Text Processing Tasks Research Articles

ChatGPT for complex text evaluation tasks

AbstractChatGPT and other large language models (LLMs) have been successful at natural and computer language processing tasks with varying degrees of complexity. This brief communication summarizes the lessons learned from a series of investigations into its use for the complex text analysis task of research quality evaluation. In summary, ChatGPT is very good at understanding and carrying out complex text processing tasks in the sense of producing plausible responses with minimum input from the researcher. Nevertheless, its outputs require systematic testing to assess their value because they can be misleading. In contrast to simple tasks, the outputs from complex tasks are highly varied and better results can be obtained by repeating the prompts multiple times in different sessions and averaging the ChatGPT outputs. Varying ChatGPT's configuration parameters from their defaults does not seem to be useful, except for the length of the output requested.

SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities

The rapid development of generative artificial intelligence has brought significant opportunities for the advancement of digital humanities research. Intelligent processing of ancient texts, as an essential part of digital humanities, is also undergoing a transformation in research methodologies in the wave of AIGC. The integration of generative pre-trained models with Chinese ancient texts, a vital carrier of Chinese culture, allows for deep mining of the content of these texts and provides services that make ancient texts more understandable and accessible to the general public. In this research, we propose a method that combines the most renowned Chinese anthology, the “Siku Quanshu,” with generative pre-trained models. We developed the SikuGPT model, a generative model for ancient text processing tasks, based on GPT-type language models by continued pretraining. This model was tested on two typical tasks of ancient text processing: translation between classical and modern Chinese, and classification of ancient texts. The findings reveal that our model achieves advantages in understanding and generating scenarios of ancient texts. The capability of SikuGPT in processing traditional Chinese texts helps to promote the organization of ancient information and knowledge services, and advances the international dissemination of traditional Chinese culture.

A case study on decompounding in Indian language IR

Abstract Decompounding is an essential preprocessing step in text-processing tasks such as machine translation, speech recognition, and information retrieval (IR). Here, the IR issues are explored from five viewpoints. (A) Does word decompounding impact the Indian language IR? If yes, to what extent? (B) Can corpus-based decompounding models be used in the Indian language IR? If yes, how? (C) Can machine learning and deep learning-based decompounding models be applied in the Indian language IR? If yes, how? (D) Among the different decompounding models (corpus-based, hybrid machine learning-based, and deep learning-based), which provides the best effectiveness in the IR domain? (E) Among the different IR models, which provides the best effectiveness from the IR perspective? This study proposes different corpus-based, hybrid machine learning-based, and deep learning-based decompounding models in Indian languages (Marathi, Hindi, and Sanskrit). Moreover, we evaluate the effectiveness of each activity from an IR perspective only. It is observed that the different decompounding models improve IR effectiveness. The deep learning-based decompounding models outperform the corpus-based and hybrid machine learning-based models in Indian language IR. Among the different deep learning-based models, the Bi-LSTM-A model performs best and improves mean average precision (MAP) by 28.02% in Marathi. Similarly, the Bi-RNN-A model improves MAP by 18.18% and 6.1% in Hindi and Sanskrit, respectively. Among the retrieval models, the In_expC2 model outperforms others in Marathi and Hindi, and the BB2 model outperforms others in Sanskrit.

Automatic text classification of drug-induced liver injury using document-term matrix and XGBoost.

Regulatory agencies generate a vast amount of textual data in the review process. For example, drug labeling serves as a valuable resource for regulatory agencies, such as U.S. Food and Drug Administration (FDA) and Europe Medical Agency (EMA), to communicate drug safety and effectiveness information to healthcare professionals and patients. Drug labeling also serves as a resource for pharmacovigilance and drug safety research. Automated text classification would significantly improve the analysis of drug labeling documents and conserve reviewer resources. We utilized artificial intelligence in this study to classify drug-induced liver injury (DILI)-related content from drug labeling documents based on FDA's DILIrank dataset. We employed text mining and XGBoost models and utilized the Preferred Terms of Medical queries for adverse event standards to simplify the elimination of common words and phrases while retaining medical standard terms for FDA and EMA drug label datasets. Then, we constructed a document term matrix using weights computed by Term Frequency-Inverse Document Frequency (TF-IDF) for each included word/term/token. The automatic text classification model exhibited robust performance in predicting DILI, achieving cross-validation AUC scores exceeding 0.90 for both drug labels from FDA and EMA and literature abstracts from the Critical Assessment of Massive Data Analysis (CAMDA). Moreover, the text mining and XGBoost functions demonstrated in this study can be applied to other text processing and classification tasks.

What Can Computers Do Now? Dreyfus Revisited for the Third Wave of Artificial Intelligence

In recent years, artificial intelligence (AI) has seen significant advances that have in fact exceeded even optimistic prognoses. Using data-driven AI, namely deep learning techniques, it has been demonstrated that computers may now be equipped with abilities of remarkable scope and quality, such as solving image and text processing tasks at human level. Large language models, in particular, have sparked debates regarding opportunities and challenges of this rapidly developing area. Will remaining fundamental challenges of data-driven AI, such as factual or logical mistakes, be overcome for good if complemented and hybridized with symbolic AI techniques, such as knowledge representation and reasoning? Will systems of artificial general intelligence (AGI) emerge from this, possessing common sense and in fact completing the decades-old quest for AI that motivated the raise of the field in the 1950s? In the light of these questions, we review the likewise, decades-old philosophical debate about capabilities and limitations of computers from a hybrid AI point of view. Here, we discuss how hybrid AI is coming closer to disproving Hubert Dreyfus’ famous statements regarding what computers can not do. At the same time, we shed light on a lesser discussed challenge for hybrid AI: the possibility that its developers might be its biggest limiters.

Exploratory analysis on the natural language processing models for task specific purposes

Natural language processing (NLP) is a technology that has become widespread in the area of human language understanding and analysis. A range of text processing tasks such as summarisation, semantic analysis, classification, question-answering, and natural language inference are commonly performed using it. The dilemma of picking a model to help us in our task is still there. It’s becoming an impediment. This is where we are trying to determine which modern NLP models are better suited for the tasks set out above in order to compare them with datasets like SQuAD and GLUE. For comparison, BERT, RoBERTa, distilBERT, BART, ALBERT, and text-to-text transfer transformer (T5) models have been used in this study. The aim is to understand the underlying architecture, its effects on the use case and also to understand where it falls short. Thus, we were able to observe that RoBERTa was more effective against the models ALBERT, distilBERT, and BERT in terms of tasks related to semantic analysis, natural language inference, and question-answering. The reason is due to the dynamic masking present in RoBERTa. For summarisation, even though BART and T5 models have very similar architecture the BART model has performed slightly better than the T5 model.

A simple and efficient filter feature selection method via document-term matrix unitization

Text processing tasks commonly grapple with the challenge of high dimensionality. One of the most effective solutions to this challenge is to preprocess text data through feature selection methods. Feature selection can select the most advantageous features for subsequent operations (e.g., classification) from the native feature space of the text. This process effectively trims the feature space’s dimensionality, enhancing subsequent operations’ efficiency and accuracy. This paper proposes a straightforward and efficient filter feature selection method based on document-term matrix unitization (DTMU) for text processing. Diverging from previous filter feature selection methods that concentrate on scoring criteria definition, our method achieves more optimal feature selection by unitizing each column of the document-term matrix. This approach mitigates feature-to-feature influence and reinforces the role of the weighting proportion within the features. Subsequently, our scoring criterion subtracts the sum of weights for negative samples from positive samples and takes the absolute value. We conduct numerical experiments to compare DTMU with four advanced filter feature selection methods: max–min ratio metric, proportional rough feature selector, least loss, and relative discrimination criterion, along with two classical filter feature selection methods: Chi-square and information gain. The experiments are performed on four ten-thousand-dimensional feature space datasets: book, dvd, music, movie and two thousand-dimensional feature space datasets: imdb, amazon_cells, sourced from Amazon product reviews and movie reviews. Experimental findings demonstrate that DTMU selects more advantageous features for subsequent operations and achieves a higher dimensionality reduction rate than those of the other six methods used for comparison. Moreover, DTMU exhibits robust generalization capabilities across various classifiers and dimensional datasets. Notably, the average CPU time for a single run of DTMU is measured at 1.455 s.

A comprehensive evaluation of large Language models on benchmark biomedical text processing tasks

Recently, Large Language Models (LLMs) have demonstrated impressive capability to solve a wide range of tasks. However, despite their success across various tasks, no prior work has investigated their capability in the biomedical domain yet. To this end, this paper aims to evaluate the performance of LLMs on benchmark biomedical tasks. For this purpose, a comprehensive evaluation of 4 popular LLMs in 6 diverse biomedical tasks across 26 datasets has been conducted. To the best of our knowledge, this is the first work that conducts an extensive evaluation and comparison of various LLMs in the biomedical domain. Interestingly, we find based on our evaluation that in biomedical datasets that have smaller training sets, zero-shot LLMs even outperform the current state-of-the-art models when they were fine-tuned only on the training set of these datasets. This suggests that pre-training on large text corpora makes LLMs quite specialized even in the biomedical domain. We also find that not a single LLM can outperform other LLMs in all tasks, with the performance of different LLMs may vary depending on the task. While their performance is still quite poor in comparison to the biomedical models that were fine-tuned on large training sets, our findings demonstrate that LLMs have the potential to be a valuable tool for various biomedical tasks that lack large annotated data.

БОЛАШАҚ МҰҒАЛІМНІҢ СӨЙЛЕУ МӘДЕНИЕТІН ҚАЛЫПТАСТЫРУ

Annotation. Linguistic and cultural bases of formation of linguistic personality of philological students can be used as classification signs in document classification, as semantic concepts in generating thesauruses and ontologies, as supporting concepts in media content analysis. In virtually all automatic text processing tasks, such as annotation, indexing, classification, machine translation, knowledge extraction, and more. it will be necessary to produce the terminology. This monograph calls for the description and systematization of these methods in order to determine the future and development issues of language identity formation. The importance of the identified issues is based on the phenomenon of information explosion in modern society. It is intended for research workers, specialists, university teachers, doctoral students, graduate students and students in the field of language personality formation. Key words: word culture, speech culture, linguistic personality, linguistic personality of the student, formation of linguistic personality.

Homomorphic Compression: Making Text Processing on Compression Unlimited

Lossless data compression is an effective way to handle the huge transmission and storage overhead of massive text data. Its utility is even more significant today when data volumes are skyrocketing. The concept of operating on compressed data infuses new blood into efficient text management by enabling mainly access-oriented text processing tasks to be done directly on compressed data without decompression. Facing limitations of the existing compressed text processing schemes such as limited types of operations supported, low efficiency, and high space occupation, we address these problems by proposing a homomorphic compression theory. It enables the generalization and characterization of algorithms with compression processing capabilities. On this basis, we develop HOCO, an efficient text data management engine that supports a variety of processing tasks on compressed text. We select three representative compression schemes and implement them combined with homomorphism in HOCO. HOCO supports the extension of homomorphic compression schemes through a modular and object-oriented design and has convenient interfaces for text processing tasks. We evaluate HOCO on six real-world datasets. The three schemes implemented in HOCO show trade-offs in terms of compression ratio, supported operation types, and efficiency. Experiments also show that HOCO can achieve higher throughput in random access and modification operations (averagely 9.18× than the state-of-the-art) and lower latency in text analytic tasks (averagely 7.16× than processing on uncompressed text) without compromising compression efficacy.

Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure

Machine learning (ML) has been extensively involved in assistant disease diagnosis and prediction systems to emancipate the serious dependence on medical resources and improve healthcare quality. Moreover, with the booming of pre-training language models (PLMs), the application prospect and promotion potential of machine learning methods in the relevant field have been further inspired. PLMs have recently achieved tremendous success in diverse text processing tasks, whereas limited by the significant semantic gap between the pre-training corpus and the structured electronic health records (EHRs), PLMs cannot converge to anticipated disease diagnosis and prediction results. Unfortunately, establishing connections between PLMs and EHRs typically requires the extraction of curated predictor variables from structured EHR resources, which is tedious and labor-intensive, and even discards vast implicit information.In this work, we propose an Input Prompting and Discriminative language model with the Mixture-of-experts framework (IPDM) by promoting the model’s capabilities to learn knowledge from heterogeneous information and facilitating the feature-aware ability of the model. Furthermore, leveraging the prompt-tuning mechanism, IPDM can inherit the impacts of the pre-training in downstream tasks exclusively through minor modifications. IPDM remarkably outperforms existing models, proved by experiments on one disease diagnosis task and two disease prediction tasks. Finally, experiments with few-feature and few-sample demonstrate that IPDM achieves significant stability and impressive performance in predicting chronic diseases with unclear early-onset characteristics or sudden diseases with insufficient data, which verifies the superiority of IPDM over existing mainstream methods, and reveals the IPDM can powerfully address the aforementioned challenges via establishing a stable and low-resource medical diagnostic system for various clinical scenarios.

On Isotropy of Multimodal Embeddings

Embeddings, i.e., vector representations of objects, such as texts, images, or graphs, play a key role in deep learning methodologies nowadays. Prior research has shown the importance of analyzing the isotropy of textual embeddings for transformer-based text encoders, such as the BERT model. Anisotropic word embeddings do not use the entire space, instead concentrating on a narrow cone in such a pretrained vector space, negatively affecting the performance of applications, such as textual semantic similarity. Transforming a vector space to optimize isotropy has been shown to be beneficial for improving performance in text processing tasks. This paper is the first comprehensive investigation of the distribution of multimodal embeddings using the example of OpenAI’s CLIP pretrained model. We aimed to deepen the understanding of the embedding space of multimodal embeddings, which has previously been unexplored in this respect, and study the impact on various end tasks. Our initial efforts were focused on measuring the alignment of image and text embedding distributions, with an emphasis on their isotropic properties. In addition, we evaluated several gradient-free approaches to enhance these properties, establishing their efficiency in improving the isotropy/alignment of the embeddings and, in certain cases, the zero-shot classification accuracy. Significantly, our analysis revealed that both CLIP and BERT models yielded embeddings situated within a cone immediately after initialization and preceding training. However, they were mostly isotropic in the local sense. We further extended our investigation to the structure of multilingual CLIP text embeddings, confirming that the observed characteristics were language-independent. By computing the few-shot classification accuracy and point-cloud metrics, we provide evidence of a strong correlation among multilingual embeddings. Embeddings transformation using the methods described in this article makes it easier to visualize embeddings. At the same time, multiple experiments that we conducted showed that, in regard to the transformed embeddings, the downstream tasks performance does not drop substantially (and sometimes is even improved). This means that one could obtain an easily visualizable embedding space, without substantially losing the quality of downstream tasks.

TPTS: Text pre-processing Techniques for Sindhi Language

The Internet is a significant source of textual data, with users generating vast amounts of information through social media and news agencies daily. The extraction of meaningful information from large datasets is a challenging and costly process. Text pre-processing is a crucial initial step in any Natural Language Processing (NLP) task, as it can impact the overall performance of the study. The main objective of text pre-processing is to transform unstructured text into a linguistically meaningful (standard form) format, making extracting information for any text-processing task easier. This paper introduces TPTS, a model for text pre-processing in the Sindhi language. TPTS performs essential NLP tasks such as text tokenization, normalization, stop-word removal, stemming, and POS tagging for the Sindhi language. The Sindhi Text Corpus (STC), consisting of 1.5k Sindhi text documents collected from various online news websites, is used for experimentation. The TF-IDF approach is employed to identify high-frequency stop-words in the Sindhi language. Furthermore, a rule-based system tags words with their part of speech in Sindhi input text. The ROUGE evaluation metric is used to assess the effectiveness of the proposed TPTS technique, achieving 89% accuracy on the STC corpus. The Sindhi language is spoken by over 30 million people globally, and the lack of adequate NLP tools and resources limits the development of technology and natural language applications that can benefit Sindhi speakers. The proposed TPTS model can aid in developing such applications, making it beneficial not only for text pre-processing tasks but also for other Sindhi language text-processing tasks such as text summarization, sentiment analysis, speech-processing applications, text mining, and information retrieval systems.

Induction of knowledge, attitude and practice of people toward a pandemic from Twitter: a comprehensive model based on opinion mining

PurposeQuestionnaire studies of knowledge, attitude and practice (KAP) are effective research in the field of health, which have many shortcomings. The purpose of this research is to propose an automatic questionnaire-free method based on deep learning techniques to address the shortcomings of common methods. Next, the aim of this research is to use the proposed method with public comments on Twitter to get the gaps in KAP of people regarding COVID-19.Design/methodology/approachIn this paper, two models are proposed to achieve the mentioned purposes, the first one for attitude and the other for people’s knowledge and practice. First, the authors collect some tweets from Twitter and label them. After that, the authors preprocess the collected textual data. Then, the text representation vector for each tweet is extracted using BERT-BiGRU or XLNet-GRU. Finally, for the knowledge and practice problem, a multi-label classifier with 16 classes representing health guidelines is proposed. Also, for the attitude problem, a multi-class classifier with three classes (positive, negative and neutral) is proposed.FindingsLabeling quality has a direct relationship with the performance of the final model, the authors calculated the inter-rater reliability using the Krippendorf alpha coefficient, which shows the reliability of the assessment in both problems. In the problem of knowledge and practice, 87% and in the problem of people’s attitude, 95% agreement was reached. The high agreement obtained indicates the reliability of the dataset and warrants the assessment. The proposed models in both problems were evaluated with some metrics, which shows that both proposed models perform better than the common methods. Our analyses for KAP are more efficient than questionnaire methods. Our method has solved many shortcomings of questionnaires, the most important of which is increasing the speed of evaluation, increasing the studied population and receiving reliable opinions to get accurate results.Research limitations/implicationsOur research is based on social network datasets. This data cannot provide the possibility to discover the public information of users definitively. Addressing this limitation can have a lot of complexity and little certainty, so in this research, the authors presented our final analysis independent of the public information of users.Practical implicationsCombining recurrent neural networks with methods based on the attention mechanism improves the performance of the model and solves the need for large training data. Also, using these methods is effective in the process of improving the implementation of KAP research and eliminating its shortcomings. These results can be used in other text processing tasks and cause their improvement. The results of the analysis on the attitude, practice and knowledge of people regarding the health guidelines lead to the effective planning and implementation of health decisions and interventions and required training by health institutions. The results of this research show the effective relationship between attitude, practice and knowledge. People are better at following health guidelines than being aware of COVID-19. Despite many tensions during the epidemic, most people still discuss the issue with a positive attitude.Originality/valueTo the best of our knowledge, so far, no text processing-based method has been proposed to perform KAP research. Also, our method benefits from the most valuable data of today’s era (i.e. social networks), which is the expression of people’s experiences, facts and free opinions. Therefore, our final analysis provides more realistic results.

Joint Reference and Relation Extraction from Legal Documents with Enhanced Decoder Input

Abstract This paper deals with an important task in legal text processing, namely reference and relation extraction from legal documents, which includes two subtasks: 1) reference extraction; 2) relation determination. Motivated by the fact that two subtasks are related and share common information, we propose a joint learning model that solves simultaneously both subtasks. Our model employs a Transformer-based encoder-decoder architecture with non-autoregressive decoding that allows relaxing the sequentiality of traditional seq2seq models and extracting references and relations in one inference step. We also propose a method to enrich the decoder input with learnable meaningful information and therefore, improve the model accuracy. Experimental results on a dataset consisting of 5031 legal documents in Vietnamese with 61,446 references show that our proposed model performs better results than several strong baselines and achieves an F1 score of 99.4% for the joint reference and relation extraction task.

Quantum semantic coding for structure extraction and matching applications

Quantum Natural Language Processing (QNLP) is coded in the semantic space using a combined semantic distribution classification model with tensor operations. Its theoretical results on quantum circuit mapping and quantum semantic coding of text have been tested in practice with the recent development of quantum back-end equipment. In view of the small scale of quantum natural language processing tasks, single sentence structure of quantum semantic coding, low text coverage and lack of application, this paper proposes a combination of syntactic structure to extract text and extend the sentence components of quantum semantic coding, which improves the utilization of text processing task of quantum computing resources. In view of the fact that quantum natural language processing has few cases in specific applications, this paper studies the application expansion possibility of quantum text matching and question answering applications. The development path of classical natural language processing is referred to enhance the usefulness and explore the practical ability of QNLP in the current resource-constrained conditions.

Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple Wizards

Wizard of Oz (WoZ) as a prototyping method has been used to simulate intelligent user interfaces, particularly for speech-based systems. However, as our societies' expectations on artificial intelligence (AI) grows, the question remains whether a single Wizard is sufficient for it to simulate smarter systems and more complex interactions. Optimistic visions of 'what artificial intelligence (AI) can do' places demands on WoZ platforms to simulate smarter systems and more complex interactions. This raises the question of whether the typical approach of employing a single Wizard is sufficient. Moreover, while existing work has employed multiple Wizards in WoZ studies, a multi-Wizard approach has not been systematically studied in terms of feasibility, effectiveness, and challenges. We offer Wizundry, a real-time, web-based WoZ platform that allows multiple Wizards to collaboratively operate a speech-to-text based system remotely. We outline the design and technical specifications of our open-source platform, which we iterated over two design phases. We report on two studies in which participant-Wizards were tasked with negotiating how to cooperatively simulate an interface that can handle natural speech for dictation and text editing as well as other intelligent text processing tasks. We offer qualitative findings on the Multi-Wizard experience for Dyads and Triads of Wizards. Our findings reveal the promises and challenges of the multi-Wizard approach and open up new research questions.

DETECTING ONLINE FAKE NEWS USING MACHINE LEARNING AND TOPOLOGICAL DATA ANALYSIS

The internet is rife with different types of disinformation openly available to the public. The spreading of accidental or malicious misinformation on social media, specifically in critical situations, such as real-world emergencies, can have negative consequences on our health, democracy and economy. This facilitates the spread of rumors on social media where users share and exchange the latest information with many readers, including a large volume of new information every second. However, updated news sharing on social media is not always true. Consequently, disinformation is rapidly being recognized as a global risk alongside terrorism, cancer and global warming. The increasing demand for fact checking at scale has stimulated a rapid development of automated solutions using technologies such as Natural Language Processing (NLP) and Machine Learning (ML) in order to reduce the required human effort. This paper will explore novel methods for automated fake news detection through the integration of two powerful approaches to data science, ML and Topological Data Analysis (TDA). The main strength of ML lies in its predictive power. Deep learning in particular has yielded some impressive practical successes in various text processing tasks. However, ML can fail in more exploratory talks aimed at understanding the nature of the data and uncovering its insights. TDA is a fairly new field that applies topology and geometry to analyze high-dimensional data and construct its compressed representation offering a more exploratory approach. This paper will explore how the strengths of those two fields can be integrated in order to create novel method for the analysis of text data, which will potentially lead to the creation of a state-of-the-art fake news detection model.

IcsBERTs: Optimizing Pre-trained Language Models in Intelligent Customer Service

Automatic processing of textual information is a growing application area in intelligent customer service platforms due to the large number of customer requests constantly provided in the form of text. Many pre-trained language models have shown their high performance on text processing tasks. However, these pre-training strategies do not leverage domain-specific information. In this paper, we propose icsBERTs optimized for intelligent customer service on both word and sentence levels. Specifically, the automatic business words extraction and multi-task training methods are designed to enhance language representations for our models. Furthermore, we also propose the use of adaptors based on these icsBERTs for more complicated tasks. Our models outperform mainstream pre-trained language models on several tasks in the field of intelligent customer service and these experimental results demonstrate that using targeted strategies can further improve the performance of pre-trained language models in this field.

Research On Text Classification Based On Deep Neural Network

Text classification is one of the classic tasks in the field of natural language processing. The goal is to identify the category to which the text belongs. Text categorization is widely used in email detection, sentiment analysis, topic marking and other fields. However, good text representation is the key to improve the performance of natural language processing tasks such as text classification. Traditional text representation adopts bag-of-words model or vector space model, which not only loses the context information of the text, but also faces the problems of high latitude and high sparsity. In recent years, with the increase of data and the improvement of computing performance, the use of deep learning technology to represent and classify texts has attracted great attention. Convolutional neural network, recurrent neural network and recurrent neural network with attention mechanism are used to represent the text, and then to classify the text and other natural language processing tasks, all of which have better performance than the traditional methods. In this paper, we design two sentence-level text representation and classification models based on the deep network. The details are as follows: (1) Text representation and classification model based on bidirectional cyclic and convolutional neural networks-BRCNN. Brcnn's input is the word vector corresponding to each word in the sentence; After using cyclic neural network to extract word order information in sentences, convolution neural network is used to extract higher-level features of sentences. After convolution, the maximum pool operation is used to obtain sentence vectors. At last, softmax classifier is used for classification. Cyclic neural network can capture the word order information in sentences, while convolutional neural network can extract useful features. Experiments on eight text classification tasks show that BRCNN model can get better text feature representation, and the classification accuracy rate is equal to or higher than that of the prior art.. (2) A text representation and classification model based on attention mechanism and convolutional neural network-ACNN. ACNN model uses the recurrent neural network with attention mechanism to obtain the context vector; Then convolution neural network is used to extract more advanced feature information. The maximum pool operation is adopted to obtain a sentence vector; At last, the softmax classifier is used to classify the text. Experiments on eight text classification benchmark data sets show that ACNN improves the stability of model convergence, and can converge to an optimal or local optimal solution better than BRCNN.