Text Processing Systems Research Articles

Detection and treatment of string events in the limit

A string event is a pattern that occurs in a stream of characters. The need to detect and handle string events in infinite texts emerges in many scenarios, including online treatment of logs, web crawling, and syntax highlighting. This paper describes a technique to specify and treat string events. Users determine patterns of interest via a markup language. From such examples, tokens are generalized via a semi-lattice of regular expressions. Such tokens are combined into a context-free language that recognizes patterns in the text stream. These techniques are implemented in a text processing system called Lushu, which runs on the Java Virtual Machine (JVM). Lushu intercepts strings emitted by the JVM. Once patterns are detected, it invokes a user-specified action handler. As a proof of concept, this paper shows that Lushu outperforms state-of-the-art parsers and parser generators, such as Comby, BeautifulSoup4 and ZheFuscator, in terms of memory consumption and running time.

An Extended Pattern Based Comprehensive Stemmer for the Urdu Language

The Urdu language is used by approximately 200 million people for spoken and written communications on a daily basis. There is a substantial amount of unstructured Urdu textual data that is available worldwide. Data mining techniques can be used to extract meaningful knowledge from such a large, potentially informative source of data. There are many text processing systems available to process unstructured textual data. However, these systems are mostly language specific and developed for a variety of languages such as English, Spanish, Chinese, etc. Unfortunately, there are not as many language processing resources available for Urdu. Stemming is one of the most important preprocessing steps in the text mining process and its goal is to reduce grammatical words form, e.g., parts of speech, gender, tense, and so on, to their root form. In this work, we have extended the stemming capabilities of our existing pattern-based comprehensive stemming system for Urdu text. In addition to the existing stemming rules in previous work, we introduce novel stemming rules for prefix, and infix stemming. We also optimize the existing suffix removal rules and extend the add character lists for word normalization. These stemming rules are generic and have the ability to generate the stem of Urdu words as well as loan words (words belonging to other languages i.e. Arabic, Persian, Turkish, etc). In the experimental evaluation, we have observed a significant improvement in the overall stemming accuracy of our proposed pattern-based Urud stemmer, which demonstrates the adoptability of the proposed stemming approach for a variety of text-processing applications.

Economic aspects of error identification in semi-structured publications in the state language

Due to the rapid growth of information on the Internet and social networks, research in the field of computational linguistics has become very relevant. The volume of information that people and machines create in natural language needs to be processed, analyzed and verified. Information retrieval systems, dialog systems, and machine translation tools are used for this. The range of automatic text processing systems is very wide, it covers various tasks. Finding errors in texts and words, identifying and correcting incorrect words is one of the most important tasks of natural language processing (NLP). The article provides an overview of semi-structured data, methods and technologies for identifying incorrect words in natural languages. The paper gives an overview of semi-structured data, methods and techniques for detecting incorrect words in natural languages. The aim of the research is to develop an effective approach for detecting and correcting errors occurring in Kazakh-language texts, especially in the context of limited resources and unstructured data. The research includes the use of machine learning techniques as well as economic analysis of the costs of developing and implementing such solutions. The proposed approach facilitates the automation of text verification, which can significantly reduce the cost of manual data processing and improve the quality of information in various spheres, including business and public administration.

METHODS OF TRANSFERRING FULL WORD-FORMATION AFFIXES IN THE NATIONAL CORPUS OF THE KAZAKH LANGUAGE

This scientific article studies methods of transferring full word-formation affixes in the National Corpus of the Kazakh language. Affixes play an essential role in the forming of word formation and semantics of a language, and the study of their transmission is crucial for linguistic theory and practice. The purpose of the study is to analyze and systematize approaches to word-forming affixes in the corpus of the Kazakh language.The main areas of research include the analysis of the morphological features of affix transfer, as well as the identification of contexts in which various transmission options for transfer are possible.A short description of the scientific and practical significance of the work: the study allows us to better understand the proc esses of word formation in the Kazakh language and develop effective methods for automatic processing of texts in this language. It is also of practical importance for the creation of linguistic resources and the development of language applications. Brief description of the research methodology: the research methodology includes the analysis of textual material from the national corpus of the Kazakh language, as well as the use of linguistic tools and data analysis to determine the features of the transmission of affixes. The main results and analysis of the research work, conclusions: the study revealed various ways of transmitting wordforming affixes, including morphological constructions, in the corpus of the Kazakh language. The analysis made it possible to identify the features and frequency of these methods. Value of the conducted research: this study contributes to the linguistic theory, and enriches our knowledge about the mechanisms of word formation and ways of transmitting linguistic elements in the language corpus. In addition, the results obtained can be useful for the development of language technologies and resources for the Kazakh language. Practical significance of the results: the study can be used in the creation of automatic systems of text processing systems in the Kazakh language, and the development of language applications, such as in linguistic training and translation. The obtained results and conclusions can become the basis for further research in the field of linguistics and computational linguistics.

Classification of Hijaiyah Letters Using Hybrid CNN-CatBoost

In this digital era, character recognition technology and letter classification are topics that are increasingly attracting attention, especially in the context of developing applications for learning Arabic, text processing, and artificial intelligence systems. There have been many previous studies examining this topic. However, there are still many opportunities to develop models for classifying hijaiyah letters to help many people learn to write hijaiyah letters.
 In this study, building a hijaiyah letter classification model using Hybrid-CNN and CatBoost, where CNN is used as a feature extractor and CatBoost as a classifier. CNN will be trained first to become a feature extractor and the results of the CNN model will be used as a feature extractor to create feature representation for training and testing CatBoost model. The AHDC dataset was used in this study and succeeded in achieving an accuracy value of 96.07%. Although it has not been able to compete with previous research, the Hybrid-CNN model with CatBoost has good potential in the future.

Text Linguistics and the Use of Linguistic Data in Modern Technologies: Prospects for Development

This study explores text linguistics as a branch of linguistics that analyses the structure, meaning, and other aspects of texts. The paper aims to analyse the use of linguistic data in modern technologies, in particular, to develop applications that can understand and generate linguistic content. To achieve this goal, the research uses the methodology of qualitative analysis and is based on the work of textual theorists. The study is based on a qualitative analysis of examples collected from the BNC corpus. The instructions for obtaining data are to use textual forms located at specific positions in the co-text. The use of qualitative analysis for data acquisition is useful for studying phonology, morphology, morphosyntax, and general linguistic facts. The results confirm the promising possibilities of text linguistics and the use of language data in technology, in particular the growth in the volume and availability of language data and the application of new methods of text analysis. The conclusions point to the connection of text linguistics with other fields and the expansion of the capabilities of semantic search engines and automatic text processing systems. In addition, the article emphasises the importance of digital resources and the study of phrases as effective tools for linguistic analysis. The analysis of three illustrative examples taken from the British linguistic corpus is carried out with the aim of studying textual and discourse phenomena in the use of language data. A certain inconsistency between the factors has been revealed, which leads to a blurring of the linguistic facts that the authors of the article tried to investigate. It should be noted that the selected examples, which were created to support the theory, are artificial, which makes it difficult to analyse the context. This makes it difficult for native English speakers to evaluate these examples. This is especially true for the first example, where the syntactic constructions do not match the context. This deviation was observed at the level of the sentence's information structure. As for the second example, the discomfort observed can be explained by the lack of motivation to use the referent that was introduced into the context as a reference marker that is independently associated with it in the place where the phrase appears in the text. There is also a clear polyphonic inconsistency in the origin of the point of view that characterises each segment of the sentence. This inconsistency is related to the choice of the reference marker.

Improving the robustness and accuracy of biomedical language models through adversarial training

Deep transformer neural network models have improved the predictive accuracy of intelligent text processing systems in the biomedical domain. They have obtained state-of-the-art performance scores on a wide variety of biomedical and clinical Natural Language Processing (NLP) benchmarks. However, the robustness and reliability of these models has been less explored so far. Neural NLP models can be easily fooled by adversarial samples, i.e. minor changes to input that preserve the meaning and understandability of the text but force the NLP system to make erroneous decisions. This raises serious concerns about the security and trust-worthiness of biomedical NLP systems, especially when they are intended to be deployed in real-world use cases. We investigated the robustness of several transformer neural language models, i.e. BioBERT, SciBERT, BioMed-RoBERTa, and Bio-ClinicalBERT, on a wide range of biomedical and clinical text processing tasks. We implemented various adversarial attack methods to test the NLP systems in different attack scenarios. Experimental results showed that the biomedical NLP models are sensitive to adversarial samples; their performance dropped in average by 21 and 18.9 absolute percent on character-level and word-level adversarial noise, respectively, on Micro-F1, Pearson Correlation, and Accuracy measures. Conducting extensive adversarial training experiments, we fine-tuned the NLP models on a mixture of clean samples and adversarial inputs. Results showed that adversarial training is an effective defense mechanism against adversarial noise; the models’ robustness improved in average by 11.3 absolute percent. In addition, the models’ performance on clean data increased in average by 2.4 absolute percent, demonstrating that adversarial training can boost generalization abilities of biomedical NLP systems. This study takes an important step towards revealing vulnerabilities of deep neural language models in biomedical NLP applications. It also provides practical and effective strategies to develop secure, trust-worthy, and accurate intelligent text processing systems in the biomedical domain.

IMPROVING MULTI-LABEL TEXT CLASSIFICATION USING WEIGHTED INFORMATION GAIN AND CO-TRAINED MULTINOMIAL NAÏVE BAYES CLASSIFIER

Over recent years, the emergence of electronic text processing systems has generated a vast amount of structured and unstructured data, thus creating a challenging situation for users to rummage through irrelevant information. Therefore, studies are continually looking to improve the classification process to produce more accurate results that would benefit users. This paper looks into the weighted information gain method that re-assigns wrongly classified features with new weights to provide better classification. The method focuses on the weights of the frequency bins, assuming every time a certain word frequency bin is iterated, it provides information on the target word feature. Therefore, the more iteration and re-assigning of weight occur within the bin, the more important the bin becomes, eventually providing better classification. The proposed algorithm was trained and tested using a corpus extracted from dedicated Facebook pages related to diabetes. The weighted information gain feature selection technique is then fed into a co-trained Multinomial Naïve Bayes classification algorithm that captures the labels' dependencies. The algorithm incorporates class value dependencies since the dataset used multi-label data before converting string vectors that allow the sparse distribution between features to be minimised, thus producing more accurate results. The results of this study show an improvement in classification to 61%.

РЕЧЕВЫЕ ТЕХНОЛОГИИ В ОБУЧЕНИИ ОПЕРАТИВНОГО ПЕРСОНАЛА ГОРОДСКИХ РЕЛЬСОВЫХ ТРАНСПОРТНЫХ СИСТЕМ

This article discusses the issues of processing audio messages between the dispatcher and the drivers in the intelligent production resource management system of urban rail transport systems. The research considers the following tasks: recognition of dispatcher commands and automation of the process of forming dispatcher voice commands. The article proposes an approach to the classi cation of dispatcher commands and creates an algorithm for forming commands for the control system. The article provides an overview of the text processing and speech recognition systems used in the article. An approach to solving the problem of classifying dispatcher commands in conditions of a limited amount of data. The article also presents a block diagram of an intelligent system. Within the framework of this scheme, the algorithms for classifying the text into: «Information», «Command», «Order». The algorithm searching commands in the text was analyzed.

Word Sense Induction in Persian and English: A Comparative Study

Words in the natural language have forms and meanings, and there might not always be a one-to-one match between them. This property of the language causes words to have more than one meaning; as a result, a text processing system faces challenges to determine the precise meaning of the target word in a sentence. Using lexical resources or lexical databases, such as WordNet, might be a help, but due to their manual development, they become outdated by passage of time and language change. Moreover, the lexical resources might be domain dependent which are unusable for open domain natural language processing tasks. These drawbacks are a strong motivation to use unsupervised machine learning approaches to induce word senses from the natural data. To reach the goal, the clustering approach can be utilized such that each cluster resembles a sense. In this paper, we study the performance of a word sense induction model by using three variables: a) the target language: in our experiments, we run the induction process on Persian and English; b) the type of the clustering algorithm: both parametric clustering algorithms, including hierarchical and partitioning, and non-parametric clustering algorithms, including probabilistic and density-based, are utilized to induce senses; c) the context of the target words to capture the information in vectors created for clustering: for the input of the clustering algorithms, the vectors are created either based on the whole sentence in which the target word is located; or based on the limited surrounding words of the target word. We evaluate the clustering performance externally. Moreover, we introduce a normalized, joint evaluation metric to compare the models. The experimental results for both Persian and English test data showed that the window-based partitioningK-means algorithm obtained the best performance.

Anonymization of German financial documents using neural network-based language models with contextual word representations

The automatization and digitalization of business processes have led to an increase in the need for efficient information extraction from business documents. However, financial and legal documents are often not utilized effectively by text processing or machine learning systems, partly due to the presence of sensitive information in these documents, which restrict their usage beyond authorized parties and purposes. To overcome this limitation, we develop an anonymization method for German financial and legal documents using state-of-the-art natural language processing methods based on recurrent neural nets and transformer architectures. We present a web-based application to anonymize financial documents and a large-scale evaluation of different deep learning techniques.

Automatic Linking of Terms from Scientific Texts with Knowledge Base Entities

Due to the growth of the number of scientific publications, the tasks related to scientific article processing become more actual. Such texts have a special structure, lexical and semantic content that should be taken into account while processing. Using information from knowledge bases can significantly improve the quality of text processing systems. This paper is dedicated to the entity linking task for scientific articles in Russian, where we consider scientific terms as entities. During our work, we annotated a corpus with scientific texts, where each term was linked with an entity from a knowledge base. Also, we implemented an algorithm for entity linking and evaluated it on the corpus. The algorithm consists of two stages: candidate generation for an input term and ranking this set of candidates to choose the best match. We used string matching of an input term and an entity in a knowledge base to generate a set of candidates. To rank the candidates and choose the most relevant entity for a term, information about the number of links to other entities within the knowledge base and to other sites is used. We analyzed the obtained results and proposed possible ways to improve the quality of the algorithm, for example, using information about the context and a knowledge base structure. The annotated corpus is publicly available and can be useful for other researchers.

Cross-Context News Corpus for Protest Event-Related Knowledge Base Construction

Abstract We describe a gold standard corpus of protest events that comprise various local and international English language sources from various countries. The corpus contains document-, sentence-, and token-level annotations. This corpus facilitates creating machine learning models that automatically classify news articles and extract protest event-related information, constructing knowledge bases that enable comparative social and political science studies. For each news source, the annotation starts with random samples of news articles and continues with samples drawn using active learning. Each batch of samples is annotated by two social and political scientists, adjudicated by an annotation supervisor, and improved by identifying annotation errors semi-automatically. We found that the corpus possesses the variety and quality that are necessary to develop and benchmark text classification and event extraction systems in a cross-context setting, contributing to the generalizability and robustness of automated text processing systems. This corpus and the reported results will establish a common foundation in automated protest event collection studies, which is currently lacking in the literature.

A Survey on Machine Learning and Text Processing for Pesticides and Fertilizer Prediction

In the Agriculture sectors the pressure is increased immensely due to the rise in population. In this present year we mainly witness a move from conventional methods to the advanced technology with advent of the technology. Machine Learning and Text Processing have transformed the quality and the quantity of agriculture. The real time monitoring systems along with hybridization of species paved away for resource efficiency. Scientists and researchers across the globe are mainly shifting towards collaborative projects and ideologies to explore this field for saving society. The optimal solution has been provided by racing in the tech industry. This application will be portable, scalable and durable that provides help for initiating new areas in the agriculture field. So, this survey focuses on machine learning methodologies along with Text Processing systems in agriculture. This startup’s private and public sector which is around the world that provides smart and sustainable solutions are briefly discussed. Based on agriculture this scenario, limitations, applications, development and future parameters are also briefly explained. The greatest challenge for the future agriculture lies in making agriculture research and extension more demand driven and client oriented. In addition, laboratory analysis has been undertaken to assess the nitrogen status and the soil carbon of fresh paddy soils and the used paddy soils as well as saplings. During the first season of 2011 the data collected on pests, disease incidence and severity and yield are being compiled for analysis. Therefore, integrated research to develop a bottom up participatory technology extension approach using new technologies is suggested for sustainable development in agriculture.

Real-time clinical note monitoring to detect conditions for rapid follow-up: A case study of clinical trial enrollment in drug-induced torsades de pointes and Stevens-Johnson syndrome.

Identifying acute events as they occur is challenging in large hospital systems. Here, we describe an automated method to detect 2 rare adverse drug events (ADEs), drug-induced torsades de pointes and Stevens-Johnson syndrome and toxic epidermal necrolysis, in near real time for participant recruitment into prospective clinical studies. A text processing system searched clinical notes from the electronic health record (EHR) for relevant keywords and alerted study personnel via email of potential patients for chart review or in-person evaluation. Between 2016 and 2018, the automated recruitment system resulted in capture of 138 true cases of drug-induced rare events, improving recall from 43% to 93%. Our focused electronic alert system maintained 2-year enrollment, including across an EHR migration from a bespoke system to Epic. Real-time monitoring of EHR notes may accelerate research for certain conditions less amenable to conventional study recruitment paradigms.

ADE Eval: An Evaluation of Text Processing Systems for Adverse Event Extraction from Drug Labels for Pharmacovigilance.

IntroductionThe US FDA is interested in a tool that would enable pharmacovigilance safety evaluators to automate the identification of adverse drug events (ADEs) mentioned in FDA prescribing information. The MITRE Corporation (MITRE) and the FDA organized a shared task—Adverse Drug Event Evaluation (ADE Eval)—to determine whether the performance of algorithms currently used for natural language processing (NLP) might be good enough for real-world use.ObjectiveADE Eval was conducted to evaluate a range of NLP techniques for identifying ADEs mentioned in publicly available FDA-approved drug labels (package inserts). It was designed specifically to reflect pharmacovigilance practices within the FDA and model possible pharmacovigilance use cases.MethodsPharmacovigilance-specific annotation guidelines and annotated corpora were created. Two metrics modeled the experiences of FDA safety evaluators: one measured the ability of an algorithm to identify correct Medical Dictionary for Regulatory Activities (MedDRA®) terms for the text from the annotated corpora, and the other assessed the quality of evidence extracted from the corpora to support the selected MedDRA® term by measuring the portion of annotated text an algorithm correctly identified. A third metric assessed the cost of correcting system output for subsequent training (averaged, weighted F1-measure for mention finding).ResultsIn total, 13 teams submitted 23 runs: the top MedDRA® coding F1-measure was 0.79, the top quality score was 0.96, and the top mention-finding F1-measure was 0.89.ConclusionWhile NLP techniques do not perform at levels that would allow them to be used without intervention, it is now worthwhile exploring making NLP outputs available in human pharmacovigilance workflows.Electronic supplementary materialThe online version of this article (10.1007/s40264-020-00996-3) contains supplementary material, which is available to authorized users.

Recognizing Transliterated English Words in Persian Texts

One of the most important problems of text processing systems is the word mismatch problem. This results in limited access to the required information in information retrieval. This problem occurs in analyzing textual data such as news, or low accuracy in text classification and clustering. In this case, if the text-processing engine does not use similar/related words in the same sense, it may not be able to guide you to the appropriate result. Various statistical techniques have been proposed to bridge the vocabulary gap problem; e.g., if two words are used in similar contexts frequently, they have similar/related meanings. Synonym and similar words, however, are only one of the categories of related words that are expected to be captured by statistical approaches. Another category of related words is the pair of an original word in one language and its transliteration from another language. This kind of related words is common in non-English languages. In non-English texts, instead of using the original word from the target language, the writer may borrow the English word and only transliterate it to the target language. Since this kind of writing style is used in limited texts, the frequency of transliterated words is not as high as original words. As a result, available corpus-based techniques are not able to capture their concept. In this article, we propose two different approaches to overcome this problem: (1) using neural network-based transliteration, (2) using available tools that are used for machine translation/transliteration, such as Google Translate and Behnevis. Our experiments on a dataset, which is provided for this purpose, shows that the combination of the two approaches can detect English words with 89.39% accuracy.

Co-occurrence graph-based context adaptation: a new unsupervised approach to word sense disambiguation

Abstract Word sense disambiguation (WSD) is the task of selecting correct sense for an ambiguous word in its context. Since WSD is one of the most challenging tasks in various text processing systems, improving its accuracy can be very beneficial. In this article, we propose a new unsupervised method based on co-occurrence graph created by monolingual corpus without any dependency on the structure and properties of the language itself. In the proposed method, the context of an ambiguous word is represented as a sub-graph extracted from a large word co-occurrence graph built based on a corpus. Most of the words are connected in this graph. To clarify the exact sense of an ambiguous word, its senses and relations are added to the context graph, and various similarity functions are employed based on the senses and context graph. In the disambiguation process, we select senses with highest similarity to the context graph. As opposite to other WSD methods, the proposed method does not use any language-dependent resources (e.g. WordNet) and it just uses a monolingual corpus. Therefore, the proposed method can be employed for other languages. Moreover, by increasing the size of corpus, it is possible to enhance the accuracy of WSD. Experimental results on English and Persian datasets show that the proposed method is competitive with existing supervised and unsupervised WSD approaches.

ПАРНЫЕ СОЧЕТАНИЯ ГЛАГОЛЬНЫХ СЛОВОИЗМЕНИТЕЛЬНЫХ АФФИКСОВ БАШКИРСКОГО ЯЗЫКА

Drawing up a model of the word form, a formal description of the inflectional system of the language is an urgent task of computer linguistics. Models of this type serve as the basis for the development of automatic text processing systems: morphological analyzers,spelling systems, machine translation, etc. Applied linguistics has considerable experience in the formalized description and modeling of the morphology of natural languages. This article discusses the modeling of the inflectional system of the verbal parts of the speech of the Bashkir language in paired combinations of affixes. This approach is close to the grammar of the Gleason orders and is promising when creating morphoanalyzers for languages of the agglutinative system.

Extraction of Rheumatoid Arthritis Disease Activity Measures From Electronic Health Records Using Automated Processing Algorithms.

ObjectiveThe accurate and efficient collection and documentation of disease activity measures (DAMs) is critical to improve clinical care and outcomes research in rheumatoid arthritis (RA). This study evaluated the performance of an automated process to extract DAMs from medical notes in the electronic health record (EHR).MethodsAn automated text processing system was developed to extract the Disease Activity Score for 28 joints (DAS28) and its clinical and laboratory elements from the Veterans Affairs EHR for patients enrolled in the Veterans Affairs Rheumatoid Arthritis (VARA) registry. After automated text processing derivation, data accuracy was assessed by comparing the automated text processing system and manual extraction with gold standard chart review in a separate validation phase.ResultsIn the validation phase, 1569 notes from 596 patients at 3 sites were evaluated, with 75 (6%) notes detected only by automated text processing, 85 (5%) detected only by manual extraction, and 1408 (90%) detected by both methods. The accuracy of automated text processing ranged from 90.7% to 96.7% and the accuracy of manual extraction ranged from 91.3% to 95.0% for the different clinical and laboratory elements. The accuracy of the two methods to calculate the DAS28 was 78.1% for automated text processing and 78.3% for manual extraction.ConclusionThe automated text processing approach is highly efficient and performed as well as the manual extraction approach. This advance has the potential for significant improvements in the collection, documentation, and extraction of these data to support clinical practice and outcomes research relevant to RA as well as the potential for broader application to other health conditions.