Development Of Natural Language Processing Research Articles

A Filter-APOSD approach for feature selection and linguistic knowledge discovery

The development of natural language processing promotes the progress of general linguistic studies. Based on the selected features and the extracted rules for word sense disambiguation (WSD), some valuable knowledge of the relations between linguistic features and word sense classes may be discovered, which may provide theoretical and practical evidence and references for lexical semantic study and natural language processing. However, many available approaches of feature selection for WSD are in the end to end operation, they can only select the optimal features for WSD, but not provide the rules for WSD, which makes knowledge discovery impossible. Therefore, a new Filter-Attribute partial ordered structure diagram (Filter-APOSD) approach is proposed in this article to fulfill both feature selection and knowledge discovery. The new approach is a combination of a Filter approach and an Attribute Partial Ordered Structure Diagram (APOSD) approach. The Filter approach is designed and used for filtering the simplest rules for WSD, and the APOSD approach is used to provide the complementary rules for WSD and visualize the structure of the datasets for knowledge discovery. The features occurring in the final rule set are selected as the optimal features. The proposed approach is verified by the benchmark data set from the SemEval-2007 preposition sense disambiguation corpus with around as the target word for WSD. The test result shows that the accuracy of WSD of around is greatly improved comparing with the one by the state of the art, and 17 out of 22 features are finally selected and ranked according to their contribution to the WSD, and some knowledge on the relations between the word senses and the selected features is discovered.

The potential and pitfalls of artificial intelligence in clinical pharmacology.

Artificial intelligence (AI) involves using data and algorithms to perform activities normally achieved through human intelligence. AI and its key component machine learning contextualize data and enhance decision making to transform how we operate, discover, and develop drugs. Transforming clinical pharmacology (CP) as AI-augmented CP (AI/CP) requires an ecosystem including digitized data collection, standardized processes, complementary technologies, and an ethical framework. This commentary aims to highlight the future perspectives of AI/CP in drug development.

Interpretable Machine Learning Enabled Inorganic Reaction Classification and Synthesis Condition Prediction

Data-driven synthesis planning with machine learning is a key step in the design and discovery of novel inorganic compounds with desirable properties. Inorganic materials synthesis is often guided by heuristics and chemists’ prior knowledge and experience, built upon experimental trial-and-error that can be both time and resource consuming. Recent developments in natural language processing have enabled large-scale text mining of scientific literature, providing open-source databases of synthesis information on realized compounds, material precursors, and reaction conditions (temperatures, times). We employ supervised classification machine learning (ML) models to distinguish between solid-state, sol–gel, and solution (hydrothermal, precipitation) synthesis routes based on specified reaction target material and/or precursor materials. We demonstrate regression ML models that are able to predict suitable temperatures and times for the crucial inorganic synthesis steps of calcination and sintering given the reaction target and precursor materials. We contrast this regression-based condition modeling with a conditional variational autoencoder neural network that can generate appropriate distributions for the synthesis conditions of interest. We evaluate model interpretability using the Shapley additive explanations approach to gain insight into factors influencing suitability of synthesis route and reaction conditions. We find that the aforementioned models are capable of learning subtle differences in target material composition, precursor compound identities, and choice of synthesis route that are present in the inorganic synthesis space. Moreover, they generalize well to unseen chemical entities, outperform common heuristics in the field, and show promise for predicting appropriate reaction routes and conditions for previously unsynthesized compounds of interest.

Topic modelling to support English text selection for translation into South Africa's other official languages

Appropriate training data is a prerequisite for the development of natural language processing (NLP) techniques. Vast amounts of language data are typically required to develop NLP tools that perform at state-of-the-art level. Such abundant resources are currently only available in a few languages. The remaining languages have to find alternative ways to become ``NLP-enabled''. The aim of the study reported on here is to make more language data available to support NLP development in the official languages of South Africa. In this paper we present the idea of generating text data by means of translation. We also propose the use of topic modelling to identify text in a highly resourced source language that will yield meaningful translations in under-resourced target languages. More specifically, the paper describes how topic modelling was used to identify English Wikipedia articles that should be suitable for translation into South Africa's 10 other official languages.

Analiza doniesień prasowych z wykorzystaniem technik przetwarzania języka naturalnego

The dynamic development of natural language processing results in a growing number of products utilizing so-called speech and language technologies. On the one hand this refers to the possibility of interacting with a computer using a language that people naturally use in speech and writing; one the other – making the information contained in all sorts of texts accessible for a computer. We present how methods for gathering and extracting information can be applied to news releases, to possibly reduce the overhead generated by republishing the same news by numerous internet information portals. We present how web syndication can be used to gather press releases; how to process those texts in order to determine mutual similarity; and how to visualize those. We present preliminary results of an experiment with application of the above-mentioned methods to selected Polish internet portals.

Advancements in Natural Language Processing for Text Understanding

Natural language processing (NLP) developments have made it possible for robots to read and analyze human language with astounding precision, revolutionizing the field of text understanding. An overview of current advancements in NLP approaches and their effects on text comprehension are provided in this abstract. It examines significant developments in fields including named entity identification, sentiment analysis, semantic analysis, and question answering, highlighting the difficulties encountered and creative solutions put forth. To sum up, recent developments in natural language processing have raised the bar for text comprehension. Deep learning models and extensive pre-training have changed methods including semantic analysis, sentiment analysis, named entity identification, and question answering. These developments have produced text comprehension systems that are increasingly precise and complex. However, issues with prejudice, coreference resolution, and contextual comprehension still need to be resolved. The future of NLP for text understanding has considerable potential with continuing study and innovation, opening the door for increasingly sophisticated applications in numerous sectors.

Using the SOCIO Chatbot for UML Modelling: A Family of Experiments

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context:</i> Recent developments in natural language processing have facilitated the adoption of chatbots in typically collaborative software engineering tasks (such as diagram modelling). Families of experiments can assess the performance of tools and processes and, at the same time, alleviate some of the typical shortcomings of individual experiments (e.g., inaccurate and potentially biased results due to a small number of participants). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> Compare the usability of a chatbot for collaborative modelling (i.e., SOCIO) and an online web tool (i.e., Creately). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Method:</i> We conducted a family of three experiments to evaluate the usability of SOCIO against the Creately online collaborative tool in academic settings. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> The student participants were faster at building class diagrams using the chatbot than with the online collaborative tool and more satisfied with SOCIO. Besides, the class diagrams built using the chatbot tended to be more concise —albeit slightly less complete. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> Chatbots appear to be helpful for building class diagrams. In fact, our study has helped us to shed light on the future direction for experimentation in this field and lays the groundwork for researching the applicability of chatbots in diagramming.

Named entity annotation schema for geological literature mining in the domain of porphyry copper deposits

Owing to the development of natural language processing and deep learning models, geological text data have become a vital resource for knowledge discovery and have attracted the attention of publishers, academic organizations, and domain scientists. However, the extraction of information from unstructured literature still remains a challenge, in which a fundamental issue is the categories and the type of discipline-specific information. This paper presents an effective workflow of building and applying ontologies in geoscience text mining, which includes a use case-driven method for building an ontology model of porphyry copper deposits, an entity annotation schema for text mining, and implementation of them to tackle real-world data. First, the Dexing porphyry copper deposit was selected as a case study to guide the construction of the ontology model. Text data in this study provided a series of entity instances. By analyzing both domain knowledge of mineral deposit models and the instance data, we built classes in the ontology. Second, with the established ontology, a named entity annotation schema comprising 21 entity tokens was designed to scale up the text mining tasks. Third, based on the annotation schema, a draft corpus with more than 200,000 words and a finely corrected corpus of 53,339 words were built for training a geological entity recognizer for porphyry copper deposits. The performance of the geological entity recognizer and the statistical distribution of entities in the corpus prove that the workflow presented in this study is effective for designing entity annotation schemas and facilitating large-scale text data mining in geoscience.

Językoznawstwo korpusowe w badaniach medioznawczych – ujęcie historyczne i praktyczne

The aim of this paper is to present the benefits and risks of implementing corpus linguistics for discourse analysis. The authors describe the origins and development of Natural Language Processing (NLP) tools in a historical perspective and provide examples of their application in social sciences, particularly in the methodology of Social Communication and Media Sciences. Fragmentary findings of studies carried out at the Institute of Journalism, Media and Social Communication at the Jagiellonian University in collaboration with the CLARIN-PL consortium illustrate a practical approach to the topic. The article presents the application of NLP tools in the analysis of the corpus of parliamentary discourse from 1989-2019 in terms of determinants for the institutionalization of civic dialogue in Poland and also in the comparative analysis of ulticulturalism in the city council discourse and media discourse in Krakow between 2014–2018 (7 th term of the Krakow City Council). The authors indicate in which phase and at which stage of communication research the use of computational linguistics can support the conclusion.

Ethical issues raised by incorporating personalized language models into brain-computer interface communication technologies: a qualitative study of individuals with neurological disease

Purpose To examine the views of individuals with neurodegenerative diseases about ethical issues related to incorporating personalized language models into brain-computer interface (BCI) communication technologies. Methods Fifteen semi-structured interviews and 51 online free response surveys were completed with individuals diagnosed with neurodegenerative disease that could lead to loss of speech and motor skills. Each participant responded to questions after six hypothetical ethics vignettes were presented that address the possibility of building language models with personal words and phrases in BCI communication technologies. Data were analyzed with consensus coding, using modified grounded theory. Results Four themes were identified. (1) The experience of a neurodegenerative disease shapes preferences for personalized language models. (2) An individual’s identity will be affected by the ability to personalize the language model. (3) The motivation for personalization is tied to how relationships can be helped or harmed. (4) Privacy is important to people who may need BCI communication technologies. Responses suggest that the inclusion of personal lexica raises ethical issues. Stakeholders want their values to be considered during development of BCI communication technologies. Conclusions With the rapid development of BCI communication technologies, it is critical to incorporate feedback from individuals regarding their ethical concerns about the storage and use of personalized language models. Stakeholder values and preferences about disability, privacy, identity and relationships should drive design, innovation and implementation. IMPLICATIONS FOR REHABILITATION Individuals with neurodegenerative diseases are important stakeholders to consider in development of natural language processing within brain-computer interface (BCI) communication technologies. The incorporation of personalized language models raises issues related to disability, identity, relationships, and privacy. People who may one day rely on BCI communication technologies care not just about usability of communication technology but about technology that supports their values and priorities. Qualitative ethics-focused research is a valuable tool for exploring stakeholder perspectives on new capabilities of BCI communication technologies, such as the storage and use of personalized language models.

Framing the Future: The Foundation Series, Foundation Models and Framing AI

Are we at risk of preserving in digital aspic current hegemonic ways of doing, talking and thinking about law and economy? Our dominant frames are those of neoclassical economics and doctrinal law, which, combined, align with neoliberal assumptions and preferences. Invisible through their ubiquity, these partially reveal and conceal problems and solutions, perpetuating inequalities. And yet, unlike social biases, our framing tends to fly largely under the radar, as we see in Isaac Asimov’s Foundation series. Taking three themes from Asimov’s work that align with current hegemonic frames, namely the centrality of the individual as the subject, a belief in scientism and technological progress to save humanity, and a dissatisfaction with the limitations of language, this paper explores what, how and why we might exercise caution in the development of natural language processing (NLP) artificial intelligence (AI). By drawing parallels with Asimov’s work and the development of foundation models that underpin all NLP AI systems, this paper asks to what extent neoliberal framing and the inequalities perpetuated therein might come to be embedded in future technologies.

Classification of Scientific Documents in the Kazakh Language Using Deep Neural Networks and a Fusion of Images and Text

The rapid development of natural language processing and deep learning techniques has boosted the performance of related algorithms in several linguistic and text mining tasks. Consequently, applications such as opinion mining, fake news detection or document classification that assign documents to predefined categories have significantly benefited from pre-trained language models, word or sentence embeddings, linguistic corpora, knowledge graphs and other resources that are in abundance for the more popular languages (e.g., English, Chinese, etc.). Less represented languages, such as the Kazakh language, balkan languages, etc., still lack the necessary linguistic resources and thus the performance of the respective methods is still low. In this work, we develop a model that classifies scientific papers written in the Kazakh language using both text and image information and demonstrate that this fusion of information can be beneficial for cases of languages that have limited resources for machine learning models’ training. With this fusion, we improve the classification accuracy by 4.4499% compared to the models that use only text or only image information. The successful use of the proposed method in scientific documents’ classification paves the way for more complex classification models and more application in other domains such as news classification, sentiment analysis, etc., in the Kazakh language.

Recent text-based research and applications in railways: A critical review and future trends

In the railway industry, a significant amount of data is stored in the textual format. The advanced development of natural language processing and text mining techniques enable automatic knowledge extraction and discovery from such documents. This paper presents a systematic review with quantitative and qualitative analyses to understand the current state of text-based research in the context of railway transport. The paper collects 107 relevant publications in the past decade and identifies different channels for researchers to obtain text data in railways and the corresponding text analysis application use-cases. Moreover, a comprehensive analysis is performed on the state-of-the-art machine learning and natural language processing methods. Four key research directions, namely multilingual NLP, digital maintenance, external data integration, and railway-centred solution pipeline, are identified from Siemens Mobility’s perspective to highlight the most prominent challenges faced in the railway industry.

Using AI to minimise bias in an employee performance review

Performance reviews are intended to be objective, but all humans experience bias. While many companies opt for group reviews as a way to de-bias and challenge the status quo, what is being said in those meetings, how those comments are said and the context for those remarks are just as important. At the same time, most people’s attention span is of shorter duration than a review and being promoted depends on what bosses remember about their direct reports, their subjective measure of employee success, and their ability to convince others that employee accomplishments are deserving of a reward. As a result of these compounding factors, meta-bias patterns emerge in company culture. Combine those limitations with the fact that reviews are often a breeding ground for subtle — and not-so-subtle — bias, and it begs the question: Why are we not using technology to help? With developments in natural language processing (NLP) and conversational AI (CAI), computers can identify biased phrases in real time. Although these technologies have a long way to go to match human nuance, we can at least flag problematic phrases during something as significant as performance reviews. And with the right inputs rooted in social science and normalised based on geography, contextual relationships and culture, we could be surfacing insidious bias throughout organisations. This paper examines how a future CAI tool could reduce bias and, eventually, teach people to re-evaluate and reframe their thinking. In a performance review setting, the system would flag problematic phrases as they are said, and committee heads would stop the conversation. The committee would then evaluate the comment, ask the presenter for further information, and only continue once there is sufficient clarity. Once the discussion concludes, the review cycle would continue until another phrase is identified. The system serves to be persistently aware throughout all conversations and highlight potential bias for everyone to learn from. Beyond pointing out biased phrases during a performance review, a combination of NLP and CAI can serve as a foundation for company-wide analytics. Organisations can track who is speaking in a majority of meetings, what was said, who challenges biased phrases, whether or not certain types of people are misrepresented in reviews more or less frequently, and so on. All this information gives a fundamentally new picture of what is happening inside a company, laying the groundwork for human resource (HR)-related metrics that individuals (and the company as a whole) can improve over time.

Multi-document summarization for patent documents based on generative adversarial network

Given the exponential growth of patent documents, automatic patent summarization methods to facilitate the patent analysis process are in strong demand. Recently, the development of natural language processing (NLP), text-mining, and deep learning has greatly improved the performance of text summarization models for general documents. However, existing models cannot be successfully applied to patent documents, because patent documents describing an inventive technology and using domain-specific words have many differences from general documents. To address this challenge, we propose in this study a multi-patent summarization approach based on deep learning to generate an abstractive summarization considering the characteristics of a patent. Single patent summarization and multi-patent summarization were performed through a patent-specific feature extraction process, a summarization model based on generative adversarial network (GAN), and an inference process using topic modeling. The proposed model was verified by applying it to a patent in the drone technology field. In consequence, the proposed model performed better than existing deep learning summarization models. The proposed approach enables high-quality information summary for a large number of patent documents, which can be used by R&D researchers and decision-makers. In addition, it can provide a guideline for deep learning research using patent data.

Enhancing Scene Text Recognition with Visual Context Information

This thesis addresses the problem of improving text spotting systems, which aim to detect and recognize text in unrestricted images (e.g., a street sign, an advertisement, a bus destination, etc.). The goal is to improve the performance of off-the-shelf vision systems by exploiting the semantic information derived from the image itself. The rationale is that knowing the content of the image or the visual context can help to decide which words are the correct candidate words. For example, the fact that an image shows a coffee shop makes it more likely that a word on a signboard reads as Dunkin and not unkind. We address this problem by drawing on successful developments in natural language processing and machine learning, in particular, learning to re-rank and neural networks, to present post-process frameworks that improve state-of-the-art text spotting systems without the need for costly data-driven re-training or tuning procedures. Discovering the degree of semantic relatedness of candidate words and their image context is a task related to assessing the semantic similarity between words or text fragments. However, semantic relatedness is more general than similarity (e.g., car, road , and traffic light are related but not similar) and requires certain adaptations. To meet the requirements of these broader perspectives of semantic similarity, we develop two approaches to learn the semantic relatedness of the spotted word and its environmental context: word-to-word (object) or word-to-sentence (caption). In the word-to-word approach, word embedding based re-rankers are developed. The re-ranker takes the words from the text spotting baseline and re-ranks them based on the visual context from the object classifier. For the second, an end-to-end neural approach is designed to drive image description (caption) at the sentence-level as well as the word-level (objects) and re-rank them based not only on the visual context but also on the co-occurrence between them. As an additional contribution, to meet the requirements of data-driven approaches such as neural networks, we propose a visual context dataset for this task, in which the publicly available COCO-text dataset 1 has been extended with information about the scene (including the objects and places appearing in the image) to enable researchers to include the semantic relations between texts and scene in their Text Spotting systems, and to offer a common evaluation baseline for such approaches. Awarded by: Universitat Politècnica de Catalunya, Barcelona, Spain on 10 September 2020. Supervised by: Lluís Padró and Francesc Moreno-Noguer. Available at: https://upcommons.upc.edu/handle/2117/334952.

A comparative analysis of transformer based models for figurative language classification

Efficient and effective methods are required to construct a model to rapidly extractdifferent sentiments from large volumes of text. To augment the performance of the models, contemporary developments in Natural Language Processing (NLP) have been utilized by researchers to work on several model architecture and pretraining tasks. This work explores several models based on transformer architecture and analyses its performance. In this work, the researchersusea dataset to answer the question of whether or not transformers work significantly well for figurative language and not just literal language classification. The results of various models are compared and have come up as a result of research over time. The studyexplains why it is necessary for computers to understand the occurrence of figurative language, why it is yet a challenge and is being intensively worked on to date, and how it is different from literal language classification. This research also covers how well these models train on a specific type of figurative language and generalize on a few other similar types.

MatSciBERT: A materials domain language model for text mining and information extraction

A large amount of materials science knowledge is generated and stored as text published in peer-reviewed scientific literature. While recent developments in natural language processing, such as Bidirectional Encoder Representations from Transformers (BERT) models, provide promising information extraction tools, these models may yield suboptimal results when applied on materials domain since they are not trained in materials science specific notations and jargons. Here, we present a materials-aware language model, namely, MatSciBERT, trained on a large corpus of peer-reviewed materials science publications. We show that MatSciBERT outperforms SciBERT, a language model trained on science corpus, and establish state-of-the-art results on three downstream tasks, named entity recognition, relation classification, and abstract classification. We make the pre-trained weights of MatSciBERT publicly accessible for accelerated materials discovery and information extraction from materials science texts.

Leveraging Multilingual News Websites for Building a Kurdish Parallel Corpus

Machine translation has been a major motivation of development in natural language processing. Despite the burgeoning achievements in creating more efficient machine translation systems, thanks to deep learning methods, parallel corpora have remained indispensable for progress in the field. In an attempt to create parallel corpora for the Kurdish language, in this article, we describe our approach in retrieving potentially alignable news articles from multi-language websites and manually align them across dialects and languages based on lexical similarity and transliteration of scripts. We present a corpus containing 12,327 translation pairs in the two major dialects of Kurdish, Sorani and Kurmanji. We also provide 1,797 and 650 translation pairs in English-Kurmanji and English-Sorani. The corpus is publicly available under the CC BY-NC-SA 4.0 license. 1

Quality indicators in colonoscopy: the chasm between ideal and reality.

Continuous measurement of quality indicators (QIs) should be a routine part of colonoscopy, as a wide variation still exists in the performance and quality levels of colonoscopy in Korea. Among the many QIs of colonoscopy, the adenoma detection rate, average withdrawal time, bowel preparation adequacy, and cecal intubation rate should be monitored in daily clinical practice to improve the quality of the procedure. The adenoma detection rate is the best indicator of the quality of colonoscopy; however, it has many limitations for universal use in daily practice. With the development of natural language processing, the adenoma detection rate is expected to become more effective and useful. It is important that colonoscopists do not strictly and mechanically maintain an average withdrawal time of 6 minutes but instead perform careful colonoscopy to maximally expose the colonic mucosa with a withdrawal time of at least 6 minutes. To achieve adequate bowel preparation, documentation of bowel preparation with the Boston Bowel Preparation Scale (BBPS) should be a routine part of colonoscopy. When colonoscopists routinely followed the bowel preparation protocols, ≥85% of outpatient screening colonoscopies had a BBPS score of ≥6. In addition, the cecal intubation rate should be ≥95% of all screening colonoscopies. The first step in improving colonoscopy quality in Korea is to apply these key performance measurements in clinical practice.