Domain-specific Vocabulary Research Articles

Framework for automation of short answer grading based on domain-specific pre-training

In the education sector, there is a growing need for efficient Automatic Short Answer Grading (ASAG) systems that require less training time, have faster inference, and provide higher accuracy. Over the past five years, researchers have made progress using deep learning for ASAG, but acquiring domain-specific labeled data remains a significant challenge. This study introduces an ASAG framework designed to learn from unlabeled domain-specific contexts while minimizing training and inference times. Our framework involves three main phases: 1) Building an unlabeled domain-specific corpus. 2) Training a pre-trained language model with this corpus. 3) Fine-tuning the pre-trained model with limited labeled domain-specific data. We explored various domain adaptations using two types of vocabulary during the pretraining stage. Our approach achieved an accuracy of 88.77% and an F1-score of 91.59% on the SPRAG dataset, improving state-of-the-art methods by 4%–5.7%. The models, including one trained from scratch with domain-specific data and vocabulary, showed robust performance, enhancing accuracy by 5.7% and F1-score by 4.84%. However, the framework's effectiveness in handling different languages or multilingual datasets remains to be explored. This framework's adaptability and efficiency enable educators to use AI for automated grading in various educational domains.

Information extraction from medical case reports using OpenAI InstructGPT

Background and objectiveResearchers commonly use automated solutions such as Natural Language Processing (NLP) systems to extract clinical information from large volumes of unstructured data. However, clinical text's poor semantic structure and domain-specific vocabulary can make it challenging to develop a one-size-fits-all solution. Large Language Models (LLMs), such as OpenAI's Generative Pre-Trained Transformer 3 (GPT-3), offer a promising solution for capturing and standardizing unstructured clinical information. This study evaluated the performance of InstructGPT, a family of models derived from LLM GPT-3, to extract relevant patient information from medical case reports and discussed the advantages and disadvantages of LLMs versus dedicated NLP methods. MethodsIn this paper, 208 articles related to case reports of foreign body injuries in children were identified by searching PubMed, Scopus, and Web of Science. A reviewer manually extracted information on sex, age, the object that caused the injury, and the injured body part for each patient to build a gold standard to compare the performance of InstructGPT. ResultsInstructGPT achieved high accuracy in classifying the sex, age, object and body part involved in the injury, with 94%, 82%, 94% and 89%, respectively. When excluding articles for which InstructGPT could not retrieve any information, the accuracy for determining the child's sex and age improved to 97%, and the accuracy for identifying the injured body part improved to 93%. InstructGPT was also able to extract information from non-English language articles. ConclusionsThe study highlights that LLMs have the potential to eliminate the necessity for task-specific training (zero-shot extraction), allowing the retrieval of clinical information from unstructured natural language text, particularly from published scientific literature like case reports, by directly utilizing the PDF file of the article without any pre-processing and without requiring any technical expertise in NLP or Machine Learning. The diverse nature of the corpus, which includes articles written in languages other than English, some of which contain a wide range of clinical details while others lack information, adds to the strength of the study.

Effects of tier 1 content literacy intervention on early-grade English learners’ reading and writing: Exploring the mediating roles of domain-specific vocabulary and oral language proficiency.

Effects of tier 1 content literacy intervention on early-grade English learners’ reading and writing: Exploring the mediating roles of domain-specific vocabulary and oral language proficiency.

Supplemental Material for Effects of Tier 1 Content Literacy Intervention on Early-Grade English Learners’ Reading and Writing: Exploring the Mediating Roles of Domain-Specific Vocabulary and Oral Language Proficiency

Supplemental Material for Effects of Tier 1 Content Literacy Intervention on Early-Grade English Learners’ Reading and Writing: Exploring the Mediating Roles of Domain-Specific Vocabulary and Oral Language Proficiency

A unified approach to publish semantic annotations of agricultural documents as knowledge graphs

The research results presented in this paper were obtained as part of the D2KAB project (Data to Knowledge in Agriculture and Biodiversity) which aims to develop semantic web-based tools to describe and make agronomical data actionable and accessible following the FAIR principles. We focus on constructing domain-specific Knowledge Graphs (KGs) from textual data sources, using Natural Language Processing (NLP) techniques to extract and structure relevant entities. Our approach is based on the formalization of a semantic data model using common linked open vocabularies such as the Web Annotation Ontology (OA) and the Provenance Ontology (PROV). The model was developed by formulating motivating scenarios and competency questions from domain experts. This model has been used to construct three different KGs from three distinct corpora: PubMed scientific publications on wheat and rice genetics and phenotyping, and French agricultural alert bulletins. The named entities to be recognized include genes, phenotypes, traits, genetic markers, taxa and phenological stages normalized using semantic resources such as the Wheat Trait and Phenotype Ontology (WTO), the French Crop Usage (FCU) thesaurus and the Plant Phenological Description Ontology (PPDO). Named entities were extracted using different NLP approaches and tools. The relevance of the semantic model was validated by implementing experts questions as SPARQL queries to be answered on the constructed RDF knowledge graphs. Our work demonstrates how domain-specific vocabularies and systematic querying of KGs can reveal hidden interactions and support agronomists in navigating vast amounts of data. The resources and transformation pipelines developed are publicly available in Git repositories.

Improving second-grade reading comprehension through a sustained content literacy intervention: A mixed-methods study examining the mediating role of domain-specific vocabulary.

Improving second-grade reading comprehension through a sustained content literacy intervention: A mixed-methods study examining the mediating role of domain-specific vocabulary.

A corpus-based real-time text classification and tagging approach for social data

With the rapid accumulation of large amounts of user-generated content through social media, social data reuse and integration have gained increasing attention recently. This has made it almost obsolete for software applications to collect, store, and work with their own data stored on local servers. While, with the provision of Application Programming Interfaces from the leading social networking sites, data acquisition and integration has become possible, the meaningful usage of such unstructured, non-uniform, and incoherent data collections needs special procedures of data summarization, understanding, and visualization. One particular aspect in this regard that needs special attention is the procedures for data (text snippets in the form of social media posts) categorization and concept tagging to filter out the relevant and most suitable data for the particular audience and for the particular purpose. In this regard, we propose a corpus-based approach for searching and successively categorizing and tagging the social data with relevant concepts in real time. The proposed approach is capable of addressing the semantical and morphological similarities, as well as domain-specific vocabularies of query strings and tagged concepts. We demonstrate the feasibility and application of our proposed approach in a web-based tool that allows searching Facebook posts and provides search results together with a concept map for further navigation, filtering, and refining of search results. The tool has been evaluated by performing multiple search queries, and resultant concept maps and annotated texts are analyzed in terms of their precision. The approach is thereby found effective in achieving its stated goal of classifying text snippets in real time.

Machine Translation and Natural Language Processing

Machine translation and natural language processing are powerful tools that allow for the automated translation of text from one language to another. Machine translation systems have evolved from rule-based systems to statistical and neural network-based approaches, with each approach having its own strengths and weaknesses. NLP, on the other hand, focuses on understanding the structure and meaning of natural language and includes tasks such as part-of-speech tagging, named entity recognition, and sentiment analysis. Thegoalofmachinetranslation,a subfield of natural language processing (NLP), is to create platform that translate text or speechautomatically.RBMT provides linguistic accuracy and control, but linguistic resources must be built and maintained meticulously. However, SMT makes use of large-scale parallel corpora to automatically learn translation patterns, making it adaptable to new language pairs and context-dependent translations.However, SMT may struggle with handling grammatical nuances and domain-specific vocabulary. Hybrid approaches combining the best of both RBMT andSMT and newer approaches like neural machine translation (NMT) are being explored to overcome the limitations and improve the quality of translation output.

Arabic Opinion Mining System for E-Services Based On Ontology and Machine Learning

With the growing use of E-Services in the Arab world, there is a need for efficient methods to analyze customer feedback and opinions. This research is proposing an Arabic opinion mining system for E-Services based on ontology and machine learning techniques. The system utilizes an Arabic ontology to capture domain-specific vocabulary and relationships between features and a machine learning model to classify customer feedback into positive, negative, or neutral sentiments. By evaluating the performance of the system using several domains datasets for Arabic customer reviews of various e-services, and the results show that the system achieves a high accuracy level. Comparing the performance of the proposed system to traditional polarity and subjectivity approaches.

ChatGPT-based biological and psychological data imputation

Missing data are a common problem for large cohort or longitudinal research and have been handled through data imputation. Based on simplified models such as linear or nonlinear interpolations, current imputation methods may not be accurate for real-life data such as biological and behavioral data. The purpose of this work was to explore the capability of ChatGPT, a powerful Large Language Model (LLM) developed by OpenAI, for biological and psychological data imputation. We tested the feasibility using data from the Human Connectome Project. Performance was evaluated by comparing the imputed data against known ground truth (GT) and measured with metrics like Pearson correlation coefficient (r), relative accuracy (MP), and mean absolute error (MAE). Comparative analyses with traditional imputation techniques are also conducted to demonstrate the superior efficacy of the ChatGPT as a data imputer. In summary, through customized data-to-text prompting engineering, ChatGPT can successfully capture intricate patterns and dependencies within biological data, resulting in precise imputations. Fine-tuning ChatGPT with domain-specific biological vocabulary with human in-loop as an interpreter enhances the accuracy and relevance of the imputations.

Identifying equipment health status from maintenance records using Lexicon based Unsupervised Sentiment Analysis Adjusted for Negation (LUSAA-N)

During periodic inspection and repair of industrial equipment, technicians create maintenance records in the form of unstructured text to document valuable information regarding the equipment condition, operating environment, failed components, failure mechanism, etc. Inferring the equipment health status from these records can help in planning future maintenance actions to avoid expensive machine failures. However, automatic inference of the equipment health status is non-trivial due to various challenges inherent in analyzing unstructured text such as domain-specific vocabulary and negation words. Further, manual labeling of the records for supervised classification is arduous and impractical. Thus, in this paper, we present a model to automatically extract the equipment health status from maintenance records in a completely unsupervised manner. We show that using general English sentiment lexicons is highly inefficient for analyzing maintenance records, and thus propose a method that leverages sentiment lexicons curated for the maintenance domain and addresses the central issue of negation. We demonstrate the effectiveness of our proposed model over extant research using data from real-life maintenance records from oil rig equipment.

Structure and associations of science vocabulary, general academic vocabulary, and science knowledge

We examined the structure of domain-specific academic science vocabulary, general academic vocabulary, and conceptual science knowledge and their impact on each other across two measurement points in a sample of 388 German third graders. Results show that the three constructs can be empirically differentiated. The high latent correlation between science vocabulary and general academic vocabulary suggests that science vocabulary is a language-related rather than a knowledge-related construct. Furthermore, science knowledge and science vocabulary predicted each other. By contrast, general academic vocabulary was only associated with science vocabulary but not with science knowledge acquisition. Neither science knowledge nor science vocabulary predicted general academic vocabulary at the end of the school year, suggesting it might be less sensitive to regular instruction.

Kontrollierte Vokabularien und historische Semantiken

Abstract Controlled vocabularies are a common component in applications of linked data in order to connect digital resources. There are applications in the Humanities and digital editions, too. Two scenarios are described in some detail. The first concerns the analysis of annotations on manuscripts of Aristotle’s De interpretatione. In conjunction with transcriptions, the usage of authority files in metadata as well of domain specific vocabularies helps in this example to find stemmatological relations between glosses and diagrams in some of the manuscripts. The second example is not about a text-critical application but about the creation of the thesaurus for a commentary on Michael Maier’s Atalanta fugiens, that is a multimodal, figurative and dense written book by today’s standards. The question is raised as to what extent a controlled vocabulary with its concepts and the implicit semantics of the relations between them can be of a heuristic value for the reconstruction of this historical, alchemical concepts in its dynamics and alterity.

An exploratory study on the relationships between positive emotions and target language vocabulary knowledge

ABSTRACT In line with increasing attention being paid to the role of positive emotions in second language (L2) proficiency and knowledge, this study set out to examine the relationships between domain-specific emotions (L2 grit, language learning curiosity, foreign language enjoyment, and foreign language classroom anxiety) and L2 vocabulary knowledge. To this end, we administered a questionnaire consisting of domain-specific instruments, which purport to measure the target emotions, as well as two widely used L2 vocabulary tests. College students studying English in the South Korean context participated in the study, and they completed both the questionnaire and the vocabulary tests. The results revealed that participants’ vocabulary test scores positively correlated with positive emotions, such as a sub-construct of L2 grit (consistency of interest), language learning curiosity, and foreign language enjoyment. In contrast, their scores negatively correlated with negative emotion – foreign language classroom anxiety. In the regression analysis, it was also found that a sub-construct of L2 grit (consistency of interest) and language learning curiosity were significant predictors for L2 vocabulary knowledge, while controlling for the effects of other variables.

Machine Learning Applications in Digital Humanities

This research study explores the potential of machine learning tools and techniques to organize knowledge objects pertaining to various aspects of the gender spectrum (LGBTQIA+) in order to address the low-resource features of the LGBTQIA+ knowledge domain in Indian libraries. It aims to develop a semi-automated subject indexing system using an open source machine learning framework (Annif) and deploying the Homosaurus, a domain-specific vocabulary system. It develops programmatically a comprehensive training dataset from open-access bibliographic data sources with the help of data carpentry tools and NLP services from OpenAI. The study also measures the efficiencies of the automated indexing framework and investigates the potential for widespread adoption of a REST/API call-based approach for rapid indexing of a substantial number of records related to the LGBTQIA+ domain.

A Speech Recognition Method Based on Domain-Specific Datasets and Confidence Decision Networks.

This paper proposes a speech recognition method based on a domain-specific language speech network (DSL-Net) and a confidence decision network (CD-Net). The method involves automatically training a domain-specific dataset, using pre-trained model parameters for migration learning, and obtaining a domain-specific speech model. Importance sampling weights were set for the trained domain-specific speech model, which was then integrated with the trained speech model from the benchmark dataset. This integration automatically expands the lexical content of the model to accommodate the input speech based on the lexicon and language model. The adaptation attempts to address the issue of out-of-vocabulary words that are likely to arise in most realistic scenarios and utilizes external knowledge sources to extend the existing language model. By doing so, the approach enhances the adaptability of the language model in new domains or scenarios and improves the prediction accuracy of the model. For domain-specific vocabulary recognition, a deep fully convolutional neural network (DFCNN) and a candidate temporal classification (CTC)-based approach were employed to achieve effective recognition of domain-specific vocabulary. Furthermore, a confidence-based classifier was added to enhance the accuracy and robustness of the overall approach. In the experiments, the method was tested on a proprietary domain audio dataset and compared with an automatic speech recognition (ASR) system trained on a large-scale dataset. Based on experimental verification, the model achieved an accuracy improvement from 82% to 91% in the medical domain. The inclusion of domain-specific datasets resulted in a 5% to 7% enhancement over the baseline, while the introduction of model confidence further improved the baseline by 3% to 5%. These findings demonstrate the significance of incorporating domain-specific datasets and model confidence in advancing speech recognition technology.

Developing a Construction Domain–Specific Artificial Intelligence Language Model for NCDOT’s CLEAR Program to Promote Organizational Innovation and Institutional Knowledge

Transportation agency personnel gain valuable knowledge through their work, but such knowledge is lost if it is not documented properly after the worker leaves the organization. The risk of losing institutional knowledge is a current problem at state departments of transportation, including the North Carolina Department of Transportation (NCDOT), due to high personnel turnover. State transportation agencies have implemented knowledge repositories in the form of lessons learned/best practices databases to address this problem. However, motivating end-users to use such databases is challenging. This paper addresses this challenge through novel artificial intelligence technology whereby a neural network–based language model is implemented as part of the NCDOT’s new knowledge management program: Communicate Lessons, Exchange Advice, Record (CLEAR). The CLEAR program encompasses a database of lessons learned/best practices and a website to access and search the database. The developed methodology involves training a language model on transportation construction texts and using that trained model in a novel algorithm enabling users to search the CLEAR database easily. The developed language-processing model provides an easily accessible interface to suggest the most relevant CLEAR data based on the end-user’s searched keywords. The model learns an inference model of construction domain–specific vocabulary extracted from various sources, such as contract documents, textbooks, and specifications, to make meaningful connections between lessons learned/best practices in the CLEAR database and project-specific knowledge. The developed model has been validated by project managers for projects at various life cycle stages. The automation of information retrieval is intended to encourage NCDOT personnel to use and embrace the CLEAR program as part of their routine work to improve project workflow. In the long run, the NCDOT will benefit from consistent usage of the CLEAR program and its high quality content, thereby leading to enhanced institutional knowledge and organizational innovation.

Fine-tuning large neural language models for biomedical natural language processing

Large neural language models have transformed modern natural language processing (NLP) applications. However, fine-tuning such models for specific tasks remains challenging as model size increases, especially with small labeled datasets, which are common in biomedical NLP. We conduct a systematic study on fine-tuning stability in biomedical NLP. We show that fine-tuning performance may be sensitive to pretraining settings and conduct an exploration of techniques for addressing fine-tuning instability. We show that these techniques can substantially improve fine-tuning performance for low-resource biomedical NLP applications. Specifically, freezing lower layers is helpful for standard BERT- models, while layerwise decay is more effective for BERT- and ELECTRA models. For low-resource text similarity tasks, such as BIOSSES, reinitializing the top layers is the optimal strategy. Overall, domain-specific vocabulary and pretraining facilitate robust models for fine-tuning. Based on these findings, we establish a new state of the art on a wide range of biomedical NLP applications.

ADS-Classif: A Robust Tool for Classifying Documents from Diverse Sources Using Weighted Domain-Specific Vocabulary

Extracting domain-specific words from the domain corpus to build vocabulary and identifying classes represented by domain-specific vocabulary is a challenging task in classifying text. Several comprehensive domain-specific techniques have been developed to build specific domain vocabulary but they lack generality and also do not consider the degree of the word's relevance. This paper proposes a supervised generic algorithmic model to create a domain-specific vocabulary. The model identifies domain-indicative word combinations from labeled real-time domain corpus, calculates their weights and logged their inclusion time to understand their semantic behavior in the present era of development. Further, a framework ADS-Classif is also developed using vocabulary to classify real-time documents extracted from different sources and test the domain-specific vocabulary's significance. Experimental results show that the approach leads to the accurate identification of domain classes for test documents compared with classical classifiers Naïve Bayes and Support Vector Machine.

How we measure language skills of children at scale: A call to move beyond domain-specific tests as a proxy for language

Purpose The aim of this research note is to encourage child language researchers and clinicians to give careful consideration to the use of domain-specific tests as a proxy for language; particularly in the context of large-scale studies and for the identification of language disorder in clinical practice. Method We report on data leveraged through the prospective Raine Study cohort. Participants included 1626 children aged 10 years (n = 104 with developmental language disorder [DLD] and n = 1522 without DLD). We assessed the predictive utility of common language measures including subtests of a standardised omnibus language assessment, non-verbal intelligence, and a domain-specific receptive vocabulary test. Result Children with DLD performed within the average range on a measure of non-verbal intelligence (z = −0.86) and receptive vocabulary (z = −0.38), as well as two out of the six subtests on the omnibus language assessment (zs > −1.50). The magnitude of the predictive relationship between language assessments and the likelihood of a child meeting criteria for DLD at 10 years was assessed using a logistic regression model, which was significant: χ 2(8) = 16.91, p = 0.031. Semantic Relationships (OR = 1.13, CI = 1.04 − 1.23, p = .004), Formulated Sentences (OR = 1.07, CI = 1.01 − 1.13, p = .028), Recalling Sentences (OR = 1.20, CI = 1.15 − 1.26, p < .001), and Sentence Assembly (OR = 1.17, CI = 1.07 − 1.30, p = .001) were significant predictors of DLD. Conclusion Domain-specific language assessments, particularly those testing receptive vocabulary, may overestimate the language ability of children with DLD. Caution is urged when using such tests by clinicians and researchers, especially those measuring language skills of children at scale. Future directions for measuring the functional impact of DLD are presented.