Domain Terminology Research Articles

PreparedLLM: effective pre-pretraining framework for domain-specific large language models

ABSTRACT The direct application of large language models (LLMs) to specific domain tasks frequently encounters challenges due to the scarcity of domain data, variations in domain semantics, and the complexity of domain knowledge. Further pretraining of advanced foundational models on extensive domain-specific corpora can infuse these models with domain-specific knowledge and enhancing their ability of solving domain-specific tasks. However, the development of most domain-specific models focuses primarily on collecting large-scale domain data, often overlooking the crucial optimization of the pre-pretraining stage, which significantly impacts both model performance and training efficiency. This paper introduces PRE -PretrAining FRamEwork for Domain-specific Large Language Models (PreparedLLM), a framework designed to enhance the pre-pretraining stage for domain specialization of LLMs. PreparedLLM employs advanced techniques in data recipe, data cleaning, vocabulary expansion, and embedding initialization. These techniques are implemented to optimize both the composition and quality of the training data, and to enhance understanding of domain terminologies and concepts, as well as improve token embedding initialization. Utilizing the geoscience domain as a case study, this paper applies PreparedLLM for the domain specialization of the Llama, a widely recognized general-purpose LLM. Experimental results demonstrate that PreparedLLM enhances model convergence speed, training speed, inference speed, the text volume of the context window, and overall performance in domain specialization. The utilization of PreparedLLM in developing domain-specific LLMs has significantly increased performance while reducing both time and resource investment. The case study provides valuable insights into the development of domain-specific LLMs.

Applying RFID and NLP for efficient warehouse picking

This paper proposes an intelligent warehouse-picking approach using radio frequency identification (RFID) indoor positioning and natural language processing (NLP) speech recognition. A forward maximum matching algorithm segments speech into domain terminology. Location was estimated by RFID signal strengths between reference tags and pickers. Simulation results demonstrated a 50% reduction in segmentation runtime versus conventional methods. Speech recognition accuracy reached 90–95%, improving by 23% over baseline. Positioning accuracy also increased substantially. The techniques can reduce picking errors and costs. Further work should evaluate performance in real-world environments.

PMD Core Ontology: Achieving semantic interoperability in materials science

Knowledge representation in the Materials Science and Engineering (MSE) domain is a vast and multi-faceted challenge: Overlap, ambiguity, and inconsistency in terminology are common. Invariant (consistent) and variant (context-specific) knowledge are difficult to align cross-domain. Generic top-level semantic terminology often is too abstract, while MSE domain terminology often is too specific. In this paper, an approach how to maintain a comprehensive MSE-centric terminology composing a mid-level ontology–the Platform MaterialDigital Core Ontology (PMDco)–via MSE community-based curation procedures is presented. The illustrated findings show how the PMDco bridges semantic gaps between high-level, MSE-specific, and other science domain semantics. Additionally, it demonstrates how the PMDco lowers development and integration thresholds. Moreover, the research highlights how to fuel it with real-world data sources ranging from manually conducted experiments and simulations with continuously automated industrial applications.

Machine Translation of Electrical Terminology Constraints

In practical applications, the accuracy of domain terminology translation is an important criterion for the performance evaluation of domain machine translation models. Aiming at the problem of phrase mismatch and improper translation caused by word-by-word translation of English terminology phrases, this paper constructs a dictionary of terminology phrases in the field of electrical engineering and proposes three schemes to integrate the dictionary knowledge into the translation model. Scheme 1 replaces the terminology phrases of the source language. Scheme 2 uses the residual connection at the encoder end after the terminology phrase is replaced. Scheme 3 uses a segmentation method of combining character segmentation and terminology segmentation for the target language and uses an additional loss module in the training process. The results show that all three schemes are superior to the baseline model in two aspects: BLEU value and correct translation rate of terminology words. In the test set, the highest accuracy of terminology words was 48.3% higher than that of the baseline model. The BLEU value is up to 3.6 higher than the baseline model. The phenomenon is also analyzed and discussed in this paper.

An Ontology-Based Cost Estimation for Offsite Construction

Design for manufacturing and assembly (DfMA) has been widely applied to support the decision-making process in offsite construction. With a DfMA approach, cost estimation requires taking product design and production processes into consideration. Current studies conduct cost estimation built upon quantity take-offs. However, they do not provide a vocabulary to relate cost estimates to offsite construction processes. This paper presents a new domain ontology, Offsite Housing Ontology (OHO) using the NeOn methodology framework to support cost estimation considering products, resources, and production processes. OHO semantically defines offsite construction domain terminology and relationships. This supports a unified model, required for efficient collaborative design management. The efficiency and effectiveness of the OHO approach are demonstrated in a real-world DfMA scenario through the development of a Knowledge-Based Engineering tool to automate cost estimation. The approach can be adapted and extended to accommodate a very wide range of offsite housing, delivering important optimization and automation benefit from DfMA solutions.

The Application of Artificial Intelligence in the Dissemination of Local Culture in Hainan Province: Taking the Dissemination of Bai Yuchan’s Thoughts as an Example

Bai Yuchan’s Taoist thought is an important part of Taoist health-preserving thought. Excavating, sorting out, and translating Bai Yuchan’s Taoist thought will not only help increase cultural self-confidence and protect traditional culture but also become an important medium for foreign exchanges. With the advent of the digital age, artificial intelligence has helped the dissemination of excellent traditional Chinese culture with its unique technological advantages, improving the effectiveness, intensity, and breadth of cultural dissemination. In domain machine translation, whether domain terms can be correctly translated plays a decisive role in the translation quality. It is of practical significance to effectively integrate domain terms into neural machine translation models and improve the translation quality of domain terms. This paper proposes a method of incorporating new Bai Yuchan’s thought term information as prior knowledge into neural machine translation. Using the term dictionary constructed from Bai Yuchan’s thought bilingual terminology knowledge base as a medium, two different knowledge integration methods are proposed and compared: (1) term replacement, which means using the target term to replace the source term on the source language side, and (2) term addition, which means splicing the source term and the target term on the source language side and both the source language side and the target language side. Use identifiers as special external knowledge to identify the beginning and end of the target term. Based on the Chinese-English bilingual alignment corpus of New Bai Yuchan’s thoughts and the constructed Chinese-English alignment termbase, the experiments are carried out. The results show that on the test set, the BLEU value of the proposed method is 6.38 and 6.38 higher than the baseline experiments, respectively, which proves that the proposed method can effectively incorporate domain terminology knowledge into the translation model and improve the translation quality of domain terminology.

Discovering novel drug-supplement interactions using SuppKG generated from the biomedical literature

ObjectiveDevelop a novel methodology to create a comprehensive knowledge graph (SuppKG) to represent a domain with limited coverage in the Unified Medical Language System (UMLS), specifically dietary supplement (DS) information for discovering drug-supplement interactions (DSI), by leveraging biomedical natural language processing (NLP) technologies and a DS domain terminology. Materials and MethodsWe created SemRepDS (an extension of an NLP tool, SemRep), capable of extracting semantic relations from abstracts by leveraging a DS-specific terminology (iDISK) containing 28,884 DS terms not found in the UMLS. PubMed abstracts were processed using SemRepDS to generate semantic relations, which were then filtered using a PubMedBERT model to remove incorrect relations before generating SuppKG. Two discovery pathways were applied to SuppKG to identify potential DSIs, which are then compared with an existing DSI database and also evaluated by medical professionals for mechanistic plausibility. ResultsSemRepDS returned 158.5% more DS entities and 206.9% more DS relations than SemRep. The fine-tuned PubMedBERT model (significantly outperformed other machine learning and BERT models) obtained an F1 score of 0.8605 and removed 43.86% of semantic relations, improving the precision of the relations by 26.4% over pre-filtering. SuppKG consists of 56,635 nodes and 595,222 directed edges with 2,928 DS-specific nodes and 164,738 edges. Manual review of findings identified 182 of 250 (72.8%) proposed DS-Gene-Drug and 77 of 100 (77%) proposed DS-Gene1-Function-Gene2-Drug pathways to be mechanistically plausible. DiscussionWith added DS terminology to the UMLS, SemRepDS has the capability to find more DS-specific semantic relationships from PubMed than SemRep. The utility of the resulting SuppKG was demonstrated using discovery patterns to find novel DSIs. ConclusionFor the domain with limited coverage in the traditional terminology (e.g., UMLS), we demonstrated an approach to leverage domain terminology and improve existing NLP tools to generate a more comprehensive knowledge graph for the downstream task. Even this study focuses on DSI, the method may be adapted to other domains.

Engineering Document Summarization: A Bidirectional Language Model-Based Approach

AbstractIn this study, the extractive summarization using sentence embeddings generated by the finetuned Bidirectional Encoder Representations from Transformers (BERT) models and the k-means clustering method has been investigated. To show how the BERT model can capture the knowledge in specific domains like engineering design and what it can produce after being finetuned based on domain-specific data sets, several BERT models are trained, and the sentence embeddings extracted from the finetuned models are used to generate summaries of a set of papers. Different evaluation methods are then applied to measure the quality of summarization results. Both the machine evaluation method Recall-Oriented Understudy for Gisting Evaluation (ROUGE) and a human-based evaluation method are used for the comparison study. The results indicate that the BERT model finetuned with a larger dataset can generate summaries with more domain terminologies than the pretrained BERT model. Moreover, the summaries generated by BERT models have more contents overlapping with original documents than those obtained through other popular non-BERT-based models. The experimental results indicate that the BERT-based method can provide better and more informative summaries to engineers. It has also been demonstrated that the contextualized representations generated by BERT-based models can capture information in text and have better performance in applications like text summarizations after being trained by domain-specific data sets.

NMF-based approach to automatic term extraction

This work describes automatic term extraction approach based on the combination of the probabilistic topic modelling (PTM) and non-negative matrix factorization (NMF). Topic modeling algorithms including NMF-based ones do not require expensive and time-consuming manual annotations for domain terms, but only a corpus of domain documents. The topics emerge from the corpus documents without any supervision as sets of most probable words. This work is aimed to investigate how fully and precisely these most probable words from topics can reflect domain terminology. We run a series of experiments on the novel, qualitatively annotated dataset ACTER that was first used in the TermEval 2020 Shared Task. We compare five different NMF algorithms and four different NMF initializations when changing the number of topics extracted from documents and the number of most probable words extracted from topics in order to determine optimal combinations for best performance of term extraction. Finally, we compare the obtained optimal combinations of NMF with the competitive methods in TermEval 2020 and prove that our approach is second only to two much more sophisticated, domain-dependent supervised methods.

OntoDomus: A Semantic Model for Ambient Assisted Living System Based on Smart Homes

Ambient assisted living (AAL) makes it possible to build assistance for older adults according to the person’s context. Understanding the person’s context sometimes involves transforming one’s home into a smart home. Typically, this is carried out using nonintrusively distributed sensors and calm technologies. Older adults often have difficulty performing activities of daily living, such as taking medication, drinking coffee, watching television, using certain electronic devices, and dressing. This difficulty is even greater when these older adults suffer from cognitive impairments. Defining an assistance solution requires a multidisciplinary and iterative collaborative approach. It is necessary, therefore, to reason about the imperatives and solutions of this multidisciplinary collaboration (e.g., clinical), as well as the adaptation of technical constraints (e.g., technologies). A common approach to reasoning is to represent knowledge using logic-based formalisms, such as ontologies. However, there is not yet an established ontology that defines concepts such as multidisciplinary collaboration in successive stages of the assistance process. This article presents OntoDomus, an ontology that describes, at several levels, the semantic interactions between ambient assisted living, context awareness, smart home, and Internet of Things, based on multidisciplinarity. It revolves around two main notions: multidisciplinarity, based on specific sub-ontologies and the ambient feedback loop. OntoDomus combines SPARQL queries and OWL 2 models to improve the reusability of domain terminology, allowing stakeholders to represent their knowledge in different collaborative and adaptive situations. The ontological model is validated, first by its reuse in more specific works—specific to an aspect of ambient assistance. Second, it is validated by the structuring of ambient knowledge and inferences of the formalization in a case study that includes instances for a particular activity of daily living. It places the ambient feedback loop at the center of the ontology by focusing on highly expressive domain ontology formalisms with a low level of expressiveness between them.

Poland’s Energy Transition: Towards an Ontology

This article depicts the process of creating a Poland’s energy transition ontology based on the extraction of domain-specific terminology from parliamentary debate transcriptions. Statements associated with the energy transition were taken from the Polish Parliamentary Corpus. Then, the corpora of statements was processed using TermoPL, a tool for extracting specialised domain terminology. Based on the generated list of key terms, an OWL ontology has been constructed using the Protégé editor. The resulting ontology comprises 90 classes, 119 instances and 11 data properties. The ontology is available online in two languages (Polish and English) for evaluation, further development or use in specialist applications in organisations.

MHeTRep: A multilingual semantically tagged health terms repository

AbstractThis paper presents MHeTRep, a multilingual medical terminology and the methodology followed for its compilation. The multilingual terminology is organised into one vocabulary for each language. All the terms in the collection are semantically tagged with a tagset corresponding to the top categories of Snomed-CT ontology. When possible, the individual terms are linked to their equivalent in the other languages. Even though many NLP resources and tools claim to be domain independent, their application to specific tasks can be restricted to specific domains, otherwise their performance degrades notably. As the accuracy of NLP resources drops heavily when applied in environments different from which they were built, a tuning to the new environment is needed. Usually, having a domain terminology facilitates and accelerates the adaptation of general domain NLP applications to a new domain. This is particularly important in medicine, a domain living moments of great expansion. The proposed method takes Snomed-CT as starting point. From this point and using 13 multilingual resources, covering the most relevant medical concepts such as drugs, anatomy, clinical findings and procedures, we built a large resource covering seven languages totalling more than two million semantically tagged terms. The resulting collection has been intensively evaluated in several ways for the involved languages and domain categories. Our hypothesis is that MHeTRep can be used advantageously over the original resources for a number of NLP use cases and likely extended to other languages.

Mining Domain Terminologies Using Search Engine's Query Log

Domain terminologies are a basic resource for various natural language processing tasks. To automatically discover terminologies for a domain of interest, most traditional approaches mostly rely on a domain-specific corpus given in advance; thus, the performance of traditional approaches can only be guaranteed when collecting a high-quality domain-specific corpus, which requires extensive human involvement and domain expertise. In this article, we propose a novel approach that is capable of automatically mining domain terminologies using search engine's query log—a type of domain-independent corpus of higher availability, coverage, and timeliness than a manually collected domain-specific corpus. In particular, we represent query log as a heterogeneous network and formulate the task of mining domain terminology as transductive learning on the heterogeneous network. In the proposed approach, the manifold structure of domain-specificity inherent in query log is captured by using a novel network embedding algorithm and further exploited to reduce the need for the manual annotation efforts for domain terminology classification. We select Agriculture and Healthcare as the target domains and experiment using a real query log from a commercial search engine. Experimental results show that the proposed approach outperforms several state-of-the-art approaches.

ITEXT-BIO: Intelligent Term EXTraction for BIOmedical analysis

Here, we introduce ITEXT-BIO, an intelligent process for biomedical domain terminology extraction from textual documents and subsequent analysis. The proposed methodology consists of two complementary approaches, including free and driven term extraction. The first is based on term extraction with statistical measures, while the second considers morphosyntactic variation rules to extract term variants from the corpus. The combination of two term extraction and analysis strategies is the keystone of ITEXT-BIO. These include combined intra-corpus strategies that enable term extraction and analysis either from a single corpus (intra), or from corpora (inter). We assessed the two approaches, the corpus or corpora to be analysed and the type of statistical measures used. Our experimental findings revealed that the proposed methodology could be used: (1) to efficiently extract representative, discriminant and new terms from a given corpus or corpora, and (2) to provide quantitative and qualitative analyses on these terms regarding the study domain.

LADM for sustainable development: An exploratory study on the application of domain-specific data models to support the SDGs

Individual land titling will not deliver land tenure security to the majority of people by 2030. Nevertheless, tenure security is implicit to the 2030 Agenda for Sustainable Development and deep-rooted in the Sustainable Development Goals (SDGs). Land tenure security underpins the SDGs related to poverty reduction, food security, disaster risk management and gender equality. The cross-cutting benefits of land tenure security with regards to sustainable development suggests equally cross-cutting conceptual frameworks, management cooperation and data sharing are needed within, and between, those sectors. Additionally, previous research in the land administration domain developed approaches, frameworks and data models linking across different domains. This latter achievement showed how developments using standardised modelling approaches and domain terminologies can be used to support gathering, organising and disseminating information relevant to reporting on and monitoring of the SDGs. From the perspective of data sharing, domain models - such as the Land Administration Domain Model LADM (ISO 19152), a globally recognized standard - are seen as supporting this capability. In response, this paper explores whether and how LADM, in terms of data model additions and adaptations, could support other SDG-relevant domains. The approach used is exploratory, including elements of literature review, case study analysis, conceptual data modelling, and synthesis. It shows the potential and added value of an integrated data modelling approach, with regards to Land Administration and its intrinsic feature to other fields of study including poverty reduction, food security, and gender equality. • Data models using standards, such as the LADM, can be used to support gathering and disseminating information relevant to the SDGs. • There is value in creating shared understandings about the value and role of shared data models between different domains, and country contexts. • Both models, LADM as well as STDM, are flexible and can accommodate other attributes and associations and are extensible to link disciplines. • The interrelationship of Land Administration with various other disciplines reflected in the SDGs becomes visible through data modelling. • A holistic approach strengthens the demand for interlinkages with the 2030 Agenda and its 5Ps and the domain of Land Administration.

Using interactive semantic networks as an augmented reality element in autonomous learning

This paper focuses on the using augmented reality elements in the educational process, in particular in autonomous learning. The possibility of using interactive semantic networks in learning foreign languages is considered. The technology of selection and structuring of specialised terminology on the basis of interactive semantic networks for their further use in the study of foreign languages and mastering automated translation systems has been developed and proposed. The criteria for creating and supplementing terminological databases with appropriate structuring of the domain terminology selected on the basis of interactive semantic networks have been defined, namely universality, structurability, convertibility, extensibility. The possibility of further use of terminology bases for foreign language learning using mobile applications, mastering Computer Aided Translation (CAT) systems, mastering Computer Aided Interpretation (CAI) are outlined. Based on the practical use of the developed technology in the process of autonomous training of translators, positive results have been concretised and areas for further activities in their technological training have been identified.

Comparison of Knowledge Databases to Be Used in Automated Monitoring of Orthopedic Medical Devices.

Surveillance and traceability of medical devices (MD) is a challenge in health care systems. In the perspective of reusing EHR data to automate the monitoring of medical devices, we carried out a comparison of the main MD knowledge bases (MD-KDB) currently available in France. Four MD-KDBs (ANSM, Gudid, Exhausmed and CIOdm) were compared quantitatively and through an example of a shoulder prosthesis. The number of MDs registered differs from one MD-KDB to another. Domain terminologies used in MD-KDBs differ in terms of granularity and in the ease of querying. Waiting EUDAMED, the European MD-KDB, it seems necessary so far to use and combine information coming from several MD-KDBs to address MD monitoring.

A Formal Framework of Software Product Line Analyses

A number of product-line analysis approaches lift analyses such as type checking, model checking, and theorem proving from the level of single programs to the level of product lines. These approaches share concepts and mechanisms that suggest an unexplored potential for reuse of key analysis steps and properties, implementation, and verification efforts. Despite the availability of taxonomies synthesizing such approaches, there still remains the underlying problem of not being able to describe product-line analyses and their properties precisely and uniformly. We propose a formal framework that models product-line analyses in a compositional manner, providing an overall understanding of the space of family-based, feature-based, and product-based analysis strategies. It defines precisely how the different types of product-line analyses compose and inter-relate. To ensure soundness, we formalize the framework, providing mechanized specification and proofs of key concepts and properties of the individual analyses. The formalization provides unambiguous definitions of domain terminology and assumptions as well as solid evidence of key properties based on rigorous formal proofs. To qualitatively assess the generality of the framework, we discuss to what extent it describes five representative product-line analyses targeting the following properties: safety, performance, dataflow facts, security, and functional program properties.

RELEVANCE OF TRANSLATION IN ESP TEACHING METHODOLOGY

This paper offers an observation and discussion on the relevance and scope of presence of translation within English for Specific Purposes courses in academic settings of the Faculty of Electronic Engineering and Faculty of Medicine, University of Niš. There is concern with methodological issues of translation skills regarding domain terminology. This subsumes translation methodology, finding precise target language equivalents, as well as determining whether to translate a text, respecting the syntax of the target language, or to paraphrase it with the primary aim to most precisely convey information contained in the original. The main argument and the issue of the survey conducted with the students of both faculties is direct correlation between translation, or absence of it for that matter, and the cognition of the domain content taught. Then, there is an emphasis on a wider social context that implies inducing in students the awareness of the responsibility they as future professionals and experts in their field have towards their own mother tongue in the dissemination of content knowledge. It is common in the Serbian scientific and professional environment that the translation of scientific texts in English is done by domain experts without consulting linguists, philologists of Serbian language. The consequence is mere (and often incorrect) transliteration of English scientific terminology, and ultimately uncurbed influx of foreign terminology. Yet another argument on the relevance of the previous one is that translation enhances soft transversal skills such as thoughtfulness and critical thinking which among others benefit comprehension of the domain knowledge, as it has been noted that professors, and then students use the English terminology while speaking Serbian, where the latter group do not fully understand the content presented in such scientific ‘newspeak’. The paper presents concrete teaching practice which has for its goal inducing the students with cautiousness when choosing professional terminology in either language.

Data modeling and evaluation of deep semantic annotation for cultural heritage images

PurposeTo meet the emerging demand for fine-grained annotation and semantic enrichment of cultural heritage images, this paper proposes a new approach that can transcend the boundary of information organization theory and Panofsky's iconography theory.Design/methodology/approachAfter a systematic review of semantic data models for organizing cultural heritage images and a comparative analysis of the concept and characteristics of deep semantic annotation (DSA) and indexing, an integrated DSA framework for cultural heritage images as well as its principles and process was designed. Two experiments were conducted on two mural images from the Mogao Caves to evaluate the DSA framework's validity based on four criteria: depth, breadth, granularity and relation.FindingsResults showed the proposed DSA framework included not only image metadata but also represented the storyline contained in the images by integrating domain terminology, ontology, thesaurus, taxonomy and natural language description into a multilevel structure.Originality/valueDSA can reveal the aboutness, ofness and isness information contained within images, which can thus meet the demand for semantic enrichment and retrieval of cultural heritage images at a fine-grained level. This method can also help contribute to building a novel infrastructure for the increasing scholarship of digital humanities.