Domain-specific Modeling Research Articles

Non-linear Embedding Methods for Identifying Similar Brain Activity in 1 Million iEEG Records Captured From 256 RNS System Patients.

Finding electrophysiological features that are similar across patients with epilepsy may facilitate identifying treatment options for one patient that worked in patients with similar brain activity patterns. Three non-linear iEEG (intracranial electroencephalogram) embedding methods of finding similar cross-patient iEEG records in a large iEEG dataset were developed and compared. About 1 million iEEG records from 256 patients with drug-resistant focal onset seizures who were treated in prospective trials of the RNS System were used for analyses. Data from 200, 25, and 31 patients were randomly selected to be in the train, validation, and test datasets. In method 1, ResNet50 convolutional neural network (CNN) model pre-trained on the ImageNet dataset was used for extracting feature maps from spectrogram images (ImageNet-ResNet) of iEEG records. In method 2, ResNet50 custom trained on an iEEG classification task using ~138,000 manually labeled iEEG records was used as the feature extractor (ESC-ResNet). Feature maps were passed through dimensionality reduction and k nearest neighbors were found in the reduced feature space. In method 3, a 256 dimensional iEEG embedding space was learned via contrastive learning by training a ResNet50 model with triplet training sets generated using within-patient iEEG clustering (CL-ResNet). All three methods had comparable performance when identifying iEEG records from the search dataset similar to test iEEG records of baseline (non-seizure) and interictal spiking activity. Epileptic interictal spikes are represented by vertical (broadband) edges in spectrogram images, and hence even generic features extracted using models trained on everyday images appear to be sufficient to represent iEEG records with similar levels of interictal spiking activity in close proximity. In the case of electrographic seizures, however, the ESC-ResNet model, identified cross-patient iEEG records with electrographic seizure morphology features that were most similar to the test iEEG records. For nuanced electrographic seizure iEEG representation learning, domain specific model training with manually generated labels had the advantage. Finally, representative iEEG records were selected from every patient using an unsupervised clustering method which effectively reduced the number of iEEG records in the search dataset from ~750,000 to 2,148, thus substantially reducing the time required for finding similar cross-patient iEEG records.

Transfer learning using attentions across atomic systems with graph neural networks (TAAG).

Recent advances in Graph Neural Networks (GNNs) have transformed the space of molecular and catalyst discovery. Despite the fact that the underlying physics across these domains remain the same, most prior work has focused on building domain-specific models either in small molecules or in materials. However, building large datasets across all domains is computationally expensive; therefore, the use of transfer learning (TL) to generalize to different domains is a promising but under-explored approach to this problem. To evaluate this hypothesis, we use a model that is pretrained on the Open Catalyst Dataset (OC20), and we study the model's behavior when fine-tuned for a set of different datasets and tasks. This includes MD17, the *CO adsorbate dataset, and OC20 across different tasks. Through extensive TL experiments, we demonstrate that the initial layers of GNNs learn a more basic representation that is consistent across domains, whereas the final layers learn more task-specific features. Moreover, these well-known strategies show significant improvement over the non-pretrained models for in-domain tasks with improvements of 53% and 17% for the *CO dataset and across the Open Catalyst Project (OCP) task, respectively. TL approaches result in up to 4× speedup in model training depending on the target data and task. However, these do not perform well for the MD17 dataset, resulting in worse performance than the non-pretrained model for few molecules. Based on these observations, we propose transfer learning using attentions across atomic systems with graph Neural Networks (TAAG), an attention-based approach that adapts to prioritize and transfer important features from the interaction layers of GNNs. The proposed method outperforms the best TL approach for out-of-domain datasets, such as MD17, and gives a mean improvement of 6% over a model trained from scratch.

Designing a Business Intelligence and Analytics Maturity Model for Higher Education: A Design Science Approach

Business Intelligence and Analytics (BIA) systems play an essential role in organizations, providing actionable insights that enable business users to make more informed, data-driven decisions. However, many Higher Education (HE) institutions do not have accessible and usable models to guide them through the incremental development of BIA solutions to realize the full potential value of BIA. The situation is becoming ever more acute as HE operates today in a complex and dynamic environment brought forward by globalization and the rapid development of information technologies. This paper proposes a domain-specific BIA maturity model (MM) for HE–the HE-BIA Maturity Model. Following a design science approach, this paper details the design, development, and evaluation of two artifacts: the MM and the maturity assessment method. The evaluation phase comprised three case studies with universities from different countries and two workshops with practitioners from more than ten countries. HE institutions reported that the assessment with the HE-BIA model was (i) useful and adequate for their needs; (ii) and contributed to a better understanding of the current status of their BIA landscape, making it explicit that a BIA program is a technology endeavor as well as an organizational development.

A systematic literature review on counterexample explanation

Context: Safety is of paramount importance for cyber-physical systems in domains such as automotive, robotics, and avionics. Formal methods such as model checking are one way to ensure the safety of cyber-physical systems. However, adoption of formal methods in industry is hindered by usability issues, particularly the difficulty of understanding model checking results. Objective: We want to provide an overview of the state of the art for counterexample explanation by investigating the contexts, techniques, and evaluation of research approaches in this field. This overview shall provide an understanding of current and guide future research. Method: To provide this overview, we conducted a systematic literature review. The survey comprises 116 publications that address counterexample explanations for model checking. Results: Most primary studies provide counterexample explanations graphically or as traces, minimize counterexamples to reduce complexity, localize errors in the models expressed in the input formats of model checkers, support linear temporal logic or computation tree logic specifications, and use model checkers of the Symbolic Model Verifier family. Several studies evaluate their approaches in safety-critical domains with industrial applications. Conclusion: We notably see a lack of research on counterexample explanation that targets probabilistic and real-time systems, leverages the explanations to domain-specific models, and evaluates approaches in user studies. We conclude by discussing the adequacy of different types of explanations for users with varying domain and formal methods expertise, showing the need to support laypersons in understanding model checking results to increase adoption of formal methods in industry.

An Integrated Intelligent Modeling and Simulation Language for Model-based Systems Engineering

Modeling and simulation are now leading the way in supporting analysis and development of system of systems. At present, to support the unified modeling formalism and dynamic simulation for different domain specific models across MBSE process, system modeling languages (such as SysML) are often required to cooperate with multi-physics modeling languages and simulation platforms (such as Modelica, Simulink), which makes it challenging to ensure the true unity of the whole system models, the consistency between the various system layers and the traceability of the modeling and simulation processes. In response to the above problems, this paper develops a new integrated intelligent modeling and simulation language, which can uniformly describe the system-level architecture and physical behavior models as a whole. On this basis, models can be simulated directly to support system verification for MBSE. Compiler and simulation engine are developed to enable X language to support the simulation of continuous, discrete event and agent models. Finally, an intelligent car system is taken as a case to verify the modeling and simulation capabilities of X language.

Benchmarking for biomedical natural language processing tasks with a domain specific ALBERT

BackgroundThe abundance of biomedical text data coupled with advances in natural language processing (NLP) is resulting in novel biomedical NLP (BioNLP) applications. These NLP applications, or tasks, are reliant on the availability of domain-specific language models (LMs) that are trained on a massive amount of data. Most of the existing domain-specific LMs adopted bidirectional encoder representations from transformers (BERT) architecture which has limitations, and their generalizability is unproven as there is an absence of baseline results among common BioNLP tasks.ResultsWe present 8 variants of BioALBERT, a domain-specific adaptation of a lite bidirectional encoder representations from transformers (ALBERT), trained on biomedical (PubMed and PubMed Central) and clinical (MIMIC-III) corpora and fine-tuned for 6 different tasks across 20 benchmark datasets. Experiments show that a large variant of BioALBERT trained on PubMed outperforms the state-of-the-art on named-entity recognition (+ 11.09% BLURB score improvement), relation extraction (+ 0.80% BLURB score), sentence similarity (+ 1.05% BLURB score), document classification (+ 0.62% F1-score), and question answering (+ 2.83% BLURB score). It represents a new state-of-the-art in 5 out of 6 benchmark BioNLP tasks.ConclusionsThe large variant of BioALBERT trained on PubMed achieved a higher BLURB score than previous state-of-the-art models on 5 of the 6 benchmark BioNLP tasks. Depending on the task, 5 different variants of BioALBERT outperformed previous state-of-the-art models on 17 of the 20 benchmark datasets, showing that our model is robust and generalizable in the common BioNLP tasks. We have made BioALBERT freely available which will help the BioNLP community avoid computational cost of training and establish a new set of baselines for future efforts across a broad range of BioNLP tasks.

Effects of data and entity ablation on multitask learning models for biomedical entity recognition

MotivationTraining domain-specific named entity recognition (NER) models requires high quality hand curated gold standard datasets which are time-consuming and expensive to create. Furthermore, the storage and memory required to deploy NLP models can be prohibitive when the number of tasks is large. In this work, we explore utilizing multi-task learning to reduce the amount of training data needed to train new domain-specific models. We evaluate our system across 22 distinct biomedical NER datasets and evaluate the extent to which transfer learning helps task performance using two forms of ablation. ResultsWe found that multitasking models generally do not improve performance, but in many cases perform on par compared to single-task models. However, we show that in some cases, new unseen tasks can be trained as a single model using less data by starting with weights from a multitask model and improve performance. AvailabilityThe software underlying this article are available in: https://github.com/NLPatVCU/multitasking_bert-1.

Electronic representations of conceptual models for simulation – A scoping review

ABSTRACT Simulation facilitates the understanding and improvement of complex systems. Conceptual modelling is a key step in simulation studies. It has gained recognition because it may both increase engagement with stakeholders and decrease the time to implement a simulation. This scoping review’s objective is to highlight approaches and platforms for electronically representing models from 1999 to 2020. The motivation is that electronic representations facilitate the sharing of conceptual models. The contribution from the review to the research of conceptual modelling and simulation is to show that conceptual models are electronically represented by broadly speaking either General-Purpose or Domain-Specific Modelling Languages. There is a slight trend towards the latter in order to better deal with application specificities and improve unambiguity in model representations, though. Thus, we identify modelling approaches, platforms, and features for electronically representing conceptual models with the potential to fill the gap between conceptual models and their corresponding simulation implementations.

Model-Based Systems Engineering Tool-Chain for Automated Parameter Value Selection

Cyber-physical systems (CPSs) integrate heterogeneous systems and process sensor data using digital services. As the complexity of CPS increases, it becomes more challenging to efficiently formalize the integrated multidomain views with flexible automated verification across the entire lifecycle. This article illustrates a model-based systems engineering tool-chain to support CPS development with an emphasis on automated parameter value selection for co-simulation. First, a domain-specific modeling approach is introduced to support the formalizations of CPS artifacts, development processes, and simulation configurations. The domain-specific models are used as the basis to generate a Web-based process management system for automated parameter value selections, which coordinates Open Services for Lifecycle Collaboration services of development information and technical resources (models, data, and tools) in order to support automated co-simulation. The services are deployed by a service orchestrator based on a decision-making algorithm for parameter value selection. Finally, developers make use of the WPMS to implement simulations and to select system parameter values for co-simulation automatically. The approach is illustrated by a case study on auto-braking system development and we evaluate the efficiency of this tool-chain by both qualitative and quantitative methods. The results show that parameter values are selected more efficiently and effectively when implementing co-simulations using our tool-chain.

Comparative Assessment of Data Mining Techniques for Flash Flood Prediction

Abstract Data mining techniques have recently drawn considerable attention from the research community for their ability to predict flash flood phenomena. These techniques can bring large-scale flood data into real practice and have become the necessary tools for impact assessment, societal resilience, and disaster control. Although numerous studies have been conducted on data mining techniques and flash flood predictions, domain-specific flash flood prediction models based on existing data mining techniques are still lacking. Notably, this study has focused on the performance of four data mining techniques, namely, logistic regression (LR), artificial neural networks (ANN), k-nearest neighbour (kNN), and support vector machine (SVM) in a comparative assessment as prediction models. The area under the curve (AUC) was utilised to validate these models. The value of AUC was higher than 0.9 for all models. Accordingly, the outcomes outlined in this study can contribute to Halim et al. the current literature by boosting the performance of data mining techniques for predicting flash floods through a comparison of the most recent data mining techniques. Keywords: Artificial neural networks (ANN), Flash flood, k-nearest neighbor (kNN), Logistic regression (LR), Support vector machine (SVM)

Domain-specific trained model to auto grade AV Nicking severity level

ArterioVenous Nicking (AVN) is a prominent vascular biomarker in predicting cardiovascular risk factors such as atherosclerosis, hypertension, coronary artery diseases, and heart stroke. The morphological change in the retinal junction calibre presents various clues to diagnose the aforesaid risk factors. In this paper, we introduced a novel hierarchical training strategy to improve the grading accuracy in assessing the abnormality. In the first level, a domain-specific weights are generated from the pretrained weights. At the second level, domain-specific weights are transferred to the target architecture called Scale Attention Region-Based Deep Architecture, which is again trained on the AV Nicked dataset. In the target architecture, the scale attention feature extractor generates adaptive weights based on the semantic information at different levels of resolution. The advantage of this module is that it extracts minute features of severity so that the grading can be done precisely. The hierarchically trained classifier’s performance for AVN grading is compared with manual grading and also with a generalised pretrained(PT) model. The ResNet101 + DST metrics are evaluated and compared with a ground truth dataset.

CancerBERT: a cancer domain-specific language model for extracting breast cancer phenotypes from electronic health records.

ObjectiveAccurate extraction of breast cancer patients’ phenotypes is important for clinical decision support and clinical research. This study developed and evaluated cancer domain pretrained CancerBERT models for extracting breast cancer phenotypes from clinical texts. We also investigated the effect of customized cancer-related vocabulary on the performance of CancerBERT models.Materials and MethodsA cancer-related corpus of breast cancer patients was extracted from the electronic health records of a local hospital. We annotated named entities in 200 pathology reports and 50 clinical notes for 8 cancer phenotypes for fine-tuning and evaluation. We kept pretraining the BlueBERT model on the cancer corpus with expanded vocabularies (using both term frequency-based and manually reviewed methods) to obtain CancerBERT models. The CancerBERT models were evaluated and compared with other baseline models on the cancer phenotype extraction task.ResultsAll CancerBERT models outperformed all other models on the cancer phenotyping NER task. Both CancerBERT models with customized vocabularies outperformed the CancerBERT with the original BERT vocabulary. The CancerBERT model with manually reviewed customized vocabulary achieved the best performance with macro F1 scores equal to 0.876 (95% CI, 0.873–0.879) and 0.904 (95% CI, 0.902–0.906) for exact match and lenient match, respectively.ConclusionsThe CancerBERT models were developed to extract the cancer phenotypes in clinical notes and pathology reports. The results validated that using customized vocabulary may further improve the performances of domain specific BERT models in clinical NLP tasks. The CancerBERT models developed in the study would further help clinical decision support.

High level architecture-based framework for modeling interdependent critical infrastructure systems

Interdependencies among urban critical infrastructure systems (CISs) significantly impact the reliability and performance of CISs and the resilience of modern societies. Although several approaches exist for modeling interdependent CISs and studying their behavior, models developed in previous studies often fail to incorporate CIS domain knowledge, capture systemic heterogeneities among the CISs, and accurately model CISs interdependencies. Consequently, existing models have a limited ability to simulate interdependent CISs with sufficient detail and accuracy. To address these limitations, this study proposes a high-level architecture (HLA)-based framework for modeling interdependent CISs that can leverage and integrate well-tested practices, knowledge, data and simulation tools accumulated over years of wide usage in various CIS domains. The framework provides a methodology for co-simulating heterogeneous fine-grained CIS domain-specific models and modeling complex interactions between them and with their external environments, hence reproducing with high fidelity the complex coupled systems. A case study of two interdependent power and water systems was conducted, which demonstrated the efficacy of the proposed framework. Simulation results revealed that the HLA-based CISs model could capture the heterogeneous behaviors of the CISs and reveal a variety of failure-induced system vulnerabilities and feedback loops which may not be observable when using other existing modeling approaches.

A Model-Based Approach for Common Representation and Description of Robotics Software Architectures

Unlike conventional software, robotic software suffers from a lack of methods and processes that could systematize and facilitate development. Thus, the application of software engineering techniques is at the heart of current issues in robotics. The work presented in this paper aims to facilitate the development of robotic software and to facilitate communication between experts in the field through the use of software engineering techniques and methods. It proposes RsaML (Robotic Software Architecture Modeling Language), a Domain Specific Modeling Language (DSML) dedicated to robotics, which takes into account the different categories of robotic software architectures and makes it possible to describe the latter independently from the implementation platform. The conceptual model defining the terminology and the hierarchy of concepts used for the description and representation of robotic software architectures in RsaML are presented in this article. RsaML is defined through a meta-model which represents the abstract syntax of the language. The real-time properties of robotic software architectures are identified and included in the meta-model. The use of RsaML is illustrated through several experimental scenarios of the language: the definition of a robotic system and the description of its software architecture, the verification of the semantics of a robotic software architecture, and the modeling of a robotic system whose software architecture does not belong to the usual categories. The support tool used for implementations and experimentation is Eclipse Modeling Framework (EMF). The results of experimentation showed good working of the proposed solution and made it possible to validate the main concepts of the RsaML language.

Transformation of Health and Social Care Systems-An Interdisciplinary Approach Toward a Foundational Architecture.

ObjectiveFor realizing pervasive and ubiquitous health and social care services in a safe and high quality as well as efficient and effective way, health and social care systems have to meet new organizational, methodological, and technological paradigms. The resulting ecosystems are highly complex, highly distributed, and highly dynamic, following inter-organizational and even international approaches. Even though based on international, but domain-specific models and standards, achieving interoperability between such systems integrating multiple domains managed by multiple disciplines and their individually skilled actors is cumbersome.MethodsUsing the abstract presentation of any system by the universal type theory as well as universal logics and combining the resulting Barendregt Cube with parameters and the engineering approach of cognitive theories, systems theory, and good modeling best practices, this study argues for a generic reference architecture model moderating between the different perspectives and disciplines involved provide on that system. To represent architectural elements consistently, an aligned system of ontologies is used.ResultsThe system-oriented, architecture-centric, and ontology-based generic reference model allows for re-engineering the existing and emerging knowledge representations, models, and standards, also considering the real-world business processes and the related development process of supporting IT systems for the sake of comprehensive systems integration and interoperability. The solution enables the analysis, design, and implementation of dynamic, interoperable multi-domain systems without requesting continuous revision of existing specifications.

Measuring the Maturity of the Business Intelligence and Analytics Initiative of a Large Norwegian University

Maturity models of Business Intelligence and Analytics (BIA) have been previously used to assess BIA development progress in organizations in many sectors, such as healthcare and business. However, there is a lack of studies reporting up-to-date knowledge on applying maturity assessment in Higher Education Institutions (HEI). It remains unclear precisely to what extent and how HEI employ maturity assessment and the benefits of such exercises. This paper addresses this gap by reporting a case study at a large Norwegian university. A domain-specific maturity model is used as a lens to observe and reflect on the BIA implementation at the Norwegian University of Science and Technology. This paper reports the assessment results and discuss the implications of the maturity assessment. The findings and discussions in the case can cater to a broader audience of BIA practitioners and researchers, contributing to understanding the value and adoption dynamics of BIA in Higher Education.

Generating Fast Specialized Simulators for Stochastic Reaction Networks via Partial Evaluation

Domain-specific modeling languages allow a clear separation between simulation model and simulator and, thus, facilitate the development of simulation models and add to the credibility of simulation results. Partial evaluation provides an effective means for efficiently executing models defined in such languages. However, it also implies some challenges of its own. We illustrate this and solutions based on a simple domain-specific language for biochemical reaction networks as well as on the network representation of the established BioNetGen language. We implement different approaches adopting the same simulation algorithms: one generic simulator that parses models at runtime and one generator that produces a simulator specialized to a given model based on partial evaluation and code generation. For the purpose of better understanding, we additionally generate intermediate variants, where only some parts are partially evaluated. Akin to profile-guided optimization, we use dynamic execution of the model to further optimize the simulators. The performance of the approaches is carefully benchmarked using representative models of small to large biochemical reaction networks. The generic simulator achieves a performance similar to state-of-the-art simulators in the domain, whereas the specialized simulator outperforms established simulation tools with a speedup of more than an order of magnitude. Technical limitations in regard to the size of the generated code are discussed and overcome using a combination of link-time optimization and code separation. A detailed performance study is undertaken, investigating how and where partial evaluation has the largest effect.

Applying BERT Embeddings to Predict Legal Textual Entailment

Textual entailment classification is one of the hardest tasks for the Natural Language Processing community. In particular, working on entailment with legal statutes comes with an increased difficulty, for example in terms of different abstraction levels, terminology and required domain knowledge to solve this task. In course of the COLIEE competition, we develop three approaches to classify entailment. The first approach combines Sentence-BERT embeddings with a graph neural network, while the second approach uses the domain-specific model LEGAL-BERT, further trained on the competition’s retrieval task and fine-tuned for entailment classification. The third approach involves embedding syntactic parse trees with the KERMIT encoder and using them with a BERT model. In this work, we discuss the potential of the latter technique and why of all our submissions, the LEGAL-BERT runs may have outperformed the graph-based approach.

A Computer Science Perspective on Digital Transformation in Production

The Industrial Internet-of-Things (IIoT) promises significant improvements for the manufacturing industry by facilitating the integration of manufacturing systems by Digital Twins. However, ecological and economic demands also require a cross-domain linkage of multiple scientific perspectives from material sciences, engineering, operations, business, and ergonomics, as optimization opportunities can be derived from any of these perspectives. To extend the IIoT to a trueInternet of Production, two concepts are required: first, a complex, interrelated network of Digital Shadows which combine domain-specific models with data-driven AI methods; and second, the integration of a large number of research labs, engineering, and production sites as a World Wide Lab which offers controlled exchange of selected, innovation-relevant data even across company boundaries. In this article, we define the underlying Computer Science challenges implied by these novel concepts in four layers:Smart human interfacesprovide access to information that has been generated bymodel-integrated AI. Given the large variety of manufacturing data, newdata modelingtechniques should enable efficient management of Digital Shadows, which is supported by aninterconnected infrastructure. Based on a detailed analysis of these challenges, we derive a systematized research roadmap to make the vision of the Internet of Production a reality.

Visualizing Standardized Model-Based Design and Inspection Data in Augmented Reality

Abstract Augmented reality (AR) has already helped manufacturers realize value across a variety of domains, including assistance in maintenance, process monitoring, and product assembly. However, coordinating traditional engineering data representations into AR systems without loss of context and information remains a challenge. A major barrier is the lack of interoperability between manufacturing-specific data models and AR-capable data representations. In response, we present a pipeline for porting standards-based design and inspection data into an AR scene. As a result, product manufacturing information with three-dimensional (3D) model data and corresponding inspection results are successfully overlaid onto a physical part. We demonstrate our pipeline by interacting with annotated parts while continuously tracking their pose and orientation. We then validate the pipeline by testing against six fully toleranced design models, accompanied by idealized inspection results. Our work (1) provides insight on how to address fundamental issues related to interoperability between domain-specific models and AR systems and (2) establishes an open software pipeline from which others can implement and further develop.