Data Integration Model Research Articles

Multidimensional Model of Information Struggle with Impulse Perturbation in Terms of Levy Approximation

The focus of this research was on building a decision support system for a model that characterizes the conflict interaction of n-dimensional complex systems with non-trivial internal structures. The interpretation of the new model was focused on information warfare as the impact of rare events that quickly change certain perceptions of a large number of people. Consequently, the support for various ideas experiences stochastic jumps, a phenomenon observable through a non-classical Levy approximation scheme. The essence of our decision support system lies in its ability to navigate the complex dynamics of conflict interaction among multifaceted systems. Through the utilization of advanced modeling techniques, our aim is to illuminate the complicated interplay of factors influencing information warfare and its cascading effects on societal perceptions and behaviors. Key components of our decision support system encompass model development, simulation capabilities, data integration, and visualization tools. The significance of our work lies in its potential to inform policy formulation, conflict resolution strategies, and societal resilience in the face of information warfare. By providing decision-makers with actionable intelligence and foresight into emerging threats and opportunities, our decision support system serves as a valuable tool for navigating the complexities of modern conflict dynamics. In conclusion, developing a decision support system for modeling conflict interaction in complex systems represents an essential step toward enhancing our understanding of information warfare and its consequences. Through interdisciplinary collaboration and innovative modeling techniques, we aim to provide stakeholders with the insights and capabilities needed to navigate the developing landscape of conflict and ensure the stability and resilience of society.

CSDMS Data Components: data–model integration tools for Earth surface processes modeling

Abstract. Progress in better understanding and modeling Earth surface systems requires an ongoing integration of data and numerical models. Advances are currently hampered by technical barriers that inhibit finding, accessing, and executing modeling software with related datasets. We propose a design framework for Data Components, which are software packages that provide access to particular research datasets or types of data. Because they use a standard interface based on the Basic Model Interface (BMI), Data Components can function as plug-and-play components within modeling frameworks to facilitate seamless data–model integration. To illustrate the design and potential applications of Data Components and their advantages, we present several case studies in Earth surface processes analysis and modeling. The results demonstrate that the Data Component design provides a consistent and efficient way to access heterogeneous datasets from multiple sources and to seamlessly integrate them with various models. This design supports the creation of open data–model integration workflows that can be discovered, accessed, and reproduced through online data sharing platforms, which promotes data reuse and improves research transparency and reproducibility.

Reconstruction of missing streamflow series in human-regulated catchments using a data integration LSTM model

Study regionYellow River Basin in China, where streamflow dynamics were significantly impacted by human activities. Study focusWe introduced a deep learning-based method, i.e., Data Integration (DI) with Long Short-Term Memory (LSTM), which leverages Global Flood Awareness System (GloFAS) streamflow data. Multiscale (Catchment, River) attributes were incorporated into the DI LSTM to represent human disturbances on land surface. We employed this method to reconstruct daily streamflow series in 60 human-regulated catchments across the Yellow River Basin, and identified the sensitivity of the DI LSTM model to the multiscale attributes. New hydrological Insights for the RegionOur findings revealed that the DI LSTM model achieved favourable performance in streamflow estimation, with the highest Kling-Gupta efficiency (KGE) reaching up to 0.9, outperforming the Regular LSTM model, which was forced by meteorological variables. Multiscale attributes can enhance the DI model performance, particularly in large catchments with significant human activities. A two-step validation demonstrated the high accuracy of the reconstructed streamflow data across the Yellow River Basin, as the KGEs for streamflow estimation in 40 catchments are over 0.6. In summary, the DI LSTM model shows great potential for reconstructing streamflow in human-regulated catchments in arid regions. The reconstructed daily streamflow data contribute valuable insights for monitoring changing hydrological conditions, especially in regions lacking extensive streamflow monitoring networks.

Connecting cohorts of Finnish biobanks creates a research resource for the study of healthy ageing.

Connecting cohorts with biobanks is a Finnish biobank collaboration, creating an infrastructure for the study of healthy ageing. We aimed to develop a model for data integration and harmonisation between different biobanks with procedures for joint access. The heart of the collaboration is the integrated datasets formed by using data from three biobanks: (a) Arctic Biobank, hosting regional birth cohorts and cohorts of elderly; (b) hospital-affiliated Borealis Biobank of Northern Finland; and (c) THL Biobank, hosting population-based cohorts. The datasets were created by developing a data dictionary, harmonising cohort data and with a joint pseudonymisation process. The connecting cohorts with biobanks resource at its widest consists altogether of almost 1.4 million individuals from collaborating biobanks. Utilising data from 107,000 cohort participants, we created harmonised datasets that contain attributes describing metabolic risk and frailty for studies of healthy ageing. These data can be complemented with medical data available from Biobank Borealis and with samples taken at hospital settings for approximately 38,000 cohort participants. In addition, the harmonised connecting cohorts with biobanks datasets can be expanded with supplementary data and samples from the collaborating biobanks. The connecting cohorts with biobanks datasets provide a unique resource for research on ageing-related personalised healthcare and for real-world evidence studies. Following the FAIR principles on findability, accessibility, interoperability, and reusability, the reused and harmonised datasets are findable and made accessible for researchers. The same approach can be further utilised to develop additional datasets for other research topics.

Circular Economy Policies for Decarbonization of US Commercial Building Stocks: Data Integration and System Dynamics Coflow Modeling Approach

Circular Economy Policies for Decarbonization of US Commercial Building Stocks: Data Integration and System Dynamics Coflow Modeling Approach

Imputation of plasma lipid species to facilitate integration of lipidomic datasets

Recent advancements in plasma lipidomic profiling methodology have significantly increased specificity and accuracy of lipid measurements. This evolution, driven by improved chromatographic and mass spectrometric resolution of newer platforms, has made it challenging to align datasets created at different times, or on different platforms. Here we present a framework for harmonising such plasma lipidomic datasets with different levels of granularity in their lipid measurements. Our method utilises elastic-net prediction models, constructed from high-resolution lipidomics reference datasets, to predict unmeasured lipid species in lower-resolution studies. The approach involves (1) constructing composite lipid measures in the reference dataset that map to less resolved lipids in the target dataset, (2) addressing discrepancies between aligned lipid species, (3) generating prediction models, (4) assessing their transferability into the targe dataset, and (5) evaluating their prediction accuracy. To demonstrate our approach, we used the AusDiab population-based cohort (747 lipid species) as the reference to impute unmeasured lipid species into the LIPID study (342 lipid species). Furthermore, we compared measured and imputed lipids in terms of parameter estimation and predictive performance, and validated imputations in an independent study. Our method for harmonising plasma lipidomic datasets will facilitate model validation and data integration efforts.

Navigating the Evolution of Digital Twins Research through Keyword Co-Occurence Network Analysis.

Digital twin technology has become increasingly popular and has revolutionized data integration and system modeling across various industries, such as manufacturing, energy, and healthcare. This study aims to explore the evolving research landscape of digital twins using Keyword Co-occurrence Network (KCN) analysis. We analyze metadata from 9639 peer-reviewed articles published between 2000 and 2023. The results unfold in two parts. The first part examines trends and keyword interconnection over time, and the second part maps sensing technology keywords to six application areas. This study reveals that research on digital twins is rapidly diversifying, with focused themes such as predictive and decision-making functions. Additionally, there is an emphasis on real-time data and point cloud technologies. The advent of federated learning and edge computing also highlights a shift toward distributed computation, prioritizing data privacy. This study confirms that digital twins have evolved into complex systems that can conduct predictive operations through advanced sensing technologies. The discussion also identifies challenges in sensor selection and empirical knowledge integration.

SpatiAlign: an unsupervised contrastive learning model for data integration of spatially resolved transcriptomics.

Integrative analysis of spatially resolved transcriptomics datasets empowers a deeper understanding of complex biological systems. However, integrating multiple tissue sections presents challenges for batch effect removal, particularly when the sections are measured by various technologies or collected at different times. We propose spatiAlign, an unsupervised contrastive learning model that employs the expression of all measured genes and the spatial location of cells, to integrate multiple tissue sections. It enables the joint downstream analysis of multiple datasets not only in low-dimensional embeddings but also in the reconstructed full expression space. In benchmarking analysis, spatiAlign outperforms state-of-the-art methods in learning joint and discriminative representations for tissue sections, each potentially characterized by complex batch effects or distinct biological characteristics. Furthermore, we demonstrate the benefits of spatiAlign for the integrative analysis of time-series brain sections, including spatial clustering, differential expression analysis, and particularly trajectory inference that requires a corrected gene expression matrix.

An empirical study on the data integration model in the comprehensive development of ideological education and student management in colleges and universities based on the concept of three-pronged parenting

Abstract In this paper, a smart campus analysis model based on multi-source data fusion technology is constructed with the study of college students’ behavior as the starting point. The K-means algorithm is utilized to mine the campus data of real college students from multiple sources, and through the fusion of multi-source data, a real, intelligent, multi-level student portrait is portrayed in the three dimensions of consumption, academics, and socialization, so as to provide a rational decision-making basis for the comprehensive development of the ideological and political education and student management of the university with the “three-whole-education” approach. The results show that multi-source data fusion technology can significantly reduce the storage space required for data. University X has a low borrowing interest for Civics and Political Science books. Sports are loved by 70% of students, but half of them don’t exercise regularly. Outside spending is a habit among some students, and 20% of them may exhibit abnormal behavior. These individual behavioral characteristics will have a significant impact on the performance of Civics.

Data- and Physics-driven Deep Learning Based Reconstruction for Fast MRI: Fundamentals and Methodologies.

Magnetic Resonance Imaging (MRI) is a pivotal clinical diagnostic tool, yet its extended scanning times often compromise patient comfort and image quality, especially in volumetric, temporal and quantitative scans. This review elucidates recent advances in MRI acceleration via data and physics-driven models, leveraging techniques from algorithm unrolling models, enhancement-based methods, and plug-and-play models to the emerging full spectrum of generative model-based methods. We also explore the synergistic integration of data models with physics-based insights, encompassing the advancements in multi-coil hardware accelerations like parallel imaging and simultaneous multi-slice imaging, and the optimization of sampling patterns. We then focus on domain-specific challenges and opportunities, including image redundancy exploitation, image integrity, evaluation metrics, data heterogeneity, and model generalization. This work also discusses potential solutions and future research directions, with an emphasis on the role of data harmonization and federated learning for further improving the general applicability and performance of these methods in MRI reconstruction.

Perspectives on AI Architectures and Codesign for Earth System Predictability

Abstract Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, laboratory, modeling, and analysis activities, called model experimentation (ModEx). BER’s ModEx is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the AI Architectures and Codesign session and associated outcomes. The AI Architectures and Codesign session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including 1) DOE high-performance computing (HPC) systems, 2) cloud HPC systems, and 3) edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this codesign area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as 1) reimagining codesign, 2) data acquisition to distribution, 3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with Earth system modeling and simulation, and 4) AI-enabled sensor integration into Earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper. Significance Statement This study aims to provide perspectives on AI architectures and codesign approaches for Earth system predictability. Such visionary perspectives are essential because AI-enabled model-data integration has shown promise in improving predictions associated with climate change, perturbations, and extreme events. Our forward-looking ideas guide what is next in codesign to enhance Earth system models, observations, and theory using state-of-the-art and futuristic computational infrastructure.

Comparative Study of Tools for Modeling, Storage, and Integration of Data on the Semantic Web for the ELLAS Network Platform

The layered architecture of the World Wide Web (W3C) defines standards, technologies, languages, and methods needed to build applications that involve the semantic web. From this W3C semantic web architecture was developed the data modeling of the ELLAS (Equality in Leadership for Latin American STEM) network platform. The ELLAS aims to map the policies and context aspects that influence women in STEM (Science, Technology, Engineering and Mathematics) in the countries of Bolivia, Brazil and Peru, which are at different levels of development. The main objective of this work is a comparison of the main storage and semantic integration tools to develop a data model (modeling, storage and integration of data) related to the mapping of policies, initiatives and factors that influence the career development of women in STEM.

Multi-level Data Integrity Model with Dual Immutable Digital Key Based Forensic Analysis in IoT Network

Multi-level Data Integrity Model with Dual Immutable Digital Key Based Forensic Analysis in IoT Network

Healthcare data quality assessment for improving the quality of the Korea Biobank Network.

Numerous studies make extensive use of healthcare data, including human materials and clinical information, and acknowledge its significance. However, limitations in data collection methods can impact the quality of healthcare data obtained from multiple institutions. In order to secure high-quality data related to human materials, research focused on data quality is necessary. This study validated the quality of data collected in 2020 from 16 institutions constituting the Korea Biobank Network using 104 validation rules. The validation rules were developed based on the DQ4HEALTH model and were divided into four dimensions: completeness, validity, accuracy, and uniqueness. Korea Biobank Network collects and manages human materials and clinical information from multiple biobanks, and is in the process of developing a common data model for data integration. The results of the data quality verification revealed an error rate of 0.74%. Furthermore, an analysis of the data from each institution was performed to examine the relationship between the institution's characteristics and error count. The results from a chi-square test indicated that there was an independent correlation between each institution and its error count. To confirm this correlation between error counts and the characteristics of each institution, a correlation analysis was conducted. The results, shown in a graph, revealed the relationship between factors that had high correlation coefficients and the error count. The findings suggest that the data quality was impacted by biases in the evaluation system, including the institution's IT environment, infrastructure, and the number of collected samples. These results highlight the need to consider the scalability of research quality when evaluating clinical epidemiological information linked to human materials in future validation studies of data quality.

Calibrating subjective data biases and model predictive uncertainties in machine learning-based thermal perception predictions

Heating, Ventilation, and Air Conditioning (HVAC) systems in large-scale buildings often struggle to ensure satisfactory thermal comfort for diverse occupants while minimizing energy waste. Achieving this goal requires developing reliable prediction models that capture the changing and varied occupant thermal perceptions in different spaces. Despite their widespread use, many machine learning (ML) based prediction models suffer from subjective data biases and model predictive uncertainties, causing inaccurate estimation for occupant needs and leading to suboptimal building controls. The authors propose a data-model integration method that identifies and calibrates the inherent uncertainties of existing ML models in both data and model dimensions, ensuring reliable thermal perception predictions. This method introduces the Multidimensional Association Rule Mining (M-ARM) algorithm to identify biased human responses by exploring interrelationships among four perception metrics: thermal sensation, comfort, acceptability, and preference. Our method reveals significant performance enhancements in seven ML models, enhancing the F1-score by up to 5.53%. By leveraging reliability diagrams and Expected Calibration Error (ECE) scores, we also expose the models’ vulnerability to miscalibration and the need for calibrated predictions. We further evaluate six calibration techniques (e.g., Platt Scaling and Isotonic Calibration) on these models and uncover their potential to enhance prediction reliability performance, highlighting a reduction of up to 80.66% in ECE scores. The authors also investigated the impacts of dataset sizes, classifiers, and calibration methods on the proposed method. Our research offers insight into creating robust data-driven strategies for thermal perception predictions, ultimately contributing to optimized occupant comfort and energy efficiency in buildings.

Application of OMOP Common Data Model for Data Integration: The Bulgarian Diabetes Register.

This paper considers mapping of the Bulgarian Diabetes Register(BDR) onto OMOP Common Data Model (CDM). Research results are referenced and plans for analysis of drug shortages from federated data sources are outlined.

Comparing contemporary models to traditional indices to estimate abundance of desert bighorn sheep

AbstractAerial surveys for large ungulates produce count data that often underrepresent the number of animals. Errors in count data can lead to erroneous estimates of abundance if they are not addressed. Our objective was to address imperfect detection probability by developing a framework that produces realistic and defensible estimates of bighorn sheep (Ovis canadensis) abundance. We applied our framework to a population of desert bighorn sheep (O. c. nelsoni) in the Great Basin, Nevada, USA. We captured and marked 24 desert bighorn sheep with global positioning system (GPS)‐collars and then conducted helicopter surveys naïve to the locations of collared animals. We developed a Bayesian integrated data model to leverage information from telemetry data, helicopter survey counts, and habitat characteristics to estimate abundance while accounting for availability and perception probability (i.e., detection given availability). Distance to ridgeline, terrain ruggedness, tree cover, and slope influenced perception probability of sheep given they were viewable from the helicopter. There was also annual variation in perception probability (2018: median = 0.64, credible interval [CrI] = 0.37–0.87; 2019: median = 0.81, CrI = 0.49–0.97). The abundance estimates from the integrated data model decreased from 2018 (594; 95% CrI = 537–656) to 2019 (487; 95% CrI = 436–551). In addition, accounting for availability and imperfect perception resulted in greater estimates of abundance compared to traditional directed search methods, which were 340 for 2018 and 320 for 2019. Our modeling framework can be used to generate more defensible population estimates of bighorn sheep and other large mammals that have been surveyed in a similar manner.

Soil legacy nutrients contribute to the decreasing stoichiometric ratio of N and P loading from the Mississippi River Basin.

Human-induced nitrogen-phosphorus (N, P) imbalance in terrestrial ecosystems can lead to disproportionate N and P loading to aquatic ecosystems, subsequently shifting the elemental ratio in estuaries and coastal oceans and impacting both the structure and functioning of aquatic ecosystems. The N:P ratio of nutrient loading to the Gulf of Mexico from the Mississippi River Basin increased before the late 1980s driven by the enhanced usage of N fertilizer over P fertilizer, whereafter the N:P loading ratio started to decrease although the N:P ratio of fertilizer application did not exhibit a similar trend. Here, we hypothesize that different release rates of soil legacy nutrients might contribute to the decreasing N:P loading ratio. Our study used a data-model integration framework to evaluate N and P dynamics and the potential for long-term accumulation or release of internal soil nutrient legacy stores to alter the ratio of N and P transported down the rivers. We show that the longer residence time of P in terrestrial ecosystems results in a much slower release of P to coastal oceans than N. If contemporary nutrient sources were reduced or suspended, P loading sustained by soil legacy P would decrease much slower than that of N, causing a decrease in the N and P loading ratio. The longer residence time of P in terrestrial ecosystems and the increasingly important role of soil legacy nutrients as a loading source may explain the decreasing N:P loading ratio in the Mississippi River Basin. Our study underscores a promising prospect for N loading control and the urgency to integrate soil P legacy into sustainable nutrient management strategies for aquatic ecosystem health and water security.

Anatomy of a km-scale fault zone controlling the Oligo-Miocene bending of the Ligurian Alps (NW Italy): integration of field and 3D high-resolution digital outcrop model data

We report the first description of a ~ 15 km long NE-SW-striking transtensive fault network crosscutting the metamorphic units of the Ligurian Alps. The main fault zone, hereby named Horse Head Fault Zone, is up to 250 m thick, involves quarzite, metarhyolite, marble and alternation of dolostone and limestone and minor pelite. Relatively narrow (~ 1–3 m-thick) fault cores are characterized by gouge and cataclasites, surrounded by brecciated damage zones as thick as tens to hundreds of meters. Damage zones show widespread evidence for dilation in the form of dilation breccia, large calcite crystals and aggregates, and centimeter- to meter-thick veins. Moreover, the fault zone contains a multitude of polished slip surfaces with multiple sets of slickensides and slickenfibers. Oblique to strike-slip kinematics dominates over the large part of the fault mirrors and both overprint and are overprinted by down-dip slip surfaces. The fault network includes dominant NE-SW right-lateral faults with a minor normal component and NW–SE left-lateral steep faults with a negligible reverse component, consistent with a km-size dextral NE-SW-striking Riedel shear zone, in turn representing an antithetic R’ of the regional sinistral shear zone constituted by the Ligurian Alps after the nappe stacking. The Horse Head Fault Zone accommodated km-scale displacement before the Early Miocene, as it is sealed by the sedimentary deposits of the Finale Ligure Basin, thus predating the Corsica-Sardinia drifting. Results of this work constraint the bending of the Ligurian Alps as part of the Western Alpine arc as accomplished through two consecutive, late Oligocene and Early Miocene, stages driven by the combination of Adria rotation and the rollback of the Apennine subduction.

Energy efficient smart manufacturing of pharmaceutical solid oral dosage forms

Background: The global pharmaceuticals market is a trillion-dollar industry which grows more than 5% annually. However, in comparison to other manufacturing industries (e.g., oil refining, automotive), the pharmaceutical sector lags in manufacturing innovation and automation. In the production of pharmaceutical solid dosage forms, the use of energy utilization as a performance measure of production efficiency has neither been implemented extensively, nor been optimized to maximize efficiency. This study will focus on the development and implementation of a smart manufacturing platform to optimize energy productivity whilst maintaining tablet quality via the consideration of different manufacturing scenarios.
 Methods: This study will consider three main unit operations (wet granulation, drying and milling) which are relatively more energy intensive in pharmaceutical downstream processing, used to produce solid dosage forms, such as tablets. Four case-studies will be considered, which are 1: baseline batch, 2: baseline continuous, 3: optimized batch and 4: optimized continuous. Smart manufacturing is implemented to present optimized cases 3: and 4: Improvements in the energy and performance metrics are quantified and compared to the baseline cases. 
 Results and conclusions: The smart manufacturing platform used in this study, integrates advanced process model development, optimization, technoeconomic analysis and data integration. The utilization of this framework contributed to a ~70% and ~80% improvement in energy utilization in the optimized batch and continuous cases, respectively, when compared to the baseline batch case. In the optimized cases, tablet quality was maintained within targeted specifications and was comparable to the baseline batch case. This smart manufacturing framework can be generalized for drug product manufacturing and other particulate-based industries such as food, agriculture, and fine chemicals.