Data Integration Research Articles

Big Data Analysis for Product Demand Prediction in Indonesian E-commerce

The Indonesian e-commerce sector has witnessed exponential growth in recent years, necessitating effective strategies for predicting product demand to meet consumer expectations and stay competitive. Leveraging big data analytics offers a promising solution to this challenge, allowing businesses to harness vast volumes of data to forecast demand accurately. This research explores the utilization of big data analysis for product demand prediction in Indonesian e-commerce through a qualitative analysis approach. Interviews with e-commerce executives, data scientists, and industry experts revealed key themes, including data quality and integration, analytical techniques and tools, and organizational culture and capability. Results highlight the challenges, opportunities, and best practices associated with leveraging big data analytics for demand prediction in Indonesian e-commerce, providing valuable insights for businesses striving to optimize their predictive capabilities and enhance decision-making processes in this dynamic and rapidly evolving market.

Brain high-throughput multi-omics data reveal molecular heterogeneity in Alzheimer's disease.

Unbiased data-driven omic approaches are revealing the molecular heterogeneity of Alzheimer disease. Here, we used machine learning approaches to integrate high-throughput transcriptomic, proteomic, metabolomic, and lipidomic profiles with clinical and neuropathological data from multiple human AD cohorts. We discovered 4 unique multimodal molecular profiles, one of them showing signs of poor cognitive function, a faster pace of disease progression, shorter survival with the disease, severe neurodegeneration and astrogliosis, and reduced levels of metabolomic profiles. We found this molecular profile to be present in multiple affected cortical regions associated with higher Braak tau scores and significant dysregulation of synapse-related genes, endocytosis, phagosome, and mTOR signaling pathways altered in AD early and late stages. AD cross-omics data integration with transcriptomic data from an SNCA mouse model revealed an overlapping signature. Furthermore, we leveraged single-nuclei RNA-seq data to identify distinct cell-types that most likely mediate molecular profiles. Lastly, we identified that the multimodal clusters uncovered cerebrospinal fluid biomarkers poised to monitor AD progression and possibly cognition. Our cross-omics analyses provide novel critical molecular insights into AD.

Genome sequence-based species classification of Enterobacter cloacae complex: a study among clinical isolates.

Standardizing taxonomy of the Enterobacter cloacae complex (ECC) is necessary for data integration across diverse studies. The study utilized whole-genome data to accurately identify 256 clinical ECC isolated from bloodstream infections using average nucleotide identity (ANI), digitalized DNA-DNA hybridization (dDDH), and phylogenetic analysis. Through comprehensive assessments including phylogenetic analysis, ANI and dDDH comparisons, virulence gene, and capsule typing of the 256 clinical isolates, it was concluded that the classification method based on subspecies exhibited better correlation and consistency with the molecular characteristics of clinical ECC isolates. In summary, this research contributes to the precise identification of clinical ECC at the species level and expands our understanding of ECC.

Mastering Data Product Development: Strategies, Architectures, and Best Practices

Abstract: The importance of data products for organizations cannot be overstated, as they enable valuable insights and informed decision-making. This article offers a thorough guide to constructing data products, with a specific emphasis on data warehouse architecture and data product management. Key concepts in data product management are explored, such as treating data as a product, understanding the data product lifecycle, and defining roles and responsibilities. The article explores the key elements of a data warehouse, including data integration and ETL processes, data modeling, and storage and retrieval techniques. This approach outlines a systematic process for developing data products, covering everything from identifying opportunities and defining requirements to design, implementation, testing, deployment, and maintenance. Emphasizing the significance of data governance, quality assurance, security, privacy, and regulatory compliance. Additionally, we delve into performance metrics, monitoring, and strategies for continuous improvement of data products. The article showcases case studies, such as Our World in Data [1], to demonstrate real-world applications and best practices. Finally, we will explore future trends, challenges, and opportunities in data p

Knowledge-guided learning methods for integrative analysis of multi-omics data

Integrative analysis of multi-omics data has the potential to yield valuable and comprehensive insights into the molecular mechanisms underlying complex diseases such as cancer and Alzheimer's disease. However, a number of analytical challenges complicate multi-omics data integration. For instance, -omics data are usually high-dimensional, and sample sizes in multi-omics studies tend to be modest. Furthermore, when genes in an important pathway have relatively weak signal, it can be difficult to detect them individually. There is a growing body of literature on knowledge-guided learning methods that can address these challenges by incorporating biological knowledge such as functional genomics and functional proteomics into multi-omics data analysis. These methods have been shown to outperform their counterparts that do not utilize biological knowledge in tasks including prediction, feature selection, clustering, and dimension reduction. In this review, we survey recently developed methods and applications of knowledge-guided multi-omics data integration methods and discuss future research directions.

Big Data and Machine Learning in Defence

In the era of data-driven warfare, the integration of big data and machine learning (ML) techniques has become paramount for enhancing defence capabilities. This research report delves into the applications of big data and ML in the defence sector, exploring their potential to revolutionize intelligence gathering, strategic decision-making, and operational efficiency. By leveraging vast amounts of data and advanced algorithms, these technologies offer unprecedented opportunities for threat detection, predictive analysis, and optimized resource allocation. However, their adoption also raises critical concerns regarding data privacy, ethical implications, and the potential for misuse. This report aims to provide a comprehensive understanding of the current state of big data and ML in defence, while examining the challenges and ethical considerations that must be addressed to ensure responsible and effective implementation.

Integration of multi-omics summary data reveals the role of N6-methyladenosine in neuropsychiatric disorders.

N6-methyladenosine (m6A) methylation regulates gene expression/protein by influencing numerous aspects of mRNA metabolism and contributes to neuropsychiatric diseases. Here, we integrated multi-omics data and genome-wide association study summary data of schizophrenia (SCZ), bipolar disorder (BP), attention deficit hyperactivity disorder (ADHD), autism spectrum disorder (ASD), major depressive disorder (MDD), Alzheimer's disease (AD), and Parkinson's disease (PD) to reveal the role of m6A in neuropsychiatric disorders by using transcriptome-wide association study (TWAS) tool and Summary-data-based Mendelian randomization (SMR). Our investigation identified 86 m6A sites associated with seven neuropsychiatric diseases and then revealed 7881 associations between m6A sites and gene expressions. Based on these results, we discovered 916 significant m6A-gene associations involving 82 disease-related m6A sites and 606 genes. Further integrating the 58 disease-related genes from TWAS and SMR analysis, we obtained 61, 8, 7, 3, and 2 associations linking m6A-disease, m6A-gene, and gene-disease for SCZ, BP, AD, MDD, and PD separately. Functional analysis showed the m6A mapped genes were enriched in "response to stimulus" pathway. In addition, we also analyzed the effect of gene expression on m6A and the post-transcription effect of m6A on protein. Our study provided new insights into the genetic component of m6A in neuropsychiatric disorders and unveiled potential pathogenic mechanisms where m6A exerts influences on disease through gene expression/protein regulation.

Current advances on single or multi-omics analysis of esophageal cancer

Esophageal cancer is associated with high mortality rates and is one of the cancers with the worst prognosis. Its incidence has significant regional specificity, particularly in China where it is much higher than in other countries. Moreover, effective diagnostic markers, therapeutic targets, and molecular subtyping biomarkers are currently lacking for esophageal cancer. Nevertheless, large-scale omics studies have identified dozens of robust genetic risk loci and prognosis-related loci, drawn genomic, epigenomic, and transcriptomic maps of esophageal cancer at multiple molecular levels, and described significant differences between esophageal squamous cell carcinoma and adenocarcinoma. These studies are of great significance for exploring the occurrence and development mechanism of esophageal cancer, guiding clinical treatment, and improving patient prognosis. This review, from the perspective of multi-omics, discusses the analytical strategies employed in these studies and summarizes their core findings. It emphasizes that the integration and analysis of multi-omics data is a key focus and development trend in the precise medical research of esophageal cancer, and has broad research and application prospects.Keywords: Esophageal cancer; GWAS; Precision medicine; Biomarker

Foundations for the scientific use of extensive health care data in Germany-results of the Data Sharing working group of the Medical Informatics Initiative

Healthcare data are an important resource in applied medical research. They are available multicentrically. However, it remains achallenge to enable standardized data exchange processes between federal states and their individual laws and regulations. The Medical Informatics Initiative (MII) was founded in 2016 to implement processes that enable cross-clinic access to healthcare data in Germany. Several working groups (WGs) have been set up to coordinate standardized data structures (WG Interoperability), patient information and declarations of consent (WG Consent), and regulations on data exchange (WG Data Sharing). Here we present the most important results of the Data Sharing working group, which include agreed terms of use, legal regulations, and data access processes. They are already being implemented by the established Data Integration Centers (DIZ) and Use and Access Committees (UACs). We describe the services that are necessary to provide researchers with standardized data access. They are implemented with the Research Data Portal for Health, among others. Since the pilot phase, the processes of 385 active researchers have been used on this basis, which, as of April 2024, has resulted in 19registered projects and 31submitted research applications.

Identification of bone mineral density associated genes with shared genetic architectures across multiple tissues: Functional insights for EPDR1, PKDCC, and SPTBN1.

Recent studies suggest a shared genetic architecture between muscle and bone, yet the underlying molecular mechanisms remain elusive. This study aims to identify the functionally annotated genes with shared genetic architecture between muscle and bone using the most up-to-date genome-wide association study (GWAS) summary statistics from bone mineral density (BMD) and fracture-related genetic variants. We employed an advanced statistical functional mapping method to investigate shared genetic architecture between muscle and bone, focusing on genes highly expressed in muscle tissue. Our analysis identified three genes, EPDR1, PKDCC, and SPTBN1, which are highly expressed in muscle tissue and previously unlinked to bone metabolism. About 90% and 85% of filtered Single-Nucleotide Polymorphisms were in the intronic and intergenic regions for the threshold at P≤5×10-8 and P≤5×10-100, respectively. EPDR1 was highly expressed in multiple tissues, including muscles, adrenal glands, blood vessels, and the thyroid. SPTBN1 was highly expressed in all 30 tissue types except blood, while PKDCC was highly expressed in all 30 tissue types except the brain, pancreas, and skin. Our study provides a framework for using GWAS findings to highlight functional evidence of crosstalk between multiple tissues based on shared genetic architecture between muscle and bone. Further research should focus on functional validation, multi-omics data integration, gene-environment interactions, and clinical relevance in musculoskeletal disorders.

Integrating Machine Learning in Metabolomics: A Path to Enhanced Diagnostics and Data Interpretation.

Metabolomics, leveraging techniques like NMR and MS, is crucial for understanding biochemical processes in pathophysiological states. This field, however, faces challenges in metabolite sensitivity, data complexity, and omics data integration. Recent machine learning advancements have enhanced data analysis and disease classification in metabolomics. This study explores machine learning integration with metabolomics to improve metabolite identification, data efficiency, and diagnostic methods. Using deep learning and traditional machine learning, it presents advancements in metabolic data analysis, including novel algorithms for accurate peak identification, robust disease classification from metabolic profiles, and improved metabolite annotation. It also highlights multiomics integration, demonstrating machine learning's potential in elucidating biological phenomena and advancing disease diagnostics. This work contributes significantly to metabolomics by merging it with machine learning, offering innovative solutions to analytical challenges and setting new standards for omics data analysis.

Integration of Sentinel-1 and Sentinel-2 Data for Ground Truth Sample Migration for Multi-Temporal Land Cover Mapping

Reliable and up-to-date training reference samples are imperative for land cover (LC) classification. However, such training datasets are not always available in practice. The sample migration method has shown remarkable success in addressing this challenge in recent years. This work investigated the application of Sentinel-1 (S1) and Sentinel-2 (S2) data in training sample migration. In addition, the impact of various spectral bands and polarizations on the accuracy of the migrated training samples was also assessed. Subsequently, combined S1 and S2 images were classified using the Support Vector Machines (SVM) and Random Forest (RF) classifiers to produce annual LC maps from 2017 to 2021. The results showed a higher accuracy (98.25%) in training sample migrations using both images in comparison to using S1 (87.68%) and S2 (96.82%) data independently. Among the LC classes, the highest accuracy in migrated training samples was found for water, built-up, bare land, grassland, cropland, and wetland. Inquiries on the efficiency of different spectral bands and polarization used in training sample migration showed that bands 4 and 8 and VV polarization in the water class were more important, while for the wetland class, bands 5, 6, 7, 8, and 8A together with VV polarization showed superior performance. The results showed that the RF classifier provided better performance than the SVM (higher overall, producer, and user accuracy). Overall, our findings suggested that shared use of S1 and S2 data can be used as a suitable means for producing up-to-date and high-quality training samples.

PRKDC-Mediated NHEJ May Play a Crucial Role in Aneuploidy of Chromosome 8-Driven Progression of Ovarian Cancer.

High malignancy is a prominent characteristic of epithelial ovarian cancer (EOC), emphasizing the necessity for further elucidation of the potential mechanisms underlying cancer progression. Aneuploidy and copy number variation (CNV) partially contribute to the heightened malignancy observed in EOC; however, the precise features of aneuploidy and their underlying molecular patterns, as well as the relationship between CNV and aneuploidy in EOC, remain unclear. In this study, we employed single-cell sequencing data along with The Cancer Genome Atlas (TCGA) to investigate aneuploidy and CNV in EOC. The technique of fluorescence in situ hybridization (FISH) was employed using specific probes. The copy number variation within the genomic region of chromosome 8 (42754568-47889815) was assessed and utilized as a representative measure for the ploidy status of individual cells in chromosome 8. Differential expression analysis was performed between different subgroups based on chromosome 8 ploidy. Gene ontology (GO), Kyoto Encyclopedia of Genes and Genomes (KEGG), protein-protein interaction (PPI), and hub-gene analyses were subsequently utilized to identify crucial genes involved. By classifying enriched tumor cells into distinct subtypes based on chromosome 8 ploidy combined with TCGA data integration, we identified key genes driving chromosome 8 aneuploidy in EOC, revealing that PRKDC gene involvement through the mediated non-homologous end-joining pathway may play a pivotal role in disease progression. Further validation through analysis of the GEO and TCGA database and survival assessment, considering both mRNA expression levels and CNV status of PRKDC, has confirmed its involvement in the progression of EOC. Further functional analysis revealed an upregulation of PRKDC in both ovarian EOC cells and tissues, with its expression showing a significant correlation with the extent of copy number variation (CNV) on chromosome 8. Taken together, CNV amplification and aneuploidy of chromosome 8 are important characteristics of EOC. PRKDC and the mediated NHEJ pathway may play a crucial role in driving aneuploidy on chromosome 8 during the progression of EOC.

Discovering Links between Geospatial Data Sources in the Web of Data: The Open Geospatial Engine Approach

The Web of Data has been fueled significantly by geospatial data over the last few years. In the current link discovery frameworks, there is still a lack of robust support for finding geospatial-aware links between geospatial data sources in the Web of Data. They are also limited in efficient association capabilities for large-scale datasets. This paper extends the data integration capability based on the spatial metrics in the open geospatial engine OGE. These metrics include topological relationships and spatial matching between geospatial entities within multiple geospatial data sources. Thus, the tool can be employed by data publishers to set geospatial-aware links to facilitate geospatial data and knowledge discovery in the Web of Data. Several geospatial data sources are used to demonstrate the usability and effectiveness of the approach and tool implementation.

A survey on semantic data management as intersection of ontology-based data access, semantic modeling and data lakes

In recent years, data lakes emerged as a way to manage large amounts of heterogeneous data for modern data analytics. One way to prevent data lakes from turning into inoperable data swamps is semantic data management. Such approaches propose the linkage of metadata to knowledge graphs based on the Linked Data principles to provide more meaning and semantics to the data in the lake. Such a semantic layer may be utilized not only for data management but also to tackle the problem of data integration from heterogeneous sources, in order to make data access more expressive and interoperable. In this survey, we review recent approaches with a specific focus on the application within data lake systems and scalability to Big Data. We classify the approaches into (i) basic semantic data management, (ii) semantic modeling approaches for enriching metadata in data lakes, and (iii) methods for ontology-based data access. In each category, we cover the main techniques and their background, and compare latest research. Finally, we point out challenges for future work in this research area, which needs a closer integration of Big Data and Semantic Web technologies.

A technical infrastructure for primary education data that contributes to data standardization

AbstractThere is a significant amount of data available about students and their learning activities in many educational systems today. However, these datasets are frequently spread across several different digital services, making it challenging to use them strategically. In addition, there are no established standards for collecting, processing, analyzing, and presenting such data. As a result, school leaders, teachers, and students do not capitalize on the possibility of making decisions based on data. This is a serious barrier to the improvement of work in schools, teacher and student progress, and the development of effective Educational Technology (EdTech) products and services. Data standards can be used as a protocol on how different IT systems communicate with each other. When working with data from different public and private institutions simultaneously (e.g., different municipalities and EdTech companies), having a trustworthy data pipeline for retrieving the data and storing it in a secure warehouse is critical. In this study, we propose a technical solution containing a data pipeline by employing a secure warehouse—the Swedish University Computer Network (SUNET), which is an interface for information exchange between operational processes in schools. We conducted a user study in collaboration with four municipalities and four EdTech companies based in Sweden. Our proposal involves introducing a data standard to facilitate the integration of educational data from diverse resources in our SUNET drive. To accomplish this, we created customized scripts for each stakeholder, tailored to their specific data formats, with the aim of merging the students’ data. The results of the first four steps show that our solution works. Once the results of the next three steps are in, we will contemplate scaling up our technical solution nationwide. With the implementation of the suggested data standard and the utilization of the proposed technical solution, diverse stakeholders can benefit from improved management, transportation, analysis, and visualization of educational data.

Next-Gen Weather Resilience: A Holistic Approach to Disaster Mitigation Through Interactive Maps, Flood Detection, and Drought Forecasting

Next-Gen Weather Resilience presents a transformative framework for disaster mitigation by utilizing interactive maps, flood detection technologies, and advanced drought forecasting methods. This holistic approach combines innovative strategies to enhance preparedness and response measures in the face of increasingly unpredictable weather patterns and climate change impacts. Through the integration of real-time data and analytics, interactive maps offer situational awareness and decision support for stakeholders at all levels. Flood detection systems enable early warning mechanisms, improving response times and reducing damages caused by flooding events. Furthermore, cutting-edge drought forecasting techniques provide valuable insights into water availability and resource management, aiding in long-term planning and adaptation strategies. By leveraging these tools in a coordinated manner, Next-Gen Weather Resilience empowers communities to build greater resilience against natural disasters while fostering sustainable development initiatives. Key Words: Weather Resilience, Disaster Mitigation, Interactive Maps, Flood Detection, Drought Forecasting.

Geological mapping and mineral exploration in Francistown region, north east Botswana from integration of ASTER TIR and aeromagnetic data

To map the region's geology and mineral exploration, this study looked at how remote sensing satellite imaging from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Thermal Infrared (TIR) sensor and aircraft geophysics may be used. The Francistown region forms part of the Zimbabwe craton, northeast of Botswana. It comprises felsic rock types, including granite and dry river sand, mafic-ultramafic rocks, and gneiss country rock. Based on their spectral emissivity responses, mafic-ultramafic units and quartz-rich units were discovered. These units are important targets for the discovery of mineral resources. The research region can be described as convoluted, fractured, intruded, metamorphosed, and ultimately worn from a geological and geophysical standpoint. Several remote sensing and aeromagnetic approaches were employed. The false color composite (FCC) and mineral indices using ASTER TIR data provided helpful information for delineating the area's geology and concentration of major-rock-forming minerals. The aeromagnetic data extracted lineaments well, and the Center for Exploration Target (CET) grid analysis system was very helpful. The CET Porphyry Analysis technique was applied in mapping near-circular porphyry features, which are good indicators of possible sites with high mineralization potential. The analysis of the lineament map shows that mineral deposits are mostly along the NE-SW direction. The results from the ASTER TIR and aeromagnetic enhancements indicate the occurrence of mafic-ultramafic rocks and felsic-rich units, which have been documented to host mineralization in the region. Porphyry structures are also mapped successfully along the general strike of the geological structure in the study area. The combination of these methods proved their success, and they can be used further in other regions of high mineral potential and lithological mapping for their cost efficiency and reliability.

Semantic Data Integration and Querying: A Survey and Challenges

Digital revolution produces massive, heterogeneous and isolated data. These latter remain underutilized, unsuitable for integrated querying and knowledge discovering. Hence the importance of this survey on data integration which identifies challenging issues and trends. First, an overview of the different generations and basics of data integration is given. Then, semantic data integration is focused, since it semantically links data allowing wider insights and decision-making. More than thirty works are reviewed. The goal is to help analysts to identify relevant criteria to compare then choose among semantic data integration approaches, focusing on the category (materialized, virtual or hybrid) and querying techniques.

The Sustainable Development Goals for Education and Research in the Ranking of Green Universities of Mahasarakham University

This research aims to review the educational and research operations of Mahasarakham University and propose development directions for the university to meet the sustainable development criteria for ranking as a green university. This involves gathering data and analyzing the results of operations over the past 3 years, then using this analysis to synthesize the lessons learned and develop guidelines for sustainable development in the coming years. The study utilizes a robust methodology involving policy analysis, strategic planning, performance evaluation, and data integration. The study found that the university’s policies and strategies implemented following the annual performance evaluation criteria align with the green university assessment criteria for education and research. This has resulted in minor improvements in the curriculum for sustainability. However, community service projects for sustainability have increased by 89.10%, while funding for sustainable research and academic publications related to sustainability has decreased. In summary, the projects under the university’s development strategy adequately support and drive suitable development activities. Nevertheless, the university must continuously review its operations to adapt to changing contexts, budget constraints, evolving competition, and long-term development towards the implementation of the sustainable development goals (SDGs) to develop in the future.