Multi-model Data Research Articles

ReIU: an efficient preliminary framework for Alzheimer patients based on multi-model data.

The rising incidence of Alzheimer's disease (AD) poses significant challenges to traditional diagnostic methods, which primarily rely on neuropsychological assessments and brain MRIs. The advent of deep learning in medical diagnosis opens new possibilities for early AD detection. In this study, we introduce retinal vessel segmentation methods based on U-Net ad iterative registration Learning (ReIU), which extract retinal vessel maps from OCT angiography (OCT-A) facilities. Our method achieved segmentation accuracies of 79.1% on the DRIVE dataset, 68.3% on the HRF dataset. Utilizing a multimodal dataset comprising both healthy and AD subjects, ReIU extracted vascular density from fundus images, facilitating primary AD screening with a classification accuracy of 79%. These results demonstrate ReIU's substantial accuracy and its potential as an economical, non-invasive screening tool for Alzheimer's disease. This study underscores the importance of integrating multi-modal data and deep learning techniques in advancing the early detection and management of Alzheimer's disease.

Revolutionizing healthcare: IoMT-enabled digital enhancement via multimodal ADL data fusion

The present research develops a framework to refine the classification of an individual’s activities and recognize wellness associated with their routine. The framework improves the accuracy of the classification of routine activities of a person, the activation time data of sensors fixed on objects linked with the routine activities of the person, and the aptness of an incessant activity pattern with the routine activities. The existing techniques need continuous monitoring and are non-adaptive to a person’s persistent habitual variations or individualities. The research involves applying Internet of Medical Things (IoMT)-based sensor information fusion to the novel multimodel data analytics to develop Activities of Daily Living (ADL) pattern, behavioral pattern generation and anomaly recognition. The novel multimodel data analytics approach is named AiCareLiving. AicareLiving is an IoMT and artificial intelligence (AI) enabled approach. The research work describes activity data using an individual’s activities within a specified area before evaluating the activity data to detect the existence of an anomaly by identifying the deviation of the activity data from the activity profile, which indicates the anticipated behavior and activity of the person. This wellness information would be shared to the caregivers, related healthcare professionals, care providers and municipalities through the secured healthcare information exchange protocol and IoMT. AiCareLiving framework aims to least false positive in terms of anomaly detection and forecasting; the high precision is close to the confidence level of 95%.

Interdisciplinary teaching strategy of multimodal data fusion in college aesthetic education curriculum using intelligence methods

In China, aesthetic education at the college level is essential for students’ quality because it improves their understanding of art, helps them progress in their professional career development, and helps them comprehend more fully the attractiveness of creative creations. As a result, it needs to prioritize aesthetic education at the institution and endeavor to nurture students’ feelings progressively and improve their aesthetic abilities at different levels. Artificial intelligence (AI) is used in this project to create a novel, interdisciplinary teaching technique that will maximize students’ artistic and intellectual potential and help them make more, better art. In this research, the Osprey Optimization method improves the interdisciplinary teaching technique for aesthetic education based on a light Exclusive gradient-boosting mechanism (OOM-LEGBM). The exploration-exploitation dynamics of the OOM are incorporated into LEGBM, providing the students with a tangible and relatable technique to understand complex-solving processes. This research develops an enhanced quality framework for college aesthetic education based on the multi-model data fusion system about the implication and necessity of aesthetic education. The influence of college aesthetic education on students’ creative capacity and artistic literacy was investigated to inform instructional activities better to develop students’ aesthetic skills. The experimental findings suggest that the proposed approach achieved an improved accuracy of 99.90%, higher precision of 99.88%, and greater recall of 99.91%. Moreover, it obtained a minimum Root Mean Square Error (RMSE) of 0.26% and a lower Mean Absolute Error (MAE) of 0.34%, showing that the suggested model greatly improved preference learning accuracy while keeping overall accuracy at an identical level. Innovation capacity building in college aesthetic education can help students become more self-aware, improve their study habits, visually literate, and more comprehensive.

Research and design of key technology and system architecture on simulation confrontation training

Aiming at the problems of unreal training scenes, unintelligent training agents and difficult data interaction in the process of simulation confrontation training, this paper combines image processing, neural network, GBB(general blackboard) technologies with system simulation technology, research around realistic complex environment modeling technology, confrontation agents modeling technology, and multi-model data interaction technology, then puts forward practical solutions. On this basis, a simulation confrontation training system integrating training management, scene development and resource management is designed, which provides a feasible idea and reference for the construction of an immersive, high-intensity and agile interactive confrontation training environment.

Center of Gravity Position Estimation of Counterbalanced Forklift Truck Based on Multi Model Data Fusion

Center of Gravity Position Estimation of Counterbalanced Forklift Truck Based on Multi Model Data Fusion

Uncertainty-guided transformer for brain tumor segmentation.

Multi-model data can enhance brain tumor segmentation for the rich information it provides. However, it also introduces some redundant information that interferes with the segmentation estimation, as some modalities may catch features irrelevant to the tissue of interest. Besides, the ambiguous boundaries and irregulate shapes of different grade tumors lead to a non-confidence estimate of segmentation quality. Given these concerns, we exploit an uncertainty-guided U-shaped transformer with multiple heads to construct drop-out format masks for robust training. Specifically, our drop-out masks are composed of boundary mask, prior probability mask, and conditional probability mask, which can help our approach focus more on uncertainty regions. Extensive experimental results show that our method achieves comparable or higher results than previous state-of-the-art brain tumor segmentation methods, achieving average dice coefficients of [Formula: see text] and Hausdorff distance of 4.91 on the BraTS2021 dataset. Our code is freely available at https://github.com/chaineypung/BTS-UGT.

Value of Multimodal Data From Clinical and Sonographic Parameters in Predicting Recurrence of Hepatocellular Carcinoma After Curative Treatment.

The objective of the work described here was to assess the value of the combination of pre-operative multimodal data-including clinical data, contrast-enhanced ultrasound (CEUS) information and liver stiffness measurement (LSM) based on 2-D shear wave elastography (SWE)-in predicting early (within 1 y) and late (after 1 y) recurrence of hepatocellular carcinoma (HCC) after curative treatment. We retrospectively included 101 patients with HCC who met the Milan criteria and received curative treatment. The multimodel data from clinical parameters, LSM by 2-D SWE and CEUS enhancement patterns were collected. The association between different variables in HCC recurrence was accessed using a Cox proportional hazard model. On the basis of the independent factors of early recurrence, models with differentsource variables were established (Clinical Model, CEUS-Clinical Model, SWE-Clinical Model, CEUS-SWE-Clinical Model). The goodness-of-fit of models was evaluated and the performance trends of different models were calculated by time-dependent area under the curve (AUC). Two-dimensional SWE, CEUS enhancement patterns and clinical parameters (spleen length, multiple tumors, α-fetoprotein, albumin and prothrombin time) were independently associated with early recurrence (all p values <0.05). Multiple tumors and a decrease in albumin independently contributed to the late recurrence. The model fit of CEUS-SWE-Clinical Model was superior to other models in predicting early recurrence (all p values <0.05). The AUCs of the CEUS-Clinical Model were higher from 2 mo to 7 mo, while the SWE-Clinical Model had higher AUCs from 9 mo to 12 mo. CEUS enhancement patterns and 2-D SWE were independent predictors of HCC early recurrence as the two factors contributed to the predictive performance at different times. The multimodal model, which included diverse data in predicting early HCC recurrence, had the best goodness-of-fit.

A multi-model data fusion methodology for reservoir water quality based on machine learning algorithms and bayesian maximum entropy

A major concern in the management of reservoirs is water quality because of the negative consequences it has on both environment and human life. Artificial Intelligence (AI) concept produces a reliable framework to recognize complicated and non-linear correlations between input and output data. Although various machine learning (ML) algorithms in recent studies were employed to predict water quality variables, the existing literature lacks exploring the combination of these algorithms, which has the potential to significantly amplify the outcomes achieved by individual models. Thus, the current study aims to bridge this knowledge gap by evaluating the precision of Random Forest Regression (RFR), Support Vector Regression, Multilayer Perceptron (MLP), and Bayesian Maximum Entropy-based Fusion (BMEF) models to estimate such water quality variables as dissolved oxygen (DO) and chlorophyll-a (Chl-a). The comparisons were conducted in two primary stages: (1) a comparison of the outcomes of different ML algorithms with each other, and (2) comparing the ML algorithms' findings with that of the BMEF model, which considers uncertainty. These comparisons were evaluated using robust statistical measures, and, finally, to indicate the utility and efficacy of the newly introduced framework, it was efficiently utilized in Wadi Dayqah Dam, which is situated in Oman. The findings indicated that, throughout both training and testing phases, the BMEF model outperformed individual machine learning models, namely MLP, RFR, and SVR by 5%, 26%, and 10%, respectively, when R2 and Chl-a are considered as evaluation index and water quality variables, respectively. Additionally, as the individual ML models are not capable of predicting electrical conductivity and oxidation-reduction potential efficiently, the BMEF model leads to better results by R2=0.89, which outperforms MLP (R2=0.81), RFR (R2=0.79), and SVR (R2=0.62) for oxidation-reduction potential. Regarding the study limits of the present study, spatio-temporal data should be collected over a long time to increase the data frequency and reduce the uncertainty related to climate variability.

Projected impacts of climate change on viticulture over French wine regions using downscaled CMIP6 multi-model data

Climate change is a major challenge for the French wine industry. Climatic conditions in French vineyards have already changed and will continue to evolve impacting viticulture.This study aims to analyse the evolution of agro- and eco-climatic indices based on phenology simulation of French wine-growing regions. This evolution was analysed on a recent-past period (1962–1991 to 1992–2021) using SAFRAN climate data and on a future projected period (1985–2014 to 2041–2070) with two SSP trajectories (SSP2-4.5 and SSP5-8.5). A set of 19 CMIP6 climate models downscaled at 8 km grid resolution over France coupled with three phenological and a water balance model were used. Phenological model parameters and training system characteristics were adapted to each region to match as much as possible current practices.Temperatures during the growing season have increased by +1 °C to +2.1 °C since the second half of the 20th century and could rise to +3.7 °C in regions around the Mediterranean by 2070. The inter-model variance concerning the precipitation is high, a significant change (decrease) in precipitation during the grapevine growing season is observed only for the regions of western France (Oceanic climate) over the period 2040–2071 with the SSP5 trajectory. All simulated phenological stages have shifted toward earlier dates. Their occurrence should be even earlier by 2070 with an average advance of up to 22 days for the mid-veraison of Pinot noir in eastern France. The theoretical maturity date (sugar content) should also be advanced from 19 to 30 days depending on the considered region and SSP. Thermal conditions closer to the photosynthetic optimum should promote onset by the early second half of the 21st century. The increase in both the number of hot days and grapevine water deficit during the period of fruit development should impact grape production in quality and quantity in all wine-growing regions. Spring frost projections show no significant change in risk for the second half of the 21st century, compared to current conditions.

Multi-model query languages: taming the variety of big data

A critical issue in Big Data management is to address the variety of data–data are produced by disparate sources, presented in various formats, and hence inherently involves multiple data models. Multi-Model DataBases (MMDBs) have emerged as a promising approach for dealing with this task as they are capable of accommodating multi-model data in a single system and querying across them with a unified query language. This article aims to offer a comprehensive survey of a wide range of multi-model query languages of MMDBs. In particular, we first present the SQL-based extensions toward multi-model data, including the standard SQL extensions such as SQL/XML, SQL/JSON, and GQL, and the non-standard SQL extensions such as SQL++ and SPASQL. We then study the manners in which document-based and graph-based query languages can be extended to support multi-model data. We also investigate the query languages that provide native support on multi-model data. Finally, this article provides insights into the open challenges and problems of multi-model query languages.

A Benchmark for Performance Evaluation of a Multi-Model Database vs. Polyglot Persistence

As the need for handling data from various sources becomes crucial for making optimal decisions, managing multi-model data has become a key area of research. Currently, it is challenging to strike a balance between two methods: polyglot persistence and multi-model databases. Moreover, existing studies suggest that current benchmarks are not completely suitable for comparing these two methods, whether in terms of test datasets, workloads, or metrics. To address this issue, the authors introduce MDBench, an end-to-end benchmark tool. Based on the multi-model dataset and proposed workloads, the experiments reveal that ArangoDB is superior at insertion operations of graph data, while the polyglot persistence instance is better at handling the deletion operations of document data. When it comes to multi-thread and associated queries to multiple tables, the polyglot persistence outperforms ArangoDB in both execution time and resource usage. However, ArangoDB has the edge over MongoDB and Neo4j regarding reliability and availability.

Decomposing Fast and Slow Responses of Global Cloud Cover to Quadrupled CO2 Forcing in CMIP6 Models

Cloud changes and their attribution under global warming still remains a challenge in climatic change studies, especially in decomposing the fast and slow cloud responses to anthropogenic forcing. In this study, the responses of global cloud cover to the quadrupled CO2 forcing are investigated quantitatively by decomposing the total response into fast and slow ones using the multi-model data from the Coupled Model Intercomparison Project Phase 6 (CMIP6). During the quasi-equilibrium period after the quadrupling of CO2 forcing, the global mean changes of simulated total cloud cover (TCC) in the total, fast, and slow responses are −2.42%, −0.64%, and −1.78%, respectively. Overall, the slow response dominates the total response in most regions over the globe with similar spatial patterns. TCC decreases at middle and low latitudes but increases at high latitudes in the total and slow responses. Whereas, it mainly decreases in the middle and low latitudes of the southern hemisphere as well as in the middle and high latitudes of the northern hemisphere in the fast response. A change in vertical motion is the major contributor to the cloud cover change at middle and low latitudes, while the decrease in upper atmospheric temperature leads to an increase in high cloud cover at high latitudes. In addition, the anomaly in water vapor convergence/diffusion also contributes to the cloud cover increase/decrease at low latitudes.

A Survey on Mapping Semi-Structured Data and Graph Data to Relational Data

The data produced by various services should be stored and managed in an appropriate format for gaining valuable knowledge conveniently. This leads to the emergence of various data models, including relational, semi-structured, and graph models, and so on. Considering the fact that the mature relational databases established on relational data models are still predominant in today’s market, it has fueled interest in storing and processing semi-structured data and graph data in relational databases so that mature and powerful relational databases’ capabilities can all be applied to these various data. In this survey, we review existing methods on mapping semi-structured data and graph data into relational tables, analyze their major features, and give a detailed classification of those methods. We also summarize the merits and demerits of each method, introduce open research challenges, and present future research directions. With this comprehensive investigation of existing methods and open problems, we hope this survey can motivate new mapping approaches through drawing lessons from each model’s mapping strategies, as well as a new research topic - mapping multi-model data into relational tables.

IoMT with Deep CNN: AI-Based Intelligent Support System for Pandemic Diseases

The Internet of Medical Things (IoMT) is an extended version of the Internet of Things (IoT). It mainly concentrates on the integration of medical things for servicing needy people who cannot get medical services easily, especially rural area people and aged peoples living alone. The main objective of this work is to design a real time interactive system for providing medical services to the needy who do not have a sufficient medical infrastructure. With the help of this system, people will get medical services at their end with minimal medical infrastructure and less treatment cost. However, the designed system could be upgraded to address the family of SARs viruses, and for experimentation, we have taken COVID-19 as a test case. The proposed system comprises of many modules, such as the user interface, analytics, cloud, etc. The proposed user interface is designed for interactive data collection. At the initial stage, it collects preliminary medical information, such as the pulse oxygen rate and RT-PCR results. With the help of a pulse oximeter, they could get the pulse oxygen level. With the help of swap test kit, they could find COVID-19 positivity. That information is uploaded as preliminary information to the designed proposed system via the designed UI. If the system identifies the COVID positivity, it requests that the person upload X-ray/CT images for ranking the severity of the disease. The system is designed for multi-model data. Hence, it can deal with X-ray, CT images, and textual data (RT-PCR results). Once X-ray/CT images are collected via the designed UI, those images are forwarded to the designed AI module for analytics. The proposed AI system is designed for multi-disease classification. It classifies the patients affected with COVID-19 or pneumonia or any other viral infection. It also measures the intensity level of lung infection for providing suitable treatment to the patients. Numerous deep convolution neural network (DCNN) architectures are available for medical image classification. We used ResNet-50, ResNet-100, ResNet-101, VGG 16, and VGG 19 for better classification. From the experimentation, it observed that ResNet101 and VGG 19 outperform, with an accuracy of 97% for CT images. ResNet101 outperforms with an accuracy of 98% for X-ray images. For obtaining enhanced accuracy, we used a major voting classifier. It combines all the classifiers result and presents the majority voted one. It results in reduced classifier bias. Finally, the proposed system presents an automatic test summary report textually. It can be accessed via user-friendly graphical user interface (GUI). It results in a reduced report generation time and individual bias.

Artificial Intelligence in power multimodal data analysis

In the past decade, electric power data has undergone a transformation from non-existent to centralized management. The management efficiency and accumulated quantity have reached a high level. Furthermore, this data is widely applied in various fields of the electric power industry and plays a significant role. This article studies the multi-model data problems in many application scenarios in the electric power industry. Combining the characteristics and actual scenarios of the electric power industry, it deeply explores the data sources and characteristics of various tasks, and integrates the path of artificial intelligence analysis in the multi-modal field to explore how to solve related problems in the electric power industry. Additionally, this article further expands on how weakly supervised and semi-supervised methods can be utilized in the electric power industry and proposes suggestions.

PBACalib: Targetless Extrinsic Calibration for High-Resolution LiDAR-Camera System Based on Plane-Constrained Bundle Adjustment

The strategy of fusing multi-model data especially from cameras, light detection and ranging sensors (LiDAR), is frequently considered in robotics to enhance the performance of the perception and navigation tasks. Extrinsic calibration, which spatially aligns different sources into a unified coordinate representation, directly determines the performance of the combined data. In this letter, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PBACalib</i> , a novel targetless extrinsic calibration algorithm aiming at the dense LiDAR-camera system based on the plane-constrained bundle adjustment (PBA). The proposed method utilizes the feature points derived from a prominent plane in the scene and iteratively minimizes the reprojection error. A maximum likelihood estimator (MLE) is designed by considering the uncertainty information of the measurements. Furthermore, we explore the distribution of collected data and characterize the robustness and solvability of the extrinsic estimates using a confidence factor. Simulation and real-world experiments both qualitatively and quantitatively demonstrate the robustness and accuracy of our method. The comparison experiments show that the proposed method outperforms another targetless method. To benefit the community, Matlab code has been publicly released on Github.

Logical design of multi-model data warehouses

Multi-model DBMSs, which support different data models with a fully integrated backend, have been shown to be beneficial to data warehouses and OLAP systems. Indeed, they can store data according to the multidimensional model and, at the same time, let each of its elements be represented through the most appropriate model. An open challenge in this context is the lack of methods for logical design. Indeed, in a multi-model context, several alternatives emerge for the logical representation of dimensions and facts. The goal of this paper is to devise a set of guidelines for the logical design of multi-model data warehouses so that the designer can achieve the best trade-off between features such as querying, storage, and ETL. To this end, for each model considered (relational, document-based, and graph-based) and for each type of multidimensional element (e.g., non-strict hierarchy) we propose some solutions and carry out a set of intra-model and inter-model comparisons. The resulting guidelines are then tested on a case study that shows all types of multidimensional elements.

A hybrid deep learning technique based integrated multi-model data fusion for forensic investigation

Digital evidence is an integral part of any trial. Data is critical facts, encrypted information that requires explanation in order to gain meaning and knowledge. The current process of digital forensic research cannot effectively address the various aspects of a complex infrastructure. Therefore, digital forensics requires the optimal processing of a complex infrastructure that differs from the current process and structure. For a long time, digital forensic research has been utilized to discuss these issues. In this research, we offer a forensic investigation hybrid deep learning approach based on integrated multi-model data fusion (HDL-DFI). First, we concentrate on digital evidence collection and management systems, which can be achieved by an integrated data fusion model with the help of an improved brain storm optimization (IBSO) algorithm. Here, we consider several multimedia data’s for evidence purposes, i.e. text, image, speech, physiological signals, and video. Then, we introduce a recurrent multiplicative neuron with a deep neural network (RM-DNN) for data de-duplication in evidence collection, which avoids repeated and redundant data. After that, we design a multistage dynamic neural network (MDNN) for sentimental analysis to decide what type of crime has transpired and classify the action on it. Finally, the accuracy, precision, recall, F1-score, G-mean, and area under the curve of our proposed HDL-DFI model implemented with the standard benchmark database and its fallouts are compared to current state-of-the-art replicas (AUC). The results of our experiments show that the computation time of the proposed model HDL-DFI is 20% and 25% lower than the previous model’s for uploading familiar and unfamiliar files, 22% and 29% lower for authentication generation, 23% and 31% lower for the index service test scenario, and 24.097% and 32.02% lower for familiarity checking.

A universal approach for multi-model schema inference

The variety feature of Big Data, represented by multi-model data, has brought a new dimension of complexity to all aspects of data management. The need to process a set of distinct but interlinked data models is a challenging task. In this paper, we focus on the problem of inference of a schema, i.e., the description of the structure of data. While several verified approaches exist in the single-model world, their application for multi-model data is not straightforward. We introduce an approach that ensures inference of a common schema of multi-model data capturing their specifics. It can infer local integrity constraints as well as intra- and inter-model references. Following the standard features of Big Data, it can cope with overlapping models, i.e., data redundancy, and it is designed to process efficiently significant amounts of data.To the best of our knowledge, ours is the first approach addressing schema inference in the world of multi-model databases.

A penalty function semi-continuous thresholding methods for constraints of hashing problems

The Hashing process is an effective tool for handling large-scale data (for example, images, videos, or multi-model data) retrieval problems. To get better retrieval accuracy, hashing models usually are imposed with three rigorous constraints, i.e., discrete binary constraint, uncorrelated condition, and the balanced constraint, which will lead to being ‘NP-hard’. In this study, we divide the whole constraints set into the uncorrelated (orthogonality) constraint and the binary discrete balance constraint and propose a fast and accurate penalty function semi-continuous thresholding (PFSCT) hash coding algorithm based on forward–backward algorithms. In addition, we theoretically analyze the equivalence between the relaxed model and the original problems. Extensive numerical experiments on diverse large-scale benchmark datasets demonstrate comparable performance and effectiveness of the proposed method.