Multi-model Feature Research Articles

Evaluating the Application of Machine Learning in Predicting the Mortality of Hospitalized COVID-19 Patients Using the Confusion Matrix and the Matthews Correlation Coefficient

Background: The COVID-19 pandemic, which occurred between 2019 and 2023, posed a significant threat to global health. Its high transmissibility, the emergence of new variants, and the novel nature of the disease made treatment and control highly challenging. Objectives: This study aimed to develop an algorithm for predicting the mortality of hospitalized COVID-19 patients using machine learning methods. Methods: This cross-sectional study was conducted on 581 hospitalized COVID-19 patients. The approach integrated multi-model features derived from computed tomography (CT) scans and electronic health record (EHR) data. High-resolution computed tomography (HRCT) images were initially processed using the Pulmonary Toolkit package in MATLAB software. Subsequently, the extracted variables were entered into the model as predictive factors, alongside demographic characteristics, underlying conditions, and laboratory results of the patients. The machine learning model was developed using the AdaBoost method by incorporating demographic and laboratory data with HRCT features. Results: In this study, 581 hospitalized COVID-19 patients were included. Among them, 199 (34.25%) patients died, while 382 (65.75%) recovered. According to the machine learning algorithm, the most effective variables for predicting COVID-19 mortality were lymphocyte variables, CRP, age, mean lung density, lung tissue percentage, RBC count, D-dimer levels, and emphysema. The MCC Index in this study was 0.73, and the area under the ROC curve was 0.96. Conclusions: According to our results, the three variables with the greatest impact on predicting mortality in COVID-19 patients were related to HRCT findings, laboratory results, and patient age. Therefore, it is recommended that, given the high cost of HRCT, this diagnostic test should only be performed if other risk factors are identified in laboratory results. If necessary, HRCT should be conducted promptly.

Adaptive Multicore Dual-Path Fusion Multimodel Extraction of Heterogeneous Features for FAIMS Spectral Analysis.

With the increasing application scenarios and detection needs of high-field asymmetric waveform ion mobility spectrometry (FAIMS) analysis, deep learning-assisted spectral analysis has become an important method to improve the analytical effect and work efficiency. However, a single model has limitations in generalizing to different types of tasks, and a model trained from one batch of spectral data is difficult to achieve good results on another task with large differences. To address this problem, this study proposes an adaptive multicore dual-path fusion multimodel extraction of heterogeneous features for FAIMS spectral analysis model in conjunction with FAIMS small-sample data analysis scenarios. Multinetwork complementarity is achieved through multimodel feature extraction, adaptive feature fusion module adjusts feature size and dimension fusion to heterogeneous features, and multicore dual-path fusion can capture and integrate information at all scales and levels. The model's performance improves dramatically when performing complex mixture multiclassification tasks: accuracy, precision, recall, f1-score, and micro-AUC reach 98.11%, 98.66%, 98.33%, 98.30%, and 98.98%. The metrics for the generalization test using the untrained xylene isomer data were 96.42%, 96.66%, 96.96%, 96.65%, and 97.60%. The model not only exhibits excellent analytical results on preexisting data but also demonstrates good generalization ability on untrained data.

A novel multi-model feature generation technique for suicide detection.

Automated expert systems (AES) analyzing depression-related content on social media have piqued the interest of researchers. Depression, often linked to suicide, requires early prediction for potential life-saving interventions. In the conventional approach, psychologists conduct patient interviews or administer questionnaires to assess depression levels. However, this traditional method is plagued by limitations. Patients might not feel comfortable disclosing their true feelings to psychologists, and counselors may struggle to accurately predict situations due to limited data. In this context, social media emerges as a potentially valuable resource. Given the widespread use of social media in daily life, individuals often express their nature and mental state through their online posts. AES can efficiently analyze vast amounts of social media content to predict depression levels in individuals at an early stage. This study contributes to this endeavor by proposing an innovative approach for predicting suicide risks using social media content and machine learning techniques. A novel multi-model feature generation technique is employed to enhance the performance of machine learning models. This technique involves the use of a feature extraction method known as term frequency-inverse document frequency (TF-IDF), combined with two machine learning models: logistic regression (LR) and support vector machine (SVM). The proposed technique calculates probabilities for each sample in the dataset, resulting in a new feature set referred to as the probability-based feature set (ProBFS). This ProBFS is compact yet highly correlated with the target classes in the dataset. The utilization of concise and correlated features yields significant outcomes. The SVM model achieves an impressive accuracy score of 0.96 using ProBFS while maintaining a low computational time of 5.63 seconds even when dealing with extensive datasets. Furthermore, a comparison with state-of-the-art approaches is conducted to demonstrate the significance of the proposed method.

Structure-function coupling in white matter uncovers the hypoconnectivity in autism spectrum disorder

BackgroundAutism Spectrum Disorder (ASD) is a neurodevelopmental disorder associated with alterations in structural and functional coupling in gray matter. However, despite the detectability and modulation of brain signals in white matter, the structure-function coupling in white matter in autism remains less explored.MethodsIn this study, we investigated structural-functional coupling in white matter (WM) regions, by integrating diffusion tensor data that contain fiber orientation information from WM tracts, with functional connectivity tensor data that reflect local functional anisotropy information. Using functional and diffusion magnetic resonance images, we analyzed a cohort of 89 ASD and 63 typically developing (TD) individuals from the Autism Brain Imaging Data Exchange II (ABIDE-II). Subsequently, the associations between structural-functional coupling in WM regions and ASD severity symptoms assessed by Autism Diagnostic Observation Schedule-2 were examined via supervised machine learning in an independent test cohort of 29 ASD individuals. Furthermore, we also compared the performance of multi-model features (i.e. structural-functional coupling) with single-model features (i.e. functional or structural models alone).ResultsIn the discovery cohort (ABIDE-II), individuals with ASD exhibited widespread reductions in structural-functional coupling in WM regions compared to TD individuals, particularly in commissural tracts (e.g. corpus callosum), association tracts (sagittal stratum), and projection tracts (e.g. internal capsule). Notably, supervised machine learning analysis in the independent test cohort revealed a significant correlation between these alterations in structural-functional coupling and ASD severity scores. Furthermore, compared to single-model features, the integration of multi-model features (i.e., structural-functional coupling) performed best in predicting ASD severity scores.ConclusionThis work provides novel evidence for atypical structural-functional coupling in ASD in white matter regions, further refining our understanding of the critical role of WM networks in the pathophysiology of ASD.

Multimodal Emotion Cognition Method Based on Multi-Channel Graphic Interaction

The relationship between the emotional components associated with images and text is a crucial way of multimodal emotion analysis. However, most of the present multimodel affective cognitive models simply associate the features of images and texts without thoroughly investigating their interactions, resulting in poor recognition. Therefore, a multimodel emotion cognition method based on multi-channel graphic interaction is proposed. Text context features are extracted, scene and image information is encoded, and useful features are obtained. Based on these results, the modal alignment module be applied to obtain information about affective regions and words, and then the cross-modal gating module be applied to combine the multimodel features. In addition, we tested extensively on three open datasets, achieving an accuracy of 0.8122 for the MSA-single dataset, 0.7307 for the MSA-MULTIPLE dataset, and 0.7159 for TumEmo. The results show that this method is effective for multimodal emotion detection.

Multi-modality risk prediction of cardiovascular diseases for breast cancer cohort in the All of Us Research Program.

This study leverages the rich diversity of the All of Us Research Program (All of Us)'s dataset to devise a predictive model for cardiovascular disease (CVD) in breast cancer (BC) survivors. Central to this endeavor is the creation of a robust data integration pipeline that synthesizes electronic health records (EHRs), patient surveys, and genomic data, while upholding fairness across demographic variables. We have developed a universal data wrangling pipeline to process and merge heterogeneous data sources of the All of Us dataset, address missingness and variance in data, and align disparate data modalities into a coherent framework for analysis. Utilizing a composite feature set including EHR, lifestyle, and social determinants of health (SDoH) data, we then employed Adaptive Lasso and Random Forest regression models to predict 6 CVD outcomes. The models were evaluated using the c-index and time-dependent Area Under the Receiver Operating Characteristic Curve over a 10-year period. The Adaptive Lasso model showed consistent performance across most CVD outcomes, while the Random Forest model excelled particularly in predicting outcomes like transient ischemic attack when incorporating the full multi-model feature set. Feature importance analysis revealed age and previous coronary events as dominant predictors across CVD outcomes, with SDoH clustering labels highlighting the nuanced impact of social factors. The development of both Cox-based predictive model and Random Forest Regression model represents the extensive application of the All of Us, in integrating EHR and patient surveys to enhance precision medicine. And the inclusion of SDoH clustering labels revealed the significant impact of sociobehavioral factors on patient outcomes, emphasizing the importance of comprehensive health determinants in predictive models. Despite these advancements, limitations include the exclusion of genetic data, broad categorization of CVD conditions, and the need for fairness analyses to ensure equitable model performance across diverse populations. Future work should refine clinical and social variable measurements, incorporate advanced imputation techniques, and explore additional predictive algorithms to enhance model precision and fairness. This study demonstrates the liability of the All of Us's diverse dataset in developing a multi-modality predictive model for CVD in BC survivors risk stratification in oncological survivorship. The data integration pipeline and subsequent predictive models establish a methodological foundation for future research into personalized healthcare.

Multi-model feature aggregation for classification of laser welding images with vision transformer

The laser welding technique is quite common in various manufacturing lines, including those for lithium-ion power batteries, due to its remarkable productivity, effectiveness, and flexibility. However, it has been observed that the consistency of the welding quality is not always optimal. This study investigates the adoption of laser welding in the manufacturing of lithium-ion batteries, such as their anode, cathode and safety vent, where consistent weld quality is crucial for battery performance and safety. Therefore, evaluating the laser-welding product is indispensable for industrial production and the public domain. Several techniques have been utilized for laser welding evaluation, both destructive and non-destructive. However, these methods are ineffective and too cumbersome to be adopted in mass manufacturing. In contrast, a machine vision strategy has recently been adopted to distinguish between successful and unsuccessful laser welding items. This opened new perspectives for evaluating the quality of the laser welding product using digital image techniques. However, these methods cannot deliver outstanding performance across multiple laser-welding products and achieve optimal classification accuracy. Deep learning has evolved remarkably in recent years and gained popularity in detecting welding defects. This paper presents the observation of a Hybrid Vision Transformer (HViT) to classify the laser welding images corresponding to their feature patch images based on multi-model feature aggregation. The VGG-16 and MobileNet, which were pre-trained on ImageNet, were utilized as the core models to extract the rich features of the laser welding image. To integrate these features, the squeeze excitation module (SE) was utilized, and a multi-layer perception (MLP) approach with a label smoothing optimizer was used for classification. To determine the effectiveness of the proposed strategy, a comparative analysis was conducted with a multitude of alternative machine learning and deep learning approaches. The results indicate that the proposed strategy surpasses all other methods in terms of classification accuracy and evaluation metrics on the test dataset.

DMFF: Deep multimodel feature fusion for building occupancy detection

The 2022 Global Status Report for Buildings and Construction (Buildings-GSR) indicates that construction activities have returned to pre-pandemic levels in most major economies, alongside more building energy consumption. To achieve the Net Zero emissions target by 2050, particularly in the post-pandemic era, accurate occupancy information is important to enhance building energy efficiency and improve occupancy comfort. While remarkable progress has been made in existing studies, they struggle to make full use of multi-sensor data to achieve high accuracy. Furthermore, there is a high expectation of the multimodel multi-temporary fusion by Transformer. In this study, we present a Transformer-based multimodal, multi-temporal feature fusion method (DMFF) for occupancy detection. To transfer domain knowledge from artificial intelligence into the building area, DMFF includes a pretrain-finetune pipeline and leverages pre-trained visual and sound models. Multiple pre-trained Transformer encoders are employed to extract features of different modalities. Then, we propose a self-attention mechanism for modality fusion to learn relationships among various sensors. Our DMFF method demonstrates superior performance on a real dataset, outperforming machine and deep learning methods (e.g., Convolutional Neural Networks, Random Forest, and Multilayer Perceptrons). Applied to a room setting, DMFF shows promising potential for building energy savings. The code and demo are accessible at https://github.com/kailaisun/multimodel_occupancy.

Object aroused emotion analysis network for image sentiment analysis

From cognitive psychology, objects are closely related to emotions, and inherently possess the ability to arouse human emotions. Hence, fully utilizing the relationships between objects and emotions can help achieve more accurate visual emotion recognition. In this paper, we propose a novel object aroused emotion analysis network to realize image sentiment classification by investigating the interactions between objects and emotions. To quantify the various emotion potencies of each object, a novel object emotion distribution module is proposed to explore the mapping among objects and emotions, and quantitatively demonstrate how various objects arouse different emotions. An object emotion-modeling mapping module is proposed to analyze the effect of objects and object combinations with emotion information on image sentiment; this module maps common objects to emotion dimensions and improves image sentiment classification with abundant object combination information. Then, we fuse the object-emotion mapping relation and multi-model features using BiGRU, thus realizing more accurate emotion recognition. Extensive experiments on widely used emotion datasets prove that our proposed method achieves excellent performance and outperforms most state-of-the-art image sentiment classification methods.

Enhancing Breast Cancer Detection and Classification Using Advanced Multi-Model Features and Ensemble Machine Learning Techniques.

Breast cancer (BC) is the most common cancer among women, making it essential to have an accurate and dependable system for diagnosing benign or malignant tumors. It is essential to detect this cancer early in order to inform subsequent treatments. Currently, fine needle aspiration (FNA) cytology and machine learning (ML) models can be used to detect and diagnose this cancer more accurately. Consequently, an effective and dependable approach needs to be developed to enhance the clinical capacity to diagnose this illness. This study aims to detect and divide BC into two categories using the Wisconsin Diagnostic Breast Cancer (WDBC) benchmark feature set and to select the fewest features to attain the highest accuracy. To this end, this study explores automated BC prediction using multi-model features and ensemble machine learning (EML) techniques. To achieve this, we propose an advanced ensemble technique, which incorporates voting, bagging, stacking, and boosting as combination techniques for the classifier in the proposed EML methods to distinguish benign breast tumors from malignant cancers. In the feature extraction process, we suggest a recursive feature elimination technique to find the most important features of the WDBC that are pertinent to BC detection and classification. Furthermore, we conducted cross-validation experiments, and the comparative results demonstrated that our method can effectively enhance classification performance and attain the highest value in six evaluation metrics, including precision, sensitivity, area under the curve (AUC), specificity, accuracy, and F1-score. Overall, the stacking model achieved the best average accuracy, at 99.89%, and its sensitivity, specificity, F1-score, precision, and AUC/ROC were 1.00%, 0.999%, 1.00%, 1.00%, and 1.00%, respectively, thus generating excellent results. The findings of this study can be used to establish a reliable clinical detection system, enabling experts to make more precise and operative decisions in the future. Additionally, the proposed technology might be used to detect a variety of cancers.

Hybrid Multi-model Feature Fusion-Based Vibration Monitoring for Rotating Machine Fault Diagnosis

Hybrid Multi-model Feature Fusion-Based Vibration Monitoring for Rotating Machine Fault Diagnosis

(Retracted) Mimicking human vision systems: deep-learning-based feature fusion for semantic image retrieval

Cross-form feature combination is an important multifeature fusion technique where the purpose is to implicitly discover the relationship between samples from different modalities, i.e., to retrieve another image encoded by similar semantics through one example image. In the past decade, cross-modal image retrieval has becoming a hotspot investigated by many academicians. Moreover, it is now a significant tool for the future performance enhancement of image retrieval. A long-short term memory (LSTM)-based feature fusion model is proposed. First, aiming at the competitiveness of nonmixed deep architecture for image retrieval, the mechanism of LSTM is introduced in detail. Among them, ground-truth-based methods are used to improve cross-modality. We notice that LSTM can mimic human visual understanding of image semantics well. To improve the accuracy of oblique-form image retrieval, systems based on binary representation are proposed to improve cross-modal similarity and effectiveness of message recovery. Second, we use a quality model to measure the commonly used image low-/high-level visual features, where the disqualified features are abandoned accordingly. This in turn achieves an optimal set of highly descriptive features for image retrieval. Furthermore, we use LSTM and the refined visual features to build a biological model for image retrieval, wherein the multimodel features can be optimally incorporated at the temporal level. Extensive experimental validations on multiple well-known image sets have shown the superiority of our method.

Research on the Gearbox Fault Diagnosis Method Based on Multi-Model Feature Fusion

The gearbox is an important component of rotating machinery and is of great significance for gearbox fault diagnosis. In this paper, a gearbox fault diagnosis model based on multi-model feature fusion was proposed that addressed the limitations of a single or few features reflecting the gearbox’s fault state. The time–frequency feature of the vibration signal was extracted, and the sensitive feature was selected. The sensitive features were extracted using a one-dimensional convolutional neural network. The parallel fusion method was used to fuse the two domain features as inputs to the support vector machine model. The radial basis kernel function and penalty factor of the support vector machine were optimized by improving the particle swarm optimization algorithm. Finally, the gearbox states were identified using the optimized support vector machine model. The results show that the recognition rate of the proposed model is 98.3%, which is higher than that of other models.

A multi-model feature fusion model for lithium-ion battery state of health prediction

State of health (SOH) prediction is key to battery health management and safety. Health indicators (HIs) are effective and feasible to predict battery SOH. The existing approaches according to HIs focused on single-source features of HIs such as voltage, current or temperature by a single model to predict SOH. The accuracy and robustness of these approaches can still be improved especially for the lack of battery datasets in applications. Multi-sources HIs can enrich the diversity of features and supply complementary information. In addition, multi-model fusion for multi-sources features can improve the robustness of prediction results. In this paper, a multi-model feature fusion based on multi-source features is proposed to improve the effectiveness and robustness of battery SOH prediction. 27 HIs are firstly extracted from multi-sources signals of the charge-discharge process, and the HIs are divided into three classes by the Pearson correlation coefficient. Subsequently, three feature vectors for the classified HIs are obtained individually by three different deep learning models according to HIs' characteristics. Finally, the feature space is fused from the three feature vectors to predict SOH by the fully connected network (FCN). The effectiveness of the proposed method is verified on the MIT dataset. Results show that the MSE and the MAPE of the proposed method achieve 0.0007 % and 0.2106 %, respectively. Compared with the results of single models and different HIs subsets, the proposed method realizes high accuracy and robustness of SOH prediction.

Anomaly detection in Internet of medical Things with Blockchain from the perspective of deep neural network

IoMT technology has many advantages in healthcare system, such as optimizing the medical service model, improving the efficiency of hospital operation and management, and improving the overall service level of the hospital. IoMT devices do not have a security authentication mechanism, and the trust between devices relies heavily on centralized third-party services. Blockchain can provide a secure interactive environment for the medical Internet of Things. However, security issues in the IoMT-Blockchain environment are also becoming increasingly prominent. Cyber-attacks targeting IoMT-Blockchain will not only compromise the security of IoT devices, but also seriously affect the security of the Internet. Therefore, how to detect abnormal traffic in the IoMT-Blockchain environment becomes particularly important. In this work, an abnormal traffic detection with deep neural network is designed for abnormal traffic detection in IoMT-Blockchain environment. First, this work proposes a feature extraction algorithm based on multi-model autoencoders. The algorithm processes the feature information in groups to reduce the complexity between traffic feature information. It builds a multi-model autoencoder to further extract fusion features between multi-model features. Second, to maximize use of traffic data information in detection network, this work proposes a multi-feature sequence anomaly detection algorithm. The algorithm extracts low-level fusion features and high-level temporal features in network traffic respectively, and applies the features to anomaly detection and classification tasks by means of residual learning.

BTG: A Bridge to Graph machine learning in telecommunications fraud detection

Telecommunications fraud runs rampant recently around the world. Therefore, how to effectively detect fraudsters has become an increasingly challenging problem. However, previous studies either assume that the samples are independent of each other and use non-graph methods, or use local subgraphs with good connectivity for graph-based anomaly detection. Few prior works have performed graph-based fraud detection on real-world Call Detail Records (CDR) meta data sets with sparse connectivity. To solve this problem, we propose an end-to-end telecommunications fraud detection framework named Bridge To Graph (BTG). BTG leverages the subscriber synergy behavior to reconstruct connectivity, which bridges the gap between sparse connectivity data and graph machine learning. Concretely, we extract multi-model features from meta data and perform Box–Cox transformation first. Then, aiming at the sparse connectivity of real-world CDR meta data, the graph is reconstructed through dimensionally selectable link prediction of node similarity. Finally, the reconstructed graph and node features are input into the graph machine learning module for node embedding representation learning and fraud node classification. Comprehensive experiments on the real-world telecommunications network CDR data set show that our proposed method outperforms the classic methods in many metrics. Beyond telecom fraud detection, our method can also be extended to anomaly detection scenarios with no graph or sparse connectivity graph.

Encoder deep interleaved network with multi-scale aggregation for RGB-D salient object detection

Recently, RGB-D salient object detection (SOD) has aroused widespread research interest. Existing RGB-D SOD approaches mainly consider the cross-modal information fusion in the decoder. And their multi-modal interaction mainly concentrates on the same level of features between RGB stream and depth stream. They do not deeply explore the coherence of multi-model features at different levels. In this paper, we design a two-stream deep interleaved encoder network to extract RGB and depth information and realize their mixing simultaneously. This network allows us to gradually learn multi-modal representation at different levels from shallow to deep. Moreover, to further fuse multi-modal features in the decoding stage, we propose a cross-modal mutual guidance module and a residual multi-scale aggregation module to implement the global guidance and local refinement of the salient region. Extensive experiments on six benchmark datasets demonstrate that the proposed approach performs favorably against most state-of-the-art methods under different evaluation metrics. During the testing stage, this model can run at a real-time speed of 93 FPS.

Sports motional characteristics modeling by leveraging multi-modal image technique

Sport motional characteristic modeling is a hot research topic in human–computer interaction (HCI). Traditional sport motional characteristic based on single signal cannot achieve satisfactory performance. To solve this problem, we propose a multi-model feature fusion-based method for sport motional feature control. More specifically, we leverage Kinect sensor to acquire real-time video stream, where the main goal is to capture hand gesture as well as arm movements. HSV-based image segmentation is used for hand image patches extraction and recognition. To improve the effectiveness of sport motional characteristics control, we design audio-based HCI system to assist sport control. Our experiment is conducted on a quadruped robot platform with manipulator. Experimental results show the effectiveness of our proposed method.

Human action identification by a quality-guided fusion of multi-model feature

Human motion recognition has become an active research area in the field of computer vision due to its wide range of implementations in domains of video monitoring, virtual reality, human–machine interaction. Dealing with the problem that the RGB images cannot provide enough depth information, a multi-modal depth neural network based on joint cost function is proposed for human motion recognition. In the architecture, the features of the RGB video frames are extracted by the 3D CNN architecture while the characteristics of human motion recognition in the SSDDI graphics utilizing depth map are extracted by the LSTM. Moreover, the model utilizes joint cost function including the cross-entropy loss and the distance constraint between the feature space of training samples and their center values within each category. The experimental results on the MSR Action 3D datasets suggest that the proposed model demonstrates a higher accuracy rate than do the other competing models.

Multimodel Feature Reinforcement Framework Using Moore-Penrose Inverse for Big Data Analysis.

Fully connected representation learning (FCRL) is one of the widely used network structures in multimodel image classification frameworks. However, most FCRL-based structures, for instance, stacked autoencoder encode features and find the final cognition with separate building blocks, resulting in loosely connected feature representation. This article achieves a robust representation by considering a low-dimensional feature and the classifier model simultaneously. Thus, a new hierarchical subnetwork-based neural network (HSNN) is proposed in this article. The novelties of this framework are as follows: 1) it is an iterative learning process, instead of stacking separate blocks to obtain the discriminative encoding and the final classification results. In this sense, the optimal global features are generated; 2) it applies Moore-Penrose (MP) inverse-based batch-by-batch learning strategy to handle large-scale data sets, so that large data set, such as Place365 containing 1.8 million images, can be processed effectively. The experimental results on multiple domains with a varying number of training samples from ∼ 1 K to ∼2 M show that the proposed feature reinforcement framework achieves better generalization performance compared with most state-of-the-art FCRL methods.