High-dimensional Medical Data Research Articles

Data-Driven Allocation of Preventive Care with Application to Diabetes Mellitus Type II

Problem definition: Increasing costs of healthcare highlight the importance of effective disease prevention. However, decision models for allocating preventive care are lacking. Methodology/results: In this paper, we develop a data-driven decision model for determining a cost-effective allocation of preventive treatments to patients at risk. Specifically, we combine counterfactual inference, machine learning, and optimization techniques to build a scalable decision model that can exploit high-dimensional medical data, such as the data found in modern electronic health records. Our decision model is evaluated based on electronic health records from 89,191 prediabetic patients. We compare the allocation of preventive treatments (metformin) prescribed by our data-driven decision model with that of current practice. We find that if our approach is applied to the U.S. population, it can yield annual savings of $1.1 billion. Finally, we analyze the cost-effectiveness under varying budget levels. Managerial implications: Our work supports decision making in health management, with the goal of achieving effective disease prevention at lower costs. Importantly, our decision model is generic and can thus be used for effective allocation of preventive care for other preventable diseases. Funding: M. Kraus acknowledges funding from Bundesministerium für Bildung und Forschung [Grant 01IS22080]. S. Feuerriegel acknowledges funding from Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung [Grant 186932]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2021.0251 .

QNMF: A quantum neural network based multimodal fusion system for intelligent diagnosis

The Internet of Medical Things (IoMT) has emerged as a significant research area in the medical field, enabling the transmission of various types of data to the cloud for analysis and diagnosis. Fusing data from multiple modalities can enhance accuracy but requires substantial computing power. Theoretically, quantum computers can rapidly process large volumes of high-dimensional medical data. Despite accelerated developments in quantum computing, research on quantum machine learning (QML) for multimodal data processing remains limited. Considering these factors, this paper presents a quantum neural network-based multimodal fusion system for intelligent diagnosis (QNMF) that can process multimodal medical data transmitted by IoMT devices, fuse data from different modalities, and improve the performance of intelligent diagnosis. This system employs a quantum convolutional neural network (QCNN) to efficiently extract features from medical images. These QCNN-based features are then fused with other modality features (such as blood test results or breast cell slices), and used to train an effective variational quantum classifier (VQC) for intelligent diagnosis. The experimental results demonstrate that a QCNN can effectively extract image data features. Furthermore, QNMF achieved an accuracy of 97.07% and 97.61% on breast cancer diagnosis and Covid-19 diagnosis experiments, respectively. In addition, the QNMF exhibits strong quantum noise robustness.

An IoT and Fuzzy aware e-Healthcare system using feature optimization tuned T-CNN with high dimensional data

Many real-time applications, including some emerging ones, rely on high-dimensional feature datasets. For simplifying the high-dimensional data, the various models are available by using the different feature optimization techniques, clustering and classification techniques. Even though the high-dimensional data is not handled effectively due to the increase in the number of features and the huge volume of data availability. In particular, the high-dimensional medical data needs to be handled effectively to predict diseases quickly. For this purpose, we propose a new Internet of Things and Fuzzy-aware e-healthcare system for predicting various diseases such as heart, diabetes, and cancer diseases effectively. The proposed system uses a newly proposed Intelligent Mahalanobis distance aware Fuzzy Weighted K-Means Clustering Algorithm (IMFWKCA) for grouping the high dimensional data and also applies a newly proposed Moth-Flame Optimization Tuned Temporal Convolutional Neural Network (MFO-TCNN) for predicting the diseases effectively. The experiments have been done by using the UCI Repository Machine Learning datasets and live streaming patient records for evaluating the proposed e-healthcare system and have proved as better than others by achieving better performance in terms of precision, recall, f-measure, and prediction accuracy.

A multi-objective evolutionary algorithm with decomposition and the information feedback for high-dimensional medical data

High-dimensional medical data often leads to a phenomenon known as the ”curse of dimensionality,” which causes additional memory and high training costs, as well as degrading the generalization capacity of learning algorithms. To address this issue, a multi-objective evolutionary algorithm that integrates decomposition and the information feedback model (IFMMOEAD) is proposed for high-dimensional medical data. This algorithm not only considers the number of selected features, but also classification accuracy and correlation measures of features when feature dimensionality reduction is executed. The property of IFMMOEAD is first verified by standard benchmarks DTLZ1–DTLZ7. Then, it is used to develop machine learning algorithms for thirty-five high-dimensional cancer gene expression data sets, showing excellent potential for high-dimensional medical machine learning. Finally, the IFMMOEAD is applied to empirical clinical data of multiple myeloma, significantly outperforming existing algorithms in terms of normalized mutual information and adjusted rand index metrics. We suggest that this algorithm could be implemented in medical information systems as a promising technique for high-dimensional medical problems.

Classification regularized dimensionality reduction improves ultrasound thyroid nodule diagnostic accuracy and inter-observer consistency

ProblemConvolutional Neural Networks (CNNs) for medical image analysis usually only output a probability value, providing no further information about the original image or inter-relationships between different images. Dimensionality Reduction Techniques (DRTs) are used for visualization of high dimensional medical image data, but they are not intended for discriminative classification analysis. AimWe develop an interactive phenotype distribution field visualization system for medical images to accurately reflect the pathological characteristics of lesions and their similarity to assist radiologists in diagnosis and medical research. MethodsWe propose a novel method, Classification Regularized Uniform Manifold Approximation and Projection (UMAP) referred as CReUMAP, combining the advantages of CNN and DRT, to project the extracted feature vector fused with the malignant probability predicted by a CNN to a two-dimensional space, and then apply a spatial segmentation classifier trained on 2614 ultrasound images for prediction of thyroid nodule malignancy and guidance to radiologists. ResultsThe CReUMAP embedding correlates well with the TI-RADS categories of thyroid nodules. The parametric version that embeds external test dataset of 303 images in presence of the training data with known pathological diagnosis improves the benign and malignant nodule diagnostic accuracy (p-value = 0.016) and confidence (p-value = 1.902 × 10−6) of eight radiologists of different experience levels significantly as well as their inter-observer agreements (kappa≥0.75). CReUMAP achieve 90.8% accuracy, 92.1% sensitivity and 88.6% specificity in test set. ConclusionCReUMAP embedding is well correlated with the pathological diagnosis of thyroid nodules, and helps radiologists achieve more accurate, confident and consistent diagnosis. It allows a medical center to generate its locally adapted embedding using an already-trained classification model in an updateable manner on an ever-growing local database as long as the extracted feature vectors and predicted diagnostic probabilities of the correspondent classification model can be outputted.

A machine learning based data modeling for medical diagnosis

High-dimensional medical data makes prediction a complex and difficult task. This study aims at modeling predictive models for medical data. Two datasets of medical data are applied in the study — one online available dataset (Heart Disease data) and another real clinical dataset (Eye Infection Data). A wide range of machine learning algorithms are applied in the modeling stage: Decision Tree, Multilayer Perceptron, Naive Bayesian, Random Forest, and Support Vector Machine. Furthermore, bagging and voting ensemble methods have also been applied with base learners. Both split and cross-validation methods are adopted for the model validation, and well-established evaluation metrics such as accuracy, precision, recall, and F-measure have been considered as evaluation metrics for the predictive models. The method applied for the modeling is comprised of two stages. The first stage uses available features for the predictions. In the second stage, selected features based on positive correlation are used. The adopted method is also for deep learning, especially Convolutional Neural Network (CNN) is applied to analyze the outcomes compared to conventional machine learning algorithms. The experimental results reveal that better predictions are achieved in the second stage. Besides, experiments also indicate split percentage produces better predictive models, and marginally better outcomes are observed in the presence of ensemble methods in comparison with base models. NB outperformed other algorithms with the highest accuracy rate as 88.90%, and MLP obtained 97.50% accuracy for Heart Disease and Eye Infection data, respectively, using 80–20 splits in the second stage. However, the CNN model performed poorly due to the size of the considered datasets.

Binary Approaches of Quantum-Based Avian Navigation Optimizer to Select Effective Features from High-Dimensional Medical Data

Many metaheuristic approaches have been developed to select effective features from different medical datasets in a feasible time. However, most of them cannot scale well to large medical datasets, where they fail to maximize the classification accuracy and simultaneously minimize the number of selected features. Therefore, this paper is devoted to developing an efficient binary version of the quantum-based avian navigation optimizer algorithm (QANA) named BQANA, utilizing the scalability of the QANA to effectively select the optimal feature subset from high-dimensional medical datasets using two different approaches. In the first approach, several binary versions of the QANA are developed using S-shaped, V-shaped, U-shaped, Z-shaped, and quadratic transfer functions to map the continuous solutions of the canonical QANA to binary ones. In the second approach, the QANA is mapped to binary space by converting each variable to 0 or 1 using a threshold. To evaluate the proposed algorithm, first, all binary versions of the QANA are assessed on different medical datasets with varied feature sizes, including Pima, HeartEW, Lymphography, SPECT Heart, PenglungEW, Parkinson, Colon, SRBCT, Leukemia, and Prostate tumor. The results show that the BQANA developed by the second approach is superior to other binary versions of the QANA to find the optimal feature subset from the medical datasets. Then, the BQANA was compared with nine well-known binary metaheuristic algorithms, and the results were statistically assessed using the Friedman test. The experimental and statistical results demonstrate that the proposed BQANA has merit for feature selection from medical datasets.

Population scale latent space cohort matching for the improved use and exploration of observational trial data.

A significant amount of clinical research is observational by nature and derived from medical records, clinical trials, and large-scale registries. While there is no substitute for randomized, controlled experimentation, such experiments or trials are often costly, time consuming, and even ethically or practically impossible to execute. Combining classical regression and structural equation modeling with matching techniques can leverage the value of observational data. Nevertheless, identifying variables of greatest interest in high-dimensional data is frequently challenging, even with application of classical dimensionality reduction and/or propensity scoring techniques. Here, we demonstrate that projecting high-dimensional medical data onto a lower-dimensional manifold using deep autoencoders and post-hoc generation of treatment/control cohorts based on proximity in the lower-dimensional space results in better matching of confounding variables compared to classical propensity score matching (PSM) in the original high-dimensional space (P<0.0001) and performs similarly to PSM models constructed by experts with prior knowledge of the underlying pathology when evaluated on predicting risk ratios from real-world clinical data. Thus, in cases when the underlying problem is poorly understood and the data is high-dimensional in nature, matching in the autoencoder latent space might be of particular benefit.

IPMOD: An efficient outlier detection model for high-dimensional medical data streams

Outlier detection in high-dimensional medical data streams in real-time is critical and challenging research, which is of great help to disease prevention and source analysis. Although academia has done a lot of research on outlier detection of time series data streams, these methods have the following two shortcomings: (1) Insufficient detection accuracy on high-dimensional data streams; (2) Insufficient accuracy in dynamic data streams scenarios low. To this end, we propose a sliding window model based on efficient pruning and information entropy, namely IPMOD(Information Entropy-Pruning Multi-dimensional Outlier Detection). In IPMOD, we first designed a new index weight measurement method combined with information entropy to quantify the weight of different indexes in multi-dimensional data, to determine the influence of different attributes on the prediction results. Then we designed a new sliding window and sub-sequence measurement mechanism to judge whether the data is abnormal based on the distance between the target sequence and the non-self-match. After that, we designed a pruning strategy to further reduce the computational complexity of the algorithm. The final comprehensive experiment shows that our proposed scheme not only has higher detection accuracy than many current schemes on multiple sets of real data-sets but also can quickly detect outliers in different medical data streams in real-time.

SURVFIT: Doubly sparse rule learning for survival data

Survival data analysis has been leveraged in medical research to study disease morbidity and mortality, and to discover significant bio-markers affecting them. A crucial objective in studying high dimensional medical data is the development of inherently interpretable models that can efficiently capture sparse underlying signals while retaining a high predictive accuracy. Recently developed rule ensemble models have been shown to effectively accomplish this objective; however, they are computationally expensive when applied to survival data and do not account for sparsity in the number of variables included in the generated rules. To address these gaps, we present SURVFIT, a “doubly sparse” rule extraction formulation for survival data. This doubly sparse method can induce sparsity both in the number of rules and in the number of variables involved in the rules. Our method has the computational efficiency needed to realistically solve the problem of rule-extraction from survival data if we consider both rule sparsity and variable sparsity, by adopting a quadratic loss function with an overlapping group regularization. Further, a systematic rule evaluation framework that includes statistical testing, decomposition analysis and sensitivity analysis is provided. We demonstrate the utility of SURVFIT via experiments carried out on a synthetic dataset and a sepsis survival dataset from MIMIC-III.

A hybrid feature selection algorithm combining ReliefF and Particle swarm optimization for high-dimensional medical data

Due to high-dimensional feature and strong correlation of features, the classification accuracy of medical data is not as good enough as expected. feature selection is a common algorithm to solve this problem, and selects effective features by reducing the dimensionality of high-dimensional data. However, traditional feature selection algorithms have the blindness of threshold setting and the search algorithms are liable to fall into a local optimal solution. Based on it, this paper proposes a hybrid feature selection algorithm combining ReliefF and Particle swarm optimization. The algorithm is mainly divided into three parts: Firstly, the ReliefF is used to calculate the feature weight, and the features are ranked by the weight. Then ranking feature is grouped according to the density equalization, where the density of features in each group is the same. Finally, the Particle Swarm Optimization algorithm is used to search the ranking feature groups, and the feature selection is performed according to a new fitness function. Experimental results show that the random forest has the highest classification accuracy on the features selected. More importantly, it has the least number of features. In addition, experimental results on 2 medical datasets show that the average accuracy of random forest reaches 90.20%, which proves that the hybrid algorithm has a certain application value.

Brain Tumor Segmentation of T1w MRI Images Based on Clustering Using Dimensionality Reduction Random Projection Technique.

Early diagnosis of a brain tumor may increase life expectancy. Magnetic resonance imaging (MRI) accompanied by several segmentation algorithms is preferred as a reliable method for assessment. The availability of high-dimensional medical image data during diagnosis places a heavy computational burden and a suitable pre-processing step is required for lower- dimensional representation. The storage requirement and complexity of image data are also a concern. To address this concern, the random projection technique (RPT) is widely used as a multivariate approach for data reduction. This study mainly focuses on T1-weighted MRI image clustering for brain tumor segmentation with dimension reduction by using the conventional principal component analysis (PCA) and RPT. Two clustering algorithms, K-means and fuzzy c-means (FCM) were used for brain tumor detection. The primary study objective was to present a comparison of the two clustering methods between MRI images subjected to PCA and RPT. In addition to the original dimension of 512 × 512, three other image sizes, 256 × 256, 128 × 128, and 64 × 64, were used to determine the effect of the methods. In terms of average reconstruction, Euclidean distance, and segmentation distance errors, the RPT produced better results than the PCA method for all the clustered images from clustering techniques. According to the values of performance metrics, RPT supported fuzzy c-means in achieving the best clustering performance and provided significant results for each new size of the MRI images.

Framelet pooling aided deep learning network: the method to process high dimensional medical data

Machine learning-based analysis of medical images often faces several hurdles, such as the lack of training data, the curse of the dimensionality problem, and generalization issues. One of the main difficulties is that there exists a computational cost problem in dealing with input data of large size matrices which represent medical images. The purpose of this paper is to introduce a framelet-pooling aided deep learning method for mitigating computational bundles caused by large dimensionality. By transforming high dimensional data into low dimensional components by filter banks and preserving detailed information, the proposed method aims to reduce the complexity of the neural network and computational costs significantly during the learning process. Various experiments show that our method is comparable to the standard unreduced learning method, while reducing computational burdens by decomposing large-sized learning tasks into several small-scale learning tasks.

Morbigenous brain region and gene detection with a genetically evolved random neural network cluster approach in late mild cognitive impairment.

The multimodal data fusion analysis becomes another important field for brain disease detection and increasing researches concentrate on using neural network algorithms to solve a range of problems. However, most current neural network optimizing strategies focus on internal nodes or hidden layer numbers, while ignoring the advantages of external optimization. Additionally, in the multimodal data fusion analysis of brain science, the problems of small sample size and high-dimensional data are often encountered due to the difficulty of data collection and the specialization of brain science data, which may result in the lower generalization performance of neural network. We propose a genetically evolved random neural network cluster (GERNNC) model. Specifically, the fusion characteristics are first constructed to be taken as the input and the best type of neural network is selected as the base classifier to form the initial random neural network cluster. Second, the cluster is adaptively genetically evolved. Based on the GERNNC model, we further construct a multi-tasking framework for the classification of patients with brain disease and the extraction of significant characteristics. In a study of genetic data and functional magnetic resonance imaging data from the Alzheimer's Disease Neuroimaging Initiative, the framework exhibits great classification performance and strong morbigenous factor detection ability. This work demonstrates that how to effectively detect pathogenic components of the brain disease on the high-dimensional medical data and small samples. The Matlab code is available at https://github.com/lizi1234560/GERNNC.git.

Efficient Shapley Explanation For Features Importance Estimation Under Uncertainty.

Complex deep learning models have shown their impressive power in analyzing high-dimensional medical image data. To increase the trust of applying deep learning models in medical field, it is essential to understand why a particular prediction was reached. Data feature importance estimation is an important approach to understand both the model and the underlying properties of data. Shapley value explanation (SHAP) is a technique to fairly evaluate input feature importance of a given model. However, the existing SHAP-based explanation works have limitations such as 1) computational complexity, which hinders their applications on high-dimensional medical image data; 2) being sensitive to noise, which can lead to serious errors. Therefore, we propose an uncertainty estimation method for the feature importance results calculated by SHAP. Then we theoretically justify the methods under a Shapley value framework. Finally we evaluate our methods on MNIST and a public neuroimaging dataset. We show the potential of our method to discover disease related biomarkers from neuroimaging data.

A Hybrid Fish – Bee Optimization Algorithm for Heart Disease Prediction using Multiple Kernel SVM Classifier

The patient’s heart disease status is obtained by using a heart disease detection model. That is used for the medical experts. In order to predict the heart disease, the existing technique use optimal classifier. Even though the existing technique achieved the better result, it has some disadvantages. In order to improve those drawbacks, the suggested technique utilizes the effective method for heart disease prediction. At first the input information is preprocessed and then the preprocessed result is forwarded to the feature selection process. For the feature selection process a proficient feature selection is used over the high dimensional medical data. Hybrid Fish Bee optimization algorithm (HFSBEE) is utilized. Thus, the proposed algorithm parallelizes the two algorithms such that the local behavior of artificial bee colony algorithm and global search of fish swarm optimization are effectively used to find the optimal solution. Classification process is performed by the transformation of medical dataset to the Multi kernel support vector machine (MKSVM). The process of our proposed technique is calculated based on the accuracy, sensitivity, specificity, precision, recall and F-measure. Here, for test analysis, the some datasets used i.e. Cleveland, Hungarian and Switzerland etc., that are given based on the UCI machine learning repository. The experimental outcome show that our presented technique is went better than the accuracy of 97.68%. This is for the Cleveland dataset when related with existing hybrid kernel support vector machine (HKSVM) method achieved 96.03% and optimal rough fuzzy classifier obtained 62.25%. The implementation of the proposed method is done by MATLAB platform.

Deep learning cardiac motion analysis for human survival prediction.

Motion analysis is used in computer vision to understand the behaviour of moving objects in sequences of images. Optimising the interpretation of dynamic biological systems requires accurate and precise motion tracking as well as efficient representations of high-dimensional motion trajectories so that these can be used for prediction tasks. Here we use image sequences of the heart, acquired using cardiac magnetic resonance imaging, to create time-resolved three-dimensional segmentations using a fully convolutional network trained on anatomical shape priors. This dense motion model formed the input to a supervised denoising autoencoder (4Dsurvival), which is a hybrid network consisting of an autoencoder that learns a task-specific latent code representation trained on observed outcome data, yielding a latent representation optimised for survival prediction. To handle right-censored survival outcomes, our network used a Cox partial likelihood loss function. In a study of 302 patients the predictive accuracy (quantified by Harrell’s C-index) was significantly higher (p = .0012) for our model C=0.75 (95% CI: 0.70 - 0.79) than the human benchmark of C=0.59 (95% CI: 0.53 - 0.65). This work demonstrates how a complex computer vision task using high-dimensional medical image data can efficiently predict human survival.

An effective feature selection for heart disease prediction with aid of hybrid kernel SVM

In today's modern world cardiovascular disease is the most lethal one. This disease attacks a person so instantly that it hardly gets any time to get treated with. So, diagnosing patients correctly on timely basis is the most challenging task for the medical fraternity. In order to reduce the risk of heart disease, effective feature selection and classification based prediction system is proposed. An efficient feature selection is applied on the high dimensional medical data, for selecting the features fish swarm optimisation algorithm is used. After that, selected features from medical dataset are fed to the HKSVM for classification. The performance of the proposed technique is evaluated by accuracy, sensitivity, specificity, precision, recall and f-measure. Experimental results indicate that the proposed classification framework have outperformed by having better accuracy of 96.03% for Cleveland dataset when compared existing SVM method only achieved 91.41% and optimal rough fuzzy classifier achieved 62.25%.

Novel ensemble method for the prediction of response to fluvoxamine treatment of obsessive-compulsive disorder.

ObjectiveAbout 30% of obsessive–compulsive disorder (OCD) patients exhibit an inadequate response to pharmacotherapy. The detection of clinical variables associated with treatment response may result in achievement of remission in shorter period, preventing illness development and reducing socioeconomic costs.MethodsIn total, 330 subjects with OCD diagnosis underwent 12-week pharmacotherapy with fluvoxamine (150–300 mg). Treatment response was ≥25% reduction in Yale-Brown Obsessive–Compulsive Scale (Y-BOCS) score. In total, 36 clinical attributes of 151 subjects who had completed their treatment course were analyzed. Data mining algorithms included missing value handling, feature selection, and new analytical method based on ensemble classification. The results were compared with those of other traditional classification algorithms such as decision tree, support vector machines, k-nearest neighbor, and random forest.ResultsSexual and contamination obsessions are high-ranked predictors of resistance to fluvoxamine pharmacotherapy as well as high Y-BOCS obsessive score. Our results showed that the proposed analysis strategy has good ability to distinguish responder and nonresponder patients according to their clinical features with 86% accuracy, 79% sensitivity, and 89% specificity.ConclusionThis study proposed an analytical approach which is an accurate and a sensitive method for the analysis of high-dimensional medical data sets containing more number of missing values. The treatment of OCD could be improved by better understanding of the predictors of pharmacotherapy, which may lead to more effective treatment of patients with OCD.

An Adaptive Harmony Search Approach for Gene Selection and Classification of High Dimensional Medical Data

In Bioinformatics, microarray data analysis has gained substantial attention for disease diagnosis. Microarray data is represented with a huge search space which imposes the foremost difficulties in selection of most relevant facts in terms of genes. In this esteem, we have recommended a hybridised harmony search and Pareto optimization approach for feature selection in high dimensional data classification problem. In the first stage an adaptive harmony search algorithm for gene selection with probability distribution factor for optimal gene ranking is implemented. This selection is further refined applying a bi-objective Pareto based feature selection technique to select optimal minimum number of top ranked genes. The importance and relevance of the selected genes are verified through a few numbers of classifiers. Experimental analysis is conducted over four well known microarray datasets. Finally statistical analysis is conducted to prove the superiority of proposed work with two other nature inspired algorithms. Simulation result reveals that the proposed hybridisation is providing high potentiality in both sample classification and feature subset prediction prospective for high dimensional databases.