Missing Data Research Articles

Prediction of heart disease based on logistic regression

Heart disease is a major threat to human health, with a variety of contributing factors, and is not easily cured. This paper will present a dataset from a cardiovascular study of residents of Framingham, Massachusetts. First, the validity of the three models, logistic regression, random forest, and decision tree, is estimated by comparing information such as accuracy, precision, recall, and F1 values. The optimal model, i.e., the logistic regression model, was selected by plotting ROC curves and using AUC as a reference criterion for assessing the predictive effectiveness of the models. Then the raw data and data were preprocessed, including dealing with missing values. Finally, a logistic regression model was developed to analyze the influencing factors of heart disease. The purpose of this study was to use the results of the logistic model to help doctors and patients in heart disease treatment. The results show that the model has a good predictive effect.

A Hybrid Model for Short-Term Energy Load Prediction Based on Transfer Learning with LightGBM for Smart Grids in Smart Energy Systems

ABSTRACT Electricity load prediction plays a vital role in energy management. The necessity of combining scattered power generation channels of sustainable energy and other substitute resources into distribution grid networks is increasing because of the 4.1 percent rise in world electricity since 2021. The utilization of probability power generators and energy storage is maximized by short-term load prediction methods, which make it possible to anticipate potential energy usage. To facilitate the growth of energy systems in quite a sustainable way, urban energy planners utilize a range of methods and technological advances, including simulation, probability optimization, comprehensive improvement, and modelling techniques for predicting power requirements. Predicting energy consumption at granular time scales (e.g., hourly or sub-hourly intervals) is common practice for short-term energy load prediction. For similar time-series forecasting challenges, the authors have decided to use transfer learning (TL) with light gradient-boosting machine (LightGBM).1 This model can benefit from models trained on larger data sets through TL, which might boost prediction accuracy. Energy prediction and effective resource utilization largely support the accurate transition of energy systems. In this paper, the authors propose a hybrid model for short-term energy load prediction. Based on TL to optimize LightGBM (OLGBM) for smart grids, The proposed model first applies data pre-processing using an abnormal supplement strategy with an immediate deviation selection technique. This phase eliminates the missing values and extracts the required features. The second phase uses a TL-OLGBM with hyperparameters. The TL method helps to learn the dynamic time scale and complex data patterns, and the hyperparameters are tuned via the Bayesian optimization technique, which addresses the challenge of short-term load forecasting (STLF) and increases forecasting accuracy. Finally, an optimized LightGBM model is applied, combining the time and energy features to predict effective energy forecasting. The proposed model gained a 2.834589 mean absolute percentage error (MAPE) value and a 0.9706495 GINI index after execution on the prescribed data set. We compare the performance of the proposed model with other comparable models, and the simulation outcomes show that the model can generate the best results for MAPE, accuracy, and root-mean-square deviation (RMSE).

Multi-metric comparison of machine learning imputation methods with application to breast cancer survival

Handling missing data in clinical prognostic studies is an essential yet challenging task. This study aimed to provide a comprehensive assessment of the effectiveness and reliability of different machine learning (ML) imputation methods across various analytical perspectives. Specifically, it focused on three distinct classes of performance metrics used to evaluate ML imputation methods: post-imputation bias of regression estimates, post-imputation predictive accuracy, and substantive model-free metrics. As an illustration, we applied data from a real-world breast cancer survival study. This comprehensive approach aimed to provide a thorough assessment of the effectiveness and reliability of ML imputation methods across various analytical perspectives. A simulated dataset with 30% Missing At Random (MAR) values was used. A number of single imputation (SI) methods - specifically KNN, missMDA, CART, missForest, missRanger, missCforest - and multiple imputation (MI) methods - specifically miceCART and miceRF - were evaluated. The performance metrics used were Gower’s distance, estimation bias, empirical standard error, coverage rate, length of confidence interval, predictive accuracy, proportion of falsely classified (PFC), normalized root mean squared error (NRMSE), AUC, and C-index scores. The analysis revealed that in terms of Gower’s distance, CART and missForest were the most accurate, while missMDA and CART excelled for binary covariates; missForest and miceCART were superior for continuous covariates. When assessing bias and accuracy in regression estimates, miceCART and miceRF exhibited the least bias. Overall, the various imputation methods demonstrated greater efficiency than complete-case analysis (CCA), with MICE methods providing optimal confidence interval coverage. In terms of predictive accuracy for Cox models, missMDA and missForest had superior AUC and C-index scores. Despite offering better predictive accuracy, the study found that SI methods introduced more bias into the regression coefficients compared to MI methods. This study underlines the importance of selecting appropriate imputation methods based on study goals and data types in time-to-event research. The varying effectiveness of methods across the different performance metrics studied highlights the value of using advanced machine learning algorithms within a multiple imputation framework to enhance research integrity and the robustness of findings.

An efficient diagnosis of heart disease using optimized cross-layer Densenet121 pyramid mutual attention network

Heart disease significantly impacts global mortality rates, underscoring the need for improved early-stage diagnosis through deep learning algorithms to enhance prognosis accuracy and prevent mortality. However, challenges such as missing values, class imbalance, redundant features and faulty predictions persist in heart disease diagnosis. To address these issues, this research proposes an Optimized Cross-layer Densenet121 Pyramid mutual attention Network (OCDPN) to improve prediction accuracy. Initially, a Hybrid Bag-boost K-means Synthetic Minority Oversampling TEchnique (HBK-SMOTE) is employed to eliminate missing values and handle class imbalance issues. Subsequently, a Two-layer Angle Kernel Linear Discriminant Extreme Learning (TK-LDE) method extracts features and the Hybrid Binary Aquila Beluga whale Optimization (HBABO)selects the most significant features. Finally, the selected features are classified using Fick's law algorithm to achieve high accuracy. The OCDPN model, comprising several dense layers and a softmax classifier, with Fick's law optimization, enhances feature understanding and classification performance. The model's efficiency is assessed using the Framingham Heart Study (FHS), Statlog (Heart), Cleveland Heart Disease (CHD) and Z-Alizadeh Sani datasets. The introduced technique demonstrates outstanding performance, achieving accuracies of 98.2%, 99.8%, 99.2% and 98.5%, respectively, for the considered datasets. These outcomes highlight the effectiveness of the introduced method in predicting heart diseases.

CRYPTOCURRENCY PRICE PREDICTION USING LSTM

Cryptocurrency, a digital form of currency, has emerged as a prominent asset class with unique characteristics such as decentralization, security, and global accessibility. As the adoption of cryptocurrencies continues to grow, the need for accurate price prediction using machine learning (ML) algorithms becomes crucial for various stakeholders in the financial ecosystem. This paper presents a comprehensive approach to cryptocurrency price prediction, focusing on the following key aspects: The data for this study is collected from the Binance platform, a leading cryptocurrency exchange known for its extensive market data and liquidity. The dataset includes historical price data across different time frames, including 1-hour, 4-hour, and daily intervals. Prior to analysis, the collected data undergoes thorough preprocessing steps to ensure data quality and consistency. This process includes handling missing values, removing outliers, and standardizing data formats for further analysis. Long Short-Term machine learning model algorithm is employed for price prediction. This model is chosen for its ability to capture complex patterns and dynamics in cryptocurrency price movements. The prediction phase involves training and testing the ML model using the preprocessed data. Performance evaluation metrics such as R-squared, Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean absolute percentage error (MAPE) are utilized to assess the accuracy and robustness of the prediction model across different time frames. Accurate cryptocurrency price prediction is essential for various stakeholders, including investors, traders, businesses, and regulators. It facilitates informed decision- making, risk management, market analysis, trading strategy development, business planning, and regulatory compliance in the dynamic cryptocurrency market. By addressing these key elements, this study aims to contribute to the advancement of cryptocurrency price prediction methodologies using ML techniques, thereby enhancing decision-making processes and fostering a more efficient and transparent digital asset market ecosystem.

How does chronic disease influence subjective life expectancy among middle-aged and older adults? Evidence from the China Health and Retirement Longitudinal Study (CHARLS) 2018 cross-sectional data.

Chronic diseases have become one of the most important factors threatening human health. Subjective life expectancy (SLE) describes an individual's expectation or subjective perception of lifespan. This article aims to explore the relationship between chronic diseases and SLE, as well as the differences among different age groups and different types of chronic diseases in this relationship. China Health and Retirement Longitudinal Study (CHARLS) is a nationwide longitudinal study that evaluates the social, economic, and health conditions of middle-aged and older adult families and individuals aged 45 and above in China. In this study, CHARLS used probability proportional to size sampling (PPS sampling) to ensure the breadth and representativeness of the sample. This study selected cross-sectional data from CHARLS 2018, removed missing values, and obtained a valid sample of 10,658 middle-aged and older individuals, of whom 8564 had chronic diseases. After controlling demographic, health behavior, socioeconomic, psychological, and social security factors, an ordered logistic regression was performed to explore the relationship between chronic disease and SLE in middle-aged and older adults. The results show that chronic diseases negatively correlate with SLE in middle-aged and older adults. Middle-aged and older adults with chronic diseases are 36.2% less likely to have high life expectancy than those without chronic diseases. Many different types of chronic diseases are negatively correlated with SLE. Cancer is most negatively correlated with SLE, far exceeding other chronic diseases. Chronic disease and SLE of middle-aged and older adults have age-heterogeneous differences. For middle-aged people aged 45-59 and young older adults aged 60-79, there is a significant correlation between chronic diseases and SLE. However, there is no correlation between chronic diseases and subjective life expectancy in the older population aged 80 and above. The government and society should pay close attention to the prevention and treatment of chronic diseases among middle-aged and older adults and adjust policies and measures according to the population's age structure. In addition, the government and society should pay attention to the spiritual needs of middle-aged and older adults. The government and society should pay more attention to cancer patients. Finally, the scientific research team should also strengthen research on chronic diseases, research and development of specific drugs and vaccines, improve the cure rate of chronic diseases, promote people's health, and make people no longer afraid of diseases.

GSAAN with War Strategy Optimization Algorithm for Cyber Security in a 5G Wireless Communication Network

In this paper, the graph sample and aggregate-attention network with war strategy optimization algorithm for cyber security in the 5G wireless communication network (CS-5GWCN-GSAAN-WSOA) is proposed in 5G mobile networks to identify cyber threats. Initially, the input data are amassed from the 5G-NIDD dataset. Then the input data are fed to preprocessing, here redundancy elimination and missing value replacement are performed utilizing the contrast limited adaptive histogram equalization filtering (CLAHEF) method. After preprocessing, optimal features are selected by the stochastic gradient boosting based recursive feature elimination process. The selected features are fed to the graph sample and aggregate-attention network (GSAAN) to detect cyber-attacks in 5G wireless communication. Generally, the GSAAN approach does not adopt any optimization techniques for determining optimal parameters and guaranteeing accurate attack detection. Therefore, the war strategy optimization algorithm (WSOA) contemplates to optimize the GSAAN weight parameters. The CS-5GWCN-GSAAN-WSOA method is activated on Python. The CS-5GWCN-GSAAN-WSOA method attains 22.51%, 20.35%, and 35.54% higher accuracy, 19.45%, 27.80%, and 18.93% higher sensitivity analysed with existing models, like cyber security attack identification in 5G wireless systems utilizing machine learning (CS-5GWCN-SVM), enhanced dropping attacks detection in 5G networks depending on deep learning models (CS-5GWCN-KNN), and cyber security issues in 5G enabled attack detection through deep learning (CS-5GWCN-RNN), respectively.

Identify the most appropriate imputation method for handling missing values in clinical structured datasets: a systematic review

Background and objectivesComprehending the research dataset is crucial for obtaining reliable and valid outcomes. Health analysts must have a deep comprehension of the data being analyzed. This comprehension allows them to suggest practical solutions for handling missing data, in a clinical data source. Accurate handling of missing values is critical for producing precise estimates and making informed decisions, especially in crucial areas like clinical research. With data's increasing diversity and complexity, numerous scholars have developed a range of imputation techniques. To address this, we conducted a systematic review to introduce various imputation techniques based on tabular dataset characteristics, including the mechanism, pattern, and ratio of missingness, to identify the most appropriate imputation methods in the healthcare field.Materials and methodsWe searched four information databases namely PubMed, Web of Science, Scopus, and IEEE Xplore, for articles published up to September 20, 2023, that discussed imputation methods for addressing missing values in a clinically structured dataset. Our investigation of selected articles focused on four key aspects: the mechanism, pattern, ratio of missingness, and various imputation strategies. By synthesizing insights from these perspectives, we constructed an evidence map to recommend suitable imputation methods for handling missing values in a tabular dataset.ResultsOut of 2955 articles, 58 were included in the analysis. The findings from the development of the evidence map, based on the structure of the missing values and the types of imputation methods used in the extracted items from these studies, revealed that 45% of the studies employed conventional statistical methods, 31% utilized machine learning and deep learning methods, and 24% applied hybrid imputation techniques for handling missing values.ConclusionConsidering the structure and characteristics of missing values in a clinical dataset is essential for choosing the most appropriate data imputation technique, especially within conventional statistical methods. Accurately estimating missing values to reflect reality enhances the likelihood of obtaining high-quality and reusable data, contributing significantly to precise medical decision-making processes. Performing this review study creates a guideline for choosing the most appropriate imputation methods in data preprocessing stages to perform analytical processes on structured clinical datasets.

Imputation of cancer proteomics data with a deep model that learns from many datasets.

Missing values are a major challenge in the analysis of mass spectrometry proteomics data. Missing values hinder reproducibility, decrease statistical power for identifying differentially expressed (DE) proteins and make it challenging to analyze low-abundance proteins. We present Lupine, a deep learning-based method for imputing, or estimating, missing values in tandem mass tag (TMT) proteomics data. Lupine is, to our knowledge, the first imputation method that is designed to learn jointly from many datasets, and we provide evidence that this approach leads to more accurate predictions. We validated Lupine by applying it to TMT data from >1,000 cancer patient samples spanning ten cancer types from the Clinical Proteomics Tumor Atlas Consortium (CPTAC). Lupine outperforms the state of the art for TMT imputation, identifies more DE proteins than other methods, corrects for TMT batch effects, and learns a meaningful representation of proteins and patient samples. Lupine is implemented as an open source Python package.

A novel and efficient risk minimisation-based missing value imputation algorithm

Missing value imputation (MVI) is a key task in data science, in which learning models are built from incomplete data. In contrast to externally driven MVI algorithms, this study proposes a novel risk minimisation-based MVI algorithm (RM-MVI) that considers both the internal characteristics of missing data and the external performance for specific classification applications. RM-MVI is technically designed for labelled data and is applied in two stages: filling with structural risk minimisation (SRM) and refining with empirical risk minimisation (ERM). In the filling stage, an autoencoder with a single hidden layer is trained on the original dataset without missing values. Missing values are first initialised with random numbers, and the imputation values are then preliminarily optimised based on the derived updating rule to minimise the structural risk-oriented objective function. After the imputation values have been preliminarily optimised in the filling stage, a neural-network-based classifier is trained in the refining stage to optimise the imputation values sophisticatedly by reducing the empirical risk. Experiments were conducted on several benchmark datasets to validate the feasibility, rationality, and effectiveness of the proposed RM-MVI algorithm. The results show that (1) the optimisation processes of the imputation values corresponding to the SRM and ERM are convergent so that the optimised imputation values can be obtained; (2) SRM can ensure distribution consistency of the imputation values that are preliminarily optimised in the filling stage, while ERM can optimise the imputation values sophisticatedly in the refining stage, which is more helpful for classifier training; and (3) the RM-MVI algorithm can yield considerably better MVI performance on benchmark datasets than 11 well-known MVI algorithms, such as a 26% higher distribution consistency ratio and 2% to 5% higher testing accuracies for 6 classifiers on average. This demonstrates that RM-MVI is a viable approach for addressing MVI problems.

Effect of in-stream ads on viewer’s attitude to purchase—The moderating role of viewer’s control

Research in the field of online advertising has focused on the effect of in-stream ads on viewers’ attitudes and intentions to purchase. However, little is known regarding the crucial role of viewer’s control in terms of the ‘skip ad option’ towards the attitude to purchase. This research aims to investigate the effect of in-stream ads on viewers’ attitudes to purchasing with the moderating role of viewer control. Primary data was collected from respondents of Vehari district of Pakistan through a questionnaire based on 5 points Likert scale. 370 questionnaires were incorporated after excluding the questionnaires having missing values. Structural equation modelling was used through SmartPLS-3 software in testing the hypotheses. The findings reveal that, in-stream (emotional, informational, and entertaining) ads have positive impact on viewers’ attitudes, and viewers’ control moderates the relationship between in-stream ads and viewers’ attitudes towards the ads. Further, viewers’ attitude toward the ads has a significant positive impact on viewers’ intention to purchase. To the best of our knowledge this is one of the first studies that examines the effect of in-stream ads on viewers’ attitudes to purchasing with the moderating role of viewer control in the context of a developing country, like Pakistan.

Small data estimation for binary variables with big data: A comparison of calibrated nearest neighbour and hierarchical Bayes methods of estimation

A recent application in machine learning has introduced a novel approach, complemented by big data sources, aimed at providing precise estimates for small geographical areas. This method employs a dual strategy: (a) hybrid estimation, involving the integration of big data sources with imputed values derived from K nearest neighbours (KNN) to address missing target variable values from the big data source; and (b) calibration of the collective sum of small area estimates to an independent yet efficient national total. Evaluating its efficacy using simulated data from the 2016 Australian population census, the calibrated KNN (CKNN) method demonstrated superior performance compared to the Fay-Herriot method based on area-level covariates. This paper enhances the comparative analysis by contrasting the CKNN method with a hierarchical Bayes method using the logit-normal model (LN) relevant for binary data. Broadly speaking, the LN method can be viewed as the Bayesian equivalent of Battese-Harter-Fuller (BHF) method, which incorporates unit-level covariates. Our results demonstrate the CKNN method’s superiority over the LN method. However, the application of hybrid estimation to the LN method significantly diminishes this superiority. Although CKNN estimates maintain better precision, they are not as accurate as the estimates from the hybridized LN method.

Generation of global 1 km all-weather instantaneous and daily mean land surface temperatures from MODIS data

Abstract. Land surface temperature (LST) serves as a crucial variable in characterizing climatological, agricultural, ecological, and hydrological processes. Thermal infrared (TIR) remote sensing provides high temporal and spatial resolutions for obtaining LST information. Nevertheless, TIR-based satellite LST products frequently exhibit missing values due to cloud interference. Prior research on estimating all-weather instantaneous LST has predominantly concentrated on regional or continental scales. This study involved generating a global all-weather instantaneous and daily mean LST product spanning from 2000 to 2020 using XGBoost. Multisource data, including Moderate-Resolution Imaging Spectroradiometer (MODIS) top-of-atmosphere (TOA) observations, surface radiation products, and reanalysis data, were employed. Validation using an independent dataset of 77 individual stations demonstrated the high accuracy of our products, yielding root mean squared errors (RMSEs) of 2.787 K (instantaneous) and 2.175 K (daily). The RMSE for clear-sky conditions was 2.614 K for the instantaneous product, which is slightly lower than the cloudy-sky RMSE of 2.931 K. Our instantaneous and daily mean LST products exhibit higher accuracy compared to the MODIS official LST product (instantaneous RMSE = 3.583 K; daily 3.105 K) and the land component of the fifth generation of the European ReAnalysis (ERA5-Land) LST product (instantaneous RMSE = 4.048 K; daily 2.988 K). Significant improvements are observed in our LST product, notably at high latitudes, compared to the official MODIS LST product. The LST dataset from 2000 to 2020 at the monthly scale, the daily mean LST on the first day of 2010 can be freely downloaded from https://doi.org/10.5281/zenodo.4292068 (Li et al., 2024), and the complete product will be available at https://glass-product.bnu.edu.cn/ (last access: 22 August 2024).

Association of Hormonal Exposures With Grip Strength in Women >45 Years: Data From the CONSTANCES Cohort Study.

Although biological findings show that estrogens are beneficial for muscular mass maintenance and bone resorption inhibition, the association of hormonal exposure with physical performance are controversial. We investigated the association of reproductive history and exogenous hormone use with hand-grip strength (GS) in women. Using the data from the CONSTANCES French prospective population-based cohort study, we ran linear mixed models to investigate the association of reproductive history and exogenous hormones use with maximal GS in 37 976 women aged 45 to 69 years recruited between 2012 and 2020. We used multiple imputation by chained equations to control missing values and corrections for multiple testing. The mean age of women was 57.2 years. Mean GS was 26.6 kg. After adjustment for age and confounders, GS increased with age at menarche (β+1 year = 0.14; 95% CI, 0.10-0.17) and duration of breastfeeding (β for ≥10 months vs <5 months = 0.39; 95% CI, 0.20-0.59; P for linear trend <.01). Compared to nonmenopausal women, postmenopausal women had significantly lower GS (β = -0.78; 95% CI, -0.98 to -0.58). GS was negatively associated with hormone therapy (HT) past use (β = -0.25; 95% CI, -0.42 to -0.07). Our results suggested that menopausal transition was strongly associated with lower GS. However, despite our hypothesis, increased age at menarche and duration of breastfeeding were associated with higher GS and HT past users presented lower GS than HT never users. These findings could help identify women at high risk of poor physical performance.

Prediction of COVID-19 Hospitalization and Mortality Using Artificial Intelligence.

COVID-19 has had a substantial influence on healthcare systems, requiring early prognosis for innovative therapies and optimal results, especially in individuals with comorbidities. AI systems have been used by healthcare practitioners for investigating, anticipating, and predicting diseases, through means including medication development, clinical trial analysis, and pandemic forecasting. This study proposes the use of AI to predict disease severity in terms of hospital mortality among COVID-19 patients. A cross-sectional study was conducted at King Abdulaziz University, Saudi Arabia. Data were cleaned by encoding categorical variables and replacing missing quantitative values with their mean. The outcome variable, hospital mortality, was labeled as death = 0 or survival = 1, with all baseline investigations, clinical symptoms, and laboratory findings used as predictors. Decision trees, SVM, and random forest algorithms were employed. The training process included splitting the data set into training and testing sets, performing 5-fold cross-validation to tune hyperparameters, and evaluating performance on the test set using accuracy. The study assessed the predictive accuracy of outcomes and mortality for COVID-19 patients based on factors such as CRP, LDH, Ferritin, ALP, Bilirubin, D-Dimers, and hospital stay (p-value ≤ 0.05). The analysis revealed that hospital stay, D-Dimers, ALP, Bilirubin, LDH, CRP, and Ferritin significantly influenced hospital mortality (p ≤ 0.0001). The results demonstrated high predictive accuracy, with decision trees achieving 76%, random forest 80%, and support vector machines (SVMs) 82%. Artificial intelligence is a tool crucial for identifying early coronavirus infections and monitoring patient conditions. It improves treatment consistency and decision-making via the development of algorithms.

Modeling the Long-Term Variability in the Surfaces of Three Lakes in Morocco with Limited Remote Sensing Image Sources

Satellite imagery has become a widespread resource for modeling variability in lake surfaces. However, the extended monitoring of a lake’s perimeter faces significant challenges due to atmospheric obstacles that cannot be rectified. Due to the atmosphere’s everchanging opacity, only half of the acquired satellite images have reliable qualitative accuracy making it possible to identify a lake’s contour. Consequently, approximately 50% of the monthly lake outline values can be determined using remote sensing methods, leaving the remaining 50% unknown. This situation is applicable to three lakes in Morocco (Abakhan, Ouiouan, and Tiglmanine), the subjects of the current research for the period between 1984 and 2022. What can we do if, during a period of time in which we monitored the evolution of the surface of a lake by satellite means, we obtain only about 50% of the possible images? Shall we just settle for this and stop the analysis? Although it may be challenging to believe, the present study introduces two statistical methods for interpolating and validating the monthly values of the lake outline: the iterative ratio method based on the autocorrelation of the monthly water balance and the Kalman filter. We estimated the reconstruction errors of the missing values and validated the methodology using an inverse philosophy, reconstructing the initial data from the table of the simulation results. Given that the difference between the initial values and the reconstructed initial values resembles white noise or an AR (1) process with a low coefficient, we deemed the methodological approach acceptable. Several comparison criteria between the two interpolation methods were employed, yet determining the more appropriate one remains challenging. Based on our surface reconstruction method, Lake Abakhan, with an average area of 22 hectares, experienced significant fluctuations, ranging from a maximum of 34 hectares in 2010 to a minimum of 0.8 hectares in 2022. Lake Ouiouan, with an average area of 14 hectares, displayed much lower variation, with a maximum of 17 hectares in 2020 and a minimum of 6.5 hectares in 1988. Lake Tiglmanine showed a pattern similar to that of Lake Abakhan but with less pronounced fluctuations. With an average area of 6.1 hectares, its maximum was 9.2 hectares in 2011 and its minimum was 4.1 hectares in 1984.

Comparison of Imputation Methods for Categorical Real-World Prostate Cancer Data with Natural Order.

Missing values (NA) often occur in cancer research, which may be due to reasons such as data protection, data loss, or missing follow-up data. Such incomplete patient information can have an impact on prediction models and other data analyses. Imputation methods are a tool for dealing with NA. Cancer data is often presented in an ordered categorical form, such as tumour grading and staging, which requires special methods. This work compares mode imputation, k nearest neighbour (knn) imputation, and, in the context of Multiple Imputation by Chained Equations (MICE), logistic regression model with proportional odds (mice_polr) and random forest (mice_rf) on a real-world prostate cancer dataset provided by the Cancer Registry of Rhineland-Palatinate in Germany. Our dataset contains relevant information for the risk classification of patients and the time between date of diagnosis and date of death. For the imputation comparison, we use Rubin's (1974) Missing Completely At Random (MCAR) mechanism to remove 10%, 20%, 30%, and 50% observations. The results are evaluated and ranked based on the accuracy per patient. Mice_rf performs significantly best for each percentage of NA, followed by knn, and mice_polr performs significantly worst. Furthermore, our findings indicate that the accuracy of imputation methods increases with a lower number of categories, a relatively even proportion of patients in the categories, or a majority of patients in a particular category.

Experimentation of AI Models Towards the Prediction of Medium-Risk Emergency Department Cases Disposition Outcome.

The overwhelming volume of patients in emergency departments (EDs) is a significant problem that hinders the delivery of high quality healthcare. Despite their great value, triage protocols are challenging to put into practice. This paper examines the utility of prediction models as a tool for clinical decision support, with a focus on medium-severity patients as defined by the ESI algorithm. 689 cases of medium-risk patients were gathered from the AHEPA hospital, evaluated, and their data fed three classifiers: XGBoost (XGB), Random Forest (RF) and Logistic Regression (LR), with the prediction goal being the outcome of their visit, i.e. admission or discharge. Essential features for the prediction task were determined using feature importance and distribution analysis. Despite having many missing values or high sparsity datasets, several symptoms and metrics are recommended as crucial for outcome prediction. When fed the patients' vital signs, XGB achieved an accuracy score of 91.30%. Several chief complaints were also proven beneficial. Prediction models can, in general, not only lessen the drawbacks of triage implementation, but also enhance its delivery.

NJGCG: A node-based joint Gaussian copula graphical model for gene networks inference across multiple states

Inferring the interactions between genes is essential for understanding the mechanisms underlying biological processes. Gene networks will change along with the change of environment and state. The accumulation of gene expression data from multiple states makes it possible to estimate the gene networks in various states based on computational methods. However, most existing gene network inference methods focus on estimating a gene network from a single state, ignoring the similarities between networks in different but related states. Moreover, in addition to individual edges, similarities and differences between different networks may also be driven by hub genes. But existing network inference methods rarely consider hub genes, which affects the accuracy of network estimation. In this paper, we propose a novel node-based joint Gaussian copula graphical (NJGCG) model to infer multiple gene networks from gene expression data containing heterogeneous samples jointly. Our model can handle various gene expression data with missing values. Furthermore, a tree-structured group lasso penalty is designed to identify the common and specific hub genes in different gene networks. Simulation studies show that our proposed method outperforms other compared methods in all cases. We also apply NJGCG to infer the gene networks for different stages of differentiation in mouse embryonic stem cells and different subtypes of breast cancer, and explore changes in gene networks across different stages of differentiation or different subtypes of breast cancer. The common and specific hub genes in the estimated gene networks are closely related to stem cell differentiation processes and heterogeneity within breast cancers.

Dynamic attention aggregated missing spatial–temporal data imputation for traffic speed prediction

Traffic speed forecasting is indispensable in Intelligent Traffic Systems (ITS). The missing traffic speed due to sensors’ failure may hamper the accurate prediction of traffic speed. Different kinds of imputation techniques were employed to replace the missing values considering the spatial and temporal information. However, previous techniques did not consider the dynamic contribution of the neighboring nodes and failed to capture the complex properties of traffic speed data precisely. The architecture of the previous imputation models was infeasible in real-time applications in terms of scalability. Therefore, a novel machine learning-based imputation technique is proposed to generate the missing traffic speed using the spatial–temporal information and dynamic contribution of neighboring nodes. A new dataset considering the characteristics of the traffic speed data is generated from the existing dataset by exploiting the non-missing traffic speed of neighboring nodes. The newly formed dataset helps in the estimation of the dynamic contribution of the neighboring nodes using the linear regression algorithm. The non-missing speeds and estimated dynamic contribution of neighboring nodes assist in calculating the missing speeds of a node. Furthermore, a novel deep learning-based prediction model is introduced to forecast traffic speed accurately. The prediction model contains graph structure learning, a deep graph neural network with skip connections, GRU, multi-head soft attention, and a fully connected neural network. Graph structure learning is proposed based on the distribution of the congestion to represent the graph in an efficient way. The deep graph neural network with skip connections and GRU help in capturing topological and temporal information, respectively. The multi-head soft attention is designed to focus on every relevant temporal information at each time step for capturing global traffic information. Finally, the extracted spatial–temporal features are fed into the fully connected neural network to predict the traffic speed. The proposed imputation and prediction models are evaluated on two real-time datasets, METR-LA and PeMSD7 datasets, under different missing rates of different missing patterns. The proposed framework generates possible spatial–temporal information for efficient traffic congestion management and evicts erroneous issues in real-time.