K-nearest Neighbor Imputation Research Articles

Methods to Handle Missing depression (PHQ-8 Score) Values among Adults in the United Arab Emirates

Abstract Background The UAE Healthy Future Study (UAEHFS) is one of the first large prospective cohort studies in the Gulf region which examines causes and risk factors for chronic diseases among adult UAE nationals. Missing values are often unavoidable in empirical research and can in many cases, lead to bias. The aim of this study is to estimate the percentage of depression in the UAEHFS pilot data using the eight-item Patient Health Questionnaire (PHQ-8) variables, using different statistical methods. Methods Five common statistical machine learning methods of handling missing values were included in this analysis. These are mode imputation, k-nearest neighbor (KNN) imputation, classification, and regression trees (CART), random forest (RF) imputations, and random sample from observed values (Sample). 100 multiple imputations were used. Results 487 (94.2 %) eligible participants were included in the analysis. 231 (44.7%) were included in the complete case analysis. The median age was 30 years (Interquartile-Range: 23 - 38). More males (67.8%) than females included in the analysis. The estimated percentage of depression was 8.4%, 8.9%, 9.9%, 12.5%, 15.4% and 17.9% by the mode, complete case, sample, RF, CART, and KNN respectively. In additional analyses, the estimated proportions of depression were 11.5% by the Complete Case, 11.9% by KNN, 13.2% by K-means clustering, and 13.2% by Random Forest. Conclusions The estimated percentage of depression in the UAEHFS pilot data varies between the applied methods of handling missing values. This shows that the problem of missing values in the variables is not negligible. Further research is needed using multiple imputations in the main UAEHFS dataset after completing recruitment. Key messages • For the depression missing values, we recommend using multiple imputations not to generate data but to prevent the exclusion of observed data. • To have a better estimate of the percentage of depression, is recommended to use different machine learning methods.

Impacts of Data Preprocessing and Sampling Techniques on Solar Flare Prediction from Multivariate Time Series Data of Photospheric Magnetic Field Parameters

The accurate prediction of solar flares is crucial due to their risks to astronauts, space equipment, and satellite communication systems. Our research enhances solar flare prediction by employing sophisticated data preprocessing and sampling techniques for the Space Weather Analytics for Solar Flares (SWAN-SF) data set, a rich source of multivariate time series data of solar active regions. Our study adopts a multifaceted approach encompassing four key methodologies. Initially, we address over 10 million missing values in the SWAN-SF data set through our innovative imputation technique called fast Pearson correlation-based k-nearest neighbors imputation. Subsequently, we propose a precise normalization technique, called LSBZM normalization, tailored for time series data, merging various strategies (log, square root, Box–Cox, Z-score, and min–max) to uniformly scale the data set's 24 attributes (photospheric magnetic field parameters), addressing issues such as skewness. We also explore the “near decision boundary sample removal” technique to enhance the classification performance of the data set by effectively resolving the challenge of class overlap. Finally, a pivotal aspect of our research is a thorough evaluation of diverse oversampling and undersampling methods, including SMOTE, ADASYN, Gaussian noise injection, TimeGAN, Tomek links, and random undersampling, to counter the severe imbalance in the SWAN-SF data set, notably a 60:1 ratio of major (X and M) to minor (C, B, and FQ) flaring events in binary classification. To demonstrate the effectiveness of our methods, we use eight classification algorithms, including advanced deep-learning-based architectures. Our analysis shows significant true skill statistic scores, underscoring the importance of data preprocessing and sampling in time-series-based solar flare prediction.

Explainable Ensemble Learning Approaches for Predicting the Compression Index of Clays

Accurate prediction of the compression index (cc) is essential for geotechnical infrastructure design, especially in clay-rich coastal regions. Traditional methods for determining cc are often time-consuming and inconsistent due to regional variability. This study presents an explainable ensemble learning framework for predicting the cc of clays. Using a comprehensive dataset of 1080 global samples, four key geotechnical input variables—liquid limit (LL), plasticity index (PI), initial void ratio (e0), and natural water content w—were leveraged for accurate cc prediction. Missing data were addressed with K-Nearest Neighbors (KNN) imputation, effectively filling data gaps while preserving the dataset’s distribution characteristics. Ensemble learning techniques, including Random Forest (RF), Gradient Boosting Decision Trees (GBDT), Extreme Gradient Boosting (XGBoost), and a Stacking model, were applied. Among these, the Stacking model demonstrated the highest predictive performance with a Root Mean Squared Error (RMSE) of 0.061, a Mean Absolute Error (MAE) of 0.043, and a Coefficient of Determination (R2) value of 0.848 on the test set. Model interpretability was ensured through SHapley Additive exPlanations (SHAP), with e0 identified as the most influential predictor. The proposed framework significantly improves both prediction accuracy and interpretability, offering a valuable tool to enhance geotechnical design efficiency in coastal and clay-rich environments.

Bilateral Matching Method for Business Resources Based on Synergy Effects and Incomplete Data.

On the third-party cloud platform, to help enterprises accurately obtain high-quality and valuable business resources from the massive information resources, a bilateral matching method for business resources, based on synergy effects and incomplete data, is proposed. The method first utilizes a k-nearest neighbor imputation algorithm, based on comprehensive similarity, to fill in missing values. Then, it constructs a satisfaction evaluation index system for business resource suppliers and demanders, and the weights of the satisfaction evaluation indices are determined, based on the fuzzy analytic hierarchy process (FAHP) and the entropy weighting method (EWM). On this basis, a bilateral matching model is constructed with the objectives of maximizing the satisfaction of both the supplier and the demander, as well as achieving the synergy effect. Finally, the model is solved using the linear weighting method to obtain the most satisfactory business resources for both supply and demand. The effectiveness of the method is verified through a practical application and comparative experiments.

A Comparative Study on Imputation Techniques: Introducing a Transformer Model for Robust and Efficient Handling of Missing EEG Amplitude Data.

In clinical datasets, missing data often occur due to various reasons including non-response, data corruption, and errors in data collection or processing. Such missing values can lead to biased statistical analyses, reduced statistical power, and potentially misleading findings, making effective imputation critical. Traditional imputation methods, such as Zero Imputation, Mean Imputation, and k-Nearest Neighbors (KNN) Imputation, attempt to address these gaps. However, these methods often fall short of accurately capturing the underlying data complexity, leading to oversimplified assumptions and errors in prediction. This study introduces a novel Imputation model employing transformer-based architectures to address these challenges. Notably, the model distinguishes between complete EEG signal amplitude data and incomplete data in two datasets: PhysioNet and CHB-MIT. By training exclusively on complete amplitude data, the TabTransformer accurately learns and predicts missing values, capturing intricate patterns and relationships inherent in EEG amplitude data. Evaluation using various error metrics and R2 score demonstrates significant enhancements over traditional methods such as Zero, Mean, and KNN imputation. The Proposed Model achieves impressive R2 scores of 0.993 for PhysioNet and 0.97 for CHB-MIT, highlighting its efficacy in handling complex clinical data patterns and improving dataset integrity. This underscores the transformative potential of transformer models in advancing the utility and reliability of clinical datasets.

Machine Learning Techniques in Enhanced Oil Recovery Screening Using Semisupervised Label Propagation

Summary Efficiently choosing the optimal enhanced oil recovery (EOR) technique is a critical requirement in reservoir engineering. Machine learning (ML) methods, with a well-established history of application, serve as a swift and dependable tool for EOR screening. In this paper, we aim to evaluate the effectiveness of various ML algorithms for EOR screening, utilizing a comprehensive database of nearly 1,000 EOR projects. This study delves into a comprehensive evaluation of regression and classification-based algorithms to develop a reliable screening system for EOR predictions and address challenges such as limited labeled data and missing values. Our analysis considered various EOR processes, including gas injection, chemical, and thermal EOR techniques. Various ML methods such as random forest (RF), K-nearest neighbors (KNN), support vector machine (SVM), shallow artificial neural networks (SANN), naive Bayes classifier (NBC), logistic regression (LR), and decision tree (DT) are applied, enabling both intermethod comparisons and evaluations against advanced methods, multiobjective deep artificial neural networks (MDANN), and multiobjective artificial neural networks (MANN). These advanced techniques provide the unique capability to concurrently address both regression and classification tasks. Considering that conventional methods can only be implemented on a single task, the RF, MANN, MDANN, and KNN algorithms demonstrated top-tier performance in our classification analysis. Regarding the regression task, KNN, RF, and MDANN displayed exceptional performance, signifying their prowess in predictive accuracy. However, MANN exhibited moderate performance in regression analysis. In addition, our study identified areas where certain algorithms, such as support vector regression (SVR), exhibited weaker performance, highlighting the importance of comprehensive model evaluation. This paper contributes novel insights into the application of ML techniques for EOR screening in the petroleum industry. By addressing challenges such as limited labeled data and missing values and by providing a thorough evaluation of various ML algorithms, our study offers valuable information for decision-makers in the oil and gas sector, aiding in the selection of suitable algorithms for EOR projects. In addition, the use of semisupervised label propagation and advanced techniques like KNN imputation adds to the existing body of literature, enhancing the state of knowledge in this domain.

Enhancing Material Property Predictions through Optimized KNN Imputation and Deep Neural Network Modeling

In materials science, the integrity and completeness of datasets are critical for robust predictive modeling. Unfortunately, material datasets frequently contain missing values due to factors such as measurement errors, data non-availability, or experimental limitations, which can significantly undermine the accuracy of property predictions. To tackle this challenge, we introduce an optimized K-Nearest Neighbors (KNN) imputation method, augmented with Deep Neural Network (DNN) modeling, to enhance the accuracy of predicting material properties. Our study compares the performance of our Enhanced KNN method against traditional imputation techniques—mean imputation and Multiple Imputation by Chained Equations (MICE). The results indicate that our Enhanced KNN method achieves a superior R² score of 0.973, which represents a significant improvement of 0.227 over Mean imputation, 0.141 over MICE, and 0.044 over KNN imputation. This enhancement not only boosts the data integrity but also preserves the statistical characteristics essential for reliable predictions in materials science.

Performance of Different Imputation Methods in Logistic Regression with Multicollinearity

The existence of missing values in the data, which must be taken into account, is an important issue in research. In particular, in the case of medical data, missing data can lead to a loss of trust in patient evaluation and make it impossible to classify people according to their level of health or disease. Furthermore, multicollinearity between independent variables can lead to misleading results. Therefore, the present research aims to study the efficiency of missing data imputation techniques for logistic regression with complete multicollinearity. The missing data imputation methods considered in this research were MEAN (mean imputation), MI (multiple imputation), KNN (k-nearest neighbor imputation), RF (random forest imputation), SRI (stochastic regression imputation), and BRI (Bayesian linear regression imputation). In this study, the simulation conducted was done with sample sizes of 20, 50, 100, 150, 200, 500, and 1000, as well as percentages of missing data at 10, 20, 30, and 40%. To compare efficiency, the EMSE (estimated mean square error) was used. The results showed that the RF method was most effective when the sample size was large and with a high percentage of missing data, whereas the MEAN method had the worst performance in all cases. When the sample size is small with a high proportion of missing data, KNN might be a preferable alternative for imputation. The inquiry into expanding the number of independent variables and different patterns of multicollinearity may be essential for future work.

Analyzing Monthly Blood Test Data to Forecast 30-Day Hospital Readmissions among Maintenance Hemodialysis Patients.

Background: The increase in the global population of hemodialysis patients is linked to aging demographics and the prevalence of conditions such as arterial hypertension and diabetes mellitus. While previous research in hemodialysis has mainly focused on mortality predictions, there is a gap in studies targeting short-term hospitalization predictions using detailed, monthly blood test data. Methods: This study employs advanced data preprocessing and machine learning techniques to predict hospitalizations within a 30-day period among hemodialysis patients. Initial steps include employing K-Nearest Neighbor (KNN) imputation to address missing data and using the Synthesized Minority Oversampling Technique (SMOTE) to ensure data balance. The study then applies a Support Vector Machine (SVM) algorithm for the predictive analysis, with an additional enhancement through ensemble learning techniques, in order to improve prediction accuracy. Results: The application of SVM in predicting hospitalizations within a 30-day period among hemodialysis patients resulted in an impressive accuracy rate of 93%. This accuracy rate further improved to 96% upon incorporating ensemble learning methods, demonstrating the efficacy of the chosen machine learning approach in this context. Conclusions: This study highlights the potential of utilizing machine learning to predict hospital readmissions within a 30-day period among hemodialysis patients based on monthly blood test data. It represents a significant leap towards precision medicine and personalized healthcare for this patient group, suggesting a paradigm shift in patient care through the proactive identification of hospitalization risks.

Predicting major adverse cardiovascular events after orthotopic liver transplantation using a supervised machine learning model: A cohort study.

Liver transplant (LT) patients have become older and sicker. The rate of post-LT major adverse cardiovascular events (MACE) has increased, and this in turn raises 30-d post-LT mortality. Noninvasive cardiac stress testing loses accuracy when applied to pre-LT cirrhotic patients. To assess the feasibility and accuracy of a machine learning model used to predict post-LT MACE in a regional cohort. This retrospective cohort study involved 575 LT patients from a Southern Brazilian academic center. We developed a predictive model for post-LT MACE (defined as a composite outcome of stroke, new-onset heart failure, severe arrhythmia, and myocardial infarction) using the extreme gradient boosting (XGBoost) machine learning model. We addressed missing data (below 20%) for relevant variables using the k-nearest neighbor imputation method, calculating the mean from the ten nearest neighbors for each case. The modeling dataset included 83 features, encompassing patient and laboratory data, cirrhosis complications, and pre-LT cardiac assessments. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC). We also employed Shapley additive explanations (SHAP) to interpret feature impacts. The dataset was split into training (75%) and testing (25%) sets. Calibration was evaluated using the Brier score. We followed Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines for reporting. Scikit-learn and SHAP in Python 3 were used for all analyses. The supplementary material includes code for model development and a user-friendly online MACE prediction calculator. Of the 537 included patients, 23 (4.46%) developed in-hospital MACE, with a mean age at transplantation of 52.9 years. The majority, 66.1%, were male. The XGBoost model achieved an impressive AUROC of 0.89 during the training stage. This model exhibited accuracy, precision, recall, and F1-score values of 0.84, 0.85, 0.80, and 0.79, respectively. Calibration, as assessed by the Brier score, indicated excellent model calibration with a score of 0.07. Furthermore, SHAP values highlighted the significance of certain variables in predicting postoperative MACE, with negative noninvasive cardiac stress testing, use of nonselective beta-blockers, direct bilirubin levels, blood type O, and dynamic alterations on myocardial perfusion scintigraphy being the most influential factors at the cohort-wide level. These results highlight the predictive capability of our XGBoost model in assessing the risk of post-LT MACE, making it a valuable tool for clinical practice. Our study successfully assessed the feasibility and accuracy of the XGBoost machine learning model in predicting post-LT MACE, using both cardiovascular and hepatic variables. The model demonstrated impressive performance, aligning with literature findings, and exhibited excellent calibration. Notably, our cautious approach to prevent overfitting and data leakage suggests the stability of results when applied to prospective data, reinforcing the model's value as a reliable tool for predicting post-LT MACE in clinical practice.

EvoImp: Multiple Imputation of Multi-label Classification data with a genetic algorithm.

Missing data is a prevalent problem that requires attention, as most data analysis techniques are unable to handle it. This is particularly critical in Multi-Label Classification (MLC), where only a few studies have investigated missing data in this application domain. MLC differs from Single-Label Classification (SLC) by allowing an instance to be associated with multiple classes. Movie classification is a didactic example since it can be "drama" and "bibliography" simultaneously. One of the most usual missing data treatment methods is data imputation, which seeks plausible values to fill in the missing ones. In this scenario, we propose a novel imputation method based on a multi-objective genetic algorithm for optimizing multiple data imputations called Multiple Imputation of Multi-label Classification data with a genetic algorithm, or simply EvoImp. We applied the proposed method in multi-label learning and evaluated its performance using six synthetic databases, considering various missing values distribution scenarios. The method was compared with other state-of-the-art imputation strategies, such as K-Means Imputation (KMI) and weighted K-Nearest Neighbors Imputation (WKNNI). The results proved that the proposed method outperformed the baseline in all the scenarios by achieving the best evaluation measures considering the Exact Match, Accuracy, and Hamming Loss. The superior results were constant in different dataset domains and sizes, demonstrating the EvoImp robustness. Thus, EvoImp represents a feasible solution to missing data treatment for multi-label learning.

Privacy-Preserving Vertical Federated KNN Feature Imputation Method

Federated learning stands as a pivotal component in the construction of data infrastructure. It significantly fortifies the safety and reliability of data circulation links, facilitating credible sharing and openness among diverse subjects. The presence of missing data poses a pervasive and challenging issue in the implementation of federated learning. Current research on imputation missing values predominantly concentrates on centralized methods and horizontal federation scenarios. However, there is a notable absence of exploration in the context of vertical federated application scenarios. In this paper, the problem of missing imputation in vertical federated learning is investigated and a novel vertical federated k-nearest neighbors (KNN) imputation method is proposed. Extensive experiments are conducted using publicly available data sets to compare existing imputation methods, the results demonstrate the effectiveness and progress of our approach.

결측치를 포함한 데이터의 k-평균 군집분석 방법 비교

Cluster analysis is an unsupervised learning method to find heterogeneous clusters that capture similarities among items and separate different items into different clusters. Various cluster analysis techniques have been proposed, and the k-means clustering method, which minimizes the sum of Euclidean distances between cluster centroids and individual entities, is widely recognized as a standard cluster analysis method. When data include missing values, it is challenging to conduct cluster analysis, because it is impossible to calculate distances between centroids of clusters and incomplete items, resulting in excluding classification of these items. Techniques have been suggested to handle missing values in k-means clustering, including conducting cluster analysis after imputation of missing values or cluster analysis based on available information. In this study, we explore methods to perform k-means cluster analysis on data with missing values and evaluate performance of these methods using a simulation. The results of simulation studies indicate that conducting k-means cluster analysis after imputation yields the better performance than the one based on available information. Among the various imputation methods, k-nearest neighbors imputation performed the best.

K-Means Clustering with KNN and Mean Imputation on CPU Benchmark Compilation Data

In the rapidly evolving digital age, data is becoming a valuable source for decision-making and analysis. Clustering, as an important technique in data analysis, has a key role in organizing and understanding complex datasets. One of the effective clustering algorithms is k-means. However, this algorithm is prone to the problem of missing values, which can significantly affect the quality of the resulting clusters. To overcome this challenge, imputation methods are used, including mean imputation and K-Nearest Neighbor (KNN) imputation. This study aims to analyze the impact of imputation methods on CPU Benchmark Compilation clustering results. Evaluation of the clustering results using the silhouette coefficient showed that clustering with mean imputation achieved a score of 0.782, while with KNN imputation it achieved a score of 0.777. In addition, the cluster interpretation results show that the KNN method produces more information that is easier for users to understand. This research provides valuable insights into the effectiveness of imputation methods in improving the quality of data clustering results in assisting CPU selection decisions on CPU Benchmark Compilation data.

Application of Imputation Method for Compositional Data with Missing Values based on Adaptive LASSO Model: the Composition of Employment Industry in Taiyuan, China

The tripartite industry classification, which divides all economic activities into three parts, is a classification method to reflect the dynamic process of economic development and the historical trend of the change of resource allocation structure.The fact shows that the proportion of each industry has become an important symbol of the level of national economic development. The proportion of each industry is compositional data,which is a kind of complex multidimensional data used in many fields. All components in the compositional data are non-negative and carry only relative information. In practice, there could be missing values in compositional data. However, general statistical analysis methods cannot be firstly used for compositional data with missing values. The complexity of the missing value of compositional data makes traditional imputation methods no longer suitable. Thus, how to carry out effective statistical inference for compositional data with missing values attracts the attention of many scholars, recently. In this paper, we focus on the imputation problem in compositional data containing missing values, and propose an Adaptive Least Absolute Shrinkage and Selection Operator (ALASSO) imputation method to obtain a complete datasets through variable selection and parameter estimation. Then, the new method is simulated and empirically analyzed, and a comparative study with mean imputation, k-nearest neighbor imputation, and iterative regression imputation is conducted. The results show that the ALASSO imputation method has the highest accuracy for different missing rates, dimensions and correlation coefficients.

Cardiac disease risk prediction using machine learning algorithms.

Heart attack is a life-threatening condition which is mostly caused due to coronary disease resulting in death in human beings. Detecting the risk of heart diseases is one of the most important problems in medical science that can be prevented and treated with early detection and appropriate medical management; it can also help to predict a large number of medical needs and reduce expenses for treatment. Predicting the occurrence of heart diseases by machine learning (ML) algorithms has become significant work in healthcare industry. This study aims to create a such system that is used for predicting whether a patient is likely to develop heart attacks, by analysing various data sources including electronic health records and clinical diagnosis reports from hospital clinics. ML is used as a process in which computers learn from data in order to make predictions about new datasets. The algorithms created for predictive data analysis are often used for commercial purposes. This paper presents an overview to forecast the likelihood of a heart attack for which many ML methodologies and techniques are applied. In order to improve medical diagnosis, the paper compares various algorithms such as Random Forest, Regression models, K-nearest neighbour imputation (KNN), Naïve Bayes algorithm etc. It is found that the Random Forest algorithm provides a better accuracy of 88.52% in forecasting heart attack risk, which could herald a revolution in the diagnosis and treatment of cardiovascular illnesses.

Missing data imputation, prediction, and feature selection in diagnosis of vaginal prolapse

BackgroundData loss often occurs in the collection of clinical data. Directly discarding the incomplete sample may lead to low accuracy of medical diagnosis. A suitable data imputation method can help researchers make better use of valuable medical data.MethodsIn this paper, five popular imputation methods including mean imputation, expectation-maximization (EM) imputation, K-nearest neighbors (KNN) imputation, denoising autoencoders (DAE) and generative adversarial imputation nets (GAIN) are employed on an incomplete clinical data with 28,274 cases for vaginal prolapse prediction. A comprehensive comparison study for the performance of these methods has been conducted through certain classification criteria. It is shown that the prediction accuracy can be greatly improved by using the imputed data, especially by GAIN. To find out the important risk factors to this disease among a large number of candidate features, three variable selection methods: the least absolute shrinkage and selection operator (LASSO), the smoothly clipped absolute deviation (SCAD) and the broken adaptive ridge (BAR) are implemented in logistic regression for feature selection on the imputed datasets. In pursuit of our primary objective, which is accurate diagnosis, we employed diagnostic accuracy (classification accuracy) as a pivotal metric to assess both imputation and feature selection techniques. This assessment encompassed seven classifiers (logistic regression (LR) classifier, random forest (RF) classifier, support machine classifier (SVC), extreme gradient boosting (XGBoost) , LASSO classifier, SCAD classifier and Elastic Net classifier)enhancing the comprehensiveness of our evaluation.ResultsThe proposed framework imputation-variable selection-prediction is quite suitable to the collected vaginal prolapse datasets. It is observed that the original dataset is well imputed by GAIN first, and then 9 most significant features were selected using BAR from the original 67 features in GAIN imputed dataset, with only negligible loss in model prediction. BAR is superior to the other two variable selection methods in our tests.ConcludesOverall, combining the imputation, classification and variable selection, we achieve good interpretability while maintaining high accuracy in computer-aided medical diagnosis.

Handling Missing Data in COVID-19 Incidence Estimation: Secondary Data Analysis.

The COVID-19 pandemic has revealed significant challenges in disease forecasting and in developing a public health response, emphasizing the need to manage missing data from various sources in making accurate forecasts. We aimed to show how handling missing data can affect estimates of the COVID-19 incidence rate (CIR) in different pandemic situations. This study used data from the COVID-19/SARS-CoV-2 surveillance system at the National Institute of Hygiene and Epidemiology, Vietnam. We separated the available data set into 3 distinct periods: zero COVID-19, transition, and new normal. We randomly removed 5% to 30% of data that were missing completely at random, with a break of 5% at each time point in the variable daily caseload of COVID-19. We selected 7 analytical methods to assess the effects of handling missing data and calculated statistical and epidemiological indices to measure the effectiveness of each method. Our study examined missing data imputation performance across 3 study time periods: zero COVID-19 (n=3149), transition (n=1290), and new normal (n=9288). Imputation analyses showed that K-nearest neighbor (KNN) had the lowest mean absolute percentage change (APC) in CIR across the range (5% to 30%) of missing data. For instance, with 15% missing data, KNN resulted in 10.6%, 10.6%, and 9.7% average bias across the zero COVID-19, transition, and new normal periods, compared to 39.9%, 51.9%, and 289.7% with the maximum likelihood method. The autoregressive integrated moving average model showed the greatest mean APC in the mean number of confirmed cases of COVID-19 during each COVID-19 containment cycle (CCC) when we imputed the missing data in the zero COVID-19 period, rising from 226.3% at the 5% missing level to 6955.7% at the 30% missing level. Imputing missing data with median imputation methods had the lowest bias in the average number of confirmed cases in each CCC at all levels of missing data. In detail, in the 20% missing scenario, while median imputation had an average bias of 16.3% for confirmed cases in each CCC, which was lower than the KNN figure, maximum likelihood imputation showed a bias on average of 92.4% for confirmed cases in each CCC, which was the highest figure. During the new normal period in the 25% and 30% missing data scenarios, KNN imputation had average biases for CIR and confirmed cases in each CCC ranging from 21% to 32% for both, while maximum likelihood and moving average imputation showed biases on average above 250% for both CIR and confirmed cases in each CCC. Our study emphasizes the importance of understanding that the specific imputation method used by investigators should be tailored to the specific epidemiological context and data collection environment to ensure reliable estimates of the CIR.

Data Analytics Enables Reconstruction of Missing Segments in Well Data History

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 208137, “Reconstruction of Missing Segments in Well Data History Using Data Analytics,” by Yuanjun Li, SPE, and Roland Horne, SPE, Stanford University, and Ahmed Alshmakhy, SPE, ADNOC, et al. The paper has not been peer reviewed. _ The problem of missing data is a common one in well-production records. An incomplete data set is commonly simplified by omitting all observations with missing values, which can lead to significant information loss. In the complete paper, the authors developed an efficient procedure that enables fast reconstruction of the entire production data set with multiple missing sections in different variables. Ultimately, the complete information can support the reservoir history-matching process and production allocation and can develop models for reservoir performance prediction. Introduction Missing values are a prevalent issue in pressure transient analysis. Their occurrence is often unavoidable and is a serious problem because many data-mining algorithms cannot work with data sets that are missing values. Sources of missing field data include operational problems, changes in operating conditions, and variable sampling frequencies. Imputation Technique of Missing Values Case Deletion. If a missing data problem exists in a multivariate series, the time series may not be evaluated appropriately. In time-series data, each record is unique. Thus, deleting a record would result in a series with gaps, which would be unusable for many analyses. Furthermore, the time-series plot would be truncated and riddled with flaws. Thus, to conduct a time-series analysis, those missing observations must be estimated and reinserted. When the number of missing values in the data set is considerable, eliminating the associated cases might result in the loss of vital and valuable information. Researchers have found that case elimination is not appropriate when the missing rate is greater than 3%. Physics-Based Imputation. Any physics-driven diagnostic or model is based on fundamental assumptions that may or may not be correct in all situations but are intended to represent reservoir behavior. While data-driven models do not make any assumptions about underlying physics, they rely on training data. Therefore, data analytics and machine learning can be an ideal substitute. Local Statistical Approach. Local statistical methods include the K-nearest-neighbor (KNN) imputation and local-least-square imputation, two of the most well-known algorithms. To impute the incomplete data, the KNN imputation employs a pairwise correlation between the target gene with missing values and the K closest reference genes. For missing-value imputation, the techniques in this local method use local similarity patterns in the data sets or a specific column of data.

Adverse maternal outcomes associated with major fetal malformations after singleton live birth

Major fetal malformations complicate 2% to 5% of live births. It is unclear what effect fetal malformations have on severe maternal morbidity. This study aimed to compare maternal outcomes between individuals with a fetus with major or minor fetal malformations and those with a fetus without major or minor fetal malformations. This was a secondary analysis of the Consortium on Safe Labor database. Our study was limited to the current analysis of pregnant individuals with a singleton live birth. Major fetal malformations based on the Centers for Disease Control and Prevention's criteria were defined. Fetal malformations that did not meet the criteria for major fetal malformations were categorized as minor fetal malformations. Our primary maternal outcome was severe maternal morbidity as defined by the Centers for Disease Control and Prevention. Missing values were imputed by multiple imputation using the k-nearest neighbor imputation method. Poisson regression with robust error variance was used to obtain adjusted relative risks with 95% confidence intervals, controlling for confounders. Of 216,881 deliveries, there were 201,860 cases (93.1%) with no congenital malformation, 12,106 cases (5.6%) with minor fetal malformations, and 2845 cases (1.3%) with major fetal malformations. Compared with individuals with no fetal malformation, those with major fetal malformations were more likely to have severe maternal morbidity (0.7% vs 1.2%; adjusted relative risk, 1.51; 95% confidence interval, 1.07-2.12), postpartum hemorrhage (3.6% vs 6.9%; adjusted relative risk, 1.76; 95% confidence interval, 1.50-2.06), preeclampsia (5.1% vs 8.3%; adjusted relative risk, 1.48; 95% confidence interval, 1.31-1.67), and cesarean delivery (26.7% vs 42.3%; adjusted relative risk, 1.51; 95% confidence interval, 1.45-1.58). Compared with individuals with no fetal malformation, those with minor fetal malformations were more likely to have severe maternal morbidity (0.7% vs 1.4%; adjusted relative risk, 1.73; 95% confidence interval, 1.48-2.02), maternal death (0.01% vs 0.03%; adjusted relative risk, 4.50; 95% confidence interval, 1.18-17.19), postpartum hemorrhage (3.6% vs 6.1%; adjusted relative risk, 1.54; 95% confidence interval, 1.41-1.68), preeclampsia (5.1% vs 8.6%; adjusted relative risk, 1.50; 95% confidence interval, 1.41-1.60), superimposed preeclampsia (1.2% vs 2.4%; adjusted relative risk, 1.25; 95% confidence interval, 1.14-1.38), cesarean delivery (26.7% vs 39.6%; adjusted relative risk, 1.38; 95% confidence interval, 1.35-1.41), chorioamnionitis (3.0% vs 4.7%; adjusted relative risk, 1.41; 95% confidence interval, 1.29-1.53), and postpartum endometritis (0.6% vs 1.0%; adjusted relative risk, 1.58; 95% confidence interval, 1.31-1.90). Major and minor congenital fetal malformations are independent risk factors for severe maternal morbidity and other pregnancy complications.