Value Imputation Research Articles

Evaluation of Neonatal and Paediatric Vancomycin Pharmacokinetic Models and the Impact of Maturation and Serum Creatinine Covariates in a Large Multicentre Data Set.

Infants and neonates present a clinical challenge for dosing drugs with high interindividual variability due to these patients' rapid growth and the interplay between maturation and organ function. Model-informed precision dosing (MIPD), which can account for interindividual variability via patient characteristics and Bayesian forecasting, promises to improve individualized dosing strategies in this complex population. Here, we assess the predictive performance of published population pharmacokinetic models describing vancomycin in neonates and infants, and analyze the robustness of these models in the face of clinical uncertainty surrounding covariate values. The predictive precision and bias of nine pharmacokinetic models were compared in a large multi-site data set (N = 2061 patients, 5794 drug levels, 28 institutions) of patients aged 0-365 days. The robustness of model predictions to errors in serum creatinine measurements and gestational age was assessed by using recorded values or by replacing covariate values with 0.3, 0.5 or 0.8 mg/dL or with 40 weeks, respectively. Of the nine models, two models (Dao and Jacqz-Aigrain) resulted in predicted concentrations within 2.5 mg/L or 15% of the measured values for at least 60% of population predictions. Within individual models, predictive performance often 2 differed in neonates (0-4 weeks) versus older infants (15-52 weeks). For preterm neonates, imputing gestational age as 40 weeks reduced the accuracy of model predictions. Measured values of serum creatinine improved model predictions compared to using imputed values even in neonates ≤1 week of age. Several available pharmacokinetic models are suitable for MIPD in infants and neonates. Availability and accuracy of model covariates for patients will be important for guiding dose decision-making.

Multimorbidity, disease count, mortality and emergency care use in persons attending the emergency department: a cross-sectional data-linkage study.

Multimorbidity (two or more concurrent chronic conditions) is associated with poorer health outcomes and increased healthcare utilisation in primary care and general populations. Less is known about the prevalence of multimorbidity in emergency department attenders, or its association with poor outcomes in this population. This study sought to explore the relationship between multimorbidity, mortality and health-care utilisation in a large urban cohort of persons attending emergency departments. Validated algorithms for the identification of 28 chronic conditions from ICD-10 codes were deployed on a cross-sectional sample of patients attending emergency departments in Glasgow, Scotland between April 2019 and March 2020. Analysis was conducted on complete cases (n=63,328) and compared with results from data with imputed missing values (n=75,723). Models adjusted for age, sex, deprivation and ethnicity were fitted to test for the association between (i) multimorbidity, (ii) complex multimorbidity, (iii) disease count and the following outcomes: admission to hospital, reattendance at 30 and 90 days, and death during admission. Multimorbidity, complex multimorbidity and disease count were significantly associated with hospital admission and emergency department reattendance. Those with 1-3 conditions were at increased risk of inpatient mortality. This study further evidences the impact of multimorbidity and disease burden on health-care use, and mortality to a lesser extent. Deployed algorithms were sufficiently sensitive to detect associations, despite limited access (21 months) to secondary-care data. This should allow for the construction of more robust models to prospectively identify persons at risk of poor outcomes in similar populations.

Item non-response imputation in the Korea National Health and Nutrition Examination Survey.

The Korea National Health and Nutrition Examination Survey (KNHANES) is a public health survey that assesses individuals' health and nutritional status and monitors the prevalence of major chronic diseases. In general, sampling weights are adjusted for unit non-responses and imputation is conducted for item non-responses. In this study, we proposed strategies for imputing item non-responses in the KNHANES in order to improve the usefulness of data, minimize bias, and increase statistical power. After applying logical imputation, we adopted 2 separate imputation methods for each variable type: unweighted sequential hot-deck imputation for categorical variables and sequential regression imputation for continuous variables. For variance estimation, multiple imputations were applied to the continuous variables. To evaluate the performance of the proposed strategies, we compared the marginal distributions of imputed variables and the results of multivariable regression analysis for the complete-case data and the expanded data with imputed values, respectively. When comparing the marginal distributions, most non-responses were imputed. The multivariable regression coefficients presented similar estimates; however, the standard errors decreased, resulting in statistically significant p-values. The proposed imputation strategies may cope with the loss of precision due to missing data, thus enhancing statistical power in analyses of the KNHANES by providing expanded data with imputed values. The proposed imputation strategy may enhance the utility of data by increasing the number of complete cases and reducing the bias in the analysis, thus laying a foundation to cope with the occurrence of item non-responses in further surveys.

A novel model-driven deterministic approach to wind power imputation

Wind data play a crucial role in the advanced planning, design, and maintenance of wind farms, scheduling, energy production, and accurate distribution. Most of the measured wind dataset comprises missing observations due to several reasons, including the failure of measuring devices, human mistakes, environmental impact, and natural calamities. Completing the missing observations in a wind dataset without losing significant information is still a challenging task currently. In the present research, a novel deterministic approach to wind power data imputation based on the wind speed-power model is proposed and validated using the 15 western wind datasets (capacity of 25–40 MW). The validation outcomes result in the minimum value of the root mean square error for the imputed value of the rated and score-lite wind power which exhibits the effectiveness of the proposed method.

Multiple Imputation to Salvage Partial Respondents: Analysis of the Forgotten Joint Score-12 After Total Hip Arthroplasty.

Missing responses are common when Asian patients complete the Forgotten Joint Score-12 (FJS-12), which is widely used to evaluate total hip arthroplasty (THA). We aimed to provide orthopaedic researchers with a solution for handling missing values in such patient-reported outcome measures (PROMs). Patients who had undergone primary THA between 1998 and 2016 (n = 1,021) were investigated in 2020. The FJS-12 and 9 other PROMs, including questions related to Asian lifestyle activities, were administered. Risk factors for missing FJS-12 items were investigated. Partial respondents were matched with complete respondents; then, in each pair, the items not completed by the partial respondent were deleted from the responses of the complete respondent. Predictive mean matching (PMM) was performed in an attempt to recover the deleted items, using 65 sets of imputation models. After the missing values had been imputed, we explored patient characteristics that affected the FJS-12, using data from all complete and partial respondents. A total of 652 patients responded to the survey (393 complete and 193 partial respondents). Partial respondents were older, more often female, and less active. Older respondents were more likely to skip items involving the bed, while those who reported a better ability to sit in the seiza style (traditional Japanese floor sitting) were more likely to skip items about chair sitting. The imputed FJS-12 value exhibited excellent reliability (intraclass correlation coefficient for agreement with the true scores, 0.985). FJS-12 values of complete respondents were significantly higher than those of respondents with 4 to 11 missing items (51.6 versus 32.8, p < 0.001). Older age was associated with higher FJS-12 values, which was revealed only via analysis of the multiply imputed data sets (p < 0.001). Analysis of only complete FJS-12 responses after THA resulted in a nonresponse bias, preferentially excluding older, female, and less active individuals and those with a traditional floor living style. Multiple imputation could provide a solution to scoring and analyzing PROMs with missing responses by permitting the inclusion of partial respondents. Therapeutic Level III . See Instructions for Authors for a complete description of levels of evidence.

Modeling air conditioning ownership and availability

In recent years, extreme heat amplified the need for indoor cooling systems. Some local governments started to provide household cooling systems for vulnerable people. However, the lack of local AC prevalence makes equitable AC dissemination more difficult. Therefore, this study developed neighborhood-level AC prevalence information for California (CA), US Properties were on the market from 2018 to 2021 with a unique real estate dataset from Zillow Transaction and Assessment Dataset (ZTRAX). The analysis selected housing attributes with theoretical or empirical linkages to AC access. Data preprocessing imputed missing values with a random forest (RF) analysis. Next, a subsequent multi-class RF estimated the types of household AC ownership (central, other, yes, and none). The RF model showed an overall accuracy of 98% (class-specific accuracies for Central: 98%, None: 96%, Other: 95%, Yes: 99%). Jackknifing revealed the latitude, longitude, heating type or system, elevation, built year, cooling degree days, and building quality exhibited the highest Gini importance values. This study visualized AC prevalence in CA and four counties with major cities: Los Angeles, San Diego, Sacramento, and San Francisco County. This result can be applied to implementing heat prevention measures such as providing household cooling systems, energy subsidies, cooling centers, and increased green space access.

Disease classification for whole-blood DNA methylation: Meta-analysis, missing values imputation, and XAI.

DNA methylation has a significant effect on gene expression and can be associated with various diseases. Meta-analysis of available DNA methylation datasets requires development of a specific workflow for joint data processing. We propose a comprehensive approach of combined DNA methylation datasets to classify controls and patients. The solution includes data harmonization, construction of machine learning classification models, dimensionality reduction of models, imputation of missing values, and explanation of model predictions by explainable artificial intelligence (XAI) algorithms. We show that harmonization can improve classification accuracy by up to 20% when preprocessing methods of the training and test datasets are different. The best accuracy results were obtained with tree ensembles, reaching above 95% for Parkinson's disease. Dimensionality reduction can substantially decrease the number of features, without detriment to the classification accuracy. The best imputation methods achieve almost the same classification accuracy for data with missing values as for the original data. XAI approaches have allowed us to explain model predictions from both populational and individual perspectives. We propose a methodologically valid and comprehensive approach to the classification of healthy individuals and patients with various diseases based on whole-blood DNA methylation data using Parkinson's disease and schizophrenia as examples. The proposed algorithm works better for the former pathology, characterized by a complex set of symptoms. It allows to solve data harmonization problems for meta-analysis of many different datasets, impute missing values, and build classification models of small dimensionality.

To Adjust or not to Adjust? Estimating the Average Treatment Effect in Randomized Experiments with Missing Covariates

Randomized experiments allow for consistent estimation of the average treatment effect based on the difference in mean outcomes without strong modeling assumptions. Appropriate use of pretreatment covariates can further improve the estimation efficiency. Missingness in covariates is nevertheless common in practice, and raises an important question: should we adjust for covariates subject to missingness, and if so, how? The unadjusted difference in means is always unbiased. The complete-covariate analysis adjusts for all completely observed covariates, and is asymptotically more efficient than the difference in means if at least one completely observed covariate is predictive of the outcome. Then what is the additional gain of adjusting for covariates subject to missingness? To reconcile the conflicting recommendations in the literature, we analyze and compare five strategies for handling missing covariates in randomized experiments under the design-based framework, and recommend the missingness-indicator method, as a known but not so popular strategy in the literature, due to its multiple advantages. First, it removes the dependence of the regression-adjusted estimators on the imputed values for the missing covariates. Second, it does not require modeling the missingness mechanism, and yields consistent estimators even when the missingness mechanism is related to the missing covariates and unobservable potential outcomes. Third, it ensures large-sample efficiency over the complete-covariate analysis and the analysis based on only the imputed covariates. Lastly, it is easy to implement via least squares. We also propose modifications to it based on asymptotic and finite sample considerations. Importantly, our theory views randomization as the basis for inference, and does not impose any modeling assumptions on the data-generating process or missingness mechanism. Supplementary materials for this article are available online.

Integrated Inventory Transshipment and Missing-Data Treatment Using Improved Imputation-Level Adjustment for Efficient Cross-Filling

This research investigates an integrated problem of transshipment for cross-filling and imputation for missing demand data. Transshipment for cross-filling has proved effective in mitigating shortages with relatively low inventory, thus reducing resource consumption in inventory management. Although accurate demand data are critical for cross-filling decision making, some demand data are inevitably incomplete. These missing data should be treated for effective transshipment operations. Despite the importance, these missing data issues have not been adequately studied for transshipment problems. This paper addresses how transshipment can be conducted under missing demand conditions. A novel integrated problem is established to combine demand-data imputation processes and transshipment decisions. Imputation strategies and new algorithms suitable for transshipment are developed to handle missing demand data. Diverse demand and transshipment cases are analyzed for cost-effectiveness. The analysis uncovers that conventional straightforward imputation methods result in inferior transshipment decisions. The study also reveals that imputed values should be adjusted to appropriate levels for transshipment to be effective. The strong interplay between imputation processes and shortage prevention is also discovered for transshipment with missing demand. This study demonstrates how inventory transshipment can be carried out successfully with appropriate treatment of missing demand data in practice.

Self-organizing maps for exploration of partially observed data and imputation of missing values

The self-organizing map is an unsupervised neural network which is widely used for data visualization and clustering in the field of chemometrics. The classical Kohonen algorithm that computes self-organizing maps is suitable only for complete data without any missing values. However, in many applications, partially observed data are the norm. In this paper, we propose an extension of self-organizing maps to incomplete data via a new criterion that also defines estimators of the missing values. In addition, an adaptation of the Kohonen algorithm, named missSOM, is provided to compute these self-organizing maps and impute missing values. An efficient implementation is provided. Numerical experiments on simulated data and a chemical dataset illustrate the short computing time of missSOM and assess its performance regarding various criteria and in comparison to the state of the art.

Handling missing data in clinical research

Because missing data are present in almost every study, it is important to handle missing data properly. First of all, the missing data mechanism should be considered. Missing data can be either completely at random (MCAR), at random (MAR), or not at random (MNAR). When missing data are MCAR, a complete case analysis can be valid. Also when missing data are MAR, in some situations a complete case analysis leads to valid results. However, in most situations, missing data imputation should be used. Regarding imputation methods, it is highly advised to use multiple imputations because multiple imputations lead to valid estimates including the uncertainty about the imputed values. When missing data are MNAR, also multiple imputations do not lead to valid results. A complication hereby is that it not possible to distinguish whether missing data are MAR or MNAR. Finally, it should be realized that preventing to have missing data is always better than the treatment of missing data.

An efficient approach for textual data classification using deep learning.

Text categorization is an effective activity that can be accomplished using a variety of classification algorithms. In machine learning, the classifier is built by learning the features of categories from a set of preset training data. Similarly, deep learning offers enormous benefits for text classification since they execute highly accurately with lower-level engineering and processing. This paper employs machine and deep learning techniques to classify textual data. Textual data contains much useless information that must be pre-processed. We clean the data, impute missing values, and eliminate the repeated columns. Next, we employ machine learning algorithms: logistic regression, random forest, K-nearest neighbors (KNN), and deep learning algorithms: long short-term memory (LSTM), artificial neural network (ANN), and gated recurrent unit (GRU) for classification. Results reveal that LSTM achieves 92% accuracy outperforming all other model and baseline studies.

Missing data estimates related to soybean production in the state of Mato Grosso, Brazil, from 1990 to 2018

PurposeThe state of Mato Grosso represents the largest producer and exporter of soybeans in Brazil; given this importance, it was aimed to propose to use the univariate imputation tool for time series, through applications of splines interpolations, in 46 of its municipalities that had missing data in the variables soybean production in thousand tons, production value and soy derivatives in R$ thousand, and also to assess the differences between the observed series and those with imputed values, in each of these municipalities, in these variables.Design/methodology/approachThe proposed methodology was based on the use of the univariate imputation method through the application of cubic spline interpolation in each of the 46 municipalities, for each of the 3 variables. Then, for each municipality, the original series were compared with each observed series plus the values imputed in these variables by the Quenouille test of correlation of time series.FindingsIt was observed that, after imputation, all series were compared with those observed and are equal by the Queinouille test in the 46 municipalities analyzed, and the Wilcoxon test also showed equality for the accumulated total of the three variables involved with the production of soybeans. And there were increases of 5.92%, 3.58% and 2.84% for soy production, soy production value and soy derivatives value accumulated in the state after imputation in the 46 municipalities.Originality/valueThe present research and its results facilitate the process of estimates and monitoring the total soy production in the state of Mato Grosso and its municipalities from 1990 to 2018.

Sequential Multiple Imputation for Real-World Health-Related Quality of Life Missing Data after Bariatric Surgery.

One of the main challenges for the successful implementation of health-related quality of life (HRQoL) assessments is missing data. The current study examined the feasibility and validity of a sequential multiple imputation (MI) method to deal with missing values in the longitudinal HRQoL data from the Scandinavian Obesity Surgery Registry. All patients in the SOReg who received bariatric surgery between 1 January 2011 and 31 March 2019 (n = 47,653) were included for the descriptive analysis and missingness pattern exploration. The patients who had completed the short-form 36 (SF-36) at baseline (year 0), and one-, two-, and five-year follow-ups were included (n = 3957) for the missingness pattern simulation and the sequential MI analysis. Eleven items of the SF-36 were selected to create the six domains of SF-6D, and the SF-6D utility index of each patient was calculated accordingly. The multiply-imputed variables in previous year were used as input to impute the missing values in later years. The performance of the sequential MI was evaluated by comparing the actual values with the imputed values of the selected SF-36 items and index at all four time points. At the baseline and year 1, where missing proportions were about 20% and 40%, respectively, there were no statistically significant discrepancies between the distributions of the actual and imputed responses (all p-values > 0.05). In year 2, where the missing proportion was about 60%, distributions of the actual and imputed responses were consistent in 9 of the 11 SF-36 items. However, in year 5, where the missing proportion was about 80%, no consistency was found between the actual and imputed responses in any of the SF-36 items. Relatively high missing proportions in HRQoL data are common in clinical registries, which brings a challenge to analyzing the HRQoL of longitudinal cohorts. The experimental sequential multiple imputation method adopted in the current study might be an ideal strategy for handling missing data (even though the follow-up survey had a missing proportion of 60%), avoiding significant information waste in the multivariate analysis. However, the imputations for data with higher missing proportions warrant more research.

A comparison of missing values imputation methods applied to precipitation of two semi-arid and humid regions of México

Climatological data with unreliable or missing values is an important area of research, and multiple methods are available to fill in missing data and evaluate data quality. Our study aims to compare the performance of different methods for estimating missing values that are explicitly designed for precipitation and multipurpose hydrological data. The climate variable used for the analysis was daily precipitation. We considered two different climate and orographic regions to evaluate the effects of altitude, precipitation regime and percentage of missing data on the Mean Absolute Error of imputed values and using a homogeneity evaluation of meteorological stations. We excluded from the analysis meteorological stations with more than 25% missing data. In the semi-arid region, ReddPrec (optimal for 9 stations), and GCIDW (optimal for 8) were the best performing methods for the 23 stations, with average MAE values of 1.63 mm/day and 1.46 mm/day, respectively. In the humid region, GCIDW was optimal in ~59% of stations, EM in ~24%, and ReddPrec in ~17%, with average MAE values of ~6.0 mm/day, 6.5 mm/day and ~9.8 mm/day, respectively. This research makes an important contribution to identifying the most appropriate methods to impute daily precipitation in different climatic regions of Mexico based on efficiency indicators and homogeneity evaluation.

Параллельный алгоритм восстановления сенсорных данных в режиме реального времени для многоядерного процессора

Currently, in many subject areas, the processing of sensor data in real time assumes imputation of values missed due to a technical failure or a human factor. This article proposes a parallel algorithm for imputation the missing values of a streaming time series in real time for a many-core processor. The algorithm employs a set of reference time series that have a semantic relationship with the original time series. The algorithm exploits the following heuristics: if there are repeated (similar) subsequences in the reference time series, then in the time series containing the missing value, repeated subsequences occur in the same time intervals. For each reference time series, a query is defined as a subsequence of a given length ending at the moment when the value in the original time series was missed. The similarity of the subsequences with the query is determined based on the DTW (Dynamic Time Warping) measure that is of quadratic computational complexity relative to the subsequence length. The algorithm employs the lower bounding technique to discard subsequences that are obviously dissimilar to the query, without calculating DTW. The lower bounds have less complexity than DTW and are calculated in parallel. The imputed value is calculated as the arithmetic mean of the last elements of the found intervals. In computational experiments, the proposed algorithm demonstrates high imputation accuracy in comparison with analogs and performance acceptable for real-time applications.

Partial Multiple Imputation With Variational Autoencoders: Tackling Not at Randomness in Healthcare Data.

Missing data can pose severe consequences in critical contexts, such as clinical research based on routinely collected healthcare data. This issue is usually handled with imputation strategies, but these tend to produce poor and biased results under the Missing Not At Random (MNAR) mechanism. A recent trend that has been showing promising results for MNAR is the use of generative models, particularly Variational Autoencoders. However, they have a limitation: the imputed values are the result of a single sample, which can be biased. To tackle it, an extension to the Variational Autoencoder that uses a partial multiple imputation procedure is introduced in this work. The proposed method was compared to 8 state-of-the-art imputation strategies, in an experimental setup with 34 datasets from the medical context, injected with the MNAR mechanism (10% to 80% rates). The results were evaluated through the Mean Absolute Error, with the new method being the overall best in 71% of the datasets, significantly outperforming the remaining ones, particularly for high missing rates. Finally, a case study of a classification task with heart failure data was also conducted, where this method induced improvements in 50% of the classifiers.

Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series

Multivariate time-series data are frequently observed in critical care settings and are typically characterized by sparsity (missing information) and irregular time intervals. Existing approaches for learning representations in this domain handle these challenges by either aggregation or imputation of values, which in-turn suppresses the fine-grained information and adds undesirable noise/overhead into the machine learning model. To tackle this problem, we propose a S elf-supervised Tra nsformer for T ime- S eries (STraTS) model, which overcomes these pitfalls by treating time-series as a set of observation triplets instead of using the standard dense matrix representation. It employs a novel Continuous Value Embedding technique to encode continuous time and variable values without the need for discretization. It is composed of a Transformer component with multi-head attention layers, which enable it to learn contextual triplet embeddings while avoiding the problems of recurrence and vanishing gradients that occur in recurrent architectures. In addition, to tackle the problem of limited availability of labeled data (which is typically observed in many healthcare applications), STraTS utilizes self-supervision by leveraging unlabeled data to learn better representations by using time-series forecasting as an auxiliary proxy task. Experiments on real-world multivariate clinical time-series benchmark datasets demonstrate that STraTS has better prediction performance than state-of-the-art methods for mortality prediction, especially when labeled data is limited. Finally, we also present an interpretable version of STraTS, which can identify important measurements in the time-series data. Our data preprocessing and model implementation codes are available at https://github.com/sindhura97/STraTS .

Combining attention with spectrum to handle missing values on time series data without imputation

In the development of predictive models, the problem of missing data is a critical issue that traditionally requires a two-step analysis. Data scientists analyze the patterns of missing values, select variables, impute missing values on the basis of domain knowledge, and then train a model. Models typically have their input sizes hardcoded, and have limitations in handling data with high missing rates or changes in available variables. We propose an attention-based neural network combined with a novel real number representation, which requires little work on manually selecting variables, and in which missing data can be overlooked, making imputation unnecessary. In this proposed model, data analysis can be one step, omitting the first step of imputing missing values. The study included data on 32,709 intensive care unit (ICU) admissions and 60 healthcare variables from the Medical Information Mart for Intensive Care (MIMIC)-IV. The proposed algorithm yielded an area under the receiver operating characteristic curve (AUC) of 0.842 (95% CIs: 0.828–0.856) when predicting prolonged length of stay in the ICU, outperforming current approaches using imputation methods. The proposed algorithm can be applied to a range of problems in data science, as it addresses the issue of incomplete data with automatic variable selection.

Implications of missingness in self-reported data for estimating racial and ethnic disparities in Medicaid quality measures.

To assess the feasibility and implications of imputing race and ethnicity for quality and utilization measurement in Medicaid. 2017 Oregon Medicaid claims from the Oregon Health Authority and electronic health records (EHR) from OCHIN, a clinical data research network, were used. We cross-sectionally assessed Hispanic-White, Black-White, and Asian-White disparities in 22 quality and utilization measures, comparing self-reported race and ethnicity to imputed values from the Bayesian Improved Surname Geocoding (BISG) algorithm. Race and ethnicity were obtained from self-reported data and imputed using BISG. 42.5%/4.9% of claims/EHR were missing self-reported data; BISG estimates were available for >99% of each and had good concordance (0.87-0.95) with Asian, Black, Hispanic, and White self-report. All estimated racial and ethnic disparities were statistically similar in self-reported and imputed EHR-based measures. However, within claims, BISG estimates and incomplete self-reported data yielded substantially different disparities in almost half of the measures, with BISG-based Black-White disparities generally larger than self-reported race and ethnicity data. BISG imputation methods are feasible for Medicaid claims data and reduced missingness to <1%. Disparities may be larger than what is estimated using self-reported data with high rates of missingness.