Comparison Of Imputation Methods Research Articles

A Comparison of Imputation Methods In CPI Calculations Used by IWGPS Organizations and Imputation Methods of robust cellwise outlier and missing data

In this study, imputation methods used by IWGPS (*) organizations in CPI (Consumer Price Index) calculations in case of missing data are discussed. Depending on the development of technological devices, methods suitable for the demand of collecting data and producing statistics from the field immediately, in a way that can be adapted to the data collection tools of statistics offices have been proposed. While the immediate imputation advantages of the proposed methods are mentioned, the proposed imputation results are compared with the method results used in the current practice and imputation results of cellwise outlier and missing data in the statistical computer programming language. The method3(i_m üd19) proposed to assist the imputation tools used in CPI calculation all over the world and produced from the statistics is intended to provide convenience to all users. It can also be considered as a common weighted imputation method for both cellwise outlier and missing data case.

A comparison of imputation methods using machine learning models

A comparison of imputation methods using machine learning models

A BAD IDEA OF USING MODE IMPUTATION METHOD

Missing data is a recurring issue in psychology questionnaire when a respondent does not respond to questions due to personal reasons. In general, two types of imputation techniques are used to replace missing data: single imputation and multiple imputation (MI). The single imputation technique generates a single value to impute each missing data. The simplest methods of single imputation are mean, mode and median. In contrast, the multiple imputation technique imputes each missing data several times resulting in multiple complete datasets. The most popular method in MI that can deal with numerical and categorical data type is the predictive mean matching (PMM). The aim of this article is to compare and visualize how the mode imputation method in the single imputation technique will lead to a biased data distribution and the PMM method in the MI techniques will reduce this issue. Both methods, mode imputation and PMM are often considered when dealing with categorical data types. The mode imputation replaces a missing data with the most frequent value of an item in a survey. Meanwhile, the predictive mean matching is an extension of regression model that apply donor selection strategy to replace a missing data. Results from bar charts visualize the multiple imputation shows less discrepancy between the original distribution and imputed distribution. Thus, in this research, it can be concluded that the PMM method in MI technique shows a less biased distribution than implementing the mode imputation method. A comparison of imputation methods with different missing rates on a survey dataset should be considered for future work.

116. Comparison of imputation methods for evaluating long-term clinical outcomes following lumbar total disc replacement

BACKGROUND CONTEXT The problem of missing data is common in clinical trials, and it can have a significant impact on the conclusions that can be drawn from the data. Long-term results are particularly susceptible as the lost to follow-up (LTFU) rates increase over the course of a trial. Imputation methods are often used to account for missing data points, but some may either under or overestimate the true treatment effect. PURPOSE In this post-hoc analysis of a randomized clinical trial, we evaluated the clinical success of three motion preserving artificial discs up to 7 years following lumbar total disc replacement (TDR) using different imputation methods. STUDY DESIGN/SETTING Randomized, controlled, multicenter clinical IDE trial. PATIENT SAMPLE A total of 324 subjects with single level degenerative disc disease and discogenic pain. OUTCOME MEASURES Radigraphic range of motion on flexion-extension, Oswestry Disability Index (ODI), neurologic exam, adverse events, subsequent surgery. METHODS Subjects were randomized to receive either a semi-constrained mobile core disc (n=218) or a control device (n=106) which was either a constrained (n=65) or unconstrained (n=41) core disc. Several imputation methods were used to calculate clinical success, including: missing values=failure, missing values=success, last observation carried forward (LOCF), complete case, best case and worst case scenarios. Clinical success scores were compared between the treatment (semi-constrained) and control (constrained or unconstrained) groups.Clinical outcomes were collected from baseline through to 7 years following single level (L4-L5 or L5-S1) TDR. Clinical success was defined as a ≥15 point improvement in ODI from baseline, maintenance or improvement in ROM and neurological evaluation (motor and sensory), the absence of index-level subsequent surgical interventions (SSI) and serious device-related adverse events (SDAE). RESULTS By imputing missing values as failures, individual success scores decreased at 5 and 7 years compared to 2 years postop for both groups. Only ROM success at 2 years was significantly greater in the treatment compared to control group (58.7% vs 42.5%, p=0.0065). The overall success score, which is a composite of the individual success scores, also decreased over time, but it was significantly greater in the treatment group at 2 years (42.2% vs 28.3%, p=0.0020). Using complete case data, clinical success at 5 and 7 years was maintained close to 2-year values for overall and individual success scores, except for SDAE success which decreased at 7 years from 2 years for both the treatment (86.1% to 67.1%) and control groups (78.7% to 63.2%). Similarly, LOCF clinical outcomes were the same between 5 and 7 years postop for both groups. By imputing missing values as successes, the results were comparable to scores yielded by complete case and LOCF imputation methods. CONCLUSIONS The missing=failure imputation method is most susceptible to LFTU and may underestimate the treatment effect if there is a large LTFU rate, which is inherent in most long-term clinical studies. Complete case and LOCF imputation methods may be more accurate estimates of long-term clinical outcomes for clinical studies with a large patient enrollment. FDA DEVICE/DRUG STATUS This abstract does not discuss or include any applicable devices or drugs.

A simulation comparison of imputation methods for quantitative data in the presence of multiple data patterns

ABSTRACTAn extensive investigation via simulation is carried out with the aim of comparing three nonparametric, single imputation methods in the presence of multiple data patterns. The ultimate goal is to provide useful hints for users needing to quickly pick the most effective imputation method among the following: Forward Imputation (), considered in the two variants of with the principal component analysis (PCA), which alternates the use of PCA and the Nearest-Neighbour Imputation (NNI) method in a forward, sequential procedure, and with the Mahalanobis distance, which involves the use of the Mahalanobis distance when performing NNI; the iterative PCA technique, which imputes missing values simultaneously via PCA; the method, which is based on random forests and is developed for mixed-type data. The performance of these methods is compared under several data patterns characterized by different levels of kurtosis or skewness and correlation structures.

Missing traffic data: comparison of imputation methods

Many traffic management and control applications require highly complete and accurate data of traffic flow. However, because of various reasons such as sensor failure or transmission error, it is common that some traffic flow data are lost. As a result, various methods were proposed by using a wide spectrum of techniques to estimate missing traffic data in the last two decades. Generally, these missing data imputation methods can be categorised into three kinds: prediction methods, interpolation methods and statistical learning methods. To assess their performance, these methods are compared from different aspects in this paper, including reconstruction errors, statistical behaviours and running speeds. Results show that statistical learning methods are more effective than the other two kinds of imputation methods when data of a single detector is utilised. Among various methods, the probabilistic principal component analysis (PPCA) yields best performance in all aspects. Numerical tests demonstrate that PPCA can be used to impute data online before making further analysis (e.g. make traffic prediction) and is robust to weather changes.

Estimation of Missing Values in the Data Mining and Comparison of Imputation Methods

Many existing, industrial, and research data sets contain missing values (MVs). There are various reasons for their existence, such as manual data entry procedures, equipment errors, and incorrect measurements. The presence of such imperfections usually requires a preprocessing stage in which the data are prepared and cleaned , in order to be useful to and sufficiently clear for the knowledge extraction process. MVs make the performance of data analysis difficult. The presence of MVs can also pose serious problems for researchers. In fact, in appropriate handling of the MVs in the analysis may introduce bias and can result in misleading conclusions being drawn from a research study and can also limit the generalize ability of the research findings. The various types of problem are usually associated with MVs in data mining are (1) loss of efficiency;(2) complications in handling and analyzing the data; and(3) bias resulting from differences between missing and complete data. We will focus our attention on the use of imputation methods. A fundamental advantage of this approach is that the MV treatment is independent of the learning algorithm used. For this reason, the user can select the most appropriate method for each situation he faces. In this paper different methods of estimation of missing values are discussed. The comparison of different imputation methods are given by using non parametric methods.

A numerical study of multiple imputation methods using nonparametric multivariate outlier identifiers and depth-based performance criteria with clinical laboratory data

It is well known that if a multivariate outlier has one or more missing component values, then multiple imputation (MI) methods tend to impute nonextreme values and make the outlier become less extreme and less likely to be detected. In this paper, nonparametric depth-based multivariate outlier identifiers are used as criteria in a numerical study comparing several established methods of MI as well as a new proposed one, nine in all, in a setting of several actual clinical laboratory data sets of different dimensions. Two criteria, an ‘outlier recovery probability’ and a ‘relative accuracy measure’, are developed, based on depth functions. Three outlier identifiers, based on Mahalanobis distance, robust Mahalanobis distance, and generalized principle component analysis are also included in the study. Consequently, not only the comparison of imputation methods but also the comparison of outlier detection methods is accomplished in this study. Our findings show that the performance of an MI method depends on the choice of depth-based outlier detection criterion, as well as the size and dimension of the data and the fraction of missing components. By taking these features into account, an MI method for a given data set can be selected more optimally.

A Comparison of Imputation Methods for Bayesian Factor Analysis Models

Imputation methods are popular for the handling of missing data in psychology. The methods generally consist of predicting missing data based on observed data, yielding a complete data set that is amiable to standard statistical analyses. In the context of Bayesian factor analysis, this article compares imputation under an unrestricted multivariate normal model (Multiple Imputation [MI]) to imputation under the statistical model of interest (Data Augmentation [DA]). The former method is popular in applied research, but the latter method is more straightforward from a Bayesian perspective. Simulations demonstrate that DA yields less-biased parameter estimates for moderate sample sizes and high missingness proportions. MI, however, yields less-biased parameter estimates for large sample sizes with misspecified models. The incorporation of auxiliary variables in DA is also addressed, and BUGS code is provided.

O-19 Comparison of imputation methods for missing immunohistochemical markers in a study of breast cancer prognosis

O-19 Comparison of imputation methods for missing immunohistochemical markers in a study of breast cancer prognosis

A comparison of imputation methods in the presence of imprecise data when employing a neural network s-Sigmoid function

This research addresses the effects of the neural network s-Sigmoid function on Knowledge Discovery of Databases (KDD) in the presence of imprecise data. ANOVA testing and Tukey's Honestly Significant Difference statistics are conducted to investigate the impact of two factors: level of data missingness and imputation method. Data mining is based upon searching the concatenation of multiple databases that usually contain some amount of missing data along with a percentage of inaccurate data and noise. Therefore, analysis depends heavily on the accuracy of the database and on the chosen sample data to be used for model training and testing.

Dealing with missing data in a multi-question depression scale: a comparison of imputation methods

BackgroundMissing data present a challenge to many research projects. The problem is often pronounced in studies utilizing self-report scales, and literature addressing different strategies for dealing with missing data in such circumstances is scarce. The objective of this study was to compare six different imputation techniques for dealing with missing data in the Zung Self-reported Depression scale (SDS).Methods1580 participants from a surgical outcomes study completed the SDS. The SDS is a 20 question scale that respondents complete by circling a value of 1 to 4 for each question. The sum of the responses is calculated and respondents are classified as exhibiting depressive symptoms when their total score is over 40. Missing values were simulated by randomly selecting questions whose values were then deleted (a missing completely at random simulation). Additionally, a missing at random and missing not at random simulation were completed. Six imputation methods were then considered; 1) multiple imputation, 2) single regression, 3) individual mean, 4) overall mean, 5) participant's preceding response, and 6) random selection of a value from 1 to 4. For each method, the imputed mean SDS score and standard deviation were compared to the population statistics. The Spearman correlation coefficient, percent misclassified and the Kappa statistic were also calculated.ResultsWhen 10% of values are missing, all the imputation methods except random selection produce Kappa statistics greater than 0.80 indicating 'near perfect' agreement. MI produces the most valid imputed values with a high Kappa statistic (0.89), although both single regression and individual mean imputation also produced favorable results. As the percent of missing information increased to 30%, or when unbalanced missing data were introduced, MI maintained a high Kappa statistic. The individual mean and single regression method produced Kappas in the 'substantial agreement' range (0.76 and 0.74 respectively).ConclusionMultiple imputation is the most accurate method for dealing with missing data in most of the missind data scenarios we assessed for the SDS. Imputing the individual's mean is also an appropriate and simple method for dealing with missing data that may be more interpretable to the majority of medical readers. Researchers should consider conducting methodological assessments such as this one when confronted with missing data. The optimal method should balance validity, ease of interpretability for readers, and analysis expertise of the research team.

A comparison of imputation methods in a longitudinal randomized clinical trial

It is common for longitudinal clinical trials to face problems of item non-response, unit non-response, and drop-out. In this paper, we compare two alternative methods of handling multivariate incomplete data across a baseline assessment and three follow-up time points in a multi-centre randomized controlled trial of a disease management programme for late-life depression. One approach combines hot-deck (HD) multiple imputation using a predictive mean matching method for item non-response and the approximate Bayesian bootstrap for unit non-response. A second method is based on a multivariate normal (MVN) model using PROC MI in SAS software V8.2. These two methods are contrasted with a last observation carried forward (LOCF) technique and available-case (AC) analysis in a simulation study where replicate analyses are performed on subsets of the originally complete cases. Missing-data patterns were simulated to be consistent with missing-data patterns found in the originally incomplete cases, and observed complete data means were taken to be the targets of estimation. Not surprisingly, the LOCF and AC methods had poor coverage properties for many of the variables evaluated. Multiple imputation under the MVN model performed well for most variables but produced less than nominal coverage for variables with highly skewed distributions. The HD method consistently produced close to nominal coverage, with interval widths that were roughly 7 per cent larger on average than those produced from the MVN model.