Percentage Of Missing Data Research Articles

Multiple Imputation of Missing Genotype Data for Unrelated Individuals

The objective of this study was to investigate the performance of multiple imputation of missing genotype data for unrelated individuals using the polytomous logistic regression model, focusing on different missingness mechanisms, percentages of missing data, and imputation models. A complete dataset of 581 individuals, each analysed for eight biallelic polymorphisms and the quantitative phenotype HDL-C, was used. From this dataset one hundred replicates with missing data were created, in different ways for different scenarios. The performance was assessed by comparing the mean bias in parameter estimates, the root mean squared standard errors, and the genotype-imputation error rates. Overall, the mean bias was small in all scenarios, and in most scenarios the mean did not differ significantly from 'no bias'. Including polymorphisms that are highly correlated in the imputation model reduced the genotype-imputation error rate and increased precision of the parameter estimates. The method works well for data that are missing completely at random, and for data that are missing at random. In conclusion, our results indicate that multiple imputation with the polytomous logistic regression model can be used for association studies to deal with the problem of missing genotype data, when attention is paid to the imputation model and the percentage of missing data.

Metric attributes of the unified Parkinson's disease rating scale 3.0 battery: Part I, feasibility, scaling assumptions, reliability, and precision

This article sought to assess the feasibility, scaling assumptions, reliability, precision, and factor analysis of the three most widely used rating scales in Parkinson's disease (PD): modified Hoehn and Yahr (HY), Schwab and England (SE), and Unified Parkinson's Disease Rating Scale (UPDRS). A multicenter sample of 1,136 PD patients was assessed. The percentage of missing data was 8% for SE, HY, and UPDRS mentation, and almost negligible (<2%) for the other UPDRS subscales. A high floor effect was found for UPDRS mentation (23%) and complications scales (36%). Item content validity, measured by multitrait scaling analysis, was adequate for all UPDRS subscales (scaling successes > 90%). Internal consistency coefficients for the UPDRS scales ranged from 0.79 (mentation) to 0.92 (activities of daily living and motor). Factor structure of the UPDRS mentation, activities of daily living, and complications subscales was replicated. As a whole, the HY, SE, and UPDRS are acceptable, consistent, and potentially sensitive rating scales.

Derivation and validation of the short version of the Malaysian Oral Health Impact Profile

This paper describes the development of a short version of the Malaysian Oral Health Impact Profile. The 45-item OHIP(M) was shortened using a method known as the 'item frequency method'. Here, the two most frequently reported items from each of the seven OHIP(M) subscales were chosen to form the short version, designated as the S-OHIP(M). Field testing was conducted to assess the effect of different modes of administration (mail versus interview) of the short form and to test its measurement properties (reliability and validity). A total of 206 respondents completed the questionnaire. In order to carry out test-retest analysis, a second administration was carried out 15 days after the first administration on a selected subsample. The mail questionnaire had a lower response rate and a higher percentage of missing data than the interview administered questionnaire. However, the mail mode of administration resulted in higher scores than interview. Cronbach's alpha was 0.89 and the ICC was also 0.89. All hypotheses developed to assess validity were confirmed. The S-OHIP(M) was found to be valid and reliable and appropriate for use in the cross-sectional studies in Malaysian adult populations.

Categorical missing data imputation for software cost estimation by multinomial logistic regression

A common problem in software cost estimation is the manipulation of incomplete or missing data in databases used for the development of prediction models. In such cases, the most popular and simple method of handling missing data is to ignore either the projects or the attributes with missing observations. This technique causes the loss of valuable information and therefore may lead to inaccurate cost estimation models. On the other hand, there are various imputation methods used to estimate the missing values in a data set. These methods are applied mainly on numerical data and produce continuous estimates. However, it is well known that the majority of the cost data sets contain software projects with mostly categorical attributes with many missing values. It is therefore reasonable to use some estimating method producing categorical rather than continuous values. The purpose of this paper is to investigate the possibility of using such a method for estimating categorical missing values in software cost databases. Specifically, the method known as multinomial logistic regression (MLR) is suggested for imputation and is applied on projects of the ISBSG multi-organizational software database. Comparisons of MLR with other techniques for handling missing data, such as listwise deletion (LD), mean imputation (MI), expectation maximization (EM) and regression imputation (RI) under different patterns and percentages of missing data, show the high efficiency of the proposed method.

Estimation of missing data using latent variable methods with auxiliary information

Estimating missing data in a matrix is often done with methods, such as the EM algorithm, using the existing data in that matrix. However, if auxiliary data that is related to the missing measurements is available, it can help to estimate the missing values. This paper presents latent variable approaches that exploit an auxiliary data information matrix, as well as the data matrix itself, for estimating missing data on raw material properties of rubbers used in formulating industrial polymer blends. The use of auxiliary information is most useful when the percentage of missing data is high, and when there exist certain combinations of missing data that show little correlation with the data that are present. Two approaches to incorporating the auxiliary information are presented: a multi-block approach and a novel two-stage projection approach. The latter approach is shown to be more flexible and to provide slightly better estimates in two industrial polymer blending problems. However, in both cases, the addition of the auxiliary information is shown to significantly improve the estimates.

Use of the EQ-5D among patients suffering from dementia

Background and objective This study was designed to determine the acceptability, feasibility, reliability, and validity of the French version of EQ-5D measuring HRQol in subjects with dementia. Methods EQ-5D was administered to 142 subjects. The feasibility and acceptability were determined by the refusal rate, the type of administration, and the percentage and distribution of missing data. Test-retest reliability was studied by kappa coefficients and validity by agreement between subjects' and proxies' assessments. Results The response rate was satisfactory. The instrument discriminated well among the subjects. Test-retest reliability was average. The validity was poor if we consider the agreement between patients' and caregivers' reports, but other criteria of validity produced better results. Subjects' responses on each dimension were related with their global judgment of health in the expected direction. Significant relations were found between the Katz index of ADL and self-rated difficulties only for expected dimensions. Relations with age and with gender were in line with expectations. Conclusion Results led to consider that patients' responses are not entirely devoid of judgment. It seems that dementia patients are capable of expressing their health-related quality of life through a brief instrument as the EQ-5D.

Treatment Of Missing Data At The Second Level Of Hierarchical Linear Models

The problem of missing data at the second level of a two-level hierarchical data structure was investigated. Using data generated to simulate the 1982 High School and Beyond data set, five missing data treatments—listwise deletion, overall mean substitution, group mean substitution, the expectation maximization (EM) algorithm, and multiple imputation—were examined under four manipulated conditions: number of Level 2 variables, Level 2 sample size, Level 1 intercept-slope correlation, and percentage of missing data. Listwise deletion, group mean substitution, and the EM algorithm performed equally well for the variable having missing values. For the variables having no missing data, listwise deletion and the EM algorithm performed satisfactorily. Only listwise deletion performed well in estimating random effects except when the Level 2 sample size was 30 and 40% of the data were missing. The practical implications of the findings are discussed.

Methods for the Analysis of Explanatory Linear Regression Models with Missing Data Not at Random

Since the work of Little and Rubin (1987) not substantial advances in the analysisof explanatory regression models for incomplete data with missing not at randomhave been achieved, mainly due to the difficulty of verifying the randomness ofthe unknown data. In practice, the analysis of nonrandom missing data is donewith techniques designed for datasets with random or completely random missingdata, as complete case analysis, mean imputation, regression imputation, maximumlikelihood or multiple imputation. However, the data conditions required to minimizethe bias derived from an incorrect analysis have not been fully determined. In thepresent work, several Monte Carlo simulations have been carried out to establishthe best strategy of analysis for random missing data applicable in datasets withnonrandom missing data. The factors involved in simulations are sample size,percentage of missing data, predictive power of the imputation model and existenceof interaction between predictors. The results show that the smallest bias is obtainedwith maximum likelihood and multiple imputation techniques, although with lowpercentages of missing data, absence of interaction and high predictive power ofthe imputation model (frequent data structures in research on child and adolescentpsychopathology) acceptable results are obtained with the simplest regression imputation.

Measuring the quality of life of people with severe and enduring mental health problems in different settings

Research has shown that quality of life (QOL) among people with severe and enduring mental health problems is better if they live in the community rather than in hospital. In the course of a study to replicate this result in one particular service, comparisons were made between different QOL measures in their suitability for routine clinical use. Forty-five hospital and community residents each took part in four out of six QOL assessments: the Life Experience Checklist (Ager, 1990) in user and keyworker report forms; the Quality of Life Questionnaire (Bigelow, McFarland & Olson, 1991); the Quality of Life Interview (Lehman, 1983); Values Satisfaction (Hartley, 1988) and Five Accomplishments (developed specifically for the study). Measures were also taken of adaptive behaviours and psychiatric symptomatology, as well as the practicalities of completion. The expected difference between hospital and community residents was found for three of the six measures (LEC-keyworker, Bigelow and Five Accomplishments), which were also strongly correlated with psychiatric symptomatology and adaptive behaviour scores. The measures differed considerably in the numbers of failures to complete, the percentage of missing data and the amount of face-to-face contact with service users required. The correlations between subscale scores on the different measures were generally low. The study raises questions about the robustness of the QOL construct, and its use as an outcome measure in mental health.

A Study of Hybrid Neural Network Approaches and the Effects of Missing Data on Traffic Forecasting

This paper describes an application of hybrid neural network approaches and an assessment of the effects of missing data on highway traffic flow forecasting. Using a self-organizing map (SOM), two hybrid approaches are developed for classifying traffic into different states. In the first hybrid approach, four auto-regressive integrated moving average (ARIMA) models are included. The second approach uses two multi-layer perception (MLP) models. The effects of missing data on neural network performance when forecasting traffic flow are analyzed, and options to replace the missing data are discussed. It is concluded that overall, ARIMA models are more sensitive to the percentage of missing data than neural networks in this context.

Evaluation of the Quality of Life in Dementia with a Generic Quality of Life Questionnaire: The Duke Health Profile

Objective: The study was designed to determine the acceptability, feasibility and validity of measuring quality of life in a representative sample of dementia patients with a generic instrument, the Duke Health Profile. Method: The French version of the Duke Health Profile was administered to 148 subjects with a mental disorder according to the DSM-III-R diagnostic criteria. The feasibility and acceptability of employing the instrument were determined by the refusal rate, the type of administration, and the percentage and distribution of missing data. Reliability was determined with Cronbach’s α coefficient. Instrument reproducibility was assessed with the intraclass correlation coefficient for test-retest values. Internal construct validity was determined by factor analysis. Discriminant capacity was determined by comparing the average scores on each measure among patients with and without an additional chronic pathology. The measurements obtained were compared by source of information (patient, family proxy and care provider proxy). Results: The feasibility and acceptability of the instrument was good. Only 2% of the patients refused to complete the questionnaire. Help from the interviewer was necessary in 79% of the cases. The average completion time was 10.6 min. Missing data exist in only 3.5% of the cases on average, except among patients with severe dementia (Mini Mental State Examination <10). For reliability, internal consistency was acceptable (Cronbach’s coefficient α = 0.5–0.7) when the self-esteem (0.23) and social health (0.26) concepts were eliminated. Reproducibility as measured by test-retest scores was moderate to good (intraclass correlation coefficient r = 0.53–0.80), except for anxiety (0.48) and perceived health (0.45). Severity of dementia mainly affected the feasibility, acceptability and reproducibility of the instrument. The family proxy seemed to agree more with the patient than did the care provider proxy. Conclusion: Quality of life can be measured in patients with dementia, but special tools need to be developed for severe dementia.

Detecting changes in morphospace occupation patterns in the fossil record: characterization and analysis of measures of disparity

Recently, there has been much interest in detecting and measuring patterns of change in disparity. Although most studies have used one or two measures of disparity to quantify and characterize the occupation of morphospace, multiple measures may be necessary to fully detect changes in patterns of morphospace occupation. Also, the ability to detect morphological trends and occupation patterns within morphospace depends on using the appropriate measure(s) of disparity. In this study, seven measures were used to determine and characterize sensitivity to sample size of the data, number of morphological characters, percentage of missing data, and changes in morphospace occupation pattern. These consist of five distance measures—sum of univariate variances, total range, mean distance, principal coordinate analysis volume, average pairwise dissimilarity—and two non-distance measures—participation ratio and number of unique pairwise character combinations. Evaluation of each measure with respect to sensitivity to sample size, number of morphological characters, and percentage of missing data was accomplished by using both simulated and Ordovician crinoid data. For simulated data, each measure of disparity was evaluated for its response to changes of morphospace occupation pattern, and with respect to simulated random and nonrandom extinction events. Changes in disparity were also measured within the Crinoidea across the Permian extinction event.Although all measures vary in sensitivity with respect to species sample size, number of morphological characters, and percentage of missing data, the non-distance measures overall produce the lowest estimates of variance (in bootstrap analyses). The non-distance measures appear to be relatively insensitive to changes in morphospace occupation pattern. All measures, except average pairwise dissimilarity, detect changes in occupation pattern in simulated nonrandom extinction events, but all measures, except number of unique pairwise character combinations and principal coordinate analysis volume, are relatively insensitive to changes in pattern in simulated random extinction events. The distance measures report similar changes in disparity over the Permian extinction event, whereas the non-distance measures differ. This study suggests that each measure of disparity is designed for different purposes, and that by using a combination of techniques a clearer picture of disparity should emerge.

Probability density estimation using incomplete data

An unsupervised method of learning probability density function parameters in the framework of mixture densities from incomplete data is developed. Unsupervised learning can be considered as learning from observations as they are received. In real-world processes, there are many imperfections in the observations. The Expectation–Maximization algorithm is used to iteratively find the maximum likelihood estimate of the missing values and the parameters of the probability density function. Reliability of the Expectation–Maximization algorithm and its convergence properties during the learning process from incomplete data are presented. Three examples of learning mixture probability density function parameters from an incomplete data set are presented and the boundaries of properly estimating the mixture probability density function parameters as the percentage of missing data increases are examined.

Robust motion and correspondence of noisy 3-D point sets with missing data

We describe RICP, a robust algorithm for registering and finding correspondences in sets of 3-D points with significant percentages of missing data, and therefore useful for both motion analysis and reverse engineering. RICP exploits LMedS robust estimation to withstand the effect of outliers. Our extensive experimental comparison of RICP with ICP shows RICP's superior robustness and reliability.

Forecasting with missing data: application to coastal wave heights

This paper presents a comparative analysis of linear and mixed models for short-term forecasting of a real data series with a high percentage of missing data. Data are the series of significant wave heights registered at regular periods of three hours by a buoy placed in the Bay of Biscay. The series is interpolated with a linear predictor which minimizes the forecast mean square error. The linear models are seasonal ARIMA models and the mixed models have a linear component and a non-linear seasonal component. The non-linear component is estimated by a non-parametric regression of data versus time. Short-term forecasts, no more than two days ahead, are of interest because they can be used by the port authorities to notify the fleet. Several models are fitted and compared by their forecasting behaviour. Copyright © 1999 John Wiley & Sons, Ltd.

Likelihood methods for incomplete longitudinal binary responses with incomplete categorical covariates.

We consider longitudinal studies in which the outcome observed over time is binary and the covariates of interest are categorical. With no missing responses or covariates, one specifies a multinomial model for the responses given the covariates and uses maximum likelihood to estimate the parameters. Unfortunately, incomplete data in the responses and covariates are a common occurrence in longitudinal studies. Here we assume the missing data are missing at random (Rubin, 1976, Biometrika 63, 581-592). Since all of the missing data (responses and covariates) are categorical, a useful technique for obtaining maximum likelihood parameter estimates is the EM algorithm by the method of weights proposed in Ibrahim (1990, Journal of the American Statistical Association 85, 765-769). In using the EM algorithm with missing responses and covariates, one specifies the joint distribution of the responses and covariates. Here we consider the parameters of the covariate distribution as a nuisance. In data sets where the percentage of missing data is high, the estimates of the nuisance parameters can lead to highly unstable estimates of the parameters of interest. We propose a conditional model for the covariate distribution that has several modeling advantages for the EM algorithm and provides a reduction in the number of nuisance parameters, thus providing more stable estimates in finite samples.

Weighted Estimating Equations with Nonignorably Missing Response Data

We propose weighted estimating equations for data with nonignorable nonresponse in order to reduce the bias that can occur with a complete case analysis. A survey concerning medical practice guidelines, malpractice litigation, and settlement provides the framework. The survey was sent to recipients in two waves: those who responded on the first or second wave are used to estimate a nonignorable nonresponse model, while the fraction of recipients who never responded is used to allow the percentage of missing data to change with each wave. We use the structure of the GEE of Liang and Zeger (1986, Biometrika 73, 13-22), adding weights equal to the inverse probability of being observed. We present simulations demonstrating the bias that can occur with an unweighted analysis and use the survey data to illustrate the methods.

Outcome evaluation in a home palliative care service

The complexity of assessing the impact of palliative care is much greater than in other fields of medicine, due to the shortcomings of traditional outcome indicators. We conducted a prospective study to describe the patient's quality of life at the outset and during palliative care at home and to define some potential indicators of palliative care outcomes with the aim of assessing the quality of home care as provided by a palliative care unit. Seventy-three patients who received care at home were assessed. The median survival in palliative home care was 29 days. To evaluate the degree of symptom distress, we used an Italian version of the Symptom Distress Scale. Assessing the quality-of-life pattern over time, we observed that palliative care was effective in mitigating pain and, at least in part, in stimulating appetite, curbing nausea, and controlling psychological aspects. The subscales referring to social and functional aspects steadily worsened. The difficulties encountered (the high percentage of missing data, the considerable number of patients treated for less than 10 days, etc.) should be a warning against using only one assessment instrument. It is worthwhile defining the various potential outcomes of palliative care even though all results will not always be measurable in every patient. Only a global assessment, a “multiple outcomes approach,” based on different indicators, would allow for evaluation of the outcome of the care process.

Forecasting with Missing Data: Application to a Real Case

This paper presents a comparative analysis of linear and mixed models for short term forecasting of a real data series with a high percentage of missing data. Data are the series of significant wave heights registered at regular periods of three hours by a buoy placed in the Bay of Biscay. The series is interpolated with a linear predictor which minimizes the forecast mean square error. The linear models are seasonal ARIMA models and the mixed models have a linear component and a non linear seasonal component. The non linear component is estimated by a non parametric regression of data versus time. Short term forecasts, no more than two days ahead, are of interest because they can be used by the port authorities to notify the fleet. Several models are fitted and compared by their forecasting behavior.

A conditional model for incomplete covariates in parametric regression models

SUMMARY Incomplete covariate data arise in many data sets. When the missing covariates are categorical, a useful technique for obtaining parameter estimates is the EM algorithm by the method of weights proposed in Ibrahim (1990). This method requires the estimation of many nuisance parameters for the distribution of the covariates. Unfortunately, in data sets when the percentage of missing data is high, and the missing covariate patterns are highly non-monotone, the estimates of the nuisance parameters can lead to highly unstable estimates of the parameters of interest. We propose a conditional model for the covariate distribution that has several modelling advantages for the E-step and provides a reduction in the number of nuisance parameters, thus providing more stable estimates in finite samples. We present a clinical trials example with six covariates, five of which have some missing values.