Incomplete Data Models Research Articles

A Generalized Nesterov-Accelerated Second-Order Latent Factor Model for High-Dimensional and Incomplete Data.

High-dimensional and incomplete (HDI) data are frequently encountered in big date-related applications for describing restricted observed interactions among large node sets. How to perform accurate and efficient representation learning on such HDI data is a hot yet thorny issue. A latent factor (LF) model has proven to be efficient in addressing it. However, the objective function of an LF model is nonconvex. Commonly adopted first-order methods cannot approach its second-order stationary point, thereby resulting in accuracy loss. On the other hand, traditional second-order methods are impractical for LF models since they suffer from high computational costs due to the required operations on the objective's huge Hessian matrix. In order to address this issue, this study proposes a generalized Nesterov-accelerated second-order LF (GNSLF) model that integrates twofold conceptions: 1) acquiring proper second-order step efficiently by adopting a Hessian-vector algorithm and 2) embedding the second-order step into a generalized Nesterov's acceleration (GNA) method for speeding up its linear search process. The analysis focuses on the local convergence for GNSLF's nonconvex cost function instead of the global convergence has been taken; its local convergence properties have been provided with theoretical proofs. Experimental results on six HDI data cases demonstrate that GNSLF performs better than state-of-the-art LF models in accuracy for missing data estimation with high efficiency, i.e., a second-order model can be accelerated by incorporating GNA without accuracy loss.

Missingness-Pattern-Adaptive Learning With Incomplete Data.

Many real-world problems deal with collections of data with missing values, e.g., RNA sequential analytics, image completion, video processing, etc. Usually, such missing data is a serious impediment to a good learning achievement. Existing methods tend to use a universal model for all incomplete data, resulting in a suboptimal model for each missingness pattern. In this paper, we present a general model for learning with incomplete data. The proposed model can be appropriately adjusted with different missingness patterns, alleviating competitions between data. Our model is based on observable features only, so it does not incur errors from data imputation. We further introduce a low-rank constraint to promote the generalization ability of our model. Analysis of the generalization error justifies our idea theoretically. In additional, a subgradient method is proposed to optimize our model with a proven convergence rate. Experiments on different types of data show that our method compares favorably with typical imputation strategies and other state-of-the-art models for incomplete data. More importantly, our method can be seamlessly incorporated into the neural networks with the best results achieved. The source code is released at https://github.com/YS-GONG/missingness-patterns.

Occupancy states forecasting with a hidden Markov model for incomplete data, exploiting daily periodicity

To automatize HVAC energy savings in buildings, it is useful to forecast the occupants’ behaviour. This article deals with such a forecasting problem by exploiting the daily periodicity of the input variables and the ability of the proposed model to learn from missing data. We propose a case study of occupancy behaviour, for which only a history of indirect information is available: CO2 and decibel levels. The unobserved classes are modeled by a Hidden Markov Model (HMM) which emits sensors’ data and periodic variables. The fact that Baum-Welch’s algorithm for HMMs is an instance of the EM algorithm allows the model to learn from incomplete data. In this case, the missing data are the class labels at every instant (past and future) and the sensors’ data in the future, but the time periodic variables are known at every instant: this enables the model to classify both the past instants (history) and the future instants (forecasting). Our method is tested on simulated data sets with 5%, 15%, 25% and 50% missing data that provide respectively an average classification accuracy of 93%, 89%, 85% and 80%. Then, experiments on one-day and one-week forecasting scenarios achieved an average accuracy of 82% and 81%. The proposed method is also applied on a real-life data set which seems to provide promising results.

Development of models and assessment of the world walnut variety fund by fruit quality

Walnut (Juglans regia L.) is a particularly significant plant for humans in terms of its useful properties and in the Russian Federation it can be attributed to the most valuable introducers for forestry and horticulture. It is grown in many countries of the world and the area of its cultivation is constantly expanding. Breeding species for food purposes requires a selection assessment of the quality of its fruits. For this, at different times, appropriate methods were developed and the existing gene pool of the culture was assessed. By now, new varieties and forms have been bred and brought, which also need to be evaluated for rational use for scientific and practical purposes. The descriptions of varieties and forms of walnut often contain insufficient information about all selected traits, which requires the development of new models to assess the variety fund using incomplete data. The purpose of the research is to develop the missing models of selection evaluation using incomplete data and evaluate the world walnut variety fund in terms of fruit quality, to identify varieties and forms of the highest quality category. The well-known methodology and new models have been used for the evaluation. When developing evaluation models for incomplete data, a database of 50 varieties evaluated for all indicators has been used. Using the method of excluding individual indicators, the corresponding multiple regression models have been calculated on its basis, taking into account a different combination of features. The licensed programs Stadia 8.0, Microsoft Excel have been used in the research. The proposed models provide an estimate with an average error of ± 0.48 – 3.72 points at R2 = 0.63 – 0.99. A promising walnut gene pool from 23 countries has been assessed, 69 representatives with fruits of the breeding category of the highest quality identified, and a list of them compiled. Of the 512 ancestors, 19.73% had the selection category of fruits of the highest quality, 62.88% of high-quality fruits, 16.80% of ordinary fruits, and 0.59% of varieties and forms of low-quality fruits.

Partial quantisation scheme for optimising the performance of hopfield network

The ideal Hopfield network would be able to remember information and recover the missing information based on what has been remembered. It is expected to have applications in areas such as associative memory, pattern recognition, optimisation computation, parallel implementation of VLSI and optical devices, but the lack of memory capacity and the tendency to generate pseudo-attractors make the network capable of handling only a very small amount of data. In order to make the network more widely used, we propose a scheme to optimise and improve its memory and resilience by introducing quantum perceptrons instead of Hebbian rules to complete its weight matrix design. Compared with the classical Hopfield network, our scheme increases the threshold of each node in the network while training the weights, and the memory space of the Hopfield network changes from being composed of the weight matrix only to being composed of the weight matrix and the threshold matrix together, resulting in a dimensional increase in the memory capacity of the network, which greatly solves the problem of the Hopfield network’s memory The problem of insufficient memory capacity and the tendency to generate pseudo-attractors was solved to a great extent. To verify the feasibility of the proposed scheme, we compare it with the classical Hopfield network in four different dimensions, namely, non-orthogonal simple matrix recovery, incomplete data recovery, memory capacity and model convergence speed. These experiments demonstrate that the improved Hopfield network with quantum perceptron has significant advantages over the classical Hopfield network in terms of memory capacity and recovery ability, which provides a possibility for practical application of the network.

Joint Models for Incomplete Longitudinal Data and Time-to-Event Data

Clinical studies often collect longitudinal and time-to-event data for each subject. Joint modeling is a powerful methodology for evaluating the association between these data. The existing models, however, have not sufficiently addressed the problem of missing data, which are commonly encountered in longitudinal studies. In this paper, we introduce a novel joint model with shared random effects for incomplete longitudinal data and time-to-event data. Our proposed joint model consists of three submodels: a linear mixed model for the longitudinal data, a Cox proportional hazard model for the time-to-event data, and a Cox proportional hazard model for the time-to-dropout from the study. By simultaneously estimating the parameters included in these submodels, the biases of estimators are expected to decrease under two missing scenarios. We estimated the proposed model by Bayesian approach, and the performance of our method was evaluated through Monte Carlo simulation studies.

Bayesian Dynamic Mode Decomposition with Variational Matrix Factorization

Dynamic mode decomposition (DMD) and its extensions are data-driven methods that have substantially contributed to our understanding of dynamical systems. However, because DMD and most of its extensions are deterministic, it is difficult to treat probabilistic representations of parameters and predictions. In this work, we propose a novel formulation of a Bayesian DMD model. Our Bayesian DMD model is consistent with the procedure of standard DMD, which is to first determine the subspace of observations, and then compute the modes on that subspace. Variational matrix factorization makes it possible to realize a fully-Bayesian scheme of DMD. Moreover, we derive a Bayesian DMD model for incomplete data, which demonstrates the advantage of probabilistic modeling. Finally, both of nonlinear simulated and real-world datasets are used to illustrate the potential of the proposed method.

A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling

The expectation–maximization (EM) algorithm is a powerful computational technique for maximum likelihood estimation in incomplete data models. When the expectation step cannot be performed in closed form, a stochastic approximation of EM (SAEM) can be used. The convergence of the SAEM toward critical points of the observed likelihood has been proved and its numerical efficiency has been demonstrated. However, sampling from the posterior distribution may be intractable or have a high computational cost. Moreover, despite appealing features, the limit position of this algorithm can strongly depend on its starting one. Sampling from an approximation of the distribution in the expectation phase of the SAEM allows coping with these two issues. This new procedure is referred to as approximated-SAEM and is proved to converge toward critical points of the observed likelihood. Experiments on synthetic and real data highlight the performance of this algorithm in comparison to the SAEM and the EM when feasible.

An alternative characterization of MAR in shared parameter models for incomplete longitudinal data and its utilization for sensitivity analysis

Dropout is a common complication in longitudinal studies, especially since the distinction between missing not at random (MNAR) and missing at random (MAR) dropout is intractable. Consequently, one starts with an analysis that is valid under MAR and then performs a sensitivity analysis by considering MNAR departures from it. To this end, specific classes of joint models, such as pattern-mixture models (PMMs) and selection models (SeMs), have been proposed. On the contrary, shared-parameter models (SPMs) have received less attention, possibly because they do not embody a characterization of MAR. A few approaches to achieve MAR in SPMs exist, but are difficult to implement in existing software. In this article, we focus on SPMs for incomplete longitudinal and time-to-dropout data and propose an alternative characterization of MAR by exploiting the conditional independence assumption, under which outcome and missingness are independent given a set of random effects. By doing so, the censoring distribution can be utilized to cover a wide range of assumptions for the missing data mechanism on the subject-specific level. This approach offers substantial advantages over its counterparts and can be easily implemented in existing software. More specifically, it offers flexibility over the assumption for the missing data generating mechanism that governs dropout by allowing subject-specific perturbations of the censoring distribution, whereas in PMMs and SeMs dropout is considered MNAR strictly.

Incomplete data classification with view-based decision tree

Data quality issues may bring serious problems in data analysis. For instance, missing values could decrease the accuracy of the classification. As traditional classification approaches can only be applied to complete data sets, we present a generic classification model for incomplete data where existing classification methods can be effectively incorporated. Firstly, we generate complete views from the incomplete data by choosing proper subsets of attributes based on Information Gain measure. Then we use these selected views to obtain multiple base classifiers. Finally, the base classifies are effectively combined as a final classifier with a decision tree. Extensive experiments results on real data sets demonstrate that the proposed method outperforms existing approaches.

Bayesian network models for incomplete and dynamic data

Bayesian networks are a versatile and powerful tool to model complex phenomena and the interplay of their components in a probabilistically principled way. Moving beyond the comparatively simple case of completely observed, static data, which has received the most attention in the literature, in this paper, we will review how Bayesian networks can model dynamic data and data with incomplete observations. Such data are the norm at the forefront of research and in practical applications, and Bayesian networks are uniquely positioned to model them due to their explainability and interpretability.

Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data.

Fusing multi-modality data is crucial for accurate identification of brain disorder as different modalities can provide complementary perspectives of complex neurodegenerative disease. However, there are at least four common issues associated with the existing fusion methods. First, many existing fusion methods simply concatenate features from each modality without considering the correlations among different modalities. Second, most existing methods often make prediction based on a single classifier, which might not be able to address the heterogeneity of the Alzheimer's disease (AD) progression. Third, many existing methods often employ feature selection (or reduction) and classifier training in two independent steps, without considering the fact that the two pipelined steps are highly related to each other. Forth, there are missing neuroimaging data for some of the participants (e.g., missing PET data), due to the participants' "no-show" or dropout. In this paper, to address the above issues, we propose an early AD diagnosis framework via novel multi-modality latent space inducing ensemble SVM classifier. Specifically, we first project the neuroimaging data from different modalities into a latent space, and then map the learned latent representations into the label space to learn multiple diversified classifiers. Finally, we obtain the more reliable classification results by using an ensemble strategy. More importantly, we present a Complete Multi-modality Latent Space (CMLS) learning model for complete multi-modality data and also an Incomplete Multi-modality Latent Space (IMLS) learning model for incomplete multi-modality data. Extensive experiments using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset have demonstrated that our proposed models outperform other state-of-the-art methods.

F-SAEM: A fast stochastic approximation of the EM algorithm for nonlinear mixed effects models

The ability to generate samples of the random effects from their conditional distributions is fundamental for inference in mixed effects models. Random walk Metropolis is widely used to perform such sampling, but this method is known to converge slowly for medium dimensional problems, or when the joint structure of the distributions to sample is spatially heterogeneous. The main contribution consists of an independent Metropolis–Hastings (MH) algorithm based on a multidimensional Gaussian proposal that takes into account the joint conditional distribution of the random effects and does not require any tuning. Indeed, this distribution is automatically obtained thanks to a Laplace approximation of the incomplete data model. Such approximation is shown to be equivalent to linearizing the structural model in the case of continuous data. Numerical experiments based on simulated and real data illustrate the performance of the proposed methods. For fitting nonlinear mixed effects models, the suggested MH algorithm is efficiently combined with a stochastic approximation version of the EM algorithm for maximum likelihood estimation of the global parameters.

Estimation of FAVAR Models for Incomplete Data with a Kalman Filter for Factors with Observable Components

This article extends the Factor-Augmented Vector Autoregression Model (FAVAR) to mixed-frequency and incomplete panel data. Within the scope of a fully parametric two-step approach, the alternating application of two expectation-maximization algorithms jointly estimates model parameters and missing data. In contrast to the existing literature, we do not require observable factor components to be part of the panel data. For this purpose, we modify the Kalman Filter for factors consisting of latent and observed components, which significantly improves the reconstruction of latent factors according to the performed simulation study. To identify model parameters uniquely, the loadings matrix is constrained. In our empirical application, the presented framework analyzes US data for measuring the effects of the monetary policy on the real economy and financial markets. Here, the consequences for the quarterly Gross Domestic Product (GDP) growth rates are of particular importance.

Spatial Incomplete Panel Data Models with Two-Way Error Components

When the panel is incomplete, which is the rule rather than the exception, standard estimation methods cannot be applied. This paper considers a model with spatial lag and two way-way error components regression with unbalanced data. The paper derives several estimators for structural parameters. It also develops more intensively ANOVA estimators for covariance components. The Monte Carlo experiments in which the design varies in, (a) the degree of unbalancedness in the data, (b) the variance components ratio, (c) the spatial matrix and (d) the spatial coefficient, compare the performance of theses estimators. Some of the basic findings are the following: (1) Better estimates of the variance components do not necessarily imply better estimates of the regression coefficients. (2) Making the data balanced, by dropping observations, worsens the performance of these estimators when compared to those from the entire unbalanced data.

Conditional kernel density estimation for some incomplete data models

A class of density estimators based on observed incomplete data are proposed. The method is to use a conditional kernel, defined as the expectation of a given kernel for the complete data conditioning on the observed data, to construct the density estimator. We study such kernel density estimators for several commonly used incomplete data models and establish their basic asymptotic properties. Some characteristics different from the classical kernel estimators are discovered. For instance, the asymptotic results of the proposed estimator do not depend on the choice of the kernel $k(\cdot )$. Simulation study is conducted to evaluate the performance of the estimator and compared with some exising methods.

A MIXED-EFFECTS MODEL FOR INCOMPLETE DATA FROM LABELING-BASED QUANTITATIVE PROTEOMICS EXPERIMENTS.

In mass spectrometry (MS) based quantitative proteomics research, the emerging iTRAQ (isobaric tag for relative and absolute quantitation) and TMT (tandem mass tags) techniques have been widely adopted for high throughput protein profiling. In a typical iTRAQ/TMT proteomics study, samples are grouped into batches, and each batch is processed by one multiplex experiment, in which the abundances of thousands of proteins/peptides in a batch of samples can be measured simultaneously. The multiplex labeling technique greatly enhances the throughput of protein quantification. However, the technical variation across different iTRAQ/TMT multiplex experiments is often large due to the dynamic nature of MS instruments. This leads to strong batch effects in the iTRAQ/TMT data. Moreover, the iTRAQ/TMT data often contain substantial batch-level nonignorable missing entries. Specifically, the abundance measures of a given protein/peptide are often either observed or missing altogether in all the samples from the same batch, with the missing probability depending on the combined batch-level abundances. We term this unique missing-data mechanism as the Batch-level Abundance-Dependent Missing-data Mechanism (BADMM). We introduce a new method- mixEMM-for analyzing iTRAQ/TMT data with batch effects and batch-level nonignorable missingness. The mixEMM method employs a linear mixed-effects model and explicitly models the batch effects and the BADMM. With simulation studies, we showed that, compared with existing approaches that utilize relative abundances and ignore the missing batches under the missing-completely-at-random assumption, the mixEMM method achieves more accurate parameter estimation and inference. We applied the method to an iTRAQ proteomics data from a breast cancer study and identified phosphopeptides differentially expressed between different breast cancer subtypes. The method can be applied to general clustered data with cluster-level nonignorable missing-data mechanisms.

Estimation and diagnostic for skew-normal partially linear models

ABSTRACTPartially linear models (PLMs) are an important tool in modelling economic and biometric data and are considered as a flexible generalization of the linear model by including a nonparametric component of some covariate into the linear predictor. Usually, the error component is assumed to follow a normal distribution. However, the theory and application (through simulation or experimentation) often generate a great amount of data sets that are skewed. The objective of this paper is to extend the PLMs allowing the errors to follow a skew-normal distribution [A. Azzalini, A class of distributions which includes the normal ones, Scand. J. Statist. 12 (1985), pp. 171–178], increasing the flexibility of the model. In particular, we develop the expectation-maximization (EM) algorithm for linear regression models and diagnostic analysis via local influence as well as generalized leverage, following [H. Zhu and S. Lee, Local influence for incomplete-data models, J. R. Stat. Soc. Ser. B 63 (2001), pp. 111–126]. A simulation study is also conducted to evaluate the efficiency of the EM algorithm. Finally, a suitable transformation is applied in a data set on ragweed pollen concentration in order to fit PLMs under asymmetric distributions. An illustrative comparison is performed between normal and skew-normal errors.

EM algorithm for longitudinal data with non-ignorable missing values: An application to health data

Longitudinal studies involves repeated observations over time on the same experimental units and missingness may occur in non-ignorable fashion. For such longitudinal missing data, a Markov model may be used to model the binary response along with a suitable non-response model for the missing portion of the data. It is of the primary interest to estimate the effects of covariates on the binary response. Similar model for such incomplete longitudinal data exists where estimation of the regression parameters are obtained using likelihood method by summing over all possible values of the missing responses. In this paper, we propose an expectation-maximization (EM) algorithm technique for the estimation of the regression parameters which is computationally simple and produces similar efficient estimates as compared to the existing complex method of estimation. A comparison of the existing and the proposed estimation methods has been made by analyzing the Health and Retirement Survey (HRS) data of United States.Bangladesh J. Sci. Res. 27(2): 133-142, December-2014

Feature projection k-NN classifier model for imbalanced and incomplete medical data

Many datasets, especially various historical medical data are incomplete. Various qualities of data can significantly hamper medical diagnosis and are bottlenecks of medical support systems. Nowadays, such systems are often used in medical diagnosis. Even great number of data can be unsuitable when data is imbalanced, missing or corrupted. In some cases these troubles can be overcome by machine learning algorithms designed for predictive modeling.Proposed approach was tested on real medical data and some benchmarks dataset form UCI repository. The liver fibrosis disease from a medical point of view is difficult to treatment and has a significant social and economic impact. Stages of liver fibrosis are diagnosed by clinical observation and evaluations, coupled with a so-called METAVIR rating scale. However, these methods may be insufficient, especially in the recognition of phase of the disease. This paper describes a newly developed algorithm to non-invasive fibrosis stage recognition using machine learning methods – a classification model based on feature projection k-NN classifier. This solution allows extracting data characteristics from the historical data which may be incomplete and may contain imbalance (unequal) sets of patients. Proposed novel solution is based on peripheral blood analysis without using any specialized biomarkers, and can be successfully included to medical diagnosis support systems and might be a powerful tool for effective estimation of liver fibrosis stages.