Mean Square Error Sense Research Articles

Online Data Dimensionality Reduction and Reconstruction Using Graph Filtering

Graphs have become a popular toolbox for capturing similarity relationships among data vectors in a variety of networks. Diffusion processes are used to model the data using the graph topology. A novel approach is put forth that exploits data similarity and diffusion models, in a graph, to improve upon the reconstruction mean-square error (MSE) performance of principal component analysis. The tasks of data dimensionality reduction and reconstruction are formulated as graph matrix filtering operations, that are utilized in a dimensionality reduction framework optimal in the MSE sense. The novel formulation is seeking MSE-optimal filter matrices that minimize the reconstruction MSE of all the graph data. Block coordinate descent techniques are employed in the graph spectral domain MSE cost to recursively determine the MSE-optimal dimensionality reducing and reconstruction filters. Online implementations are put forth to work with continuous streams of graph data, while almost sure convergence to a stationary point of the ensemble reconstruction MSE cost is established. Analysis of the reconstruction MSE reveals that the lowest MSE achieved by the novel approach is not larger than the standard PCA MSE, while there is a bound for the filter orders after which no more MSE reduction occurs. Extensive numerical tests using synthetic data, as well as real image datasets demonstrate the advantage of the novel graph-based framework over standard PCA and related methods.

Privacy With Estimation Guarantees

We study the central problem in data privacy: how to share data with an analyst while providing both privacy and utility guarantees to the user that owns the data. In this setting, we present an estimation-theoretic analysis of the privacy-utility trade-off (PUT). Here, an analyst is allowed to reconstruct (in a mean-squared error sense) certain functions of the data (utility), while other private functions should not be reconstructed with distortion below a certain threshold (privacy). We demonstrate how chi-square information captures the fundamental PUT in this case and provide bounds for the best PUT. We propose a convex program to compute privacy-assuring mappings when the functions to be disclosed and hidden are known a priori and the data distribution is known. We derive lower bounds on the minimum mean-squared error of estimating a target function from the disclosed data and evaluate the robustness of our approach when an empirical distribution is used to compute the privacy-assuring mappings instead of the true data distribution. We illustrate the proposed approach through two numerical experiments.

Recursive state estimation for linear systems with lossy measurements under time-correlated multiplicative noises

This paper is concerned with the recursive state estimation problem for a class of linear discrete-time systems with lossy measurements and time-correlated multiplicative noises (TCMNs). The lossy measurements result from one-step transmission delays and packet dropouts. Different from the traditional white multiplicative noises, TCMNs are included in the measurement model in order to reflect engineering practice. Utilizing the state augmentation approach, the system under investigation is first converted into a stochastic parameter system, and some new recursive terms (including the estimation for the product of state and multiplicative noises) are introduced to handle the difficulties caused by the TCMNs. Then, by the well-known projection theorem, recursive state estimation algorithms are developed in the sense of minimum mean-square error, which facilitate the design of the filter, the multi-step predictor and the smoothers. The proposed algorithms are explicitly dependant on the key system parameters including the covariances of the TCMNs, the occurrence probabilities of the transmission delays and the packet losses. Finally, simulation results illustrate the effectiveness of the presented estimation algorithms.

On linearized ridge logistic estimator in the presence of multicollinearity

Logistic Regression is a very popular method to model the dichotomous data. The maximum likelihood estimator (MLE) of unknown regression parameters of the logistic regression is not too accurate when multicollinearity exists among the covariates. It is well known that the presence of multicollinearity increases the variance of the MLE. To diminish the inflated mean square error (MSE) of the MLE due to the presence of multicollinearity, we proposed a new estimator designated as linearized ridge logistic estimator. The conditional superiority of the proposed estimator over the other existing estimators is derived theoretically and the optimal choice of shrinkage parameter is suggested. Monte Carlo simulations are performed to study the performance of the proposed estimator through MSE sense. Also, a numerical example is presented to support the results.

Extended-Object or Group-Target Tracking Using Random Matrix With Nonlinear Measurements

For extended-object/group-target tracking (EOT/ GTT), the random-matrix approach assumes that measurements are linear in the state and noise with a covariance being a random matrix to represent the object extension or target group. In practice, however, most measurements are nonlinear in the state and noise. This paper proposes a random-matrix approach for EOT/GTT using nonlinear measurements. First, a matched linearization (ML) is proposed to linearize nonlinear measurements. The linearized form has two parts. The first is linear in the state, and it is optimized in the sense of minimum mean square error (MMSE). The second part is linear in the extension-related noise with a preserved second moment, which is important since extension information is contained in the covariance of this noise. The linearized measurements can be incorporated into existing random-matrix algorithms after a simple conversion under certain conditions. Second, a variational Bayesian (VB) scheme is proposed for EOT/GTT using the linearized measurements. This approach can be generally applied no matter whether the linearized measurements are converted or not. The effectiveness of the proposed ML and VB approach is demonstrated by simulation results compared with existing random-matrix algorithms.

Indirect estimation of interregional freight flows with a real-valued genetic algorithm

This paper introduces a method for estimating the interregional transportation of certain commodities in those cases where the commodity flows are not readily available and only aggregated flows per origin-destination pair are provided. We use a doubly-constrained gravity model to find a matrix of aggregated flows that is as similar as possible to the available data, in the sense of the standardized root mean square error. This model is calibrated via a real-valued genetic algorithm that uses a combination of global and local searches to find a set of optimal parameters of the deterrence function under study in the gravity model. This method is introduced as an application to estimating the disaggregated flows of ten different products among the fifteen regions of peninsular Spain between 2007 and 2016. After testing several formulations, we conclude that an exponential deterrence function calibrated with data from 2010 is as effective to estimate the flows in this 10-year span as other more complex options, which emphasizes the time transferability of our model.

An Optimal Data Fusion Algorithm in the Presence of Unknown Cross Covariances

This paper presents an optimal data fusion formulation and algorithm in the sense of minimum mean-square error when some or all cross covariances are unknown. This algorithm is generic in that it is capable of processing any number of measurement vectors of any dimension with any pattern of unknown cross covariances. Closed-form solution is provided for the case when all cross covariances are unknown and all measurement covariance matrices are diagonal. Numerical projected subgradient optimal fusion algorithm is provided for the most generic case. The well known covariance intersection method is shown to have a weaker upper bound of this formulation.

On LMVDR Estimators for LDSS Models: Conditions for Existence and Further Applications

For linear discrete state-space models, under certain conditions, the linear least mean squares (LLMS) filter estimate has a recursive format, a.k.a. the Kalman filter (KF). Interestingly, the linear minimum variance distortionless response (LMVDR) filter, when it exists, shares exactly the same recursion as the KF, except for the initialization. If LMVDR estimators are suboptimal in mean-squared error sense, they do not depend on the prior knowledge on the initial state. Thus, the LMVDR estimators may outperform the usual LLMS estimators in case of misspecification of the prior knowledge on the initial state. In this perspective, we establish the general conditions under which existence of the LMVDRF is guaranteed. An immediate benefit is the introduction of LMVDR fixed-point and fixed-lag smoothers (and possibly other smoothers or predictors), which has not been possible so far. Indeed, the LMVDR fixed-point smoother can be used to compute recursively the solution of a generalization of the deterministic least-squares problem.

Mean-square performance of the modified frequency-domain block LMS algorithm

The mean weight vector of the normalized frequency-domain block least-mean-square (NFBLMS) algorithm cannot converge to the optimal solution in the mean-square error sense for the non-causal and under-modeling cases. A modified frequency-domain block least-mean-square (MFBLMS) algorithm was proposed to resolve this problem, which was claimed to have optimal steady-state performance. In this paper, we present a comprehensive statistical analysis of the MFBLMS algorithm in both the full- and under-modeling conditions. We first present the equivalent time-domain expressions for the update equation and the error vector of the MFBLMS, which allows us to carry out the performance analysis completely in the time domain. The analytical model for both the mean and mean-square performance of the MFBLMS are provided without assuming a specific input distribution, and the closed-form solution of the step-size bound is given. It is found that the upper step-size bound of the MFBLMS algorithm is much smaller than that of the NFBLMS algorithm for the correlated inputs, and the MFBLMS algorithm does not always achieve a better steady-state performance than the NFBLMS algorithm. Simulation results agree with our theoretical analysis quite well.

The r – d class estimator in generalized linear models: applications on gamma, Poisson and binomial distributed responses

ABSTRACTIn order to combat multicollinearity, the r – d class estimator was introduced in linear and binary logistic regression models. Since the generalized linear models (GLMs) are the models that include logistic regression model, Poisson regression model, etc., we introduce the iterative and first-order approximated r – d class estimator in GLMs. The sampling distribution of the class estimator at convergence is given and the test on the regression coefficients is provided. The properties of the first-order approximated r – d class estimator in GLMs are discussed, comparisons of the three constitute estimators in the sense of bias, variance and scalar mean square errors are done, and a cross-validation method and a mean square error method for the selection of the shrinkage parameter are given. Finally, a simulation study is conducted for Poisson response and two real data analyses are done for gamma and binomial response data to examine the performance of the first-order approximated r – d class estimator versus the first-order approximated maximum likelihood, Liu, PCR and ridge estimators in GLMs.

A further prediction method in linear mixed models: Liu prediction

We propose the Liu estimator and the Liu predictor via the penalized log-likelihood approach in linear mixed models when multicollinearity is present. The necessary and sufficient conditions for the superiority of the Liu predictor over the best linear unbiased predictor and the ridge predictor of linear combinations of fixed and random effects in the sense of matrix and scalar mean square errors are examined. Furthermore, the selection of the Liu biasing parameter is given and the findings are illustrated with both a real data set and a simulation study. The study show that the Liu estimator and predictor outperform the ridge estimator and predictor and the blue and blup in the sense of mean square error for large degree of correlation and the degree of supremacy of the Liu estimator and predictor over the ridge estimator and predictor and the blue and blup increase as the Liu biasing parameter decreases.

Sequential Fault-Tolerant Fusion Estimation for Multisensor Time-Varying Systems

In this paper, the impact of the fusion framework on the fault diagnosis process is discussed. The centralized fusion framework makes it difficult to locate and estimate the sensor fault. Based on the fault detection method, the sequential fusion framework could locate and estimate the sensor fault and realize the fault-tolerant estimation of system states. In the sense of minimum mean square error (MMSE), based on the sequential detection of bias fault of sensors, a sequential fault-tolerant fusion estimation approach is presented to estimate the sensor fault and the system state, simultaneously, optimally, in real time. Further, a novel alternate fault-tolerant fusion estimation method is proposed to alternately estimate the sensor fault and the system state, in the frame of sequential fusion. What is more, the equivalency of the two proposed methods is proved. And the feasibility and equivalency of them are also verified by the computer simulation.

Team-optimal online estimation of dynamic parameters over distributed tree networks

Abstract We study online parameter estimation over a distributed network, where the nodes in the network collaboratively estimate a dynamically evolving parameter using noisy observations. The nodes in the network are equipped with processing and communication capabilities and can share their observations or local estimates with their neighbors. The conventional distributed estimation algorithms cannot perform the team-optimal online estimation in the finite horizon global mean-square error sense (MSE). To this end, we present a team-optimal distributed estimation algorithm through the disclosure of local estimates for tracking an underlying dynamic parameter. We first show that the optimal estimation can be achieved through the diffusion of all the time stamped observations for any arbitrary network and prove that the team optimality through disclosure of local estimates is only possible for certain network topologies such as tree networks. We then derive an iterative algorithm to recursively calculate the combination weights of the disclosed information and construct the team-optimal estimate for each time step. Through series of simulations, we demonstrate the superior performance of the proposed algorithm with respect to the state-of-the-art diffusion distributed estimation algorithms regarding the convergence rate and the finite horizon MSE levels. We also show that while conventional distributed estimation schemes cannot track highly dynamic parameters, through optimal weight and estimate construction, the proposed algorithm presents a stable MSE performance.

Investigating the two parameter analysis of Lipovetsky for simultaneous systems

Two stage least squares regression analysis is the most practical statistical technique that is used in the analysis of structural equations. This study demonstrates the utility of some biased estimation methods when there is inherent multicollinearity among exogenous variables of a simultaneous system. Two stage ridge estimator is one of the biased estimators encountered in this model. In the light of the two parameter ridge estimator of Lipovetsky and Conklin (Appl Stoch Mod Bus Ind 21:525–540, 2005) in linear regression model, we propose two stage two parameter ridge estimator in the simultaneous equations model. Due to the fact that Lipovetsky and Conklin’s two parameter ridge estimator is superior to the ridge and ordinary least squares estimators, we expect the superiority of our new two stage two parameter ridge estimator to the two stage ridge estimator and the two stage least squares estimator. This superiority is proved theoretically. A data analysis involved with Klein Model I is examined to illustrate the mastery of the two stage two parameter ridge estimator in the sense of mean square error. In addition, an extensive Monte Carlo experiment is conducted. It is recommended that investigators who develop simultaneous equations models may perform the new estimator to eliminate the effect of multicollinearity on parameter estimates.

Distributed prediction of solar farm status by probabilistic fuzzy opinion formation model

Although photovoltaic arrays are a highly reliable solution to generating energy, the full scale use of these integrated arrays presents formidable challenges since their future generation carries high uncertainty. In this paper, the success of a large-scale solar farm in generating different percentages of its committed value is forecasted by a new opinion formation model. In the proposed model, each node uses local generation data, transforms them to a fuzzy set called perception, and evolves its fuzzy opinion about the entire farm's future generation by using this perception and negotiating with other nodes. As a result, the fuzzy predictions, which can be used in fuzzy energy management systems, are achieved with simple, distributed, and scalable computations. As a case study, Belgium is considered here as the entire solar farm with its provinces as the local generation units. To evaluate the predicted fuzzy opinions, we extract the lower/upper and point-wise predictions from them and compare these extracted values with the outputs of central data-driven forecasting algorithms. Furthermore, to evaluate the proposed aggregation mechanism, the obtained opinions are compared with the fusion of perceptions by conventional fusion algorithms. The results show that in both experiments, the proposed strategy performs better than its competitors in the sense of maximum absolute percentage error and mean-squared error.

The optimal extended balanced loss function estimators

We derive the optimal heterogeneous, homogeneous and homogeneous unbiased estimators of the coefficient vector in a linear regression model under the extended balanced loss function of Shalabh et al. (2009). Risk functions and optimal predictors of the new estimators are evaluated and comparisons among the estimators are made with respect to the extended balanced loss function. Some of the theoretical results are illustrated by a numerical example. Moreover, the behavior of the proposed estimators is studied via a Monte-Carlo experiment in the sense of mean square error.

Estimation of Time-Varying Channels in MIMO Two-Way Multi-Relay Systems

In this paper, we study time-varying channel estimation and optimal pilot design for multiple-input multiple-output two-way multi-relay systems in the sense of minimizing the total estimation mean square error (MSE) under the power constraints at two source nodes and multiple relays. A particularly challenging issue in the optimal pilot design is to completely eliminate the inter-link interference while most efficiently using the channel resource. To address this challenging issue, we consider a pilot design approach based on the phase rotation technique at multiple relays. First, we establish various optimality conditions for the pilot design in the sense of the minimum total MSE. Then, we propose the optimal pilot scheme by designing the pilot signals, phase rotations, and pilot symbol positions to satisfy the established optimality conditions. Finally, we propose the asymptotically optimal pilot scheme in the high signal-to-noise ratio (SNR) regime with low complexity. Simulation results show that the proposed schemes significantly outperform the existing schemes in terms of the channel estimation and the bit error rate, and the proposed asymptotically optimal scheme provides the near-optimal performance even in the low to moderate SNR range.

A Framework of Covariance Projection on Constraint Manifold for Data Fusion.

A general framework of data fusion is presented based on projecting the probability distribution of true states and measurements around the predicted states and actual measurements onto the constraint manifold. The constraint manifold represents the constraints to be satisfied among true states and measurements, which is defined in the extended space with all the redundant sources of data such as state predictions and measurements considered as independent variables. By the general framework, we mean that it is able to fuse any correlated data sources while directly incorporating constraints and identifying inconsistent data without any prior information. The proposed method, referred to here as the Covariance Projection (CP) method, provides an unbiased and optimal solution in the sense of minimum mean square error (MMSE), if the projection is based on the minimum weighted distance on the constraint manifold. The proposed method not only offers a generalization of the conventional formula for handling constraints and data inconsistency, but also provides a new insight into data fusion in terms of a geometric-algebraic point of view. Simulation results are provided to show the effectiveness of the proposed method in handling constraints and data inconsistency.

Quantized Kalman Filter Tracking in Directional Sensor Networks

Moving target tracking in directional sensor networks has recently attracted attention by right of special directional sensing features. Unlike omnidirectional sensors, the directional sensor senses the target only in the direction of its orientation. It can provide quantized direction that indicates the presence or absence of the target in the sensing field, rather than just the analog measurement of sensing signal with respect to the detected target. A quantized Kalman filter (QKF) based on both quantized directions and analog ranging measurements is derived in the minimum mean-square error (MMSE) sense. Its performances of mean square estimation error (MSE) and complexity are also analyzed. Then, a reduced-complexity QKF of high-accuracy is pursued to facilitate its implementation. It is proved that the QKF yields a smaller MSE than the traditional extended Kalman filter (EKF) merely based on analog measurements. The posterior Cram $\acute{e}$ r-Rao lower bound (PCRLB) is introduced as the performance measure. The performance advantages of the proposed QKF are demonstrated using Monte Carlo simulations in a target tracking application using ultrasonic ranging sensors.

Kalman Filter Reinforced by Least Mean Square for Systems with Unknown Inputs

This paper addresses the state estimation problem of linear discrete-time time-varying stochastic systems with unknown inputs (UIs). It is shown that the globally optimal unbiased minimum-variance filters may not satisfy the minimum-variance property, and hence they cannot eliminate noises appropriately. If this is the case, the well-known Kalman filter may give a better solution, which however may also not be the best one due to that the imbedded unknown input model may not be practical. To remedy the filtering degradation problem, a robust filter named as the KFLMS, which has good noise rejection property for such systems, is developed in this paper, where the UI estimates are obtained by using least mean square algorithm and the state estimation is achieved via the previous proposed two-stage Kalman filtering approach. Numerical examples are provided to show the effectiveness of the proposed results. Specifically, simulation results illustrate the goodness of the new method in the sense of lower root mean square error and better noise rejection property.