Linear Least-squares Estimation Research Articles

Least-squares linear estimation of signals from observations with Markovian delays

The least-squares linear estimation of signals from randomly delayed measurements is addressed when the delay is modeled by a homogeneous Markov chain. To estimate the signal, recursive filtering and fixed-point smoothing algorithms are derived, using an innovation approach, assuming that the covariance functions of the processes involved in the observation equation are known. Recursive formulas for filtering and fixed-point smoothing error covariance matrices are obtained to measure the goodness of the proposed estimators.

Pilot Matrix Design for Estimating Cascaded Channels in Two-Hop MIMO Amplify-and-Forward Relay Systems

In this paper, we consider a two-hop multi-input-multi-output (MIMO) amplify-and-forward (AF) relay system consisting of a source node (SN), a destination node (DN), and a relay node (RN) that simply amplifies and forwards its received signal to the DN without any further processing. In this system, the overall channel from the SN to the DN is a cascade of the backward relay channel over the SN-RN hop, the amplifying matrix at the RN, and the forward relay channel over the RN-DN hop. We investigate the estimation of the two cascaded relay channels at the DN based on the predefined amplifying matrix applied at the RN and the corresponding overall channel obtained through the conventional channel estimation algorithms with the help of pilots transmitted by the SN. In particular, we find necessary and sufficient conditions on the pilot amplifying matrix sequence at the RN to ensure feasible relay channel estimation at the DN. Based on these conditions, we present rules to design diagonal or quasi-diagonal pilot amplifying matrices so that the cascaded relay channels can be estimated with minimum complexity at the RN. In the presence of imperfect overall channel state information at the DN, we further develop the approximate linear least-square estimation of the relay channels based on the designed pilot matrix sequence and demonstrate its performance by simulation results.

Fundamental limits and improved algorithms for linear least‐squares wireless position estimation

ABSTRACTIn this paper, theoretical lower bounds on performance of linear least‐squares (LLS) position estimators are obtained, and performance differences between LLS and nonlinear least‐squares (NLS) position estimators are quantified. In addition, two techniques are proposed in order to improve the performance of the LLS approach. First, a reference selection algorithm is proposed to optimally select the measurement that is used for linearizing the other measurements in an LLS estimator. Then, a maximum likelihood approach is proposed, which takes correlations between different measurements into account in order to reduce average position estimation errors. Simulations are performed to evaluate the theoretical limits and to compare performance of various LLS estimators. Copyright © 2010 John Wiley & Sons, Ltd.

Resolution improvement in vertical-cavity-surface-emitting-laser diode interferometry based on linear least-squares estimation of phase gradients of phase-locked fringes

A frequency scanning interferometer using a vertical-cavity-surface-emitting-laser diode (VCSEL) capable of wide-frequency scanning has been constructed for precise distance measurements. The frequency scanning velocity of the VCSEL has been stabilized by the phase-locked loop technique, which enables us to precisely determine the phase gradient of the scanned interference fringe by linear least-squares fitting. In our test measurements, the absolute lengths from 8 to 14mm have been measured with a resolution of nearly sub-micrometer. Compared with a conventional frequency scanning interferometer using a Fabry-Perot laser diode, the resolution of length measurement has been improved by two orders of magnitude.

Linear and quadratic estimation using uncertain observations from multiple sensors with correlated uncertainty

In this paper, filtering algorithms are derived for the least-squares linear and quadratic estimation problems in linear systems with uncertain observations coming from multiple sensors with different uncertainty characteristics. It is assumed that, at each sensor, the state is measured in the presence of additive white noise and that the Bernoulli random variables describing the uncertainty are correlated at consecutive sampling times but independent otherwise. The least-squares linear estimation problem is solved by using an innovation approach, and the quadratic estimation problem is reduced to a linear estimation one in a suitable augmented system. The performance of the linear and quadratic estimators is illustrated by a numerical simulation example wherein a scalar signal is estimated from correlated uncertain observations coming from two sensors with different uncertainty characteristics.

A New Estimation Algorithm from Measurements with Multiple-Step Random Delays and Packet Dropouts

The least-squares linear estimation problem using covariance information is addressed in discrete-time linear stochastic systems with bounded random observation delays which can lead to bounded packet dropouts. A recursive algorithm, including the computation of predictor, filter, and fixed-point smoother, is obtained by an innovation approach. The random delays are modeled by introducing some Bernoulli random variables with known distributions in the system description. The derivation of the proposed estimation algorithm does not require full knowledge of the state-space model generating the signal to be estimated, but only the delay probabilities and the covariance functions of the processes involved in the observation equation.

Analysis of Fluorescence Lifetime Imaging Microscopy (FLIM) Data

A novel Fluorescence Lifetime Imaging Microscopy (FLIM) deconvolution method based on the linear expansion of fluorescence decays on a set of orthonormal Laguerre functions was recently proposed. The Laguerre deconvolution method applies linear least-square estimation to estimate the expansion coefficients of all pixel decays simultaneously, performing at least two orders of magnitude faster than the other algorithms. In the original Laguerre FLIM deconvolution implementation, however, the Laguerre parameter α is selected using a heuristic approach, making it unsuitable for online applications. In this study, we present a fully automated implementation of the Laguerre FLIM deconvolution, whereby the Laguerre parameter α is treated as a free parameter within a nonlinear least-squares optimization scheme. The performance of this method has been successfully validated on simulated data, and experimental FLIM images of standard fluorescent dyes and endogenous tissue fluorescence. The main advantage of the proposed method is that it does not require any user intervention for tuning up the deconvolution process. Thus, we believe this method will facilitate the translation of FLIM to online applications, including real-time clinical diagnosis.

Segmentation of the pelvic girdle in pediatric computed tomographic images

Identification, localization, and segmentation of the thoracic, abdominal, and pelvic organs are important steps in computer-aided diagnosis, treatment planning, landmarking, and content-based retrieval of biomedical images. In this context, to aid the identification of the lower abdominal organs, to assist in image-guided surgery or treatment planning, to separate the abdominal cavity from the lower pelvic region, and to improve the process of localization of abdominal pathology, we propose methods to identify and segment automatically the pelvic girdle in pediatric computed tomographic (CT) images. The opening-by-reconstruction procedure was used for segmentation of the pelvic girdle. The methods include procedures to represent the pelvic surface by a quadratic model using linear least-squares estimation and to refine the model using deformable contours. The result of segmentation of the pelvic girdle was assessed quantitatively and qualitatively by comparing with the segmentation performed independently by a radiologist. On the basis of quantitative analysis with 13 CT exams of six patients, including a total of 277 slices with the pelvis, the average Hausdorff distance was determined to be 5.95 mm, and the average mean distance to the closest point (MDCP) was 0.53 mm. The average MDCP is comparable to the size of one pixel, on the average.

Blended learning fitting algorithm for polarization curves of fuel cells

Fuel cell polarization curves, characterized by nonlinear models and the parameters of which are time-consuming to be identified, can represent fuel cell performance but will alter as the fuel cell degrades. For getting the information on degradation in time, a less time-consuming and an easily programmed algorithm, based on blended learning technique and linear least square estimation (LSE), is proposed to fit polarization curves obtained from the fuel cell systems. Simulations show that the proposed algorithm, compared with classical nonlinear LSE algorithms, converges much faster, features better extrapolation and less average quadratic error, and is easy to be programmed by C language. Therefore, the algorithm is a good option not only for fitting the polarization curves but also for implementation in embedded systems.

Chronobiological analysis techniques. Application to blood pressure

Most variables of clinical interest show predictable changes with different frequencies, mainly, but not exclusively, along the rest-activity cycle (circadian variation). Methods of linear least-squares estimation have been designed for the detection of periodic components in sparse and noisy time series (as they are usually present in clinical situations). They include the single and population-mean cosinor methods. In cases where more than one period is statistically significant over the span of time investigated, or when the waveform is non-sinusoidal, the use of multiple components analysis to fit a model consisting of several cosine functions (harmonics or not from a given fundamental period) is recommended. We describe these methods, from the characterization of the underlying models to the process of parameter estimation. As an application example, we describe the modelling of the circadian variation of blood pressure (BP). In most individuals, BP presents a morning increase, a small postprandial valley and a deeper descent during nocturnal rest. This pattern can be easily modelled by means of a model with periods of 24 and 12 hours. Individuals that differ from this model might be considered to present increased cardiovascular risk.

Signal estimation based on covariance information from observations featuring correlated uncertainty and coming from multiple sensors

The linear least-squares estimation problem of signals from observations coming from multiple sensors is addressed when there is a non-zero probability that each observation does not contain the signal to be estimated ( uncertain observations). At each sensor, this uncertainty in the observations is modeled by a sequence of Bernoulli random variables correlated at consecutive sampling times. To estimate the signal, recursive filtering and (fixed-point and fixed-interval) smoothing algorithms are derived without requiring the knowledge of the signal state-space model but only the means and covariance functions of the processes involved in the observation equations, the uncertainty probabilities and the correlation between the variables modeling the uncertainty. To measure the estimation accuracy, recursive expressions for the estimation error covariance matrices are also proposed. The theoretical results are illustrated by a numerical simulation example where a signal is estimated from observations featuring correlated uncertainty and coming from two sensors with different uncertainty characteristics.

Modeling of a 6/4 Switched Reluctance Motor Using Adaptive Neural Fuzzy Inference System

The magnetic saturation and strong nonlinearity of switched reluctance machines (SRMs) makes it very difficult to derive a comprehensive mathematical model for the behavior of the machine. We propose a new method of modeling SRMs based on an adaptive neural fuzzy inference system (ANFIS). First, we use an indirect method to measure the static flux linkage and then use the co-energy method (via the principle of virtual displacement) to calculate the torque characteristics from data on flux linkage versus current and rotor position. A hybrid learning algorithm, which combines the back propagation algorithm and the linear least-squares estimation algorithm, identifies the parameters of the ANFIS. After training, the ANFIS flux linkage model and ANFIS torque model are in excellent agreement with experimental flux linkage measurements and the calculated torque data. Finally, we use an ANFIS current model and an ANFIS torque model to study SRM dynamic performance. The accuracy of the model was evaluated by comparison to laboratory measurements of the machine's current-speed and torque-speed characteristics. The model is quite accurate.

On channel estimation and optimal training design for amplify and forward relay networks

In this paper, we provide a complete study on the training based channel estimation issues for relay networks that employ the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">amplify-and-forward</i> (AF) transmission scheme. We first point out that separately estimating the channel from source to relay and relay to destination suffers from many drawbacks. Then we provide a new estimation scheme that directly estimates the overall channels from the source to the destination. The proposed channel estimation well serves the AF based space time coding (STC) that was recently developed. There exists many differences between the proposed channel estimation and that in the traditional single input single out (SISO) and multiple input single output (MISO) systems. For example, a relay must linearly precode its received training sequence by a sophisticatedly designed matrix in order to minimize the channel estimation error. Besides, each relay node is individually constrained by a different power requirement because of the non-cooperation among all relay nodes. We study both the linear least-square (LS) estimator and the minimum mean-square-error (MMSE) estimator. The corresponding optimal training sequences, as well as the optimal preceding matrices are derived from an efficient convex optimization process.

Recursive fixed-point smoothing algorithm from covariances based on uncertain observations with correlation in the uncertainty

This paper considers the least-squares linear estimation problem of a discrete-time signal from noisy observations in which the signal can be randomly missing. The uncertainty about the signal being present or missing at the observations is characterized by a set of Bernoulli variables which are correlated when the difference between times is equal to a certain value m. The marginal distribution of each one of these variables, specified by the probability that the signal exists at each observation, as well as their correlation function, are known. A linear recursive filtering and fixed-point smoothing algorithm is obtained using an innovation approach without requiring the state-space model generating the signal, but just the covariance functions of the processes involved in the observation equation.

On the estimation of a large sparse Bayesian system: The Snaer program

The Snaer program calculates the posterior mean and variance of variables on some of which we have data (with precisions), on some we have prior information (with precisions), and on some prior indicator ratios (with precisions) are available. The variables must satisfy a number of exact restrictions. The system is both large and sparse. Two aspects of the statistical and computational development are a practical procedure for solving a linear integer system, and a stable linearization routine for ratios. The numerical method for solving large sparse linear least-squares estimation problems is tested and found to perform well, even when the n × k design matrix is large ( n k = O ( 10 8 ) ).

Linear least-squares estimation based on covariances from multiple correlated uncertain observations

In this paper, the linear least-squares estimation problem of signals from correlated uncertain observations coming from multiple sensors is addressed. It is assumed that, at each sensor, the signal is measured in the presence of additive white noise and that the uncertainty in the observations is characterized by a set of Bernoulli random variables which are only correlated at consecutive time instants. Assuming that the probability and correlation of such variables are not necessarily the same for all the sensors, a recursive filtering and fixed-point smoothing algorithm is proposed. The derivation of such algorithm does not require the knowledge of the signal state-space model, but only the covariance functions of the processes involved in the observation equation of each sensor, as well as the probability and correlation of the Bernoulli variables modeling the uncertainty. Recursive expressions for the estimation error covariance matrices are also provided, and the performance of the estimators is illustrated by a numerical simulation example wherein a signal is estimated from correlated uncertain observations coming from two sensors with different uncertainty characteristics.

Linear and quadratic least-squares estimation using measurements with correlated one-step random delay

This paper considers the linear and quadratic least-squares estimation of a discrete-time signal from observations randomly delayed by one sampling time such that the delay at a given time instant depends on a previous delay. It is assumed that the signal is measured with an additive white noise and that the delay in the observations is characterized by a set of Bernoulli variables which are correlated when the difference between times is equal to a certain value m. Linear and quadratic recursive filtering and fixed-point smoothing algorithms for such a class of models are constructed using an innovation approach; they do not require full knowledge of the state-space model for the signal process, but just the moments up to the fourth order of the signal (admitting a separable form) and the observation noise, as well as the probability and correlation of the Bernoulli variables modelling the delay. Recursive expressions for the estimation error covariance matrices are also given, and the performance of the different estimators is illustrated by means of a numerical example.

Linear system identification using proper orthogonal decomposition

A least-square-based method to identify the system matrices of linear dynamical systems is proposed. The primary focus is on the identification of a reduced-order model of the system operating in the mid-frequency range of vibration. Proper orthogonal decomposition (POD) is used for the model reduction. Such reduced-order model circumvents the limitations of traditional modal analysis which, although well-adapted in the low-frequency range, is prone to computational and conceptual difficulties in the mid-frequency range. The inverse problem involving the identification of the mass, damping and stiffness matrices is posed in the framework of a linear least-square estimation problem. To achieve this objective, Kronecker algebra is aptly exploited for a concise mathematical formulation to identify these matrix-valued variables. Tikhonov regularisation is used to satisfy the symmetry property of the system matrices. The application of the proposed methodology is demonstrated using an example of multiple degree-of-freedom discrete linear dynamical system. The robustness of the new methodology is investigated using a noise sensitivity study.

Least-squares methods for identifying biochemical regulatory networks from noisy measurements.

BackgroundWe consider the problem of identifying the dynamic interactions in biochemical networks from noisy experimental data. Typically, approaches for solving this problem make use of an estimation algorithm such as the well-known linear Least-Squares (LS) estimation technique. We demonstrate that when time-series measurements are corrupted by white noise and/or drift noise, more accurate and reliable identification of network interactions can be achieved by employing an estimation algorithm known as Constrained Total Least Squares (CTLS). The Total Least Squares (TLS) technique is a generalised least squares method to solve an overdetermined set of equations whose coefficients are noisy. The CTLS is a natural extension of TLS to the case where the noise components of the coefficients are correlated, as is usually the case with time-series measurements of concentrations and expression profiles in gene networks.ResultsThe superior performance of the CTLS method in identifying network interactions is demonstrated on three examples: a genetic network containing four genes, a network describing p53 activity and mdm2 messenger RNA interactions, and a recently proposed kinetic model for interleukin (IL)-6 and (IL)-12b messenger RNA expression as a function of ATF3 and NF-κB promoter binding. For the first example, the CTLS significantly reduces the errors in the estimation of the Jacobian for the gene network. For the second, the CTLS reduces the errors from the measurements that are corrupted by white noise and the effect of neglected kinetics. For the third, it allows the correct identification, from noisy data, of the negative regulation of (IL)-6 and (IL)-12b by ATF3.ConclusionThe significant improvements in performance demonstrated by the CTLS method under the wide range of conditions tested here, including different levels and types of measurement noise and different numbers of data points, suggests that its application will enable more accurate and reliable identification and modelling of biochemical networks.

Training-based MIMO channel estimation: a study of estimator tradeoffs and optimal training signals

In this paper, we study the performance of multiple-input multiple-output channel estimation methods using training sequences. We consider the popular linear least squares (LS) and minimum mean-square-error (MMSE) approaches and propose new scaled LS (SLS) and relaxed MMSE techniques which require less knowledge of the channel second-order statistics and/or have better performance than the conventional LS and MMSE channel estimators. The optimal choice of training signals is investigated for the aforementioned techniques. In the case of multiple LS channel estimates, the best linear unbiased estimation (BLUE) scheme for their linear combining is developed and studied.