Regularized Least-squares Approach Research Articles

A Distributed Optimization Scheme for State Estimation of Nonlinear Networks With Norm-Bounded Uncertainties

This article investigates state estimation for a class of complex networks, in which the dynamics of each node is subject to Gaussian noise, system uncertainties, and nonlinearities. Based on a regularized least-squares approach, the estimation problem is reformulated as an optimization problem, solving for a solution in a distributed way by utilizing a decoupling technique. Then, based on this solution, a class of estimators is designed to handle the system dynamics and constraints. A novel feature of this design lies in the unified modeling of uncertainties and nonlinearities, the decoupling of nodes, and the construction of recursive approximate covariance matrices for the optimization problem. Furthermore, the feasibility of the proposed estimators and the boundedness of mean-square estimation errors are ensured under a developed criterion, which is easier to check than some typical estimation strategies including the linear matrix inequalities-based and the variance-constrained ones. Finally, the effectiveness of the theoretical results is verified by a numerical simulation.

Nonlinear estimation of pressure projection of ultrasound fields in background-oriented schlieren imaging.

Background-oriented schlieren imaging is a recently proposed method for measuring projections of ultrasound fields. The method is based on observing deflection of light in a heterogeneous refractive index field that is induced by ultrasound via an acousto-optic effect. The deflection of light manifests as apparent perturbations in an imaged target, forming a potential flow estimation problem. In this work, the potential flow approach is formulated as a nonlinear regularized least-squares approach to alleviate limitations of approaches that linearize the problem. The nonlinear approach is shown to outperform the linear one when estimating projections of medically relevant ultrasound fields.

Distributed state estimation for uncertain linear systems: A regularized least-squares approach

This paper addresses the state estimation problem for a discrete-time uncertain system with a network of sensors, where the system is not necessarily observable by each sensor and deterministic uncertainties exist in the system matrices. A new robust estimator is designed for each sensor, using only its own and neighbor’s information, which is fully distributed. Moreover, a novel information fusion strategy is developed to guarantee the estimation performance, based on the collective observability of the sensor network, which greatly relaxes the technical assumption of the proposed estimator. Theoretically, it can be ensured that if the observed system is time-varying, the gains of the estimator will be bounded. Furthermore, if the system is time-invariant, these gains will be convergent. Subsequently, the estimation error covariance will be ultimately bounded if the observed system is quadratically bounded. In the end, the superiority of the proposed robust distributed state estimation algorithm is illustrated by several numerical simulation examples.

An Atlas of Coronal Electron Density at 5R⊙. II. A Spherical Harmonic Method for Density Reconstruction

Abstract This is the second of a series of three papers that present a methodology with the aim of creating a set of maps of the coronal density over a period of many years. This paper describes a method for reconstructing the coronal electron density based on spherical harmonics. By assuming a radial structure to the corona at the height of interest, line-of-sight integrations can be made individually on each harmonic basis prior to determining coefficients, i.e., the computationally expensive integrations are calculated only once during initialization. This approach reduces the problem to finding the set of coefficients that best match the observed brightness using a regularized least-squares approach and is very efficient. The method is demonstrated on synthetic data created from both a simple and an intricate coronal density model. The quality of reconstruction is found to be reasonable in the presence of noise and large gaps in the data. The method is applied to both Large Angle and Spectrometric Coronagraph Experiment C2 and Solar Terrestrial Relations Observatory Cor2 coronagraph observations from 2009 March 20, and the results from both spacecraft compared. Future work will apply the method to large data sets.

Identification of conductivity in inhomogeneous orthotropic media

Purpose The purpose of this paper is to solve numerically the identification of the thermal conductivity of an inhomogeneous and possibly anisotropic medium from interior/internal temperature measurements. Design/methodology/approach The formulated coefficient identification problem is inverse and ill-posed, and therefore, to obtain a stable solution, a non-linear regularized least-squares approach is used. For the numerical discretization of the orthotropic heat equation, the finite-difference method is applied, while the non-linear minimization is performed using the MATLAB toolbox routine lsqnonlin. Findings Numerical results show the accuracy and stability of solution even in the presence of noise (modelling inexact measurements) in the input temperature data. Research limitations/implications The mathematical formulation uses temporal temperature measurements taken at many points inside the sample, and this may be too much information that is provided to identify a space-wise dependent only conductivity tensor. Practical implications As noisy data are inverted, the paper models real situations in which practical temperature measurements recorded using thermocouples are inherently contaminated with random noise. Social implications The identification of the conductivity of inhomogeneous and orthotropic media will be of great interest to the inverse problems community with applications in geophysics, groundwater flow and heat transfer. Originality/value The current investigation advances the field of coefficient identification problems by generalizing the conductivity to be anisotropic in addition of being heterogeneous. The originality lies in performing, for the first time, numerical simulations of inversion to find the orthotropic and inhomogeneous thermal conductivity from noisy temperature measurements. Further value and physical significance are brought in by determining the degree of cure in a resin transfer molding process, in addition to obtaining the inhomogeneous thermal conductivity of the tested material.

Automatic alignment for three-dimensional tomographic reconstruction

In tomographic reconstruction, the goal is to reconstruct an unknown object from a collection of line integrals. Given a complete sampling of such line integrals for various angles and directions, explicit inverse formulas exist to reconstruct the object. Given noisy and incomplete measurements, the inverse problem is typically solved through a regularized least-squares approach. A challenge for both approaches is that in practice the exact directions and offsets of the x-rays are only known approximately due to, e.g. calibration errors. Such errors lead to artifacts in the reconstructed image. In the case of sufficient sampling and geometrically simple misalignment, the measurements can be corrected by exploiting so-called consistency conditions. In other cases, such conditions may not apply and we have to solve an additional inverse problem to retrieve the angles and shifts. In this paper we propose a general algorithmic framework for retrieving these parameters in conjunction with an algebraic reconstruction technique. The proposed approach is illustrated by numerical examples for both simulated data and an electron tomography dataset.

Comparison of Regularized Least-Squares Approach and Predictive Error Method: Case Studies * *This work is supported by the National Science Foundation of China (No. 61273191) and by 973 Program of China (No. 2012CB720500). Corresponding author Yucai Zhu. Tel. +15381085960. Fax +0571-87953353.

Abstract There are recently studies on linear system identification with high order finite impulse response (FIR) models using the regularized least-squares approach. The regularized approach allows some bias to reduce variance so that the error of the regularized FIR model is minimized. This paper concerns the comparison between the regularized least-square approach and the traditional predictive error method (PEM), using a number of case studies. More cases are included than those used by the authors of the regularized approach, for example, non-white input signal, non-white output noise, low-order systems and closed-loop tests. In most cases PEM outperforms the regularized approach.

A Fast Active Set Block Coordinate Descent Algorithm for $\ell_1$-Regularized Least Squares

The problem of finding sparse solutions to underdetermined systems of linear equations arises in several applications (e.g. signal and image processing, compressive sensing, statistical inference). A standard tool for dealing with sparse recovery is the $\ell_1$-regularized least-squares approach that has been recently attracting the attention of many researchers. In this paper, we describe an active set estimate (i.e. an estimate of the indices of the zero variables in the optimal solution) for the considered problem that tries to quickly identify as many active variables as possible at a given point, while guaranteeing that some approximate optimality conditions are satisfied. A relevant feature of the estimate is that it gives a significant reduction of the objective function when setting to zero all those variables estimated active. This enables to easily embed it into a given globally converging algorithmic framework. In particular, we include our estimate into a block coordinate descent algorithm for $\ell_1$-regularized least squares, analyze the convergence properties of this new active set method, and prove that its basic version converges with linear rate. Finally, we report some numerical results showing the effectiveness of the approach.

Distributed iterative thresholding for ℓ0/ℓ1-regularized linear inverse problems

The $\ell _{0}/\ell _{1}$ -regularized least-squares approach is used to deal with linear inverse problems under sparsity constraints, which arise in mathematical and engineering fields. In particular, multiagent models have recently emerged in this context to describe diverse kinds of networked systems, ranging from medical databases to wireless sensor networks. In this paper, we study methods for solving $\ell _{0}/\ell _{1}$ -regularized least-squares problems in such multiagent systems. We propose a novel class of distributed protocols based on iterative thresholding and input driven consensus techniques, which are well-suited to work in-network when the communication to a central processing unit is not allowed. Estimation is performed by the agents themselves, which typically consist of devices with limited computational capabilities. This motivates us to develop low-complexity and low-memory algorithms that are feasible in real applications. Our main result is a rigorous proof of the convergence of these methods in regular networks. We introduce a suitable distributed, regularized, least-squares functional, and we prove that our algorithms reach their minima using results from dynamical systems theory. Furthermore, we propose numerical comparisons with the alternating direction method of multipliers and the distributed subgradient methods, in terms of performance, complexity, and memory usage. We conclude that our techniques are preferable for their good memory-accuracy tradeoff.

A variable fixing version of the two-block nonlinear constrained Gauss–Seidel algorithm for $$\ell _1$$ ℓ 1 -regularized least-squares

The problem of finding sparse solutions to underdetermined systems of linear equations is very common in many fields as e.g. signal/image processing and statistics. A standard tool for dealing with sparse recovery is the $$\ell _1$$ l 1 -regularized least-squares approach that has recently attracted the attention of many researchers. In this paper, we describe a new version of the two-block nonlinear constrained Gauss---Seidel algorithm for solving $$\ell _1$$ l 1 -regularized least-squares that at each step of the iteration process fixes some variables to zero according to a simple active-set strategy. We prove the global convergence of the new algorithm and we show its efficiency reporting the results of some preliminary numerical experiments.

Implementation of algorithms for tuning parameters in regularized least squares problems in system identification

There has been recently a trend to study linear system identification with high order finite impulse response (FIR) models using the regularized least-squares approach. One key of this approach is to solve the hyper-parameter estimation problem that is usually nonconvex. Our goal here is to investigate implementation of algorithms for solving the hyper-parameter estimation problem that can deal with both large data sets and possibly ill-conditioned computations. In particular, a QR factorization based matrix-inversion-free algorithm is proposed to evaluate the cost function in an efficient and accurate way. It is also shown that the gradient and Hessian of the cost function can be computed based on the same QR factorization. Finally, the proposed algorithm and ideas are verified by Monte-Carlo simulations on a large data-bank of test systems and data sets.

Convergence and error analysis of a numerical method for the identification of matrix parameters in elliptic PDEs

We analyze a numerical method for solving the inverse problem of identifying the diffusion matrix in an elliptic PDE from distributed noisy measurements. We use a regularized least-squares approach in which the state equations are given by a finite element discretization of the elliptic PDE. The unknown matrix parameters act as control variables and are handled with the help of variational discretization as introduced in (Hinze M 2005 Comput. Optim. Appl. 30 45–61). For a suitable coupling of Tikhonov regularization parameter, finite element grid size and noise level we are able to prove L2-convergence of the discrete solutions to the unique norm-minimal solution of the identification problem; corresponding convergence rates can be obtained provided that a suitable projected source condition is fulfilled. Finally, we present a numerical experiment which supports our theoretical findings.

Leave-one-out prediction error of systolic arterial pressure time series under paced breathing

In this paper, we consider systolic arterial pressure time series from healthy subjects and chronic heart failure patients, undergoing paced respiration, and show that different physiological states and pathological conditions may be characterized in terms of predictability of time series signals from the underlying biological system. We model time series by the regularized least-squares approach and quantify predictability by the leave-one-out error. We find that the entrainment mechanism connected to paced breath, that renders the arterial blood pressure signal more regular and thus more predictable, is less effective in patients, and this effect correlates with the seriousness of the heart failure. Using a Gaussian kernel, so that all orders of nonlinearity are taken into account, the leave-one-out error separates controls from patients (probability less than 10−7), and alive patients from patients for whom cardiac death occurred (probability less than 0.01).

Dynamic field map estimation using a spiral-in/spiral-out acquisition.

The long readout times of single-shot acquisitions and the high field strengths desired for functional MRI (fMRI) using blood oxygenation level-dependent (BOLD) contrast make functional scans sensitive to magnetic field inhomogeneity. If it is not corrected during image reconstruction, field inhomogeneity can cause geometric distortions in the images when Cartesian k-space trajectories are used or blurring with spiral acquisitions. Many traditional methods to correct for field inhomogeneity distortions rely on a static field map measured with the use of images that are themselves distorted. In this work, we employ a regularized least-squares approach to jointly estimate both the undistorted image and field map at each acquisition using a spiral-in/spiral-out pulse sequence. Simulation and phantom studies show that this method is accurate and stable over a time series. Human functional studies show that the jointly estimated field map may be more accurate than standard field map estimates in the presence of respiration-induced phase oscillations, leading to better detection of functional activation. The proposed method measures a dynamic field map that accurately tracks magnetic field drift and respiration-induced phase oscillations during the course of a functional study.

Enhanced frequency-domain optical image reconstruction in tissues through total-variation minimization

Optical image reconstruction in heterogeneous turbid media is sensitive to noise, especially when the signal-to-noise ratio of a measurement system is low. A total-variation-minimization-based iterative algorithm is described in this paper that enhances the quality of reconstructed images with frequency-domain data over that obtained previously with a regularized least-squares approach. Simulation experiments in an 8.6-cm-diameter circular heterogeneous region with low- and high-contrast levels between the target and the background show that the quality of the reconstructed images can be improved considerably when total-variation minimization is included. These simulated results are further verified and confirmed by images reconstructed from experimental data by the use of the same geometry and optically tissue-equivalent phantoms. Measures of imaging performance, including the location, size, and shape of the reconstructed heterogeneity, along with absolute errors in the predicted optical-property values are used to quantify the enhancements afforded by this new approach to optical image reconstruction with diffuse light. The results show improvements of up to 5 mm in terms of geometric information and an order of magnitude or more decrease in the absolute errors in the reconstructed optical-property values for the test cases examined.