Constrained Least Research Articles

A Dual estimator as a tool for solving regression problems

This paper discusses a parameter estimation method that employs an unusual estimator called the Dual estimator. For a linear regression model, we obtain two alternative estimators by subtracting or adding a certain vector to the vector of the Ordinary Least Squares Estimator (OLSE). One of them strictly dominates the latter. Moreover, under the normality assumption this estimator is unbiased, and consistent, and has significantly smaller variance than the OLSE. The use of a priori information is a universal way to choose a better alternative. An important property of the proposed method is the possibility of using the strict inequalities as a priori information. In particular, if the external information is that the L2-norm of the OLS estimate exceeds the same norm of a vector of true coefficients, one can choose a better alternative without additional parameters. If it is known that the parameter is restricted by a linear non-strict inequality, the method has a smaller Mean Squared Error than a Constrained Least Squares technique. Finally, a priori information on two possible parameter values can be successfully used for the experimental confirmation of one of two alternative theories, which is illustrated by a verification of the General Theory of Relativity based upon astronomical data.

Two-dimensional bar code out-of-focus deblurring via the Increment Constrained Least Squares filter

When a two-dimensional bar code is away from a camera’s focus, the image is blurred by the convolution of the point spread function. In the presence of noise, the out-of-focus deblurring is an ill-posed problem. The two-dimensional bar code image has a very special form, making deblurring feasible. This paper proposes a fast deblurring algorithm called the Increment Constrained Least Squares filter that is specifically designed for two-dimensional bar code images. After analyzing the bar code image, the standard deviation of the Gaussian blur kernel is obtained. Then, the bar code image is restored through on iterative computations. In each iteration, the bi-level constraint of the bar code image is efficiently incorporated. Experimental results show that our algorithm can obtain better bar code image quality compared with existing methods. Our method can also improve the reading depth of field, which is an important performance parameter for bar code readers.

A Sound Sources Locating Method of Constrained Least Squares Based on Time Delay Estimation

The sound sources positioning method based on time delay estimation is through the iteration of a group of nonlinear equation to solve the target position. Most of the existing methods is to solve under the hypothesis that the sound spread velocity in air is known. This paper puts forward several effective methods of source orientation and propagation velocity estimation. These methods are only using the time delay which comes from the received signal between sensors separated in space to deduce the solving method of sound source location and sound’s speed estimation based on least squares, total least squares, limited data not sure and constraint least squares method, and then compare the characteristics of these methods through the computer simulation.

Design of Recurrent Neural Networks for Solving Constrained Least Absolute Deviation Problems

Recurrent neural networks for solving constrained least absolute deviation (LAD) problems or L(1)-norm optimization problems have attracted much interest in recent years. But so far most neural networks can only deal with some special linear constraints efficiently. In this paper, two neural networks are proposed for solving LAD problems with various linear constraints including equality, two-sided inequality and bound constraints. When tailored to solve some special cases of LAD problems in which not all types of constraints are present, the two networks can yield simpler architectures than most existing ones in the literature. In particular, for solving problems with both equality and one-sided inequality constraints, another network is invented. All of the networks proposed in this paper are rigorously shown to be capable of solving the corresponding problems. The different networks designed for solving the same types of problems possess the same structural complexity, which is due to the fact these architectures share the same computing blocks and only differ in connections between some blocks. By this means, some flexibility for circuits realization is provided. Numerical simulations are carried out to illustrate the theoretical results and compare the convergence rates of the networks.

Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method

Nonnegative matrix factorization (NMF) determines a lower rank approximation of a matrix $A \in \mathbb{R}^{m \times n} \approx WH$ where an integer $k \ll \min(m,n)$ is given and nonnegativity is imposed on all components of the factors $W \in \mathbb{R}^{m \times k}$ and $H \in \mathbb{R}^{k \times n}$. NMF has attracted much attention for over a decade and has been successfully applied to numerous data analysis problems. In applications where the components of the data are necessarily nonnegative, such as chemical concentrations in experimental results or pixels in digital images, NMF provides a more relevant interpretation of the results since it gives nonsubtractive combinations of nonnegative basis vectors. In this paper, we introduce an algorithm for NMF based on alternating nonnegativity constrained least squares (NMF/ANLS) and the active set-based fast algorithm for nonnegativity constrained least squares with multiple right-hand side vectors, and we discuss its convergence properties and a rigorous convergence criterion based on the Karush-Kuhn-Tucker (KKT) conditions. In addition, we also describe algorithms for sparse NMFs and regularized NMF. We show how we impose a sparsity constraint on one of the factors by $L_1$-norm minimization and discuss its convergence properties. Our algorithms are compared to other commonly used NMF algorithms in the literature on several test data sets in terms of their convergence behavior.

Constrained Least Squares Filtering Algorithm for Ultrasound Image Deconvolution

A new medical ultrasound tissue model is considered in this paper, which incorporates random fluctuations of the tissue response and provides more realistic interpretation of the received pulse-echo ultrasound signal. Using this new model, we propose an algorithm for restoration of the degraded ultrasound image. The proposed deconvolution is a modification of the classical regularization technique which combines Wiener filter and the constrained least squares (LS) algorithm for restoration of the ultrasound image. The performance of the algorithm is evaluated based on both the simulated phantom images and real ultrasound radio frequency (RF) data. The results show that the algorithm can provide improved ultrasound imaging performance in terms of the resolution gain. The deconvolved images visually show better resolved tissue structures and reduce speckle, which are confirmed by a medical expert.

A Constrained Least Squares Approach to Mobile Positioning: Algorithms and Optimality

The problem of locating a mobile terminal has received significant attention in the field of wireless communications. Time-of-arrival (TOA), received signal strength (RSS), time-difference-of-arrival (TDOA), and angle-of-arrival (AOA) are commonly used measurements for estimating the position of the mobile station. In this paper, we present a constrained weighted least squares (CWLS) mobile positioning approach that encompasses all the above described measurement cases. The advantages of CWLS include performance optimality and capability of extension to hybrid measurement cases (e.g., mobile positioning using TDOA and AOA measurements jointly). Assuming zero-mean uncorrelated measurement errors, we show by mean and variance analysis that all the developed CWLS location estimators achieve zero bias and the Cramer-Rao lower bound approximately when measurement error variances are small. The asymptotic optimum performance is also confirmed by simulation results.

Reliability constrained routing in QoS networks

The issue of handling network failures is becoming increasingly important. In this paper, we address the problem of constrained routing by treating reliability as one of the QoS requirements. The problem is to create a feasible path from a given node to the destination such that the bandwidth and reliability requirements of the path are satisfied and the cost of the path is minimized (Reliability Constrained Least Cost Routing Problem). To solve the problem, we propose an approach which employs a novel concept, called partial protection, wherein backup paths are created for a selected set of domains in the network so as to meet the reliability constraints. The Partial Protection Approach (PPA) has two steps: Primary Path Creation and Backup Path Creation if necessary. To implement PPA, we propose three scalable two-pass resource reservation schemes, viz., Conservative, Optimistic, and Hybrid schemes. These schemes differ depending on whether the backup paths are created during the forward pass, reverse pass, or both. We evaluate the performance of the proposed schemes for dynamic multicast groups with different bandwidth and reliability requirements using average call acceptance rate and average tree cost as performance metrics. Our studies show that group dynamics and reliability requirements have significant impact on the performance of the schemes.

Local Compliance Estimation via Positive Semidefinite Constrained Least Squares

We present a method to estimate a positive semidefinite matrix by linear least squares, and we apply this method to the estimation of local compliance matrices during deformable object modeling. Estimation of physical quantities from measurements has to consider noise due to measurement and modeling inaccuracy. Enforcing constraints during estimation can guarantee physically plausible results even under difficult measurement conditions.

A Krylov Subspace Method for Quadratic Matrix Polynomials with Application to Constrained Least Squares Problems

We present a Krylov subspace--type projection method for a quadratic matrix polynomial $\lambda^2 I -\lambda A - B$ that works directly with A and B without going through any linearization. We discuss a special case when one matrix is a low rank perturbation of the other matrix. We also apply the method to solve quadratically constrained linear least squares problem through a reformulation of Gander, Golub, and von Matt as a quadratic eigenvalue problem, and we demonstrate the effectiveness of this approach. Numerical examples are given to illustrate the efficiency of the algorithms.

IMAGE RESTORATION BASED ON GENERALIZED FINITE AUTOMATA ENCODED EDGE PRESERVING REGULARIZATION

We present an edge preserving regularization scheme for the restoration of degraded images compressed by the lossy compression method based on Generalized Finite Automata edge encoding and the Iterative Constrained Least Square Regularization technique. In this scheme, the degraded image reconstructed from lossy image compressions is treated as the input to the image restoration process. The edge information extracted from the source image is utilized as a priori knowledge for the subsequent reconstruction. In order to compromise the overall bit rate incurred by the additional edge information, a generalized finite automata encoding technique is adopted to encode the bit-planes of the edge image. The generalized finite automata method ensures an efficient and adaptive image-independent encoding of the edge image. The experiment has shown that the proposed scheme could significantly improve both the objective and subjective quality of the reconstructed image over that of the set partitioning in hierarchical trees by recovering more image details and edge structures under the same bit rates.

GSOR Method for the Equality Constrained Least Squares Problems and the Generalized Least Squares Problems

In a recent paper [4], Li et al . gave a generalized successive overrelaxation (GSOR) method for the least squares problems. In this paper, we show that the GSOR method can be applied to the equality constrained least squares (LSE) problems and the generalized least squares (GLS) problems.

Spectral unmixing ofcompact airborne spectrographic imager (CASI) data for quantifying sub-pixel proportions of biophysical parameters in a coastal dune system

In environmentally sensitive and large coastal dune systems, identification and mapping of favourable (sandy), and unfavourable (scrub rich) habitats form the key to coastal conservation and management. In highly mixed floral environments, however, such an identification is difficult with low resolution multispectral imagery. In such cases, spectral unmixing is useful to resolve the “mixed pixel” effects. The Kenfig NNR (National Nature Reserve), South Wales. UK, bestowed with high biodiversity, is facing loss of successionally young slack habitats due to dune stabilisation and vegetation succession. To map such habitats, linear spectral unmixing of airborne MSS (CASI-Compact Airborne Spectrographic Imager) data was performed using the Constrained Least Square (CLS) method, and the sub-pixel proportions of the spectral end members viz. sand, vegetation and shade/moisture were defined. Comparison of the estimated fractions with the growth forms of the dominant vegetation species Salix repens (Creeping willow) with classified digital aerial photographs shows a positive correlation, thus giving us confidence in the mixture modelling technique. Apart from aiding in conservation management, such a fuzzy classification of multi-date imagery helps to delineate sandy and vegetated areas for change detection and landscape/ habitat succession studies.

Row-Wise Backward Stable Elimination Methods for the Equality Constrained Least Squares Problem

It is well known that the solution ofthe equality constrained least squares (LSE) problem minBx=d||b-Ax||2 is the limit of the solution of the unconstrained weighted least squares problem $$ \min_x\left\| \bmatrix{ \mu d \cr b } - \bmatrix{\mu B \cr A } x \right\|_2 $$ as the weight $\mu$ tends to infinity, assuming that $\bmatrix{B^T & A^T \cr}^T$ has full rank. We derive a method for the LSE problem by applying Householder QR factorization with column pivoting to this weighted problem and taking the limit analytically, with an appropriate rescaling of rows. The method obtained is a type of direct elimination method. We adapt existing error analysis for the unconstrained problem to obtain a row-wise backward error bound for the method. The bound shows that, provided row pivoting or row sorting is used, the method is well-suited to problems in which the rows of A and B vary widely in norm. As a by-product of our analysis, we derive a row-wise backward error bound of precisely the same form for the standard elimination method for solving the LSE problem. We illustrate our results with numerical tests.

Backward Error Bounds for Constrained Least Squares Problems

We derive an upper bound on the normwise backward error of an approximate solution to the equality constrained least squares problem $\min_{Bx=d}\|b-Ax\|_2$. Instead of minimizing over the four perturbations to $A$, $b$, $B$ and $d$, we fix those to $B$ and $d$ and minimize over the remaining two; we obtain an explicit solution of this simplified minimization problem. Our experiments show that backward error bounds of practical use are obtained when $B$ and $d$ are chosen as the optimal normwise relative backward perturbations to the constraint system, and we find that when the bounds are weak they can be improved by direct search optimization. We also derive upper and lower backward error bounds for the problem of least squares minimization over a sphere: $\min_{\|x\|_2\le\alpha}\|b-Ax\|_2$.

Efficient Solution of Constrained Least Squares Problems with Kronecker Product Structure

A computational method for efficient solution of linear constrained least squares problems with Kronecker product structure is presented. The equality constraints are assumed to be linearly independent. The computational efficiency of the method is analyzed. Conditions for uniqueness of solutions are given.

Blind restoration of images degraded by space-variant blurs using iterative algorithms for both blur identification and image restoration

This paper addresses the problem of digital blind restoration of images degraded by space-variant blurs and noise. The existing Expectation-Maximization (EM) algorithm reported in the literature is extended in this paper and combined with the region adaptive technique to handle the problem of identifying spatially variant blurs. The proposed algorithm is a two-step interative process. The expectation step of the EM algorithm is modified by the use of iterative image restoration. The entire image is divided into disjointed regions and the blur is identified in these regions using the proposed modified form of the EM algorithm. The iterative Constrained Least Squares (CLS) algorithm used in space-invariant image restoration is extended to restore the space variant blur images. Spatially adaptive algorithms for restoration are also applied. Experiments have been carried out to evaluate the performances of the proposed algorithms.

Constrained Least Squares Identification of Deterministic Nicely Nonlinear Models

A system M is said to be a nicely nonlinear model of a given plant P if: i) M is tangent to P at the nominal equilibrium, ii) M is rigorously linearizable by output feedback. The present paper deals with the problem of identifying the parameters of a simple class of nicely nonlinear models. From a computational point of view, the problem to solve is a convex semi-infinite quadratic programming problem, for which a suitable algorithm has been set up.

On Row Relaxation Methods for Large Constrained Least Squares Problems

This paper addresses the question of how to construct a row relaxation method for solving large unstructured linear least squares problems, with or without linear constraints. The proposed approach combines the Herman–Lent–Hurwitz scheme for solving regularized least squares problems with the Lent–Censor–Hildreth method for solving linear constraints. However, numerical experiments show that the Herman–Lent–Hurwitz scheme has difficulty reaching a least squares solution. This difficulty is resolved by applying the Riley–Golub iterative improvement process.

Implicit Nullspace Iterative Methods for Constrained Least Squares Problems

A class of iterative algorithms is proposed for solving equality constrained least squares problems, generalizing an order-reducing algorithm first analyzed by Barlow, Nichols, and Plemmons (algorithm BNP). The new algorithms, called implicit null space methods, are based on the classical nullspace method, except that the basis for the nullspace of the constraint matrix is not explicitly formed. The implicit basis acts as a preconditioner for a set of normal equations in factored form. Implicit nullspace methods allow great flexibility in the choice of preconditioner, and can be used to solve certain problems for which algorithm BNP is not well suited. In addition, they offer the opportunity for parallel implementation on substructured problems. Some numerical results based on both structural engineering applications and Stokes flow are included.