Weighted Least-squares Formulation Research Articles

A Bias-Reduced Solution for Multistatic Localization Using Differential Delays and Doppler Shifts

In this paper, the multistatic localization problem with unknown propagation speed is investigated using differential delays and Doppler shifts between the signals from direct and indirect paths. A series of pseudo-linear equations are formulated via the transformation of measurement models. A weighted least squares (WLS) formulation is then proposed after ignoring the second-order error terms, which can be rewritten as a non-convex optimization problem with the relationships among variables included as constraints. To deal with the non-convexity of the problem, semidefinite relaxation is applied, resulting in a convex semidefinite program (SDP). Several reasonable second-order cone constraints constructed via basic inequality and Cauchy-Schwarz inequality are added to tighten the relaxed SDP problem. By preserving the second-order error terms in equations, the bias of the estimate from the WLS formulation is also derived and then subtracted to nearly eliminate the bias and reach a bias-reduced solution. Simulation results show that the mean square error (MSE) of the proposed method approaches the Cramer-Rao lower bound (CRLB) and the bias is reduced significantly.

A Complex Exponential Structure for Low-Complexity Variable Fractional Delay FIR Filters

Variable fractional delay (VFD) filters generally require significantly greater computational resources for implementation than static filters. The motivation of this work is therefore to develop ways for implementation complexity reduction. The Farrow structure is adopted by most of the variable fractional delay (VFD) filters due to their effectiveness. This structure essentially assumes an algebraic polynomial approximation to the continuously varying impulse response. In this paper, we dispense with the polynomial function and instead propose the use of complex exponential functions for approximation. This new approach leads to a Farrow-like implementation structure with complex coefficient subfilters and a shape parameter. The complex exponential (CE) VFD filter is analyzed, and various types of symmetry properties are derived. Accordingly, a simplified implementation structure is obtained. The complexity analysis shows that the design and implementation complexities can be reduced to something comparable to the classical real algebraic polynomial (AP) VFD filters with the same number of subfilters and filter order. However, the CE filters can achieve a better approximation accuracy to the desired fractional delay characteristics, compared to the AP filters. Moreover, this superiority becomes significant, when only a few subfilters are used, or when the filter orders are high. Therefore, the reduction in complexity can be achieved for the same design specification. The design algorithm is developed using a weighted least-squares formulation. Using a matrix-based approach, the closed-form solution is derived firstly for a fixed shape parameter. This is then followed by optimization of the shape parameter using the Golden Cut method. Comprehensive design examples will be presented that compare the CE filters with the AP filters.

Weighted reproducing kernel collocation method based on error analysis for solving inverse elasticity problems

For inverse problems equipped with incomplete boundary conditions, a simple solution strategy to obtain approximations remains a challenge in the fields of engineering and science. Based on our previous study, the weighted reproducing kernel collocation method (W-RKCM) shows optimal convergence in solving inverse Cauchy problems. As such, this work further introduces the W-RKCM to solve inverse problems in elasticity. From mathematical error estimate and numerical convergence study, it is shown that the weighted least-squares formulation can properly balance the errors in the domain and on the boundary. By comparing the approximations obtained by W-RKCM with those obtained by the direct collocation method, the reproducing kernel shape function can retain the locality without using a large support size, and the corresponding approximations exhibit extremely high solution accuracy. The stability of the W-RKCM is demonstrated by adding noise on the boundary conditions. This work shows the efficacy of the proposed W-RKCM in solving inverse elasticity problems as no additional technique is involved to reach the desired solution accuracy in comparison with the existing methods in the literature.

New Semidefinite Relaxation Method for Acoustic Energy-Based Source Localization

The acoustic energy-based source localization problem is addressed. Based on the noisy acoustic energy measurements, we obtain a nonlinear and non-convex weighted least squares (WLS) formulation for this problem, whose globally optimal solution is hardly obtained without a good initial estimate. To overcome this difficulty, we first transform the original measurement model to obtain an approximate WLS formulation, and then employ the semidefinite relaxation (SDR) technique to obtain a semidefinite programming (SDP), which can be solved very efficiently. The SDP solution is further refined using the conventional Gaussian randomization procedure. Moreover, we propose an alternating estimation procedure to handle the localization problem when the decay factor is unknown. Simulation results show that the proposed SDR method significantly outperforms the existing methods.

Cooperative Spectrum Sensing for Cognitive Radios Using Kriged Kalman Filtering

A cooperative cognitive radio (CR) sensing problem is considered, where a number of CRs collaboratively detect the presence of primary users (PUs) by exploiting the novel notion of channel gain (CG) maps. The CG maps capture the propagation medium per frequency from any point in space and time to each CR user. They are updated in real-time using Kriged Kalman filtering (KKF), a tool with well-appreciated merits in geostatistics. In addition, the CG maps enable tracking the transmit-power and location of an unknown number of PUs, via a sparse regression technique. The latter exploits the sparsity inherent to the PU activities in a geographical area, using an <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ℓ</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm regularized, sparsity-promoting weighted least-squares formulation. The resulting sparsity-cognizant tracker is developed in both centralized and distributed formats, to reduce computational complexity and memory requirements of a batch alternative. Numerical tests demonstrate considerable performance gains achieved by the proposed algorithms .

A Semidefinite Relaxation Method for Energy-Based Source Localization in Sensor Networks

In this paper, the energy-based localization problem in wireless sensor networks is addressed. We focus on the weighted least squares (WLS) estimation of the source location. Due to the nonconvex nature of the WLS formulation, its global solution is hard to obtain without a good initial estimate. We propose a semidefinite relaxation method for this localization problem. To do so, we transform the original WLS formulation into a nonconvex approximate WLS (AWLS) formulation, which is then relaxed as a semidefinite programming (SDP). We show that it is possible for the SDP to be tight, i.e., the SDP solves the original AWLS problem. For the cases where the SDP is not tight, a procedure called Gaussian randomization is applied to further refine the SDP solution. Simulation results show that the proposed method can outperform the existing methods at high noise levels.

Discontinuous Discretization for Least-Squares Formulation of Singularly Perturbed Reaction-Diffusion Problems in One and Two Dimensions

In this paper, we consider the singularly perturbed reaction-diffusion problem in one and two dimensions. The boundary value problem is decomposed into a first-order system to which a suitable weighted least-squares formulation is proposed. A robust, stable, and efficient approach is developed based on local discontinuous Galerkin (LDG) discretization for the weak form. Uniform error estimates are derived. Numerical examples are presented to illustrate the method and the theoretical results. Comparison studies are made between the proposed method and other methods.

An algorithm for the estimation of parameters in models with stochastic differential equations

An algorithm for the estimation of parameters of stochastic differential equations (SDEs) is presented. It is based on a nonlinear weighted least-squares formulation, in which the objective function is evaluated based on mean values of the measured variables predicted through an Euler discretisation of the SDEs and their integration by Monte-Carlo simulation. The problem is solved using a Levenberg–Marquardt algorithm. The presence of simulation noise is handled by choosing a convergence criterion based on the noise level and by ensuring that the optimality criterion is met for a large simulation size and hence a low noise level. In order to increase the reliability of the algorithm and to decrease its computational cost, stochastic sensitivity equations are derived. Furthermore, the number of trajectories used in the Monte-Carlo simulations is changed adaptively throughout the execution of the algorithm. This leads to a significant decrease in computational requirements. These concepts are illustrated on a simple example and a more complex model of polymer rheology. In all cases, parameter estimates close to the true parameter values are identified.

Advantages of Unbiased Support Vector Classifiers for Data Mining Applications

Many learning algorithms have been used for data mining applications, including Support Vector Classifiers (SVC), which have shown improved capabilities with respect to other approaches, since they provide a natural mechanism for implementing Structural Risk Minimization (SRM), obtaining machines with good generalization properties. SVC leads to the optimal hyperplane (maximal margin) criterion for separable datasets but, in the nonseparable case, the SVC minimizes the L1 norm of the training errors plus a regularizing term, to control the machine complexity. The L1 norm is chosen because it allows to solve the minimization with a Quadratic Programming (QP) scheme, as in the separable case. But the L1 norm is not truly an "error counting" term as the Empirical Risk Minimization (ERM) inductive principle indicates, leading therefore to a biased solution. This effect is specially severe in low complexity machines, such as linear classifiers or machines with few nodes (neurons, kernels, basis functions). Since one of the main goals in data mining is that of explanation, these reduced architectures are of great interest because they represent the origins of other techniques such as input selection or rule extraction. Training SVMs as accurately as possible in these situations (i.e., without this bias) is, therefore, an interesting goal. We propose here an unbiased implementation of SVC by introducing a more appropriate "error counting" term. This way, the number of classification errors is truly minimized, while the maximal margin solution is obtained in the separable case. QP can no longer be used for solving the new minimization problem, and we apply instead an iterated Weighted Least Squares (WLS) procedure. This modification in the cost function of the Support Vector Machine to solve ERM was not possible up to date given the Quadratic or Linear Programming techniques commonly used, but it is now possible using the iterated WLS formulation. Computer experiments show that the proposed method is superior to the classical approach in the sense that it truly solves the ERM problem.

Improved sparse least-squares support vector machines

Suykens et al. (Neurocomputing (2002), in press) describe a weighted least-squares formulation of the support vector machine for regression problems and present a simple algorithm for sparse approximation of the typically fully dense kernel expansions obtained using this method. In this paper, we present an improved method for achieving sparsity in least-squares support vector machines, which takes into account the residuals for all training patterns, rather than only those incorporated in the sparse kernel expansion. The superiority of this algorithm is demonstrated on the motorcycle and Boston housing data sets.

A least-squares approach for uniform strain triangular and tetrahedral finite elements

A least-squares approach is presented for implementing uniform strain triangular and tetrahedral finite elements. The basis for the method is a weighted least-squares formulation in which a linear displacement field is fit to an element's nodal displacements. By including a greater number of nodes on the element boundary than is required to define the linear displacement field, it is possible to eliminate volumetric locking common to fully integrated lower-order elements. Such results can also be obtained using selective or reduced integration schemes, but the present approach is fundamentally different from those. The method is computationally efficient and can be used to distribute surface loads on an element edge or face in a continuously varying manner between vertex, mid-edge and mid-face nodes. Example problems in two- and three-dimensional linear elasticity are presented. Element types considered in the examples include a six-node triangle, eight-node tetrahedron, and ten-node tetrahedron. © 1998 John Wiley & Sons, Ltd.

Novel deblurring algorithms for images captured with CCD cameras

Weighted least-squares algorithms are proposed for reconstructing images from photon-limited data captured with CCD cameras. These cameras introduce blur and signal-independent noise, while forces of nature give rise to so-called background noise. The errors in the data are addressed by using a weighted least-squares formulation that emphasizes more reliable data values. The performance of the proposed algorithms is investigated by using synthetic and real data obtained from the Space Telescope Science Institute. The real data were captured with the wide-field/planetary camera aboard the original Hubble Space Telescope. Simulation studies with the synthetic data indicate that the proposed algorithms converge faster and have fewer “false stars” than a modified Richardson–Lucy algorithm.