Least-squares Problem Research Articles

Joint Denoising-Demosaicking Network for Long-Wave Infrared Division-of-Focal-Plane Polarization Images with Mixed Noise Level Estimation.

Denoising and demosaicking long-wave infrared (LWIR) division-of-focal-plane (DoFP) polarization images are crucial for various vision applications. However, existing methods rely on the sequential application of individual denoising and demosaicking processes, which may result in the accumulation of errors produced by each process. To address this issue, we propose a joint denoising and demosaicking method for LWIR DoFP images based on a three-stage progressive deep convolutional neural network. To ensure the generalization ability of this network, it is essential to have adequate training data that closely resembles real data. Therefore, we model the complex noise sources that affect LWIR DoFP images as mixed Poisson-Additive-Stripe noise and construct a least-squares problem based on the polarization measurement redundancy error to estimate the parameters of this model on real images. Subsequently, the estimated noise parameters are used to generate training data that enables the network to learn accurate polarization image statistics and improve its generalization ability. The experimental results demonstrate the effectiveness of the proposed method in enhancing the image restoration performance on real LWIR DoFP polarization data.

WCA: A New Efficient Nonlinear Adaptive Control Allocation for Planar Hexacopters

This paper presents an efficient control allocation (CA) strategy which significantly improves the flight performance of a planar hexacopter. This allocation maps the desired control vector v = [ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T, L,M,N</i> ]⟙ composed of the thrust and torques in roll, pitch, and yaw axis, respectively, to propellers’ speed. This paper shows that a CA strategy based on the classical approach of pseudo-inverse matrix only exploits a limited range of the vehicle capabilities to generate thrust and moments. A novel approach is presented, which is based on a weighted pseudo-inverse matrix method, called WCA, capable of exploiting a much larger domain in v. The three weights involved in WCA are adapted online according to nonlinear laws which are analytically derived by solving symbolically the equivalent constraint least-squares problem, thus removing the need for online-optimization calculations. This solution allows for very fast real-time operations, suitable for autopilots with limited computing resources. This paper provides 1) a detailed analysis of the limitations of the classical control allocation scheme, 2) the mathematical development of the WCA algorithm, 3) simulations and real experiments which show that this WCA strategy outperforms the classical CA approach in terms of a) capability to generate the maximum roll and pitch torques possible, without generating undesired yaw torques, b) prioritizing the generation of thrust over attitude torques, thus achieving better altitude tracking despite aggressive maneuvers or the presence of a payload, and c) better behavior in case of actuator saturation, faults or even failures.

Analysis of Interpolating Regression Models and the Double Descent Phenomenon

A regression model with more parameters than data points in the training data is overparametrized and has the capability to interpolate the training data. Based on the classical bias-variance tradeoff expressions, It is commonly assumed that models which interpolate noisy training data are poor to generalize. In some cases, this is not true. The best models obtained are overparametrized and the testing error exhibits the double descent behavior as the model order increases. In this contribution, we provide some analysis to explain the double descent phenomenon, first reported in the machine learning literature. We focus on interpolating models derived from the minimum norm solution to the classical least-squares problem and also briefly discuss model fitting using ridge regression. We derive a result based on the behavior of the smallest singular value of the regression matrix that explains the peak location and the double descent shape of the testing error as a function of model order.

Low-rank signal subspace: parameterization, projection and signal estimation

The paper contains several theoretical results related to the weighted nonlinear least-squares problem for low-rank signal estimation, which can be considered as a Hankel structured low-rank approximation problem. A parameterization of the subspace of low-rank time series connected with generalized linear recurrence relations (GLRRs) is described and its features are investigated. It is shown how the obtained results help to describe the tangent plane, prove optimization problem features and construct stable algorithms for solving low-rank approximation problems. For the latter, a stable algorithm for constructing the projection onto a subspace of time series that satisfy a given GLRR is proposed and justified. This algorithm is used for a new implementation of the known Gauss-Newton method using the variable projection approach. The comparison by stability and computational cost is performed theoretically and with the help of an example.

A Novel Approach for Unit-Modulus Least-Squares Optimization Problems

We describe a novel constrained least-squares (LS) optimization problem and propose an iterative solution that deals with the constraints of placing variables on the complex unit circle. We reformulate the LS problem with unit magnitude constraints so that we obtain efficient closed-form local updates based on Procrustes orthogonal solutions that monotonically improve the objective function. We show the performance of the proposed algorithm in a synthetic experiment and an application to hybrid precoding/combining in mmWave MIMO communications.

A Function Approximation Approach for Parametric Optimization

We present a novel approach for approximating the primal and dual parameter-dependent solution functions of parametric optimization problems. We start with an equation reformulation of the first-order necessary optimality conditions. Then, we replace the primal and dual solutions with some approximating functions and find for some test parameters optimal coefficients as solution of a single nonlinear least-squares problem. Under mild assumptions it can be shown that stationary points are global minima and that the function approximations interpolate the solution functions at all test parameters. Further, we have a cheap function evaluation criterion to estimate the approximation error. Finally, we present some preliminary numerical results showing the viability of our approach.

A hybrid equivalent source—particle swarm optimization model for accurate near-field to far-field conversion

This paper proposes an improved equivalent source (ES) model based on particle swarm optimization (PSO) to predict the radiated emissions of a 2.4 GHz ISM band patch antenna. The proposed model uses near-field data to extract an equivalent set of infinitesimal electric dipoles representation. Characterized by their positions, orientations, and currents, these dipoles can reproduce the same field distribution as the target device under test (DUT). To build an accurate equivalent dipole model, a numerical experiment is performed and presented in this paper, where many variable selections of observation points and dipole numbers and locations of the dipole plane are examined, with the aspiration of determining a suitable conversion problem. Based on the proper combination of the two methods (ES-PSO), the complete modeling procedure consists of the application of the pseudo-inverse method to find the minimum norm solution to the least-square problem. Then, the obtained solution is provided to the PSO algorithm to initialize the search for the best-fit dipole parameters. The PSO method can further enhance the stability of the obtained dipole solution and improve the accuracy of the far field prediction. Promising and encouraging results in terms of field prediction at higher planes above the DUT are obtained.

A one-shot overlapping Schwarz method for component-based model reduction: application to nonlinear elasticity

We propose a component-based (CB) parametric model order reduction (pMOR) formulation for parameterized nonlinear elliptic partial differential equations (PDEs) based on overlapping subdomains. Our approach reads as a constrained optimization statement that penalizes the jump at the components’ interfaces subject to the approximate satisfaction of the PDE in each local subdomain. Furthermore, the approach relies on the decomposition of the local states into a port component – associated with the solution on interior boundaries – and a bubble component that vanishes at ports: since the bubble components are uniquely determined by the solution value at the corresponding port, we can recast the constrained optimization statement into an unconstrained statement, which reads as a nonlinear least-squares problem and can be solved using the Gauss–Newton method. We present thorough numerical investigations for a two-dimensional neo-Hookean nonlinear mechanics problem to validate our method; we further discuss the well-posedness of the mathematical formulation and the a priori error analysis for linear coercive problems.

A General FRI Sampling System for Pulse Streams and the Multichannel Synchronization Method

Pulse streams are a kind of finite rate of innovation (FRI) signal that widely exist in many engineering applications. According to the FRI theory, nonconsecutive sampled Fourier coefficients (FCs) have better parameter estimation effects than the consecutive ones. In this brief, we propose a general FRI sampling system for pulse streams based on multichannel random modulation (MRM). Jointly using multichannel samples, multiple sets of consecutive FCs of distinct frequency bands can be analytically calculated by solving the least-squares problems. The minimum sampling rate for each channel is determined by the number of pulses rather than the pulse bandwidth. The proposed sampling system is general for pulses of distinct frequency bands, and we extend it to the frequency-agile radar (FAR). Besides, the multichannel synchronization problem is analyzed, and a calibration method when the pseudo-random sequences (PRSs) are not synchronized is proposed. Simulation experiments demonstrate the effectiveness of the proposed sampling system.

Method for inverse calculation of erosion parameters in slag-refractory systems

Refractory erosion due to fluid flow is caused by the shear stresses acting on the liquid/solid interface. This mechanism of continuous wear has not been extensively investigated based on appropriate physical considerations. This study proposes a novel method for the inverse calculation of erosion parameters in slag-refractory systems. A computational fluid dynamic model is coupled with optimisation software to solve a nonlinear least-squares problem for the identification of erosion parameters from experimental erosion profiles. Erosion-rate modelling is performed based on a three-parameter law of the acting shear stress. A test problem is studied using artificially generated erosion profiles. The feasibility of the approach with two parameters is proven; however, using three parameters causes the problem to become ill-posed. Moreover, the method is successfully applied to a real experiment with an alumina coarse-grain refractory. Statistical analysis of the solutions is performed for both the test problem and real experimental profile. Confidence intervals and regions calculated via linearisation are examined, but their use is limited and should be evaluated for each case. This method can serve as a basis for further investigations concerning the quantification of refractory erosion. The data that support the findings of this study are available on reasonable request from the corresponding author.

An Iterative Implementation of Variable Projection for Separable Nonlinear Optimization Problems

The separable nonlinear least-squares (SNLLS) problems considered in this article frequently appear in a wide range of research fields, such as machine learning, computer vision, system identification, and signal processing. The variable projection algorithm proposed by Golub and Pereyra, which reduces the dimension of the parameters by projecting the linear parameters out of the problem, is quite valuable in solving SNLLS problems. Previous implementations of the variable projection algorithm are based on matrix factorization. In this article, we propose an iterative implementation of the variable projection algorithm. Compared with previous implementations based on matrix decomposition, the proposed method can effectively avoid suffering from large condition number of the matrix or even matrix decomposition failure when dealing with ill-posed SNLLS problems. Numerical experiments on real-world data and synthetic data show the efficiency and robustness of the proposed iterative variable projection algorithm.

Model-Based Derivative-Free Methods for Convex-Constrained Optimization

We present a model-based derivative-free method for optimization subject to general convex constraints, which we assume are unrelaxable and accessed only through a projection operator that is cheap to evaluate. We prove global convergence and a worst-case complexity of $O(\epsilon^{-2})$ iterations and objective evaluations for nonconvex functions, matching results for the unconstrained case. We introduce new, weaker requirements on model accuracy compared to existing methods. As a result, sufficiently accurate interpolation models can be constructed only using feasible points. We develop a comprehensive theory of interpolation set management in this regime for linear and composite linear models. We implement our approach for nonlinear least-squares problems and demonstrate strong practical performance compared to general-purpose solvers.

Synchronous distributed ADMM for consensus convex optimization problems with self-loops

In this paper, a distributed algorithm based on the alternating direction method of multipliers is proposed to solve an unconstrained consensus optimization problem. The network topology among the agents is assumed to be connected and self-loops of each agent are considered. This algorithm takes less time than sequential algorithms because the optimization variables of the agents can be updated synchronously. In addition, it is shown that the algorithm is actually a Jacobi-proximal ADMM algorithm and the requirement of the proximal matrix is relaxed. A convergence analysis of the algorithm is provided, and its convergence rate is found to be O(1/k). Finally, the characteristics of the proposed algorithm are illustrated by numerical experiments, and comparisons are made with other ADMM algorithms in solving a least-squares problem.

Voxel-Based Localization and Mapping for Multirobot System in GPS-Denied Environments

Collaboration localization and mapping in GPS-denied environments is a challenging problem, due to the computing power and communication bandwidth constraints of onboard equipment. In this article, a voxel-based localization and mapping system is developed, which achieves real-time trajectory estimation for the multirobot system utilizing only onboard sensors, computing power, and communication module. The key idea of the voxel-based matching approach is to formulate the registration problem as a maximum <i>a posteriori</i> (MAP) problem and to simplify it to a nonlinear least-squares problem using Gauss approximation and principal component analysis (PCA). The voxel-based matching method significantly improves the computational efficiency and reduces the communication bandwidth of the localization and mapping system. The effectiveness of the proposed method is demonstrated through experiments in real-world environments for various scenarios.

Online State Estimation for Time-Varying Systems

The article investigates the problem of estimating the state of a time-varying system with a linear measurement model; in particular, the article considers the case where the number of measurements available can be smaller than the number of states. In lieu of a batch linear least-squares approach—well-suited for static networks, where a sufficient number of measurements could be collected to obtain a full-rank design matrix—the article proposes an online algorithm to estimate the possibly time-varying state by processing measurements as and when available. The design of the algorithm hinges on a generalized least-squares cost augmented with a proximal-point-type regularization. With the solution of the regularized least-squares problem available in closed-form, the online algorithm is written as a linear dynamical system where the state is updated based on the previous estimate and based on the new available measurements. Conditions under which the algorithmic steps are in fact a contractive mapping are shown, and bounds on the estimation error are derived for different noise models. Numerical simulations are provided to corroborate the analytical findings.

An Enhanced Pathfinder Algorithm based MCSA for rotor breakage detection of induction motor

This paper proposes a new broken rotor bar (BRB) finding technique based on a novel hybrid algorithm, namely the Enhanced Pathfinder Algorithm (EPFA). Compared to existing optimization algorithms, this proposed approach provides more balanced exploration and exploitation. The main task regarding rotor fault detection is to identify sideband frequency components (SFCs) corresponding to the fault and in this context proposed method overcomes the disadvantages of traditional frequency domain analysis, such as short data length, effects of spectral leakage, and insufficient sampling frequency. To achieve the proposed diagnosis approach, a signal model for faulty current and a function of residual error are used for the formulation of a nonlinear least-square problem. The method is first tested on a synthetic fault signal, and finally, to validate the proposed technique, a Finite Element Model (FEM) of the induction motor is simulated in the ANSYS 2D workbench.

Effective 3D-transient electromagnetic inversion using finite-element method with a parallel direct solver

The correct interpretation of time-domain or transient electromagnetic (TEM) data relies on effective inversion techniques. Herein, we have developed an effective 3D-TEM inversion method based on the vector finite-element method. The domain was discretized with unstructured tetrahedra to simulate complex geologic structures. We develop a second-order backward-Euler scheme, rigorously with nonuniform time-step sizes, to approximate the time derivative of the electric field. The waveform was directly simulated and considered in the forward modeling and inversion. In a 3D inversion, the most time-consuming element of the workflow is solving a large system of linear equations. To alleviate this problem, we adopt a parallel direct solver for solving a large system of equations involved in forward modeling and sensitivity calculations. We use the Gauss-Newton approach to solve the inverse problem and formulate the least-squares problem to obtain model updates. We avoided the explicit sensitivity calculation by solving the matrix-vector multiplication between the sensitivity matrix (or its transpose) and vector. The developed inversion algorithm is validated using several realistic synthetic models. Finally, we apply the method to practical TEM data with complex topography for coal mine goaf detection. We thus obtain a detailed 3D resistivity model that revealed the layering structure and some localized anomalies related to the coal mine goaf.

Four-Operator Splitting via a Forward–Backward–Half-Forward Algorithm with Line Search

In this article, we provide a splitting method for solving monotone inclusions in a real Hilbert space involving four operators: a maximally monotone, a monotone-Lipschitzian, a cocoercive, and a monotone-continuous operator. The proposed method takes advantage of the intrinsic properties of each operator, generalizing the forward–backward–half-forward splitting and the Tseng’s algorithm with line search. At each iteration, our algorithm defines the step size by using a line search in which the monotone-Lipschitzian and the cocoercive operators need only one activation. We also derive a method for solving nonlinearly constrained composite convex optimization problems in real Hilbert spaces. Finally, we implement our algorithm in a nonlinearly constrained least-square problem and we compare its performance with available methods in the literature.

Distributed and scalable optimization for robust proton treatment planning.

The importance of robust proton treatment planning to mitigate the impact of uncertainty is well understood. However, its computational cost grows with the number of uncertainty scenarios, prolonging the treatment planning process. We developed a fast and scalable distributed optimization platform that parallelizes the robust proton treatment plan computation over the uncertaintyscenarios. We modeled the robust proton treatment planning problem as a weighted least-squares problem. To solve it, we employed an optimization technique called the alternating direction method of multipliers with Barzilai-Borwein step size (ADMM-BB). We reformulated the problem in such a way as to split the main problem into smaller subproblems, one for each proton therapy uncertainty scenario. The subproblems can be solved in parallel, allowing the computational load to be distributed across multiple processors (e.g., CPU threads/cores). We evaluated ADMM-BB on four head-and-neck proton therapy patients, each with 13 scenarios accounting for 3 mm setup and 3.5% range uncertainties. We then compared the performance of ADMM-BB with projected gradient descent (PGD) applied to the sameproblem. For each patient, ADMM-BB generated a robust proton treatment plan that satisfied all clinical criteria with comparable or better dosimetric quality than the plan generated by PGD. However, ADMM-BB's total runtime averaged about 6 to 7 times faster. This speedup increased with the number ofscenarios. ADMM-BB is a powerful distributed optimization method that leverages parallel processing platforms, such as multicore CPUs, GPUs, and cloud servers, to accelerate the computationally intensive work of robust proton treatment planning. This results in (1) a shorter treatment planning process and (2) the ability to consider more uncertainty scenarios, which improves planquality.

Spherical Harmonic-Based Anisotropic Point Scatterer Models for Radar Applications Using Inverse Optimization

High-performance computing-based electromagnetic (EM) emulators are used to simulate real-time complex EM wave interactions between multiple radar targets, transmitters, and receivers. The radar cross section (RCS) of the radar targets are required to be stored as a table; however, the needed storage size increases dramatically with the angle and frequency sampling density. In this article, we present an innovative approach of constructing a concise spherical harmonic-based anisotropic point scatterer model that the emulators can use as part of the computations. The point scatterer model is constructed directly from the precomputed RCS data. First, we use only the monostatic RCS data and compute the spherical harmonic-based monostatic point scatterer model by solving a linear least-squares problem which has a group sparsity constraint. Then, we further compute the spherical harmonic-based bistatic point scatterer model using the full bistatic RCS data. The problem is formulated as a bilinear least-squares problem. The problem is solved using the normalized iterative algorithm, which linearly solves two parameters in a back and forth manner. The results show that the point scatterer model can effectively represent the bistatic RCS data of a radar target.