Least-squares Formulation Research Articles

Machine Learning and Hyperspectral Imaging for Analysis of Human Papillomaviruses (HPV) Vaccine Self-Healing Particles.

Human papillomaviruses (HPV) are known to cause a variety of diseases, including cervical cancer and genital warts. HPV is a highly prevalent virus and is considered the most common sexually transmitted disease. Because of the risks associated with HPV, Gardasil, a quadrivalent recombinant vaccine, was developed by Merck & Co., Inc., Rahway, NJ, USA, and approved by the Food and Drug Administration (FDA) in 2006. The second generation of the vaccine, Gardasil9, was subsequently approved by the FDA in 2014, providing significant protection against HPV. The HPV vaccine may be given as 2 or 3 doses; however, vaccine administration as a single dose with a sustained release mechanism may potentially offer benefits to meet emerging health needs. To explore this, HPV vaccines were formulated within microporous self-healing particles (SHPs) to enable potential controlled release of HPV virus-like particle (VLP) antigen. Machine learning, in the form of multivariate curve resolution-alternating least-squares (MCR-ALS), with Raman hyperspectral imaging was used to determine the molecular identity and spatial distribution of all relevant species within this HPV vaccine formulation. The results indicate that machine learning with Raman hyperspectral imaging was able to spatially resolve HPV VLP antigens within SHP vaccines for the first time, providing crucial information necessary for vaccine development.

Accurate Flow Decomposition via Robust Integer Linear Programming.

Minimum flow decomposition (MFD) is a common problem across various fields of Computer Science, where a flow is decomposed into a minimum set of weighted paths. However, in Bioinformatics applications, such as RNA transcript or quasi-species assembly, the flow is erroneous since it is obtained from noisy read coverages. Typical generalizations of the MFD problem to handle errors are based on least-squares formulations or modelling the erroneous flow values as ranges. All of these are thus focused on error handling at the level of individual edges. In this paper, we interpret the flow decomposition problem as a robust optimization problem and lift error-handling from individual edges to solution paths. As such, we introduce a new minimum path-error flow decomposition problem, for which we give an Integer Linear Programming formulation. Our experimental results reveal that our formulation can account for errors significantly better, by lowering the inaccuracy rate by 30-50% compared to previous error-handling formulations, with computational requirements that remain practical.

Quadratic hyper-surface kernel-free large margin distribution machine-based regression and its least-square form

ε-Support vector regression (ε-SVR) is a powerful machine learning approach that focuses on minimizing the margin, which represents the tolerance range between predicted and actual values. However, recent theoretical studies have highlighted that simply minimizing structural risk does not necessarily result in well margin distribution. Instead, it has been shown that the distribution of margins plays a more crucial role in achieving better generalization performance. Furthermore, the kernel-free technique offers a significant advantage as it effectively reduces the overall running time and simplifies the parameter selection process compared to the kernel trick. Based on existing kernel-free regression methods, we present two efficient and robust approaches named quadratic hyper-surface kernel-free large margin distribution machine-based regression (QLDMR) and quadratic hyper-surface kernel-free least squares large margin distribution machine-based regression (QLSLDMR). The QLDMR optimizes the margin distribution by considering both ε-insensitive loss and quadratic loss function similar to the large-margin distribution machine-based regression (LDMR). QLSLDMR aims to reduce the cost of the computing process of QLDMR, which transforms inequality constraints into an equality constraint inspired by least squares support vector machines (LSSVR). Both models combined the spirit of optimal margin distribution with kernel-free technique and after simplification are convex so that they can be solved by some classical methods. Experimental results demonstrate the superiority of the optimal margin distribution combined with the kernel-free technique in robustness, generalization, and efficiency.

Understanding Composite-Based Structural Equation Modeling Methods From the Perspective of Regression Component Analysis

Regression component analysis (RCA) replaces the factors in a factor analysis model with weighted composites of the model’s observed variables. The weight matrix may be calculated from the factor model’s parameter estimates. Thus, RCA parameter estimates can be obtained using factor model software, but RCA composites have determinate scores, rather than the indeterminate scores of factors. Analytically, RCA equates to modeling with “regression method” factor scores, except that, while those scores will be inconsistent with the original factor model, they are strictly consistent with the RCA model. When the original factor model is strictly correct in the population and the composites in RCA are standardized, RCA parameter estimates replicate those from regression-weighted forms of partial least squares (PLS) path modeling and generalized structured component analysis (GSCA)—affirming that those methods also equate to modeling with regression method factor scores under the same conditions. Parallel measurement allows RCA to replicate both correlation weight and regression weight versions of PLS and GSCA. These results suggest that RCA and regression-weighted forms of PLS and GSCA are all consistent approaches for modeling data that conforms to a factor model. All analytical methods are described using one consistent symbol palette. Complete R syntax is provided.

Phase unwrapping for phase imaging using the plug-and-play proximal algorithm.

Phase unwrapping (PU) is essential for various scientific optical applications. This process aims to estimate continuous phase values from acquired wrapped values, which are limited to the interval (-π,π]. However, the PU process can be challenging due to factors such as insufficient sampling, measurement errors, and inadequate equipment calibration, which can introduce excessive noise and unexpected phase discontinuities. This paper presents a robust iterative method based on the plug-and-play (PnP) proximal algorithm to unwrap two-dimensional phase values while simultaneously removing noise at each iteration. Using a least-squares formulation based on local phase differences and reformulating it as a partially differentiable equation, it is possible to employ the fast cosine transform to obtain a closed-form solution for one of the subproblems within the PnP framework. As a result, reliable phase reconstruction can be achieved even in scenarios with extremely high noise levels.

Computationally Efficient Sphere Decoding Algorithm Based on Artificial Neural Networks for Long-Horizon FCS-MPC

Successful application of finite control set model predictive control (FCS-MPC) strategies with long prediction horizon depends on the careful design of the optimization algorithm. The conventional method involves transforming the problem to an equivalent box-constrained integer least-squares (BILS) formulation that can be solved with branch-and-bound techniques such as the sphere decoding algorithm (SDA). In this work, it is proposed to define an artificial neural network (ANN) to replace the SDA, avoiding its inherent computational variability. Similarly to practical applications of the SDA, the ANN finds an approximate solution of the underlying optimization problem. In contrast, the main benefit of the proposed approach is that it can be implemented in a low-cost microprocessing platform, greatly improving the performance in terms of resources in comparison with other advanced techniques proposed in the literature.

Two-Level Integrity-Monitoring Method for Multi-Source Information Fusion Navigation

To address the issue of integrity monitoring for a multi-source information fusion navigation system, a theoretical framework of two-level integrity monitoring is proposed. Firstly, at the system level, a system-integrity-monitoring method based on the Kalman filter weighted least-squares form is established to detect and isolate faulty navigation sources. Secondly, at the sensor level, considering the redundancy of the faulty navigation sources, this paper presents the design of a multi-mode comprehensive fault-detection method for non-redundant navigation sources. Additionally, an extended-dimension matrix optimized sensor-fault detection and verification method for redundant navigation sources is proposed. Finally, integrity risk allocation criteria are established based on the effectiveness of navigation sources to calculate the system protection level and trigger integrity alarms. The two-level integrity-monitoring method was tested on a multi-source information fusion navigation system integrated with an inertial navigation system (INS), global positioning system (GPS), BeiDou satellite navigation system (BDS), Doppler velocity log (DVL), barometric altimeter (BA), and terrain-aided navigation (TAN). Test results demonstrate that the proposed method can effectively isolate the faulty navigation source within 10 s. Furthermore, it can detect the faulty sensors within the faulty navigation sources, thereby enhancing the reliability and robustness of the multi-source information fusion navigation system.

Indentation over a transversely isotropic, poroelastic, and layered half-space

Poroelastic materials are common in nature and have applications in many engineering fields. In this paper, we derive a general solution of indentation over a multilayered half-space consisting of transversely isotropic and poroelastic materials. The rigid disc-shaped indenter is subjected to a vertical force of Heaviside time-variation. The solution is expressed in terms of the recently introduced powerful Fourier-Bessel series (FBS) system of vector functions combined with the unconditionally stable dual-variable and position method for dealing with layering. Since the problem is a mixed boundary-value one, the Green's functions due to a vertical ring-load are first derived which are then utilized in the integral least-square formulation to derive the solution. In terms of the FBS method, the expansion coefficients, which are further called Love numbers, are discrete, and therefore can be pre-calculated and used repeatedly for different field points on the surface. As such, the solution based on the new FBS method is more efficient and accurate than previous integral-transform methods. This new FBS method is particularly attractive when dealing with mixed boundary-value problems where time-variation is further involved. Numerical examples are conducted to validate the accuracy of the proposed solution and to demonstrate the effects of material layering, geometry, and hydraulic boundary conditions on the contact performance of the material system.

Physics-informed machine learning method with space-time Karhunen-Loève expansions for forward and inverse partial differential equations

We propose a physics-informed machine-learning method based on space-time-dependent Karhunen-Loève expansions (KLEs) of the state variables and the residual least-square formulation of the solution of partial differential equations. This method, which we name dPICKLE, results in a reduced-order model for solving forward and inverse time-dependent partial differential equations. By conditioning KLEs on data, dPICKLE seamlessly assimilates data in forward and inverse solutions. KLEs are linear in unknown parameters. Because of this, and unlike physics-informed deep-learning methods based on the residual least-square formulation, for well-posed partial differential equation (PDE) problems, dPICKLE leads to linear least-square problems (directly for linear PDEs and after linearization for nonlinear PDEs), which guarantees a unique solution. The efficiency and accuracy of dPICKLE are demonstrated for linear and nonlinear forward and inverse problems via comparison with analytical, finite difference, and physics-informed neural network (PINN) solutions.

Global strong convexity and characterization of critical points of time-of-arrival-based source localization

In this work, we study a least-squares formulation of the source localization problem given time-of-arrival measurements. We show that the formulation, albeit non-convex in general, is globally strongly convex under certain condition on the geometric configuration of the anchors and the source and on the measurement noise. Next, we derive a characterization of the critical points of the least-squares formulation, leading to a bound on the maximum number of critical points under a very mild assumption on the measurement noise. In particular, the result provides a sufficient condition for the critical points of the least-squares formulation to be isolated. Prior to our work, the isolation of the critical points is treated as an assumption without any justification in the localization literature. The said characterization also leads to an algorithm that can find a global optimum of the least-squares formulation by searching through all critical points. We then establish an upper bound of the estimation error of the least-squares estimator. Finally, our numerical results corroborate the theoretical findings and show that our proposed algorithm can obtain a global solution regardless of the geometric configuration of the anchors and the source.

In situ process analytical technology for real time viable cell density and cell viability during live-virus vaccine production

Viable cell density (VCD) and cell viability (CV) are key performance indicators of cell culture processes in biopharmaceutical production of biologics and vaccines. Traditional methods for monitoring VCD and CV involve offline cell counting assays that are both labor intensive and prone to high variability, resulting in sparse sampling and uncertainty in the obtained data. Process analytical technology (PAT) approaches offer a means to address these challenges. Specifically, in situ probe-based measurements of dielectric spectroscopy (also commonly known as capacitance) can characterize VCD and CV continuously in real time throughout an entire process, enabling robust process characterization. In this work, we propose in situ dielectric spectroscopy as a PAT tool for real time analysis of live-virus vaccine (LVV) production. Dielectric spectroscopy was collected across 25 discreet frequencies, offering a thorough evaluation of the proposed technology. Correlation of this PAT methodology to traditional offline cell counting assays was performed, in which VCD and CV were both successfully predicted using dielectric spectroscopy. Both univariate and multivariate data analysis approaches were evaluated for their potential to establish correlation between the in situ dielectric spectroscopy and offline measurements. Univariate analysis strategies are presented for optimal single frequency selection. Multivariate analysis, in the form of partial least squares (PLS) regression, produced significantly higher correlations between dielectric spectroscopy and offline VCD and CV data, as compared to univariate analysis. Specifically, by leveraging multivariate analysis of dielectric information from all 25 spectroscopic frequencies measured, PLS models performed significantly better than univariate models. This is particularly evident during cell death, where tracking VCD and CV have historically presented the greatest challenge. The results of this work demonstrate the potential of both single and multiple frequency dielectric spectroscopy measurements for enabling robust LVV process characterization, suggesting that broader application of in situ dielectric spectroscopy as a PAT tool in LVV processes can provide significantly improved process understanding. To the best of our knowledge, this is the first report of in situ dielectric spectroscopy with multivariate analysis to successfully predict VCD and CV in real time during live virus-based vaccine production.

Solver algorithm for stabilized space-time formulation of advection-dominated diffusion problem

This article shows how to develop an efficient solver for a stabilized numerical space-time formulation of the advection-dominated diffusion transient equation. At the discrete space-time level, we approximate the solution using higher-order continuous B-spline basis functions in both spatial and temporal dimensions. This problem is very difficult to solve numerically using the standard Galerkin finite element method due to artificial oscillations present when the advection term dominates the diffusion term. However, a first-order constraint least-square formulation allows us to obtain numerical solutions avoiding oscillations. The advantages of space-time formulations are the use of high-order methods and the feasibility of developing space-time mesh adaptive techniques on well-defined discrete problems. We develop a solver for a least-square formulation to obtain a stabilized and symmetric problem on finite element meshes. The computational cost of our solver is bounded by the cost of the inversion of the space-time mass and stiffness (with one value fixed at a point) matrices and the cost of the GMRES solver applied for the symmetric and positive definite problem. We illustrate our findings on an advection-dominated diffusion space-time model problem and present two numerical examples: one with isogeometric analysis discretizations and the second one with an adaptive space-time finite element method.

Hybrid neural-network FEM approximation of diffusion coefficient in elliptic and parabolic problems

Abstract In this work we investigate the numerical identification of the diffusion coefficient in elliptic and parabolic problems using neural networks (NNs). The numerical scheme is based on the standard output least-squares formulation where the Galerkin finite element method (FEM) is employed to approximate the state and NNs act as a smoothness prior to approximate the unknown diffusion coefficient. A projection operation is applied to the NN approximation in order to preserve the physical box constraint on the unknown coefficient. The hybrid approach enjoys both rigorous mathematical foundation of the FEM and inductive bias/approximation properties of NNs. We derive a priori error estimates in the standard $L^2(\varOmega )$ norm for the numerical reconstruction, under a positivity condition which can be verified for a large class of problem data. The error bounds depend explicitly on the noise level, regularization parameter and discretization parameters (e.g., spatial mesh size, time step size and depth, upper bound and number of nonzero parameters of NNs). We also provide extensive numerical experiments, indicating that the hybrid method is very robust for large noise when compared with the pure FEM approximation.

A recursive system identification inertia estimator for traditional and converter-interfaced generators

This paper proposes a method to estimate the inertia constant of synchronous generators (SGs) and the virtual inertia provided by converter-interfaced generators (CIGs). It can be applied when the system is under normal operating conditions or after power imbalance events. The algorithm is based on a recursive system identification approach and uses the active power and bus frequency measurements proceeding from phasor measurement units. The measurements are preprocessed to improve the estimation accuracy. Subsequently, a model-independent rotor speed estimator is used to compute rotor speed variations (or internal frequency variations, in the case of CIGs). Afterwards, a recursive least-square equation error formulation is implemented to compute the SG and CIG parameters in the z-domain, which are later converted into their respective counterparts in the s-domain. The method is applied to several generation devices in the IEEE 39-bus and IEEE 118-bus test systems, including traditional SGs and CIGs providing virtual inertia. Numerical results show the accuracy and the low computational burden of the proposed method.

Hybrid Sphere Decoder for Long Prediction Horizon FCS-MPC

In finite control set model predictive control (FCS-MPC) strategies, extending the prediction horizon length provides important closed-loop performance improvements. However, the computational costs are increased in exponential fashion. Transforming the problem to an equivalent box-constrained integer least-squares (ILS) formulation enables the usage of sphere decoding algorithms (SDA) that can efficiently solve this problem. Recently, a K-best sphere decoder was proposed and designed for hardware platforms. This algorithm follows a breadth-first strategy different to the conventional SDA. In this work, a hybrid SDA that combines the merits of both the K-best SDA and the conventional SDA is proposed with the objective of increasing optimality likelihood and improve control performance. In particular, it is proposed that a K-best sphere decoder delivers a preliminary optimal solution. Then, a conventional SDA uses the available calculation time to search for a better solution. Simulation and experimental results confirm the validity of the proposal in terms of performance and computational efficiency.

Least-squares neural network (LSNN) method for scalar nonlinear hyperbolic conservation laws: Discrete divergence operator

A least-squares neural network (LSNN) method was introduced for solving scalar linear and nonlinear hyperbolic conservation laws (HCLs) in Cai et al. (2021, 2022). This method is based on an equivalent least-squares (LS) formulation and uses ReLU neural network as approximating functions, making it ideal for approximating discontinuous functions with unknown interface location. In the design of the LSNN method for HCLs, the numerical approximation of differential operators is a critical factor, and standard numerical or automatic differentiation along coordinate directions can often lead to a failed NN-based method. To overcome this challenge, this paper rewrites HCLs in their divergence form of space and time and introduces a new discrete divergence operator. As a result, the proposed LSNN method is free of penalization of artificial viscosity.Theoretically, the accuracy of the discrete divergence operator is estimated even for discontinuous solutions. Numerically, the LSNN method with the new discrete divergence operator was tested for several benchmark problems with both convex and non-convex fluxes, and was able to compute the correct physical solution for problems with rarefaction, shock or compound waves. The method is capable of capturing the shock of the underlying problem without oscillation or smearing, even without any penalization of the entropy condition, total variation, and/or artificial viscosity.

Absolute Nodal Coordinate Formulation-Based Shape Sensing Approach for Large Deformation: Plane Beam

The inverse finite element method (IFEM) is currently one of the most studied methods in the field of shape sensing, in other words, the reconstruction of the displacement field of a structure from discrete strain measures. The current research is still insufficient in applying IFEM to flexible structures undergoing large deformation that are in increasing demand, especially in terms of computational efficiency. Hence, an element-by-element IFEM approach based on absolute nodal coordinate formulation (ANCF) is developed in the paper. Taking the plane beam as the object, a class of gradient-deficient ANCF plane beam element is introduced to provide a concise nonlinear nodal displacement/strain relationship. Similar to IFEM, the inverse ANCF (IANCF) plane beam element is obtained in the form of least-square formulation, which means IANCF describes the deformation reconstruction problem as a nonlinear optimization problem. Because the computational complexity of solving nonlinear optimization problems increases rapidly with the increase of the number of decision variables, an element-by-element solution algorithm that solves each element relatively independently is adopted, and the explicit iterative formula is given by the Newton method. Besides, a curvature continuity constraint is introduced to improve the well-posed-ness of this problem and the smoothness of the reconstructed shape. Through numerical analysis, IANCF exhibits remarkable accuracy in various deformation degrees and its insensitivity to the weight factors inherited from IFEM. In the experiment conducted with surface-mounted distributed optical fiber sensors, the effectiveness of IANCF for practical structures is verified.

Convergence rate analysis of Galerkin approximation of inverse potential problem

In this work we analyze the inverse problem of recovering a space-dependent potential coefficient in an elliptic / parabolic problem from distributed observation. We establish novel (weighted) conditional stability estimates under very mild conditions on the problem data. Then we provide an error analysis of a standard reconstruction scheme based on the standard output least-squares formulation with Tikhonov regularization (by an H 1-seminorm penalty), which is then discretized by the Galerkin finite element method with continuous piecewise linear finite elements in space (and also backward Euler method in time for parabolic problems). We present a detailed error analysis of the discrete scheme, and provide convergence rates in a weighted for discrete approximations with respect to the exact potential. The error bounds explicitly depend on the noise level, regularization parameter and discretization parameter(s). Under suitable conditions, we also derive error estimates in the standard and interior L 2 norms. The analysis employs sharp a priori error estimates and nonstandard test functions. Several numerical experiments are given to complement the theoretical analysis.

Least-Squares Spectral Methods for ODE Eigenvalue Problems

We develop spectral methods for ODEs and operator eigenvalue problems that are based on a least-squares formulation of the problem. The key tool is a method for rectangular generalized eigenvalue problems, which we extend to quasimatrices and objects combining quasimatrices and matrices. The strength of the approach is its flexibility that lies in the quasimatrix formulation allowing the basis functions to be chosen arbitrarily (e.g. those obtained by solving nearby problems), and often giving high accuracy. We also show how our algorithm can easily be modified to solve problems with eigenvalue-dependent boundary conditions, and discuss reformulations as an integral equation, which often improves the accuracy.

Recovery of a space-time-dependent diffusion coefficient in subdiffusion: stability, approximation and error analysis

Abstract In this work we study an inverse problem of recovering a space-time-dependent diffusion coefficient in the subdiffusion model from the distributed observation, where the mathematical model involves a Djrbashian–Caputo fractional derivative of order $\alpha \in (0,1)$ in time. The main technical challenges of both theoretical and numerical analyses lie in the limited smoothing properties due to the fractional differential operator and high degree of nonlinearity of the forward map from the unknown diffusion coefficient to the distributed observation. We establish two conditional stability results using a novel test function, which leads to a stability bound in $L^2(0,T;L^2(\varOmega ))$ under a suitable positivity condition. The positivity condition is verified for a large class of problem data. Numerically, we develop a rigorous procedure for recovering the diffusion coefficient based on a regularized least-squares formulation, which is then discretized by the standard Galerkin method with continuous piecewise linear elements in space and backward Euler convolution quadrature in time. We provide a complete error analysis of the fully discrete formulation, by combining several new error estimates for the direct problem (optimal in terms of data regularity), a discrete version of fractional maximal $L^p$ regularity and a nonstandard energy argument. Under the positivity condition, we obtain a standard $\ell ^2(L^2(\varOmega ))$ error estimate consistent with the conditional stability. Further, we illustrate the analysis with some numerical examples.