Least-squares Objective Function Research Articles

Identification of the thermo-physical properties of a stratified tissue. Adiabatic hypodermic wall

Biological living tissues possess thermophysical properties that are difficult to measure directly and their estimation is very important in order to decide the appropriate treatment procedure (e.g. place of inoculation and appropriate amount of drug dosage) in case an abnormality is detected. In the present paper, we numerically investigate, for the first time, the determination of several thermo-physical blood-tissue properties of a one-dimensional, multi-layered biological skin tissue subjected to external heating. In contrast to the usual parabolic model that assumes infinitely fast propagation of the heat signal, in the present paper, the bio-heat transfer in such a biological body is mathematically modelled by a hyperbolic partial differential equation, which takes into account that the thermal wave speed is finite. On the skin surface a convective boundary condition holds taking into account the heat exchange with the environment, whilst on the most inner wall of the tissue an adiabatic boundary condition applies. Then, in this framework, the piecewise constant thermo-physical properties of interest given by the thermal conductivity, heat capacity and blood perfusion rate are accurately and stably reconstructed from non-intrusive temperature measurements on the inner and outer boundaries of the multi-layered tissue by minimizing a weighted nonlinear least-squares objective function.

Nonlinear conjugate gradient method for identifying Young's modulus of the elasticity imaging inverse problem

Application of elasticity imaging inverse problem to identify Young's modulus in the elasticity problems in human's life is an interesting research area. In this study, we identify the modulus of elasticity for solving elasticity imaging inverse problem using a modified output least-squares method. Numerical convergence in the displacements of the direct problem for elasticity is investigated. To study the elasticity imaging inverse problem in an optimization framework, we utilize the sensitivity and adjoint problems to conceptualize a new model for computing the gradient of the minimizer. Discrete formulae in the model are then used to devise a scheme for an efficient computation gradient of the modified output least-squares objective function using the nonlinear conjugate gradient method. Numerical experiments demonstrate the effectiveness of the proposed technique.

Crawling subsampling for multivariate spatial autoregression model in large-scale networks

In network data analysis, multivariate spatial autoregression (MSAR) models may be used to analyze the autocorrelation among multiple responses. With large-scale networks, the estimation for MSAR on the entire network is computationally expensive. In this case, the subsampling method could be adopted. This approach selects a sample of nodes and then uses the estimate based on the sample to approximate the estimate on the full data. However, traditional sampling methods cannot obtain unbiased parameter estimates. Considering the second-order friend information of sampled nodes, we propose the crawling subsampling (CS) method for a general framework. Thereafter, based on the sampled data only, we construct the least-squares objective function. Under certain conditions, the computational complexity of optimizing the objective function is linear with the sample size ns. The identification condition for the parameters on the sampled network is theoretically provided. The sample size order requirement is provided, and the asymptotic properties of the least-squares estimators are investigated. The numerical results for the simulated and real data are presented to demonstrate the performance of the proposed CS method and least-squares estimator for the MSAR model.

Estimation and clustering for partially heterogeneous single index model

In this paper, our goal is to estimate the homogeneous parameter and cluster the heterogeneous parameters in a partially heterogeneous single index model (PHSIM). To achieve the goal, the minimization criterion for such a single index model is first transformed into a least-squares optimization problem in the population form. Based on the least-squares objective function, we introduce an empirical version for the PHSIM. By minimizing such an empirical version, we estimate the homogeneous parameter and the subgroup-averages of the heterogeneous index directions, and then use a fusion penalized method to identify the subgroup structure of the PHSIM. By the proposed methodologies, the homogeneous parameter and the heterogeneous index directions can be consistently estimated, and the heterogeneous parameters can be consistently clustered. Moreover, the new clustering procedure is simple and robust. Simulation studies are carried out to examine the performance of the proposed methodologies.

Contact resistance analysis applied to simultaneous estimation of thermal properties of metals

This work aims to present a method for simultaneously estimating the thermal conductivity, k, and specific heat, cp, in samples of stainless steel 304 and cemented carbide by accounting the imperfect thermal contact at the heater–sample interface. A transient one-dimensional heat conduction model with constant thermal properties is used. The metallic sample is placed in the middle of a polyimide heater and a thermal insulator. A constant heat flux is applied on the specimen upside surface and a thermal insulation condition is maintained on the opposing surface, where a type T thermocouple measures the temperature response. Contact resistance is determined and accounted for as a reducing factor on heat flux. Instead of considering a lumped model, a microscopic approach of the contacting regions is used to describe both the surface roughness and the fluid gap. With the intention of guaranteeing simultaneous and reliable estimates for both thermal properties, sensitivity analysis is used to establish heat flux intensity, experiment duration, time step for data acquisition, and other characteristics of the experiments. To simultaneously achieve both thermal properties, the optimization method BFGS (Broyden–Fletcher–Goldfarb–Shanno) is employed to minimize a least-squares objective function comparing experientially measured and numerically calculated temperatures. The numerical solution for the transient conduction problem is determined with COMSOL, which solves the transient heat diffusion problem by discretizing it applying the finite element method (FEM). The linear system of equations is solved using the PARDISO solver and backward differentiation formula (BDF) method. Heat flux is estimated employing the sequential function specification method (SFSM) as a means of verifying the robustness of parameter estimation and experimental procedure. Moreover, uncertainty analysis is performed to confirm the quality of the results obtained. Finally, the uncertainty analysis outcomes and thermal properties estimated are compared with literature.

Self-Paced Nonnegative Matrix Factorization for Hyperspectral Unmixing

The presence of mixed pixels in the hyperspectral data makes unmixing to be a key step for many applications. Unsupervised unmixing needs to estimate the number of endmembers, their spectral signatures, and their abundances at each pixel. Since both endmember and abundance matrices are unknown, unsupervised unmixing can be considered as a blind source separation problem and can be solved by nonnegative matrix factorization (NMF). However, most of the existing NMF unmixing methods use a least-squares objective function that is sensitive to the noise and outliers. To deal with different types of noises in hyperspectral data, such as the noise in different bands (band noise), the noise in different pixels (pixel noise), and the noise in different elements of hyperspectral data matrix (element noise), we propose three self-paced learning based NMF (SpNMF) unmixing models in this article. The SpNMF models replace the least-squares loss in the standard NMF model with weighted least-squares losses and adopt a self-paced learning (SPL) strategy to learn the weights adaptively. In each iteration of SPL, atoms (bands or pixels or elements) with weight zero are considered as complex atoms and are excluded, while atoms with nonzero weights are considered as easy atoms and are included in the current unmixing model. By gradually enlarging the size of the current model set, SpNMF can select atoms from easy to complex. Usually, noisy or outlying atoms are complex atoms that are excluded from the unmixing model. Thus, SpNMF models are robust to noise and outliers. Experimental results on the simulated and two real hyperspectral data sets demonstrate that our proposed SpNMF methods are more accurate and robust than the existing NMF methods, especially in the case of heavy noise.

From constant- to variable-density inverse extended Born modeling

For quantitative seismic imaging, iterative least-squares reverse time migration is the recommended approach. The existence of an inverse of the forward modeling operator would considerably reduce the number of required iterations. In the context of the extended model, such a pseudoinverse does exist and is built as a weighted version of the adjoint that accounts for the deconvolution, geometric spreading, and uneven illumination. The application of the pseudoinverse Born modeling is based on constant-density acoustic media, which is a limiting factor for practical applications. To consider density perturbations, we have adopted and investigated two approaches. The first one is a generalization of a recent study proposing to recover acoustic perturbations from the angle-dependent response of the pseudoinverse Born modeling operator. This new version is based on the weighted least-squares objective function. The method not only provides more robust results, but it also offers the flexibility to include constraints in the objective function to reduce the parameters’ crosstalk. We also propose an alternative approach based on Taylor expansion that does not require any Radon transform. Numerical examples based on a simple model and the Marmousi2 models using correct and incorrect background models for the variable density pseudoinverse Born modeling verify the effectiveness of the weighted least-squares method when compared with the other two approaches. The Taylor expansion approach appears to contain too many artifacts for successful applicability.

Calibration of Johnson-Cook Model Using Heterogeneous Thermo-Mechanical Tests

In the present work, a calibration methodology based on full-field measurements from heterogeneous thermo-mechanical tests is introduced. In order to evaluate the feasibility of this methodology, the widely adopted Johnson-Cook model is chosen. This calibration methodology relies on the Finite Element Model Updating (FEMU) method to take full advantage of the information contained in full-field measurements and thus, simultaneously calibrate the three terms of the Johnson-Cook model regarding strain hardening, temperature and strain rate. A virtual experimental database composed of strain fields and load output from three heterogeneous tests performed at different average strain rates is used. The minimisation of the least-square objective function is performed by the gradient-based Levenberg-Marquardt optimisation algorithm. A detailed analysis of the virtual database and objective function is performed and discussed. Furthermore, the robustness of the proposed methodology is tested with noisy data.

An Integrated Approach to Registration and Fusion of Hyperspectral and Multispectral Images

Combining a hyperspectral (HS) image and a multispectral (MS) image - an example of image fusion - can result in a spatially and spectrally high-resolution image. Despite the plethora of fusion algorithms in remote sensing, a necessary prerequisite, namely registration, is mostly ignored. This limits their application to well-registered images from the same source. In this article, we propose and validate an integrated registration and fusion approach (code available at https://github.com/zhouyuanzxcv/Hyperspectral). The registration algorithm minimizes a least-squares (LSQ) objective function with the point spread function (PSF) incorporated together with a nonrigid freeform transformation applied to the HS image and a rigid transformation applied to the MS image. It can handle images with significant scale differences and spatial distortion. The fusion algorithm takes the full high-resolution HS image as an unknown in the objective function. Assuming that the pixels lie on a low-dimensional manifold invariant to local linear transformations from spectral degradation, the fusion optimization problem leads to a closed-form solution. The method was validated on the Pavia University, Salton Sea, and the Mississippi Gulfport datasets. When the proposed registration algorithm is compared to its rigid variant and two mutual information-based methods, it has the best accuracy for both the nonrigid simulated dataset and the real dataset, with an average error less than 0.15 pixels for nonrigid distortion of maximum 1 HS pixel. When the fusion algorithm is compared with current state-of-the-art algorithms, it has the best performance on images with registration errors as well as on simulations that do not consider registration effects.

Real2Sim

This paper presents a method for optimizing visco-elastic material parameters of a finite element simulation to best approximate the dynamic motion of real-world soft objects. We compute the gradient with respect to the material parameters of a least-squares error objective function using either direct sensitivity analysis or an adjoint state method. We then optimize the material parameters such that the simulated motion matches real-world observations as closely as possible. In this way, we can directly build a useful simulation model that captures the visco-elastic behaviour of the specimen of interest. We demonstrate the effectiveness of our method on various examples such as numerical coarsening, custom-designed objective functions, and of course real-world flexible elastic objects made of foam or 3D printed lattice structures, including a demo application in soft robotics.

Gauss–Newton Optimization for Phase Recovery From the Bispectrum

Phase recovery from the bispectrum is a central problem in speckle interferometry which can be posed as an optimization problem minimizing a weighted nonlinear least-squares objective function. We look at two different formulations of the phase recovery problem from the literature, both of which can be minimized with respect to either the recovered phase or the recovered image. Previously, strategies for solving these formulations have been limited to gradient descent or quasi-Newton methods. This article explores Gauss–Newton optimization schemes for the problem of phase recovery from the bispectrum. We implement efficient Gauss–Newton optimization schemes for all the formulations. For the two of these formulations which optimize with respect to the recovered image, we also extend to projected Gauss–Newton to enforce element-wise lower and upper bounds on the pixel intensities of the recovered image. We show that our efficient Gauss–Newton schemes result in better image reconstructions with no or limited additional computational cost compared to previously implemented first-order optimization schemes for phase recovery from the bispectrum. MATLAB implementations of all methods and simulations are made publicly available in the BiBox repository on Github.

Super-Resolution Phase Retrieval from Designed Coded Diffraction Patterns.

Super-resolution phase retrieval is an inverse problem that appears in diffractive optical imaging (DOI) and consists in estimating a high-resolution image from low-resolution phaseless measurements. DOI has three diffraction zones where the data can be acquired, known as near, middle, and far fields. Recent works have studied super-resolution phase retrieval under a setup that records coded diffraction patterns at the near and far fields. However, the attainable resolution of the image is mainly governed by the sensor characteristics, whose cost increases in proportion to the resolution. Also, these methodologies lack theoretical analysis. Hence, this work derives super-resolution models from low-resolution coded phaseless measurements at any diffraction zone that in contrast to prior contributions, the attainable resolution of the image is determined by the resolution of the coded aperture. For the proposed models, the existence of a unique solution (up to a global unimodular constant) is guaranteed with high probability, which can be increased by designing the coded aperture. Therefore, a strategy that designs the spatial distribution of the coded aperture is developed. Additionally, a super-resolution phase retrieval algorithm that minimizes a smoothed nonconvex least-squares objective function is proposed. The method first approximates the image by a spectral algorithm, which is then refined based upon a sequence of alternate steps. Simulation results show that the proposed algorithm overcomes state-of-the-art methods in reconstructing the high-resolution image. In addition, the reconstruction quality using designed coded apertures is higher than that of the non-designed ensembles.

Globally Optimal Least-Squares ARMA Model Identification is an Eigenvalue Problem

We show that globally optimal least-squares identification of autoregressive moving-average (ARMA) models is an eigenvalue problem (EP). The first order optimality conditions of this identification problem constitute a system of multivariate polynomial equations, in which most variables appear linearly. This system is basically a multiparameter eigenvalue problem (MEP), which we solve by iteratively building a so-called block Macaulay matrix, the null space of which is block multi-shift-invariant. The set of all stationary points of the optimization problem, i.e., the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> -tuples of eigenvalues and eigenvectors of the MEP, follows from a standard EP related to the multidimensional realization problem in that null space. At least one of these <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> -tuples corresponds to the global minimum of the original least-squares objective function. Contrary to existing heuristic techniques, this approach yields the globally optimal parameters of the ARMA model. We provide a numerical example to illustrate the new identification method.

Land seismic multiparameter full waveform inversion in elastic VTI media by simultaneously interpreting body waves and surface waves with an optimal transport based objective function

SUMMARY Land seismic multiparameter full waveform inversion in anisotropic media is challenging because of high medium contrasts and surface waves. With a data-residual least-squares objective function, the surface wave energy usually masks the body waves and the gradient of the objective function exhibits high values in the very shallow depths preventing from recovering the deeper part of the earth model parameters. The optimal transport objective function, coupled with a Gaussian time-windowing strategy, allows to overcome this issue by more focusing on phase shifts and by balancing the contributions of the different events in the adjoint-source and the gradients. We first illustrate the advantages of the optimal transport function with respect to the least-squares one, with two realistic examples. We then discuss a vertical transverse isotropic (VTI) example starting from a quasi 1-D isotropic initial model. Despite some cycle-skipping issues in the initial model, the inversion based on the windowed optimal transport approach converges. Both the near-surface complexities and the variations at depth are recovered.

Compressive least-squares migration with on-the-fly Fourier transforms

Least-squares reverse time migration is a powerful approach for true-amplitude seismic imaging of complex geologic structures, but the successful application of this method is currently hindered by its enormous computational cost, as well as its high memory requirements for computing the gradient of the objective function. We have tackled these problems by introducing an algorithm for low-cost sparsity-promoting least-squares migration using on-the-fly Fourier transforms. We formulate the least-squares migration objective function in the frequency domain (FD) and compute gradients for randomized subsets of shot records and frequencies, thus significantly reducing data movement and the number of overall wave equations solves. By using on-the-fly Fourier transforms, we can compute an arbitrary number of monochromatic FD wavefields with a time-domain (TD) modeling code, instead of having to solve individual Helmholtz equations for each frequency, which becomes computationally infeasible when moving to high frequencies. Our numerical examples demonstrate that compressive imaging with on-the-fly Fourier transforms provides a fast and memory-efficient alternative to TD imaging with optimal checkpointing, whose memory requirements for a fixed background model and source wavelet are independent of the number of time steps. Instead, the memory and additional computational costs grow with the number of frequencies and determine the amount of subsampling artifacts and crosstalk. In contrast to optimal checkpointing, this offers the possibility to trade the memory and computational costs for image quality or a larger number of iterations and is advantageous in new computing environments such as the cloud, where computing is often cheaper than memory and data movement.

A least-squares correlation-based full traveltime inversion for shallow subsurface velocity reconstruction

Accurate estimation of shallow subsurface velocity models with complex topography is crucial for statics corrections and imaging deep structures. The correlation-based wave-equation traveltime inversion (CWTI) method is suitable for estimating such shallow subsurface velocity structures. However, the CWTI objective function suffers an inherent resolution-loss problem because the traveltime weighted crosscorrelation misfit does not fall to zero even when the model is perfectly matched. Furthermore, the Born-approximation-based CWTI gradient cannot provide an effective model update during each iteration. To overcome these problems, we have developed a least-squares correlation-based full traveltime inversion (LCFTI) method, in which the least-squares correlation-based objective function was designed to minimize the traveltime weighted difference between the autocorrelation and the crosscorrelation. By incorporating the autocorrelation, LCFTI indicates better convergence and higher resolution than CWTI. The LCFTI model updates are derived using the Rytov approximation to avoid incorrect model updates by emphasizing phase matching. Furthermore, to accurately simulate wave propagation, we use the spectral-element method as the modeling engine, in which the mesh of the complex topography is flexibly represented. Synthetic and field data examples are performed to demonstrate the effectiveness of the proposed method in shallow subsurface velocity reconstruction.

Kinematics performance oriented smoothing method to plan tool orientations for 5-axis ball-end CNC machining

When adopting 5-axis machine to mill the parts, it is desired to avoid the drastic change of tool orientation for improving the kinematics performance of 5-axis machining while ensuring no machining interferences. For this purpose, a kinematics performance oriented smoothing method is proposed to plan the tool orientations, which is focused specifically on minimizing the angular accelerations imposed on the rotary axes of 5-axis machine. In this method, with several specified representative tool orientations (RTOs), two B-spline curves, which represent the displacements of the rotary axes, are used to join smoothly the RTOs together and then to determine the tool orientations at other areas. The solutions for the two B-spline curves are achieved by solving a least-square objective function which minimizes the angular accelerations of the rotary axes. To restricting simultaneously the interpolated tool orientations in the geometric feasible domains (GFDs) of tool motion, a simple alternate strategy of first smoothing the tool orientation and then checking the machining interference is developed so that tool orientation planning and its geometric constraints are decoupled and the complicated constraint optimization process of tool orientation can be greatly simplified. Since the proposed method works in the machine coordinate system (MCS), it can not only ensure the smooth motions of the rotary axes without the machining interferences, but also can generate directly the rotary axis orders. Finally, the proposed method is validated by the experiments.

Frequency-Resolved Optical Gating Recovery via Smoothing Gradient

Frequency-resolved optical gating (FROG) is a popular technique for complete characterization of ultrashort laser pulses. The acquired data in FROG, called FROG trace, is the Fourier magnitude of the product of the unknown pulse with a time-shifted version of itself, for several different shifts. To estimate the pulse from the FROG trace, we propose an algorithm that minimizes a smoothed non-convex least-squares objective function. The method consists of two steps. First, we approximate the pulse by an iterative spectral algorithm. Then, the attained initialization is refined based upon a sequence of block stochastic gradient iterations. The algorithm is theoretically simple, numerically scalable, and easy-to-implement. Empirically, our approach outperforms the state-of-the-art when the FROG trace is incomplete, that is, when only few shifts are recorded. Simulations also suggest that the proposed algorithm exhibits similar computational cost compared to a state-of-the-art technique for both complete and incomplete data. In addition, we prove that in the vicinity of the true solution, the algorithm converges to a critical point. A Matlab implementation is publicly available at https://github.com/samuelpinilla/FROG.

Fuzzy Adaptive Regularization Method for Inverse Steady-State Heat Transfer Problem

For the steady-state heat transfer process, a fuzzy adaptive regularization method (FARM) is proposed to estimate the distributed thermal boundary condition in heat transfer system. First, the relationship model between temperatures at measurement points and parameters to be estimated is established based on sensitivity matrix. The regularization term is introduced into the least-squares objective function, and then the distributed thermal boundary condition is estimated by optimizing the new objective function. A fuzzy inference mechanism is developed to ensure the adaptive ability of FARM in which the regularization parameter is updated based on the residual norm between calculated and measured temperatures at measurement points and the norm of inversion parameters. Taking the plate heat conduction system and fluid–solid conjugate heat transfer system as research objects, the effects of the parameter distribution, the number of measurement points, and measurement errors on the inversion results are discussed by numerical experiments, and comparison with the classical regularization method is also conducted. Results indicate that FARM exhibits a good adaptive ability.

Interpreting and Extending the Guided Filter via Cyclic Coordinate Descent.

The guided filter (GF) is a widely used smoothing tool in computer vision and image processing. However, to the best of our knowledge, few papers investigate the mathematical connection between this filter and the least-squares optimization. In this paper, we first interpret the guided filter as the cyclic coordinate descent (CCD) solver of a least-squares objective function. This discovery implies an extension approach to generalize the guided filter since we can change the least-squares objective function and define new filters as the first pass iteration of the CCD solver of modified objective functions. In addition, referring to the iterative minimizing procedure of the CCD, we can derive new rolling filtering schemes. So, we are reasonable to say that our discovery not only reveals an approach to design new GF-like filters adapting to specific requirements of applications but also offers thorough explanations for two rolling filtering schemes of the guided filter as well as the method to extend them. Experiments prove our new proposed filters and rolling filtering schemes could produce state-of-the-art results.