Least-squares Optimization Research Articles

Theory and Design of Multizone Soundfield Reproduction Using Sparse Methods

Multizone soundfield reproduction over an extended spatial region is a challenging problem in acoustic signal processing. We introduce a method of reproducing a multizone soundfield within a desired region in reverberant environments. It is based on the identification of the acoustic transfer function (ATF) from the loudspeaker over the desired reproduction region using a limited number of microphone measurements. We assume that the soundfield is sparse in the domain of planewave decomposition and identify the ATF using sparse methods. The estimates of the ATFs are then used to derive the optimal least-squares solution for the loudspeaker filters that minimize the reproduction error over the entire reproduction region. Simulations confirm that the method leads to a significantly reduced number of required microphones for accurate multizone sound reproduction, while it also facilitates the reproduction over a wide frequency range. Practical experiments are used to verify the sparse planewave representation of the reverberant soundfield in a real-world listening environment.

Simulation of Water Temperature in a Small Pond Using Parametric Statistical Models: Implications of Climate Warming

AbstractChanges in temperature and precipitation patterns due to global warming are likely to affect the quantity and quality of water in different water bodies. Water temperature modeling techniques are usually employed to study the effects of global climate change on stream and river ecosystems. This study aims to identify a suitable air–water temperature relationship for a small aquatic pond in a semiarid region of India and examine the effects of increased water temperature on the small pond’s attributes. The performance of two parametric statistical models—simple linear regression (SLR) and four-parameter nonlinear logistic regression (NLR) models—was evaluated. The developed models were field tested for mean, minimum, and maximum air–water temperatures on daily, weekly, and monthly timescales. The model parameters were estimated from the measured air–water temperature time-series data using the least-squares optimization method. Model performance was evaluated using three statistical indicators—the ...

Phase and amplitude tracking for seismic event separation

We have developed a method to decompose seismic records into atomic events, each defined by a smooth phase function and a smooth amplitude function. This decomposition is intrinsically nonlinear and calls for a nonconvex least-squares optimization formulation, along the lines of full-waveform inversion. To overcome the lack of convexity, we have developed an iterative refinement-expansion scheme to initialize and track the phase and amplitude for each atomic event. For short, we called the method phase tracking. The initialization is carried out by applying multiple signal classification to a few seed traces in which events can be separated and identified by their arrival times and amplitudes. We then construct the initial solution at the seed traces using linear phase functions from the arrival times and constant amplitude functions, assuming the medium is mostly dispersion free. We refine this initial solution to account for dispersion and imperfect knowledge of the wavelet at the seed traces by fitting the observed data using a gradient descent method. The resulting phase and amplitude functions are then carefully expanded across the traces in an adequately smooth way to match the whole data record. We have evaluated the proposed method on two synthetic records and a field record. Because the parametrization of the seismic events is physically meaningful, it also enables a simple form of bandwidth extension of the observed shot record to unobserved low and high frequencies. We tested this procedure on the same shot records. Bandwidth extension is in principle helpful to initialize full-waveform inversion with frequency sweeps and enhanced its resolution.

Mathematical modelling and parameter identification of a stainless steel annealing furnace

A new, comprehensive mathematical model of continuous annealing furnaces is developed, under consideration of both the radiative and convective heat transfer of the furnace components. Based on measured normal operating data from an industrial stainless steel plant, parameter identification is basically carried out using a nonlinear least-squares optimization algorithm for the whole annealing furnace, to estimate optimal values of uncertain parameters, such as emissivities. Due to the complexity of the model, a sequential approach for parameter identification is proposed and implemented, i.e. the parameter set is divided into different subsets, and the parameter estimation is carried out sequentially in several steps and iterations. The performance of the model with the estimated parameters is then evaluated on a different test data set. It is shown that the obtained model can predict temperature evolutions along the furnace in good agreement to measured data, under both steady-state and transient conditions. The presented model is suitable for controller design and process optimization.

Implementation of a rigorous least-squares modification of Stokes’ formula to compute a gravimetric geoid model over Saudi Arabia (SAGEO13)

A gravimetric geoid model (SAGEO13) is computed for the Kingdom of Saudi Arabia using a rigorous stochastic computational method. The computational methodology is based on a combination of least-squares (LS) modification of Stokes’ formula and the additive corrections for topographic, ellipsoidal, atmospheric, and downward continuation effects on the geoid solution. In this study, we used terrestrial gravity data, a digital elevation model (SRTM3), and seven global geopotential models (GGMs) to compute a new geoid model for Saudi Arabia. The least-squares coefficients are derived based on the optimisation of the input modification parameters. The gravimetric solution and its additive corrections are computed based on the optimum LS coefficients. Compared to GPS-levelling data, SAGEO13 shows a fit of 18 cm (RMS) after using a 4-parameter fitting model.

Two-Level Space–Time Domain Decomposition Methods for Three-Dimensional Unsteady Inverse Source Problems

As the number of processor cores on supercomputers becomes larger and larger, algorithms with high degree of parallelism attract more attention. In this work, we propose a two-level space---time domain decomposition method for solving an inverse source problem associated with the time-dependent convection---diffusion equation in three dimensions. We introduce a mixed finite element/finite difference method and a one-level and a two-level space---time parallel domain decomposition preconditioner for the Karush---Kuhn---Tucker system induced from reformulating the inverse problem as an output least-squares optimization problem in the entire space-time domain. The new full space---time approach eliminates the sequential steps in the optimization outer loop and the inner forward and backward time marching processes, thus achieves high degree of parallelism. Numerical experiments validate that this approach is effective and robust for recovering unsteady moving sources. We will present strong scalability results obtained on a supercomputer with more than 1000 processors.

X-ray computed tomography using sparsity based regularization

Sparsity based iterative reconstruction methods have been demonstrated to significantly improve the image quality using only part of the large volume complete projection data of computed tomography (CT). In this work, CT reconstruction was treated as a penalized weighted least-squares optimization with the consideration of photon statistics of X-ray detection and converted it to a constrained total variation (TV) minimization based on sparsity of the first order derivative of images. This constrained optimization was solved by alternate TV minimization through the first order primal-dual (FOPD) algorithm and enforcement of the constraints of data fidelity and non-negativity through projection onto convex sets (POCS). The proposed FOPD–POCS algorithm implemented an implicit balance mechanism between FOPD and POCS, which starts from strong POCS and weak FOPD TV minimization to quickly reach the feasible region, and then reinforces FOPD by skipping POCS steps given that TV solution satisfies the data fidelity constraint. The FOPD–POCS algorithm is compared with the classical adaptive-steepest-descent-POCS (ASD-POCS) method for reconstruction performance and with a convergent simultaneous FOPD algorithm (“Sidkey-A7”) for reconstruction stability. The results demonstrate that 1) although FOPD is theoretically more rigorous than steepest descent for TV minimization, no significant advantage of FOPD over ASD is observed; 2) the weighted least-squares criterion is better suited than the least-squares that was used in the original ASD-POCS reconstruction, particularly at high noise levels when using low radiation dose at each projection view; and 3) with the implicit balance mechanism between FOPD and POCS, FOPD–POCS can achieve a stable solution with a simple selection of the FOPD step size and significantly reduce computational cost. In summary, FOPD–POCS can provide an easily controlled reconstruction to avoid complicated parameter tuning and to yield satisfying images in a short time.

Achieving Target Emulsion Drop Size Distributions Using Population Balance Equation Models of High-Pressure Homogenization

Population balance equation (PBE) models have been used extensively to predict drop size distributions (DSDs) of dispersed phase systems. In our previous publications, we have used the PBE framework to develop increasingly sophisticated process models for oil-in-water emulsification in high-pressure homogenizers. The goal of this study was to utilize these PBE models for integrated emulsion product and process design through the formulation and solution of homogenizer optimization problems. For a specified number of homogenization passes, the initial amount of surfactant and the pressure of each pass were determined by solving a nonlinear least-squares optimization problem such that the target DSD were achieved. Three alternative objective functions that differed with respect to the distribution specification and the penalty on surfactant usage were formulated and solved for different target DSDs. The model predictions were successfully validated by performing homogenization experiments using the optimized formulation and homogenization variables.

Error analysis of aspheric surface with reference datum.

Severe requirements of location tolerance provide new challenges for optical component measurement, evaluation, and manufacture. Form error, location error, and the relationship between form error and location error need to be analyzed together during error analysis of aspheric surface with reference datum. Based on the least-squares optimization method, we develop a least-squares local optimization method to evaluate form error of aspheric surface with reference datum, and then calculate the location error. According to the error analysis of a machined aspheric surface, the relationship between form error and location error is revealed, and the influence on the machining process is stated. In different radius and aperture of aspheric surface, the change laws are simulated by superimposing normally distributed random noise on an ideal surface. It establishes linkages between machining and error analysis, and provides an effective guideline for error correcting.

Note: Improving low-light-level image detection sensitivity with higher speed using auxiliary sinusoidal light signal.

An improved active imaging method, which upgraded the detection sensitivity by applying an auxiliary sawtooth wave light signal, was reported. Nevertheless, such method sacrificed the imaging speed. To speed up imaging, a sinusoidal light signal is used instead and superposed with the undetectable low-light-level signal on the image sensor. After acquiring a superimposed image set in one sine wave cycle, an unbiased low-light-level image estimation is obtained by using least-square optimization. Through probabilistic analysis and experimental study, we demonstrate that the sinusoidal signal could improve the detection sensitivity 1/3 faster than the sawtooth wave signal.

Kinetic modeling of polyurethane pyrolysis using non-isothermal thermogravimetric analysis

The pyrolysis of polyurethane was studied by dynamic thermogravimetry analysis (TGA). The studied polyurethane is used as organic binder in casting process to make sand cores and molds. A semi-empirical model is presented that can be used to describe polyurethane pyrolysis occurring during TGA experiments. This model assumes that the polyurethane is pyrolysed by several parallel independent reactions. The kinetic parameters of polyurethane pyrolysis were evaluated by fitting the model to the experimental data obtained by TGA over a wide variety of heating rates. A nonlinear least-squares optimization method is employed in the fitting procedure. A hybrid objectives based simultaneously on the mass (TG) and mass loss rate (DTG) curves has been used in the least-squares method. The values of the activation energy obtained by the non-linear fitting were then recalculated by the methods of Kissinger and Friedmand. Furthermore, the parameters obtained in the present paper were then compared with those reported in the literature.

Radio interferometric gain calibration as a complex optimization problem

Recent developments in optimization theory have extended some traditional algorithms for least-squares optimization of real-valued functions (Gauss–Newton, Levenberg–Marquardt, etc.) into the domain of complex functions of a complex variable. This employs a formalism called the Wirtinger derivative, and derives a full-complex Jacobian counterpart to the conventional real Jacobian. We apply these developments to the problem of radio interferometric gain calibration, and show how the general complex Jacobian formalism, when combined with conventional optimization approaches, yields a whole new family of calibration algorithms, including those for the polarized and direction-dependent gain regime. We further extend the Wirtinger calculus to an operator-based matrix calculus for describing the polarized calibration regime. Using approximate matrix inversion results in computationally efficient implementations; we show that some recently proposed calibration algorithms such as StefCal and peeling can be understood as special cases of this, and place them in the context of the general formalism. Finally, we present an implementation and some applied results of CohJones, another specialized direction-dependent calibration algorithm derived from the formalism.

Uncertainty quantification for nuclear density functional theory and information content of new measurements.

Statistical tools of uncertainty quantification can be used to assess the information content of measured observables with respect to present-day theoretical models, to estimate model errors and thereby improve predictive capability, to extrapolate beyond the regions reached by experiment, and to provide meaningful input to applications and planned measurements. To showcase new opportunities offered by such tools, we make a rigorous analysis of theoretical statistical uncertainties in nuclear density functional theory using Bayesian inference methods. By considering the recent mass measurements from the Canadian Penning Trap at Argonne National Laboratory, we demonstrate how the Bayesian analysis and a direct least-squares optimization, combined with high-performance computing, can be used to assess the information content of the new data with respect to a model based on the Skyrme energy density functional approach. Employing the posterior probability distribution computed with a Gaussian process emulator, we apply the Bayesian framework to propagate theoretical statistical uncertainties in predictions of nuclear masses, two-neutron dripline, and fission barriers. Overall, we find that the new mass measurements do not impose a constraint that is strong enough to lead to significant changes in the model parameters. The example discussed in this study sets the stage for quantifying and maximizing the impact of new measurements with respect to current modeling and guiding future experimental efforts, thus enhancing the experiment-theory cycle in the scientific method.

Climate sensitivity

ABSTRACTEarth has been habitable through most of its history, but the anthropogenically mediated greenhouse effect, if sufficiently strong, can threaten Earth's long-standing equability. This paper's main aim is to determine the strength of the anthropogenic greenhouse effect (the climate sensitivity) from observational data and basic physics alone, without recourse to the parameterisations of earth-system models and their inevitable uncertainties. A key finding is that the sensitivity can be constrained by harmonising historical records of land and ocean temperatures with observations of potential climate-change drivers in a non-steady state, energy-balance equation via a least-squares optimisation. The global temperature increase, for a CO2doubling, is found to lie (95 % confidence limits) between 3.0oC and 6.3oC, with a best estimate of +4oC. Under a business-as-usual scenario, which assumes that there will be no significant change in people's attitudes and priorities, Earth's surface temperature is forecast to rise by 7.9oC over the land, and by 3.6oC over the oceans, by the year 2100. Global temperature rise has slowed in the last decade, leading some to question climate predictions of substantial 21st-Century warming. A formal runs test, however, shows that the recent slowdown is part of the normal behaviour of the climate system.

COMPARISON OF LEVENBERG-MARQUARDT BASED LEAST SQUARES METHOD AND A HEURISTIC TECHNIQUE FOR ELECTRICITY DEMAND ESTIMATION

This paper focuses on demand estimation of electricity in Iran using artificial bee swarm optimization (ABSO) algorithm which is a recently invented metaheuristic technique. For this aim, two types of exponential and quadratic models are investigated to estimate the Iran’s electricity demand. These models are defined based on socio-economic indicators of population, gross domestic product (GDP), import and export figures. Owing to the fluctuations of the economic indicators and nonlinearity of the electricity demand, an efficient technique must be employed to find optimal or near optimal values of the models’ weighting factors. This paper proposes ABSO as an efficient approach for solving this problem. The available data of electricity demand in Iran from 1981 to 1999 is used for finding the optimal weighing factors and the data from 2000 to 2005 is used for testing the models. In order to evaluate the performance of the proposed methodology, the results are compared with the result obtained by the traditional nonlinear least-squares optimization method of and Levenberg–Marquardt (LM)

Analysis of the scintillation mechanism in a pressurized 4He fast neutron detector using pulse shape fitting

An empirical investigation of the scintillation mechanism in a pressurized 4He gas fast neutron detector was conducted using pulse shape fitting. Scintillation signals from neutron interactions were measured and averaged to produce a single generic neutron pulse shape from both a 252Cf spontaneous fission source and a (d,d) neutron generator. An expression for light output over time was then developed by treating the decay of helium excited states in the same manner as the decay of radioactive isotopes. This pulse shape expression was fitted to the measured neutron pulse shape using a least-squares optimization algorithm, allowing an empirical analysis of the mechanism of scintillation inside the 4He detector. A further understanding of this mechanism in the 4He detector will advance the use of this system as a neutron spectrometer. For 252Cf neutrons, the triplet and singlet time constants were found to be 970 ns and 686 ns, respectively. For neutrons from the (d,d) generator, the time constants were found to be 884 ns and 636 ns. Differences were noted in the magnitude of these parameters compared to previously published data, however the general relationships were noted to be the same and checked with expected trends from theory. Of the excited helium states produced from a 252Cf neutron interaction, 76% were found to be born as triplet states, similar to the result from the neutron generator of 71%. The two sources yielded similar pulse shapes despite having very different neutron energy spectra, validating the robustness of the fits across various neutron energies.

Integrated 3-D stress determination by hydraulic fracturing in multiple inclined boreholes beneath an underground cavern

This paper presents a complete three-dimensional stress determination using hydraulic fracturing data from three inclined boreholes, drilled from the floor of an underground cavern at a depth of about 100m. Both conventional hydraulic fracturing (HF) and hydraulic testing of pre-existing fractures (HTPF) were carried out at all test points to acquire reliable data and conduct the integrated stress analysis. We determined 3-D stress states using a numerical inversion code that integrates the entire data set from HF and HTPF methods, and employs a nonlinear least-squares optimization routine based on a modified Levenberg–Marquardt method and a finite-difference Jacobian algorithm. We present the trend of complete three-dimensional stress states, with depth expressed by correlation equations. We compare the 3-D stress inversion result for the integration of all data from the three boreholes with the results determined independently for each individual borehole. The results showed that the maximum principal stress was subhorizontal and oriented approximately NNE–SSW, and the ratio of maximum to minimum principal stress was 2.0 on average. The inverted scatter of misfit remained within 10% for both the integrated analysis of the entire data set and the measurement data for each individual borehole. From these findings, we conclude that the 3-D stress determination made by integration of HF and HTPF data from multiple inclined boreholes resulted in a reliable inversion of the 3-D stress field.

Parameter estimation in a Holzapfel-Ogden law for healthy myocardium.

A central problem in biomechanical studies of personalized human left ventricular (LV) modelling is to estimate material properties from in vivo clinical measurements. In this work we evaluate the passive myocardial mechanical properties inversely from the in vivo LV chamber pressure–volume and strain data. The LV myocardium is described using a structure-based orthotropic Holzapfel–Ogden constitutive law with eight parameters. In the first part of the paper we demonstrate how to use a multi-step non-linear least-squares optimization procedure to inversely estimate the parameters from the pressure–volume and strain data obtained from a synthetic LV model in diastole. In the second part, we show that to apply this procedure to clinical situations with limited in vivo data, additional constraints are required in the optimization procedure. Our study, based on three different healthy volunteers, demonstrates that the parameters of the Holzapfel–Ogden law could be extracted from pressure–volume and strain data with a suitable multi-step optimization procedure. Although the uniqueness of the solution cannot be addressed using our approaches, the material response is shown to be robustly determined.

A measure theoretic approach to linear inverse atmospheric dispersion problems

Using measure theoretic arguments, we provide a general framework for describing and studying the general linear inverse dispersion problem where no a priori assumptions on the source function has been made (other than assuming that it is indeed a source, i.e. not a sink). We investigate the source-sensor relationship and rigorously state solvability conditions for when the inverse problem can be solved using a least-squares optimization method. That is, we derive conditions for when the least-squares problem is well-defined.

Full Waveform Inversion Using Student’s t Distribution: a Numerical Study for Elastic Waveform Inversion and Simultaneous-Source Method

Seismic full waveform inversion (FWI) has primarily been based on a least-squares optimization problem for data residuals. However, the least-squares objective function can suffer from its weakness and sensitivity to noise. There have been numerous studies to enhance the robustness of FWI by using robust objective functions, such as l 1-norm-based objective functions. However, the l 1-norm can suffer from a singularity problem when the residual wavefield is very close to zero. Recently, Student’s t distribution has been applied to acoustic FWI to give reasonable results for noisy data. Student’s t distribution has an overdispersed density function compared with the normal distribution, and is thus useful for data with outliers. In this study, we investigate the feasibility of Student’s t distribution for elastic FWI by comparing its basic properties with those of the l 2-norm and l 1-norm objective functions and by applying the three methods to noisy data. Our experiments show that the l 2-norm is sensitive to noise, whereas the l 1-norm and Student’s t distribution objective functions give relatively stable and reasonable results for noisy data. When noise patterns are complicated, i.e., due to a combination of missing traces, unexpected outliers, and random noise, FWI based on Student’s t distribution gives better results than l 1- and l 2-norm FWI. We also examine the application of simultaneous-source methods to acoustic FWI based on Student’s t distribution. Computing the expectation of the coefficients of gradient and crosstalk noise terms and plotting the signal-to-noise ratio with iteration, we were able to confirm that crosstalk noise is suppressed as the iteration progresses, even when simultaneous-source FWI is combined with Student’s t distribution. From our experiments, we conclude that FWI based on Student’s t distribution can retrieve subsurface material properties with less distortion from noise than l 1- and l 2-norm FWI, and the simultaneous-source method can be adopted to improve the computational efficiency of FWI based on Student’s t distribution.