Least-squares Objective Function Research Articles

Semi-numerical simulation of groundwater flow induced by periodic forcing with a case-study at an island aquifer

This paper presents an efficient method to analyze the aquifer response to periodic forcing. A complex transformation is used to change the time-dependent linear aquifer model into an equivalent time-independent problem of a complex function (referred to as the “complex amplitude” hereafter). The magnitude of the complex amplitude represents the amplitude of the sinusoidal fluctuation of the groundwater level, and the argument of the complex amplitude, the phase shift of the fluctuation. Different iterative methods can be used to obtain the numerical solution. A semi-analytical numerical solution of the groundwater level, which is numerical for the spatial variables and analytical for the time variable, can be obtained by implementing a simple inverse complex transform to the complex amplitude. The advantages of the time-independent method include ruling out of initial conditions, direct outputs of the amplitude and phase shift of different sinusoidal components of the water level fluctuations. On the contrary, the numerical solutions of the time-dependent model need further computational efforts for the analyses of the amplitudes and phase shifts of different sinusoidal components, and have to consider an initial condition whose impacts on the periodicity have to be eliminated by running sufficiently great number of tidal circles. A case study at an unconfined aquifer formed by fill materials near Lam Chau Island, Hong Kong SAR, PR China shows the applicability of the semi-numerical solution to analyze tide-induced watertable fluctuations. The aquifer parameters were estimated by minimizing a least-squares objective function to fit the complex amplitude of semidiurnal and diurnal components of the observed water level fluctuations. Compared with the parameter estimations of some of the previous analytical solutions, the numerical method gives not only better fitting to the water level observations, but also more aquifer parameters.

Automated water content reconstruction of zero-offset borehole ground penetrating radar data using simulated annealing

The automated inversion of water content profiles from first arrival travel time data collected with zero-offset borehole ground penetrating radar is discussed. The inversion algorithm sets out to find the water content profile that minimizes a least-squares objective function representing the difference between the modeled and measured first arrival travel time. Ray-tracing analysis is used to determine the travel time for direct and critically refracted paths to identify the first arrival travel time. This automated method offers improvement over a previously presented graphical solution that considers both direct and critical refractions. Specifically, this approach can identify thinner layers and allow for the incorporation of uncertainty in the travel time measurements to determine the depth-specific uncertainty of the inferred water content profile through multiple simulations using a stochastic approach.

Gap detection with electromagnetic terahertz signals

We apply an inverse problem formulation to determine characteristics of a defect from a perturbed electromagnetic interrogating signal. A defect (gap) inside of a dielectric material causes a disruption, via reflections and refractions at the material interfaces, of the windowed interrogating signal. We model the electromagnetic waves inside the material with Maxwell's equations. This leads to a non-standard, nonlinear optimization problem for the dimensions and location of the defect. Using simulations as forward solves, we employ a Newton-based, iterative optimization scheme to a novel modified least-squares objective function. Numerical results are given in tables and plots, standard errors are calculated, and computational issues are addressed.

The ℓ1oblique procrustes problem

In this paper, we reconsider the well-known oblique Procrustes problem where the usual least-squares objective function is replaced by a more robust discrepancy measure, based on the e1 norm or smooth approximations of it. We propose two approaches to the solution of this problem. One approach is based on convex analysis and uses the structure of the problem to permit a solution to the e1 norm problem. An alternative approach is to smooth the problem by working with smooth approximations to the e1 norm, and this leads to a solution process based on the solution of ordinary differential equations on manifolds. The general weighted Procrustes problem (both orthogonal and oblique) can also be solved by the latter approach. Numerical examples to illustrate the algorithms which have been developed are reported and analyzed.

A curvilinear gradient path method for optimization of biological systems models

We describe a novel curvilinear gradient method for least squares parameter estimation suitable for a wide class of biological systems models. At each iteration step, a minimum search is performed along a curvilinear trajectory corresponding to the steepest descent path of the least-squares objective function. The approach combines the benefits of both local (steepest-descent) and non-local (Newton-step) descent methods to offer a robust performance, especially suited to medium and large scale problems associated with biological models.

Robust registration for computer-integrated orthopedic surgery: Laboratory validation and clinical experience

In order to provide navigational guidance during computer-integrated orthopedic surgery, the anatomy of the patient must first be registered to a medical image or model. A common registration approach is to digitize points from the surface of a bone and then find the rigid transformation that best matches the points to the model by constrained optimization. Many optimization criteria, including a least-squares objective function, perform poorly if the data include spurious data points (outliers). This paper describes a statistically robust, surface-based registration algorithm that we have developed for orthopedic surgery. To find an initial estimate, the user digitizes points from predefined regions of bone that are large enough to reliably locate even in the absence of anatomic landmarks. Outliers are automatically detected and managed by integrating a statistically robust M-estimator with the iterative-closest-point algorithm. Our in vitro validation method simulated the registration process by drawing registration data points from several sets of densely digitized surface points. The method has been used clinically in computer-integrated surgery for high tibial osteotomy, distal radius osteotomy, and excision of osteoid osteoma.

Aminoglycoside nephrotoxicity: modeling, simulation, and control.

The main constraints on the administration of aminoglycosides are the risks of nephrotoxicity and ototoxicity, which can lead to acute, renal, vestibular, and auditory toxicities. In the present study we focused on nephrotoxicity. No reliable predictor of nephrotoxicity has been found to date. We have developed a deterministic model which describes the pharmacokinetic behavior of aminoglycosides (with a two-compartment model), the kinetics of aminoglycoside accumulation in the renal cortex, the effects of aminoglycosides on renal cells, the resulting effects on renal function by tubuloglomerular feedback, and the resulting effects on serum creatinine concentrations. The pharmacokinetic parameter values were estimated by use of the NPEM program. The estimated pharmacodynamic parameter values were obtained after minimization of the least-squares objective function between the measured and the calculated serum creatinine concentrations. A simulation program assessed the influences of the dosage regimens on the occurrence of nephrotoxicity. We have also demonstrated the relevancy of modeling of the circadian rhythm of the renal function. We have shown the ability of the model to fit with 49 observed serum creatinine concentrations for a group of eight patients treated for endocarditis by comparison with 49 calculated serum creatinine concentrations (r(2) = 0.988; P < 0.001). We have found that for the same daily dose, the nephrotoxicity observed with a thrice-daily administration schedule appears more rapidly, induces a greater decrease in renal function, and is more prolonged than those that occur with less frequent administration schedules (for example, once-daily administration). Moreover, for once-daily administration, we have demonstrated that the time of day of administration can influence the incidence of aminoglycoside nephrotoxicity. The lowest level of nephrotoxicity was observed when aminoglycosides were administered at 1:30 p.m. Clinical application of this model might make it possible to adjust aminoglycoside dosage regimens by taking into account both the efficacies and toxicities of the drugs.

Optimal weighting design for distributed parameter systems estimation

AbstractThis paper presents a method which aims at improving parameter estimation in dynamical systems. The general principle of the method is based on a modification of the least‐squares objective function by means of a weighting operator, in view to improve the conditioning of the identification problem. First we recall a previous work using variational calculus in order to obtain the weighting operators through a linear equation. Then we propose a new approach which consists of determining the weights by formulating an optimization problem including positive semidefinite constraints (linear matrix inequalities, LMI). Copyright © 2001 John Wiley &amp; Sons, Ltd.

Investigation of shape optimization for the design of life extension options for an F/A-18 airframe FS 470 bulkhead

Shape reworking and bonded reinforcement are two procedures available to extend the fatigue life of cracked metallic aircraft components already in service. Typically for realistic applications, the design of reworks has been undertaken through trial-and-error finite element analyses, guided by simplified analytical formulations, the aim being to achieve reduced stresses while generally restricting the shape boundaries to circular and straight segments. In the present work, an automated sensitivity-based shape optimization procedure has been developed for the optimal design of free-form reworks and bonded reinforcements and demonstrated through application to a realistic practical problem, the F/A-18 FS 470 bulkhead. Here, a least-squares objective function written in terms of selected stress quantities is used, together with multiple basis shape vectors to specify allowable shape changes. For the rework option, a unique optimal solution has been determined, which achieves a region of constant boundary hoop stress which is 27 per cent less than for the nominal initial uncracked geometry, even though material removal at the critical location is accounted for. Subsequently, for the bonded reinforcement analyses, two distinct optimal designs were determined, corresponding to the case where either shear or peel stresses in the adhesive layer are minimized. Both the shape of the adhesive layer and the reinforcement are allowed to vary, and significant improvements over a conventional reinforcement design are obtained, as assessed by the reduction in peak stresses. These results indicate that the numerical shape optimization procedures presented can provide designs of reworks and bonded reinforcements, which offer significant improvements over standard designs.

A Method for Numerical Simulation of Single Limb Ground Contact Events: Application to Heel-Toe Running

The objective of this work was to develop a method to simulate single-limb ground contact events, which may be applied to study musculoskeletal injuries associated with such movements. To achieve this objective, a three-dimensional musculoskeletal model was developed consisting of the equations of motion for the musculoskeletal system, and models for the muscle force generation and ground contact elements. An optimization framework and a weighted least-squares objective function were presented that generated muscle stimulation patterns that optimally reproduced subject-specific movement data. Experimental data were collected from a single subject to provide initial conditions for the simulation and tracking data for the optimization. As an example application, a simulation of the stance phase of running was generated. The results showed that the average difference between the simulation and subject's ground reaction force and joint angle data was less than two inter-trial standard deviations. Further, there was good agreement between the model's muscle excitation patterns and experimentally collected electromyography data. These results give confidence in the model to examine musculoskeletal loading during a variety of landing movements and to study the effects of various factors associated with injury. Limitations were examined and areas of improvement for the model were presented.

UCODE, a computer code for universal inverse modeling

This article presents the US Geological Survey computer program UCODE, which was developed in collaboration with the US Army Corps of Engineers Waterways Experiment Station and the International Ground Water Modeling Center of the Colorado School of Mines. UCODE performs inverse modeling, posed as a parameter-estimation problem, using nonlinear regression. Any application model or set of models can be used; the only requirement is that they have numerical (ASCII or text only) input and output files and that the numbers in these files have sufficient significant digits. Application models can include preprocessors and postprocessors as well as models related to the processes of interest (physical, chemical and so on), making UCODE extremely powerful for model calibration. Estimated parameters can be defined flexibly with user-specified functions. Observations to be matched in the regression can be any quantity for which a simulated equivalent value can be produced, thus simulated equivalent values are calculated using values that appear in the application model output files and can be manipulated with additive and multiplicative functions, if necessary. Prior, or direct, information on estimated parameters also can be included in the regression. The nonlinear regression problem is solved by minimizing a weighted least-squares objective function with respect to the parameter values using a modified Gauss–Newton method. Sensitivities needed for the method are calculated approximately by forward or central differences and problems and solutions related to this approximation are discussed. Statistics are calculated and printed for use in (1) diagnosing inadequate data or identifying parameters that probably cannot be estimated with the available data, (2) evaluating estimated parameter values, (3) evaluating the model representation of the actual processes and (4) quantifying the uncertainty of model simulated values. UCODE is intended for use on any computer operating system: it consists of algorithms programmed in perl, a freeware language designed for text manipulation and Fortran90, which efficiently performs numerical calculations.

Conjugate-gradient preconditioning methods for shift-variant PET image reconstruction

Gradient-based iterative methods often converge slowly for tomographic image reconstruction and image restoration problems, but can be accelerated by suitable preconditioners. Diagonal preconditioners offer some improvement in convergence rate, but do not incorporate the structure of the Hessian matrices in imaging problems. Circulant preconditioners can provide remarkable acceleration for inverse problems that are approximately shift-invariant, i.e., for those with approximately block-Toeplitz or block-circulant Hessians. However, in applications with nonuniform noise variance, such as arises from Poisson statistics in emission tomography and in quantum-limited optical imaging, the Hessian of the weighted least-squares objective function is quite shift-variant, and circulant preconditioners perform poorly. Additional shift-variance is caused by edge-preserving regularization methods based on nonquadratic penalty functions. This paper describes new preconditioners that approximate more accurately the Hessian matrices of shift-variant imaging problems. Compared to diagonal or circulant preconditioning, the new preconditioners lead to significantly faster convergence rates for the unconstrained conjugate-gradient (CG) iteration. We also propose a new efficient method for the line-search step required by CG methods. Applications to positron emission tomography (PET) illustrate the method.

Improving the Prediction Capability of Radial Basis Function Networks

Radial basis function networks (RBFNs) have been well established as a class of supervised neural networks. Because they do not require a priori process information and they possess localized interpolation properties, they are attractive for the empirical modeling of complex nonlinear multivariable processes such as those associated with the chemical industry. In this paper, classical regularization theory is used to develop a technique for improving the prediction capabilities of RBFNs by incorporating process knowledge obtained from physicochemical relationships through modification of the objective function. An analysis of this new objective function is provided and compared to the conventional least-squares objective function. Several chemical process examples are provided to demonstrate the improved predictive capability of this modified RBFN.

Inversion for interface structure using teleseismic traveltime residuals

An inversion method is presented for the reconstruction of interface geometry between two or more crustal layers from teleseismic traveltime residuals. The method is applied to 2-D models consisting of continuous interfaces separating constant-velocity layers. The forward problem of determining ray paths and traveltimes between incident wave fronts below the structure and receivers located on the Earth—s surface is solved by an efficient and robust shooting method. A conjugate gradient method is employed to solve the inverse problem of minimizing a least-squares type objective function based on the difference between observed and calculated traveltimes. Teleseismic data do not accurately constrain average vertical structure, so a priori information in the form of layer velocities and average layer thicknesses is required. Synthetic tests show that the method can be used to reconstruct interface geometry accurately, even in the presence of data noise. Tests also show that, if layer velocities and initial interface positions are poorly chosen, lateral structure is still recoverable. The inversion method was applied to previously published teleseismic data recorded by an in-line array of portable seismographs that traversed the northern margin of the Musgrave Block, central Australia. The solution based on interface parametrization is consistent with models given by other studies that used the same data but different methods, most notably the standard tomographic approach that inverts for velocity rather than interface structure.

Computational aspects of finite element modeling in EEG source localization.

A comparison is made of two different implementations of the finite element method (FEM) for calculating the potential due to dipole sources in electroencephalography (EEG). In one formulation (the direct method) the total potential is the unknown that is solved for and the dipole source is directly incorporated into the model. In the second formulation (the subtraction method) the unknown is the difference between the total potential and the potential due to the same dipole in an infinite region of homogeneous conductivity, corresponding to the region where the dipole is located. Both methods have the same FEM system matrix. However, the subtraction method requires an additional calculation of flux integrations along the edges of the elements in the computation of the right-hand side (RHS) vector. It is shown that the subtraction method is usually more accurate in the forward modeling, provided the flux integrations are computed accurately. Errors in calculating the flux integrations may result in large errors in the forward solution due to the ill-conditioned nature of the FEM system matrix caused by the Neumann boundary condition. To minimize the errors, closed-form expressions for the flux integrations are used for both linear and quadratic triangular elements. It is also found that FEM forward modeling errors may cause false extrema in the least-square objective function obtained from the boundary potential, near boundaries between media of differing conductivity. Multiple initial guesses help eliminate the possibility of the solution getting trapped in these false extrema.

Experimental Evaluation of Nonisothermal, First-Order Reaction Kinetics

A procedure for the analyses of nonisothermal, batch reaction data is presented. Nonlinear regression analyses based on a least-squares objective function yield kinetic rate expression parameters from a single experiment. Data collected for the thermal decomposition of benzoyl peroxide in xylene were analyzed, illustrating differential and integral analyses. The Arrhenius frequency factor and activation energy determined were in agreement with published results obtained by isothermal experiments. In addition, the regression analyses determined the initial concentration of the reactant.

Reliability of Horizontal Well Performance On a Field Scale Through Automatic History Matching

Abstract An efficient procedure for the determination of the error bounds (95% confidence intervals) of forecasted horizontal well production rates is presented. In particular, we advocate the use of automatic history matching (based on non-linear regression) to arrive at the best reservoir parameter values (typically porosities and permeabilities) for a given reservoir structure (i.e., for a given grid block representation). If the Gauss-Newton-method is used for the parameter search, the sensitivity coefficients required for the estimation of the standard error of the parameters and the well production rates are computed as part of the parameter search algorithm. Using typical techniques from statistical analysis (and the statistical properties of multivariable non-linear regression estimates), one can readily compute the confidence intervals of the predicted production rates. These confidence intervals around the expected reservoir behaviour incorporate the uncertainty in all reservoir parameters. As a result, a realistic measure of the uncertainty in the model predictions of future reservoir performance is obtained. The proposed procedure is illustrated with a case study. Introduction The final task of a reservoir simulation study is often the computation of future production levels of the history matched reservoir under alternative depletion plans. Moreover, the reservoir engineer often examines the sensitivity of the forecasted performance to different reservoir descriptions in an effort to establish the risks associated with a particular depletion plan. Typically, this is accomplished by varying key reservoir parameters such as porosities and permeabilities and determining how much the performance of a particular well or of the whole reservoir is affected. With the continuous development of powerful workstations and desktop personal computers, a renewed interest in automatic history marching procedures can be observed in the past few years. For the minimization of the resulting least squares (LS) objective function, two different approaches have been widely used: nonlinear regression(1–6) and optimal control methods(7–11). Recently, the simulated annealing(12) method has also proven to be a reliable procedure. In practical applications some form of reguralization(13,14) is always required to overcome the very serious problem of ill-conditioning, particularly as the number of parameters increases to more than ten or so. In our approach to automatic history matching, we minimize the resulting nonlinear LS objective function using an efficient implementation of the Gauss-Newton method(1). The necessary computation of the sensitivity coefficients when he Gauss-Newton method is used enables us to readily determine the uncertainty in the estimated parameter values(2). Furthermore, if we compute the sensitivity coefficients at future times where the performance of the reservoir is forecasted, we can generate 95% confidence intervals of the model predictions. The confidence intervals around the expected reservoir behaviour incorporate the uncertainty in all reservoir parameters and hence this approach provides a more realistic measure of the uncertainty in the model predictions of future reservoir behaviour(15). When several different grid representations of the reservoir are available with a known probability of being correct, the above described approach yields conditional expectations of future production rates which can subsequently be combined to arrive at an overall probability distribution of key variables, like cumulative oil production, WOR, GOR, etc.

The dual regularization approach to seismic velocity inversion

Differential semblance optimization (DSO) is a novel way of approaching a class of inverse problems arising in exploration seismology. The promising feature of the DSO method is that it replaces a non-smooth, highly non-convex cost function (the output least-squares (OLS) objective function) with a smooth cost function that is amenable to standard (local) optimization algorithms. The OLS problem can be written abstractly as a partially linear least-squares problem with linear constraints. The DSO objective function is derived from the associated quadratic penalty function. One way to view the DSO objective function is as a regularization of a function that is dual (in a certain sense) to the OLS objective function. Under suitable assumptions, the DSO method defines a parametrized path of minimizers converging to the desired solution and that, for certain values of the parameter, standard optimization techniques can be used to find a point on the path. The results of the theory are illustrated in the plane wave detection problem, a simple model problem for velocity inversion.

Penalized weighted least-squares image reconstruction for positron emission tomography

Presents an image reconstruction method for positron-emission tomography (PET) based on a penalized, weighted least-squares (PWLS) objective. For PET measurements that are precorrected for accidental coincidences, the author argues statistically that a least-squares objective function is as appropriate, if not more so, than the popular Poisson likelihood objective. The author proposes a simple data-based method for determining the weights that accounts for attenuation and detector efficiency. A nonnegative successive over-relaxation (+SOR) algorithm converges rapidly to the global minimum of the PWLS objective. Quantitative simulation results demonstrate that the bias/variance tradeoff of the PWLS+SOR method is comparable to the maximum-likelihood expectation-maximization (ML-EM) method (but with fewer iterations), and is improved relative to the conventional filtered backprojection (FBP) method. Qualitative results suggest that the streak artifacts common to the FBP method are nearly eliminated by the PWLS+SOR method, and indicate that the proposed method for weighting the measurements is a significant factor in the improvement over FBP.

Elastic constants of orthotropic composite materials using plate resonance frequencies, classical lamination theory and an optimized three-mode rayleigh formulation

This paper shows how the four independent elastic constants (longitudinal and transverse Young's moduli, in-plane shear modulus and major Poisson's ratio) of an orthotropic material may be extracted from the modal resonance data of a freely-supported rectangular thin plate made from the material, using the classical lamination theory and an optimized three-mode Rayleigh formulation with a suitably formed least-squares objective function. Results are obtained for orthotropy ratios of unity (aluminum), about three (unidirectional E-glass/vinylester) and about 13 (graphite/epoxy), and plate aspect ratios of unity and about two. The results suggest that the method proposed has potential for rapid characterization of elastic properties of advanced composites.