Selection Of Regularization Parameter Research Articles

An efficient inversion algorithm for atmospheric remote sensing with application to UV limb observations

In this paper we present a retrieval algorithm for atmospheric remote sensing. The algorithm combines Tikhonov regularization and the iteratively regularized Gauss–Newton method and is devoted to the solution of multi-parameter inverse problems with simple bounds on the variables. The basic features of the algorithm: the solution of the bound-constrained minimization problem, the selection of the optimal regularization parameter, the derivation of the global regularization matrix and the characterization of the solution (error analysis) are discussed in detailed. The inversion algorithm is applied to ozone retrieval from SCIAMACHY limb scatter measurements in the ultraviolet spectral range.

A regularized inverse approach to ultrasonic pulse-echo imaging

An inverse-theoretic approach to ultrasonic pulse-echo imaging based on nonquadratic regularization is presented, and its effectiveness is investigated computationally by: 1) evaluating the quality of the reconstruction of speckle-based images as a function of the transmit-receive bandwidth and focal number of the transducer; 2) comparing the reconstructed images with those obtained by using conventional B-mode imaging. The L-curve and the generalized cross-validation methods were evaluated as automatic regularization parameter selection techniques. The inversion using regularization produced better results than conventional B-mode imaging for high signal-to-noise ratios (SNRs). A lower bound of 30 dB for the SNR was found for this study, below which several of the image features were lost during the reconstruction process in order to control the distortion due to the noise.

Application of Bayesian Regularized BP Neural Network Model for Trend Analysis, Acidity and Chemical Composition of Precipitation in North Carolina

Bayesian regularized back-propagation neural network (BRBPNN) was developed for trend analysis, acidity and chemical composition of precipitation in North Carolina using precipitation chemistry data in NADP. This study included two BRBPNN application problems: (i) the relationship between precipitation acidity (pH) and other ions (NH4 +, NO3 −, SO4 2−, Ca2+, Mg2+, K+, Cl− and Na+) was performed by BRBPNN and the achieved optimal network structure was 8-15-1. Then the relative importance index, obtained through the sum of square weights between each input neuron and the hidden layer of BRBPNN(8-15-1), indicated that the ions' contribution to the acidity declined in the order of NH4 + > SO4 2− > NO3 −; and (ii) investigations were also carried out using BRBPNN with respect to temporal variation of monthly mean NH4 +, SO4 2− and NO3 − concentrations and their optimal architectures for the 1990–2003 data were 4-6-1, 4-6-1 and 4-4-1, respectively. All the estimated results of the optimal BRBPNNs showed that the relationship between the acidity and other ions or that between NH4 +, SO4 2−, NO3 − concentrations with regard to precipitation amount and time variable was obviously nonlinear, since in contrast to multiple linear regression (MLR), BRBPNN was clearly better with less error in prediction and of higher correlation coefficients. Meanwhile, results also exhibited that BRBPNN was of automated regularization parameter selection capability and may ensure the excellent fitting and robustness. Thus, this study laid the foundation for the application of BRBPNN in the analysis of acid precipitation.

Scale Recognition, Regularization Parameter Selection, and Meyer's G Norm in Total Variation Regularization

We investigate how TV regularization naturally recognizes the scale of individual features of an image, and we show how this perception of scale depends on the amount of regularization applied to the image. We give an automatic method driven by the geometry of the image for finding the minimum value of the regularization parameter needed to remove all features below a user-chosen threshold. We explain the relation of Meyer's G norm to the perception of scale, which provides a more intuitive understanding of this norm. We consider other applications of this ability to recognize scale, including the multiscale effects of TV regularization and the rate of loss of image features of various scales as a function of increasing amounts of regularization. Several numerical results are given.

Modern regularisation techniques in data analysis for space plasma observations

This paper is aimed to summarize the main philosophy of the Regularisation technique for inverse problems in space plasma observations. This unified approach is based around a general criterion for the Regularisation parameter selection. This approach was tested on various space plasma observation problems. It is demonstrated that for these problems a deviation is necessary from the traditional Least Squares Method in favour of the Regularisation technique. Particular problems were considered in more details in other publications by the authors. However, this paper is a first attempt to emphasize mathematical peculiarities of the Regularisation approach in space plasma observations in a general manner in order to provide a useful and versatile schema for the wide range of space plasma problems.

Iteratively adaptive regularization in inverse modeling with Bayesian outlook – application on geophysical data

An algorithm of model-based nonlinear inversion scheme with applications on geophysical examples is proposed. In the framework of classical least squares optimization, the algorithm uses adaptively controlled smoothness constraint in an iterative minimization scheme through an appropriate selection of regularization parameter. A new formula in computing regularization parameter, which is a variant of Engl's formalism of a posteriori regularization is proposed. A regularized Gauss–Newton type optimization scheme along with conjugate gradient algorithm is used for nonlinear error minimization. The applicability of the algorithm is demonstrated through numerical experiments inverting synthetically generated vertical electrical sounding, first arrival travel time refraction seismic data and travel time data of Rayleigh wave generated due to acoustic emission source over homogeneous and isotropic medium.

Patch near-field acoustical holography in cylindrical geometry

Spatial discrete Fourier transform-based near-field acoustical holography is known to suffer from windowing effects since the measurement aperture is necessarily finite. The latter effect can be mitigated by using patch holography, in which the measured field is extended beyond the measurement aperture based on successive smoothing operations. In this article, the application of a patch holography algorithm to cylindrical geometries is described. In planar geometry, initial zero-padding can be applied to the hologram pressure to an arbitrary degree in both in-plane directions since the pressure magnitude is expected to limit ultimately towards zero in both directions. In a cylindrical geometry, zero-padding can be implemented axially in the same way, but the number of zeros added in the circumferential direction is necessarily determined by the angular sample spacing owing to the periodic nature of the field in this direction. By using both numerical simulation and experimental results, it is shown that the patch extrapolation procedure works well in the cylindrical case, and a discussion of rigid reflecting boundaries is also included. Finally, reconstruction results obtained by the combined use of pressures measured in the finite aperture and two different regularization parameter selection techniques are compared, and the effect of projection distance is discussed.

3D near-to-surface conductivity reconstruction by inversion of VETEM data using the distorted Born iterative method

In this paper, three-dimensional (3D) subsurface imaging by inversion of data obtained from the very early time electromagnetic system (VETEM) is presented. The distorted Born iterative method is used to match the internal nonlinear property of the 3D inversion problem. The frequency explored is from 0.062 to 3.052 MHz, the high end of which is beyond the induction range. No reported 3D approximation works well for this frequency range, thus the 3D stabilized bi-conjugate-gradient fast Fourier transform method is invoked as the forward solver, in which the total current formulation other than the induced current formulation is used to achieve higher stability. With the use of the reciprocity theorem, the Fréchet derivatives are computed rapidly by two calls to the forward solver. Tikhonov regularization is used so that the contribution from small singular values is suppressed to warrant a smooth solution. The selection of regularization parameter follows a heuristic rule as used in the Levenberg–Marquardt algorithm so that the iteration is stable. Processing the whole data set at one time is prohibitively time-consuming and expensive. The localized property of the geophysical problem is exploited to decompose the original large problem into a number of smaller subproblems. The nonlinear inversion is implemented subsequently on each subproblem first. The inverted conductivity profiles are then combined together to arrive at a complete conductivity image under the whole data domain. The proposed method is first tested on synthetic data and then applied to the VETEM data from the Pit 9 complex of the Department of Energy.

Application of discrete data consistency conditions for selecting regularization parameters in PET attenuation map reconstruction

Simultaneous emission and transmission measurement is appealing in PET due to the matching of geometrical conditions between emission and transmission and reduced acquisition time for the study. A potential problem remains: when transmission statistics are low, attenuation correction could be very noisy. Although noise in the attenuation map can be controlled through regularization during statistical reconstruction, the selection of regularization parameters is usually empirical. In this paper, we investigate the use of discrete data consistency conditions (DDCC) to optimally select one or two regularization parameters. The advantages of the method are that the reconstructed attenuation map is consistent with the emission data and that it accounts for particularity in the emission reconstruction algorithm and acquisition geometry. The methodology is validated using a computer-generated whole-body phantom for both emission and transmission, neglecting random events and scattered radiation. MAP-TR was used for attenuation map reconstruction, while 3D OS-EM is used for estimating the emission image. The estimation of regularization parameters depends on the resolution of the emission image controlled by the number of iterations in OS-EM. The computer simulation shows that, on one hand, DDCC regularized attenuation map reduces propagation of the transmission scan noise to the emission image, while on the other hand DDCC prevents excessive attenuation map smoothing that could result in resolution mismatch artefacts between emission and transmission.

A boundary elements pseudo heat source method formulation for inverse analysis of solidification problems

An efficient methodology is presented to solve inverse solidification problems. In the procedure, the latent heat effects are implemented by introducing pseudo heat sources near the moving interface. The material properties can be temperature dependent. To account for the nonlinear part of the governing differential equations, a finite-boundary element formulation is employed. To reduce the oscillations in the solution, a sequential regularization scheme is used. A procedure for proper selection of regularization parameters is presented. To smooth the solutions further, a secondary regularization scheme is introduced and employed. Two complete examples are presented to demonstrate the applicability and the accuracy of the methods.

Selection of Multiple Regularization Parameters in Local Ridge Regression Using Evolutionary Algorithms and Prediction Risk Optimization

Selection of Multiple Regularization Parameters in Local Ridge Regression Using Evolutionary Algorithms and Prediction Risk Optimization

Direct analytic model of the L-curve for Tikhonov regularization parameterselection

We present an approximation to a direct, non-parametrized analytic expressionfor the L-curve used in the regularization of ill-conditioned linear systems which isconstructed starting from the exact solution for the case of a linear operator withjust one singular value. For the general case, this expression suggests onechoice of Tikhonov regularization parameter per singular value at whichto compute the curvature of the log–log L-curve plot. It suffices to testa small number of candidate values in the vicinity of the mean squareresidual of the unregularized solution. Monte Carlo simulations show thatchoosing the maximum curvature generated from within this set of candidatevalues selects an optimum value of the regularization parameter whichyields a solution whose norm typically lies within a fraction 0.125/κL of the exact L-curvevalue, where κLis the maximum log–log curvature at the L-curve corner. For large scale ill-posedproblems, a new regularization parameter selection method based on theconstruction of an L-curve from a single sequence of conjugate gradient iterates ispresented. It includes a new stopping rule for terminating the iterative solution oflarge scale least squares problems. Numerical experiments show that the accuracyof the regularized solution obtained with the value of the regularization parameterat the corner of this curve is close to that of the exact L-curve solution.

An information approach to regularization parameter selection under modelmisspecification

We review the information approach to regularization parameter selection and itsinformation complexity extension for the solution of discrete ill posedproblems. An information criterion for regularization parameter selectionwas first proposed by Shibata in the context of ridge regression as anextension of Takeuchi’s information criterion. In the information approach, theregularization parameter value is chosen to maximize the mean expected loglikelihood (MELL) of a model whose parameters are estimated usingthe maximum penalized likelihood method. Under the Gaussian noiseassumption such a choice coincides with the minimum of mean predictiveerror choice. Maximization of the MELL corresponds to minimization ofthe mean Kullback–Leibler information, that measures the deviation ofthe approximating (model) distribution from the true one. The resultingregularization parameter selection methods can handle possible functional anddistributional misspecifications when the usual assumptions of Gaussian noiseand/or linear relationship have been made but not met. We also suggest that inengineering applications it is beneficial to find ways of lowering the risk ofgetting grossly under-regularized solutions and that the new informationcomplexity regularization parameter selection method (RPSM) is one of thepossibilities. Several examples of applying the reviewed RPSMs are given.

Efficient determination of multiple regularization parameters in a generalizedL-curve framework

The selection of multiple regularization parameters is considered in a generalizedL-curve framework. Multiple-dimensional extensions of the L-curve for selectingmultiple regularization parameters are introduced, and a minimum distancefunction (MDF) is developed for approximating the regularization parameterscorresponding to the generalized corner of the L-hypersurface. For thesingle-parameter (i.e. L-curve) case, it is shown through a model that theregularization parameters minimizing the MDF essentially maximize the curvatureof the L-curve. Furthermore, for both the single-and multiple-parameter cases theMDF approach leads to a simple fixed-point iterative algorithm for computingregularization parameters. Examples indicate that the algorithm convergesrapidly thereby making the problem of computing parameters according tothe generalized corner of the L-hypersurface computationally tractable.

Least third-order cumulant method with adaptive regularization parameter selection for neural networks

This paper introduces an interesting property of the least third-order cumulant objective function. The property is that the solution is optimal when the gradients of Mean Squares error and third-order cumulant error are zero vectors. The optimal solutions are independent of the value of regularization parameter λ. Also, an adaptive regularization parameter selection method is derived to control the convergences of Mean Squares error and the cumulant error terms. The proposed selection method is able to tunnel through the sub-optimal solutions, of which the locations are controllable, via changing the value of the regularization parameter. Consequently, the least third-order cumulant method with the adaptive regularization parameter selection method is theoretically capable of estimating an optimal solution when it is applied to regression problems.

Stable Recovery of Analytic Functions using Basic Hypergeometric Series

Two stable sampling formulas for reconstructing analytic functions from exponentially spaced samples are considered. Criterion for selecting regularization parameter and error estimates are obtained.

Regularization parameter selection for Bayesian reconstruction of attenuation maps

Previously we developed algorithms to obtain transmission reconstructions from truncated projections and from emission data without transmission measurements. The optimal basis set derived from "knowledge set" was used to create an approximate attenuation map, and the expansion coefficients were estimated using optimization algorithms. Since a truncated expansion does not represent an image precisely, and the projections of the basis vectors are not orthogonal, the estimated coefficients may be unstable in the presence of systematic errors. In addition to this, a nonlinear problem of reconstruction of an attenuation map from emission data has a nonunique solution. A constraint based on distribution of the expansion coefficients is considered in this paper to regularize the estimation problems. The parameter selection methods based on different assumptions are applied to find the optimum regularization parameter. The selected regularization parameter obtained from a projection data set has been shown to provide satisfactory reconstruction results.

Adaptive regularization parameter selection method for enhancing generalization capability of neural networks

A novel adaptive regularization parameter selection (ARPS) method is proposed in this paper to enhance the performance of the regularization method. The proposed ARPS method enables a gradient descent type training to tunnel through some of the undesired sub-optimal solutions on the composite error surface by means of changing the value of the regularization parameter. Undesired sub-optimal solutions are introduced inherently from regularized objective functions. Hence, the proposed ARPS method is capable of enhancing the regularization method without getting stuck at these sub-optimal solutions.

Rational fraction model for regularized inverse dynamics of a flexible manipulator

The rational fraction model for the regularized inverse dynamics of a flexible manipulator is presented, the effect of this model on alleviating the ill-posed behavior of the calculated torque is demonstrated, and the implication of the regularization parameter selection on the trade-off between the tracking accuracy and the frequency bandwidth of the calculated torque is addressed as well. The validity of the proposed method is illustrated by means of a computer numeric simulation.

Constrained and regularized system identification

Prior knowledge can be introduced into system identification problems in terms of constraints on the parameter space, or regularizing penalty functions in a prediction error criterion. The contribution of this work is mainly an extension of the well known FPE (Final Production Error) statistic to the case when the system identification problem is constrained and contains a regularization penalty. The FPECR statistic (Final Production Error with Constraints and Regularization) is of potential interest as a criterion for selection of both regularization parameters and structural parameters such as order.