Optimal Regularization Parameter Research Articles

Robust Extreme Learning Machine for estimation of triangular, rectangular, and parabolic weirs

In this paper, for the first time, the DC of triangular, rectangular, and parabolic weirs is simulated by a new learning machine called “Robust Extreme Learning Machine” (RELM). The used laboratory data are divided into two categories: training (70% data) and testing (30% data). In the next step, the number of neurons inside the hidden layer is examined. For the structure of the proposed RELM algorithm, 10 hidden layer neurons are embedded. For the learning machine used in the present study, six different activation functions are evaluated that the Sigmoid activation function has better performance and is used for the RELM structure. Next, the calibration parameter of the RELM algorithm is discussed. The optimal regularization parameter is selected for the present study equal to 0.0001. Then, using parameters affecting the DC, four RELM models are developed. By performing various analyzes, the superior RELM model and the most effective input parameters are identified. Also, comparing the performance of the RELM model with ELM shows the superiority of the RELM algorithm. For the superior model, a relative derivative sensitivity analysis is performed to investigate the behaviour of the input parameters on the DC. Finally, an equation for estimating the DCs of triangular, rectangular, and parabolic weirs is proposed.

Cucumber powdery mildew detection method based on hyperspectra-terahertz.

To explore the use of information technology in detecting crop diseases, a method based on hyperspectra-terahertz for detecting cucumber powdery mildew is proposed. Specifically, a method of effective hyperspectrum establishment, a method of spectral preprocessing, a method of selecting the feature wavelength, and a method of establishing discriminant models are studied. Firstly, the effective spectral information under visible light and near infrared is preprocessed by Savitzky-Golay (SG) smoothing, discrete wavelet transform, and move sliding window, which determine the optimal preprocessing method to be wavelet transform. Then stepwise discriminant analysis is used to select the feature wavelengths in the visible and near-infrared bands, forming the feature space. According to the features, a linear discriminant model is established for the wave bands, and the average recognition rate of cucumber powdery mildew is 93% in the whole wave band. The preprocessing method of terahertz data, the screening method of terahertz effective spectrum, the selection method of feature wavelength and the establishment method of classification model are studied. Python 3.8 is used to preprocess the terahertz raw data and establish the terahertz effective spectral data set for subsequent processing. Through iterative variable subset optimization - iterative retaining informative variables (IVSO-IRIV), the terahertz effective spectrum is screened twice to form the terahertz feature space. After that, the optimal regularization parameter and regularization solution methods are selected, and a sparse representation classification model is established. The accuracy of cucumber powdery mildew identification under the terahertz scale is 87.78%. The extraction and analysis methods of terahertz and hyperspectral feature images are studied, and more details of lesion samples are restored. Hence, the use of hyperspectral and terahertz technology can realize the detection of cucumber powdery mildew, which provides a basis for research on the hyperspectral and terahertz technology in detection of crop diseases.

3D inversion of gravity data with unstructured mesh and least-squares QR-factorization (LSQR)

Once inverting potential-field geophysical data an appropriate discretization of the model is required to accurately construct complicated geometries of the causative sources. Rectangular prisms (structured meshes) have limitations to recover and preserve the edges of real geological sources. Here an isoparametric finite-element (FE) methodology is used to design an unstructured mesh for use in 3D inverse modeling of gravity data. The calculation of the sensitivity kernel of the forward operator uses Gauss-Legendre quadrature rather than the analytic formulation. For the sake of instability of inversion operator in gravity data and to solve the Tikhonov norms term, the least-squares QR-factorization (LSQR) technique is used. This method is accompanied with weighted generalized cross-validation (WGCV) for selecting the optimum regularization parameter value. The depth weighting function is also incorporated in the formulation of the objective function to suppress the impact of shallow features and recover sources at an appropriate depth. The proposed algorithm was applied to noise-corrupted synthetic data along with a real case study where a gravity survey was used for iron exploration in Yazd province, central Iran. The obtained results of synthetic example indicated that the proposed 3D inversion method recovers the geometry and density contrast values which are similar to the true structures and its application on a real example data set recovers geologically reasonable complex structures. So, this modified algorithm is a type of smooth inversion for boundary detection and whole understanding of the physical property distribution of the subsurface.

Application of distribution of relaxation times method in polymer electrolyte membrane water electrolyzer

Distribution function of relaxation times (DRT or DFRT) is a well-established method of solving electrochemical impedance spectroscopy (EIS) data from electrochemical component, such as fuel cells and battery. It can be used to develop the most accurate equivalent circuit models without the input of any circuit architecture information, due to the complexity in equivalent circuit model (ECM). For the first time, this paper implemented DRT methods in polymer electrolyte membrane water electrolyzers (PEMECs). The regularization parameter, characteristic parameters and frequency factors of the solving process are introduced. The results show that (1) DRT method represents the original impedance spectrum of hydrogen production in the form of three characteristic peaks, which correspond to the main polarization processes in hydrogen production system, including water/gas diffusion, hydrogen evolution reaction (HER), oxygen evolution reaction (OER) reactions, and proton transfer processes. (2) The optimal regularization parameter is 0.01 for different current and temperature. (3) The frequency of water/gas diffusion, HER /OER reactions and proton transfer processes is 0.1–60 Hz, 100–1000 Hz and 2000–4000 Hz, respectively. (4) DRT methods couples with genetic algorithm can effectively identify various types of polarization impedance corresponding to different characteristic frequencies without equivalent circuit model.

ADMM-Based Residual Whiteness Principle for Automatic Parameter Selection in Single Image Super-Resolution Problems

We propose an automatic parameter selection strategy for the single image super-resolution problem for images corrupted by blur and additive white Gaussian noise with unknown standard deviation. The proposed approach exploits the structure of both the down-sampling and the blur operators in the frequency domain and computes the optimal regularisation parameter as the one optimising a suitably defined residual whiteness measure. Computationally, the proposed strategy relies on the fast solution of generalised Tikhonov ell _2–ell _2 problems as proposed in Zhao et al. (IEEE Trans Image Process 25:3683–3697, 2016). These problems naturally appear as substeps of the Alternating Direction Method of Multipliers used to solve single image super-resolution problems with non-quadratic, non-smooth, sparsity-promoting regularisers both in convex and in non-convex regimes. After detailing the theoretical properties allowing to express the whiteness functional in a compact way, we report an exhaustive list of numerical experiments proving the effectiveness of the proposed approach for different type of problems, in comparison with well-known parameter selection strategies such as, e.g., the discrepancy principle.

Feature-guided regularization parameter selection in sparse de-noising for fault diagnosis

Using the sparse and redundant representation model to de-noise vibration signals is an effective method for mechanical fault diagnosis. The selection of the regularization parameter in the sparse de-noising algorithm plays a vital role in reconstructing fault features. The current choices, however, are primarily based on fixed-form parameters derived under the assumption of additive white noise or based on the “trial and error” method. As a result, the extracted de-noised signals often fail to show fault features. To solve this difficulty, a self-adaptive regularization parameter selection strategy is proposed, which directly measures the significance of fault features in sparse de-noised signals. Firstly, the entire solution path is obtained by the computationally efficient least angle regression (LARS) algorithm. Besides, a feature presence probability index (FP) is well designed to measure the significance of fault features in the reconstructed signal envelope demodulation spectrum. Accordingly, with the help of the maximal FP indicator, the optimal regularization parameter and LARS solution is determined. The proposed method is not limited by noise probability distributions; therefore, it shows the superiority of extracting fault features in both the simulation and experimental data analysis.

Investigation of a non-Newtonian MR fluid flow between parallel plates by developed CFD code for different numerical schemes

Magnetorheological (MR) fluids are composed of a solution of micron-sized magnetizable particles dispersed in a carrier fluid, typically oil. Thus, the MR fluids are classified as smart fluids by showing variable apparent viscosity under a precisely controllable external magnetic field. This feature of the MR fluids gave the possibility to use them in the MR dampers to control variational damping forces. In this work, an MR damper on relatively small scales is considered to investigate the MR fluid flow behavior through the piston annuli where the magnetic field exists. This work mainly focuses on the numerical solutions of the flow variables in a magnetically excited non-Newtonian flow medium by comparing analytical results for various Reynolds numbers. The Herschel–Bulkley (HB) viscous model is used for the non-Newtonian characteristic of the MR fluid. For the numerical modeling of the HB viscous model, the regularized approaches are used to avoid numerical errors. However, contrary to the actual HB model, the regularized models can be incapable to give true apparent viscous values at very low shear rates depending on the regularization parameters. Thus, the present study aims to give a better understanding of choosing the optimal regularization parameters for the studied flow conditions. The second part of the study discusses the numerical discretization schemes aiming to present their performances while changing the Reynolds number between 0.002 and 1. In this manner, a Computational Fluid Mechanics (CFD) solver has been developed for two-dimensional geometries which are meshed with structured grids by using six different discretization schemes including two of the most known total variation diminishing (TVD) schemes. For simplicity of the problem, a two-dimensional parallel plates geometry and a constant magnitude of the magnetic field intensity along the piston annuli are assumed. The study also summarizes the CFD technique by evaluating the physical meaning of the flow field variables.

Time-lapse adaptive inversion on resistivity monitoring data with Lp norm regularization and cross-time weight

Time-lapse electrical resistivity tomography (ERT) can describe the dynamic processes occurring underground and has been widely employed in the fields of hydrology, environmental management, and engineering. The time-lapse inversion method is based on the space- and time-dimension regularization technique as it considers complementary information about more than one time interval and reduces the potential causes of the geophysical data inversion of the artifact. However, the L2 norm inversions smear out the models and cannot distinguish between the resistivity interfaces. Moreover, the determination of the two optimal regularization parameters for time-lapse inversion remains challenging, and the fixed-time regularization parameter smooths out the low-sensitivity area and inhibits the real variation on the target body of the deep region. To overcome this problem, we propose a time-lapse ERT inversion method of adaptively imaging the time-lapse model and focusing on the resistivity variation on the time dimension. This method is based on the Lp norm regularization provided by the M-estimator and uses the adaptive regularization parameters based on the model roughness and data misfit. In addition, a cross-time matrix improving the sensitivity in the deep region of model space is added to the time regularization term. We present two synthetic examples and a field dataset to validate the method. Numerical simulation and field dataset inversion results revealed that the proposed time-lapse inversion strategy can avoid the excessive smoothness caused by the low sensitivity in the model deep region and can thus enhance the response from the true ground variation to improve the accuracy of the inversion results. Owing to the adaptive regularization parameters that can preserve the dynamic balance between the objective function components, the inversion iteration progress can remain stable and can maintain the trend toward convergence.

An Inversion Framework for Optimizing Non‐Methane VOC Emissions Using Remote Sensing and Airborne Observations in Northeast Asia During the KORUS‐AQ Field Campaign

We aim to reduce uncertainties in CH2O and other volatile organic carbon (VOC) emissions through assimilation of remote sensing data. We first update a three‐dimensional (3D) chemical transport model, GEOS‐Chem with the KORUSv5 anthropogenic emission inventory and inclusion of chemistry for aromatics and C2H4, leading to modest improvements in simulation of CH2O (normalized mean bias (NMB): −0.57 to −0.51) and O3 (NMB: −0.25 to −0.19) compared against DC‐8 aircraft measurements during KORUS‐AQ; the mixing ratio of most VOC species are still underestimated. We next constrain VOC emissions using CH2O observations from two satellites (OMI and OMPS) and the DC‐8 aircraft during KORUS‐AQ. To utilize data from multiple platforms in a consistent manner, we develop a two‐step Hybrid Iterative Finite Difference Mass Balance and four‐dimensional variational inversion system (Hybrid IFDMB‐4DVar). The total VOC emissions throughout the domain increase by 47%. The a posteriori simulation reduces the low biases of simulated CH2O (NMB: −0.51 to −0.15), O3 (NMB: −0.19 to −0.06), and VOCs. Alterations to the VOC speciation from the 4D‐Var inversion include increases of biogenic isoprene emissions in Korea and anthropogenic emissions in Eastern China. We find that the IFDMB method alone is adequate for reducing the low biases of VOCs in general; however, 4D‐Var provides additional refinement of high‐resolution emissions and their speciation. Defining reasonable emission errors and choosing optimal regularization parameters are crucial parts of the inversion system. Our new hybrid inversion framework can be applied for future air quality campaigns, maximizing the value of integrating measurements from current and upcoming geostationary satellite instruments.

Stochastic Convergence of Regularized Solutions and Their Finite Element Approximations to Inverse Source Problems

In this work, we investigate the regularized solutions and their finite element solutions to the inverse source problems governed by partial differential equations, and establish the stochastic convergence and optimal finite element convergence rates of these solutions, under pointwise measurement data with random noise. Unlike most existing regularization theories, the regularization error estimates are derived without any source conditions, while the error estimates of finite element solutions show their explicit dependence on the noise level, regularization parameter, mesh size, and time step size, which can guide practical choices among these key parameters in real applications. The error estimates also suggest an iterative algorithm for determining an optimal regularization parameter. Numerical experiments are presented to demonstrate the effectiveness of the analytical results.

Solving two-dimensional bioheat optimal control problem in solid and vessel domain by pseudospectral discretization

In this paper, an optimal control problem subject to an energy transfer equation in thermally significant blood vessel coupled with the Pennes governing equation is solved. The proposed dynamical system is modeled by a system of conduction-reaction and convection-conduction equations. Chebyshev–Gauss–Lobatto (CGL) and Legendre–Gauss–Lobatto (LGL) points are applied to discretize optimal control problem. Then, the problem transferred into a quadratic programming problem using domain decomposition and matching the solution and its first derivative across the related interfaces. The Tikhonov regularization method is used to compute the optimal regularization parameter for the objective function. Some theories have been discussed to prove that the discrete problem has a solution and it is also consistent with the continuous problem. Numerical results are compared and numerical convergent is shown. Optimal and non-optimal external heat source is compared and it is shown that the mathematical modeling is effective in saving the external heat energy. Numerical results show that by this process of heating at the final time, the temperature is very close to the desired temperature without damaging the vessel domain. Numerical results and corresponding graphs are depicted.

Estimation of the depth-domain seismic wavelet based on velocity substitution and a generalized seismic wavelet model

Depth-domain seismic wavelet estimation is the essential foundation for depth-imaged data inversion, which is increasingly used for hydrocarbon reservoir characterization in geophysical prospecting. The seismic wavelet in the depth domain stretches with increasing medium velocity and compresses with decreasing medium velocity. The commonly used convolutional model cannot be directly used to estimate depth-domain seismic wavelets due to velocity-dependent wavelet variations. We have developed a separate parameter estimation method for estimating depth-domain seismic wavelets from poststack depth-domain seismic and well-log data. Our method is based on the velocity substitution and depth-domain generalized seismic wavelet model defined by the fractional derivative and reference wavenumber. Velocity substitution allows wavelet estimation with the convolutional model in the constant-velocity depth domain. The depth-domain generalized seismic wavelet model allows for a simple workflow that estimates the depth-domain wavelet by estimating two wavelet model parameters. In addition, this simple workflow does not need to perform searches for the optimal regularization parameter and wavelet length, which are time-consuming in least-squares (LS)-based methods. The limited numerical search ranges of the two wavelet model parameters can easily be calculated using the constant phase and peak wavenumber of the depth-domain seismic data. Our method is verified using synthetic and real seismic data and further compared with LS-based methods. The results indicate that our method is effective and stable even for data with a low signal-to-noise ratio.

A note on a bilevel problem for parameter learning for inverse problems with the wave equation

In this paper we consider a bilevel problem for determining the optimal regularization parameter in an inverse problem with the linear wave equation transferring results from Holler, Kunisch, and Barnard (2018), where a general function space setting and applications to (bilinear) elliptic problems have been addressed. We analyze the well-posedness and derive the optimality conditions for the bilevel problem for the wave equation. Moreover, for given noisy data the numerical performance of the approach to find the regularization parameter is compared for different choices of priors in the Tikohonov regularization term of the lower level problem.

Tensor-Based Recursive Least-Squares Adaptive Algorithms with Low-Complexity and High Robustness Features

The recently proposed tensor-based recursive least-squares dichotomous coordinate descent algorithm, namely RLS-DCD-T, was designed for the identification of multilinear forms. In this context, a high-dimensional system identification problem can be efficiently addressed (gaining in terms of both performance and complexity), based on tensor decomposition and modeling. In this paper, following the framework of the RLS-DCD-T, we propose a regularized version of this algorithm, where the regularization terms are incorporated within the cost functions. Furthermore, the optimal regularization parameters are derived, aiming to attenuate the effects of the system noise. Simulation results support the performance features of the proposed algorithm, especially in terms of its robustness in noisy environments.

Analysis of terrestrial water storage changes in the Shaan-Gan-Ning Region using GPS and GRACE/GFO

Both the Global Positioning System (GPS) and Gravity Recovery and Climate Experiment (GRACE)/GRACE Follow-On (GFO) provide effective tools to infer surface mass changes. In this paper, we combined GPS, GRACE/GFO spherical harmonic (SH) solutions and GRACE/GFO mascon solutions to analyze the total surface mass changes and terrestrial water storage (TWS) changes in the Shaan-Gan-Ning Region (SGNR) over the period from December 2010 to February 2021. To improve the reliability of GPS inversion results, an improved regularization Laplace matrix and monthly optimal regularization parameter estimation strategy were employed to solve the ill-posed problem. The results show that the improved Laplace matrix can suppress the edge effects better than that of the traditional Laplace matrix, and the correlation coefficient and standard deviation (STD) between the original signal and inversion results from the traditional and improved Laplace matrix are 0.84 and 0.88, and 17.49 mm and 15.16 mm, respectively. The spatial distributions of annual amplitudes and time series changes for total surface mass changes derived from GPS agree well with GRACE/GFO SH solutions and mascon solutions, and the correlation coefficients of total surface mass change time series between GPS and GRACE/GFO SH solutions, GPS and GRACE/GFO mascon solutions are 0.80 and 0.77. However, the obvious differences still exist in local regions. In addition, the seasonal characteristics, increasing and decreasing rate of TWS change time series derived from GPS, GRACE/GFO SH and mascon solutions agree well with the Global Land Data Assimilation System (GLDAS) hydrological model in the studied area, and generally consistent with the precipitation data. Meanwhile, TWS changes derived from GPS and GRACE mascon solutions in the SGNR are more reliable than those of GRACE SH solutions over the period from January 2016 to June 2017 (the final operation phase of the GRACE mission).

On an unsupervised method for parameter selection for the elastic net

<abstract><p>Despite recent advances in regularization theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularization parameter from noisy data. In this work, we improve their results and extend the analysis to the elastic net regularization. Furthermore, we design a data-driven, automated algorithm for the computation of an approximate regularization parameter. Our analysis combines statistical learning theory with insights from regularization theory. We compare our approach with state-of-the-art parameter selection criteria and show that it has superior accuracy.</p></abstract>

An Efficient Cross-Gradient Joint Inversion Algorithm for Gravity and Magnetic Data Using a Sequential Strategy

The joint inversion methods of gravity and magnetic data have been presented in numerous literatures and implemented in many real applications. However, almost all of these existing methods still face the problem of low computational efficiency, although some targeted improvements have been put forwarded. To further address this issue, this study proposes an efficient algorithm for the joint inversion of gravity and magnetic data. The new algorithm is developed from the idea of structural coupling by using the cross-gradient function and employs a sequential strategy. A new formula for efficient minimization of the structural similarity term is derived innovatively, in which the minimization of the structural coupling term is obtained by a technical combination of an alternating strategy and the randomized singular value decomposition (RSVD) algorithm. Using the proposed new formula, the dimension of the original inverse problem can be reduced greatly. Furthermore, the RSVD algorithm is utilized to perform the independent inversion and determine an optimal regularization parameter. Several comparative tests on a synthetic example show that the proposed efficient method poses great advantage in efficiency compared with the conventional sequential cross-gradient joint inversion (CSCGJI) method. The proposed algorithm is also successfully applied to the real data from a metallic deposit area in Xinjiang province, China.

A Joint Inversion Algorithm for the Establishment of High-Accuracy 3-D Marine Gravity Field

The 3-D marine gravity field plays an important role in many practical applications, such as matching navigation, object monitoring, and resource exploitation. However, the accuracy of the gravity field generated by most existing methods is relatively low. To solve this problem, this study innovatively puts forward a new method for establishing a high-accuracy 3-D marine gravity field by using a joint inversion algorithm. The presented method incorporates gravity fields observed at different altitudes, especially those observed near the seafloor, because they contain more prominent short-wavelength signals of the sources. The final joint inversion formulas are developed from the standard form of independent inversion of single gravity datasets, so it is conducive to include different weighting matrices. To achieve the efficiency of the presented method, the nonlinear objective function is minimized iteratively using a fast randomized singular value decomposition (RSVD) technique. The employing of RSVD also facilitates the determination of an optimal regularization parameter based on the generalized cross-validation criterion. The presented method is tested on both synthetic gravity fields simulated by a theoretical density model and real gravity fields collected in the South China Sea. Numerical results demonstrate that the presented method can provide more accurate 3-D gravity fields than the commonly used iterative downward continuation method in the wavenumber domain (DCW) and the conventional equivalent source (CES) method.

Portfolio Optimization Using a Consistent Vector-Based MSE Estimation Approach

This paper is concerned with optimizing the weights of the global minimum-variance portfolio (GMVP) in high-dimensional settings where both observation and population dimensions grow at a bounded ratio. Optimizing the GMVP weights is highly influenced by the data covariance matrix estimation. In a high-dimensional setting, it is well known that the sample covariance matrix is not a proper estimator of the true covariance matrix since it is not invertible when we have fewer observations than the data dimension. Even with more observations, the sample covariance matrix may not be well-conditioned. This paper determines the GMVP weights based on a regularized covariance matrix estimator to overcome the abovementioned difficulties. Unlike other methods, the proper selection of the regularization parameter is achieved by minimizing the mean squared error of an estimate of the noise vector that accounts for the uncertainty in the data mean estimation. Using random-matrix-theory tools, we derive a consistent estimator of the achievable mean squared error that allows us to find the optimal regularization parameter using a simple line search. Simulation results demonstrate the effectiveness of the proposed method when the data dimension is larger than, or of the same order of, the number of data samples.

Spectral variable selection based on least absolute shrinkage and selection operator with ridge-adding homotopy

The least absolute shrinkage and selection operator (LASSO) is an established sparse representation approach for variable selection, and its performance relies on finding a good value for the regularization parameter, typically through cross-validation. However, cross-validation is a computationally intensive step and requires a properly determined search range and step size. In the present study, the ridge-adding homotopy (RAH) algorithm is applied with LASSO to overcome the aforementioned shortcomings. The homotopy algorithm can fit the entire solution of the LASSO problem by tracking the Karush-Kuhn Tucker (KKT) conditions and yields a finite number of potential regularization parameters. Considering the singularities, a M×1 random ridge vector will be added to the KKT conditions, which ensures that only one element is added to or removed from the active set. Finally, we can select the optimal regularization parameter by traversing the potential parameters with modelling and evaluation metrics. The selected variables are the nonzero elements in the sparse regression coefficient vector derived by the optimal regularization parameter. The proposed method has been demonstrated on three near-infrared (NIR) datasets with regard to wavelength selection and calibration. The results suggested that the “RAH-LASSO ​+ ​PLS” outperforms “LASSO ​+ ​PLS” and “full-wavelength PLS” in most cases. Importantly, the RAH method provides a systematic, as opposed to trial-and-error, procedure to determine the regularization parameter in LASSO.