Conjugate Gradient Method For Optimization Research Articles

Conjugate gradient‐based active noise cancellation

AbstractActive noise cancellation schemes should synthesize proper secondary perturbations without relevant prior information. Their performance depends strongly on their adaptive mechanisms, which, sometimes, do not fulfill the severe requirements of practical setups. This research demonstrates for the first time that the adaptive conjugate gradient optimization method can be used for active noise cancellation tasks. Additionally, a stochastic model is devised for estimating its steady‐state solution, even when the secondary path is not correctly estimated. Such a theoretical analysis demonstrates the asymptotic unbiasedness of the proposed scheme when the secondary plant is perfectly estimated. Simulations reveal that the devised method can attain better asymptotic performance and a higher convergence rate than the traditional FX‐LMS algorithm, at the expense of an increased computational burden. Furthermore, simulations show that the method outperforms the standard ones for nonstationary inputs (such as speech signals).

A Quadratic Model based Conjugate Gradient Optimization Method

In this paper, we introduce a nonlinear scaled conjugate gradient method, operating on the premise of a descent and conjugacy relationship. The proposed algorithm employs a conjugacy parameter that is determined to ensure that the method generates conjugate directions. It also utilizes a parameter that scales the gradient to enhance the convergence behavior of the method. The derived method not only exhibits the crucial feature of global convergence but also maintains the generation of descent directions. The efficiency of the method is established through numerical tests conducted on a variety of high-dimensional nonlinear test functions. The obtained results attest to the improved behavior of the derived algorithm and support the presented theory.

An inversion scheme for sizing crack from signals of the motion-induced eddy current testing method based on a new formula of signal gradient of ferromagnetic materials

In this paper, an inversion scheme for the profile reconstruction of a fatigue crack in rails from the motion-induced eddy current testing (MIECT) signals is proposed based on the conjugate gradient (CG) optimization method and the fast MIECT forward simulator developed by authors. To calculate the gradient of MIECT signals with respect to the crack parameters, a simplified analytical gradient formula is deduced, which enables a high efficiency crack reconstruction from MIECT signals for a crack in ferromagnetic material. The validity of the inversion schemes is demonstrated at first by reconstructing cracks with numerical simulated signals for ferromagnetic materials with an oblique elliptic crack model. The good reconstruction results reveal that the new signal grandient formula is efficient for reconstructing crack in ferromagnetic material with the CG algorithm. Finally, sizing of artificial cracks are carried out with use of measured MIECT signals due to cracks in plates of both non-magnetic and ferromagnetic materials. The reconstructed crack parameters are in good agreement with the true values, which experimentally proved the validity and efficiency of the proposed inversion scheme and the new signal gradient formula for MIECT inversion.

Multiparameter 2-D viscoelastic full-waveform inversion of Rayleigh waves: a field experiment at Krauthausen test site

SUMMARYFull-waveform inversion (FWI) has been proven to be an effective tool for high-resolution multiparameter imaging of the shallow subsurface. It has been shown that the Gauss–Newton (GN) optimization method uses the off-diagonal information contained in the Hessian matrix and can increase resolution and mitigate crosstalk in multiparameter viscoelastic FWI. In this work, we demonstrate the advantages of GN viscoelastic FWI over the conventional FWI with a conjugate gradient optimization method by using synthetic examples. We also investigate the potential of shallow seismic-wave 2-D viscoelastic FWI as a method for high-resolution hydrogeological characterization. The GN viscoelastic FWI is applied to two orthogonal profiles acquired at the Krauthausen natural laboratory (Germany). The groundwater table is located at around 2 m, which nicely agrees with an abrupt increase of P-wave velocity in the inverted results. FWI also reveals a low S-wave velocity layer at the depth of 4–6 m with high Poisson’s ratio values close to 0.5, which corresponds to a saturated sand layer known from previous studies. A K-mean cluster analysis is used to further analyse the multiparameter FWI results. By considering the derived Poisson’s ratio, P- and S-wave velocities, we convert the complex relationship between the multivariate data into a lithological meaningful zonation of the shallow subsurface. By comparing the lithological units in the alluvial aquifer with the cone penetration tests clusters, we conclude that the divided facies describe valuable characterization information about the heterogeneity and connectivity of the aquifer. This experiment indicates that the multiparameter models derived by viscoelastic FWI contain useful information for high-resolution aquifer characterization, and the potential of multiparameter FWI combined with cluster analysis in shallow subsurface characterization is encouraging.

Prediction of Suitable Crops Using Stacked Scaling Conjugant Neural Classifier

Agriculture plays a vital role in economic development. The major problem faced by the farmers are the selection of suitable crops based on environmental conditions such as weather, soil nutrients, etc. The farmers were following ancestral patterns, which could sometimes lead to the wrong selection of crops. In this research work, the feature selection method is adopted to improve the performance of the classification. The most relevant features from the dataset are obtained using a Probabilistic Feature Selection (PFS) approach, and classification is done using a Neural Fuzzy Classifier (NFC). Scaling Conjugate Gradient (SCG) optimization method is used to update the weights. The data set used for analysis contain various parameters such as soil characteristics, geographical location, and environmental factors such as temperature and rainfall. The proposed method recommends suitable crops for cultivation based on site-specific parameters. Experimental result shows that the proposed method provides high accuracy and efficiency as compared to existing methodologies.

Gaussian process latent variable model factorization for context-aware recommender systems

• We propose a novel algorithm named GPLVMF to achieve a Bayesian factorization for Context-aware recommendation. • We model the non-zero mean in Gaussian processes to capture the bias and provide a generalized variational inference. • Our method can flexibly deal with both real-valued and categorical contexts. • Both scaled conjugate gradient and stochastic gradient descent optimization methods are applied to the model. • Experiment results show that GPLVMF not only improves accuracy but can automatically infer the influence of context. Context-aware recommender systems (CARS) have gained increasing attention due to their ability to utilize contextual information. Compared to traditional recommender systems, CARS are, in general, able to generate more accurate recommendations. The latent factors approach accounts for a large proportion of CARS. Recently, a non-linear Gaussian Process (GP) based factorization method was proven to outperform the state-of-the-art methods in CARS. Despite its effectiveness, standard GP model-based methods can suffer from over-fitting and may not be able to determine the impact of each context automatically. In order to address such shortcomings, we propose a Gaussian Process Latent Variable Model Factorization (GPLVMF) method, where we apply an appropriate prior to the original GP model. Our work is primarily inspired by the Gaussian Process Latent Variable Model (GPLVM), which is a non-linear dimensionality reduction method. Since the traditional maximum likelihood approach for standard GP cannot be applied to our model, we adopt a variational inference method to solve the GPLVMF model. As a result, we improve the results on the real datasets significantly as well as capturing the importance of each context. In addition to the general advantages, our method flexibly provides two contributions regarding recommender system settings: (1) addressing the influence of bias by setting a non-zero mean function, and (2) utilizing real-valued contexts by fixing the latent space with real values.

Taking the 4D Nature of fMRI Data Into Account Promises Significant Gains in Data Completion.

Functional magnetic resonance imaging (fMRI) is a powerful, noninvasive tool that has significantly contributed to the understanding of the human brain. FMRI data provide a sequence of whole-brain volumes over time and hence are inherently four dimensional (4D). Missing data in fMRI experiments arise from image acquisition limits, susceptibility and motion artifacts or during confounding noise removal. Hence, significant brain regions may be excluded from the data, which can seriously undermine the quality of subsequent analyses due to the significant number of missing voxels. We take advantage of the four dimensional (4D) nature of fMRI data through a tensor representation and introduce an effective algorithm to estimate missing samples in fMRI data. The proposed Riemannian nonlinear spectral conjugate gradient (RSCG) optimization method uses tensor train (TT) decomposition, which enables compact representations and provides efficient linear algebra operations. Exploiting the Riemannian structure boosts algorithm performance significantly, as evidenced by the comparison of RSCG-TT with state-of-the-art stochastic gradient methods, which are developed in the Euclidean space. We thus provide an effective method for estimating missing brain voxels and, more importantly, clearly show that taking the full 4D structure of fMRI data into account provides important gains when compared with three-dimensional (3D) and the most commonly used two-dimensional (2D) representations of fMRI data.

The Cauchy Conjugate Gradient Algorithm with Random Fourier Features

Random Fourier mapping (RFM) in kernel adaptive filters (KAFs) provides an efficient method to curb the linear growth of the dictionary by projecting the original input data into a finite-dimensional space. The commonly used measure in RFM-based KAFs is the minimum mean square error (MMSE), which causes performance deterioration in the presence of non-Gaussian noises. To address this issue, the minimum Cauchy loss (MCL) criterion has been successfully applied for combating non-Gaussian noises in KAFs. However, these KAFs using the well-known stochastic gradient descent (SGD) optimization method may suffer from slow convergence rate and low filtering accuracy. To this end, we propose a novel robust random Fourier features Cauchy conjugate gradient (RFFCCG) algorithm using the conjugate gradient (CG) optimization method in this paper. The proposed RFFCCG algorithm with low complexity can achieve better filtering performance than the KAFs with sparsification, such as the kernel recursive maximum correntropy algorithm with novelty criterion (KRMC-NC), in stationary and non-stationary environments. Monte Carlo simulations conducted in the time-series prediction and nonlinear system identification confirm the superiorities of the proposed algorithm.

Multi-Target Strike Path Planning Based on Improved Decomposition Evolutionary Algorithm

This study proposes a path-finding model for multi-target strike planning. The model evaluates three elements, i.e., the target value, the aircraft’s threat tolerance, and the battlefield threat, and optimizes the striking path by constraining the balance between mission execution and the combat survival. In order to improve the speed of the Multi-Objective Evolutionary Algorithm Based on Decomposition (MOEA/D), we use the conjugate gradient method for optimization. A Gaussian perturbation is added to the search points to make their distribution closer to the population distribution. The simulation shows that the proposed method effectively chooses its target according to the target value and the aircraft’s acceptable threat value, completes the strike on high value targets, evades threats, and verifies the feasibility and effectiveness of the multi-objective optimization model.

A Method for Diagnosis of Poloidal Field Coils’ Current Based on Inversion of Magnetic Field

Tokamak devices such as ITER, EAST, and HL-2M have been designed and constructed (or under construction) to challenge the magnetic-confined nuclear fusion technology for the sustainable energy problem of human beings. The magnet system of the tokamak devices is mainly composed with poloidal field (PF) coils, toroidal field (TF) coils, and central solenoid coils. Among them, the PF coils play a key role in adjusting the confining magnetic field for the control of the fusion plasma. In order to avoid the plasma disruption, it is necessary to know the correlation of the magnetic field in a vacuum vessel (VV) with the current in PF coils and to predict the current distribution of PF coils which can generate a required configuration of confining magnetic field. In this paper, taking the HL-2M tokamak device as an example, an inverse analysis scheme to predict and identify the perturbation of current in PF coils from the target magnetic field signals in the VV was proposed. First, based on the Biot–Savart law, a forward code to calculate the magnetic field due to PF coil currents and TF coil currents was developed. Second, an inversion code to predict and identify current distribution in PF coils from the target magnetic field was developed based on the conjugate gradient optimization method and the singular value decomposition method. The validity of the proposed inversion scheme and the corresponding numerical codes was investigated by reconstructing PF coil current distributions from several groups of perturbed magnetic fields. In addition, the effect of the measurement points and the measurement noises on the reconstruction results was also discussed. Finally, a simplified experiment on a coil system was conducted to verify the validity and efficiency of the proposed forward and inversion analysis code.

Direct Integration Method for Time-Delayed Control of Second-Order Dynamic Systems

A direct integration method (DIM) for time-delayed control (TDC) is proposed in this research. For a second-order dynamic system with time-delayed controllers, a Volterra integral equation of the second kind is used instead of a state derivative equation. With the proposed DIM where matrix exponentials are avoided, semi-analytical representation of the Floquet transition matrix for stability analysis can be derived, the stability region on the parametric space comprising control variables can also be plotted. Within this stability region, optimal control variables are subsequently obtained using a multilevel conjugate gradient optimization method. Further simulation examples demonstrated the superiority of the proposed DIM in terms of computational efficiency and accuracy, as well as the effectiveness of the optimization-based controller design approach.

A Riemannian conjugate gradient method for optimization on the Stiefel manifold

In this paper we propose a new Riemannian conjugate gradient method for optimization on the Stiefel manifold. We introduce two novel vector transports associated with the retraction constructed by the Cayley transform. Both of them satisfy the Ring-Wirth nonexpansive condition, which is fundamental for convergence analysis of Riemannian conjugate gradient methods, and one of them is also isometric. It is known that the Ring-Wirth nonexpansive condition does not hold for traditional vector transports as the differentiated retractions of QR and polar decompositions. Practical formulae of the new vector transports for low-rank matrices are obtained. Dai's nonmonotone conjugate gradient method is generalized to the Riemannian case and global convergence of the new algorithm is established under standard assumptions. Numerical results on a variety of low-rank test problems demonstrate the effectiveness of the new method.

A nonlinear mathematical modeling of daily pan evaporation based on conjugate gradient method

The mathematical form to estimate the evaporation plays an important role in sustainable development and optimal water resources management of the countries facing water crisis. In this paper, a general regression analysis is proposed to estimate the daily pan evaporation with respect to input variables including the air temperature (T), relative humidity (R), wind speed (W) and sunshine hours (H) from two weather stations (i.e. Chahnimeh Zabol reservoirs and Iranshar station) in Iran. A conjugate gradient optimization method is used to calibrate the nonlinear mathematical function for accurate prediction of daily pan evaporation. Three mathematical functions with nonlinear forms are calibrated on data points of daily pan evaporation using proposed general regression analysis by proposed conjugate line search method. Then, the accuracies of three nonlinear models are compared with adaptive neuro-fuzzy inference system (ANFIS) and M5 Model Tree (M5Tree) models for two stations. Results indicated that that the conjugate gradient method was successfully applied to estimate the nonlinear models for predictions of daily pan evaporation with explicit mathematical function. The proposed mathematical models (RMSE of 4.14mm for Chahnimeh and 1.94mm for Iranshar) performed better than the ANFIS predictions for Iranshar (RMSE of 2.41mm) and M5tree (RMSE of 4.31mm for Chahnimeh and 2.19mm for Iranshar).

A hybrid inverse method for small scale parameter estimation of FG nanobeams

As a first attempt, an inverse hybrid numerical method for small scale parameter estimation of functionally graded (FG) nanobeams using measured frequencies is presented. The governing equations are obtained with the Eringen's nonlocal elasticity assumptions and the first-order shear deformation theory (FSDT). The equations are discretized by using the differential quadrature method (DQM). The discretized equations are transferred from temporal domain to frequency domain and frequencies of the nanobeam are obtained. By applying random error to these frequencies, measured frequencies are generated. The measured frequencies are considered as input data and inversely, the small scale parameter of the beam is obtained by minimizing a defined functional. The functional is defined as root mean square error between the measured frequencies and calculated frequencies by the DQM. Then, the conjugate gradient (CG) optimization method is employed to minimize the functional and the small scale parameter is obtained. Efficiency, convergence and accuracy of the presented hybrid method for small scale parameter estimation of the beams for different applied random error, boundary conditions, length-to-thickness ratio and volume fraction coefficients are demonstrated.

A Parallel Nonrigid Registration Algorithm Based on B-Spline for Medical Images

The nonrigid registration algorithm based on B-spline Free-Form Deformation (FFD) plays a key role and is widely applied in medical image processing due to the good flexibility and robustness. However, it requires a tremendous amount of computing time to obtain more accurate registration results especially for a large amount of medical image data. To address the issue, a parallel nonrigid registration algorithm based on B-spline is proposed in this paper. First, the Logarithm Squared Difference (LSD) is considered as the similarity metric in the B-spline registration algorithm to improve registration precision. After that, we create a parallel computing strategy and lookup tables (LUTs) to reduce the complexity of the B-spline registration algorithm. As a result, the computing time of three time-consuming steps including B-splines interpolation, LSD computation, and the analytic gradient computation of LSD, is efficiently reduced, for the B-spline registration algorithm employs the Nonlinear Conjugate Gradient (NCG) optimization method. Experimental results of registration quality and execution efficiency on the large amount of medical images show that our algorithm achieves a better registration accuracy in terms of the differences between the best deformation fields and ground truth and a speedup of 17 times over the single-threaded CPU implementation due to the powerful parallel computing ability of Graphics Processing Unit (GPU).

An adaptive multigrid conjugate gradient method for permeability identification of nonlinear diffusion equation

Abstract In this paper, the adaptive multigrid conjugate gradient method is applied to the inversion of the space-dependent permeability function in a nonlinear diffusion equation, which plays an important role in promoting the permeability estimation within multiphase porous media flow. The forward problem is discretized using finite-difference methods and the estimation is formulated as a least-square minimization problem with a regularization term. Firstly, the conjugate gradient method for optimization is used to solve this parameter identification problem. Secondly, to reduce the amount of calculation, the idea of multigrid inversion and the necessary condition of “the optimal solution should be the fixed point of multigrid inversion” is considered. Then, by combining the multigrid multiscale method with the conjugate gradient method, a new inversion method called the adaptive multigrid conjugate gradient method is designed. The results of numerical simulations indicate that the proposed method can effectively reduce the required computation burden and have good ability in anti-noise.

Optimal control of energy extraction in wind-farm boundary layers

In very large wind farms, the vertical interaction with the atmospheric boundary layer plays an important role, i.e. the total energy extraction is governed by the vertical transport of kinetic energy from higher regions in the boundary layer towards the turbine level. In the current study, we investigate optimal control of wind-farm boundary layers, considering the individual wind turbines as flow actuators, whose energy extraction can be dynamically regulated in time so as to optimally influence the flow field and the vertical energy transport. To this end, we use large-eddy simulations of a fully developed pressure-driven wind-farm boundary layer in a receding-horizon optimal control framework. For the optimization of the wind-turbine controls, a conjugate-gradient optimization method is used in combination with adjoint large-eddy simulations for the determination of the gradients of the cost functional. In a first control study, wind-farm energy extraction is optimized in an aligned wind farm. Results are accumulated over one hour of operation. We find that the energy extraction is increased by 16 % compared to the uncontrolled reference. This is directly related to an increase of the vertical fluxes of energy towards the wind turbines, and vertical shear stresses increase considerably. A further analysis, decomposing the total stresses into dispersive and Reynolds stresses, shows that the dispersive stresses increase drastically, and that the Reynolds stresses decrease on average, but increase in the wake region, leading to better wake recovery. We further observe also that turbulent dissipation levels in the boundary layer increase, and overall the outer layer of the boundary layer enters into a transient decelerating regime, while the inner layer and the turbine region attain a new statistically steady equilibrium within approximately one wind-farm through-flow time. Two additional optimal control cases study penalization of turbulent dissipation. For the current wind-farm geometry, it is found that the ratio between wind-farm energy extraction and turbulent boundary-layer dissipation remains roughly around 70 %, but can be slightly increased by a few per cent by penalizing the dissipation in the optimization objective. For a pressure-driven boundary layer in equilibrium, we estimate that such a shift can lead to an increase in wind-farm energy extraction of 6 %.

Optimizing material properties of composite plates for sound transmission problem

To calculate the specific transmission loss (TL) of a composite plate, the conjugate gradient optimization method is utilized to estimate and optimize material properties of the composite plate in this study. For an n-layer composite plate, a nonlinear dynamic stiffness matrix based on the thick plate theory is formulated. To avoid huge computational efforts due to the combination of different composite material plates, a transfer matrix approach is proposed to restrict the dynamic stiffness matrix of the composite plate to a 4×4 matrix. Moreover, the transfer matrix approach has also been used to simplify the complexity of the objective function gradient for the optimization method. Numerical simulations are performed to validate the present algorithm by comparing the TL of the optimal composite plate with that of the original plate. Small number of iterations required during convergence tests illustrates the efficiency of the optimization method. The results indicate that an excellent estimation for the composite plate can be obtained for the desired sound transmission.

An Efficient Numerical Scheme for Sizing of Cavity Defect in Metallic Foam From Signals of DC Potential Drop Method

Quantitative nondestructive testing is important to guarantee the integrity of metallic foam (MF) structures. To predict the profile of a cavity defect in an MF material, a database-type fast forward scheme is upgraded at first by introducing a kind of multimedium element (MME) for the efficient simulation of dc potential drop (DCPD) signals of MF with defect of complicated shape. Second, a code of the hybrid strategy combining the neural network and the conjugate gradient optimization method is proposed to obtain the size and the position parameters of the defect. Both simulated and measured DCPD signals are adopted to reconstruct the bubble defects in MF. The good consistency of the true and the reconstructed results demonstrated the validity of the new scheme. In addition, it is also proved that the updated database-type fast-forward scheme is efficient for the signal simulation of MF with defect of complicated shape with the help of MME, and the hybrid inverse strategy has a better numerical performance for the defect sizing.

Wave-equation traveltime inversion: Comparison of three numerical optimization methods

Compared to traditional traveltime inversion methods, no traveltime picking, high-frequency assumption or ray tracing is necessary in wave-equation traveltime inversion (WT). Another merit of WT is that it is insensitive to the starting model. Although WT offers less detailed parts of velocity model than the full waveform inversion (FWI), it can provide a good initial velocity model for FWI. In this paper, the steepest descent, conjugate gradient and limited-memory BFGS (L-BFGS) optimization methods are used in the implementation of acoustic WT. We use synthetic crosswell data for testing and the numerical results show that L-BFGS has a faster rate of convergence and offers a reconstructed velocity model with better resolution compared to the other two gradient methods.