Gradient Method Research Articles

Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

In a Hilbertian framework, for the minimization of a general convex differentiable function f, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t, is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document}. In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document}. By adjusting the speed of convergence of ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document} towards zero, we will obtain both rapid convergence towards the infimal value of f, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f, we study the proximal algorithms in detail, and show that they benefit from similar properties.

켤레 계산 기울기 방법을 이용한 기계학습

켤레 계산 기울기 방법을 이용한 기계학습

Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

Delayed Weighted Gradient Method with simultaneous step-sizes for strongly convex optimization

Hybrid Nanosystem Formed by DOX-Loaded Liposomes and Extracellular Vesicles from MDA-MB-231 Is Effective against Breast Cancer Cells with Different Molecular Profiles.

Drug delivery selectivity is a challenge for cancer treatment. A hybrid pegylated pH-sensitive liposome-extracellular vesicle isolated from human breast cancer cell MDA-MB-231 was developed to investigate its in vitro activity against breast cancer cells of different molecular profiles to overcome this inconvenience. The hybrid nanosystem was produced by film hydration, and doxorubicin (DOX) was encapsulated in this system using the ammonium sulfate gradient method. The characterization of this hybrid nanosystem revealed a mean diameter of 140.20 ± 2.70 nm, a polydispersity index of 0.102 ± 0.033, an encapsulation efficiency of doxorubicin of 88.9% ± 2.4, and a great storage stability for 90 days at 4 °C. The fusion of extracellular vesicles with liposomes was confirmed by nanoflow cytometry using PE-conjugated human anti-CD63. This hybrid nanosystem demonstrated cytotoxicity against human breast cancer cell lines with different molecular subtypes, enhanced anti-migration properties, and exhibited similar cellular uptake to the free DOX treatment. Preliminary acute toxicity assessments using Balb/C female mice indicated a median lethal dose of 15-17.5 mg/kg, with no evidence of splenic, liver, heart, bone marrow, and renal damage at a dose of 15 mg/kg. These findings suggest the hybrid formulation as a versatile nanocarrier for the treatment of various breast cancer subtypes.

Stress concentration factor estimation for corroded steel plates using the surface gradient method

Pitting corrosion occurs in steel components as a stress-accumulating surface irregularity that is vulnerable to fatigue crack nucleation. In this study, stress distribution in sinusoidal surfaces was investigated to devise an estimation method for the stress concentration factor (SCF) fluctuations of corroded steel. An SCF equation for sinusoidal surfaces was developed from empirical data based on three parameters: wavelength, thickness, and position. For corroded surfaces, successive numerical differentiations can be performed on each point to determine the sinusoidal surface having a point sharing the same gradient information. Notably, the SCFs of corresponding points are comparable; SCF estimation is made feasible for irregular surfaces through sinusoidal surfaces. In two-dimensional sections, maximum error was below 5% at pits and 15% for the surface. In three-dimensional models, maximum error was below 20% at pits for wide-ranging corroded specimens. Therefore, the proposed method effectively estimates SCF for identifying areas at risk of fatigue cracking.

Modified Memoryless Spectral-Scaling Broyden Family on Riemannian Manifolds

This paper presents modified memoryless quasi-Newton methods based on the spectral-scaling Broyden family on Riemannian manifolds. The method involves adding one parameter to the search direction of the memoryless self-scaling Broyden family on the manifold. Moreover, it uses a general map instead of vector transport. This idea has already been proposed within a general framework of Riemannian conjugate gradient methods where one can use vector transport, scaled vector transport, or an inverse retraction. We show that the search direction satisfies the sufficient descent condition under some assumptions on the parameters. In addition, we show global convergence of the proposed method under the Wolfe conditions. We numerically compare it with existing methods, including Riemannian conjugate gradient methods and the memoryless spectral-scaling Broyden family. The numerical results indicate that the proposed method with the BFGS formula is suitable for solving an off-diagonal cost function minimization problem on an oblique manifold.

A three-term conjugate gradient descent method with some applications

The stationary point of optimization problems can be obtained via conjugate gradient (CG) methods without the second derivative. Many researchers have used this method to solve applications in various fields, such as neural networks and image restoration. In this study, we construct a three-term CG method that fulfills convergence analysis and a descent property. Next, in the second term, we employ a Hestenses-Stiefel CG formula with some restrictions to be positive. The third term includes a negative gradient used as a search direction multiplied by an accelerating expression. We also provide some numerical results collected using a strong Wolfe line search with different sigma values over 166 optimization functions from the CUTEr library. The result shows the proposed approach is far more efficient than alternative prevalent CG methods regarding central processing unit (CPU) time, number of iterations, number of function evaluations, and gradient evaluations. Moreover, we present some applications for the proposed three-term search direction in image restoration, and we compare the results with well-known CG methods with respect to the number of iterations, CPU time, as well as root-mean-square error (RMSE). Finally, we present three applications in regression analysis, image restoration, and electrical engineering.

A New Preconditioned Nonlinear Conjugate Gradient Method in Real Arithmetic for Computing the Ground States of Rotational Bose–Einstein Condensate

A New Preconditioned Nonlinear Conjugate Gradient Method in Real Arithmetic for Computing the Ground States of Rotational Bose–Einstein Condensate

Optimization of Welded Joints under Fatigue Loadings

In this paper, the notch effect in weldments has been investigated, and the optimal configuration of different types of welded joints has been analysed using the implicit gradient approach. By considering this implicit gradient method, it is possible to calculate the effective stress related to fatigue damage, with the effective stress being a continuous scalar function of the real stress tensor components, even in the presence of sharp edges. Hence, the search for the optimal configuration that maximises fatigue life can be tackled as the condition of minimum effective stress obtained by changing the weld shape and geometrical parameters. Both load-carrying cruciform joints and spot welds made of steel have been considered. The structural details have been studied by modelling actual shapes without any geometric simplification. Moreover, the same numerical procedure has been considered independently of the size, shape or load condition without imposing restrictive rules on the FE mesh.

A stabilized SAV difference scheme and its accelerated solver for spatial fractional Cahn–Hilliard equations

A novel energy-stable scheme is proposed to solve the spatial fractional Cahn–Hilliard equations, using the idea of scalar auxiliary variable (SAV) approach and stabilization technique. Thanks to the stabilization technique, it is shown that larger temporal stepsizes can be applied in numerical simulations. Moreover, the proposed SAV finite difference scheme is non-coupled and linearly implicit, which can be efficiently solved by the preconditioned conjugate gradient (PCG) method with a sine transform based preconditioner. Optimal error estimates of the fully-discrete scheme are obtained rigorously. Numerical examples are given to confirm the theoretical results and show the higher efficiency of the proposed scheme than the previous SAV schemes.

New vector transport operators extending a Riemannian CG algorithm to generalized Stiefel manifold with low-rank applications

This paper proposes two innovative vector transport operators, leveraging the Cayley transform, for the generalized Stiefel manifold embedded with a non-standard metric. Specifically, it introduces the differentiated retraction and an approximation of the Cayley transform to the differentiated matrix exponential. These vector transports are demonstrated to satisfy the Ring–Wirth non-expansive condition under non-standard metrics, and one of them is also isometric. Building upon the novel vector transport operators, we extend the modified Polak–Ribière–Polyak (PRP) conjugate gradient method to the generalized Stiefel manifold. Under a non-monotone line search condition, we prove our algorithm globally converges to a stationary point. The efficiency of the proposed vector transport operators is empirically validated through numerical experiments involving generalized eigenvalue problems and canonical correlation analysis.

Adaptive accelerated proximal gradient algorithm for auto-regressive exogenous models with outliers

This study introduces an enhanced recursive least-squares algorithm that applies the adaptive accelerated proximal gradient method to identify Auto-Regressive Exogenous models with output outliers. First, the outlier problem was converted into a robust principal component analysis problem. The adaptive accelerated proximal gradient method is then introduced to recover the information matrix, and the recursive least squares algorithm is applied to estimate the model parameters. Furthermore, we demonstrated that the parameter estimation is unbiased and that the parameter estimation error converges to zero under persistent excitation. A series of bench tests consistently highlighted the practicality and effectiveness of the proposed algorithm.

Characterization of Complex Proteoform Mixtures by Online Nanoflow Ion-Exchange Chromatography-Native Mass Spectrometry.

The characterization of proteins and complexes in biological systems is essential to establish their critical properties and to understand their unique functions in a plethora of bioprocesses. However, it is highly difficult to analyze low levels of intact proteins in their native states (especially those exceeding 30 kDa) with liquid chromatography (LC)-mass spectrometry (MS). Herein, we describe for the first time the use of nanoflow ion-exchange chromatography directly coupled with native MS to resolve mixtures of intact proteins. Reference proteins and protein complexes with molecular weights between 10 and 150 kDa and a model cell lysate were separated using a salt-mediated pH gradient method with volatile additives. The method allowed for low detection limits (0.22 pmol of monoclonal antibodies), while proteins presented nondenatured MS (low number of charges and limited charge state distributions), and the oligomeric state of the complexes analyzed was mostly kept. Excellent chromatographic separations including the resolution of different proteoforms of large proteins (>140 kDa) and a peak capacity of 82 in a 30 min gradient were obtained. The proposed setup and workflows show great potential for analyzing diverse proteoforms in native top-down proteomics, opening unprecedented opportunities for clinical studies and other sample-limited applications.

Holographic phase retrieval via Wirtinger flow: Cartesian form with auxiliary amplitude

We propose a new gradient method for holography, where a phase-only hologram is parameterized by not only the phase but also amplitude. The key idea of our approach is the formulation of a phase-only hologram using an auxiliary amplitude. We optimize the parameters using the so-called Wirtinger flow algorithm in the Cartesian domain, which is a gradient method defined on the basis of the Wirtinger calculus. At the early stage of optimization, each element of the hologram exists inside a complex circle, and it can take a large gradient while diverging from the origin. This characteristic contributes to accelerating the gradient descent. Meanwhile, at the final stage of optimization, each element evolves along a complex circle, similar to previous state-of-the-art gradient methods. The experimental results demonstrate that our method outperforms previous methods, primarily due to the optimization of the amplitude.

Color-to-Gray Conversion Method Based on Chroma Difference

Most of the images encountered in daily life are color images, yet grayscale remains prevalent in many fields due to its reduced data size and simplified operations. When reducing the dimensions of a three-channel color image to a single-channel grayscale, a portion of the original color information is inevitably lost. To yield high-quality grayscale images, this study introduces a color-to-gray conversion method based on chroma difference. This method defines a novel color distance metric for color-to-grayscale conversion, incorporating pixel chromaticity differences alongside brightness variations. The discrepancy in gray pixel values in the output grayscale image effectively mirrors the overall differences among input color image pixels. Optimization is conducted using the conjugate gradient method, ensuring appropriate reflection of luminance information from the input image and chromaticity data from the original color image within the grayscale rendering. Experimental validation confirms the efficacy of this approach. However, as the method accounts for all pixel pairs, it occasionally considers unnecessary pairs, leading to potential distortion in color differences between pixels and consequent inadequacies in chromaticity variation. To address this issue, a weighting factor is introduced, prioritizing color combinations with similar color differences. Experimental findings demonstrate that grayscale images produced using the proposed method outperform those generated by alternative approaches in preserving color differentials and retaining detailed features of the original image. The resulting output exhibits clear and natural outlines, as supported by both subjective and objective assessments.

Robust-optimization-guiding deep reinforcement learning for chemical material production scheduling

This study presents a novel guiding framework embedded with robust optimization (RO) for the training phase of reinforcement learning (RL), specifically tailored for dynamic scheduling of a single-stage multi-product chemical reactor in an uncertain environment. The proposed framework addresses the challenge of local optima in policy gradient methods by integrating optimization methods, enhancing both the objective value and learning stability of the RL. We further enhance the robustness of the proposed model against parameter distortions by incorporating RO as a guiding engine. A numerical study of chemical material production scheduling is conducted to validate the proposed model and the results demonstrate the effectiveness to address demand volatility with several metrics including sensitivity analysis, solution quality analysis, computational time, compared to a typical actor-critic RL.

Constrained tensorial total variation problem based on an alternating conditional gradient algorithm

The present paper aims to regularize ill-posed high-dimensional problems using a constrained total variation minimization approach. The analysis of the continuous model via the dual formulation converts this minimization problem into a constrained minimax problem. Three main issues stand before the resolution of this problem: The minimization over a constraint, the high dimensional representation of the model, and the non-linearity and non-differentiability of the cost function. On the one hand, multidimensional data analysis will be performed using the tensor representation. On the other hand, a new alternating approach based on a proposed pseudo-Lagrangian operator will be developed to deal with the non-linearity and the non-differentiability of this constrained optimization problem. The proposed technique aims to decompose the main tensorial problem into feasible subproblems that can be solved using the conditional gradient method. The convergence of the proposed algorithm is proved and some numerical results are given to illustrate the effectiveness of the presented approach in different fields of image and video processing.

Modeling of the Complexity Propagation of Crack in a Ductile Material Under Complex Solicitation in Crack Tip: Introduction of a Matrix of Stress Intensity Factors of Bifurcation

In fracture and damage mechanics, modeling of crack propagation has always been a source of difficulties. Numerous works have been carried out on this case at the crack tip, introducing new parameters: the Stress Intensity Factor (K); which is the local Irwin parameter, and also the Rice integral (J), the Griffith's energizing method, in which J and G are the global parameters around the crack tip. The problem of the crack remains very complex and difficult problem to be solved. Several methods are used to investigate the crack problem, namely the method of gradient, the numerical methods by finite elements, as well as the thermodynamic approach and the classical methods of Irwin, Griffith or Rice, according to the Intensity Stress Factor. This study adds to the work already carried out. Using the analytical analysis method of equations, we manage to show that the Stress Intensity Factor has a matrix of rank 3 at the crack tip, which is a better form since it includes complex combination cases of crack mode and bifurcation. Furthermore, when the material is subjected to complex stress, after analysis we emerge from a new singularity in (r) which is different from the classical mode. Finally, we are shown the new form of singularity, which is frequency dependent. This work can explain many situations, for example, the case of certain structural disasters showing the presence of cracks for complex or uncontrollable stress.

Artificial Neural Network Control Applied to a Photovoltaic-Battery Microgrid System

This paper deals with artificial neural network (ANN) applied to control a standalone microgrid in French Guiana. ANN is an artificial intelligence technique used to control non-linear and complex systems. ANN associated with the Levenberg–Marquardt (LM) algorithm has many advantages, such as rapid decision-making and improved system transients. Therefore, this technique should be adapted for the control of photovoltaic (PV) systems in the tropical climate of French Guiana with high variation in irradiance. The microgrid is composed of a PV source and a storage battery to supply an isolated building which is modeled by a DC load. The PV source is controlled by an ANN-based MPPT (Maximum Power Point Tracking) controller. To validate our ANN-MPPT, we compared it with one of the very popular MPPT algorithms, which is the P&O-MPPT algorithm. The comparison results show that our ANN-MPPT works well because it can find the maximum power point quickly. In the case of battery control, we tested two feed-forward backpropagation neural network (FFBNN) configurations called method1 and method2 associated with the Levenberg–Marquardt (LM) algorithm. We varied the number of hidden layers in each of these two FFBNN configurations to obtain the optimal number of hidden layers for each configuration which optimizes battery control. Method1 is chosen because it is better than method2, in a sense that it respects the maximum amplitude of the battery current for our application and improves the transient regimes of this current. This best configuration (method1) is then tested with two other learning algorithms for comparison: Bayesian regularization (BR) and scaled conjugate gradient (SCG) methods. The system performance with LM algorithm is better than SCG and BR algorithms. LM algorithm improves the performance of the system in transient regimes while the results obtained with the SGG and BR algorithms are similar. Then, we focused on the advantage of using ANN control compared to the conventional proportional integral control (PI control). The comparison results showed that ANN control associated with the LM algorithm (ANN-LM) made it possible to reduce battery current peaks by 26% in transient regimes compared to conventional PI control. Finally, we present and discuss the results of our simulation obtained with the MATLAB Simulink software.

Modification of a conjugate gradient approach for convex constrained nonlinear monotone equations with applications in signal recovery and image restoration

Conjugate gradient (CG )methods have received widespread use in recent years for recovering damaged signals in compressive sensing. This study attempts to create a CG method that is more effective than the recent existing scheme for recovering disrupted signals. The newly created method is a modification of the method proposed by Halilu et al. (2021). Because the authors’ CG parameter can be undefined while solving general functions. As a result, to accomplish the desired result, a new CG method combined with the projection scheme of Solodov and Svaiter (1999) is presented. In addition, under mild conditions, the proposed method converges globally. Numerical experiments conducted with recent methods show the effectiveness of the proposed method. Furthermore, the proposed method is applied efficiently to address the signal recovery and image restoration problems, yielding a better result than some existing methods in the literature.