Improved Convergence Rate Research Articles

Osteoporosis Disease Detection using Optimized Elman Recurrent Neural Network based on Hybrid Bacterial Colony Optimization and Tabu Search Algorithm

Bone loss and fragility are indications of osteoporosis, a condition caused by calcium deficiency. The detection of osteoporosis is a significant and difficult diagnostic endeavor. Elman recurrent neural network (ERNN) is a well-known medical disease detection method due to its modeling sequential data and capturing temporal dependencies. ERNN training can be computationally costly and necessitates precise adjustment of hyperparameters. In this research, optimized ERNN is used to predict osteoporosis diseases to achieve high detection accuracy and to improve the global convergence rate. The new hybrid method is used to optimize the hyperparameters of ERNN based on the bacterial colony optimization (BCO) and tabu search (TS) algorithm, which is called IBCO-ERNN. The hybrid technique can efficiently explore the solution space by combining BCO's global exploration capabilities and TS's local exploitation capability, perhaps leading to better solutions to hyperparameter optimization problems. The hybrid BCO-TS strategy trains the ERNN model to prevent local optima and improve convergence rate. The experimental results demonstrated that the proposed IBCO-ERNN obtained high accuracy and fast convergence compared to other detection methods.

Fast symplectic integrator for Nesterov-type acceleration method

AbstractIn this paper, explicit stable integrators based on symplectic and contact geometries are proposed for a family of non-autonomous ordinarily differential equations (ODEs) found in improving convergence rate of Nesterov’s accelerated gradient method. Symplectic geometry is known to be suitable for describing Hamiltonian mechanics, and contact geometry is known as an odd-dimensional counterpart of symplectic geometry. Moreover, a procedure, called symplectization, is a known way to construct a symplectic manifold from a contact manifold, yielding autonomous Hamiltonian systems from contact ones. It is found in this paper that a previously investigated non-autonomous ODEs can be written as a contact Hamiltonian system family. Then, by developing and applying a symplectization of non-autonomous contact Hamiltonian vector fields expressing the non-autonomous ODEs, novel symplectic integrators are derived. Because the proposed symplectic integrators preserve hidden symplectic and contact structures in the ODEs, they are expected to be more stable than the Runge–Kutta method. Numerical experiments demonstrate that, as expected, the second-order symplectic integrator is stable and high convergence rates are achieved.

Stability of the F∗ Algorithm on Strong Pseudocontractive Mapping and Its Application

This paper investigates the stability of the F∗ iterative algorithm applied to strongly pseudocontractive mappings within the context of uniformly convex Banach spaces. The study leverages both analytic and numerical methods to demonstrate the convergence and stability of the algorithm. In comparison to previous works, where weak-contraction mappings were utilized, the strongly pseudocontractive mappings used in this study preserve the convergence property, exhibit greater stability, and have broader applicability in optimization and fixed point theory. Additionally, this work shows that the type of mapping employed converges faster than those in earlier studies. The results are applied to a mixed-type Volterra–Fredholm nonlinear integral equation, and numerical examples are provided to validate the theoretical findings. Key contributions of this work include the following: (i) the use of strongly pseudocontractive mappings, which offer a more stable and efficient convergence rate compared to weak-contraction mappings; (ii) the application of the F∗ algorithm to a wider range of problems; and (iii) the proposal of future directions for improving convergence rates and exploring the algorithm’s behavior in Hilbert and reflexive Banach spaces.

A Low Mach Preconditioned Harmonic Balance Solver for Cavity Flutter Computations

Abstract Labyrinth seal flutter is a critical phenomenon in turbomachinery, as it can lead to severe structural vibrations and potential component damage. Accurate prediction and mitigation of flutter are paramount to ensuring the reliability and performance of modern turbomachinery systems. This paper explores the numerical computation of a labyrinth seal flutter test case using a low Mach preconditioned harmonic balance (HB) solver and investigates how this approach can improve the accuracy and response time of flutter computations. HB solvers have gained prominence in turbomachinery computations for their ability to efficiently capture unsteady flow phenomena and significantly reduce computational time compared to time-domain analyses. In labyrinth seals, however, the flow is often characterized by low Mach numbers, and preconditioning for these conditions has been shown to significantly improve convergence and accuracy. The goal of this paper is to demonstrate how to implement low Mach preconditioning in a HB solver in the frequency domain. We employ iterative preconditioning to alleviate the stiffness associated with density-based solvers under low Mach conditions and analyze the effect of the preconditioning parameters on the convergence rate. Furthermore, we address inaccuracies linked to the classical Roe solver in low Mach scenarios by adapting it to the low Mach preconditioned governing equations. Through the combined utilization of iterative preconditioning and a preconditioned Roe solver, this study aims to improve convergence rates and the overall quality of flutter predictions. We demonstrate the method with an academic labyrinth seal test case originally presented by Corral et al. (“Higher Order Conceptual Model for Labyrinth Seal Flutter,” ASME J. Turbomach., 143(7), p. 071006). While previous investigations have primarily relied on linearized frequency domain solvers and reduce-order models, in this research a preconditioned HB solver is applied to this test case.

A circuit-generated quantum subspace algorithm for the variational quantum eigensolver.

Recent research has shown that wavefunction evolution in real and imaginary time can generate quantum subspaces with significant utility for obtaining accurate ground state energies. Inspired by these methods, we propose combining quantum subspace techniques with the variational quantum eigensolver (VQE). In our approach, the parameterized quantum circuit is divided into a series of smaller subcircuits. The sequential application of these subcircuits to an initial state generates a set of wavefunctions that we use as a quantum subspace to obtain high-accuracy groundstate energies. We call this technique the circuit subspace variational quantum eigensolver (CSVQE) algorithm. By benchmarking CSVQE on a range of quantum chemistry problems, we show that it can achieve significant error reduction in the best case compared to conventional VQE, particularly for poorly optimized circuits, greatly improving convergence rates. Furthermore, we demonstrate that when applied to circuits trapped at local minima, CSVQE can produce energies close to the global minimum of the energy landscape, making it a potentially powerful tool for diagnosing local minima.

Design and Applicability of Two-Step Fractional Newton–Raphson Method

Developing two-step fractional numerical methods for finding the solution of nonlinear equations is the main objective of this research article. In addition, we present a detailed study of convergence analysis for the methods that have been proposed. By comparing numerically, we can see that the proposed methods significantly improve convergence rate and accuracy. Additionally, we demonstrate how our main results can be applied to basins of attraction.

Distributed learning with compressed gradient differences*

Training large machine learning models requires a distributed computing approach, with communication of the model updates being the bottleneck. For this reason, several methods based on the compression of updates were recently proposed using sparsification or quantization. However, none of the prior methods are able to learn the gradients, which renders them incapable of converging to the true optimum in the batch mode. In this work we propose a new method – DIANA – which resolves this issue via compression of gradient differences. We provide theory in the strongly convex and nonconvex settings that shows improved convergence rates, and use it to obtain the first convergence rate for the previously proposed method TernGrad. Finally, we provide theory to support non-smooth regularizers.

A Combined Quasi-Newton-Type Method Using 4th-Order Directional Derivatives for Solving Degenerate Problems of Unconstrained Optimization

Introduction. Methods of unconstrained optimization play a significant role in machine learning [1–6]. When solving practical problems in machine learning, such as tuning nonlinear regression models, the extremum point of the chosen optimality criterion is often degenerate, which significantly complicates its search. Therefore, degenerate problems are the most challenging in optimization. Known numerical methods for solving the general unconstrained optimization problem, up to the second order, have very low convergence rates when solving degenerate problems [7, 8]. This is explained by the fact that for a significant improvement in convergence rate in this case, it is necessary to use higher-order derivatives in the method than the second order [10]. The purpose of the paper is to develop an efficient quasi-Newton method for solving degenerate unconstrained optimization problems, the idea of which (unlike regularization) involves dividing the entire space into the sum of two orthogonal subspaces. This idea was introduced in [23]. The space division at each iteration of the method is based on the spectral decomposition of the matrix approximating the Hessian of the objective function using the BFGS formula [3]. Each subspace exhibits its own behavior of the objective function, and therefore, an appropriate minimization method is applied on it. Results. A combined quasi-Newton method is presented for solving degenerate unconstrained optimization problems, based on orthogonal decomposition of the Hessian approximation matrix and division of the entire space into the sum of two orthogonal subspaces. On one subspace (the kernel of the Hessian approximation matrix), a method is applied where derivatives in the direction of the 4th order are computed, while on the orthogonal complement to it, a quasi-Newtonian method is applied. A separate one-dimensional search is applied on each of these subspaces to determine the step multiplier in the respective direction. The effectiveness of the presented combined method is confirmed by numerical experiments conducted on widely accepted test functions for unconstrained optimization problems. The proposed method allows obtaining fairly accurate solutions to test tasks in case of degeneracy of the minimum point with significantly lower computational costs for gradient calculations compared to optimization procedures of well-known mathematical packages. Conclusions. The idea of dividing the entire space into the sum of two (or possibly more) orthogonal subspaces when solving complex optimization problems is quite promising in terms of applying a combination of different numerical methods on separate subspaces. In the future, it is planned to conduct theoretical research on the convergence rate of the presented combined method in solving degenerate unconstrained optimization problems. Keywords: unconstrained optimization, quasi-Newton methods, degenerate minimum point, spectral matrix decomposition, Machine Learning.

Nonlinear FOPID controller design for pressure regulation of steam condenser via improved metaheuristic algorithm

Shell and tube heat exchangers are pivotal for efficient heat transfer in various industrial processes. Effective control of these structures is essential for optimizing energy usage and ensuring industrial system reliability. In this regard, this study focuses on adopting a fractional-order proportional-integral-derivative (FOPID) controller for efficient control of shell and tube heat exchanger. The novelty of this work lies in the utilization of an enhanced version of cooperation search algorithm (CSA) for FOPID controller tuning, offering a novel approach to optimization. The enhanced optimizer (en-CSA) integrates a control randomization operator, linear transfer function, and adaptive p-best mutation integrated with original CSA. Through rigorous testing on CEC2020 benchmark functions, en-CSA demonstrates robust performance, surpassing other optimization algorithms. Specifically, en-CSA achieves an average convergence rate improvement of 23% and an enhancement in solution accuracy by 17% compared to standard CSAs. Subsequently, en-CSA is applied to optimize the FOPID controller for steam condenser pressure regulation, a crucial aspect of heat exchanger operation. Nonlinear comparative analysis with contemporary optimization algorithms confirms en-CSA’s superiority, achieving up to 11% faster settling time and up to 55% reduced overshooting. Additionally, en-CSA improves the steady-state error by 8% and enhances the overall stability margin by 12%.

Anderson accelerated preconditioning iterative method for RBF interpolation

Traditional RBF interpolation involves solving a linear system, making it computationally expensive for large datasets. Iterative-based quasi-interpolation combines RBF interpolation with iterative methods to enhance accuracy and convergence. To enhance efficiency and accuracy, we in this paper propose a novel method for RBF quasi-interpolation that combines Anderson acceleration with the asynchronous DCPI, termed Anderson-DCPI. The method alternates between the preconditioning iterative method and Anderson extrapolation, aiming to improve convergence rates. We demonstrate the convergence of Anderson-DCPI for positive definite RBF kernel functions and validate its effectiveness through a series of numerical examples.

Transfer learning‐based physics‐informed neural networks for magnetostatic field simulation with domain variations

AbstractPhysics‐informed neural networks (PINNs) provide a new class of mesh‐free methods for solving differential equations. However, due to their long training times, PINNs are currently not as competitive as established numerical methods. A promising approach to bridge this gap is transfer learning (TL), that is, reusing the weights and biases of readily trained neural network models to accelerate model training for new learning tasks. This work applies TL to improve the performance of PINNs in the context of magnetostatic field simulation, in particular to resolve boundary value problems with geometrical variations of the computational domain. The suggested TL workflow consists of three steps. (a) A numerical solution based on the finite element method (FEM). (b) A neural network that approximates the FEM solution using standard supervised learning. (c) A PINN initialized with the weights and biases of the pre‐trained neural network and further trained using the deep Ritz method. The FEM solution and its neural network‐based approximation refer to an computational domain of fixed geometry, while the PINN is trained for a geometrical variation of the domain. The TL workflow is first applied to Poisson's equation on different 2D domains and then to a 2D quadrupole magnet model. Comparisons against randomly initialized PINNs reveal that the performance of TL is ultimately dependent on the type of geometry variation considered, leading to significantly improved convergence rates and training times for some variations, but also to no improvement or even to performance deterioration in other cases.

User friendly FE Formulation for anisotropic distortional hardening model based on non-associated flow plasticity and its application to springback prediction

Based on non-associated flow plasticity, a newly developed anisotropic distortional hardening model developed by Hu and Yoon [15] is implemented in finite element analysis in a user-friendly manner. The derivatives of complex hardening models are calculated using the Finite Difference Method (FDM), which is much more convenient than using the analytical derivatives. To further improve the accuracy of the proposed method, the step size analysis in FDM is performed by analyzing the derivative formation. To evaluate the accuracy and computational efficiency of a proposed step size for FDM, single element simulations are performed under different loading paths. It has been found that the maximum absolute error of the flow curves between the simulation and the theoretical result is less than 0.3%. The U-bending tests for DP600 and TRIP1180 are used to verify the ability of the distortional hardening model for springback prediction. The simulation result of the strain hardening model is in good agreement with the experiment. The computational efficiency is also increased by 24% due to the improved convergence rate.

Many-objective artificial hummingbird algorithm: an effective many-objective algorithm for engineering design problems

Abstract Many-objective optimization presents unique challenges in balancing diversity and convergence of solutions. Traditional approaches struggle with this balance, leading to suboptimal solution distributions in the objective space especially at higher number of objectives. This necessitates the need for innovative strategies to adeptly manage these complexities. This study introduces a Many-Objective Artificial Hummingbird Algorithm (MaOAHA), an advanced evolutionary algorithm designed to overcome the limitations of existing many-objective optimization methods. The objectives are to improve convergence rates, maintain solution diversity, and achieve a uniform distribution in the objective space. MaOAHA implements information feedback mechanism (IFM), reference point-based selection and association, non-dominated sorting, and niche preservation. The IFM utilizes historical data from previous generations to inform the update process, thereby improving the algorithm’s the exploration and exploitation capabilities. Reference point-based selection, along with non-dominated sorting, ensures solutions are both close to the Pareto front and evenly spread in the objective space. Niche preservation and density estimation strategies are employed to maintain diversity and prevent overcrowding. The comprehensive experimental analysis benchmarks MaOAHA against four leading algorithms viz. Many-Objective Gradient-Based Optimizer, Many-Objective Particle Swarm Optimizer, Reference Vector Guided Evolutionary Algorithm, and Nondominated Sorting Genetic Algorithm III. The DTLZ1–DTLZ7 benchmark sets with four, six, and eight objectives and five real-world problems (RWMaOP1–RWMaOP5) are considered for performance assessment of the selected algorithms. The results demonstrate that internal parameter-free MaOAHA significantly outperforms its counterparts, achieving better generational distance by up to 52.38%, inverse generational distance by up to 38.09%, spacing by up to 56%, spread by up to 71.42%, hypervolume by up to 44%, and runtime by up to 52%. These metrics affirm the MaOAHA’s capability to enhance the decision-making processes through its adept balance of convergence, diversity, and uniformity.

Dual-Adaptive Heterojunction Synaptic Transistors for Efficient Machine Vision in Harsh Lighting Conditions.

Photoadaptive synaptic devices enable in-sensor processing of complex illumination scenes, while second-order adaptive synaptic plasticity improves learning efficiency by modifying the learning rate in a given environment. The integration of above adaptations in one phototransistor device will provide opportunities for developing high-efficient machine vision system. Here, a dually adaptable organic heterojunction transistor as a working unit in the system, which facilitates precise contrast enhancement and improves convergence rate under harsh lighting conditions, is reported. The photoadaptive threshold sliding originates from the bidirectional photoconductivity caused by the light intensity-dependent photogating effect. Metaplasticity is successfully implemented owing to the combination of ambipolar behavior and charge trapping effect. By utilizing the transistor array in a machine vision system, the details and edges can be highlighted in the 0.4% low-contrast images, and a high recognition accuracy of 93.8% with a significantly promoted convergence rate by about 5 times are also achieved. These results open a strategy to fully implement metaplasticity in optoelectronic devices and suggest their vision processing applications in complex lighting scenes.

Revolutionizing connectivity: Unleashing the power of 5G wireless networks enhanced by artificial intelligence for a smarter future

In the context of 5G wireless networks, this paper addresses a crucial concern—data loss in Analog-to-Digital Converters (ADCs) embedded within Multiple Input Multiple Output (MIMO) channels, a prevalent challenge in modern wireless communication systems. Despite the integration of ADCs in MIMO configurations to mitigate this issue, their implementation introduces significant complexities. In response to this, the paper introduces an innovative and efficient approach aimed at enhancing channel estimation in 5G MIMO systems. The proposed methodology relies on a heuristic-based optimization technique within a channel prediction framework. The framework uses feedback information collected through a Cascade Fusion Serial Network (CFSN) to analyze the receiver-side error ratio and estimate the channel coefficients of MIMO systems at the transmitter. An attention mechanism is incorporated into the model design through the use of a Convolutional Neural Networks - Gated Recurrent Units (CNN-GRU) and a Multi-Scaled Stacked Auto encoder. Parameters of Attention CNN-GRU, Stacked Auto encoder, and CFSN are refined using the Whale Optimization method. The optimization objective is to minimize key metrics, including Root Mean Squared Error (RMSE), Bit Error Rate (BER), and Mean Squared Error (MSE) associated with the estimated channel. Experimental evaluations underscore the efficacy of the proposed MIMO model in the context of 5G networks. The results demonstrate improved convergence rates, heightened prediction performance, and reduced computational costs compared to existing methods. This research contributes valuable insights into addressing data loss challenges specific to 5G MIMO systems, providing a pioneering solution that integrates heuristic optimization, advanced neural network architectures, and nature-inspired algorithms for parameter tuning.

Linear Mixed-Effects Models for Dependent Data: Power and Accuracy in Parameter Estimation

Linear mixed-effects models have been increasingly used to analyze dependent data in psychological research. Despite their many advantages over ANOVA, critical issues in their analyses remain. Due to increasing random effects and model complexity, estimation computation is demanding, and convergence becomes challenging. Applied users need help choosing appropriate methods to estimate random effects. The present Monte Carlo simulation study investigated the impacts when the restricted maximum likelihood (REML) and Bayesian estimation models were misspecified in the estimation. We also compared the performance of Akaike information criterion (AIC) and deviance information criterion (DIC) in model selection. Results showed that models neglecting the existing random effects had inflated Type I errors, unacceptable coverage, and inaccurate R-squared measures of fixed and random effects variation. Furthermore, models with redundant random effects had convergence problems, lower statistical power, and inaccurate R-squared measures for Bayesian estimation. The convergence problem is more severe for REML, while reduced power and inaccurate R-squared measures were more severe for Bayesian estimation. Notably, DIC was better than AIC in identifying the true models (especially for models including person random intercept only), improving convergence rates, and providing more accurate effect size estimates, despite AIC having higher power than DIC with 10 items and the most complicated true model.

Two novel iterative approaches for improved LSPIA convergence

This paper introduces two improved variants of the least squares progressive-iterative approximation (LSPIA) by leveraging momentum techniques. Specifically, based on the Polyak's and Nesterov's momentum techniques, the proposed methods utilize the previous iteration information to update the control points. We name these two methods PmLSPIA and NmLSPIA, respectively. The introduction of momentum enhances the determination of the search directions, leading to a significant improvement in convergence rate. The geometric interpretations of PmLSPIA and NmLSPIA are elucidated, providing insights into the underlying principles of these accelerated algorithms. Rigorous convergence analyses are conducted, revealing that both PmLSPIA and NmLSPIA exhibit faster convergence than LSPIA. Numerical results further validate the efficacy of the proposed algorithms.

An inspired chaos-based estimation-theory optimization for low-density parity-check (LDPC) code decoding

Optimization techniques have emerged as powerful tools for solving complex engineering problems in real-time applications, surpassing traditional heuristic methods. This paper introduces a novel chaos optimization algorithm that integrates the bisection search method to enhance the convergence ability, solution accuracy, and search speed of conventional optimization problems. The proposed algorithm is applied to the design and analysis of a reduced complexity decoding scheme using Low-Density Parity-Check (LDPC) codes. By leveraging the Linear Minimum Mean Square Error (LMMSE) estimation criterion, the proposed method approximates the estimated parameters acquired during the message computation process. This combined approach effectively mitigates error propagation and premature convergence issues. The presented research offers a low-complexity decoding algorithm for LDPC codes, leveraging an optimized chaos bisection search method. The adoption of the LMMSE criterion overcomes data dependency and error propagation concerns during the estimation process. The modified chaos optimization technique, integrated with the bisection search, efficiently determines optimal values from perturbed observations, leading to improved convergence rates and data processing precision by reducing the observation dimension (search space). Simulation results validate the proposed approach, showcasing its ability to mitigate error propagation and premature convergence while maintaining coding gain accuracy. Experimental validation on an FPGA RF-SoC device (model XCZU27DR) demonstrates that the proposed method achieves a maximum throughput of 8.7 Gbps at a clock frequency of 386.72 MHz, with reduced hardware complexity. The experimental verification on an FPGA RF-SoC device highlights the algorithm's minimal hardware complexity, superior decoding throughput, and Bit Error Rate (BER) performance.

The G-invariant graph Laplacian Part I: Convergence rate and eigendecomposition

Graph Laplacian based algorithms for data lying on a manifold have been proven effective for tasks such as dimensionality reduction, clustering, and denoising. In this work, we consider data sets whose data points lie on a manifold that is closed under the action of a known unitary matrix Lie group G. We propose to construct the graph Laplacian by incorporating the distances between all the pairs of points generated by the action of G on the data set. We deem the latter construction the “G-invariant Graph Laplacian” (G-GL). We show that the G-GL converges to the Laplace-Beltrami operator on the data manifold, while enjoying a significantly improved convergence rate compared to the standard graph Laplacian which only utilizes the distances between the points in the given data set. Furthermore, we show that the G-GL admits a set of eigenfunctions that have the form of certain products between the group elements and eigenvectors of certain matrices, which can be estimated from the data efficiently using FFT-type algorithms. We demonstrate our construction and its advantages on the problem of filtering data on a noisy manifold closed under the action of the special unitary group SU(2).

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent.