Modern Optimization Algorithms Research Articles

Controlled gradient descent: A control theoretical perspective for optimization

The Gradient Descent (GD) paradigm is a foundational principle of modern optimization algorithms. The GD algorithm and its variants, including accelerated optimization algorithms, geodesic optimization, natural gradient, and contraction-based optimization, to name a few, are used in machine learning and the system and control domain. Here, we proposed a new algorithm based on the control theoretical perspective, labeled as the Controlled Gradient Descent (CGD). Specifically, this approach overcomes the challenges of the abovementioned algorithms, which rely on the choice of a suitable geometric structure, particularly in machine learning. The proposed CGD approach visualizes the optimization as a Manifold Stabilization Problem (MSP) through the notion of an invariant manifold and its attractivity. The CGD approach leads to an exponential contraction of trajectories under the influence of a pseudo-Riemannian metric generated through the control procedure as an additional outcome. The efficacy of the CGD is demonstrated with various test objective functions like the benchmark Rosenbrock function, objective function with a lack of flatness, and semi-contracting objective functions often encountered in machine learning applications.

Using sequential statistical tests for efficient hyperparameter tuning

AbstractHyperparameter tuning is one of the most time-consuming parts in machine learning. Despite the existence of modern optimization algorithms that minimize the number of evaluations needed, evaluations of a single setting may still be expensive. Usually a resampling technique is used, where the machine learning method has to be fitted a fixed number of k times on different training datasets. The respective mean performance of the k fits is then used as performance estimator. Many hyperparameter settings could be discarded after less than k resampling iterations if they are clearly inferior to high-performing settings. However, resampling is often performed until the very end, wasting a lot of computational effort. To this end, we propose the sequential random search (SQRS) which extends the regular random search algorithm by a sequential testing procedure aimed at detecting and eliminating inferior parameter configurations early. We compared our SQRS with regular random search using multiple publicly available regression and classification datasets. Our simulation study showed that the SQRS is able to find similarly well-performing parameter settings while requiring noticeably fewer evaluations. Our results underscore the potential for integrating sequential tests into hyperparameter tuning.

Characteristics Analysis and Comparison of a Cylindrical Linear Induction Motor with Composite Secondary Structure

The cylinder linear induction motor (CLIM) is a variation of the rotary induction motor. Its structure is simple, it has a low manufacturing cost, and it can generate linear thrust without the need for a conversion mechanism. It is particularly suitable for electromagnetic catapults, magnetic levitation transport, and industrial production fields, due to its strong environmental adaptability. Designing a high-thrust and high-efficiency CLIM is a great challenge due to its inherent drawbacks, such as the low thrust density and power density of induction motors. In this article, two CLIMs with different topologies are proposed to meet the demand for control-rod drives in high-temperature and high-pressure environments. The article elucidates the topologies of the two CLIMs and proposes an analytical computational approach for the CLIM. Modern optimization algorithms were utilized to optimize the design of the structural parameters of both CLIMs. A 3D-FEA simulation was used to compare and analyze the air-gap magnetism and thrust characteristics of two CLIMs. The results indicate that the copper-ring secondary CLIM has a higher thrust density and is more suitable for use in control-rod drive mechanism (CRDM) systems.

Optimization of numerical and engineering problems using altered differential evolution algorithm

In this study, an altered differential evolution (ADE) is presented for numerical and engineering problem optimization. It incorporates innovative mutation strategy with new control parameters using the perception of particle swarm optimization (PSO) process, to enhance exploration and exploitation activities extra profusely and increase the global search capacity. Also, a new crossover rate is employed in ADE, to attain higher convergence accuracy and quality optimal solutions. Finally, a novel selection strategy is introduced in ADE, to facilitate information sharing as well as for escaping local minima and keeps progressing. To investigate the suggested ADE performance, a collection of 13 classical benchmark functions, CEC2014 and CEC2017 benchmark suite are solved. Furthermore, the superiority and applicability of the ADE algorithm are further demonstrated through experimentation on six famous real-life engineering problems. The experimental and statistical test outcomes, collectively indicate that compared to other modern optimization algorithms, overall ADE exhibits superior performance. Also, comparison results show that ADE has powerful exploration and exploitation capabilities, excellent convergence performance, and strong ability for gaining high quality solution.

An Enhanced Genetic Algorithm using Directional-Based Crossover and normal mutation For Global Optimization Problems

Global optimization has been employed in many practical modeling processes. Using gradient methods to solve optimization problems may be computationally inefficient and time-consuming, particularly when convexity or differentiability is not guaranteed. On the other hand, nature-inspired techniques offer an effective gradient-free approach for solving complex, non-convex, or non-differentiable problems. Genetic algorithms are one of the most effective and widely used nature-inspired techniques. However, canonical genetic algorithms do not always guarantee convergence to the optimum point owing to the stochastic nature of the genetic operators, and typically require more work to ensure convergence and increase performance. Improving the genetic operators remains an open issue and usually involves a trade-off between the speed of convergence and searchability. In this study, we propose an enhanced genetic algorithm that relies on directional-based crossover and normal mutation operators to increase the speed of convergence while preserving searchability. The proposed algorithm is evaluated using a set of 40 typical benchmark functions in two dimensions. In addition, to examine its performance at higher dimensions, 16 functions from the test set were tested at 10 and 100 dimensions. The evaluation results of the proposed algorithm are compared to the outcomes of three modern optimization algorithms, namely (Whale optimization algorithm, Teacher-Learner based algorithm, and Covariance matrix adaptation evolution strategy). The results revealed that the proposed algorithm outperformed the conventional algorithms at lower dimensions in all test functions and showed a relatively better performance than the other algorithms at higher dimensions.

Adaptive DBSCAN Clustering and GASA Optimization for Underdetermined Mixing Matrix Estimation in Fault Diagnosis of Reciprocating Compressors.

Underdetermined blind source separation (UBSS) has garnered significant attention in recent years due to its ability to separate source signals without prior knowledge, even when sensors are limited. To accurately estimate the mixed matrix, various clustering algorithms are typically employed to enhance the sparsity of the mixed matrix. Traditional clustering methods require prior knowledge of the number of direct signal sources, while modern artificial intelligence optimization algorithms are sensitive to outliers, which can affect accuracy. To address these challenges, we propose a novel approach called the Genetic Simulated Annealing Optimization (GASA) method with Adaptive Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering as initialization, named the CYYM method. This approach incorporates two key components: an Adaptive DBSCAN to discard noise points and identify the number of source signals and GASA optimization for automatic cluster center determination. GASA combines the global spatial search capabilities of a genetic algorithm (GA) with the local search abilities of a simulated annealing algorithm (SA). Signal simulations and experimental analysis of compressor fault signals demonstrate that the CYYM method can accurately calculate the mixing matrix, facilitating successful source signal recovery. Subsequently, we analyze the recovered signals using the Refined Composite Multiscale Fuzzy Entropy (RCMFE), which, in turn, enables effective compressor connecting rod fault diagnosis. This research provides a promising approach for underdetermined source separation and offers practical applications in fault diagnosis and other fields.

The Research on Pricing and Replenishment Optimization of Fresh Supermarket Vegetable Products based on Sales Data

This study delves into the challenges and complexities of vegetable sales management in modern commercial environments, particularly in fresh food supermarkets. By utilizing descriptive statistics and visual analysis of sales data, the study reveals the correlations between different vegetable categories and quantifies these correlations using Pearson's correlation coefficient. Subsequently, by integrating time series analysis and multi-objective programming, a mathematical model is constructed, aimed at maximizing profits under specific constraints. The innovation of this research lies in its comprehensive consideration of category-level sales management and the application of modern optimization algorithms for replenishment and pricing strategies. The uniqueness of this paper is in its integrative approach to the vegetable sales problem, providing refined mathematical models and advanced optimization methods. Finally, the paper thoroughly describes the steps of the research design, including sales data analysis, cost markup analysis, and the construction of an optimization model based on time series analysis and multi-objective programming, intending to offer supermarkets a vegetable sales management plan adaptable to market changes.

Heuristic Logic Resynthesis Algorithms at the Core of Peephole Optimization

Logic resynthesis is one of the core problems in modern peephole logic optimization algorithms. Given a target function and a set of existing functions, logic resynthesis asks for a circuit reusing some of the existing functions and generating the target. While exact methods such as enumeration and SATbased synthesis guarantee optimal solutions, limitations on the problem size are inevitable due to scalability concerns. In this work, we propose heuristic resynthesis algorithms for ANDbased, majority-based, and multiplexer-based circuits, which are scalable in all aspects. Used as the core of high-effort optimization, our heuristic resynthesis algorithms play a key role in enabling 2-3% further size reduction on benchmarks that are already processed by state-of-the-art optimization flows.

Reliability constrained dynamic generation expansion planning using honey badger algorithm

Generation expansion planning (GEP) is a complex, highly constrained, non-linear, discrete and dynamic optimization task aimed at determining the optimum generation technology mix of the best expansion alternative for long-term planning horizon. This paper presents a new framework to study the GEP in a multi-stage horizon with reliability constrained. GEP problem is presented to minimize the capital investment costs, salvage value cost, operation and maintenance, and outage cost under several constraints over planning horizon. Added to that, the spinning reserve, fuel mix ratio and reliability in terms of Loss of Load Probability are maintained. Moreover, to decrease the GEP problem search space and reduce the computational time, some modifications are proposed such as the Virtual mapping procedure, penalty factor approach, and the modified of intelligent initial population generation. For solving the proposed reliability constrained GEP problem, a novel honey badger algorithm (HBA) is developed. It is a meta-heuristic search algorithm inspired from the intelligent foraging behavior of honey badger to reach its prey. In HBA, the dynamic search behavior of honey badger with digging and honey finding approaches is formulated into exploration and exploitation phases. Added to that, several modern meta-heuristic optimization algorithms are employed which are crow search algorithm, aquila optimizer, bald eagle search and particle swarm optimization. These algorithms are applied, in a comparative manner, for three test case studies for 6-year, 12-year, and 24-year of short- and long-term planning horizon having five types of candidate units. The obtained results by all these proposed algorithms are compared and validated the effectiveness and superiority of the HBA over the other applied algorithms.

Boosting Power Density of Proton Exchange Membrane Fuel Cell Using Artificial Intelligence and Optimization Algorithms.

The adoption of Proton Exchange Membrane (PEM) fuel cells (FCs) is of great significance in diverse industries, as they provide high efficiency and environmental advantages, enabling the transition to sustainable and clean energy solutions. This study aims to enhance the output power of PEM-FCs by employing the Adaptive Neuro-Fuzzy Inference System (ANFIS) and modern optimization algorithms. Initially, an ANFIS model is developed based on empirical data to simulate the output power density of the PEM-FC, considering factors such as pressure, relative humidity, and membrane compression. The Salp swarm algorithm (SSA) is subsequently utilized to determine the optimal values of the input control parameters. The three input control parameters of the PEM-FC are treated as decision variables during the optimization process, with the objective to maximize the output power density. During the modeling phase, the training and testing data exhibit root mean square error (RMSE) values of 0.0003 and 24.5, respectively. The coefficient of determination values for training and testing are 1.0 and 0.9598, respectively, indicating the successfulness of the modeling process. The reliability of SSA is further validated by comparing its outcomes with those obtained from particle swarm optimization (PSO), evolutionary optimization (EO), and grey wolf optimizer (GWO). Among these methods, SSA achieves the highest average power density of 716.63 mW/cm2, followed by GWO at 709.95 mW/cm2. The lowest average power density of 695.27 mW/cm2 is obtained using PSO.

Performance prediction and optimization of a hybrid renewable-energy-based multigeneration system using machine learning

This paper proposes an innovative multi-generation hybrid renewable energy system (HRES). The proposed HRES is optimized using the recently developed equilibrium optimizer algorithm and the efficacy of this algorithm is investigated by comparing the optimized sizing with results obtained from more traditional optimization algorithms, i.e., gray wolf optimizer, lightning search algorithm, and artificial bee colony. The results indicate that using the equilibrium optimizer algorithm outperforms conventional optimization algorithms in minimizing the levelized cost of electricity to $0.83 per kWh. This proves applicability of this modern optimization algorithm in improving HRES, which is a novelty of this research. Moreover, machine learning is implemented to predict the exergy efficiency of the proposed HRES, which provides insight into the performance and sustainability of the system. A thermodynamic database is established to train the machine learning model using secondary data. Using the multi-layer perceptron class, the exergy efficiency of the system is predicted; the trained model is able to deliver a prediction with an R-Squared value of 0.98. Thus, the appropriateness of machine learning algorithms in predicting the performance of HRES is verified. This research will, therefore, pave the way towards expanding real-world application of HRES's. This study provides insights into the optimization and economic and environmental benefits of hybrid renewable energy systems and inform policymakers on how to incentivize their implementation. As the study is performed for a specific location, exploring the feasibility of implementing the proposed system in other locations is recommended for future research.

Modern Optimization Algorithm for Improved Performance of Maximum Power Point Tracker of Partially Shaded PV Systems

Due to the rapid advancement in the use of photovoltaic (PV) energy systems, it has become critical to look for ways to improve the energy generated by them. The extracted power from the PV modules is proportional to the output voltage. The relationship between output power and array voltage has only one peak under uniform irradiance, whereas it has multiple peaks under partial shade conditions (PSCs). There is only one global peak (GP) and many local peaks (LPs), where the typical maximum power point trackers (MPPTs) may become locked in one of the LPs, significantly reducing the PV system’s generated power and efficiency. The metaheuristic optimization algorithms (MOAs) solved this problem, albeit at the expense of the convergence time, which is one of these algorithms’ key shortcomings. Most MOAs attempt to lower the convergence time at the cost of the failure rate and the accuracy of the findings because these two factors are interdependent. To address these issues, this work introduces the dandelion optimization algorithm (DOA), a novel optimization algorithm. The DOA’s convergence time and failure rate are compared to other modern MOAs in critical scenarios of partial shade PV systems to demonstrate the DOA’s superiority. The results obtained from this study showed substantial performance improvement compared to other MOAs, where the convergence time was reduced to 0.4 s with zero failure rate compared to 0.9 s, 1.25 s, and 0.43 s for other MOAs under study. The optimal number of search agents in the swarm, the best initialization of search agents, and the optimal design of the dc–dc converter are introduced for optimal MPPT performance.

Learning to Break Symmetries for Efficient Optimization in Answer Set Programming

The ability to efficiently solve hard combinatorial optimization problems is a key prerequisite to various applications of declarative programming paradigms. Symmetries in solution candidates pose a significant challenge to modern optimization algorithms since the enumeration of such candidates might substantially reduce their performance. This paper proposes a novel approach using Inductive Logic Programming (ILP) to lift symmetry-breaking constraints for optimization problems modeled in Answer Set Programming (ASP). Given an ASP encoding with optimization statements and a set of small representative instances, our method augments ground ASP programs with auxiliary normal rules enabling the identification of symmetries using existing tools, like SBASS. Then, the obtained symmetries are lifted to first-order constraints with ILP. We prove the correctness of our method and evaluate it on real-world optimization problems from the domain of automated configuration. Our experiments show significant improvements of optimization performance due to the learned first-order constraints.

Temperature Self-Adaptive Ultra-Thin Solar Absorber Based on Optimization Algorithm

In solar applications, the solar absorber is paramount to converting solar radiation to heat energy. We systematically examined the relationship between the efficiency of the solar absorber and operating temperature and other factors. By combining inverse designs with surface plasmonic and Fabry-Perot cavity solar absorption theories, we have developed several solar absorber devices with excellent performance at different temperatures. One of these devices displays a solar spectral absorption of 95.6%, an ultra-low emission rate of 5.7%, and optical-to-thermal conversion efficiency exceeding 90%, all within an ultra-thin depth of 0.45 μm under working temperatures of 600 K. The device has the potential to surpass the Shockley-Queisser limit (S-Q limit) in solar power generation systems. Our method is adaptable, enabling the design of optimal-performance devices to the greatest extent possible. The design was optimized using modern optimization algorithms to meet complex conditions and offers new insights for further study of the conversion from solar to thermal energy and the advancement of solar energy applications.

Maximizing Green Hydrogen Production from Water Electrocatalysis: Modeling and Optimization

The use of green hydrogen as a fuel source for marine applications has the potential to significantly reduce the carbon footprint of the industry. The development of a sustainable and cost-effective method for producing green hydrogen has gained a lot of attention. Water electrolysis is the best and most environmentally friendly method for producing green hydrogen-based renewable energy. Therefore, identifying the ideal operating parameters of the water electrolysis process is critical to hydrogen production. Three controlling factors must be appropriately identified to boost hydrogen generation, namely electrolysis time (min), electric voltage (V), and catalyst amount (μg). The proposed methodology contains the following two phases: modeling and optimization. Initially, a robust model of the water electrolysis process in terms of controlling factors was established using an adaptive neuro-fuzzy inference system (ANFIS) based on the experimental dataset. After that, a modern pelican optimization algorithm (POA) was employed to identify the ideal parameters of electrolysis duration, electric voltage, and catalyst amount to enhance hydrogen production. Compared to the measured datasets and response surface methodology (RSM), the integration of ANFIS and POA improved the generated hydrogen by around 1.3% and 1.7%, respectively. Overall, this study highlights the potential of ANFIS modeling and optimal parameter identification in optimizing the performance of solar-powered water electrocatalysis systems for green hydrogen production in marine applications. This research could pave the way for the more widespread adoption of this technology in the marine industry, which would help to reduce the industry’s carbon footprint and promote sustainability.

Research Progress and Prospect of Satellite Constellation Optimization Design

Satellite constellation optimization has gradually developed from simple single-objective optimization to complex multi-objective optimization. Among them, the high-dimensional and multi-objective dynamic optimization problem poses a great challenge to the performance of the algorithm. To improve convergence speed and optimization performance, researchers continue to improve modern optimization algorithms. Firstly, the construction process of the optimization model is reviewed. The model takes the coverage performance, network performance, and system cost as objective functions or constraints. Secondly, the general forms of multi-objective constellation optimization design models under various constraints are given, and the applications of modern optimization algorithms such as genetic algorithm (GA), non-dominated sorting genetic algorithm (NSGA-2), multi-objective particle swarm optimization algorithm (MOPSO), differential evolution algorithm (DE) in different constellation design tasks, as well as their characteristics and development trends are introduced. Finally, it is pointed out that in the era of vigorous development of Leo mega-constellations, the combination of modern optimization algorithms and machine learning is the future research direction to solve high-dimensional and multi-objective dynamic optimization problems.

Continuous global optimization of multivariable functions based on Sergeev and Kvasov diagonal approach

Abstract. One of modern global optimization algorithms is method of Strongin and Piyavskii modified by Sergeev and Kvasov diagonal approach. In recent paper we propose an extension of this approach to continuous multivariable functions defined on the multidimensional parallelepiped. It is known that Sergeev and Kvasov method applies only to a Lipschitz continuous function though it effectively extends one-dimensional algorithm to multidimensional case. So authors modify We modify mentioned method to a continuous functions using introduced by Vanderbei ε-Lipschitz property that generalizes conventional Lipschitz inequality. Vanderbei proved that a real valued function is uniformly continuous on a convex domain if and only if it is ε-Lipschitz. Because multidimensional parallelepiped is a convex compact set, we demand objective function to be only continuous on a search domain. We describe extended Strongin’s and Piyavskii’s methods in the Sergeev and Kvasov modification and prove the sufficient conditions for the convergence. As an example of proposed method’s application, at the end of this article we show numerical optimization results of different continuous but not Lipschitz functions using three known partition strategies: “partition on 2”, “partition on 2N” and “effective”. For the first two of them we present formulas for computing a new iteration point and for recalculating the ε-Lipschitz constant estimate. We also show algorithm modification that allows to find a new search point on any algorithm’s step.

Improving Moth-Flame Optimization Algorithm by using Slime-Mould Algorithm

The MFO algorithm is one of the modern optimization algorithms based on swarm intelligence, and the SMA algorithm is also one of the latest algorithms in the same field and has the advantages of fast convergence, high convergence accuracy, robust and robust. In this research paper, we introduce an optimized algorithm for MFO based on the SMA algorithm to get better performance using the features in the two algorithms, and two different algorithms are proposed in this field. The two predicted new algorithms were tested with standard test functions and the results were encouraging compared to the standard algorithms.

An interior search algorithm based on chaotic and crossover strategies for parameter extraction of polyphase induction machines

The accuracy of the extracted parameters is important for studying the polyphase induction motor performance and/or the motor control schemes. An investigated and improved interior search algorithm (IISA) is presented in this study for extracting the optimal values of estimated parameters of six-phase and three-phase induction motors. This investigation was carried out on two polyphase induction motors as experimental research cases, utilizing features of manufacturer's operation. The estimated parameters show the high capability regarding the performance of the desired IISA optimizer. The performance of the proposed IISA is compared with different modern optimization algorithms including the basic ISA, and other state-of-the-art approaches. Experimental verifications are validated on two polyphase induction motors, called six-phase and three-phase induction motors. The obtained results show that the proposed method is very competitive in extracting the unknown parameters of different induction motor models with a high degree of closeness to the experimental records. Moreover, various statistical tests, such as the Wilcoxon rank test, stability analysis, and convergence analysis, have been conducted to justify the performance of the proposed IISA. From all the analyses, it has been revealed that the proposed IISA is a competitive method compared to other popular state-of-the-art competitors and ISA variant with accurately identified parameters.

Comparative study of metaheuristic optimization algorithms for image steganography based on discrete Fourier transform domain

Image steganography and watermarking have demonstrated their efficiency in creating a covert communication channel and in digital image authentication respectively. The importance of cybersecurity tasks sets high efficiency requirements for information embedding methods. Many studies in this area focus on finding new ways to improve the embedding quality. One of them is to use metaheuristic optimization algorithms. The use of classical metaheuristics is common enough for a data hiding area. However, a lot of new metaheuristics have been proposed over the past few years. Their applicability has not yet been evaluated in the field of steganography and digital watermarking. In this paper, we present a comparative study on the efficiency of seven metaheuristic optimization algorithms for finding the best embedding options in the phase spectrum of the Discrete Fourier Transform (DFT). One of the contributions of our research is an improved DFT-based data hiding algorithm, which is a development of the error-free embedding algorithm we obtained earlier. The new algorithm also provides error-free extraction of embedded data, and it is distinguished by the high invisibility of embedding through the use of metaheuristic optimization. The use of metaheuristic optimization led to an increase in the PSNR value by an average of 2%–6% and an increase in the capacity value by an average of 16%–25%. Another important contribution of our research is the original formulation of the optimization problem for different classes of metaheuristics. The experimental results demonstrate the efficiency of metaheuristic optimization for solving data hiding problem. The Differential Evolution and the Particle Swarm Optimization showed the best values of embedding indicators among the classical metaheuristic optimization algorithms. The Gradient-Based Optimizer showed the highest efficiency among modern algorithms. Thus, our research indicates the relevance of further studies of other modern optimization algorithms for a data hiding area.