Numerical Optimization Algorithm Research Articles

Universal Symmetry of Optimal Control at the Microscale

Optimizing the energy efficiency of driving processes provides valuable insights into the underlying physics and is of crucial importance for numerous applications, from biological processes to the design of machines and robots. Knowledge of optimal driving protocols is particularly valuable at the microscale, where energy supply is often limited. Here, we experimentally and theoretically investigate the paradigmatic optimization problem of moving a potential carrying a load through a fluid, in a finite time and over a given distance, in such a way that the required work is minimized. An important step towards more realistic systems is the consideration of memory effects in the surrounding fluid, which are ubiquitous in real-world applications. Therefore, our experiments were performed in viscous and viscoelastic media, which are typical environments for synthetic and biological processes on the microscale. Despite marked differences between the protocols in both fluids, we find that the optimal control protocol and the corresponding average particle trajectory always obey a time-reversal symmetry. We show that this symmetry, which surprisingly applies here to a class of processes far from thermal equilibrium, holds universally for various systems, including active, granular, and long-range correlated media in their linear regimes. The uncovered symmetry provides a rigorous and versatile criterion for optimal control that greatly facilitates the search for energy-efficient transport strategies in a wide range of systems. Using a machine learning algorithm, we demonstrate that the algorithmic exploitation of time-reversal symmetry can significantly enhance the performance of numerical optimization algorithms. Published by the American Physical Society 2024

FPDclustering: a comprehensive R package for probabilistic distance clustering based methods

AbstractData clustering has a long history and refers to a vast range of models and methods that exploit the ever-more-performing numerical optimization algorithms and are designed to find homogeneous groups of observations in data. In this framework, the probability distance clustering (PDC) family methods offer a numerically effective alternative to model-based clustering methods and a more flexible opportunity in the framework of geometric data clustering. Given nJ-dimensional data vectors arranged in a data matrix and the number K of clusters, PDC maximizes the joint density function that is defined as the sum of the products between the distance and the probability, both of which are measured for each data vector from each center. This article shows the capabilities of the PDC family, illustrating the package .

A Modified Sine Cosine Algorithm for Numerical Optimization

Sine cosine algorithm (SCA) is a recently developed meta-heuristic method based on the characteristics of sine and cosine functions. However, SCA algorithm may suffer premature convergence in solving complex optimization problems. To mitigate this limitation, a modified SCA (MSCA) is developed to achieve an effective balance between exploration and exploitation. Specifically, the conversion parameter can be configured adaptively by using the individual fitness information in the evolution process to guide the individual search behavior. After that, an inertia weight is embedded in the position updating equation to further improve the search accuracy and accelerate the convergence speed. The performance of MSCA algorithm is investigated on 13 benchmark functions and the chaotic time series prediction problem. Experimental results demonstrate that MSCA algorithm is more effective than the other optimization methods.

Towards Hybrid-Optimization Video Coding

Video coding that pursues the highest compression efficiency is the art of computing for rate-distortion optimization. The optimization has been approached in different ways, exemplified by two typical frameworks: block-based hybrid video coding and end-to-end learned video coding. The block-based hybrid framework encompasses more and more coding modes that are available at the decoder side; an encoder tries to search for the optimal coding mode for each block to be coded. This is an online, discrete, search-based optimization strategy. The end-to-end learned framework embraces more and more sophisticated neural networks; the network parameters are learned from a collection of videos, typically using gradient descent-based methods. This is an offline, continuous, numerical optimization strategy. Having analyzed these two strategies, both conceptually and with concrete schemes, this paper suggests investigating hybrid -optimization video coding, that is to combine online and offline, discrete and continuous, search-based and numerical optimization. For instance, we propose a hybrid-optimization video coding scheme, where the decoder consists of trained neural networks and supports several coding modes, and the encoder adopts both numerical and search-based algorithms for the online optimization. Our scheme achieves promising compression efficiency on par with H.265/HM for the random-access configuration.

Multi-swarm surrogate model assisted PSO algorithm to minimize distribution network energy losses

The computational analysis of real-world engineering problems often relies on time-consuming simulation calculations. This presents challenges in balancing computational burden and precision when applying traditional metaheuristic algorithms to large-scale complex problems. While there have been numerous surrogate-assisted metaheuristic optimization algorithms proposed, most of them are designed for solving expensive problems within 30–50 dimensions. In this paper, a novel approach called Parallel-multi-swarm Gaussian Process Regression surrogate-based particle swarm optimization (Parallel-MS-GPRS-PSO) is proposed for high-dimensional problems. This approach combines a standard particle swarm optimization (PSO) algorithm, multi-swarm cooperative PSO (MSCPSO), and a GPR surrogate-based PSO (PSO-GPR). By integrating PSO-GPR and MSCPSO, the proposed method aims to effectively explore and exploit the search space while enhancing the global and local performance of the surrogate model. To validate the effectiveness of the proposed algorithm, it is compared with several existing PSO algorithms on seven benchmark functions and an engineering problem. The results illustrate that the Parallel-MS-GPRS-PSO approach outperforms the existing algorithms. Moreover, the method shows promising performance in computationally expensive engineering optimization problems with 288 variables.

Improved Marine Predators Algorithm and Extreme Gradient Boosting (XGBoost) for shipment status time prediction

Shipment Status Time Prediction (STP) is a complex problem requiring expertise in many disciplines, including Machine Learning (ML) and logistics management, to develop effective solutions. Estimating every possible shipment step before creation plays a vital role in e-commerce sale channels. In this point of view, a novel STP approach was proposed to predict shipment status times. The proposed approach involves two phases. The first leg of the novel STP approach is to build an ML model for estimating shipment statuses using a dataset acquired from a real-world application. The Extreme Gradient Boosting (XGB) and popular ML algorithms were compared for the classification of shipment status prediction first. The XGB algorithm performed best among the compared algorithms, with 99.92% and 96.16% accuracy for training and test sets. Moreover, the ML algorithms were run on the public New York City taxi trip dataset. The XGB algorithm exhibited the best performance. The accuracy for training and test sets are 97.40% and 97.33%, respectively. The second phase of the proposed approach is shipment status time estimation, designed as an optimization problem. The Marine Predators Algorithm (MPA) is a recently proposed optimization algorithm for numerical function optimization. An improved MPA algorithm for STP (STPMPA) was proposed in this study. The performance of the STPMPA algorithm was scrutinized on numerical benchmark problems first. The STPMPA algorithm outperformed all the algorithms in the experiment. Then, the most feasible shipment status times are searched by optimizers using the XGB model. The proposed STPMPA algorithm put forth a superior performance for the STP problem than the compared optimization algorithms. Consequently, experimental studies reveal that the proposed STP approach is able to generate efficient estimations in reasonable times for real-time systems.

Multi-axis fields boost SABRE hyperpolarization

The inherently low signal-to-noise ratio of NMR and MRI is now being addressed by hyperpolarization methods. For example, iridium-based catalysts that reversibly bind both parahydrogen and ligands in solution can hyperpolarize protons (SABRE) or heteronuclei (X-SABRE) on a wide variety of ligands, using a complex interplay of spin dynamics and chemical exchange processes, with common signal enhancements between 103 and 104. This does not approach obvious theoretical limits, and further enhancement would be valuable in many applications (such as imaging mM concentration species in vivo). Most SABRE/X-SABRE implementations require far lower fields (μT-mT) than standard magnetic resonance (>1T), and this gives an additional degree of freedom: the ability to fully modulate fields in three dimensions. However, this has been underexplored because the standard simplifying theoretical assumptions in magnetic resonance need to be revisited. Here, we take a different approach, an evolutionary strategy algorithm for numerical optimization, multi-axis computer-aided heteronuclear transfer enhancement for SABRE (MACHETE-SABRE). We find nonintuitive but highly efficient multiaxial pulse sequences which experimentally can produce a sevenfold improvement in polarization over continuous excitation. This approach optimizes polarization differently than traditional methods, thus gaining extra efficiency.

Bayesian Optimization via Exact Penalty

Constrained optimization problems pose challenges when the objective function and constraints are nonconvex and their evaluation requires expensive black-box simulations. Recently, hybrid optimization methods that integrate statistical surrogate modeling with numerical optimization algorithms have shown great promise, as they inherit the properties of global convergence from statistical surrogate modeling and fast local convergence from numerical optimization algorithms. However, the computational efficiency is not satisfied by practical needs under limited budgets and in the presence of equality constraints. In this article, we propose a novel hybrid optimization method, called exact penalty Bayesian optimization (EPBO), which employs Bayesian optimization within the exact penalty framework. We model the composite penalty function by a weighted sum of Gaussian processes, where the qualitative components of the constraint violations are smoothed by their predictive means. The proposed method features (i) closed-form acquisition functions, (ii) robustness to initial designs, (iii) the capability to start from infeasible points, and (iv) effective handling of equality constraints. We demonstrate the superiority of EPBO to state-of-the-art competitors using a suite of benchmark synthetic test problems and two real-world engineering design problems.

Toward unique electrical ladder network model synthesis of a transformer winding high-frequency modeling using K-means and metaheuristic-based method

PurposeAlthough, numerous optimization algorithms have been devoted to construct an electrical ladder network model (ELNM), they suffer from some frail points such as insufficient accuracy as well as the majority of them are unconstrained, which result in optimal solutions that violate certain security operational constraints. For this purpose, this paper aims to propose a flexible-constraint coyote optimization algorithm; the novelty lies in these points: penalty function is introduced in the objective function to discard any unfeasible solution, an advanced constraint handling technique and empirical relationship between the physical estimated parameters and their natural frequencies.Design/methodology/approachFrequency response analysis (FRA) is very significant for transformer winding diagnosis. Interpreting results of a transformer winding FRA is quite challenging. This paper proposes a new methodology to synthesize a nearly unique ELNM physically and electrically coupled for power transformer winding, basing on K-means and metaheuristic algorithm. To this end, the K-means method is used to cluster the setting of control variables, including the self-mutual inductances/capacitances, and the resistances parameters. Afterward, metaheuristic algorithm is applied to determine the cluster centers with high precision and efficiency.FindingsFRA is performed on a power transformer winding model. Basing on the proposed methodology, the prior knowledge in selecting the initial guess and search space is avoided and the global solution is ensured. The performance of the abovementioned methodology is compared using evaluation expressions to verify its feasibility and accuracy.Originality/valueThe proposed method could be generalized for diagnosis of faults in power transformer winding.

Adaptation of the Scaling Factor Based on the Success Rate in Differential Evolution

Differential evolution is a popular heuristic black-box numerical optimization algorithm which is often used due to its simplicity and efficiency. Parameter adaptation is one of the main directions of study regarding the differential evolution algorithm. The main reason for this is that differential evolution is highly sensitive to the scaling factor and crossover rate parameters. In this study, a novel adaptation technique is proposed which uses the success rate to replace the popular success history-based adaptation for scaling factor tuning. In particular, the scaling factor is sampled with a Cauchy distribution, whose location parameter is set as an nth order root of the current success rate, i.e., the ratio of improved solutions to the current population size. The proposed technique is universal and can be applied to any differential evolution variant. Here it is tested with several state-of-the-art variants of differential evolution, and on two benchmark sets, CEC 2017 and CEC 2022. The performed experiments, which include modifications of algorithms developed by other authors, show that in many cases using the success rate to determine the scaling factor can be beneficial, especially with relatively small computational resource.

Near-Optimal Feedback Guidance for Low-Thrust Earth Orbit Transfers

AbstractThis research describes a near-optimal feedback guidance, based on nonlinear orbit control, for low-thrust Earth orbit transfers. Lyapunov stability theory leads to proving that although several equilibria exist, only the desired operational conditions are associated with a stable equilibrium. This ensures quasi-global asymptotic convergence toward the desired final orbit. The dynamical model includes the effect of eclipsing on the available thrust, as well as all the relevant orbit perturbations, such as several harmonics of the geopotential, solar radiation pressure, aerodynamic drag, and gravitational attraction due to the Sun and the Moon. Near-optimality of the feedback guidance comes from careful selection of the control gains. They are identified in two steps. Step (a) is an extensive table search in which the gains are changed in a large interval. Step (b) uses a numerical optimization algorithm that refines the gains found in (a), while minimizing the time of flight. For the numerical simulations, two scenarios are defined: (i) nominal conditions and (ii) nonnominal conditions, which arise from orbit injection errors and stochastic failures of the propulsion system. For case (i), gain optimization leads to obtaining numerical results very close to those corresponding to a known optimal orbit transfer with eclipse arcs. Moreover, for case (ii), extensive Monte Carlo simulations demonstrate that the nonlinear feedback guidance at hand is effective in driving a spacecraft from a low Earth orbit to a geostationary orbit, also in the presence of nonnominal flight conditions.

Enhancing feature selection with a novel hybrid approach incorporating genetic algorithms and swarm intelligence techniques

Computing advances in data storage are leading to rapid growth in large-scale datasets. Using all features increases temporal/spatial complexity and negatively influences performance. Feature selection is a fundamental stage in data preprocessing, removing redundant and irrelevant features to minimize the number of features and enhance the performance of classification accuracy. Numerous optimization algorithms were employed to handle feature selection (FS) problems, and they outperform conventional FS techniques. However, there is no metaheuristic FS method that outperforms other optimization algorithms in many datasets. This motivated our study to incorporate the advantages of various optimization techniques to obtain a powerful technique that outperforms other methods in many datasets from different domains. In this article, a novel combined method GASI is developed using swarm intelligence (SI) based feature selection techniques and genetic algorithms (GA) that uses a multi-objective fitness function to seek the optimal subset of features. To assess the performance of the proposed approach, seven datasets have been collected from the UCI repository and exploited to test the newly established feature selection technique. The experimental results demonstrate that the suggested method GASI outperforms many powerful SI-based feature selection techniques studied. GASI obtains a better average fitness value and improves classification performance.

Estimation of tunnel axial orientation in the interlayered rock mass using a comprehensive algorithm

The axial selection of tunnels constructed in the interlayered soft-hard rock mass affects the stability and safety during construction. Previous optimization is primarily based on experience or comparison and selection of alternative values under specific geological conditions. In this work, an intelligent optimization framework has been proposed by combining numerical analysis, machine learning (ML) and optimization algorithm. An automatic and intelligent numerical analysis process was proposed and coded to reduce redundant manual intervention. The conventional optimization algorithm was developed from two aspects and applied to the hyperparameters estimation of the support vector machine (SVM) model and the axial orientation optimization of the tunnel. Finally, the comprehensive framework was applied to a numerical case study, and the results were compared with those of other studies. The results of this study indicate that the determination coefficients between the predicted and the numerical stability evaluation indices (STIs) on the training and testing datasets are 0.998 and 0.997, respectively. For a given geological condition, the STI which changed with the axial orientation shows the trend of first decreasing and then increasing, and the optimal tunnel axial orientation is estimated to be 87°. This method provides an alternative and quick approach to the overall design of the tunnels.

EABOA: Enhanced adaptive butterfly optimization algorithm for numerical optimization and engineering design problems

The butterfly optimization algorithm (BOA) is a meta-heuristic algorithm that mimics foraging and mating behavior of butterflies. In order to alleviate the problems of slow convergence, local optimum and lack of population diversity of BOA, an enhanced adaptive butterfly optimization algorithm (EABOA) is proposed in this paper. First, a new adaptive fragrance model is designed, which provided a finer fragrance perception way and effectively enhanced the convergence speed and accuracy. Second, Lévy flight with high-frequency short-step jumping and low-frequency long-step walking is adopted to help the algorithm jump out of the local optimum. Third, the dimension learning-based hunting is employed to enhance information exchange by creating neighbors for each butterfly, thus improving the balance between local and global search and maintaining population diversity. In addition, the Fitness-Distance-Constraint (FDC) method is introduced to enhance constraint handling in EABOA (named FDC-EABOA). The proposed EABOA is compared with 8 well-known algorithms and 8 BOA variants in CEC 2022 test suite and the results were statistically analyzed using Friedman, Friedman aligned rank, Wilcoxon signed rank, Quade rank and multiple comparisons, analysis of variance (ANOVA) and range analysis. Finally, EABOA and FDC-EABOA are applied to seven engineering problems (parameter identification of photovoltaic module model, speed reducer design, tension/compression spring design, pressure vessel design, gear train design, welded beam design, SOPWM for 3-level inverters), and metrics such as Improvement Index (IF) and Mean Constraint Violation (MV) confirm that the proposed algorithms are satisfactory. Experimental results and statistical analysis show that the proposed algorithms outperform the comparison algorithms and demonstrate the strong potential for solving numerical optimization and engineering design problems.

NoQSM-net: Combining Convolutional Neural Network With Numerical Optimization Algorithm for Quantitative Susceptibility Mapping Reconstruction

NoQSM-net: Combining Convolutional Neural Network With Numerical Optimization Algorithm for Quantitative Susceptibility Mapping Reconstruction

Tailoring transverse beam characteristics with the new CERN PS Booster charge-exchange injection system

A key aspect of the LHC Injectors Upgrade project is the connection of the PSB to the newly built Linac4 and the related installation of a new 160 MeV charge-exchange injection system. The new injection system was commissioned in winter 2020/21 and is now used operationally to tailor the transverse characteristics for the various beam types at CERN, such as high-intensity fixed target beams, LHC single bunch beams, and high-brightness beams for LHC. This contribution outlines the different injection strategies for producing the various beam types and discusses the application of numerical optimization algorithms to adjust injection settings in operation efficiently.

Mathematical Model for Optimal Dubins Path to Intercept a Stationary Target

This paper presents a comprehensive mathematical model designed to determine the most efficient trajectory for intercepting a stationary target, with a primary focus on minimizing interception time and resource utilization. The proposed model incorporates key parameters such as target location, interception speed, and environmental constraints to formulate a set of differential equations that describe the motion dynamics of both the pursuer and the target. The objective function is defined to optimize the interception path. The study employs advanced mathematical techniques, including optimal control theory and numerical optimization algorithms, to solve the formulated equations and derive the optimal path. The model is adaptable to various scenarios, accounting for different pursuit vehicle dynamics and target characteristics. Simulation results are presented to validate the effectiveness of the proposed mathematical model in practical applications.
 GANIT J. Bangladesh Math. Soc. 43.1 (2023) 049- 064

A Numerical Technique for Solving Optimization Problems

The aim of this paper is to calculate a better approximation value (whether it is maximize or minimize ) for one- and two-dimensional nonlinear equations using the best numerical optimization algorithms, which is Newton's method. The idea of this technique is based on approximating the function by expanding the Taylor series expansion and iteratively updating the estimate of the optimal solution. we have obtained good results in terms of accuracy and speed of approach, as shown in the examples mentioned. We also mentioned the applications of Newton’s method in multiple disciplines, including engineering, physics, economics, finance, computer graphics, machine learning, image processing, and other applications.

Colony-Based Search Algorithm for numerical optimization

The efficiency of an Evolutionary Algorithm is highly sensitive to the mutation and crossover operators it possesses, as well as to the strategy used for determining the direction of numerical evolution and the values of evolutionary step sizes. There is no analytical method to efficiently define the direction of numerical evolution and the value of evolutionary step size for EAs. The efficiency of EAs' search processes is also influenced by their ability to maintain numerical diversity within the population. This paper introduces the Colony-Based Search Algorithm (CSA). The development of CSA was motivated by the scientific and industrial need for a relatively more efficient EA. CSA possesses relatively more efficient artificial genetic operators and strategies for producing evolutionary direction and step size, and the ability to maintain numerical diversity. CSA generates the Clan Matrix containing the pattern vectors to be evolved in the current iteration by randomly selecting pattern vectors from the Colony Matrix at the beginning of each iteration. This makes it easier for CSA to maintain numerical diversity among pattern vectors for a long time. CSA's mutation method includes three randomly blended components with different properties. The problem-solving performance of CSA is statistically compared with the problem-solving performance of eight popular evolutionary search methods (i.e., SADE, SHADE, LSHADE, COBIDE, JADE, CK, GWO, and SFS) by using benchmark functions of CEC'2017 and CEC’2022. In the experiments, the 3D viewshed analysis was addressed as a real-world problem, employing the CSA. The statistical analyses conducted on the experimental results indicate that CSA performs relatively better than the compared methods to solve numerical problems.

A generalization of the Taylor's theorem

In mathematics, Taylor's theorem is a formula that uses the information of a function to describe its proximity to a point. If the function is smooth enough, the conductive values can be used to construct a multiform to approximate the the function in the neighboring area of that point, which can even be extended to the convergence radius of the scale. Taylor’s theorem also gives the deviation between this multiform and the actual function value. The essay includes the proof and application of Taylor’s theorem and the Taylor’s theorem in multivariate functions. Special functions are set with unknown coefficients to deduce the Taylor’s theorem. Specific questions are solved by using the Taylor’s theorem as some examples. Taylor's theorem is frequently applied in real-world settings to solve problems. For instance, in engineering, it can be used to build a variety of numerical simulations and optimization algorithms. In physics, it can be used to analyze data from physical experiments. In finance, it can be used to manage risk and set prices for financial transactions.