Numerical Optimization Algorithm Research Articles

Convergence analysis of a primal–dual optimization-by-continuation algorithm

We present a numerical iterative optimization algorithm for the minimization of a cost function consisting of a linear combination of three convex terms, one of which is differentiable, a second one is prox-simple and the third one is the composition of a linear map and a prox-simple function. The algorithm’s special feature lies in its ability to approximate, in a single iteration run, the minimizers of the cost function for many different values of the parameters determining the relative weight of the three terms in the cost function. A proof of convergence of the algorithm, based on an inexact variable metric approach, is also provided. As a special case, one recovers a generalization of the primal–dual algorithm of Chambolle and Pock, and also of the proximal-gradient algorithm. Finally, we show how it is related to a primal–dual iterative algorithm based on inexact proximal evaluations of the non-smooth terms of the cost function.

Analytical equation for camera imaging with refractive interfaces

Camera imaging through refractive interfaces is a crucial issue in photogrammetric measurements. Most past studies adopted numerical optimization algorithms based on refractive ray tracing procedures. In these studies, the camera and interface parameters are usually calculated iteratively with numerical optimization algorithms. Inappropriate initial values can cause iterations to diverge. Meanwhile, these iterations cannot efficiently reveal the accurate nature of refractive imaging. Therefore, obtaining camera calibration results that are both flexible and physically interpretable continues to be challenging. Consequently, in this study, we modeled refractive imaging by employing ray transfer matrix analysis. Subsequently, we deduced an analytical refractive imaging (ARI) equation that explicitly describes the refractive geometry in a matrix form. Although this equation is built upon the paraxial approximation, we executed a numerical experiment that shows that the developed analytical equation can accurately illustrate refractive imaging with a considerable object distance and a slightly tilted angle of the flat interface. This ARI equation can be used to define the expansion center and the normal vector of the flat interface. Finally, we also propose a flexible measurement method to determine the orientation of the flat interface, wherein the orientation can be measured rather than calculated by iterative procedures.

A triple population adaptive differential evolution

The Differential Evolution (DE) algorithm is one of the most efficient algorithms for complex numerical optimization. However, the nature of differential mutation and crossover hinders the individuals from a major change and always guides them toward their superior neighbors. There's a lack of useful directional information to help the population escape from early convergence. To solve the above problem, this paper proposes a novel Triple-population-based Adaptive Differential Evolution (TPADE) to enhance the evolutionary efficiency in solving various complex numerical optimization problems. First, a population division method with symmetrical linear reduction is designed to divide the parent population of each iteration into three sub-populations of different sizes, i.e., superior sub-population, medium sub-population, and inferior sub-population. Each sub-population adopts distinct differential mutation and crossover operators to maintain balanced search directions. Second, a superior-trial-preserved selection mechanism is proposed to screen useful directional information to guide the next iteration of evolution. Third, an effective parameter adaptation strategy is designed with the linear population size reduction strategy to avoid redundant search. Experiments are then conducted to show that the TPADE exhibits well performance compared with eleven state-of-the-art DE variants, CEC winners, and their variants on the CEC'2014, CEC'2017, and CEC'2022 benchmark suites. The C++ source code of TPADE can be downloaded from https://github.com/DoubleGong/TPADE.

From constraints fusion to manifold optimization: A new directional transport manifold metaheuristic algorithm

The ascent of geometry-based models and methodologies, exemplified by geometric deep learning and manifold numerical optimization algorithms, has inaugurated a novel domain across various applications that grapple with geometric data complexities, such as electroencephalogram signals represented by symmetric positive definite matrix manifold, hierarchical data represented by hyperbolic manifold. The imperative fuels this inevitable paradigm shift to encapsulate the intricacies and richness inherent in data, areas where traditional methods prove inadequate. While metaheuristic algorithms are renowned for their versatile adaptability across applications, offering practical solutions within reasonable timeframes. However, the conventional metaheuristic algorithms fail on manifold applications with meaningless solutions. From an extrinsic optimization perspective, we treat manifold optimization problems as general optimization problems with multiple fused constraints that limit the optimization path to the manifold. This study pioneered the proposal and implementation of a metaheuristic manifold optimization, introducing a novel directional transport operator to rectify previously identified issues. Through experimentation across five sets of 25 problems, comparing against five algorithms, including both gradient-free and gradient-dominant counterparts, our proposed algorithm emerges as the optimal performer within the gradient-free category, demonstrating competitiveness even against gradient-dominant algorithms. Furthermore, we applied the proposed algorithm to the robot dynamic manipulation problem, achieving a close-optimal solution that eludes gradient-dominant approaches. This paper delves into the inherent capabilities and establishes the generalization of a metaheuristic algorithm within non-Euclidean functional landscapes. The source code will be available at https://github.com/lingping-fuzzy/metaheuristic-manifold-optimization.

MODIFICATION OF A QUANTUM-INSPIRED GENETIC ALGORITHM FOR NUMERICAL OPTIMIZATION USING QUDIT UNDER CONDITIONS OF SIMULATING QUANTUM DECOHERENCE

The genetic algorithm for numerical optimization (GA) of the metaheuristic class is a method for finding optimal solutions based on the biological principles of natural selection and variability. GA is characterized by high operating speed, resistance to noise in the data, low probability of hitting the local extremum of the multimodal objective function, as well as the simultaneous application of probabilistic and deterministic rules for generating search space points. An alternative to the classical GA is the quantum-inspired genetic algorithm for numerical optimization (QIGA), which has advantages that are unattainable for GA by using the concepts and principles of quantum computing. The article proposes a new approach to the implementation of a quantum-inspired genetic numerical optimization algorithm for searching for the global maximum of the objective function, based on modeling the functioning of the GA by simulating the execution of quantum calculations based on qudit in the conditions of the existence of quantum decoherence in the era of noisy medium-scale quantum algorithms. For this purpose, to carry out quantum operations of rotating the states of multilevel quantum systems, the paper presents a density matrix based on Heisenberg-Weyl operators as an analogue of the Bloch sphere for qudits. The simulation of quantum decoherence is interpreted from the point of view of the influence of extraneous noise emanating from the environment on the qudit and is presented as the use of a normal random variable with zero mathematical expectation and unit variance in quantum gates. At the same time, the work presents detailed pseudocodes of the functioning of both the most modified quantum-inspired genetic algorithm for numerical optimization and its individual operations. Testing is carried out by conducting computational experiments with the implementation of a modified algorithm on two-dimensional and multidimensional functions of test optimization problems, as well as when solving an applied optimization problem of planning hybrid flow production in the manufacturing industry based on financial costs and solving the problem of increasing forecasting accuracy based on compact extreme learning machines. The experimental results demonstrate the superiority of the new algorithm over QIGA and classical optimization algorithms in the accuracy of the solution, the speed of convergence with the target value of the global maximum and the execution time of the algorithm.

Advanced optimization-based weighted features for ensemble deep learning smart occupancy detection network for road traffic parking

In day-to-day activities, the advanced technology like Internet of Things (IoT) emerges to improve the lifestyle of people. In metropolitan cities, real-time parking is the long-lasting problem that we face in our daily life activities. Urban parking regulation gained more attention because of its capability to diminish energy consumption, congested traffic, and manifestation. The parking space detection for vehicles in real-time is a crucial role for on-street parking control models where the data is delivered to the drivers through internet with the help of Global Positioning System (GPS). Hence, the network congestion gets increased, where the existing model does not performed well in the parking space effectively. This research paper recommends a novel strategy for detecting occupancy in road traffic parking by IoT and deep networks to overcome these difficulties. At first, the required information is gathered from the standard datasets as CNRPark + EXT for further processing. After the data collection process, the preprocessing phase is executed, where the data cleaning and data transformation approaches are done with a multi-scale retinex network. The pre-processing technique is effectively performed to reduce background noise. Further, the feature extraction is progressed through the Residual Attention Network, where the relevant features are extracted from the pre-processed data. Subsequently, the optimal features and weights are selected by the novel Advanced Pelican Optimization Algorithm (APOA) to get the optimal weighted feature selection. This optimal weighted feature is further given to the Ensemble Deep Networks (EDN) by integrating the Deep Conditional Random Field (DCRF), and Extreme Deep Learning (EDL) model. In final process, the detection of occupancy in road traffic parking is determined by averaging both classifiers outcomes. Finally, the results are compared with numerous optimization algorithms and classifiers to ensure the developed system's efficacy. Throughout the validation, the developed model outperforms with an accuracy as 96.6, precision as 92.9 and Mathew's Correlation Coefficient (MCC) as 92.3. Thus, the developed model shows better performance than the existing traditional approaches.

The technology of precise tapping of remaining oil during the high water cut stage of offshore multi-layer reservoirs driven by big data intelligence

Abstract Based on traditional numerical simulation and optimization algorithm, combined with the “hard data” monitored by stratified injection and production real-time monitoring and automatic control technology, a fine reservoir model integrated reservoir knowledge and development law was established, and the remaining oil potential exploitation during high water cut stage was quantified and evaluated by deep forest method to adjust well productivity, greatly improving the remaining oil potential exploitation effect. The innovation has formed a set of accurate remaining oil potential tapping technology based on big data intelligent drive in offshore multi-layer reservoirs with high water cuts. This technology fully adopts the automatic and intelligent characteristics of data-driven and deep forest algorithms, and the historical fitting rate of production dynamic during the high water cut stage is more than 90%, which can accurately guide the design and implementation of the remaining oil potential tapping scheme and provide a reference for similar oil fields.

Universal Symmetry of Optimal Control at the Microscale

Optimizing the energy efficiency of driving processes provides valuable insights into the underlying physics and is of crucial importance for numerous applications, from biological processes to the design of machines and robots. Knowledge of optimal driving protocols is particularly valuable at the microscale, where energy supply is often limited. Here, we experimentally and theoretically investigate the paradigmatic optimization problem of moving a potential carrying a load through a fluid, in a finite time and over a given distance, in such a way that the required work is minimized. An important step towards more realistic systems is the consideration of memory effects in the surrounding fluid, which are ubiquitous in real-world applications. Therefore, our experiments were performed in viscous and viscoelastic media, which are typical environments for synthetic and biological processes on the microscale. Despite marked differences between the protocols in both fluids, we find that the optimal control protocol and the corresponding average particle trajectory always obey a time-reversal symmetry. We show that this symmetry, which surprisingly applies here to a class of processes far from thermal equilibrium, holds universally for various systems, including active, granular, and long-range correlated media in their linear regimes. The uncovered symmetry provides a rigorous and versatile criterion for optimal control that greatly facilitates the search for energy-efficient transport strategies in a wide range of systems. Using a machine learning algorithm, we demonstrate that the algorithmic exploitation of time-reversal symmetry can significantly enhance the performance of numerical optimization algorithms. Published by the American Physical Society 2024

FPDclustering: a comprehensive R package for probabilistic distance clustering based methods

AbstractData clustering has a long history and refers to a vast range of models and methods that exploit the ever-more-performing numerical optimization algorithms and are designed to find homogeneous groups of observations in data. In this framework, the probability distance clustering (PDC) family methods offer a numerically effective alternative to model-based clustering methods and a more flexible opportunity in the framework of geometric data clustering. Given nJ-dimensional data vectors arranged in a data matrix and the number K of clusters, PDC maximizes the joint density function that is defined as the sum of the products between the distance and the probability, both of which are measured for each data vector from each center. This article shows the capabilities of the PDC family, illustrating the package .

Special aspects of patch antennas synthesis using genetic algorithm

Microstrip-fed patch antenna synthesis can be performed using one of the numerous optimization algorithms, such as a genetic algorithm (GA). VBA or IronPython script engines implemented in modern CAD systems make it possible to implement GA. However, the capabilities of these languages for processing matrix data are limited. The use of a MATLAB environment script could combine the capabilities of MATLAB and CAD for electromagnetic modelling. It would make it possible to generate chromosomes of individuals of a given length, build antenna models in CAD that reflect the genetic code of individuals, process modelling results in MATLAB, carry out crossing over and mutations with a given probability for obtaining patch antennas that provide a user-specified value of the goal function. The primary purpose of this work is to propose a script in the MATLAB environment that allows complete automation of microstrip antennas' structural synthesis in MATLAB - CAD using GA, as well as to study GA selection methods and their influence on the results of this algorithm. As a result, a MATLAB script has been developed, combining the capabilities of electromagnetic modelling CAD and the MATLAB environment for processing matrix data for fully automated synthesis of the microstrip-fed patch antenna. It is shown that the proposed script randomly generates an initial population, creates a CAD model corresponding to each individual, determines the value of the goal function for each individual, performs crossing over and mutations, and then iteratively repeats the above actions until the GA convergence condition is satisfied. This paper highlights that the GA convergence speed depends on the selection method chosen before crossing over. Recommendations for choosing a selection method are formulated. The results of synthesizing a microstrip antenna with a frequency band 5.2 times greater than the frequency band of the reference antenna are presented. The presented script combines the capabilities of electromagnetic modelling CAD and MATLAB to fully automate the process of structural synthesis of microstrip antennas with a given frequency response and the main lobe direction of the radiation pattern.

A Modified Sine Cosine Algorithm for Numerical Optimization

Sine cosine algorithm (SCA) is a recently developed meta-heuristic method based on the characteristics of sine and cosine functions. However, SCA algorithm may suffer premature convergence in solving complex optimization problems. To mitigate this limitation, a modified SCA (MSCA) is developed to achieve an effective balance between exploration and exploitation. Specifically, the conversion parameter can be configured adaptively by using the individual fitness information in the evolution process to guide the individual search behavior. After that, an inertia weight is embedded in the position updating equation to further improve the search accuracy and accelerate the convergence speed. The performance of MSCA algorithm is investigated on 13 benchmark functions and the chaotic time series prediction problem. Experimental results demonstrate that MSCA algorithm is more effective than the other optimization methods.

Towards Hybrid-Optimization Video Coding

Video coding that pursues the highest compression efficiency is the art of computing for rate-distortion optimization. The optimization has been approached in different ways, exemplified by two typical frameworks: block-based hybrid video coding and end-to-end learned video coding. The block-based hybrid framework encompasses more and more coding modes that are available at the decoder side; an encoder tries to search for the optimal coding mode for each block to be coded. This is an online, discrete, search-based optimization strategy. The end-to-end learned framework embraces more and more sophisticated neural networks; the network parameters are learned from a collection of videos, typically using gradient descent-based methods. This is an offline, continuous, numerical optimization strategy. Having analyzed these two strategies, both conceptually and with concrete schemes, this paper suggests investigating hybrid -optimization video coding, that is to combine online and offline, discrete and continuous, search-based and numerical optimization. For instance, we propose a hybrid-optimization video coding scheme, where the decoder consists of trained neural networks and supports several coding modes, and the encoder adopts both numerical and search-based algorithms for the online optimization. Our scheme achieves promising compression efficiency on par with H.265/HM for the random-access configuration.

Multi-swarm surrogate model assisted PSO algorithm to minimize distribution network energy losses

The computational analysis of real-world engineering problems often relies on time-consuming simulation calculations. This presents challenges in balancing computational burden and precision when applying traditional metaheuristic algorithms to large-scale complex problems. While there have been numerous surrogate-assisted metaheuristic optimization algorithms proposed, most of them are designed for solving expensive problems within 30–50 dimensions. In this paper, a novel approach called Parallel-multi-swarm Gaussian Process Regression surrogate-based particle swarm optimization (Parallel-MS-GPRS-PSO) is proposed for high-dimensional problems. This approach combines a standard particle swarm optimization (PSO) algorithm, multi-swarm cooperative PSO (MSCPSO), and a GPR surrogate-based PSO (PSO-GPR). By integrating PSO-GPR and MSCPSO, the proposed method aims to effectively explore and exploit the search space while enhancing the global and local performance of the surrogate model. To validate the effectiveness of the proposed algorithm, it is compared with several existing PSO algorithms on seven benchmark functions and an engineering problem. The results illustrate that the Parallel-MS-GPRS-PSO approach outperforms the existing algorithms. Moreover, the method shows promising performance in computationally expensive engineering optimization problems with 288 variables.

Improved Marine Predators Algorithm and Extreme Gradient Boosting (XGBoost) for shipment status time prediction

Shipment Status Time Prediction (STP) is a complex problem requiring expertise in many disciplines, including Machine Learning (ML) and logistics management, to develop effective solutions. Estimating every possible shipment step before creation plays a vital role in e-commerce sale channels. In this point of view, a novel STP approach was proposed to predict shipment status times. The proposed approach involves two phases. The first leg of the novel STP approach is to build an ML model for estimating shipment statuses using a dataset acquired from a real-world application. The Extreme Gradient Boosting (XGB) and popular ML algorithms were compared for the classification of shipment status prediction first. The XGB algorithm performed best among the compared algorithms, with 99.92% and 96.16% accuracy for training and test sets. Moreover, the ML algorithms were run on the public New York City taxi trip dataset. The XGB algorithm exhibited the best performance. The accuracy for training and test sets are 97.40% and 97.33%, respectively. The second phase of the proposed approach is shipment status time estimation, designed as an optimization problem. The Marine Predators Algorithm (MPA) is a recently proposed optimization algorithm for numerical function optimization. An improved MPA algorithm for STP (STPMPA) was proposed in this study. The performance of the STPMPA algorithm was scrutinized on numerical benchmark problems first. The STPMPA algorithm outperformed all the algorithms in the experiment. Then, the most feasible shipment status times are searched by optimizers using the XGB model. The proposed STPMPA algorithm put forth a superior performance for the STP problem than the compared optimization algorithms. Consequently, experimental studies reveal that the proposed STP approach is able to generate efficient estimations in reasonable times for real-time systems.

Multi-axis fields boost SABRE hyperpolarization

The inherently low signal-to-noise ratio of NMR and MRI is now being addressed by hyperpolarization methods. For example, iridium-based catalysts that reversibly bind both parahydrogen and ligands in solution can hyperpolarize protons (SABRE) or heteronuclei (X-SABRE) on a wide variety of ligands, using a complex interplay of spin dynamics and chemical exchange processes, with common signal enhancements between 103 and 104. This does not approach obvious theoretical limits, and further enhancement would be valuable in many applications (such as imaging mM concentration species in vivo). Most SABRE/X-SABRE implementations require far lower fields (μT-mT) than standard magnetic resonance (>1T), and this gives an additional degree of freedom: the ability to fully modulate fields in three dimensions. However, this has been underexplored because the standard simplifying theoretical assumptions in magnetic resonance need to be revisited. Here, we take a different approach, an evolutionary strategy algorithm for numerical optimization, multi-axis computer-aided heteronuclear transfer enhancement for SABRE (MACHETE-SABRE). We find nonintuitive but highly efficient multiaxial pulse sequences which experimentally can produce a sevenfold improvement in polarization over continuous excitation. This approach optimizes polarization differently than traditional methods, thus gaining extra efficiency.

Bayesian Optimization via Exact Penalty

Constrained optimization problems pose challenges when the objective function and constraints are nonconvex and their evaluation requires expensive black-box simulations. Recently, hybrid optimization methods that integrate statistical surrogate modeling with numerical optimization algorithms have shown great promise, as they inherit the properties of global convergence from statistical surrogate modeling and fast local convergence from numerical optimization algorithms. However, the computational efficiency is not satisfied by practical needs under limited budgets and in the presence of equality constraints. In this article, we propose a novel hybrid optimization method, called exact penalty Bayesian optimization (EPBO), which employs Bayesian optimization within the exact penalty framework. We model the composite penalty function by a weighted sum of Gaussian processes, where the qualitative components of the constraint violations are smoothed by their predictive means. The proposed method features (i) closed-form acquisition functions, (ii) robustness to initial designs, (iii) the capability to start from infeasible points, and (iv) effective handling of equality constraints. We demonstrate the superiority of EPBO to state-of-the-art competitors using a suite of benchmark synthetic test problems and two real-world engineering design problems.

Toward unique electrical ladder network model synthesis of a transformer winding high-frequency modeling using K-means and metaheuristic-based method

PurposeAlthough, numerous optimization algorithms have been devoted to construct an electrical ladder network model (ELNM), they suffer from some frail points such as insufficient accuracy as well as the majority of them are unconstrained, which result in optimal solutions that violate certain security operational constraints. For this purpose, this paper aims to propose a flexible-constraint coyote optimization algorithm; the novelty lies in these points: penalty function is introduced in the objective function to discard any unfeasible solution, an advanced constraint handling technique and empirical relationship between the physical estimated parameters and their natural frequencies.Design/methodology/approachFrequency response analysis (FRA) is very significant for transformer winding diagnosis. Interpreting results of a transformer winding FRA is quite challenging. This paper proposes a new methodology to synthesize a nearly unique ELNM physically and electrically coupled for power transformer winding, basing on K-means and metaheuristic algorithm. To this end, the K-means method is used to cluster the setting of control variables, including the self-mutual inductances/capacitances, and the resistances parameters. Afterward, metaheuristic algorithm is applied to determine the cluster centers with high precision and efficiency.FindingsFRA is performed on a power transformer winding model. Basing on the proposed methodology, the prior knowledge in selecting the initial guess and search space is avoided and the global solution is ensured. The performance of the abovementioned methodology is compared using evaluation expressions to verify its feasibility and accuracy.Originality/valueThe proposed method could be generalized for diagnosis of faults in power transformer winding.

Adaptation of the Scaling Factor Based on the Success Rate in Differential Evolution

Differential evolution is a popular heuristic black-box numerical optimization algorithm which is often used due to its simplicity and efficiency. Parameter adaptation is one of the main directions of study regarding the differential evolution algorithm. The main reason for this is that differential evolution is highly sensitive to the scaling factor and crossover rate parameters. In this study, a novel adaptation technique is proposed which uses the success rate to replace the popular success history-based adaptation for scaling factor tuning. In particular, the scaling factor is sampled with a Cauchy distribution, whose location parameter is set as an nth order root of the current success rate, i.e., the ratio of improved solutions to the current population size. The proposed technique is universal and can be applied to any differential evolution variant. Here it is tested with several state-of-the-art variants of differential evolution, and on two benchmark sets, CEC 2017 and CEC 2022. The performed experiments, which include modifications of algorithms developed by other authors, show that in many cases using the success rate to determine the scaling factor can be beneficial, especially with relatively small computational resource.

Near-Optimal Feedback Guidance for Low-Thrust Earth Orbit Transfers

This research describes a near-optimal feedback guidance, based on nonlinear orbit control, for low-thrust Earth orbit transfers. Lyapunov stability theory leads to proving that although several equilibria exist, only the desired operational conditions are associated with a stable equilibrium. This ensures quasi-global asymptotic convergence toward the desired final orbit. The dynamical model includes the effect of eclipsing on the available thrust, as well as all the relevant orbit perturbations, such as several harmonics of the geopotential, solar radiation pressure, aerodynamic drag, and gravitational attraction due to the Sun and the Moon. Near-optimality of the feedback guidance comes from careful selection of the control gains. They are identified in two steps. Step (a) is an extensive table search in which the gains are changed in a large interval. Step (b) uses a numerical optimization algorithm that refines the gains found in (a), while minimizing the time of flight. For the numerical simulations, two scenarios are defined: (i) nominal conditions and (ii) nonnominal conditions, which arise from orbit injection errors and stochastic failures of the propulsion system. For case (i), gain optimization leads to obtaining numerical results very close to those corresponding to a known optimal orbit transfer with eclipse arcs. Moreover, for case (ii), extensive Monte Carlo simulations demonstrate that the nonlinear feedback guidance at hand is effective in driving a spacecraft from a low Earth orbit to a geostationary orbit, also in the presence of nonnominal flight conditions.

Enhancing feature selection with a novel hybrid approach incorporating genetic algorithms and swarm intelligence techniques

Computing advances in data storage are leading to rapid growth in large-scale datasets. Using all features increases temporal/spatial complexity and negatively influences performance. Feature selection is a fundamental stage in data preprocessing, removing redundant and irrelevant features to minimize the number of features and enhance the performance of classification accuracy. Numerous optimization algorithms were employed to handle feature selection (FS) problems, and they outperform conventional FS techniques. However, there is no metaheuristic FS method that outperforms other optimization algorithms in many datasets. This motivated our study to incorporate the advantages of various optimization techniques to obtain a powerful technique that outperforms other methods in many datasets from different domains. In this article, a novel combined method GASI is developed using swarm intelligence (SI) based feature selection techniques and genetic algorithms (GA) that uses a multi-objective fitness function to seek the optimal subset of features. To assess the performance of the proposed approach, seven datasets have been collected from the UCI repository and exploited to test the newly established feature selection technique. The experimental results demonstrate that the suggested method GASI outperforms many powerful SI-based feature selection techniques studied. GASI obtains a better average fitness value and improves classification performance.