Convergence Of Optimization Algorithm Research Articles

На пути к автоматизированному и оптимальному проектированию систем IIoT

In today’s world, the Internet of Things has become an integral part of our lives. The increasing number of intelligent devices and their pervasiveness has made it challenging for developers and system architects to plan and implement systems of Internet of Things and Industrial Internet of Things effectively. The primary objective of this work is to automate the design process of Industrial Internet of Things systems while optimizing the quality of service parameters, battery life, and cost. To achieve this goal, a general four-layer fog-computing model based on mathematical sets, constraints, and objective functions is introduced. This model takes into consideration the various parameters that affect the performance of the system, such as network latency, bandwidth, and power consumption. The Non-dominated Sorting Genetic Algorithm II is employed to find Pareto optimal solutions, while the Technique for Order of Preference by Similarity to Ideal Solution is used to identify compromise solutions on the Pareto front. The optimal solutions generated by this approach represent servers, communication links, and gateways whose information is stored in a database. These resources are chosen based on their ability to enhance the overall performance of the system. The proposed strategy follows a three-stage approach to minimize the dimensionality and reduce dependencies while exploring the search space. Additionally, the convergence of optimization algorithms is improved by using a biased initial population that exploits existing knowledge about how the solution should look. The algorithms used to generate this initial biased population are described in detail. To illustrate the effectiveness of this automated design strategy, an example of its application is presented.

A dynamic parameter identification method for the 5-DOF hybrid robot based on sensitivity analysis

Purpose This paper aims to propose a dynamic parameter identification method based on sensitivity analysis for the 5-degree of freedom (DOF) hybrid robots, to solve the problems of too many identification parameters, complex model, difficult convergence of optimization algorithms and easy-to-fall into a locally optimal solution, and improve the efficiency and accuracy of dynamic parameter identification. Design/methodology/approach First, the dynamic parameter identification model of the 5-DOF hybrid robot was established based on the principle of virtual work. Then, the sensitivity of the parameters to be identified is analyzed by Sobol’s sensitivity method and verified by simulation. Finally, an identification strategy based on sensitivity analysis was designed, experiments were carried out on the real robot and the results were verified. Findings Compared with the traditional full-parameter identification method, the dynamic parameter identification method based on sensitivity analysis proposed in this paper converges faster when optimized using the genetic algorithm, and the identified dynamic model has higher prediction accuracy for joint drive forces and torques than the full-parameter identification models. Originality/value This work analyzes the sensitivity of the parameters to be identified in the dynamic parameter identification model for the first time. Then a parameter identification method is proposed based on the results of the sensitivity analysis, which can effectively reduce the parameters to be identified, simplify the identification model, accelerate the convergence of the optimization algorithm and improve the prediction accuracy of the identified model for the joint driving forces and torques.

Computation Offloading for Tasks With Bound Constraints in Multiaccess Edge Computing

Multi-Access Edge Computing (MEC) provides task offloading services to facilitate the integration of idle resources with the network and bring cloud services closer to the end user. By selecting suitable servers and properly managing resources, task offloading can reduce task completion latency while maintaining the Quality of Service (QoS). Prior research, however, has primarily focused on tasks with strict time constraints, ignoring the possibility that tasks with soft constraints may exceed the bound limits and failing to analyze this complex task constraint issue. Furthermore, considering additional constraint features makes convergent optimization algorithms challenging when dealing with such complex and high-dimensional situations. In this paper, we propose a new computational offloading decision framework by minimizing the long-term payment of computational tasks with mixed bound constraints. In addition, redundant experiences are gotten rid of before the training of the algorithm. The most advantageous transitions in the experience pool are used for training in order to improve the learning efficiency and convergence speed of the algorithm as well as increase the accuracy of offloading decisions. The findings of our experiments indicate that the method we have presented is capable of achieving fast convergence rates while also reducing sample redundancy.

Covid-19 Forecasting Using CNN Approach With A Halbinomial Distribution And A Linear Decreasing Inertia Weight-Based Cat Swarm Optimization

In recent times, the COVID-19 epidemic has spread to over 170 nations. Authorities all around the world are feeling the strain of COVID-19 since the total of infected people is rising as well as they does not familiar to handle the problem. The majority of current research effort is thus being directed on the analysis of COVID-19 data within the framework of the machines learning method. Researchers looked the COVID 19 data to make predictions about who would be treated, who would die, and who would get infected in the future. This might prompt governments worldwide to develop strategies for protecting the health of the public. Previous systems rely on Long Short- Term Memory (LSTM) networks for predicting new instances of COVID-19. The LSTM network findings suggest that the pandemic might be over by June of 2020. However, LSTM may have an over-fitting issue, and it may fall short of expectations in terms of true positive. For this issue in COVID-19 forecasting, we suggest using two methods such as Cat Swarm Optimization (CSO) for reducing the inertia weight linearly and then artificial intelligence based binomial distribution is used. In this proposed study, we take the COVID-19 predicting database as an contribution and normalise it using the min-max approach. The accuracy of classification is improved with the use of the first method to choose the optimal features. In this method, inertia weight is added to the CSO optimization algorithm convergence. Death and confirmed cases are predicted for a certain time period throughout India using Convolutional Neural Network with Partial Binomial Distribution based on carefully chosen characteristics. The experimental findings validate that the suggested scheme performs better than the baseline system in terms of f-measure, recall, precision, and accuracy.

Accelerating Distributed Optimization via Fixed-Time Convergent Flows

Distributed optimization has gained significant attention in recent years, primarily fueled by the availability of a large amount of data and privacy-preserving requirements. This paper presents a fixed-time convergent optimization algorithm for solving a potentially non-convex optimization problem using a first-order multi-agent system. Each agent in the network can access only its private objective function, while local information exchange is permitted between the neighbors. The proposed optimization algorithm combines a fixed-time convergent distributed parameter estimation scheme with a fixed-time distributed consensus scheme as its solution methodology. The results are presented under the assumption that the team objective function is strongly convex, as opposed to the common assumptions in the literature requiring each of the local objective functions to be strongly convex. The results extend to the class of possibly non-convex team objective functions satisfying only the Polyak-Lojasiewicz (PL) inequality. It is also shown that the proposed continuous-time scheme, when discretized using Euler's method, leads to consistent discretization.

Generalized left-localized Cayley parametrization for optimization with orthogonality constraints

We present a reformulation of optimization problems over the Stiefel manifold by using a Cayley-type transform, named the generalized left-localized Cayley transform, for the Stiefel manifold. The reformulated optimization problem is defined over a vector space, whereby we can apply directly powerful computational arts designed for optimization over a vector space. The proposed Cayley-type transform enjoys several key properties which are useful to (i) study relations between the original problem and the proposed problem; (ii) check the conditions to guarantee the global convergence of optimization algorithms. Numerical experiments demonstrate that the proposed algorithm outperforms the standard algorithms designed with a retraction on the Stiefel manifold.

An Improved Phase Space Reconstruction Method‐Based Hybrid Model for Chaotic Traffic Flow Prediction

Traffic flow is chaotic due to nonstationary realistic factors, and revealing the internal nonlinear dynamics of chaotic data and making high‐accuracy predictions is the key to traffic control and inducement. Given that high‐quality phase space reconstruction is the foundation of predictive modeling. Firstly, an improved C‐C method based on the fused norm search domain is proposed to address the issue that the C‐C method in the phase space reconstruction algorithm does not meet the Euclidean metric accuracy and reduces the reconstruction quality when the infinite norm metric is used. Secondly, to address the problem of insufficient learning ability of traditional convolutional combinatorial modeling for complex phase space laws of chaotic traffic flow, the high‐dimensional phase space features are extracted using the layer‐by‐layer pretraining mechanism of convolutional deep belief networks (CDBNs), and the temporal features are extracted by combining with long short‐term memory (LSTM). Finally, an improved probabilistic dynamic reproduction‐based genetic algorithm (PDRGA) is proposed to address the problem of the hybrid model falling into a local optimum when learning the phase space law. Experiments are conducted in three aspects: phase space reconstruction quality analysis, comparison of optimization algorithm convergence, and prediction model performance comparison. The experimentation with two data sets demonstrates that the improved C‐C method combines the advantages of the high accuracy metric of the L2 norm with the low operational complexity of the infinite norm, achieving a balance between reconstruction quality and algorithm efficiency. The proposed PDRGA optimization algorithm is a lightweight improvement of the traditional genetic algorithm (GA) and solves the problem that the model tends to fall into a local optimum by optimizing the initial weights of CDBN. Meanwhile, the five error evaluation indexes of the proposed PDRGA‐CDBN‐LSTM hybrid model are lower than those of the baseline model, providing a new modeling idea for chaotic traffic flow prediction.

Research on improved artificial bee colony algorithm based on large-scale multi-objective group combination optimization

Large power system contains many generating units, and many factors need to be considered during operation. The unit combination optimization is a nonlinear large-scale optimization problem with multi-objective and multi-constraints, and the existing methods have many shortcomings. Artificial bee colony algorithm has good performance in solving nonlinear optimization problems, but it has some disadvantages such as low optimization efficiency and local extremum. To solve these problems, an improved artificial bee colony algorithm is proposed. The algorithm introduces variable field of vision, adjusts the initial solution and selection strategy, and combines with mutation operation in genetic algorithm. A multi - objective optimization model considering economy and environmental protection is constructed. In order to solve the problem that the calculation time is too long due to the expansion of the unit scale, a phased optimization method is adopted, and the improved algorithm is applied to the start-stop scheduling stage. After determining the start-stop state of the unit, the mixed integer programming method is used for load distribution. For unit contains up to 1000 units of large power grid optimization examples are simulated experiment, the experimental results show that the improved optimization algorithm convergence and global search ability is improved, greatly reduces the computation time of the mass unit combination, multiple objective conditions have also made the ideal results, verify the effectiveness of the proposed method.

MIMO Interference Channels Assisted by Reconfigurable Intelligent Surfaces: Mutual Coupling Aware Sum-Rate Optimization Based on a Mutual Impedance Channel Model

We study a multi-user multiple-input multiple-output interference network in the presence of multiple reconfigurable intelligent surfaces (RISs). The entire system is described by using a circuit-based model for the transmitters, receivers, and RISs. This is obtained by leveraging the electromagnetic tool of mutual impedances, which accounts for the signal propagation and the mutual coupling among closely-spaced scattering elements. An iterative and provably convergent optimization algorithm that maximizes the sum-rate of RIS-assisted multi-user interference channels is introduced. Numerical results show that the sum-rate is enhanced if the mutual coupling among the RIS elements is accounted for at the optimization stage.

Efficient spreading strategy for global solution search using a trust-region optimization method: An application to the common-reflection-surface stacking problem

Some processes in seismic imaging can be formulated as a coherence-based problem, such as common-reflection-surface (CRS) stacking. The approach consists of obtaining the CRS attributes that provide the best-fitting CRS surface in the multicoverage data. The problem can be described as an optimization problem and is solved by an optimization algorithm. In general, quick convergent optimization algorithms are local solvers. To obtain a global solution, an efficient strategy has been developed to be used combined with a trust-region local optimization method. This strategy can be divided into two features: the sequential parameter search and the spreading solution. The idea is to first find solutions on a coarse output grid by a sequential parameter search. This feature is based on constructing splines to estimate the maxima of the objective function in one dimension. These estimated maxima are the initial approximations to the local solver. The optimization algorithm obtains the parameters by sequentially solving 1D, 2D, and 3D problems. Once the solutions are found on the coarse grid, useful information is propagated in the neighborhood to obtain the solutions on all output grids. Although the idea of spreading a solution seems easy, its implementation is complex. It is essential to consider the properties of the problem as well as the properties of the optimization algorithm. Through some numerical experiments, the results using this strategy are shown. The use of sequential parameter search and spreading solution provides an improvement not only in the parameters but also in computational time.

Learning with smooth Hinge losses

Due to the non-smoothness of the Hinge loss in SVM, it is difficult to obtain a faster convergence rate with modern optimization algorithms. In this paper, we introduce two smooth Hinge losses ψG(α;σ) and ψM(α;σ) which are infinitely differentiable and converge to the Hinge loss uniformly in α as σ tends to 0. By replacing the Hinge loss with these two smooth Hinge losses, we obtain two smooth support vector machines (SSVMs), respectively. Solving the SSVMs with the Trust Region Newton method (TRON) leads to two quadratically convergent algorithms. Experiments in text classification tasks show that the proposed SSVMs are effective in real-world applications. We also introduce a general smooth convex loss function to unify several commonly-used convex loss functions in machine learning. The general framework provides smooth approximation functions to non-smooth convex loss functions, which can be used to obtain smooth models that can be solved with faster convergent optimization algorithms.

The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems

In this paper, we propose new numerical algorithms in the setting of unconstrained optimization problems and we prove the discrete rate of convergence of order in the iterates of the convex objective function. Our optimization algorithms are obtained via discretizations from dynamical systems with Hessian-driven damping. Finally, some numerical experiments are presented in order to validate the theoretical results.

Optimizing vertical and deviated wells based on advanced initialization using new productivity potential map

Well placement optimization is an important part during the development of different types of reservoirs. As a highly nonlinear problem, it is difficult to obtain the most effective placement simply relying on screening different scenarios generated with engineering judgement. Automatic well placement optimization based on numerical simulation models and optimization algorithms can effectively solve this nonlinear problem, and greatly save the human labor. However, there is still space for improvement to avoid overwhelming computational cost and premature convergence of optimization algorithms. This paper focused on developing an advanced initialization procedure for well placement optimization, where multiple high-quality well placement scenarios are automatically generated and used as initial guesses. Specifically, we proposed a new form of productivity-potential map (NPPM), which measures not only the productivity potential for producers, but also the injectivity potential for injectors. What is more, compared to the traditional PPM which is defined only for vertical wells, NPPM is generalized to also consider deviated wells. Moreover, the proposed NPPM takes into account inter-well distance which penalizes cases where wells would otherwise be placed too close together. The advanced initialization with NPPM is applied to the PUNQ-S3 reservoir and compared to the initialization with a traditional PPM and a Random Initialization. Results based on the optimization of both vertical wells and deviated wells demonstrate that initialization with NPPM could improve both the quality of the optimal solutions and the robustness of a stochastic optimization algorithm (e.g., particle swarm optimization algorithm in this paper) compared to the rest of the two initialization strategies. Considering that NPPM is used as an initialization procedure, it is easy to couple it with other optimizers.

Separable Nonlinear Least-Squares Parameter Estimation for Complex Dynamic Systems.

Nonlinear dynamic models are widely used for characterizing processes that govern complex biological pathway systems. Over the past decade, validation and further development of these models became possible due to data collected via high-throughput experiments using methods from molecular biology. While these data are very beneficial, they are typically incomplete and noisy, which renders the inference of parameter values for complex dynamic models challenging. Fortunately, many biological systems have embedded linear mathematical features, which may be exploited, thereby improving fits and leading to better convergence of optimization algorithms. In this paper, we explore options of inference for dynamic models using a novel method of separable nonlinear least-squares optimization and compare its performance to the traditional nonlinear least-squares method. The numerical results from extensive simulations suggest that the proposed approach is at least as accurate as the traditional nonlinear least-squares, but usually superior, while also enjoying a substantial reduction in computational time.

Algorithms and complexity for functions on general domains

Error bounds and complexity bounds in numerical analysis and information-based complexity are often proved for functions that are defined on very simple domains, such as a cube, a torus, or a sphere. We study optimal error bounds for the approximation or integration of functions defined on Dd⊂Rd and only assume that Dd is a bounded Lipschitz domain. Some results are even more general. We study three different concepts to measure the complexity: order of convergence, asymptotic constant, and explicit uniform bounds, i.e., bounds that hold for all n (number of pieces of information) and all (normalized) domains. It is known for many problems that the order of convergence of optimal algorithms does not depend on the domain Dd⊂Rd. We present examples for which the following statements are true:1. Also the asymptotic constant does not depend on the shape of Dd or the imposed boundary values, it only depends on the volume of the domain.2. There are explicit and uniform lower (or upper, respectively) bounds for the error that are only slightly smaller (or larger, respectively) than the asymptotic error bound.

Perspective maximum likelihood-type estimation via proximal decomposition

We introduce a flexible optimization model for maximum likelihood-type estimation (M-estimation) that encompasses and generalizes a large class of existing statistical models, including Huber’s concomitant M-estimator, Owen’s Huber/Berhu concomitant estimator, the scaled lasso, support vector machine regression, and penalized estimation with structured sparsity. The model, termed perspective M-estimation, leverages the observation that convex M-estimators with concomitant scale as well as various regularizers are instances of perspective functions, a construction that extends a convex function to a jointly convex one in terms of an additional scale variable. These nonsmooth functions are shown to be amenable to proximal analysis, which leads to principled and provably convergent optimization algorithms via proximal splitting. We derive novel proximity operators for several perspective functions of interest via a geometrical approach based on duality. We then devise a new proximal splitting algorithm to solve the proposed M-estimation problem and establish the convergence of both the scale and regression iterates it produces to a solution. Numerical experiments on synthetic and real-world data illustrate the broad applicability of the proposed framework.

Damage identification of bridge structure considering temperature variations based on particle swarm optimization - cuckoo search algorithm

Structures are always exposed to environmental conditions such as varying temperatures and noises; as a consequence, the dynamic features of structures are changed accordingly. But the model-based methods, used to detect damage using optimization algorithms to get global optimal solution, are highly sensitive to environmental conditions, experimental noises, or numerical errors. While the mechanisms of optimization algorithms are limited by local optimal solution, their convergences are not always assured. In the study, a model-based damage-identification method considering temperature variations, comprised of particle swarm optimization and cuckoo search, is implemented to detect structural damage. First, to eliminate the influence of environmental temperature, temperature change is considered as a parameter of structural material elastic modulus. A function relationship is established between environmental temperature and the material elastic modulus, and an objective function composed of natural frequency, mode shape and modal strain energy with different weight coefficients is constructed. Second, the hybrid optimization algorithm, a combination of particle swarm optimization and cuckoo search, is proposed. Third, to solve the problem of optimization algorithm convergence, the optimization performance of the hybrid optimization algorithm is validated by utilizing four benchmark functions, and it is found that the performance of the hybrid optimization algorithm is the best. In order to test the performance of the three algorithms in damage identification, a numerical simply supported beam is adopted. The results show that the hybrid optimization algorithm can identify the damage location and severity under four different damage cases without considering temperature variations and two cases considering temperature variations. Finally, the hybrid optimization algorithm is introduced to test the damage-identification performance of I-40 Bridge, an actual steel–concrete composite bridge under temperature variations, whose results show that the hybrid optimization algorithm can preferably distinguish between real damages and temperature effects (temperature gradient included); its good robustness and engineering applicability are validated.

On the Powerball Method: Variants of Descent Methods for Accelerated Optimization

We propose a new method to accelerate the convergence of optimization algorithms. This method simply adds a power coefficient $\gamma\in[0,1)$ to the gradient during optimization. We call this the Powerball method and analyze the convergence rate for the Powerball method for strongly convex functions. While theoretically the Powerball method is guaranteed to have a linear convergence rate in the same order of the gradient method, we show that empirically it significantly outperforms the gradient descent and Newton's method, especially during the initial iterations. We demonstrate that the Powerball method provides a $10$-fold speedup of the convergence of both gradient descent and L-BFGS on multiple real datasets.

Improved Dynamic Performance and Hierarchical Energy Management of Microgrids With Energy Routing

In this paper, a hierarchical distributed energy management of multimicrogrids (MMGs) with energy routing is proposed. Existing control strategies for power sharing, transient performance, and economic-emission dispatch in microgrids with distributed generators (DGs) fall short in providing good dynamic performance. To address this issue, a hierarchical distributed optimization is proposed by using top–down approach, which decomposes original economic-emission dispatch of MMG scenario into individual microgrid (MG) and energy routing subproblems. Distributed electric vehicle charging, intermittent photovoltaic source, and battery energy storage system are incorporated in the optimization model. Using multiagent system model for DG, a dynamic performance controller (DPC) is proposed for each MG to achieve improved performance during transients. Convergence of optimization algorithm is proved using Lyapunov theory. Performance evaluation results show that the proposed DPC for economic-emission dispatch improves system performance significantly during either load or generator switching.

Multiple Kernel Clustering With Global and Local Structure Alignment

Multiple kernel clustering (MKC) based on global structure alignment (GSA) has unified many existing MKC algorithms, and shown outstanding clustering performance. However, we observe that most of existing GSA-based MKC algorithms only maximally align global structure of data with an ideal similarity matrix, while ignoring the local geometrical structure hidden in data, which is regarded to be important in improving the clustering performance. To address this issue, we propose a global and local structure alignment framework for MKC (GLSAMKC) which well considers both the alignment between the global structure and local structure of data with the same ideal similarity matrix. To illustrate the effectiveness of the proposed framework, we instantiate two specific GLSAMKC-based algorithms by exploiting the local structure with local linear embedding and locality preserving projection, respectively. A two-step alternate iterative and convergent optimization algorithm is developed to implement the resultant optimization problem. Extensive experimental results on five benchmark data sets demonstrate the superiority of proposed algorithms compared with the many state-of-the-art MKC algorithms, indicating the effectiveness of the proposed framework.