Non-differentiable Objective Functions Research Articles

Cellular gradient algorithm for solving complex mechanical optimization design problems

In mechanical optimization design problems, there are often some non-continuous or non-differentiable objective functions. For these non-continuous and non-differentiable optimization objectives, it is often difficult for existing optimal design algorithms to find the desired optimal solutions. In this paper, we incorporate the idea of gradient descent into cellular automata and propose a Cellular Gradient (CG) method. First, we have given the basic rules and algorithmic framework of CG and designed three kinds of growth and extinction rules respectively. Then, the three evolutionary rules for cellular within a single cycle are analyzed separately for form and ordering. The best expressions for the cellular jealous neighbor rule and the solitary regeneration rule are given, and the most appropriate order in which the rules are run is selected. Finally, the solution results of the cellular gradient algorithm and other classical optimization design algorithms are compared with a multi-objective multi-parameter mechanical optimization design problem as an example. The computational results show that the cellular gradient algorithm has an advantage over other algorithms in solving global and dynamic mechanical optimal design problems. The novelty of CG is to provide a new way of thinking for solving optimization problems with global discontinuities.

A novel interval estimation framework for wind power forecasting using multi-objective gradient descent optimization

Effective wind power forecasting enables reliable integration of renewable energy into the electric grid and enhances wind farm management and power system dispatch. To mitigate uncertainty in wind power forecasting, interval prediction is commonly used to provide information on confidence levels. The lower upper bound estimation (LUBE) method is a quality-driven solution that constructs prediction intervals (PIs) based on an ensemble loss function comprising two PI quality metrics without assuming any specific distribution function. However, the conventional LUBE method suffers from three major drawbacks: 1) it is incompatible with gradient descent (GD) due to its non-differentiable objective function; 2) its objective function is formulated based on qualitative assessment rather than on a statistical basis; and 3) the ensemble loss function cannot fully represent the PI quality information measured by two conflicting metrics, leading to potential performance loss. In this study, we propose a novel multi-objective gradient descent (MOGD)-based LUBE model to address these drawbacks. The loss function is derived based on the maximum likelihood estimation and is approximated by a sigmoid function to ensure its statistical basis and differentiable form. In the training process, MOGD is applied to search for the direction of GD subject to both PI metrics. The proposed model is compared with several benchmarks using four wind power datasets from different spatial locations. The numerical results indicate the superiority of the proposed MOGD-LUBE model over its counterparts in terms of the reliability and sharpness for efficient wind power forecasting.

Duality Method for Multidimensional Nonsmooth Constrained Linear Convex Stochastic Control

In this paper, we discuss a general multidimensional linear convex stochastic control problem with nondifferentiable objective function, control constraints, and random coefficients. We formulate an equivalent dual problem, prove the dual stochastic maximum principle and the relation of the optimal control, optimal state, and adjoint processes between primal and dual problems, and illustrate the usefulness of the dual approach with some examples.

РОЗВ’ЯЗАННЯ ЗАДАЧ ОПТИМІЗАЦІЇ В ЕНЕРГЕТИЦІ ЗА ДОПОМОГОЮ ГЕНЕТИЧНОГО АЛГОРИТМУ

The article discusses the application of genetic algorithms in the field of energy optimization. Linear programming is commonly used for optimization problems in energy systems. Linear programming is a mathematical optimization method that seeks the optimal solution under constraints, where all constraints and the objective function are linear functions. In the realm of artificial intelligence,genetic algorithms are employed for optimization tasks. genetic algorithms mimic natural evolution processes, including selection, crossover, mutation, and adaptation, to solve optimization and search problems. The article outlines the process of a genetic algorithm, starting with the formation of an initial population and proceeding through selection, crossover, mutation, and evaluation. This cycle repeats until an optimal solution is achieved. Advantages of genetic algorithms include their ability to handle complex solution spaces, find global optima, adapt to changing conditions, optimize multi-objective functions, and work with non-linear and non-differentiable objective functions. However, they may require significant computational resources and parameter tuning. The article then presents a case study of applying a genetic algorithm to optimize the allocation of a power load in an energy system. The mathematical model is developed, and the simplex method is initially used for solution. Subsequently, a Python program for genetic algorithm implementation is provided. The algorithm's efficiency and convergence are demonstrated through a graphical representation of the optimization process. In conclusion, the article highlights the effectiveness of genetic algorithms in energy optimization, showcasing their rapid convergence and ability to find near-optimal solutions in complex scenarios.

A Modified Simulated Annealing (MSA) Algorithm to Solve the Supplier Selection and Order Quantity Allocation Problem with Non-Linear Freight Rates

Economic Order Quantity (EOQ) is an important optimization problem for inventory management with an impact on various industries; however, their mathematical models may be complex with non-convex, non-linear, and non-differentiable objective functions. Metaheuristic algorithms have emerged as powerful tools for solving complex optimization problems (including EOQ). They are iterative search techniques that can efficiently explore large solution spaces and obtain near-optimal solutions. Simulated Annealing (SA) is a widely used metaheuristic method able to avoid local suboptimal solutions. The traditional SA algorithm is based on a single agent, which may result in a low convergence rate for complex problems. This article proposes a modified multiple-agent (population-based) adaptive SA algorithm; the adaptive algorithm imposes a slight attraction of all agents to the current best solution. As a proof of concept, the proposed algorithm was tested on a particular EOQ problem (recently studied in the literature and interesting by itself) in which the objective function is non-linear, non-convex, and non-differentiable. With these new mechanisms, the algorithm allows for the exploration of different regions of the solution space and determines the global optimum in a faster manner. The analysis showed that the proposed algorithm performed well in finding good solutions in a reasonably short amount of time.

An Extensive Study Using the Beetle Swarm Method to Optimize Single and Multiple Objectives of Various Optimal Power Flow Problems

An electric energy generation system, under the economic operation mode, is an imperative mission in the power system function. This article deals with the use of beetle swarm optimization algorithm (BSOA), for optimal power flow (OPF) solution, in an effective approach. BSOA is a competent optimization technique, to handle multimodal, nonlinear, and nondifferentiable objective functions. The proposed OPF is modeled by numerous objective functions, formulations with constraints, examined with thirty-one different cases, on the three distinguished test systems (IEEE 30, 57, and 118-bus), using single and weighted sum multiobjectives. Six new multiobjective cases are also studied. The control variables, such as real generation of power, tap setting ratio of transformers, bus voltages magnitudes, and the values of shunt capacitor, are also optimized. Potency and robustness of this proposed method were investigated and evaluated with more recent findings reported in the literature. This extensive study revealed the preeminence of the presented technique, applied to OPF problem, with intricate and nonsmooth objective functions.

Biomechanical Trajectory Optimization of Human Sit-to-Stand Motion With Stochastic Motion Planning Framework

Trajectory optimization has been an important approach in biomechanics for the analysis and prediction of the limb movement. Such approaches have paved the way for the motion planning of biped and quadruped robots as well. Most of these methods are deterministic, utilizing first-order iterative gradient-based algorithms incorporating the constrained differentiable objective functions. However, the limitation of prevailing methods concerning differentiability hinders the implementation of non-differentiable objective functions such as metabolic energy expenditure (MEE) function, which is highly relevant for physiological systems and can even be implemented across the muscular space. This paper consolidates the implementation of the prevalent direct collocation-based optimal control method with the stochastic trajectory optimization method based on Policy Improvement with Path Integral (PI2) for comprehending the human sit-to-stand (STS) motion. PI2 method, which utilizes reinforcement learning of Dynamic Movement Primitive (DMP) to learn a goal-based trajectory is implemented and validated by comparing with the experimental result in joint-space and muscle-space.

A Joint estimation approach to sparse additive ordinary differential equations.

Ordinary differential equations (ODEs) are widely used to characterize the dynamics of complex systems in real applications. In this article, we propose a novel joint estimation approach for generalized sparse additive ODEs where observations are allowed to be non-Gaussian. The new method is unified with existing collocation methods by considering the likelihood, ODE fidelity and sparse regularization simultaneously. We design a block coordinate descent algorithm for optimizing the non-convex and non-differentiable objective function. The global convergence of the algorithm is established. The simulation study and two applications demonstrate the superior performance of the proposed method in estimation and improved performance of identifying the sparse structure.

A Novel Range-Free Node Localization Method for Wireless Sensor Networks

Solving the nonconvex and non-differentiable objective function of traditional range-free node localization methods will result in low localization accuracy or high computational complexity. To this end, a novel iterative localization algorithm called CVX-DV-hop is proposed in this letter. It first performs matrix transformation to reformulate the original optimization problem into one with a convex function and nonconvex constraints. And then, the first-order Taylor expansions are employed to tighten the nonconvex constraints into linear inequality constraints. Finally, a successive convex approximation method is designed to iteratively solve the optimization problem. Simulation results show that the proposed algorithm has higher localization accuracy and lower computational burden than those of the particle swarm optimization algorithm.

Selecting Some Variables to Update-Based Algorithm for Solving Optimization Problems.

With the advancement of science and technology, new complex optimization problems have emerged, and the achievement of optimal solutions has become increasingly important. Many of these problems have features and difficulties such as non-convex, nonlinear, discrete search space, and a non-differentiable objective function. Achieving the optimal solution to such problems has become a major challenge. To address this challenge and provide a solution to deal with the complexities and difficulties of optimization applications, a new stochastic-based optimization algorithm is proposed in this study. Optimization algorithms are a type of stochastic approach for addressing optimization issues that use random scanning of the search space to produce quasi-optimal answers. The Selecting Some Variables to Update-Based Algorithm (SSVUBA) is a new optimization algorithm developed in this study to handle optimization issues in various fields. The suggested algorithm’s key principles are to make better use of the information provided by different members of the population and to adjust the number of variables used to update the algorithm population during the iterations of the algorithm. The theory of the proposed SSVUBA is described, and then its mathematical model is offered for use in solving optimization issues. Fifty-three objective functions, including unimodal, multimodal, and CEC 2017 test functions, are utilized to assess the ability and usefulness of the proposed SSVUBA in addressing optimization issues. SSVUBA’s performance in optimizing real-world applications is evaluated on four engineering design issues. Furthermore, the performance of SSVUBA in optimization was compared to the performance of eight well-known algorithms to further evaluate its quality. The simulation results reveal that the proposed SSVUBA has a significant ability to handle various optimization issues and that it outperforms other competitor algorithms by giving appropriate quasi-optimal solutions that are closer to the global optima.

Safe learning-based gradient-free model predictive control based on cross-entropy method

In this paper, a safe and learning-based control framework for model predictive control (MPC) is proposed to optimize nonlinear systems with a non-differentiable objective function under uncertain environmental disturbances. The control framework integrates a learning-based MPC with an auxiliary controller in a way of minimal intervention. The learning-based MPC augments the prior nominal model with incremental Gaussian Processes to learn the uncertain disturbances. The cross-entropy method (CEM) is utilized as the sampling-based optimizer for the MPC with a non-differentiable objective function. A minimal intervention controller is devised with a control Lyapunov function and a control barrier function to guide the sampling process and endow the system with high probabilistic safety. The proposed algorithm shows a safe and adaptive control performance on a simulated quadrotor in the tasks of trajectory tracking and obstacle avoidance under uncertain wind disturbances.

Optimization and learning with nonlocal calculus

<p style='text-indent:20px;'>Nonlocal models have recently had a major impact in nonlinear continuum mechanics and are used to describe physical systems/processes which cannot be accurately described by classical, calculus based "local" approaches. In part, this is due to their multiscale nature that enables aggregation of micro-level behavior to obtain a macro-level description of singular/irregular phenomena such as peridynamics, crack propagation, anomalous diffusion and transport phenomena. At the core of these models are <i>nonlocal</i> differential operators, including nonlocal analogs of the gradient/Hessian. This paper initiates the use of such nonlocal operators in the context of optimization and learning. We define and analyze the convergence properties of nonlocal analogs of (stochastic) gradient descent and Newton's method on Euclidean spaces. Our results indicate that as the nonlocal interactions become less noticeable, the optima corresponding to nonlocal optimization converge to the "usual" optima. At the same time, we argue that nonlocal learning is possible in situations where standard calculus fails. As a stylized numerical example of this, we consider the problem of non-differentiable parameter estimation on a non-smooth translation manifold and show that our <i>nonlocal</i> gradient descent recovers the unknown translation parameter from a non-differentiable objective function.</p>

A Method for Completing Economic Load Dispatch Using the Technique of Narrowing Down Area

Economic load dispatch solutions based on published methods, both conventional and artificial, have been very well-formulated through point-to-point movement methodologies to reach a convergence point. Iteration always starts from the starting point to obtain the following solution point, leading to the convergence point. This paper presents a new method to solve economic load dispatch problems by narrowing the minimum and maximum power limits between generator units. This idea approximates the solution point with a tiny space formed by the very narrow power limits of each generator. The methodology used is the distance between the minimum and maximum power limits of each generator divided into several segments. Then, the best segment is determined by the minimum total cost calculated based on the center point of the segment. Continue to the following iteration process until the best segment is the smallest. This iteration process is another artificial method that works without calculus calculations, so it does not depend on the objective function. This method has been validated using two generator units with differentiable objective functions, with calculation accuracy less than 0.00001 MW of the power distance of the generator limit, and the iteration stops at the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$23^{\mathrm {rd}}$ </tex-math></inline-formula> step. Furthermore, this method has been successfully applied to the nondifferentiable objective function, piecewise and valve point effects.

Learning per-machine linear dispatching rule for heterogeneous multi-machines control

ABSTRACT This paper proposes a per-machine linear dispatching rule learning approach to improve the scheduling of re-entrant flow shop such as semiconductor fab. Finding an optimal schedule of a complex manufacturing system is intractable; hence, a dispatching rule as a heuristic approach is widely used in actual practice. Also, to develop a good dispatching rule, an automated methodology for developing heuristics, also known as a hyper-heuristic, has been studied extensively. However, most of the literature has focused on finding a single-sophisticated dispatching rule, in which every machine uses the same rule. Such an approach often shows suboptimal performance when the optimal dispatching rule is different on each machine. To solve this problem, we introduce a simple and effective per-machine dispatching rule learning approach, in which each machine has one linear dispatching rule that is optimised by the Gradient-based Evolutionary Strategy (GES). This method is sample-efficient and can be applied to non-differentiable objective functions such as average Cycle Time. The proposed approach was mainly compared to two popular methods based on Genetic Programming (GP) and the Genetic Algorithm (GA) on a four-station and eight-machine re-entrant flow shop. Numerical results show that the proposed approach outperforms widely used methods.

On generalization of E-convex and composite functions

In this paper the concepts of some generalizations of E-convexity such as logarithmic, and exponential E-convexity are introduced for real-valued functions defined on a Banach space and their relationships with known concepts have been discussed. Further, some applications to optimization of non-differentiable objective functions have been proposed. Also, it is proved that the composition of an affine and a G-convex functional is G-convex, where G stands for convex, quasi-convex, r-quasi convex and m-quasi convex.

Subgradient averaging for multi-agent optimisation with different constraint sets

We consider a multi-agent setting with agents exchanging information over a possibly time-varying network, aiming at minimising a separable objective function subject to constraints. To achieve this objective we propose a novel subgradient averaging algorithm that allows for non-differentiable objective functions and different constraint sets per agent. Allowing different constraints per agent simultaneously with a time-varying communication network constitutes a distinctive feature of our approach, extending existing results on distributed subgradient methods. To highlight the necessity of dealing with a different constraint set within a distributed optimisation context, we analyse a problem instance where an existing algorithm does not exhibit a convergent behaviour if adapted to account for different constraint sets. For our proposed iterative scheme we show asymptotic convergence of the iterates to a minimum of the underlying optimisation problem for step sizes of the form ηk+1, η>0. We also analyse this scheme under a step size choice of ηk+1, η>0, and establish a convergence rate of O(lnkk) in objective value. To demonstrate the efficacy of the proposed method, we investigate a robust regression problem and an ℓ2 regression problem with regularisation.

Comparison between optimal configuration algorithms of fiber phased array

Optical fiber phased array can be used in high-power laser beam combination, lidar and other areas. The configuration of the optical fiber array is different from the microwave phased array, which has periodic problems that affect the energy intensity distribution of the main lobe. Starting from the physical model, in this paper we establish a theoretical model of optical phased array antenna array based on a set of concentric circular ring lattices, and propose a theory of the rapid synthesis of randomly configured interference field strengths through using analytical continuation method and Fourier transform method. The problem of sampling bandwidth and sampling number that should be paid attention to in the numerical simulation of discrete sampling are discussed, and the problem of quickly realizing the numerical simulation of multi-beam interference field is solved. Genetic algorithm and particle swarm algorithm for optimizing the configuration of optical phased array antennas are investigated with different populations. The convergence speeds and optimization efficiencies of the two algorithms are compared and analyzed. It is demonstrated that the peak side-lobe ratio PSR can be achieved to be better than 0.270 by the genetic algorithm optimized configuration array under the real fabricate parameter. The proposed method is expected to be used in the actual optical phased array antenna configuration to guide the optimal design of the antenna with low side lobes, and the proposed model is also expected to provide a certain reference value for the study of optimizing the non-differentiable objective function.

Novel strategies for modal-based structural material identification

In this work, we present modal-based methods for model calibration in structural dynamics, and address several key challenges in the solution of gradient-based optimization problems with eigenvalues and eigenvectors, including the solution of singular Helmholtz problems encountered in sensitivity calculations, non-differentiable objective functions caused by mode swapping during optimization, and cases with repeated eigenvalues. Unlike previous literature that relied on direct solution of the eigenvector adjoint equations, we present a parallel iterative domain decomposition strategy (Adjoint Computation via Modal Superposition with Truncation Augmentation) for the solution of the singular Helmholtz problems. For problems with repeated eigenvalues we present a novel Mode Separation via Projection algorithm, and in order to address mode swapping between inverse iterations we present a novel Injective mode ordering metric. We present the implementation of these methods in a massively parallel finite element framework with the ability to use measured modal data to extract unknown structural model parameters from large complex problems. A series of increasingly complex numerical examples are presented that demonstrate the implementation and performance of the methods in a massively parallel finite element framework [7,5], using gradient-based optimization techniques in the Rapid Optimization Library (ROL) [21].

Robust MIMO radar target localization based on lagrange programming neural network

In a multiple-input multiple-output (MIMO) radar system, there are a number of transmitters and receivers. We can use a set of range measurements from MIMO system to locate a target. Each range measurement is the sum of the transmitter-to-target distance and target-to-receiver distance, which corresponds to elliptic localization. This paper addresses the MIMO radar target localization problem with possibly outlier measurements. We formulate the problem via non-smooth constrained optimization with an ℓ1-norm objective function, which is non-differentiable, and the Lagrange programming neural network (LPNN) is adopted as the solver. As the LPNN framework cannot handle non-differentiable objective functions, we utilize two techniques, namely, approximation of the ℓ1-norm and locally competitive algorithm, to develop two LPNN based algorithms. Moreover, the stability of the LPNN-based algorithms is studied. Simulation results demonstrate that the proposed algorithms outperform two state-of-the-art algorithms.

Optimization-Based Decentralized Coded Caching for Files and Caches With Arbitrary Sizes

Existing decentralized coded caching solutions cannot guarantee small loads in the general scenario with arbitrary file sizes and cache sizes. In this paper, we propose an optimization framework for decentralized coded caching in the general scenario to minimize the worst-case load and average load (under an arbitrary file popularity), respectively. Specifically, we first propose a class of decentralized coded caching schemes for the general scenario, which are specified by a general caching parameter and include several known schemes as special cases. Then, we optimize the caching parameter to minimize the worst-case load and average load, respectively. Each of the two optimization problems is a challenging nonconvex problem with a nondifferentiable objective function. For each optimization problem, we develop an iterative algorithm to obtain a stationary point using techniques for solving Complementary Geometric Programming (GP). We also obtain a low-complexity approximate solution by solving an approximate problem with a differentiable objective function which is an upper bound on the original nondifferentiable one, and characterize the performance loss caused by the approximation. Finally, we present two information-theoretic converse bounds on the worst-case load and average load (under an arbitrary file popularity) in the general scenario, respectively. To the best of our knowledge, this is the first work that provides optimization-based decentralized coded caching schemes and information-theoretic converse bounds for the general scenario.