Original Optimization Problem Research Articles

A new approach for solving the multidimensional control optimization problems with first‐order partial differential equations constraints

AbstractThe purpose of this paper is to provide the linearization technique to solve the multidimensional control optimization problem (MCOP) involving first‐order partial differential equation (PDEs) constraints. Firstly, we use the modified objective function approach for simplifying the aforesaid extremum problem (MCOP) and show that the solution sets of the original control optimization problem and its modified control optimization problem (MCOP) are equivalent under convexity assumptions. Further, we use the absolute value exact penalty function method to transform (MCOP) into a penalized control problem (MCOP) . Then, we establish the equivalence between a minimizer of the modified penalized optimization problem (MCOP) and a saddle point of the Lagrangian defined for the modified optimization problem (MCOP) under appropriate convexity hypotheses. Moreover, the results established in the paper are illustrated by some examples of MCOPs involving first‐order PDEs constraints.

State Space Emulation and Annealed Sequential Monte Carlo for High Dimensional Optimization

Many high dimensional optimization problems can be reformulated into a problem of finding theoptimal state path under an equivalent state space model setting. In this article, we present a general emulation strategy for developing a state space model whose likelihood function (or posterior distribution) shares the same general landscape as the objective function of the original optimization problem. Then the solution of the optimization problem is the same as the optimal state path that maximizes the likelihood function or the posterior distribution under the emulated system. To find such an optimal path, we adapt a simulated annealing approach by inserting a temperature control into the emulated dynamic system and propose a novel annealed Sequential Monte Carlo (SMC) method that effectively generating Monte Carlo sample paths utilizing samples obtained on the high temperature scale. Compared to the vanilla simulated annealing implementation, annealed SMC is an iterative algorithm for state space model optimization that directly generates state paths from the equilibrium distributions with a decreasing sequence of temperatures through sequential importance sampling which does not require burn-in or mixing iterations to ensure quasi-equilibrium condition. Several applications of state space model emulation and the corresponding annealed SMC results are demonstrated.

On the Use of DELEqC-III in Bilevel Problems with Linear Equality Constraints

The bilevel programming problem (BLP) is an optimization problem with another optimization problem in its constraints. This framework finds utility in modeling decentralized scenarios, which arise in real-world applications such as traffic management, transportation, and economic policy. Differential Evolution (DE) techniques have emerged in literature for addressing such complex problems. However, handling linear equality constraints poses a significant challenge for DE and other metaheuristics. To address this issue, we previously introduced DELEqC, enhancing DE with a mechanism to manipulate the linear equality constraints. A specialized variant, BL-DELEqC, was further proposed specifically for tackling general BLPs. Another variant, DELEqC-III, transforms the original constrained optimization problem into a lower-dimensional unconstrained one, offering applicability to BLPs with linear equality constraints. Thus, we explore in this study the efficacy of DELEqC-III in handling BLPs with linear equality constraints. The proposed BL-DELEqC-III is compared to BL-DELEqC on a selection of benchmark BLPs, demonstrating superior results.

A Memory-Efficient Federated Kernel Support Vector Machine for Edge Devices.

A federated learning (FL) scheme (denoted as Fed-KSVM) is designed to train kernel support vector machines (SVMs) over multiple edge devices with low memory consumption. To decompose the training process of kernel SVM, each edge device first constructs high-dimensional random feature vectors of its local data, and then trains a local SVM model over the random feature vectors. To reduce the memory consumption on each edge device, the optimization problem of the local model is divided into several subproblems. Each subproblem only optimizes a subset of the model parameters over a block of random feature vectors with a low dimension. To achieve the same optimal solution to the original optimization problem, an incremental learning algorithm called block boosting is designed to solve these subproblems sequentially. After training of the local models, the central server constructs a global SVM model by averaging the model parameters of these local models. Fed-KSVM only increases the iterations of training the local SVM models to save the memory consumption, while the communication rounds between the edge devices and the central server are not affected. Theoretical analysis shows that the kernel SVM model trained by Fed-KSVM converges to the optimal model with a linear convergence rate. Because of such a fast convergence rate, Fed-KSVM reduces the communication cost during training by up to 99% compared with the centralized training method. The experimental results also show that Fed-KSVM reduces the memory consumption on the edge devices by nearly 90% while achieving the highest test accuracy, compared with the state-of-the-art schemes.

A Majorization-Minimization Gauss-Newton Method for 1-Bit Matrix Completion

In 1-bit matrix completion, the aim is to estimate an underlying low-rank matrix from a partial set of binary observations. We propose a novel method for 1-bit matrix completion called Majorization-Minimization Gauss-Newton (MMGN). Our method is based on the majorization-minimization principle, which converts the original optimization problem into a sequence of standard low-rank matrix completion problems. We solve each of these sub-problems by a factorization approach that explicitly enforces the assumed low-rank structure and then apply a Gauss-Newton method. Using simulations and a real data example, we illustrate that in comparison to existing 1-bit matrix completion methods, MMGN outputs comparable if not more accurate estimates. In addition, it is often significantly faster, and less sensitive to the spikiness of the underlying matrix. In comparison with three standard generic optimization approaches that directly minimize the original objective, MMGN also exhibits a clear computational advantage, especially when the fraction of observed entries is small.

Three six-degree-of-freedom dynamic descent convex optimization guidance methods

A key technology for soft landing of powered descent of aircraft is to solve the six-degree-of-freedom powered descent guidance problem in real time, which can be described as the fuel-saving trajectory optimization problem under multiple constraints. Three optimization models are established by selecting three independent variables: flight time, time substitution variable, and trajectory height. The original trajectory optimization problem is transformed into a solvable form of sequential convex optimization for iterative solution, forming three online guidance methods. The three guidance methods are compared in terms of convergence, real-time performance, optimality, and solution accuracy. The results show that all three guidance methods can solve the six-degree-of-freedom powered descent problem. The guidance method with flight time as the independent variable has the shortest calculation time and the least fuel consumption, but the powered descent flight time needs to be determined in advance. The guidance methods based on the other two types of independent variables can optimize the powered descent flight time, but they are all suboptimal solutions, and the calculation time is significantly increased. The solution accuracy of the three methods is similar under the same number of discrete points. If sequential convex optimization is used as the online guidance scheme for powered descent, how to determine the optimal flight time, approach the optimal solution for fuel, and further shorten the calculation time still needs to be further studied.

Optimal resource allocation for rapid convergence to stable healthy state in epidemic spreading models

The networked Susceptible-Infected-Susceptible (SIS) model has been widely investigated as a model for the spread of epidemics within networked systems. In networks with SIS dynamics and unstable healthy states, a critical question is how to distribute compensatory curing resources (with constrained total cost) among individuals, ensuring that the network converges to a healthy state as fast as possible. This paper introduces a novel approach to this problem by developing an algorithm for the optimal allocation of compensatory curing resources required by agents in a given network. This solution approach has been made possible by reformulating the original convergence rate optimization problem and an additional constraint guaranteeing a minimum convergence rate as a standard semidefinite programming problem. The applicability of the proposed algorithm to arbitrary undirected topologies and other variations of the SIS model, including the one with a weighted cost function, has been demonstrated. In the case of symmetry preserving curing and infection rates, it has been shown that the problem over a given network with a symmetric topology can be reduced to a smaller network with each orbit acting as a node. Additionally, for networks with one or two orbits, the problem has been addressed analytically, and several examples have been included. Based on two different scenarios inspired by the SARS outbreak in Hong Kong and the COVID-19 outbreak in the USA, it is shown that the algorithm's optimal results outperform the uniform distribution of additional curing resources. The paper also explores how optimal performance metrics change with the given upper limit on the total amount of additional curing resources.

Vector optimization problems in near vector space

ABSTRACT The conventional vector optimization problems intend to optimize the objective functions, which are defined on a vector space into another vector space. In this paper, we are going to study the vector optimization problems by minimizing the objective functions, which are defined on a vector space into another so-called near vector space. The concept of near vector space is axiomatically defined, and it does not necessarily contain the additive inverse element. A so-called pointed-like convex cone is introduced to propose the concept of minimizer of this new type vector optimization problem. Its corresponding scalar optimization problem is also formulated. Finally, we show that the optimal solution of the corresponding scalar optimization problem is also a minimizer of the original vector optimization problem based on the near vector space.

A Wasserstein perspective of Vanilla GANs

The empirical success of Generative Adversarial Networks (GANs) caused an increasing interest in theoretical research. The statistical literature is mainly focused on Wasserstein GANs and generalizations thereof, which especially allow for good dimension reduction properties. Statistical results for Vanilla GANs, the original optimization problem, are still rather limited and require assumptions such as smooth activation functions and equal dimensions of the latent space and the ambient space. To bridge this gap, we draw a connection from Vanilla GANs to the Wasserstein distance. By doing so, existing results for Wasserstein GANs can be extended to Vanilla GANs. In particular, we obtain an oracle inequality for Vanilla GANs in Wasserstein distance. The assumptions of this oracle inequality are designed to be satisfied by network architectures commonly used in practice, such as feedforward ReLU networks. By providing a quantitative result for the approximation of a Lipschitz function by a feedforward ReLU network with bounded Hölder norm, we conclude a rate of convergence for Vanilla GANs as well as Wasserstein GANs as estimators of the unknown probability distribution.

A distributionally robust optimization approach of multi-park integrated energy systems considering shared energy storage and Uncertainty of Demand Response

To enhance the economic efficiency and renewable energy integration capacity of multi-park integrated energy systems (MPIES) and address the issue of insufficient consideration of demand response uncertainty in existing studies, this paper proposes a distributionally robust optimization approach for multi-park integrated energy systems, considering shared energy storage and the uncertainty of demand response. First, models of the shared energy system and demand response are established. Based on these models, a deterministic optimization scheduling model for MPIES is developed, aiming to minimize system costs while considering constraints such as grid power balance. To address uncertainty in demand response, this paper employs Interval Type-2 Fuzzy Theory to construct uncertainty sets and considers system reliability constraints. The original optimization problem is then transformed into an equivalent robust counterpart model, and the optimal distributionally robust solution is obtained through parameter domain decomposition methods. Finally, the proposed method is validated using the IEEE 33-bus system. The results show that considering shared energy storage and demand response individually can reduce total system costs by 4.86 % and 26.46 %, respectively. After accounting for the uncertainty in demand response, the total system cost increases only slightly by 4.36 %, but this improves the system's robustness.

HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search.

Recent neural architecture search (NAS)-based approaches have made great progress in the hyperspectral image (HSI) classification tasks. However, the architectures are usually optimized independently of the network weights, increasing searching time, and restricting model performances. To tackle these issues, in this article, different from previous methods that extra define structural parameters, we propose to directly generate structural parameters by utilizing the specifically designed hyper kernels, ingeniously converting the original complex dual optimization problem into easily implemented one-tier optimizations, and greatly shrinking searching costs. Then, we develop a hierarchical multimodule search space whose candidate operations only contain convolutions, and these operations can be integrated into unified kernels. Using the above searching strategy and searching space, we obtain three kinds of networks to separately conduct pixel-level or image-level classifications with 1-D or 3-D convolutions. In addition, by combining the proposed hyper kernel searching scheme with the 3-D convolution decomposition mechanism, we obtain diverse architectures to simulate 3-D convolutions, greatly improving network flexibilities. A series of quantitative and qualitative experiments on six public datasets demonstrate that the proposed methods achieve state-of-the-art results compared with other advanced NAS-based HSI classification approaches.

Olfactory-based powered descent guidance for Mars methane plume exploration in long-time-average wind environment

This paper investigates an olfactory-based powered descent guidance method that enable a lander to autonomously locate and target the vent source of any methane plume in long-time-average wind environment on Mars. The episodic methane plumes emanating from the Martian surface invite the possibility of direct access to the subsurface methane reservoir to acquire pristine organic materials. Constrained by the insufficient knowledge of the accurate plume vent location, Mars landers must infer the target location — the plume source — autonomously during the powered descent. However, existing powered descent guidance methods require the target location as a terminal constraint, rendering them ineffective in plume exploration missions. We propose an olfactory-based powered descent guidance method that could steer a lander to target the methane vent source by tracking the gradient of methane concentration in long-time-average wind environment, imposing sub-optimal fuel consumption. A novel olfactory navigation method based on tracking the negative logarithmic concentration gradient, and the corresponding gradient measurement method, is proposed. By implementing the negative logarithmic gradient field, gradient-dependent dynamic equations for the powered descending lander are proposed. The gradient-dependent equations provide a way to transform the original optimal guidance problem, which is non-convex and terminal-free, into a convex and terminal-fixed problem. A suboptimal guidance law, similar to E-guidance, is then derived by solving this problem. The proposed guidance could target the vent source of any methane plume in long-time-average wind environment, imposing modest fuel consumption by feeding back the concentration gradient. Numerical simulations illustrate that, in comparison to the terminal-fixed E-guidance and the open-loop fuel-optimal guidance, the lander imposes additional fuel consumption of 2.57% and 12.15% respectively, using the proposed guidance law.

Synergistic optimization strategy for beamforming and power allocation in dual-functional radar-communication systems

The dual-functional radar-communication (DFRC) integrated system presents an ideal solution to address the challenge of spectrum resource congestion in future networks. This paper explores an adaptive power allocation technique based on beamforming to enhance the word error probability (WEP) performance of the DFRC system. Initially, a joint optimization model is developed to minimize the WEP while adhering to constraints on radar signal-to-interference-noise ratio (SINR), peak-to-average-power ratio, sidelobe level, and total transmit power. This model incorporates dual-function transmit beam, radar, and communication receive beam patterns. Subsequently, the proposed subproblem convex relaxation alternating update (SCRAU) algorithm is introduced to achieve a locally optimal solution for multi-carrier power allocation. This algorithm decomposes the original non-convex optimization problem into three sub-problems with lower complexity and iteratively optimizes them. Simulation experiments validate that the SCRAU algorithm can simultaneously fulfill radar and communication functions. The SCRAU algorithm demonstrates superior WEP performance compared to current advanced algorithms.

A novel set-induced robust optimization model under correlated uncertainty: a comparative theoretical and computational study

In practical scenarios, optimization problems frequently involve uncertain parameters that exhibit correlated fluctuations. The widely adopted set-induced robust optimization approach is employed to address correlated uncertainty in environments with limited data. This approach focuses on deriving a deterministic reformulation of the original uncertain optimization problem, ensuring that the constructed solution remains feasible for all potential realizations of input parameters within a predefined uncertainty set. However, this approach often yields overly conservative solutions. In this study, an innovative uncertainty set is developed to mitigate the over-conservatism associated with the set-induced robust optimization approach under correlated uncertainty. Moreover, from the proposed description of data uncertainty, a robust linear reformulation of the original uncertain optimization problem is established to strike a balance between conservativeness and optimality. Computational studies exploring production planning optimization programs demonstrate the superior performance of the proposed robust model over other alternatives, considering various levels of correlation among uncertain parameters.

Edge-Intelligence-Powered Joint Computation Offloading and Unmanned Aerial Vehicle Trajectory Optimization Strategy

UAV-based air-ground integrated networks offer a significant benefit in terms of providing ubiquitous communications and computing services for Internet of Things (IoT) devices. With the empowerment of edge intelligence (EI) technology, they can efficiently deploy various intelligent IoT applications. However, the trajectory of UAVs can significantly affect the quality of service (QoS) and resource optimization decisions. Joint computation offloading and UAV trajectory optimization bring many challenges, including coupled decision variables, information uncertainty, and long-term queue delay constraints. Therefore, this paper introduces an air-ground integrated architecture with EI and proposes a TD3-based joint computation offloading and UAV trajectory optimization (TCOTO) algorithm. Specifically, we use the principle of the TD3 algorithm to transform the original problem into a cumulative reward maximization problem in deep reinforcement learning (DRL) to obtain the UAV trajectory and offloading strategy. Additionally, the Lyapunov framework is used to convert the original long-term optimization problem into a deterministic short-term time-slot problem to ensure the long-term stability of the UAV queue. Based on the simulation results, it can be concluded that our novel TD3-based algorithm effectively solves the joint computation offloading and UAV trajectory optimization problems. The proposed algorithm improves the performance of the system energy efficiency by 3.77%, 22.90%, and 67.62%, respectively, compared to the other three benchmark schemes.

Real-time pricing considering carbon constraints for smart grid with multiple energy sources

With the escalation of fossil fuel pollution and the challenges of climate change, the smart grid is forced to address the carbon emissions associated with fossil fuel use. Real-time pricing of smart grid is an effective means to regulate the balance of supply and demand in the grid as well as to achieve energy conservation and environmental protection. This paper takes traditional fossil energy and clean energy power supply as the background, takes social welfare maximization as the goal, combines carbon emission cost with traditional power generation cost, and constructs a real-time pricing model that takes into account carbon emission cost, so that thermal power enterprises participate in the power market and carbon market at the same time. By using duality methods, we transformed the original optimization problem into an equivalent one. Furthermore, a distributed iterative algorithm based on the subgradient projection method is employed to solve the dual problem. The simulation results verify the feasibility and effectiveness of the proposed real-time pricing strategy.

A verified training support vector machine in bearing fault diagnosis

Abstract The combination of support vector machine (SVM) and deep learning is widely used in bearing fault diagnosis. The SVM-based bearing fault diagnosis method usually uses features extracted from bearing vibration signals as input. However, environmental noise in industrial applications can affect the feature representation, potentially leading to failures in SVM-based fault diagnosis methods. Adversarial robustness verification methods can evaluate the robustness of models to small perturbations in feature representations. Certified defense methods achieve robustness enhancement by embedding adversarial robustness verification into the learning process of the model. Thus, we propose a certified defense method named verified training SVM (VT-SVM) based on Lagrange duality. The proposed method incorporates adversarial robustness verification into the SVM learning framework, thereby enhancing the robustness of SVM in noisy environments. First, the certified defense optimization problem of SVM is constructed by combining the original optimization problem of SVM with the adversarial robustness verification problem. Then, the certified defense optimization problem is converted into an equivalent convex quadratic programming problem by introducing slack variables. Next, the Lagrange duality is used to transform the convex quadratic programming problem into a dual problem. Finally, the solution to the original optimization problem can be obtained by solving the dual problem. We validate the effectiveness of VT-SVM on the artificial dataset, the rolling bearing dataset from Case Western Reserve University, and the VBL-VA001 dataset from Sepuluh Nopember Institute of Technology. Experimental results demonstrate that our method both ensures high predictive performance and improves the resistance of the model to small noise.

Portfolio and reinsurance optimization under unknown market price of risk

We investigate the optimal investment-and-reinsurance problem for insurance company with partial information on the market price of the risk. Through the use of filtering techniques, we convert the original optimization problem involving different filtrations into an equivalent stochastic control problem under the observation filtration, i.e. the so-called separated problem. The Markovian structure of the separated problem allows us to apply a classical approach to stochastic optimization based on the Hamilton–Jacobi–Bellman equation, and to provide explicit formulas for the value function and the optimal investment-and-reinsurance strategy. We finally discuss some comparisons between the optimal strategies pursued by a partially informed insurer and that followed by a fully informed insurer, and we evaluate the value of information using the idea of indifference pricing. These results are also supported by numerical experiments.

Efficient radiation treatment planning based on voxel importance

Objective. Radiation treatment planning (RTP) involves optimization over a large number of voxels, many of which carry limited information about the clinical problem. We propose an approach to reduce the large optimization problem by only using a representative subset of informative voxels. This way, we drastically improve planning efficiency while maintaining the plan quality. Approach. Within an initial probing step, we pre-solve an easier optimization problem involving a simplified objective from which we derive an importance score per voxel. This importance score is then turned into a sampling distribution, which allows us to subsample a small set of informative voxels using importance sampling. By solving a—now reduced—version of the original optimization problem using this subset, we effectively reduce the problem’s size and computational demands while accounting for regions where satisfactory dose deliveries are challenging. Main results. In contrast to other stochastic (sub-)sampling methods, our technique only requires a single probing and sampling step to define a reduced optimization problem. This problem can be efficiently solved using established solvers without the need of modifying or adapting them. Empirical experiments on open benchmark data highlight substantially reduced optimization times, up to 50 times faster than the original ones, for intensity-modulated radiation therapy, all while upholding plan quality comparable to traditional methods. Significance. Our novel approach has the potential to significantly accelerate RTP by addressing its inherent computational challenges. We reduce the treatment planning time by reducing the size of the optimization problem rather than modifying and improving the optimization method. Our efforts are thus complementary to many previous developments.

Revisiting Possibilistic Fuzzy C-Means Clustering Using the Majorization-Minimization Method.

Possibilistic fuzzy c-means (PFCM) clustering is a kind of hybrid clustering method based on fuzzy c-means (FCM) and possibilistic c-means (PCM), which not only has the stability of FCM but also partly inherits the robustness of PCM. However, as an extension of FCM on the objective function, PFCM tends to find a suboptimal local minimum, which affects its performance. In this paper, we rederive PFCM using the majorization-minimization (MM) method, which is a new derivation approach not seen in other studies. In addition, we propose an effective optimization method to solve the above problem, called MMPFCM. Firstly, by eliminating the variable V∈Rp×c, the original optimization problem is transformed into a simplified model with fewer variables but a proportional term. Therefore, we introduce a new intermediate variable s∈Rc to convert the model with the proportional term into an easily solvable equivalent form. Subsequently, we design an iterative sub-problem using the MM method. The complexity analysis indicates that MMPFCM and PFCM share the same computational complexity. However, MMPFCM requires less memory per iteration. Extensive experiments, including objective function value comparison and clustering performance comparison, demonstrate that MMPFCM converges to a better local minimum compared to PFCM.