Convex Quadratic Programming Research Articles

New Generalized Derivatives for Solving Variational Inequalities Using the Nonsmooth Newton Methods

AbstractVariational inequality (VI) generalizes many mathematical programming problems and has a wide variety of applications. One class of VI solution methods is to reformulate a VI into a normal map nonsmooth equation system, which is then solved using nonsmooth equation-solving techniques. In this article, we propose a first practical approach for furnishing B-subdifferential elements of the normal map, which in turn enables solving the normal map equation system using variants of the B-subdifferential-based nonsmooth Newton method. It is shown that our new method requires less stringent conditions to achieve local convergence than some other established methods, and thus guarantees convergence in certain cases where other methods may fail. We compute a B-subdifferential element using the LD-derivative, which is a recently established generalized derivative concept. In our new approach, an LD-derivative is computed by solving a sequence of strictly convex quadratic programs, which can be terminated early under certain conditions. Numerical examples are provided to illustrate the convergence properties of our new method, based on a proof-of-concept implementation in Python.

Using dual relaxations in multiobjective mixed-integer convex quadratic programming

Using dual relaxations in multiobjective mixed-integer convex quadratic programming

A Quasi-Newton Subspace Trust Region Algorithm for Nonmonotone Variational Inequalities in Adversarial Learning over Box Constraints

The first-order optimality condition of convexly constrained nonconvex nonconcave min-max optimization problems with box constraints formulates a nonmonotone variational inequality (VI), which is equivalent to a system of nonsmooth equations. In this paper, we propose a quasi-Newton subspace trust region (QNSTR) algorithm for the least squares problems defined by the smoothing approximation of nonsmooth equations. Based on the structure of the nonmonotone VI, we use an adaptive quasi-Newton formula to approximate the Hessian matrix and solve a low-dimensional strongly convex quadratic program with ellipse constraints in a subspace at each step of the QNSTR algorithm efficiently. We prove the global convergence of the QNSTR algorithm to an ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon $$\\end{document}-first-order stationary point of the min-max optimization problem. Moreover, we present numerical results based on the QNSTR algorithm with different subspaces for a mixed generative adversarial networks in eye image segmentation using real data to show the efficiency and effectiveness of the QNSTR algorithm for solving large-scale min-max optimization problems.

A verified training support vector machine in bearing fault diagnosis

Abstract The combination of support vector machine (SVM) and deep learning is widely used in bearing fault diagnosis. The SVM-based bearing fault diagnosis method usually uses features extracted from bearing vibration signals as input. However, environmental noise in industrial applications can affect the feature representation, potentially leading to failures in SVM-based fault diagnosis methods. Adversarial robustness verification methods can evaluate the robustness of models to small perturbations in feature representations. Certified defense methods achieve robustness enhancement by embedding adversarial robustness verification into the learning process of the model. Thus, we propose a certified defense method named verified training SVM (VT-SVM) based on Lagrange duality. The proposed method incorporates adversarial robustness verification into the SVM learning framework, thereby enhancing the robustness of SVM in noisy environments. First, the certified defense optimization problem of SVM is constructed by combining the original optimization problem of SVM with the adversarial robustness verification problem. Then, the certified defense optimization problem is converted into an equivalent convex quadratic programming problem by introducing slack variables. Next, the Lagrange duality is used to transform the convex quadratic programming problem into a dual problem. Finally, the solution to the original optimization problem can be obtained by solving the dual problem. We validate the effectiveness of VT-SVM on the artificial dataset, the rolling bearing dataset from Case Western Reserve University, and the VBL-VA001 dataset from Sepuluh Nopember Institute of Technology. Experimental results demonstrate that our method both ensures high predictive performance and improves the resistance of the model to small noise.

Bi-affine scaling iterative method for convex quadratic programming with bound constraints

To solve general convex quadratic programming problems with bound constraints, this paper proposes a new interior point iterative method that is easy to be implemented. The method exhibits a simple and sufficiently smooth search direction, and possesses the characteristics of affine scaling. Under the limited optimal stepsize rule, starting from an arbitrary interior point, any accumulation point of the generated sequence is an optimal solution of the corresponding problem. Furthermore, due to the absence of introducing dual variables and solving equations, the proposed method is more suitable for solving large-scale problems. Preliminary numerical results indicate that the new method has advantages in terms of both efficiency and accuracy.

Active Fault Tolerant Nonsingular Terminal Sliding Mode Control for Electromechanical System Based on Support Vector Machine

Effective fault diagnosis and fault-tolerant control method for aeronautics electromechanical actuator is concerned in this paper. By borrowing the advantages of model-driven and data-driven methods, a fault tolerant nonsingular terminal sliding mode control method based on support vector machine (SVM) is proposed. A SVM is designed to estimate the fault by off-line learning from small sample data with solving convex quadratic programming method and is introduced into a high-gain observer, so as to improve the state estimation and fault detection accuracy when the fault occurs. The state estimation value of the observer is used for state reconfiguration. A novel nonsingular terminal sliding mode surface is designed, and Lyapunov theorem is used to derive a parameter adaptation law and a control law. It is guaranteed that the proposed controller can achieve asymptotical stability which is superior to many advanced fault-tolerant controllers. In addition, the parameter estimation also can help to diagnose the system faults because the faults can be reflected by the parameters variation. Extensive comparative simulation and experimental results illustrate the effectiveness and advancement of the proposed controller compared with several other main-stream controllers.

A Full-Newton Step Interior-Point Method for Weighted Quadratic Programming Based on the Algebraic Equivalent Transformation

In this paper, a new full-Newton step feasible interior-point method for convex quadratic programming is presented and analyzed. The idea behind this method is to replace the complementarity condition with a non-negative weight vector and use the algebraic equivalent transformation for the obtained equation. Under the selection of appropriate parameters, the quadratic rate of convergence of the new algorithm is established. In addition, the iteration complexity of the algorithm is obtained. Finally, some numerical results are presented to demonstrate the practical performance of the proposed algorithm.

Convex Quadratic Programming for Computing Geodesic Distances on Triangle Meshes

Querying the geodesic distance field on a given smooth surface is a fundamental research pursuit in computer graphics. Both accuracy and smoothness serve as common indicators for evaluating geodesic algorithms. In this study, we argue that ensuring that the norm of the triangle-wise estimated gradients is not larger than 1 is preferable compared to the widely used eikonal condition. Inspired by this, we formulate the geodesic distance field problem as a Quadratically Constrained Linear Programming (QCLP) problem. This formulation can be further adapted into a Quadratically Constrained Quadratic Programming (QCQP) problem by incorporating considerations for smoothness requirements. Specifically, when enforcing a Hessian-energy-based smoothing term, our formulation, named QCQP-Hessian, effectively mitigates the cusps in the geodesic isolines within the near-ridge area while maintaining accuracy in the off-ridge area. We conducted extensive experiments to demonstrate the accuracy and smoothness advantages of QCQP-Hessian.

Network partition and distributed voltage coordination control strategy of active distribution network system considering photovoltaic uncertainty

Aiming at the voltage exceeding problem caused by the introduction of Photovoltaic (PV) in Active distribution network (ADN), a comprehensive electrical distance index was established considering active and reactive voltage sensitivity, and the uncertainty of traditional PV probability density function (PDF) was optimized by random simulation method. The electrical distance expectation matrix between nodes was established by discretization of PV output probability distribution characteristics, and the ADN system was partitioned by affinity propagation (AP) algorithm. Taking active power loss as the objective function, the non-convex reactive power optimization control problem is transformed into a convex quadratic programming problem by LinDistFlow equation. An improved alternating direction method of multipliers (ADMM) which adaptively adjusts the penalty parameters is used to convert the finite boundaries of adjacent regions into data. By coordinating on-load tap-changer (OLTC), capacitor banks (CBs), and PV inverters on different time scales, the fast optimal control of global voltage in ADN is realized. The proposed method is tested on IEEE 33-bus and IEEE 123-bus distribution systems. After the optimization control algorithm described in this paper, the system network loss was reduced by 3.88%. Compared with the semi-definite programming relaxation, the calculation speed of LinDistFlow equation is increased by 72.2%. The results fully verify the feasibility and high efficiency of the control strategy described in this paper.

Evaluation method of distributed photovoltaic maximum access capacity based on ADMM

With the increasing penetration rate of distributed photovoltaics in the distribution network, new requirements are put forward for the operation and control of distribution networks. In order to ensure that the distribution network can operate stably within a safe range, it is important to evaluate the maximum capacity of the distribution network that can be connected to distributed photovoltaics. Considering the high cost of centralized computing, high communication requirements, and poor reliability, it is difficult to adapt to the actual needs of the distribution network with large-scale access to distributed photovoltaics. Therefore, this paper proposes an ADMM-based distributed PV maximum access capacity evaluation method. This method takes the distributed photovoltaic access capacity and the active power loss of the distribution network line as the objective function and considers the power flow equation constraints, node voltage constraints, and line current carrying capacity constraints of the distribution network. By simplifying the model, the non-convex optimization problem is transformed into a convex quadratic programming problem, and the ADMM algorithm is used to solve the distributed solution to obtain the access strategy of distributed PV in various regions under different operating conditions. Finally, the IEEE-33 system is used as an example to simulate the calculation, and the experimental results verify the correctness and effectiveness of the method.

On the number of pivots of Dantzig's simplex methods for linear and convex quadratic programs

Refining and extending works by Ye and Kitahara-Mizuno, this paper presents new results on the number of pivots of simplex-type methods for solving linear programs of the Leontief kind, certain linear complementarity problems of the P kind, and nonnegatively constrained convex quadratic programs. Our results contribute to the further understanding of the complexity and efficiency of simplex-type methods for solving these problems. Two applications of the quadratic programming results are presented.

A multi-view ensemble model based on semi-supervised feature learning for small sample classification of PolSAR images

ABSTRACT Classification with a few samples of training set has been a long-standing issue in the field of polarimetric synthetic aperture radar (PolSAR) image analysis and processing. In fact, one of the most important challenges of the PolSAR image classification task is the number of labelled samples. In essence, training network by utilizing just few numbers of training samples cannot be led to the reliable result since the neural network is so sensitive to the number of training samples. In addition, in the real PolSAR image classification task, the huge number of training samples is not accessible. On the other hand, classification performance using just small number of labelled samples is not an accurate result. So, in this paper, aiming at the small number of training samples of the PolSAR image classification task, a novel ensemble self-supervised feature-learning (ESSFL) model is designed. The designed ESSFL can automatically extract PolSAR features conducive to PolSAR image classification with a small number of training samples. In addition, it can significantly decrease the dependence of neural network algorithms on large labelled samples of training set. First, to utilize the spatial–polarimetric features of PolSAR data perfectly, the EfficientNet-B0 is presented and utilized as the main section of the deep learning (DL) model to extract DL features of PolSAR data. Then, using an optimization function that constrains the cross-correlation matrix of various distortions of each sample to the identity matrix, the designed deep learning model can obtain the effective features of homogeneous samples gathering together and heterogeneous samples separating from each other in a self-supervised manner. Moreover, two ensemble learning (EL) models, feature-level and view-level ensemble, are proposed to increase the feature extraction capability and classification result by jointly using spatial features at different scales and polarimetric information at different bands. Finally, the stack of the obtained features and the main polarimetric information of PolSAR data can be input into classifier for the classification of PolSAR data. In this paper we use two different classifiers, the first one is the Random Forest (RF) classifier and the second one is the Support Vector Machine (SVM) classifier. The advantages of SVM include that it can be used to avoid the difficulties of using linear functions in the high-dimensional feature space, and the optimization problem is transformed into dual convex quadratic programs that are so usable for an actual image classification task. In addition, among all the available classification methods, random forests provide one of the highest accuracies. The random forest technique can also handle big data with numerous variables running into thousands. In fact, it can automatically balance data sets when a class is more infrequent than other classes in the data that is so important for a real PolSAR image classification task. Experimental results on three well-known PolSAR data sets illustrate that the designed ESSFL can extract more discriminant features using the designed deep learning model. In the end, the experiments prove that the designed ESSFL has a significant classification performance compared with different deep learning models in the case of small number of training samples, and also it can achieve a better result in the case of large number of training samples by comparison with the most of the deep learning models.

Anomaly Detection Method for Rocket Engines Based on Convex Optimized Information Fusion.

The power system, as a core component of a launch vehicle, has a crucial impact on the reliability and safety of a rocket launch. Due to the limited measurement information inside the engine, it is often challenging to realize fast and accurate anomaly detection. For this reason, this paper introduces the rocket flight state data to expand the information source for anomaly detection. However, engine measurement and rocket flight state information have different data distribution characteristics. To find the optimal data fusion scheme for anomaly detection, a data set information fusion algorithm based on convex optimization is proposed, which solves the optimal fusion parameter using the convex quadratic programming problem and then adopts the adaptive CUSUM algorithm to realize the fast and accurate anomaly detection of engine faults. Numerical simulation tests show that the algorithm proposed in this paper has a higher detection accuracy and lower detection time than the traditional algorithm.

Evaluation of Grid Support Capacity From Electrified Railways

High power level and temporal operation flexibility endow electric trains with great potential for improving grid resilience. It becomes crucial to quantify the grid support capacity (GSC) from large-scale electrified railways. This paper innovatively proposes a hierarchical train coordination and trajectory regulation method to evaluate and deliver the GSC from electrified railways. In the light of the mechanical energy consumption analysis of train running, the energy-oriented train operation model is proposed to avoid the nonlinear differential dynamics of train kinetics when coordinating large-scale trains. Then the aggregation and disaggregation of trains for grid support can be formulated as linear and convex quadratic programming problems respectively. To closely follow the reference power and enhance the feasibility of train operation, a novel iterative discretization approach is proposed to generate the precise reference trajectory. A sliding mode controller with disturbance bound adaption is devised to overcome the modeling errors and disturbances when deploying GSC in realistic environment. Comprehensive numerical examples verify the effectiveness of the proposed scheme, and demonstrate the significant grid support potential from collaborative electrified railways.

A steepest feasible direction method for linear programming. Derivation and embedding in the simplex method

A feasible direction method for linear programming has been proposed. The method is embedded in the framework of the simplex method, even though it works with non-edge feasible directions. The direction used is the steepest in the space of all variables or an approximation thereof, and it is found by solving a strictly convex quadratic program in the space of the nonbasic variables. Further, this program guarantees the feasibility of the direction even in the case of degeneracy. To remain within the simplex framework, the direction is represented by an auxiliary, or external, nonbasic column, which is a nonnegative linear combination of original nonbasic columns. We have made an experimental evaluation of the suggested method on both nondegenerate and highly degenerate problem instances. The overall results are very promising for continued research along this line, especially concerning various computational strategies that can be applied when the method is implemented.

Numerical solution of box constrained separable convex quadratic programming problems

In this paper, minimization problems with a separable convex quadratic objective function subject to a linear equality constraint/linear equality constraints, and bounds on the variables (box constraints) are considered. A necessary and sufficient condition for a feasible solution to be an optimal solution to each of these problems has been established. A convergent polynomial algorithm for solving the problem with a linear equality constraint and bounded variables is proposed, and some numerical results, obtained by this algorithm, are presented.

A High Order Integration Enhanced Zeroing Neural Network for Time-Varying Convex Quadratic Program Against Nonlinear Noise

A High Order Integration Enhanced Zeroing Neural Network for Time-Varying Convex Quadratic Program Against Nonlinear Noise

A coordinated optimization method of energy management and trajectory optimization for hybrid electric UAVs with PV/Fuel Cell/Battery

There are complex coupling relationships between flight motion and energy management for hybrid electric fixed wing UAVs with photovoltaic/fuel cell/battery. Aiming at realizing high-energy-efficiency long endurance autonomous flight, this paper proposes a coordinated optimization method of trajectory optimization and energy management for hybrid electric UAVs. Numerical simulation and experiment are carried out. Separated by surplus demand energy except that produced by PV, this method includes an optimal energy flight trajectory optimization layer and an energy management layer. For the former, fuzzy neural network sequential convex optimization (FNNSCP) is proposed to handle the flight trajectory optimization problem for maximizing utilization solar energy. For the latter, an online energy management strategy based on convex quadratic programming MPC (CQPMPC) is proposed to realize the power allocation. Numerical simulation results show that FNNSCP reduced 2% energy requirement compared with Radau pseudo-spectral method (RPM) from flight trajectory optimization level. From energy management level, CQPMPC saves hydrogen consumption by 3.1% and 16.3% compared with nonlinear model predictive control (NMPC) and fuzzy logic state machine (FLSM), respectively. In the experiment, less hydrogen fuel is consumed by proposed method by 5.2% and 15.6% compared with NMPC and FLSM, respectively. Experimental results show that the proposed method has better energy-saving performance and excellent real time performance. It indicates that optimizing flight trajectory and improving energy management efficiency can realize the high-energy-efficiency flight of hybrid electric UAVs. This method is conducive to promoting the application of hydrogen energy in the field of UAVs.

Improved fuzzy sparse multi-class least squares support vector machine

The classification problem is a key area of research in machine learning. The Least Squares Support Vector Machine (LSSVM) is an important classifier that is commonly used to solve classification problems. Its widespread use stems from its replacement of the inequality constraint in the Support Vector Machine (SVM) with the equality constraint, which transforms the convex quadratic programming (QP) problem of SVM into the solution of linear equations. However, when dealing with multi-class classification problems, LSSVM faces the challenges of lack of sparsity and sample noises, which can negatively impact its performance. Based on the modeling characteristics and data distribution of the multi-class LSSVM model, this paper proposes two improvements and establishes an improved fuzzy sparse multi-class least squares support vector machine (IF-S-M-LSSVM). The first improvement adopts a non-iterative sparse algorithm, which can delete training sample points to different degrees by adjusting the sparse ratio. The second improvement addresses the impact of sample noise on determining the optimal hyperplane by adding a fuzzy membership degree based on sample density. The advantages of the new model, in terms of training speed and classification accuracy, are verified through UCI machine learning standard data set experiments. Finally, the statistical significance of the IF-S-M-LSSVM model is tested using the Friedman and Bonferroni-Dunn tests.

Flatness-Based Quadcopter Trajectory Planning and Tracking With Continuous-Time Safety Guarantees

This work proposes a convex optimization-based framework for the trajectory planning and tracking of quadcopters that ensures continuous-time safety guarantees. Using the convexity property of B-spline curves and the differential flatness property of quadcopters, a second-order cone program is formulated to generate an optimal nominal trajectory that respects state and input constraints, including position, linear velocity, angle, angular velocity, thrust, waypoints, and obstacle avoidance constraints, rigorously in the continuous-time sense. To ensure safe trajectory tracking, a convex quadratic program is proposed based on control barrier functions, which guarantees that the actual trajectory of the quadcopter remains within a prescribed safe tube of the nominal trajectory in continuous time. Furthermore, conditions that ensure the safe tracking controller respects thrust, roll, and pitch constraints are also presented. Both the planning and control approaches are suitable for online implementation, and the effectiveness of the proposed framework is demonstrated through simulations and experiments with a Crazyflie2.1 nano quadcopter.