Convex Quadratic Programming Problem Research Articles

A linearly convergent self-adaptive gradient projection algorithm for sparse signal reconstruction in compressive sensing

<abstract><p>For sparse signal reconstruction (SSR) problem in compressive sensing (CS), by the splitting technique, we first transform it into a continuously differentiable convex optimization problem, and then a new self-adaptive gradient projection algorithm is proposed to solve the SSR problem, which has fast solving speed and pinpoint accuracy when the dimension increases. Global convergence of the proposed algorithm is established in detail. Without any assumptions, we establish global $ R- $linear convergence rate of the proposed algorithm, which is a new result for constrained convex (rather than strictly convex) quadratic programming problem. Furthermore, we can also obtain an approximate optimal solution in a finite number of iterations. Some numerical experiments are made on the sparse signal recovery and image restoration to exhibit the efficiency of the proposed algorithm. Compared with the state-of-the-art algorithms in SSR problem, the proposed algorithm is more accurate and efficient.</p></abstract>

A Discrete-time Projection Neural Network for Solving Convex Quadratic Programming Problems with Hybrid Constraints

A Discrete-time Projection Neural Network for Solving Convex Quadratic Programming Problems with Hybrid Constraints

Ensemble Support Vector Recurrent Neural Network for Brain Signal Detection.

The brain-computer interface (BCI) P300 speller analyzes the P300 signals from the brain to achieve direct communication between humans and machines, which can assist patients with severe disabilities to control external machines or robots to complete expected tasks. Therefore, the classification method of P300 signals plays an important role in the development of BCI systems and technologies. In this article, a novel ensemble support vector recurrent neural network (E-SVRNN) framework is proposed and developed to acquire more accurate and efficient electroencephalogram (EEG) signal classification results. First, we construct a support vector machine (SVM) to formulate EEG signals recognizing model. Second, the SVM formulation is transformed into a standard convex quadratic programming (QP) problem. Third, the convex QP problem is solved by combining a varying parameter recurrent neural network (VPRNN) with a penalty function. Experimental results on BCI competition II and BCI competition III datasets demonstrate that the proposed E-SVRNN framework can achieve accuracy rates as high as 100% and 99%, respectively. In addition, the results of comparison experiments verify that the proposed E-SVRNN possesses the best recognition accuracy and information transfer rate (ITR) compared with most of the state-of-the-art algorithms.

QPPAL: A Two-phase Proximal Augmented Lagrangian Method for High-dimensional Convex Quadratic Programming Problems

In this article, we aim to solve high-dimensional convex quadratic programming (QP) problems with a large number of quadratic terms, linear equality, and inequality constraints. To solve the targeted QP problem to a desired accuracy efficiently, we consider the restricted-Wolfe dual problem and develop a two-phase Proximal Augmented Lagrangian method (QPPAL), with Phase I to generate a reasonably good initial point to warm start Phase II to obtain an accurate solution efficiently. More specifically, in Phase I, based on the recently developed symmetric Gauss-Seidel (sGS) decomposition technique, we design a novel sGS-based semi-proximal augmented Lagrangian method for the purpose of finding a solution of low to medium accuracy. Then, in Phase II, a proximal augmented Lagrangian algorithm is proposed to obtain a more accurate solution efficiently. Extensive numerical results evaluating the performance of QPPAL against existing state-of-the-art solvers Gurobi, OSQP, and QPALM are presented to demonstrate the high efficiency and robustness of our proposed algorithm for solving various classes of large-scale convex QP problems. The MATLAB implementation of the software package QPPAL is available at https://blog.nus.edu.sg/mattohkc/softwares/qppal/ .

A novel robust support vector machine classifier with feature mapping

Support vector machines with different kernel functions are useful for binary classification. In this paper, a novel support vector machine model with feature mapping is proposed. Feature mapping is achieved based on the equivalence of the recently proposed soft quadratic surface support vector machine. In this model, a technique for pre-selecting training points is initially utilized to improve robustness. Then, the feature mapping and kernel trick are combined to handle datasets with different distributions, particularly nonlinearly separable datasets. This model can be transformed into a convex quadratic programming problem, which is efficiently solved by applying the sequential minimal optimization algorithm. Finally, numerical tests on several artificial and benchmark datasets reveal the superior performance of the proposed method over other well-known classification methods.

Hyperspectral Image Classification Based on a Least Square Bias Constraint Additional Empirical Risk Minimization Nonparallel Support Vector Machine

Hyperspectral image classification technology is important for the application of hyperspectral technology. Support vector machines (SVMs) work well in supervised classifications of hyperspectral images; however, they still have some shortcomings, and their use of a parallel decision plane makes it difficult to conform to real hyperspectral data distribution. The improved nonparallel support vector machine based on SVMs, i.e., the bias constraint additional empirical risk minimization nonparallel support vector machine (BC-AERM-NSVM), has improved classification accuracy compared its predecessor. However, BC-AERM-NSVMs have a more complicated solution problem than SVMs, and if the dataset is too large, the training speed is significantly reduced. To solve this problem, this paper proposes a least squares algorithm, i.e., the least square bias constraint additional empirical risk minimization nonparallel support vector machine (LS-BC-AERM-NSVM). The dual problem of the LS-BC-AERM-NSVM is an unconstrained convex quadratic programming problem, so its solution speed is greatly improved. Experiments on hyperspectral image data demonstrate that the LS-BC-AERM-NSVM displays a vast improvement in terms of solution speed compared with the BC-AERM-NSVM and achieves good classification accuracy.

Computationally efficient train timetable generation of metro networks with uncertain transfer walking time to reduce passenger waiting time: A generalized Benders decomposition-based method

With more and more interchange stations in a large-scale metro network, passengers tend to transfer between different metro lines from origination to destination, sometimes even more than once. Passenger waiting time is one of the critical standards for measuring the quality of urban public transport services. To support high service quality, this paper proposes a mixed integer nonlinear programming (MINLP) model for the train timetable generation problem of a metro network that minimizes the transfer waiting times and access passenger waiting times. In the mathematical formulation of the model, the transfer walking times at the interchange stations between two connected lines are treated as uncertain parameters. The robust train timetable generation model is formulated to optimize timetables by adjusting arrival and departure times of each train in the metro network to reduce access and transfer passenger waiting times. A robust counterpart is further derived that transforms the formulated robust model into a deterministic one. Moreover, a generalized Benders decomposition technique based approach is developed to decompose the robust counterpart into a subproblem and a master problem. The subproblem is a convex quadratic programming problem that can be solved efficiently. Finally, two sets of numerical examples, consisting of a small case and a large-scale case based on a real-world portion of the Beijing metro network, are performed to demonstrate the validity and practicability of the proposed model and solution approach.

Stability analysis of proportional delayed projection neural network for quadratic programming problem

At present, projection neural network (PNN) with bounded time delay has been widely used for solving convex quadratic programming problem (QPP). However, there is little research concerning PNN with unbounded time delay. In this paper, we propose the proportional delayed PNN to solve QPP with equality constraints. By utilizing homeomorphism mapping principle, we prove the proportional delayed PNN exists with unique equilibrium point which is the optimal solution of QPP. Simultaneously, delay-dependent criteria about global exponential stability (GES) and global polynomial stability (GPS) are also acquired by applying the method of variation of constants and inequality techniques. On the other hand, when proportional delay factor [Formula: see text] is equal to 1, the proportional delayed PNN becomes the one without time delay which still can be utilized for solving QPP. But in most situations, [Formula: see text] is not equal to 1, and time delay is unpredictable and may be unbounded in the actual neural network, which causes instability of system. Therefore, it is necessary to consider proportional delayed PNN. A numerical example demonstrates that, compared with the proportional delayed Lagrange neural network, the proportional delayed PNN is faster in terms of convergence rate. The possible reason is that appropriate parameters make the model converge to the equilibrium point along the direction of gradient descent.

A Support Vector Neural Network for P300 EEG Signal Classification

Brain computer interface (BCI) P300 speller can help severely disabled patients communicate and control with external machines or robots, so that the classification methods of P300 Electroencephalogram (EEG) signal play an important role in the development of BCI system and technology. In this paper, a novel support vector neural network (SVNN) is proposed and developed to obtain more accurate and effective EEG classification results. It is the first time to combine linear variational inequality (LVI) based primal-dual neural network with convex quadratic programming problem based on support vector machine to solve the classification problem. It has been proved that the support vector neural network globally converges to the optimal solution of convex optimization problem and iterates the parameters in the form of matrix, which means that the method has global convergence and parallelism. The proposed SVNN method is used to solve the classification problem of P300 EEG signals. Experimental results on data set IIb from BCI competition II and data set II from BCI competition III show that the accuracy of the proposed SVNN method is 100% and 98%, respectively. Compared with most of the state-of-theart algorithms, SVNN has the highest recognition accuracy and information transfer rate.

An efficient arc-search interior-point algorithm for convex quadratic programming with box constraints

Although the classical LQR design method has been very successful in real world engineering designs, in some cases, the classical design method needs modifications because of the saturation in actuators. This modified problem is sometimes called the constrained LQR design. For discrete systems, the constrained LQR design problem is equivalent to a convex quadratic programming problem with box constraints. We will show that the interior-point method is very efficient for this problem because an initial interior point is available, a condition which is not true for general convex quadratic programming problem. We will devise an effective and efficient algorithm for the constrained LQR design problem using the special structure of the box constraints and a recently introduced arc-search technique for the interior-point algorithm. We will prove that the algorithm is polynomial and has the best-known complexity bound for the convex quadratic programming. The proposed algorithm is implemented in MATLAB. An example for the constrained LQR design is provided to show the effectiveness and efficiency of the design method. The proposed algorithm can easily be used for model predictive control.

A new projection neural network for linear and convex quadratic second-order cone programming

A new projection neural network approach is presented for the linear and convex quadratic second-order cone programming. In the method, the optimal conditions of the linear and convex second-order cone programming are equivalent to the cone projection equations. A Lyapunov function is given based on the G-norm distance function. Based on the cone projection function, the descent direction of Lyapunov function is used to design the new projection neural network. For the proposed neural network, we give the Lyapunov stability analysis and prove the global convergence. Finally, some numerical examples and two kinds of grasping force optimization problems are used to test the efficiency of the proposed neural network. The simulation results show that the proposed neural network is efficient for solving some linear and convex quadratic second-order cone programming problems. Especially, the proposed neural network can overcome the oscillating trajectory of the exist projection neural network for some linear second-order cone programming examples and the min-max grasping force optimization problem.

Twin Support Vector Regression Model Based on Heteroscedastic Gaussian Noise and Its Application

The main purpose of twin support vector regression (TSVR) is to find linear or nonlinear relationships in sample data, and then predict future data. TSVR is the decomposition of a large convex quadratic programming problem into two small convex quadratic programming problems. Therefore, TSVR not only has the advantages of fast computation and low computational complexity, but also has better regression performance. Classic SVR, TSVR is assuming that accords with mean zero, variance of noise with the variance of the gaussian distribution or not to consider the effects of noise, but in some practical applications, such as wind speed forecasting, noise characteristic is more in line with mean zero, variance for heteroscedasticity of gaussian distribution, therefore, the return of the existing technology is not the best. In this paper, the characteristics of heteroscedasticity Gaussian noise are introduced into the model TSVR, and the twin support vector regression model based on heteroscedasticity Gaussian noise (TSVR-HGN) is constructed. The Lagrange multiplier method is used to solve the problem, and the optimization algorithm is used to find the global optimization. The artificial data set, UCI data set and wind speed data set were selected for experimental comparison. The experimental results show that TSVR-HGN has better prediction accuracy.

Energy-Optimal End-to-End Network Slicing in Cloud-Based Architecture

Network slicing is a promising technology for realizing the vision of supporting a wide range of services with diverse and heterogeneous service requirements. With network slicing, the network is partitioned into multiple individual dedicated networks tailored and customized for specific services. However, this causes extra energy consumption to reserve resources for each slice. On the other hand, minimizing the energy consumption in radio access network (RAN) may result in increasing the energy consumption in the cloud and the fronthaul due to higher required processing and data transport. Therefore, the energy should be evaluated from an end-to-end perspective. In this study, we address the problem of minimizing the end-to-end energy consumption of a network with network slicing by jointly reserving communication and computation resources among slices. First, we propose an end-to-end delay and energy model for each slice. We take into account the delays and energy consumption of the radio site, midhaul/fronthaul transport, and the cloud site in an Ethernet-based cloud RAN (C-RAN). Then, we formulate a non-convex optimization problem to minimize the total energy consumption of the network by jointly allocating transmission bandwidth and processing resources in the digital unit pool of the cloud, respectively, to each slice. The constraints of the optimization problem are the total delay requirement of each slice, the maximum allowable bandwidth at each radio unit, the maximum rate limitation of the Ethernet links, and the total processing limit of the cloud. To solve the problem optimally, we transform it into a convex quadratic programming problem. The simulation results show that end-to-end network slicing can decrease the total energy consumption of the network compared to only RAN slicing. We also investigate the impact of the 5G numerology on the allocated resources to each slice and the total energy consumption. We show that using mixed numerology depending on the slice type, we can interplay between delay and energy consumption for each slice.

A Circadian Rhythms Neural Network for Solving the Redundant Robot Manipulators Tracking Problem Perturbed by Periodic Noise

Redundant robot manipulators are usually applied to various complex scenarios where harmful noise especially periodic calculating noise always exists. In order to avoid the task failure affected by the periodic noise, a circadian rhythms neural network (CRNN) is applied to solve motion planning problems of redundant robot manipulators suffering from the periodic noise. First, in this article, we formulate a tracking problem of redundant manipulators as a convex quadratic programming (QP) problem. Second, the QP problem is converted into a matrix equation. Third, based on the neural dynamic design method, a CRNN model is exploited and developed to solve the matrix equation of the tracking problem. Comparative simulations between the proposed CRNN and the traditional zeroing neural network show that the CRNN has better robustness and performance on solving the end-effector tracking task. Two physical experiments are conducted to further certify the effectiveness, robustness, and practicability of the proposed CRNN.

An augmented Lagrangian method with constraint generation for shape-constrained convex regression problems

Shape-constrained convex regression problem deals with fitting a convex function to the observed data, where additional constraints are imposed, such as component-wise monotonicity and uniform Lipschitz continuity. This paper provides a unified framework for computing the least squares estimator of a multivariate shape-constrained convex regression function in $${\mathbb {R}}^d$$ . We prove that the least squares estimator is computable via solving an essentially constrained convex quadratic programming (QP) problem with $$(d+1)n$$ variables, $$n(n-1)$$ linear inequality constraints and n possibly non-polyhedral inequality constraints, where n is the number of data points. To efficiently solve the generally very large-scale convex QP, we design a proximal augmented Lagrangian method (proxALM) whose subproblems are solved by the semismooth Newton method. To further accelerate the computation when n is huge, we design a practical implementation of the constraint generation method such that each reduced problem is efficiently solved by our proposed proxALM. Comprehensive numerical experiments, including those in the pricing of basket options and estimation of production functions in economics, demonstrate that our proposed proxALM outperforms the state-of-the-art algorithms, and the proposed acceleration technique further shortens the computation time by a large margin.

Inference on estimators defined by mathematical programming

We propose an inference procedure for a class of estimators defined as the solutions to linear and convex quadratic programming problems in which the coefficients in both the objective function and the constraints of the problem are estimated from data and hence involve sampling error. We argue that the Karush–Kuhn–Tucker conditions that characterize the solutions to these programming problems can be treated as moment conditions; by doing so, we transform the problem of inference on the solution to a constrained optimization problem (which is non-standard) into one involving inference on inequalities with pre-estimated coefficients, which is better understood. Our approach is valid regardless of whether the problem has a unique solution or multiple solutions. We apply our method to various portfolio selection models, in which the confidence sets can be non-convex, lower-dimensional manifolds.

Robust optimal predictive control for real-time bus regulation strategy with passenger demand uncertainties in urban rapid transit

The bus is inevitably affected by various factors during the trip, which causes that the phenomenon of bus bunching often occurs. This paper proposed a nonlinear optimal control model with passenger demand and disturbance uncertainties for the real-time bus regulation in urban rapid transit. The aim of this model is to assure the stability of bus headway in the line, reduce passengers’ waiting time and improve bus service level. Considering the system uncertainties and the real-time control requirement of bus regulation, a robust optimal predictive control algorithm is put forward to generate the real-time optimal bus regulation strategy based on updated delay feedback of buses by a rolling horizon strategy. Simultaneously, by using continuous interval number to describe the uncertain parameters, the robust counterpart of the formulated robust optimization model is obtained using the duality theory, which is further converted to a mixed integer convex quadratic programming problem by a linearization method, which can be easily solved to obtain the robust solution with a good performance for any value of uncertain parameters. Finally, some numerical experiments are conducted to verify the validity of the presented optimization model in improving the stability of the bus headway.

The Weighted Lower and Upper Admissible Mean Downside Semi-variance Portfolio Selection

In this paper, the lower and upper admissible mean, downside semi-covariances and downside semi-variances of assets are first defined. Considering the transaction costs, borrowing constraints and threshold constraints, we present a new weighted lower and upper admissible mean downside semi-variance portfolio selection model which is a convex quadratic programming problem. The improved pivoting algorithm is presented to obtain the optimal portfolio strategy. Finally, the efficient frontiers comparison analyses of different optimistic coefficients, borrowing constraints and threshold constraints are given to examine the model and the designed algorithm.

RNN for Receding Horizon Control of Redundant Robot Manipulators

Redundant manipulators have been studied and applied in many fields. The trajectory tracking of redundant manipulators is an important topic to explore for applications. This article aims to develop a planning scheme for achieving the trajectory tracking of redundant manipulators, from the receding horizon control (RHC) perspective. For the nonlinear model of manipulators, the linearization operation is conducted to obtain predictive outputs through the forward kinematic equation. Subsequently, an RHC scheme, which minimizes tracking error, velocity norm, and acceleration norm, and directly considers joint limits at three levels as well as the terminal equality constraint, is constructed and further simplified as a convex quadratic programming problem. Furthermore, a recurrent neural network (RNN) model is designed for the constructed RHC scheme, with the help of the technique of converting inequality constraints into equality constraints. The proposed RHC scheme solved by the RNN model is compared with other existing planning schemes and solvers through computer simulations and experiments, without and with the sudden external interference. Simulation and experiment results show that the proposed RHC scheme solved by the RNN model is able to make the redundant manipulator track the given trajectory excellently, and is superior to other existing schemes and solvers in terms of high efficiency, quick-response capacity, and strong robustness.

Convex MPC for exclusion constraints

This article develops model predictive control for exclusion constraints with a priori guaranteed strong system theoretic properties, which is implementable via computationally highly efficient, strictly convex quadratic programming. The proposed approach deploys safe tubes in order to ensure intrinsically nonconvex exclusion constraints via closed polyhedral constraints. A safe tube is constructed by utilizing the separation theorem for convex sets, and it is practically obtained from the solution of a strictly convex quadratic programming problem. A safe tube is deployed to efficiently optimize a predicted finite horizon control process via another strictly convex quadratic programming problem.