Unconstrained Minimization Problem Research Articles

On Infinite Horizon Optimal Control Problems with a Feed Forward Neural Network Scheme

In this paper, a class of infinite-horizon nonlinear optimal control problems is considered. The main idea is to convert the infinite horizon problem to an equivalent finite-horizon optimal control problem. According to the Pontryagin minimum principle for optimal control problems and by constructing an error function, we define an unconstrained minimization problem. In the optimization problem, we use trial solutions for the state, costate and control functions where these trial solutions are constructed by using two-layer perceptron. We then minimize the error function where weights and biases associated with all neurons are unknown. Substituting the optimal values of the weights and biases into the trial solutions, we obtain the optimal solution of the original problem. We also use a dynamic optimization scheme to learning process and discuss the stability and convergence properties of it. Some examples are given to show the efficiency of the method.

Localization of IoT Networks via Low-Rank Matrix Completion

Location awareness, providing the ability to identify the location of sensor, machine, vehicle, and wearable device, is a rapidly growing trend of hyper-connected society and one of the key ingredients for the Internet of Things (IoT) era. In order to make a proper reaction to the collected information from things , location information of things should be available at the data center. One challenge for the IoT networks is to identify the location map of whole nodes from partially observed distance information. The aim of this paper is to present an algorithm to recover the Euclidean distance matrix (and eventually the location map) from partially observed distance information. By casting the low-rank matrix completion problem into the unconstrained minimization problem in a Riemannian manifold in which a notion of differentiability can be defined, we solve the low-rank matrix completion problem using a modified conjugate gradient algorithm. From the convergence analysis, we show that localization in Riemannian manifold using conjugate gradient (LRM-CG) converges linearly to the original Euclidean distance matrix under the extended Wolfe’s conditions. From the numerical experiments, we demonstrate that the proposed method, called LRM-CG, is effective in recovering the Euclidean distance matrix.

An efficient neural network for solving convex optimization problems with a nonlinear complementarity problem function

In this paper, we present a one-layer recurrent neural network (NN) for solving convex optimization problems by using the Mangasarian and Solodov (MS) implicit Lagrangian function. In this paper by using Krush–Kuhn–Tucker conditions and MS function the NN model was derived from an unconstrained minimization problem. The proposed NN model is one layer and compared to the available NNs for solving convex optimization problems, which has a better performance in convergence time. The proposed NN model is stable in the sense of Lyapunov and globally convergent to optimal solution of the original problem. Finally, simulation results on several numerical examples are presented and the validity of the proposed NN model is demonstrated.

Scalar Parameter of a Spectral PRP Conjugate Gradient Method for Unconstrained optimization

In recent times, conjugate gradient method (CG) have been broadly used to solve nonlinear unconstrained minimization problems as a result of fewer storage locations and computational expensive in dealing with large-scale problems. In this work, we present a spectral PRP CG method which derived from the CG search direction without secant condition and utilized some of the benchmark test problem functions with several variables to prove its global convergence properties and satisfies sufficient descent condition, the results are validated by exact line search techniques.

Multi-category ternion support vector machine

This paper proposes a three-class classifier termed as ‘Ternion support vector machine’ (TerSVM) and its tree based multi-category classification approach termed as ‘Multi-category ternion support vector machine’ (M-TerSVM). The proposed classifier, TerSVM, is motivated by twin multi-class support vector classification (Twin-KSVC) and evaluates the data patterns for three outputs (+1,−1,0). Twin-KSVC has very high computational complexity, which makes it infeasible for real-world problems. Our proposed classifier, TerSVM, overcomes this limitation, as it formulates three unconstrained minimization problems (UMPs), instead of quadratic programming problems as solved by Twin-KSVC. The UMPs of TerSVM are solved as systems of linear equations which determine three proximal nonparallel hyperplanes. TerSVM can also be used as a binary classifier. This work also proposes a multi-category classification algorithm, M-TerSVM, that extents our three-class classifier (TerSVM) into multi-category framework. For a K-class problem, M-TerSVM constructs a classifier model in the form of a ternion tree of height ⌊K∕2⌋, where the data is partitioned into three groups at each level. Our algorithm uses a novel procedure to identify a reduced training set which improves its learning time. Numerical experiments performed on synthetic and benchmark datasets indicate that M-TerSVM outperforms other classical multi-category approaches like one-against-all and Twin-KSVC, in terms of generalization ability and learning time. This paper proposes the application of M-TerSVM for handwritten digit recognition and color image classification.

Unconstrained convex minimization based implicit Lagrangian twin random vector Functional-link networks for binary classification (ULTRVFLC)

In the spirit of twin support vector machine (TWSVM) and based on random vector Functional-link (RVFL) network, we propose implicit Lagrangian twin random vector Functional-link network via unconstrained minimization problem for better generalization performance. Unlike RVFL, which is a least-squares method, inequality constraints are employed to the proposed optimization problem to allow tolerating certain amount of error for handling overfitting problem. In this paper, the proposed constrained optimization problem is first reformulated to an unconstrained minimization problem and then is solved by iterative convergent schemes to eliminate the requirement to solving quadratic programming problems (QPPs) of its corresponding dual formulation. The ‘plus’ function present in the reformulated dual formulations of the proposed schemes is not differentiable which is handled by applying the generalized derivative approach or smooth approximation techniques and further solved by applying Newton–Armijo stepsize method. To establish the efficacy of the proposed algorithms, numerical experiments are carried out and comparison of results are carried out with support vector machine (SVM), twin support vector machine (TWSVM), kernel ridge regression (KRR), random vector Functional-link (RVFL) net, extreme learning machine (ELM), constrained extreme learning machine (CoELM) and twin extreme learning machine (TELM). The classification results are presented in terms of accuracy and further, statistical comparison of the algorithms’ performance is carried out to verify the significance of the proposed methods.

An efficient numerical scheme for solving fractional infinite-horizon optimal control problems

This paper presents an efficient numerical method to solve fractional infinite-horizon optimal control problems, where the dynamic control system depends on Caputo fractional derivatives. First, by a suitable change of variable, we transform the fractional infinite-horizon optimal control problem to a finite-horizon one. Then, with the help of an approximation, we replace the Caputo derivative to integer order derivative. According to the Pontryagin minimum principle (PMP) for optimal control problems and by constructing an error function, we define an unconstrained minimization problem. In the optimization problem, we use trial solutions for state, costate and control functions where these trial solutions are constructed by using two-layered perceptron neural network. Some numerical results are introduced to explain our main results.

Correction of trust region method with a new modified Newton method

ABSTRACTOne of the iterative techniques for solving unconstrained minimization problems is trust region method. Trust region methods are robust and can be applied to ill-conditioned problems. In this paper, we give a new nonmonotone trust region algorithm to solve problems arising from unconstrained optimization. The curvilinear paths we set are dogleg paths, generated mainly by employing Q.I.F. factorization (Quadrant Interlocking Factorisation) for general symmetric matrices which may be indefinite. Convergence properties of the trust region algorithm are given. Moreover, numerical experiments show that the new algorithm is competitive with some existing methods and performs very well.

Sinkhorn–Knopp theorem for PPT states

Given a PPT state $A=\sum_{i=1}^nA_i\otimes B_i \in M_k\otimes M_k$ and a vector $v\in\Im(A)\subset\mathbb{C}^k\otimes\mathbb{C}^k$ with tensor rank $k$, we provide an algorithm that checks whether the positive map $G_A:M_k\rightarrow M_k$, $G_A(X)=\sum_{i=1}^n tr(A_iX)B_i$, is equivalent to a doubly stochastic map. This procedure is based on the search for Perron eigenvectors of completely positive maps and unique solutions of, at most, $k$ unconstrained quadratic minimization problems. As a corollary, we can check whether this state can be put in the filter normal form. This normal form is an important tool for studying quantum entanglement. An extension of this procedure to PPT states in $M_k\otimes M_m$ is also presented.

On the relation between MPECs and optimization problems in abs-normal form

ABSTRACTWe show that the problem of unconstrained minimization of a function in abs-normal form is equivalent to identifying a certain stationary point of a counterpart Mathematical Program with Equilibrium Constraints (MPEC). Hence, concepts introduced for the abs-normal forms turn out to be closely related to established concepts in the theory of MPECs. We give a number of proofs of equivalence or implication for the kink qualifications LIKQ and MFKQ. We also show that the counterpart MPEC always satisfies MPEC-ACQ. We then consider non-smooth nonlinear optimization problems (NLPs) where both the objective function and the constraints are presented in the abs-normal form. We show that this extended problem class also has a counterpart MPEC problem.

Using a new regularized factorization method for unconstrained optimization problems

AbstractIn this paper, a new regularized factorization method is presented for solving unconstrained minimization problems in which the Hessian matrix may not be a positive definite or may be close to singular. With employing the modified quadrant interlocking factorization (QIF), an efficient algorithm is developed to find a suitable method. Moreover, under suitable conditions, the global convergence of the proposed method is established. Some interesting examples are solved by the proposed method and the improved Cholesky factorization method. The results show that the proposed method, in the most cases, the algorithm has a faster convergence than the improved Cholesky factorization method.

A projection wavelet weighted twin support vector regression and its primal solution

In this paper, an efficient projection wavelet weighted twin support vector regression (PWWTSVR) algorithm is proposed. PWWTSVR determines the regression function by solving a pair of smaller unconstrained minimization problems in primal space, which can reduce computational costs. Classical SVR algorithms give the same emphasis to all training samples, which degrades performance. PWWTSVR gives samples penalty weights determined by wavelet transforms. These are applied to both the quadratic empirical risks term and the first-degree empirical risks term to reduce the influence of outliers. A projection axis in each objective function is sought to minimize the variance of the projected points due to the utilization of a priori information of training data. Therefore, data structure terms are added to the penalty functions. The final regressor can avoid the overfitting problem to a certain extent, and yields great generalization ability. Numerical experiments on artificial and benchmark datasets demonstrate the feasibility and validity of the proposed algorithm.

A Stochastic Trust Region Method for Unconstrained Optimization Problems

In this paper, a stochastic trust region method is proposed to solve unconstrained minimization problems with stochastic objectives. Particularly, this method can be used to deal with nonconvex problems. At each iteration, we construct a quadratic model of the objective function. In the model, stochastic gradient is used to take the place of deterministic gradient for both the determination of descent directions and the approximation of the Hessians of the objective function. The behavior and the convergence properties of the proposed method are discussed under some reasonable conditions. Some preliminary numerical results show that our method is potentially efficient.

P.1.11 Inflammatory markers in anorexia nervosa: An exploratory study

In this paper, we propose a semi-supervised classifier termed as Fast Laplacian Twin Support Vector Machine (FLap−TWSVM) with an objective to reduce the requirement of labeled data and simultaneously lessen the training time complexity of a traditional Laplacian Twin Support Vector Machine semi-supervised classifier. FLap−TWSVM is faster than existing Laplacian twin support vector machine as it solves a smaller size Quadratic Programming Problem (QPP) along with an Unconstrained Minimization Problem (UMP) to obtain decision hyperplanes which can also handle heteroscedastic noise present in the training data. Traditional semi-supervised classifiers generally have no explicit control over the choice of labeled data available for training, hence to overcome this limitation, we propose a pool-based active learning framework which identifies most informative examples to train the learning model. Moreover, the aforementioned framework has been extended to deal with multi-category classification scenarios. Several experiments have been performed on machine learning benchmark datasets which proves the utility of the proposed classifier over traditional Laplacian Twin Support Vector Machine (Lap−TWSVM) and active learning based Support Vector Machine (SVMAL). The efficacy of the proposed framework has also been tested on human activity recognition problem and content based image retrieval system.

Optimality conditions for fuzzy non-linear unconstrained minimization problems

Optimality conditions for fuzzy non-linear unconstrained minimization problems

A new class of three-term conjugate Gradient methods for solving unconstrained minimization problems

A new class of three-term conjugate Gradient methods for solving unconstrained minimization problems

Fast Laplacian twin support vector machine with active learning for pattern classification

In this paper, we propose a semi-supervised classifier termed as Fast Laplacian Twin Support Vector Machine (FLap−TWSVM) with an objective to reduce the requirement of labeled data and simultaneously lessen the training time complexity of a traditional Laplacian Twin Support Vector Machine semi-supervised classifier. FLap−TWSVM is faster than existing Laplacian twin support vector machine as it solves a smaller size Quadratic Programming Problem (QPP) along with an Unconstrained Minimization Problem (UMP) to obtain decision hyperplanes which can also handle heteroscedastic noise present in the training data. Traditional semi-supervised classifiers generally have no explicit control over the choice of labeled data available for training, hence to overcome this limitation, we propose a pool-based active learning framework which identifies most informative examples to train the learning model. Moreover, the aforementioned framework has been extended to deal with multi-category classification scenarios. Several experiments have been performed on machine learning benchmark datasets which proves the utility of the proposed classifier over traditional Laplacian Twin Support Vector Machine (Lap−TWSVM) and active learning based Support Vector Machine (SVMAL). The efficacy of the proposed framework has also been tested on human activity recognition problem and content based image retrieval system.

A Riemannian variant of the Fletcher–Reeves conjugate gradient method for stochastic inverse eigenvalue problems with partial eigendata

SummaryIn this paper, we focus on the stochastic inverse eigenvalue problem with partial eigendata of constructing a stochastic matrix from the prescribed partial eigendata. A Riemannian variant of the Fletcher–Reeves conjugate gradient method is proposed for solving a general unconstrained minimization problem on a Riemannian manifold, and the corresponding global convergence is established under some assumptions. Then, we reformulate the inverse problem as a nonlinear least squares problem over a matrix oblique manifold, and the application of the proposed geometric method to the nonlinear least squares problem is investigated. The proposed geometric method is also applied to the case of prescribed entries and the case of column stochastic matrix. Finally, some numerical tests are reported to illustrate that the proposed geometric method is effective for solving the inverse problem.

Lagrangian supervised and semi-supervised extreme learning machine

Two extreme learning machine (ELM) frameworks are proposed to handle supervised and semi-supervised classifications. The first is called lagrangian extreme learning machine (LELM), which is based on optimality conditions and dual theory. Then LELM is extended to semi-supervised setting to obtain a semi-supervised extreme learning machine (called Lap-LELM), which incorporates the manifold regularization into LELM to improve performance when insufficient training information is available. In order to avoid the inconvenience caused by matrix inversion, Sherman-Morrison-Woodbury (SMW) identity is used in LELM and Lap-LELM, which leads to two smaller sized unconstrained minimization problems. The proposed models are solvable in a space of dimensionality equal to the number of sample points. The resulting iteration algorithms converge globally and have low computational burden. So as to verify the feasibility and effectiveness of the proposed method, we perform a series of experiments on a synthetic dataset, near-infrared (NIR) spectroscopy datasets and benchmark datasets. Compared with the traditional methods, experimental results demonstrate that the proposed methods achieve better performances than the traditional supervised and semi-supervised methods in most of considered datasets.

Iterative Signal Detection for Large-Scale GSM-MIMO Systems

Generalized spatial modulations (GSM) represent a novel multiple input multiple output (MIMO) scheme, which can be regarded as a compromise between spatial multiplexing MIMO and conventional spatial modulations, achieving both spectral efficiency and energy efficiency. Due to the high computational complexity of the maximum likelihood detector in large antenna settings and symbol constellations, in this correspondence paper we propose a lower complexity iterative suboptimal detector. The derived algorithm comprises a sequence of simple processing steps, namely, an unconstrained Euclidean distance minimization problem, an element-wise projection over the signal constellation, and a projection over the set of valid active antenna combinations. To deal with scenarios where the number of possible active antenna combinations is large, an alternative version of the algorithm, which adopts a simpler cardinality projection, is also presented. Simulation results show that, compared with other existing approaches, both versions of the proposed algorithm are effective in challenging underdetermined scenarios where the number of receiver antennas is lower than the number of transmitter antennas.