Unconstrained Minimization Problem Research Articles

Numerical solution of systems of non linear equations defined by convex functions

In this paper, the problem of numerical solution of systems of non linear equations, defined by convex differentiable functions, is studied. With this system, non-smooth (non-differentiable) unconstrained minimization problems are associated, with objective functions, which are based on the discrete l 1 - and l∞ - norm, respectively, that is, using these two norms as proximity criteria. Also, a “differentiable” unconstrained minimization problem (least squares data fitting problem) is associated with the considered system of non linear equations, with objective function, which is based on the discrete l 2 - norm. The special case when non linear equations are defined by separable convex differentiable functions is also studied. The considered system of non linear equations is solved, in the case of l 1 - norm, by minimizing the sum of absolute values of residuals, and, in the case of l∞ - norm, by minimizing the maximal residual, respectively. In the case of l 2 - norm, the system of non linear equations is solved by minimizing the sum of squares of the residuals. Subgradients of the objective functions of the associated unconstrained minimization problems are calculated, and the subgradient method for solving these problems is described. The gradient method for solving the corresponding “differentiable” discrete least squares problem, which is based on l 2 - norm, is also presented, and gradients of the objective functions of the corresponding “differentiable” unconstrained minimization problems are calculated.

Maximum Penalty Function in Linear Programming

A linear program can be equivalently reformulated as an unconstrained nonsmooth minimization problem, whose objective is the sum of the original objective and a penalty function with a sufficiently large penalty parameter. The article presents two methods for choosing this parameter. The first one applies to linear programs with usual linear inequality constraints. Then, we use a corresponding theorem by N.Z. Shor on the equivalence of a convex program to an unconstrained nonsmooth minimization problem. The second method is for linear programs of a special type. This means that all inequalities are of the form that a linear expression on the left-hand side is less or equal to a positive constant on the right-hand side. For this special type, we use a corresponding theorem of B.N. Pshenichny on establishing a penalty parameter for convex programs. For differently sized linear programs of the special type, we demonstrate that suitable penalty parameters can be computed by a procedure in GNU Octave based on GLPK software.

HSH-norm optimal MOR for the MIMO linear time-invariant systems on the Stiefel manifold

This paper focuses on the Hilbert–Schmidt-Hankel-norm ( norm) optimal model order reduction (MOR) of large-scale multiple-input multiple-output (MIMO) linear time-invariant (LTI) systems on the Stiefel manifold. First, a cost function is constructed in regard to the norm. By introducing the orthogonality constraints, the norm optimal MOR problem is converted into an unconstrained minimisation problem on the Stiefel manifold. We derive the Riemannian gradient of the cost function on the Stiefel manifold. The MOR algorithm is developed associated with the Riemannian conjugate gradient method. Global convergence is guaranteed with mild conditions. Finally, the effectiveness of the proposed method is illustrated by two numerical examples.

On asymptotically stabilizing control of nonlinear fractional control systems using an optimization scheme

In this paper, stabilization of the nonlinear fractional order systems with unknown control coefficients is considered where the dynamic control system depends on the Caputo fractional derivative. Related to the nonlinear fractional control (NFC) system, an infinite-horizon optimal control (OC) problem is first proposed. It is shown that the obtained OC problem can be an asymptotically stabilizing control for the NFC system. Using the help of an approximation, the Caputo derivative is replaced with the integer order derivative. The achieved infinite-horizon OC problem is then converted into an equivalent finite-horizon one. According to the Pontryagin minimum principle for OC problems and by constructing an error function, an unconstrained minimization problem is defined. In the optimization problem, trial solutions are used for state, costate and control functions where these trial solutions are constructed by using a two-layered perceptron neural network. A learning algorithm with convergence properties is also provided. Two numerical results are introduced to explain the main results.

A novel bearing-only localization for generalized Gaussian noise

A maximum likelihood estimator for bearing only source localization problem is addressed. Bearing measurements are assumed to be corrupted by additive generalized Gaussian noise, making the proposed estimator capable of keeping its performance high for a range of noise densities, i.e., from highly impulsive to uniform noises. Most previous works rely on small noise approximation and adopt recursive methods; therefore they need a good initial guess and high computational load. The novelty of this paper is to propose a non-iterative importance sampling based method to find a solution to the unconstrained lp-norm minimization problem for highly non-linear cost functions not known before. Our estimator is simple, does not need derivation or matrix multiplication operations, and requires generating a fair number of random particles.

Handling Ensemble N-Representability Constraint in Explicit-by-Implicit Manner.

The convergence issues caused by the improper treatment of the ensemble N-representability constraint have severely affected the applicability and reproducibility of reduced density matrix functional theory (RDMFT). Unlike the commonly used Lagrange methods explicitly bringing the constraint into the objective functions, we present a different idea to handle the constraint in an implicit manner, which is achieved by introducing implicit functions to exactly consider the nonlinear unclear connection embedded in the explicit constraint. This explicit-by-implicit idea, denoted as EBI, thus transforms the constrained optimization problem into an unconstrained minimization problem. The tests on different systems, initial guesses, and functionals demonstrate the superiority of EBI in the treatment of the ensemble N-representability constraint. Therefore, EBI solves the convergence issues of the Lagrange methods, which is essential for further development and application of RDMFT. Besides, the idea of EBI is helpful for the treatment of different constrained problems in modern physics and chemistry.

Chaos and intensification enhanced flower pollination algorithm to solve mechanical design and unconstrained function optimization problems

Nature-inspired computation has enjoyed a visible position among the soft computational techniques. Flower Pollination Algorithm (FPA), which is known as one of the outstanding algorithms in this domain, has been shown to be promising in numerous publications. FPA is comprised of two main phases, which are referred to as abiotic and biotic pollination that correspond to local and global search, respectively. It makes use of a user-supplied parameter to switch between them. This parameter is referred to as the switching probability. Thus, it can be put forward that switching probability defines the search characteristic of FPA. The present work introduces several FPA modifications that adopt chaotic maps. Moreover, the developed modifications are further enhanced by using intensifying step sizing procedure that allows a more intensified search towards the end of search. With the help of the introduced chaos in switching probability and incrementally intensifying search, developed FPA modifications are expected to find the hard-to-detect promising regions. Next, such capabilities of chaotic maps are utilized in various building blocks of FPA. Performances of all developed FPA modifications are analysed on the well-known unconstrained real-valued function minimization and mechanical design problems. Finally, appropriate non-parametric statistical analysis is carried out to observe possible statistically significant improvements over the standard FPA. As shown by the experimental study, obtained results induce success of the developed procedures, which clearly add to the capability of the canonical FPA.

Stabilisation of distributed time-delay systems: a smoothed spectral abscissa optimisation approach

We address the stabilisation problem of linear time-invariant systems with pointwise and distributed delays. We first introduce the concept of the smoothed spectral abscissa, a robust stability measure previously studied for pointwise delay systems. We show that the new stability measure for distributed systems is smooth with respect to the system parameters, and robust in the sense that it provides a trade-off between the decay rate of the solution and the norm of the system. Next, we reformulate the stabilisation problem as an unconstrained minimisation problem considering the smoothed spectral abscissa as the objective function, and show that its properties enable solving the stabilisation problem by using standard gradient-based optimisation techniques. We show the potential of the proposed approach by using it for the stabilisation of a fleet of vehicles, and an oscillator with a class of time-varying delays in the input.

Numerical solution of some systems of nonlinear algebraic equations

The problem of numerical solution of systems of nonlinear algebraic equations with even powers is studied in this paper. Nondifferentiable (nonsmooth) unconstrained minimization problems are associated with the system under consideration, with objective functions, based on discrete l 1 - and l ∞ - norm, respectively, by using these two norms as proximity criteria, and the considered system is solved by minimizing the sum of absolute values of residuals, in the case of l 1 - norm, or by minimizing the maximal residual, in the case of l ∞ - norm, respectively. Also, a “differentiable” unconstrained minimization problem (least squares problem) is associated with the considered system of nonlinear equations, with objective function, based on the discrete l 2 -norm, and the considered system is solved by minimizing the sum of squares of the residuals. Subgradients of the objective functions of unconstrained minimization problems are calculated, and a subgradient method for solving the associated problems is presented. The iterative gradient method for solving the corresponding “differentiable” discrete least squares problem, based on l 2 - norm, is also outlined, and gradients of the objective function of the corresponding “differentiable” unconstrained minimization problems are calculated.

Use of the Shor’s r-Algorithm in Linear Robust Optimization Problems

The paper is devoted to the description of a new approach to the construction of algorithms for solving linear programming problems (LP-problems), in which the number of constraints is much greater than the number of variables. It is based on the use of a modification of the r-algorithm to solve the problem of minimizing a nonsmooth function, which is equivalent to LP problem. The advantages of the approach are demonstrated on the linear robust optimization problem and the robust parameters estimation problem using the least moduli method. The developed octave programs are designed to solve LP problems with a very large number of constraints, for which the use of standard software from linear programming is either impossible or impractical, because it requires significant computing resources. The material of the paper is presented in three sections. In the first section for the problem of minimizing a convex function we describe a modification of the r-algorithm with a constant coefficient of space dilation in the direction of the difference of two successive subgradients and an adaptive method for step size adjustment in the direction of the antisubgradient in the transformed space of variables. The software implementation of this modification is presented in the form of Octave function ralgb5a, which allows to find or approximation of the minimum point of a convex function, or approximation of the maximum point of the concave function. The code of the ralgb5a function is given with a brief description of its input and output parameters. In the second section, a method for solving the LP problem is presented using a nonsmooth penalty function in the form of maximum function and the construction of an auxiliary problem of unconstrained minimization of a convex piecewise linear function. The choice of the finite penalty coefficient ensures equivalence between the LP-problem and the auxiliary problem, and the latter is solved using the ralgb5a program. The results of computational experiments in GNU Octave for solving test LP-problems with the number of constraints from two hundred thousand to fifty million and the number of variables from ten to fifty are presented. The third section presents least moduli method that is robust to abnormal observations or "outliers". The method uses the problem of unconstrained minimization of a convex piecewise linear function, and is solved using the ralgb5a program. The results of computational experiments in GNU Octave for solving test problems with a large number of observations (from two hundred thousand to five million) and a small number of unknown parameters (from ten to one hundred) are presented. They demonstrate the superiority of the developed programs over well-known linear programming software such as the GLPK package. Keywords: robust optimization, linear programming problem, nonsmooth penalty function, r-algorithm, least modulus method, GNU Octave.

An efficient and globally optimal solution to perspective-n-line problem

This research develops an accurate and efficient method for the Perspective-n-Line (PnL) problem. The developed method addresses and solves PnL via exploiting the problem’s geometry in a non-linear least squares fashion. Specifically, by representing the rotation matrix with a novel quaternion parameterization, the PnL problem is first decomposed into four independent subproblems. Then, each subproblem is reformulated as an unconstrained minimization problem, in which the Kronecker product is adopted to write the cost function in a more compact form. Finally, the Gröbner basis technique is used to solve the polynomial system derived from the first-order optimality conditions of the cost function. Moreover, a novel strategy is presented to improve the efficiency of the algorithm. It is improved by exploiting structure information embedded in the rotation parameterization to accelerate the computing of coefficient matrix of a cost function. Experiments on synthetic data and real images show that the developed method is comparable to or better than state-of-the-art methods in accuracy, but with reduced computational requirements.

Generalized Twin Support Vector Machines

In this paper, we propose two efficient approaches of twin support vector machines (TWSVM). The first approach is to reformulate the TWSVM formulation by introducing $$L_1$$ and $$L_\infty $$ norms in the objective functions, and convert into linear programming problems termed as LTWSVM for binary classification. The second approach is to solve the primal TWSVM, and convert into completely unconstrained minimization problem. Since the objective function is convex, piecewise quadratic but not twice differentiable, we present an efficient algorithm using the generalized Newton’s method termed as GTWSVM. Computational comparisons of the proposed LTWSVM and GTWSVM on synthetic and several real-world benchmark datasets exhibits significantly better performance with remarkably less computational time in comparison to relevant baseline methods.

A partially proximal linearized alternating minimization method for finding Dantzig selectors

The Dantzig selector (DS) is an efficient estimator designed for high-dimensional linear regression problems, especially for the case where the number of samples n is much less than the dimension of features (or variables) p. In this paper, we first reformulate the underlying DS model as an unconstrained minimization problem of the sum of two nonsmooth convex functions and a smooth coupled function. Then by exploiting the structure of the resulting model, we propose a partially proximal linearized alternating minimization method (P-PLAM), whose two subproblems are easy enough with closed-form solutions. Another remarkable advantage is that P-PLAM requires only one starting point, which is potentially helpful for saving computing time. A series of computational experiments on synthetic and real-world data sets demonstrate that the proposed P-PLAM has promising numerical performance in the sense that P-PLAM can obtain higher quality solutions by taking less computing time than some existing state-of-the-art first-order solvers.

Spatial Degrees of Freedom Exploration and Analog Beamforming Designs for Signature Spatial Modulation

In this paper, we focus on developing efficient multi-configuration selection mechanisms by exploiting the spatial degrees of freedom (DoF), and leveraging the simple design benefits of spatial modulation (SM). Notably, the SM technique, as well as its variants, faces the following critical challenges: (i) the performance degradation and difficulty in improving the system performance for higher-level QAM constellations, and (ii) the vast complexity cost in precoder designs particularly for the increasing system dimension and amplitude-phase modulation (APM) constellation dimension. Given this situation, we first investigate two independent modulation domains, i.e., the original signal- and spatial-constellations. By exploiting the analog shift weighting and the virtual spatial signature technologies, we introduce the signature spatial modulation (SSM) concept, which is capable of guaranteing superior trade-offs among spectral- and cost-efficiencies, and system bit error rate (BER) performance. Besides, we develop an analog beamforming for SSM by solving the introduced unconstrained Lagrange dual function minimization problem. Numerical results manifest the performance gain brought by our developed analog beamforming for SSM.

On robust asymmetric Lagrangian [formula omitted]-twin support vector regression using pinball loss function

The main objective of twin support vector regression (TSVR) is to find the optimum regression function based on the ε-insensitive up- and down-bound with equal influences on the regression function where all the data points have a different location above the up-bound points and below the down-bound points. However, the effects of all data points must be distinct based on their distribution in the regression function. Recently, asymmetric ν-twin support vector regression (Asy-ν-TSVR) is encouraged on the same subject but still, the present matrices in the mathematical formulation have faced the problem of semi-definite. In order to handle this problem effectively, a new regressor model named as robust asymmetric Lagrangian ν-twin support vector regression using pinball loss function (URALTSVR) proposes as a pair of the unconstrained minimization problem to handle not only the noise sensitivity and instability of re-sampling but also consist positive definite matrices. Here, we suggest the proposed model URALTSVR in such a way where the pinball loss function is playing a vital role to control the fitting error inside the asymmetric tube. One of the advantages is that unlike TSVR and Asy-ν-TSVR, it considers the concept of structural risk minimization principle through the inclusion of regularization term as well as change the one-norm of the vector of the slack variable by two-norm, which yields the dual problem to be strongly convex, stable and well-posed. Aforementioned, the proposed formulation has a continuous and piecewise quadratic problem that is solved by their gradients based iterative approaches. Specifically, we analyze the three implementations of URALTSVR with the baselines approaches support vector regression (SVR), TSVR and Asy-ν-TSVR, which discard the dependencies to solve a pair of quadratic programming problem (QPP) for obtaining the unique global solution. Overall, SRALTSVR1 based on smooth approximation function performs outstanding for artificial and real-world datasets.

Regularized based implicit Lagrangian twin extreme learning machine in primal for pattern classification

In this paper, we suggest a novel approach termed as regularized based implicit Lagrangian twin extreme learning machine in primal as a pair of unconstrained convex minimization problem (RILTELM) where regularization term is added to follow the structural risk minimization principle. Here, we consider 2-norm of the slack vector of variables to make the problem strongly convex which results in a unique solution. Since it has non-smooth plus functions in their objective function, so we find an approximate solution by replacing the non-smooth plus function with smooth approximation function because to find an approximation solution in primal space is always superior to its dual. Due to non-smooth plus function, we solve the problem by either smooth approximation approach or generalized derivative approach. In addition, a functional iterative scheme is also suggested to find the optimal solution. Hence, no external optimization toolbox is required unlike in twin extreme learning machine (TELM) and twin support vector machine (TWSVM). The numerical experiments are demonstrated on artificial and real-world datasets and compared with TWSVM, ELM, TELM and LSTELM to establish the efficacy and applicability of proposed RILTELM.

Image Reconstruction Algorithm Based on Total Least Squares Target Correction for ECT.

In the image reconstruction of the electrical capacitance tomography (ECT) system, the application of the total least squares theory transforms the ill-posed problem into a nonlinear unconstrained minimization problem, which avoids calculating the matrix inversion. But in the iterative process of the coefficient matrix, the ill-posed problem is also produced. For the effect on the final image reconstruction accuracy of this problem, combined with the principle of the ECT system, the coefficient matrix is targeted and updated in the overall least squares iteration process. The new coefficient matrix is calculated, and then, the regularization matrix is corrected according to the adaptive targeting singular value, which can reduce the ill-posed effect. In this study, the total least squares iterative method is improved by introducing the mathematical model of EIV to deal with the errors in the measured capacitance data and coefficient matrix. The effect of noise interference on the measurement capacitance data is reduced, and finally, the high-quality reconstructed images are calculated iteratively.

A New Spectral on the Gradient Methods for Unconstrained Optimization Minimization Problem

One simple and well-known method for minimizing the functions is a spectral conjugate gradient method. In this paper, we derive a new spectral on the method of gradient, which can give a new path of search. The new spectral method holds the property of descent and we have shown that the spectral method is convergent globally. The empirical results show that for the test problems, the given approach is competitive with the other conjugate gradient methods.

On fractional optimal control problems with an application in fractional chaotic systems using a Legendre collocation-optimization technique

In this paper, an intelligence method based on single layer legendre neural network is proposed to solve fractional optimal control problems where the dynamic control system depends on Caputo fractional derivatives. First, with the help of an approximation, the Caputo derivative is replaced to integer order derivative. According to the Pontryagin minimum principle for optimal control problems and by constructing an error function, an unconstrained minimization problem is then defined. In the optimization problem, trial solutions are used for state, costate and control functions, where these trial solutions are constructed by using Legendre polynomial based functional link artificial neural network. In the following, error back propagation algorithm is used for updating the network parameters (weights). At the end, some illustrative examples are included to demonstrate the validity and capability of the proposed method. Three applicable examples about chaos control of Malkus waterwheel, finance fractional chaotic models and fractional-order geomagnetic field models are also considered.

Efficient implicit Lagrangian twin parametric insensitive support vector regression via unconstrained minimization problems

In this paper, an efficient implicit Lagrangian twin parametric insensitive support vector regression is proposed which leads to a pair of unconstrained minimization problems, motivated by the works on twin parametric insensitive support vector regression (Peng: Neurocomputing. 79, 26–38, 2012), and Lagrangian twin support vector regression (Balasundaram and Tanveer: Neural Comput. Applic. 22(1), 257–267, 2013). Since its objective function is strongly convex, piece-wise quadratic and differentiable, it can be solved by gradient-based iterative methods. Notice that its objective function having non-smooth ‘plus’ function, so one can consider either generalized Hessian, or smooth approximation function to replace the ‘plus’ function and further apply the simple Newton-Armijo step size algorithm. These algorithms can be easily implemented in MATLAB and do not require any optimization toolbox. The advantage of this method is that proposed algorithms take less training time and can deal with data having heteroscedastic noise structure. To demonstrate the effectiveness of the proposed method, computational results are obtained on synthetic and real-world datasets which clearly show comparable generalization performance and improved learning speed in accordance with support vector regression, twin support vector regression, and twin parametric insensitive support vector regression.