Quadratic Programming Technique Research Articles

Advantages of Unbiased Support Vector Classifiers for Data Mining Applications

Many learning algorithms have been used for data mining applications, including Support Vector Classifiers (SVC), which have shown improved capabilities with respect to other approaches, since they provide a natural mechanism for implementing Structural Risk Minimization (SRM), obtaining machines with good generalization properties. SVC leads to the optimal hyperplane (maximal margin) criterion for separable datasets but, in the nonseparable case, the SVC minimizes the L1 norm of the training errors plus a regularizing term, to control the machine complexity. The L1 norm is chosen because it allows to solve the minimization with a Quadratic Programming (QP) scheme, as in the separable case. But the L1 norm is not truly an "error counting" term as the Empirical Risk Minimization (ERM) inductive principle indicates, leading therefore to a biased solution. This effect is specially severe in low complexity machines, such as linear classifiers or machines with few nodes (neurons, kernels, basis functions). Since one of the main goals in data mining is that of explanation, these reduced architectures are of great interest because they represent the origins of other techniques such as input selection or rule extraction. Training SVMs as accurately as possible in these situations (i.e., without this bias) is, therefore, an interesting goal. We propose here an unbiased implementation of SVC by introducing a more appropriate "error counting" term. This way, the number of classification errors is truly minimized, while the maximal margin solution is obtained in the separable case. QP can no longer be used for solving the new minimization problem, and we apply instead an iterated Weighted Least Squares (WLS) procedure. This modification in the cost function of the Support Vector Machine to solve ERM was not possible up to date given the Quadratic or Linear Programming techniques commonly used, but it is now possible using the iterated WLS formulation. Computer experiments show that the proposed method is superior to the classical approach in the sense that it truly solves the ERM problem.

Hybrid PSO–SQP for economic dispatch with valve-point effect

This paper presents a novel and efficient method for solving the economic dispatch problem (EDP), by integrating the particle swarm optimization (PSO) technique with the sequential quadratic programming (SQP) technique. PSO is the main optimizer and the SQP is used to fine tune for every improvement in the solution of the PSO run. PSO is a derivative free optimization technique which produces results quickly and proves itself fit for solving large-scale complex EDP without considering the nature of the incremental fuel cost function it minimizes. SQP is a nonlinear programming method which starts from a single searching point and finds a solution using the gradient information. The effectiveness of the proposed method is validated by carrying out extensive tests on three different EDP with incremental fuel cost function takes into account the valve-point loadings effects. The proposed method out-performs and provides quality solutions compared to other existing techniques for EDP considering valve-point effects are shown in general.

Analysis of a Rocking and Walking Punch—Part II: General Numerical Solution Using Quadratic Programming

AbstractThis paper pursues a numerical approach to the solution of the contact problem of a rigid punch on an incompressible half-plane, subjected to a shearing force together with a normal force which may be offset from the centerline of the punch. A piecewise-linear representation of the shear tractions is employed and quadratic programming techniques are used to solve the problem. This method enables an arbitrary load history to be followed. Results are presented which show excellent agreement with other solutions to the special cases of monotonic and steady-state cyclic loading. It is shown that the traction distribution reaches a steady-state cycle after only a few cycles of loading. The existence of an interesting stick-slip regime, where a central zone of slip is bordered by two stick regions is highlighted.

Ants colony algorithm approach for multi-objective optimisation of surface grinding operations

An ant colony based optimisation procedure has been developed to optimise grinding conditions, viz. wheel speed, workpiece speed, depth of dressing and lead of dressing, using a multi-objective function model with a weighted approach for the surface grinding process. The procedure evaluates the production cost and production rate for the optimum grinding condition, subjected to constraints such as thermal damage, wheel wear parameters, machine tool stiffness and surface finish. The results are compared with Genetic Algorithm (GA) and Quadratic Programming (QP) techniques.

Evaluation of optimization techniques for an extractive alcoholic fermentation process.

The mathematical optimization of a continuous alcoholic fermentation process combined with a flash column under vacuum was studied. The objective was to maximize % yield and productivity in the fermentor. The results using surface response analysis combined with modeling and simulation were compared withy those obtained when the problem was written as a nonlinear programming problem and was solved with a successive quadratic programming (SQP) technique. Two process models were evaluated when the process was optimized using the SQP technique. The first one is a deterministic model, whose kinetic parameters were experimentally determined as functions of the temperature, and the second is a statistical model obtained using the factorial design technique combined with simulation. Although the best result was the one obtained using the rigorous model, the values for productivity and % yield obtained using the simplified model are acceptable, and these models can be used when the development of a rigorous model is excessively difficult, slow, or expensive.

Investigation of a System Identification Methodology in the Context of the ASCE Benchmark Problem

This article briefly presents the theory for a system identification and damage detection algorithm for linear systems, and discusses the effectiveness of such a methodology in the context of a benchmark problem that was proposed by the ASCE Task Group in Health Monitoring. The proposed approach has two well-defined phases: (1) identification of a state space model using the Observer/Kalman filter identification algorithm, the eigensystem realization algorithm, and a nonlinear optimization approach based on sequential quadratic programming techniques, and (2) identification of the second-order dynamic model parameters from the realized state space model. Structural changes (damage) are characterized by investigating the changes in the second-order parameters of the reference and damaged models. An extensive numerical analysis, along with the underlying theory, is presented in order to assess the advantages and disadvantages of the proposed identification methodology.

An Analysis of VaR-based Capital Requirements

We study the dynamic investment and reporting problem of a Financial institution subject to capital requirements based on self-reported VaR estimates, as in the Basel Committee's Internal Models Approach (IMA). We characterize the solution of this problem using martingale duality and parametric quadratic programming techniques. With constant price coefficients, we show that optimal portfolios display a local three-fund separation property. VaR-based capital requirements induce financial institutions to tilt their portfolios towards assets with high expected return (and high systematic risk), but result nevertheless in a decrease of the overall risk of trading portfolios. In general, an institution may optimally under-report or over-report its true VaR, depending on its risk aversion and the stringency of capital requirements. Overall, we find that capital requirements determined on the basis of the IMA can be very effective not only in curbing portfolio risk but also in inducing truthful revelation of this risk.

Optimum design of structures with stress and displacement constraints using the force method

A new structural analysis and optimization algorithm is developed to determine the minimum weight of structures with the truss and beam-type members under displacement and stress constraints. The algorithm combines the mathematical programming based on the sequential quadratic programming (SQP) technique and the finite element technique based on the integrated force method. The equilibrium matrix is generated automatically through the finite element analysis while the compatibility matrix is obtained directly using the displacement–deformation relations and the single value decomposition (SVD) technique. By combining the equilibrium and compatibility matrices with the force–displacement relations, the equations of equilibrium with the element forces as variables are obtained. The proposed method is extremely efficient to analyze and optimize the truss and beam structures under stress and displacement constraints. The computational effort required by the force method is found to be significantly lower than that of the displacement method. The effect of the geometric nonlinearity in the structural optimization problems under the stress and displacement constraints were also investigated and it is illustrated that the geometric nonlinearity is not an important issue in these types of problems and hence, it does not affect the final optimum solution significantly. Four examples illustrate the procedure and allow the results to be compared with those reported in the literatures.

A sequential quadratic programming with a dual parametrization approach to nonlinear semi-infinite programming

In the sequel of the work reported in Liu et al. (1999), in which a method based on a dual parametrization is used to solve linear-quadratic semi-infinite programming (SIP) problems, a sequential quadratic programming technique is proposed to solve nonlinear SIP problems. A merit function to measure progress toward the solution and a procedure to compute the penalty parameter are also proposed.

Nonlinear System Identification Tools Applied to the Modelling of Submarine Dynamics

This paper presents initial results on the use of nonlinear systems identification algorithms to estimate the hydrodynamic coefficients in fully coupled, nonlinear submarine motion equations. Parameter identification algorithms rely on the sequential quadratic programming technique. The paper briefly describes the form of the non-linear equations used in the submarine simulation, describes the system identification algorithms developed and then illustrates their application to three test cases of interest. These are based on horizontal motion only and aim to identify the model coefficients in the cases when these are completely unknown, partially known, and partially known but with an error on the known values respectively.

On the Identification of the Parameters of a Viscoelastic Material Law for Polymeric Foams

AbstractIn this paper, the gradient‐based sequential quadratic programming (SQP) technique is used to determine the parameters of a finite viscoelastic material law for the description of polymeric foams. Due to the cellular structure of foams, the resulting non‐linear behaviour is reproduced by an Ogden‐type law in the framework of the Theory of Porous Media (TPM) [1]. The conformity between experimental data and numerical simulation is improved by the minimization of a least squares functional under consideration of equality and inequality constraints. The result is shown by a finite element based back‐analysis of unconfined compression tests.

On finding global optima for the hinge fitting problem

This paper considers the data fitting of n given points in R m by a hinge function, as it appears in Breiman (IEEE Trans. Inform. Theory 39(3) (1993) 999) and Pucar and Sjöberg (IEEE Trans. Inform. Theory 44(3) (1998) 1310). This problem can be seen as a mathematical programming problem with a convex objective function and equilibrium constraints. For the euclidean error, an enumerative approach is proposed, which is a polynomial method in the sample size n, for a fixed dimension m. An alternative formulation for the l 1 error is also introduced, which is processed by a Sequential Linear Complementarity Problem approach. Some numerical results with both algorithms are included to highlight the efficiency of those procedures. Scope and purpose The problem addressed here is found in applications of neural networks, nonlinear regression and data classification. Due to its nonconvex character, the application of classical nonlinear programming tools lead only to (usually suboptimal) local minima. This work proposes a global optimization method for the Hinge Fitting problem, combining combinatorial optimization and quadratic programming techniques. By achieving global optimality in reasonable time, the method compares favorably to previously proposed approaches.

Optimization of the Growth Rate of Probiotics in Fermented Milk Using Genetic Algorithms and Sequential Quadratic Programming Techniques

Prebiotics (peptides, N-acetyglucoamine, fructo-oligosaccharides, isomalto-oligosaccharides and galacto- oligosaccharides) were added to skim milk in order to improve the growth rate of contained Lactobacillus acidophilus, Lactobacillus casei, Bifidobacterium longum and Bifidobacterium bifidum. The purpose of this research was to study the potential synergy between probiotics and prebiotics when present in milk, and to apply modern optimization techniques to obtain optimal design and performance for the growth rate of the probiotics using a response surface-modeling technique. To carry out response surface modeling, the regression method was performed on experimental results to build mathematical models. The models were then formulated as an objective function in an optimization problem that was consequently optimized using a genetic algorithm and sequential quadratic programming approach to obtain the maximum growth rate of the probiotics. The results showed that the quadratic models appeared to have the most accurate response surface fit. Both SQP and GA were able to identify the optimal combination of prebiotics to stimulate the growth of probiotics in milk. Comparing both methods, SQP appeared to be more efficient than GA at such a task. (Asian-Aust. J. Anim. Sci. 2003. Vol 16, No. 6 : 894-902)

A multi-objective genetic algorithm (GA) approach for optimization of surface grinding operations

A genetic algorithm (GA) based optimization procedure has been developed to optimize grinding conditions, viz. wheel speed, workpiece speed, depth of dressing and lead of dressing, using multi-objective function model with a weighted approach for surface grinding process. The procedure evaluates the production cost and production rate for the optimum grinding condition, subjected to constraints such as thermal damage, wheel wear parameters, machine tool stiffness and surface finish. New GA procedure is illustrated with an example and optimum results such as production cost, surface finish, metal removal rate are compared with quadratic programming techniques.

Gradient-based optimization of a viscoelastic Ogden type model for cellular polymers

The parameter identification is the interface between a theoretical material model and its application in numerical computations. Only by an accurate identification of the theoretically introduced material parameters, an applicable simulation of the material is achieved. An increasing standard of the parameter identification is set by the requirements of complex material models used in computer-aided engineering. A common identification strategy is a gradient-based optimization of a least-squares functional, e. g. the sequential quadratic programming (SQP) technique. In this paper, the SQP method is used to optimize material models of cellular polymers. In particular, the optimization is shown for a viscoelastic polyurethane (PU) foam. Due to the high-grade nonlinear material behaviour, the foam is modelled by a finite viscoelastic Ogden type law in the framework of the Theory of Porous Media (TPM).

Evaluation of modal transformation matrices for overhead transmission lines and underground cables by optimization method

The paper develops a numerical procedure based on constrained optimization for evaluating the modal transformation matrices of overhead transmission lines and underground cables. In the method, the eigenvalue-eigenvector equation is transformed into an objective function which is then minimized for finding eigenvalues and eigenvectors. The method developed avoids completely the problem of eigenvalue switching which causes the discontinuities of eigenvectors encountered in the QR-method. Initial values for starting the optimization at each frequency are the eigenvalues and eigenvectors evaluated at the preceding frequency. This technique leads to the set of smooth eigenvectors that define the modal transformation matrices. The form of the objective function derived in the paper for minimization together with the sequential quadratic programming technique guarantees that convergence to valid eigenvalues and eigenvectors is achieved. Using the method, transformation matrices for a wide range of frequency for representative double-circuit line and underground cable are evaluated.

An Interior Point Constrained Trust Region Method for a Special Class of Nonlinear Semidefinite Programming Problems

In this paper, an interior point trust region algorithm for the solution of a class of nonlinear semidefinite programming (SDP) problems is described and analyzed. Such nonlinear and nonconvex programs arise, e.g., in the design of optimal static or reduced order output feedback control laws and have the structure of abstract optimal control problems in a finite dimensional Hilbert space. The algorithm treats the abstract states and controls as independent variables. In particular, an algorithm for minimizing a nonlinear matrix objective functional subject to a nonlinear SDP-condition, a positive definiteness condition, and a nonlinear matrix equation is considered. The algorithm is designed to take advantage of the structure of the problem. It is an extension of an interior point trust region method to nonlinear and nonconvex SDPs, with a special structure which applies sequential quadratic programming techniques to a sequence of barrier problems and uses trust regions to ensure robustness of the iteration. Some convergence results are given, and, finally, several numerical examples demonstrate the applicability of the considered algorithm.

Structural damage identification using piezoelectric sensors

In this paper, a two-stage damage identification method has been proposed using the data obtained from piezoelectric sensors. In the first stage, a first-order approximation technique is presented for numerically predicting the transient response of electrical potential change on sensors caused by damages. Both numerical and experimental data in the time domain are transformed into the frequency domain using the FFT technique. The damage locations can then be detected by matching the numerical data and the experimental data in the frequency domain through a proposed detection technique. After identifying the possible damage locations, in the second stage, an iterative estimation scheme for solving nonlinear optimization programming problems, based on the quadratic programming technique, is put forward to predict damage extents. A beam example has been employed to illustrate the effectiveness of the algorithm numerically. Furthermore, various investigations, such as the accuracy of the proposed first-order approximation technique, the influences of the excitation frequency of external force, and modelling errors and measurement noises on the results have been carried out.

Evaluation of Modal Transformation Matrices for Overhead Transmission Lines and Underground Cables by Optimization Methods

This article develops a numerical procedure based on constrained optimization for evaluating the modal transformation matrices of overhead transmission lines and underground cables. In the method, the eigenvalue-eigenvector equation is transformed into an objective function, which is then minimized for finding eigenvalues and eigenvectors. The developed method completely avoids the problem of eigenvalue switching, which causes the discontinuities of eigenvectors encountered in the QR method. Initial values for starting the optimization at each frequency are the eigenvalues and eigenvectors evaluated at the preceding frequency. This technique leads to the set of smooth eigenvectors that define the modal transformation matrices. The form of the objective function derived in the paper for minimization, together with the sequential quadratic programming technique, guarantees that convergence to valid eigenvalues and eigenvectors is achieved. Using the method, transformation matrices for a wide range of frequency for representative double-circuit line and underground cable are evaluated.

Optimal weight design of rotor systems with oil-film bearings subjected to frequency constraints

The purpose of this study is to investigate optimal weight design of rotors. The analysis results of this work for dynamic analysis of rotors are compared to those available in the literature. An excellent agreement is observed, which validates the formulation and implementation of the model developed. Further analysis is conducted for rotors with oil-film bearings. The stability of the system is studied by examining the real parts of the eigenvalues and the logarithmic decrement. The sequential quadratic programming technique is used for the minimum weight design. The effect of minimum weight design on the distribution of rotor shape, weight saving, and system stability is examined. It is demonstrated that significant reduction of rotor weight can be achieved. Weight reduction of more than 80% is possible for a simple rotor analyzed, whereas for a realistic complex rotor with oil-lubricated bearings, an original rotor weight of 5.4886 kg can be reduced to 1.7267 kg after being optimized.