Unconstrained Minimization Problem Research Articles

A fictitious domain decomposition method for a nonlinear bonded structure

We study a fictitious domain decomposition method for a nonlinearly bonded structure. Starting with a strongly convex unconstrained minimization problem, we introduce an interface unknown such that the displacement problems on each subdomain become uncoupled in the saddle-point equations. The interface unknown is eliminated and a Uzawa conjugate gradient domain decomposition method is derived from the saddle-point equations of the stabilized Lagrangian functional. To avoid interface fitted meshes we use a fictitious domain approach, inspired by XFEM, which consists in cutting the finite element basis functions around the interface. Some numerical experiments are proposed to illustrate the efficiency of the proposed method.

On pairing Huber support vector regression

In this paper, a novel and efficient pairing support vector regression learning method using ε−insensitive Huber loss function (PHSVR) is proposed where the ε−insensitive zone having flexible shape is determined by tightly fitting the training samples. Our approach leads to solving a pair of unconstrained minimization problems in primal and the solutions are obtained by two algorithms: a functional iterative (FPHSVR) and Newton iterative (NPHSVR) algorithms. The finite termination of the Newton method to its global minimum solution is proved. The significant advantages of the proposed method are the robustness, generalization ability and learning speed. Experiments performed on a series of synthetic data sets, polluted by different types of noise including heteroscedastic noise and outliers, and on real-world benchmark data sets confirm the effectiveness and superiority of the proposed method.

Semi-supervised Weighted Ternary Decision Structure for Multi-category Classification

Semi-supervised learning has attracted researchers due to its advantages over supervised learning. In this paper, an extremely fast multi-category classification algorithm, termed as weighted ternary decision structure (WTDS) is proposed. WTDS is a generic algorithm that can extend any binary classifier into multi-category framework. This work also proposes a novel semi-supervised binary classifier termed as Weighted Laplacian least-squares twin support vector machine which is further extended using WTDS. The novel semi-supervised classifier obtains the solution by formulating a pair of Unconstrained Minimization Problems which are solved as systems of linear equation. WTDS takes advantage of the strengths of the classifier and efficiently constructs the multi-category classifier model in the form of a decision structure. WTDS outperforms other state-of-the-art multi-category approaches in terms of classification accuracy and time complexity. To confirm the feasibility and efficacy of proposed algorithm, experiments are conducted on benchmark UCI datasets.

Sensitivity analysis and lattice density optimization for sequential inherent strain method used in additive manufacturing process

Compensation of the thermal distortion that occurs during the fabrication process is an important issue in the field of metal additive manufacturing. Considering the problem in forming a lattice structure inside an object to reduce the thermal distortion, we developed a lattice volume fraction distribution optimization method. Assuming that the linear elastic problem is solved using the finite element method (FEM), an inherent strain method applying a layer-by-layer process utilizing the element activation during the FEM is formed as a recurrence relation, and the sensitivity of an objective function is derived based on the adjoint method. The unit lattice shape is a simple cube with a cube or a sphere-shaped air hole, and its distribution is optimized by considering the minimum thickness of the wall surrounding it as a design variable. The effective stiffness tensor of the lattice is derived using a homogenization method. The functions of the effective properties with respect to the design variables are approximated through polynomial functions. The optimization problem is formulated as an unconstrained minimization problem. The design variables are optimized using the method of moving asymptotes. Herein, the validity of the proposed method is discussed based on quasi two-dimensional and three-dimensional numerical studies including a re-analysis through full-scale thermo-mechanical analysis.

An inexact scalarization proximal point method for multiobjective quasiconvex minimization

In this paper we present an inexact scalarization proximal point algorithm to solve unconstrained multiobjective minimization problems where the objective functions are quasiconvex and locally Lipschitz. Under some natural assumptions on the problem, we prove that the sequence generated by the algorithm is well defined and converges. Then providing two error criteria we obtain two versions of the algorithm and it is proved that each sequence converges to a Pareto–Clarke critical point of the problem; furthermore, it is also proved that assuming an extra condition, the convergence rate of one of these versions is linear when the regularization parameters are bounded and superlinear when these parameters converge to zero.

A Stochastic Derivative-Free Optimization Method with Importance Sampling: Theory and Learning to Control

We consider the problem of unconstrained minimization of a smooth objective function in ℝn in a setting where only function evaluations are possible. While importance sampling is one of the most popular techniques used by machine learning practitioners to accelerate the convergence of their models when applicable, there is not much existing theory for this acceleration in the derivative-free setting. In this paper, we propose the first derivative free optimization method with importance sampling and derive new improved complexity results on non-convex, convex and strongly convex functions. We conduct extensive experiments on various synthetic and real LIBSVM datasets confirming our theoretical results. We test our method on a collection of continuous control tasks on MuJoCo environments with varying difficulty. Experiments show that our algorithm is practical for high dimensional continuous control problems where importance sampling results in a significant sample complexity improvement.

Approximate versions of proximal iteratively reweighted algorithms including an extended IP-ICMM for signal and image processing problems

Iteratively reweighted algorithms are popular methods for solving nonconvex unconstrained minimization problems. Applications are notably mathematical models in image processing or signal processing. They often have a convex subproblem and do not have closed form solutions in general. In this paper, we propose approximate versions of proximal iteratively reweighted algorithms for nonconvex and nonsmooth unconstrained minimization problems. Specifically, we can achieve an approximate solution for the subproblem by applying a computable inexact stopping rule. The convergence of our method is proved based on an inexact unified framework. Numerical applications for image deblurring or denoising problems validate the effectiveness of the proposed algorithms.

A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing

Combining the projection method of Solodov and Svaiter with the Liu-Storey and Fletcher Reeves conjugate gradient algorithm of Djordjević for unconstrained minimization problems, a hybrid conjugate gradient algorithm is proposed and extended to solve convex constrained nonlinear monotone equations. Under some suitable conditions, the global convergence result of the proposed method is established. Furthermore, the proposed method is applied to solve the ℓ1-norm regularized problems to restore sparse signal and image in compressive sensing. Numerical comparisons of the proposed algorithm versus some other conjugate gradient algorithms on a set of benchmark test problems, sparse signal reconstruction and image restoration in compressive sensing show that the proposed scheme is computationally more efficient and robust than the compared schemes.

On fractional infinite-horizon optimal control problems with a combination of conformable and Caputo–Fabrizio fractional derivatives

Fractional calculus is a powerful and effective tool for modeling of nonlinear systems. In this paper, we first introduce a modification of conformable fractional derivative to solve fractional infinite horizon optimal control problems. We point out that the term t1−α in definition of conformable derivative has some disadvantages and thus must be refined. Using an interesting property of relationship between the presented fractional derivative and the usual first-order derivative, fractional dynamic system in the infinite horizon optimal control problem is transformed into a non-fractional one. By a suitable change of variable, the obtained infinite horizon problem is reduced to a finite-horizon one. According to the Pontryagin minimum principle for optimal control problems and by constructing an error function, an unconstrained minimization problem is then defined. In the achieved minimization problem, trial solutions for state, co-state and control functions are utilized where these trial solutions are constructed by using two-layered perceptron neural network. Some numerical results are solved to explain our main results. Two applicable examples as stabilization and chaos control of fractional order systems are also provided.

Fractional Chebyshev functional link neural network‐optimization method for solving delay fractional optimal control problems with Atangana‐Baleanu derivative

SummaryIn this article, we propose a higher order neural network, namely the functional link neural network (FLNN), for the model of linear and nonlinear delay fractional optimal control problems (DFOCPs) with mixed control‐state constraints. We consider DFOCPs using a new fractional derivative with nonlocal and nonsingular kernel that was recently proposed by Atangana and Baleanu. The derivative possesses more important characteristics that are very useful in modelling. In the proposed method, a fractional Chebyshev FLNN is developed. At the first step, the delay problem is transformed to a nondelay problem, using a Padé approximation. The necessary optimality condition is stated in a form of fractional two‐point boundary value problem. By applying the fractional integration by parts and by constructing an error function, we then define an unconstrained minimization problem. In the optimization problem, trial solutions for state, co‐state and control functions are utilized where these trial solutions are constructed by using single‐layer fractional Chebyshev neural network model. We then minimize the error function using an unconstrained optimization scheme based on the gradient descent algorithm for updating the network parameters (weights and bias) associated with all neurons. To show the effectiveness of the proposed neural network, some numerical results are provided.

Unconstrained convex minimization based implicit Lagrangian twin extreme learning machine for classification (ULTELMC)

The recently proposed twin extreme learning machine (TELM) requires solving two quadratic programming problems (QPPs) in order to find two non-parallel hypersurfaces in the feature that brings in the additional requirement of external optimization toolbox such as MOSEK. In this paper, we propose implicit Lagrangian TELM for classification via unconstrained convex minimization problem (ULTELMC) and further suggest iterative convergent schemes which eliminates the requirement of external optimization toolbox generally required in solving the quadratic programming problems (QPPs) of TELM. The solutions to the dual variables of the proposed ULTELMC are obtained using iterative schemes containing ‘plus’ function which is not differentiable. To overcome this shortcoming, the generalized derivative approach and smooth approximation approaches are suggested. Further, to test the performance of the proposed approaches, classification performances are compared with support vector machine (SVM), twin support vector machine (TWSVM), extreme learning machine (ELM), twin extreme learning machine (TELM) and Lagrangian extreme learning machine (LELM). Moreover, non-requirement to solve QPPs makes the iterative schemes find the solution faster as compared to the reported methods that finds the solution in dual space. Computational times required in finding the solutions are also presented for comparison.

A Robust Spectral Three-Term Conjugate Gradient Algorithm for Solving Unconstrained Minimization Problems

In this paper, we investigated a new improved Conjugate Gradient (CG) algorithm of a Three-Term type (TTCG) based on Dai and Liao procedure to improve the CG algorithm of (Hamoda, Rivaie, and Mamat / HRM). The new CG-algorithm satisfies both the conjugacy condition and the sufficient descent condition. The step-size of this TTCG-algorithm would be computed by accelerating the Wolfe-Powell line search technique. The proposed new TTCG algorithms have demonstrated their global affinity in certain specific circumstances given in this paper.

Stochastic Three Points Method for Unconstrained Smooth Minimization

In this paper we consider the unconstrained minimization problem of a smooth function in $\mathbb{R}^n$ in a setting where only function evaluations are possible. We design a novel randomized deriv...

Anti-Predatory NIA for Unconstrained Mathematical Optimization Problems

Nature-Inspired Algorithms (NIAs) are one of the most efficient methods to solve the optimization problems. A recently proposed NIA is the anti-predatory NIA, which is based on the anti-predatory behavior of frogs. This algorithm uses five different types of self-defense mechanisms in order to improve its anti-predatory strength. This paper demonstrates the computation steps of anti-predatory for solving the Rastrigin function and attempts to solve 20 unconstrained minimization problems using anti-predatory NIA. The performance of anti-predatory NIA is compared with the six competing meta-heuristic algorithms. A comparative study reveals that the anti-predatory NIA is a more promising than the other algorithms. To quantify the performance comparison between the algorithms, Friedman rank test and Holm-Sidak test are used as statistical analysis methods. Anti-predatory NIA ranks first in both cases of “Mean Result” and “Standard Deviation.” Result measures the robustness and correctness of the anti-predatory NIA. This signifies the worth of anti-predatory NIA in the domain of mathematical optimization.

Comparison of modern heuristics on solving the phase stability testing problem

<p style='text-indent:20px;'>In this paper, we are concerned with the phase stability testing at constant volume, temperature, and moles (<inline-formula><tex-math id="M1">\begin{document}$ VTN $\end{document}</tex-math></inline-formula>-specification) of a multicomponent mixture, which is an unconstrained minimization problem. We present and compare the performance of five chosen optimization algorithms: Differential Evolution, Cuckoo Search, Harmony Search, CMA-ES, and Elephant Herding Optimization. For the comparison of the evolution strategies, we use the Wilcoxon signed-rank test. In addition, we compare the evolution strategies with the classical Newton-Raphson method based on the computation times. Moreover, we present the expanded mirroring technique, which mirrors the computed solution into a given simplex.

An ellipsoid method for minimization of convex function

We consider the generalized ellipsoid method – an algorithm with the space dilation. For a certain choice of the dilation coefficient, this is a method of outer approximation of semi-ellipsoids by ellipsoids with a monotonous decrease in their volume. The Yudin-Nemirovski-Shor ellipsoid method is a specific case. The paper provides properties of two algorithmic realizations of the generalized ellipsoid method. The first algorithm is based on updating nonsymmetric matrix B, as in the Shor ellipsoid method, and the second is based on updating symmetric matrix H = BBT, as in the Yudin-Nemirovski ellipsoid method. We present the Emshor algorithm (Ellipsoid Method of Shor) for computing a solution of the problem of unconstrained minimization of a convex function. It updates nonsymmetric matrix B and uses a stopping criterion that, for a convex function, guarantees to find a point at which the function value does not deviate more than a specified tolerance from the optimal function value. It is shown that the Emshor algorithm finds sufficiently accurate approximations to the minimum point of a ravine convex function, and for functions of twenty variables it takes no longer than a few seconds on a usual PC. Hence, the algorithm can be useful for solving small optimization problems. The Emshor algorithm is planned to be used to develop a dual method for solving a two-stage transportation problem, when the number of intermediate points is not greater than twenty. The computational complexity of the dual method is determined by the complexity of calculating the value of the dual function and its supergradient, for which we need not more than twenty times to find the minimum elements in two one-dimensional arrays whose lengths m and n correspond to the numbers of suppliers and consumers. This means that the dual method can be oriented to the case of large m and n (thousands, tens of thousands), for which solving linear programming problems corresponding to the two-stage transportation problem by general-purpose programs is impossible or requires significant computational resources.

Müntz–Legendre neural network construction for solving delay optimal control problems of fractional order with equality and inequality constraints

In this paper, an artificial intelligence approach using neural networks is described to solve a class of delay optimal control problems of fractional order with equality and inequality constraints. In the proposed method, a functional link neural network based on the Muntz–Legendre polynomial is developed. The problem is first transformed into an equivalent problem with a fractional dynamical system without delay, using a Pade approximation. According to the Pontryagin’s minimum principle for optimal control problems of fractional order and by constructing an error function, the authors then define an unconstrained minimization problem. The authors use trial solutions for the states, Lagrange multipliers and control functions where these trial solutions are constructed by a single-layer Muntz–Legendre neural network model. The authors then exploit an unconstrained optimization scheme for adjusting the network parameters (weights and bias) and to minimize the computed error function. Some numerical examples are given to illustrate the effectiveness of the proposed method.

Fractional infinite-horizon optimal control problems with a feed forward neural network scheme

ABSTRACTThis paper presents a method based on neural networks to solve fractional infinite-horizon optimal control problems s(FIHOCP)s, where the dynamic control system depends on Caputo fractional derivatives. First, with the help of an approximation, the Caputo derivative is replaced to integer-order derivative. Using a suitable change of variable, the IHOCP is transformed into a finite-horizon one. According to the Pontryagin minimum principle (PMP) for optimal control problems and by constructing an error function, an unconstrained minimization problem is defined. In the optimization problem, the trial solutions are used for state, costate and control functions where these trial solutions are constructed by using two-layered perceptron neural network. Two numerical results are introduced to explain our main results.

A fractional power series neural network for solving a class of fractional optimal control problems with equality and inequality constraints

ABSTRACTThis paper solved fractional order optimal control problems, in which the dynamic control system involves integer and fractional order derivatives with equality and inequality constraints. According to the Pontryagin minimum principle (PMP) for fractional optimal control problem (FOCP) with fractional derivative in the Caputo sense and by constructing a suitable error function, an unconstrained minimization problem is defined. In the optimization problem, trial solutions are used for the states, Lagrange multipliers and control functions where these trial solutions are constructed by fractional power series neural network models. An error function is then minimized using a numerical optimization scheme where weight parameters (or coefficients of the series) and biases associated with all neurons are unknown. Some computational simulations are discussed in details.

A generalized matrix Krylov subspace method for TV regularization

This paper presents efficient algorithms to solve both TV/L1 and TV/L2 models of images contaminated by blur and noise. The unconstrained structure of the problems suggests that one can solve a constrained optimization problem by transforming the original unconstrained minimization problem to an equivalent constrained minimization one. An augmented Lagrangian method is developed to handle the constraints when the model is given with matrix variables, and an alternating direction method (ADM) is used to iteratively find solutions of the subproblems. The solutions of some subproblems are belonging to subspaces generated by application of successive orthogonal projections onto a class of generalized matrix Krylov subspaces of increasing dimension. We give some theoretical results and report some numerical experiments to show the effectiveness of the proposed algorithms.