Unconstrained Optimization Problems Research Articles

A New Dai-Liao Conjugate Gradient Method based on Approximately Optimal Stepsize for Unconstrained Optimization

Conjugate gradient methods are a class of very effective iterative methods for large-scale unconstrained optimization. In this paper, a new Dai-Liao conjugate gradient method for solving large-scale unconstrained optimization problem is proposed. Based on the approximately optimal stepsize for the gradient method, we derive three new choices for the important parameters tk in Dai-Liao conjugate gradient method. The search direction satisfies the sufficient descent condition, and the global convergences of the proposed method for uniformly convex and general functions are proved under some mild conditions. Numerical experiments on a set of test problems from the CUTEst library show that the proposed method is superior to some well-known conjugate gradient methods.

A New Hybrid Three-Term LS-CD Conjugate Gradient In Solving Unconstrained Optimization Problems

The Conjugate Gradient (CG) method is renowned for its rapid convergence in optimization applications. Over the years, several modifications to CG methods have emerged to improve computational efficiency and tackle practical challenges. This paper presents a new three-term hybrid CG method for solving unconstrained optimization problems. This algorithm utilizes a search direction that combines Liu-Storey (LS) and Conjugate Descent (CD) CG coefficients and standardizes it using a spectral which acts as a scheme for the choices of the conjugate parameters. This resultant direction closely approximates the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton direction, known for its bounded nature and compliance with the sufficient descent condition. The paper establishes the global convergence under standard Wolfe conditions and some appropriate assumptions. Additionally, the numerical experiments were conducted to emphasize the robustness and superior efficiency of this hybrid algorithm in comparison to existing approaches.

Uncertainty-resilient constrained rendezvous trajectory optimization via stochastic feedback control and unscented transformation

This paper proposes an innovative approach for constrained rendezvous trajectory optimization (CRTO) using stochastic optimal feedback control and unscented transform (UT) uncertainty quantification. The method overcomes limitations of traditional deterministic CRTO solutions that suffer from uncontrolled terminal state error growth and severely deteriorated path constraint satisfaction when initial state uncertainty, dynamics uncertainty, and navigation uncertainty are considered. The approach involves constructing a stochastic optimal feedback control (SOFC) problem with chance constraints and introducing linear feedback control to regulate both the mean and variance of terminal state errors. UT is employed to approximately quantify the state errors and their propagation, transforming the SOFC problem into an unconstrained deterministic optimization (UDO) problem. Differential dynamic programming (DDP) is then used to solve the UDO problem. The obtained stochastic optimal solution provides a robust rendezvous trajectory and a corresponding explicit closed-loop guidance law, improving terminal accuracy and path constraint satisfaction in the presence of various considered uncertainties. The influence of different term weights in the objective function on terminal accuracy and path constraint satisfaction is also studied. The effectiveness of the proposed approach is verified through Monte Carlo simulation, demonstrating the robustness of the closed-loop control strategy. The results highlight the potential of the method in enhancing terminal accuracy and path constraint satisfaction in uncertain rendezvous scenarios.

Annealing for prediction of grand canonical crystal structures: Implementation of n -body atomic interactions

We propose an annealing scheme usable on modern Ising machines for crystal structures prediction (CSP) by taking into account the general n-body atomic interactions and in particular three-body interactions which are necessary to simulate covalent bonds. The crystal structure is represented by discretizing a unit cell and placing binary variables which express the existence or nonexistence of an atom on every grid point. The resulting quadratic unconstrained binary optimization (QUBO) or higher-order unconstrained binary optimization (HUBO) problems implement the CSP problem and is solved using simulated and quantum annealing. Using the example of Lennard-Jones clusters we show that it is not necessary to include the target atom number in the formulation allowing for simultaneous optimization of both the particle density and the configuration and argue that this is advantageous for use on annealing machines as it reduces the total amount of interactions. We further provide a scheme that allows for reduction of higher-order interaction terms that is inspired by the underlying physics. We show for a covalently bonded monolayer MoS2 crystal that we can simultaneously optimize for the particle density as well as the crystal structure using simulated annealing. We also show that we reproduce ground states of the interatomic potential with high probability that are not represented on the initial discretization of the unit cell. Published by the American Physical Society 2024

A Second-Order Finite-Difference Method for Derivative-Free Optimization

In this paper, a second-order finite-difference method is proposed for finding the second-order stationary point of derivative-free nonconvex unconstrained optimization problems. The forward-difference or the central-difference technique is used to approximate the gradient and Hessian matrix of objective function, respectively. The traditional trust-region framework is used, and we minimize the approximation trust region subproblem to obtain the search direction. The global convergence of the algorithm is given without the fully quadratic assumption. Numerical results show the effectiveness of the algorithm using the forward-difference and central-difference approximations.

Convergence analysis of the DFP algorithm for unconstrained optimization problems on Riemannian manifolds

Convergence analysis of the DFP algorithm for unconstrained optimization problems on Riemannian manifolds

Global convergence properties of a Dai-Liao-type CGM for unconstrained optimization

A popular optimization technique called the conjugate gradient method (CGM) is renowned for its effectiveness in addressing problems involving unconstrained optimization. Several conjugate gradient (CG) techniques have been proven to possess global convergence properties both theoretically and numerically. The Dai-Liao-type CGM is a variant that incorporates certain modifications to enhance its convergence properties. This paper examines the global convergence properties of a Dai-Liao-type CGM for unconstrained optimization problems. Theoretically, this study investigates the conditions under which the method ensures convergence to the global minimum of the objective function, focusing on the algorithm’s descent directions, the necessary reduction in objective function values, and termination criteria. A numerical experiment is conducted on a set of unconstrained optimization problems to validate the theoretical results obtained in this work. The numerical findings of this study demonstrate the robustness and reliability of the Dai-Liao-type CGM, showing its ability to find the global optimal solution in a wide range of unconstrained optimization problems.

Boosting constrained teaching–learning based optimization algorithm through group discussion

Teaching–learning based optimization algorithm (TLBO) is an efficient population-based nature-inspired meta-heuristic designed initially to solve unconstrained optimization problems. TLBO being a swam intelligence based algorithm is renowned for fast convergence. This characteristic, however, occasionally causes it to converge too soon to a non-globally optimal solution. Group discussion techniques are used in educational institutions all over the world to help participants to exchange ideas and information and to improve their skills. This work imitates the outlined strategy, therefore innovating TLBO. Here, a group of virtual students is selected corresponding to each student in the learner phase of TLBO to produce diversity in the population and to balance the exploration and exploitation capabilities of it, hence reducing the likelihood of early convergence. GTLBO stands for the new algorithm that incorporates group discussion. It is essential to incorporate some constraint handling techniques (CHTs) into the TLBO framework in order to handle constrained optimization problems (COPs). The most recent, effective, and enhanced version of the widely used CHT stochastic ranking (SR) is called hybrid stochastic ranking (HSR). HSR is added to the GTLBO structure as a selection operator for managing COPs; the resulting setup is referred to as HSR-GTLBO. Additionally, the suggested algorithm’s group size for discussion is sensitive, resulting in a number of constrained variants. The efficacy of these variations is assessed with the suit CEC 2017’s constrained problems. In conformity with the parameters settlement, comparison, and ranking criteria of CEC 2017, the simulation results of the proposed versions are compared and ranked using the most advanced state-of-the-art algorithms. As a result, in the competition, the newly developed HSR-GTLBO variants placed first, second, and third.

A warm-start FE-dABCD algorithm for elliptic optimal control problems with constraints on the control and the gradient of the state

In this paper, elliptic control problems with the integral constraint on the gradient of the state and the box constraint on the control are considered. The optimality conditions for the problem are proved. To numerically solve the problem, a finite element duality-based inexact majorized accelerated block coordinate descent (FE-dABCD) algorithm is proposed. Specifically, both the state and the control are discretized by piecewise linear functions. An inexact majorized ABCD algorithm is employed to solve the discretized problem via its dual, which is a multi-block unconstrained convex optimization problem, but the primal variables are also generated in each iteration. Thanks to the inexactness of the FE-dABCD algorithm, the subproblems at each iteration are allowed to be solved inexactly. For the smooth subproblem, we use the preconditioned generalized minimal residual (GMRES) method to solve it. For the two nonsmooth subproblems, one of them has a closed form solution through introducing an appropriate proximal term, and another one is solved by the line search Newton's method. Based on these efficient strategies, we prove that our proposed FE-dABCD algorithm enjoys O(1k2) iteration complexity. Moreover, to make the algorithm more efficient and further reduce its computation cost, based on the mesh-independence of ABCD method, we propose an FE-dABCD algorithm with a warm-start strategy (wFE-dABCD). Some numerical experiments are done and the numerical results show the efficiency of the FE-dABCD algorithm and wFE-dABCD algorithm.

Convergence of successive linear programming algorithms for noisy functions

AbstractGradient-based methods have been highly successful for solving a variety of both unconstrained and constrained nonlinear optimization problems. In real-world applications, such as optimal control or machine learning, the necessary function and derivative information may be corrupted by noise, however. Sun and Nocedal have recently proposed a remedy for smooth unconstrained problems by means of a stabilization of the acceptance criterion for computed iterates, which leads to convergence of the iterates of a trust-region method to a region of criticality (Sun and Nocedal in Math Program 66:1–28, 2023. https://doi.org/10.1007/s10107-023-01941-9). We extend their analysis to the successive linear programming algorithm (Byrd et al. in Math Program 100(1):27–48, 2003. https://doi.org/10.1007/s10107-003-0485-4, SIAM J Optim 16(2):471–489, 2005. https://doi.org/10.1137/S1052623403426532) for unconstrained optimization problems with objectives that can be characterized as the composition of a polyhedral function with a smooth function, where the latter and its gradient may be corrupted by noise. This gives the flexibility to cover, for example, (sub)problems arising in image reconstruction or constrained optimization algorithms. We provide computational examples that illustrate the findings and point to possible strategies for practical determination of the stabilization parameter that balances the size of the critical region with a relaxation of the acceptance criterion (or descent property) of the algorithm.

Transformation of Elliptic Curve Discrete Logarithm Problem to QUBO Using Direct Method in Quantum Annealing Applications

This paper investigates how to reduce the elliptic curve discrete logarithm problem over prime fields to the quadratic unconstrained binary optimization (QUBO) problem in order to obtain as few logical qubits as possible. In the best case scenario, if n is the bitlength of a characteristic of prime field Fp, approximately 3n³ logical qubits are required for such a reduction in the Edwards curve case. We present a practical attack on an elliptic curve discrete logarithm problem over the 3-bit prime field F7 for an elliptic curve with the subgroup of order 8. We solved this problem using the D-Wave Advantage QPU. To the best of the authors' knowledge, no one has made, so far, a practical attack on the elliptic curve discrete logarithm over a prime field using the direct quantum method.

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent.

A New Hybrid Three-Term HS-DY Conjugate Gradient In Solving Unconstrained Optimization Problems

Conjugate Gradient (CG) method is an interesting tool to solve optimization problems in many fields, such design, economics, physics and engineering. Until now, many CG methods have been developed to improve computational performance and have applied in the real-world problems. Combining two CG parameters with distinct denominators may result in non-optimal outcomes and congestion.In this paper, a new hybrid three-term CG method is proposed for solving unconstrained optimization problems. The hybrid threeterm search direction combines Hestenes-Stiefel (HS) and Dai-Yuan (DY) CG parameters which standardized by using a spectral to determine the suitable conjugate parameter choice and it satisfies the sufficient descent condition. Additionally, the global convergence was proved under standard Wolfe conditions and some suitable assumptions. Furthermore, the numerical experiments showed the proposed method is most robust and superior efficiency compared to some existing methods.

Waveform Design for Target Information Maximization over a Complex Circle Manifold

The cognitive radar framework presents a closed-loop adaptive processing paradigm that ensures the efficient acquisition of target information while exploring the environment and enhancing overall sensing performance. In this study, instead of mutual information, we employed the squared Pearson correlation coefficient (SPCC) to measure the target information in observations specifically considering only linear dependency. A waveform design method is proposed that simultaneously maximizes target information and minimizes the integrated sidelobe level (ISL) under the constant modulus constraint (CMC). To enhance computational efficiency, we reformulated the constrained problem as an unconstrained optimization problem by leveraging the inherent geometric property of CMC. Additionally, we present two conditional equivalences associated with waveform design in relation to target information. The simulation results validate the feasibility and effectiveness of the proposed method.

Graph signal reconstruction based on spatio-temporal features learning

This paper presents a new algorithm for reconstructing time-varying graph signals using spatiotemporal feature learning. We introduce a time series analysis method to capture the temporal stationarity of graph signals and propose a reconstruction model based on spatiotemporal coupling features. However, prior knowledge of the temporal stationarity of graph signals is required for this task. To address the issue, we analyze the spatio-temporal coupling feature in a graph-spectral view and design a learning framework to capture only the temporal feature based on the sampled signals. We reconstruct the graph signals using the proposed reconstruction model and feature-learning framework by solving an unconstrained optimization problem consisting of data fidelity and two regularization terms. Numerical results using both synthetic and real-world datasets demonstrate the superiority of the proposed reconstruction algorithm over existing methods.

Quantum annealing-based computed tomography using variational approach for a real-number image reconstruction

Objective. Despite recent advancements in quantum computing, the limited number of available qubits has hindered progress in CT reconstruction. This study investigates the feasibility of utilizing quantum annealing-based computed tomography (QACT) with current quantum bit levels. Approach. The QACT algorithm aims to precisely solve quadratic unconstrained binary optimization problems. Furthermore, a novel approach is proposed to reconstruct images by approximating real numbers using the variational method. This approach allows for accurate CT image reconstruction using a small number of qubits. The study examines the impact of projection data quantity and noise on various image sizes ranging from 4 × 4 to 24 × 24 pixels. The reconstructed results are compared against conventional reconstruction algorithms, namely maximum likelihood expectation maximization (MLEM) and filtered back projection (FBP). Main result. By employing the variational approach and utilizing two qubits for each pixel of the image, accurate reconstruction was achieved with an adequate number of projections. Under conditions of abundant projections and lower noise levels, the image quality in QACT algorithm outperformed that of MLEM and FBP algorithms. However, in situations with limited projection data and in the presence of noise, the image quality in QACT was inferior to that in MLEM. Significance. This study developed the QACT reconstruction algorithm using the variational approach for real-number reconstruction. Remarkably, only 2 qubits were required for each pixel representation, demonstrating their sufficiency for accurate reconstruction.

A new Q‐function structure for model‐free adaptive optimal tracking control with asymmetric constrained inputs

SummaryThis article aims to design a model‐free adaptive tracking controller for discrete‐time nonlinear systems with unknown dynamics and asymmetric control constraints. First, a new Q‐function structure is designed by introducing the control input into the tracking error of the next moment, in order to eliminate the final tracking error, avoid the steady control, and ignore the discount factor. Second, via system transformation, a general performance index is developed to overcome the challenge caused by asymmetric constraints of implicit control inputs. By this operation, the constrained tracking problem is converted to an unconstrained optimal tracking problem without the traditional nonquadratic performance function that is only applicable to explicit control inputs. Then, a value‐iteration‐based Q‐learning (VIQL) algorithm is derived to seek the optimal Q‐function and the optimal control policy by using offline data rather than the mathematical model. Next, the convergence, monotonicity, and stability properties of VIQL are investigated to demonstrate that the iterative Q‐function sequence can converge to the optimal Q‐function under ideal conditions. To realize the VIQL algorithm, the critic neural network is employed to approximate the Q‐function. Finally, simulation results and comparative experiments are conducted to demonstrate the validity and effectiveness of the present VIQL scheme.

Solving Large-Scale Unconstrained Optimization Problems with an Efficient Conjugate Gradient Class

The main goal of this paper is to introduce an appropriate conjugate gradient class to solve unconstrained optimization problems. The presented class enjoys the benefits of having three free parameters, its directions are descent, and it can fulfill the Dai–Liao conjugacy condition. Global convergence property of the new class is proved under the weak-Wolfe–Powell line search technique. Numerical efficiency of the proposed class is confirmed in three sets of experiments including 210 test problems and 11 disparate conjugate gradient methods.

Quantum computer based feature selection in machine learning

AbstractThe problem of selecting an appropriate number of features in supervised learning problems is investigated. Starting with common methods in machine learning, the feature selection task is treated as a quadratic unconstrained optimisation problem (QUBO), which can be tackled with classical numerical methods as well as within a quantum computing framework. The different results in small problem instances are compared. According to the results of the authors’ study, whether the QUBO method outperforms other feature selection methods depends on the data set. In an extension to a larger data set with 27 features, the authors compare the convergence behaviour of the QUBO methods via quantum computing with classical stochastic optimisation methods. Due to persisting error rates, the classical stochastic optimisation methods are still superior.

Supervised probabilistic latent semantic analysis with applications to controversy analysis of legislative bills

Probabilistic Latent Semantic Analysis (PLSA) is a fundamental text analysis technique that models each word in a document as a sample from a mixture of topics. PLSA is the precursor of probabilistic topic models including Latent Dirichlet Allocation (LDA). PLSA, LDA and their numerous extensions have been successfully applied to many text mining and retrieval tasks. One important extension of LDA is supervised LDA (sLDA), which distinguishes itself from most topic models in that it is supervised. However, to the best of our knowledge, no prior work extends PLSA in a similar manner sLDA extends LDA by jointly modeling the contents and the responses of documents. In this paper, we propose supervised PLSA (sPLSA) which can efficiently infer latent topics and their factorized response values from the contents and the responses of documents. The major challenge lies in estimating a document’s topic distribution which is a constrained probability that is dictated by both the content and the response of the document. To tackle this challenge, we introduce an auxiliary variable to transform the constrained optimization problem to an unconstrained optimization problem. This allows us to derive an efficient Expectation and Maximization (EM) algorithm for parameter estimation. Compared to sLDA, sPLSA converges much faster and requires less hyperparameter tuning, while performing similarly on topic modeling and better in response factorization. This makes sPLSA an appealing choice for latent response analysis such as ranking latent topics by their factorized response values. We apply the proposed sPLSA model to analyze the controversy of bills from the United States Congress. We demonstrate the effectiveness of our model by identifying contentious legislative issues.