Large-scale Nonlinear Optimization Problems Research Articles

GPU-accelerated Bloch simulations and MR-STAT reconstructions using the Julia programming language.

MR-STAT is a relatively new multiparametric quantitative MRI technique in which quantitative paramater maps are obtained by solving a large-scale nonlinear optimization problem. Managing reconstruction times is one of the main challenges of MR-STAT. In this work we leverage GPU hardware to reduce MR-STAT reconstruction times. A highly optimized, GPU-compatible Bloch simulation toolbox is developed as part of this work that can be utilized for other quantitative MRI techniques as well. The Julia programming language was used to develop a flexible yet highly performant and GPU-compatible Bloch simulation toolbox called BlochSimulators.jl. The runtime performance of the toolbox is benchmarked against other Bloch simulation toolboxes. Furthermore, a (partially matrix-free) modification of a previously presented (matrix-free) MR-STAT reconstruction algorithm is proposed and implemented using the Julia language on GPU hardware. The proposed algorithm is combined with BlochSimulators.jl and the resulting MR-STAT reconstruction times on GPU hardware are compared to previously presented MR-STAT reconstruction times. The BlochSimulators.jl package demonstrates superior runtime performance on both CPU and GPU hardware when compared to other existing Bloch simulation toolboxes. The GPU-accelerated partially matrix-free MR-STAT reconstruction algorithm, which relies on BlochSimulators.jl, allows for reconstructions of 68 seconds per two-dimensional (2D slice). By combining the proposed Bloch simulation toolbox and the partially matrix-free reconstruction algorithm, 2D MR-STAT reconstructions can be performed in the order of one minute on a modern GPU card. The Bloch simulation toolbox can be utilized for other quantitative MRI techniques as well, for example for online dictionary generation for MR Fingerprinting.

Nonlinear inversion of electrical resistivity sounding data for multi-layered 1-D earth model using global particle swarm optimization (GPSO)

Interpreting geophysical data requires solving nonlinear optimization problem(s) in inversion. Analytical methods such as least-square have some intrinsic limitations, which include slow convergence and dimensionality, making heuristic-based swarm intelligence a better alternative. Large-scale nonlinear optimization problems in inversion can be solved effectively using a technique within the swarm intelligence family called Particle Swarm Optimization (PSO). This study evaluates the inversion of geoelectrical resistivity data with global particle swarm optimization (GPSO). We attempted to invert field vertical electrical sounding data for a multi-layered 1-D earth model using the developed particle swarm optimization algorithm. The result of the PSO-interpreted VES data was compared with that of the least square inversion result from Winresist 1.0. According to the PSO-interpreted VES results, satisfactory solutions may be attained with a swarm of 200 or fewer particles, and convergence can be reached in fewer than 100 iterations. The GPSO inversion approach has a maximum capacity of 100 iterations, more than the least square inversion algorithm of the Winresist, which has a maximum capacity of 30 iterations. The misfit error of GPSO inversion is 6.14×10−7, much lower than that of the least square inversion of 4.0. The GPSO inversion model has lower and upper limit values of the geoelectric layer parameters model to fit the true model better. The limitations of the developed PSO inversion scheme include a slower execution time of the inversion procedures than the least-square inversion. There is a need for a priori knowledge of the number of layers from borehole reports in the study area. The PSO inversion scheme, however, estimates inverted models closer to the true solutions with greater accuracy than the least-square inversion scheme.

Cartesian vs radial MR-STAT: An efficiency and robustness study

MR Spin TomogrAphy in Time-domain (“MR-STAT”) is quantitative MR technique in which multiple quantitative parameters are estimated from a single short scan by solving a large-scale non-linear optimization problem. In this work we extended the MR-STAT framework to non-Cartesian gradient trajectories. Cartesian MR-STAT and radial MR-STAT were compared in terms of time-efficiency and robustness in simulations, gel phantom measurements and in vivo measurements.In simulations, we observed that both Cartesian and radial MR-STAT are highly robust against undersampling. Radial MR-STAT does have a lower spatial encoding power because the outer corners of k-space are never sampled. However, especially in T2, this is compensated by a higher dynamic encoding power that comes from sampling the k-space center with each readout. In gel phantom measurements, Cartesian MR-STAT was observed to be robust against overfitting whereas radial MR-STAT suffered from high-frequency artefacts in the parameter maps at later iterations. These artefacts are hypothesized to be related to hardware imperfections and were (partially) suppressed with image filters. The time-efficiencies were higher for Cartesian MR-STAT in all vials. In-vivo, the radial reconstruction again suffered from overfitting artefacts. The robustness of Cartesian MR-STAT over the entire range of experiments may make it preferable in a clinical setting, despite radial MR-STAT resulting in a higher T1 time-efficiency in white matter.

Quantum computing for future real-time building HVAC controls

Buildings contribute to more than 70% of overall U.S. electricity usage and greenhouse gas (GHG) emissions. HVAC systems in buildings often consume more than 40% of the total building energy usage. To reduce such high energy use, numerous control strategies including optimal and predictive controls have been developed and demonstrated. To achieve a near real-time solution, most previous research has simplified the non-linearity of building thermodynamics and provided an approximate optimal solution. The future HVAC control optimizes more connected devices in buildings, which requires a rapid and accurate response, not only to the building itself but also to the grid signals. It also poses the challenge of solving non-linear problems with discrete variables. With the recent development of quantum computers, this has become feasible. In this paper, we developed a new optimization solution based on quantum annealing for model predictive control (MPC) of a rooftop unit (RTU). Compared to traditional optimization methods, we obtained similar solutions with less than 2% differences and improved computational speed from hours to seconds. We also demonstrated an 80% reduction in total electricity consumption and a 21% reduction in electricity bills by considering day-ahead price time-of-use demand response signals. Quantum computing has proven capable of solving large-scale non-linear discrete optimization problems for building energy systems.

Computational aspects of column generation for nonlinear and conic optimization: classical and linearized schemes

Solving large scale nonlinear optimization problems requires either significant computing resources or the development of specialized algorithms. For Linear Programming (LP) problems, decomposition methods can take advantage of problem structure, gradually constructing the full problem by generating variables or constraints. We first present a direct adaptation of the Column Generation (CG) methodology for nonlinear optimization problems, such that when optimizing over a structured set $${\mathcal {X}}$$ plus a moderate number of complicating constraints, we solve a succession of (1) restricted master problems on a smaller set $${\mathcal {S}}\subset {\mathcal {X}}$$ and (2) pricing problems that are Lagrangean relaxations wrt the complicating constraints. The former provides feasible solutions and feeds dual information to the latter. In turn, the pricing problem identifies a variable of interest that is then taken into account into an updated subset $${\mathcal {S}}'\subset {\mathcal {X}}$$ . Our approach is valid whenever the master problem has zero Lagrangean duality gap wrt to the complicating constraints, and not only when $${\mathcal {S}}$$ is the convex hull of the generated variables as in CG for LPs, but also with a variety of subsets such as the conic hull, the linear span, and a special variable aggregation set. We discuss how the structure of $${\mathcal {S}}$$ and its update mechanism influence the number of iterations required to reach near-optimality and the difficulty of solving the restricted master problems, and present linearized schemes that alleviate the computational burden of solving the pricing problem. We test our methods on synthetic portfolio optimization instances with up to 5 million variables including nonlinear objective functions and second order cone constraints. We show that some CGs with linearized pricing are 2–3 times faster than solving the complete problem directly and are able to provide solutions within 1% of optimality in 6 h for the larger instances, whereas solving the complete problem runs out of memory.

Research on Volt/Var Control of Distribution Networks Based on PPO Algorithm

In this paper, a model free volt/var control (VVC) algorithm is developed by using deep reinforcement learning (DRL). We transform the VVC problem of distribution networks into the network framework of PPO algorithm, in order to avoid directly solving a large-scale nonlinear optimization problem. We select photovoltaic inverters as agents to adjust system voltage in a distribution network, taking the reactive power output of inverters as action variables. An appropriate reward function is designed to guide the interaction between photovoltaic inverters and the distribution network environment. OPENDSS is used to output system node voltage and network loss. This method realizes the goal of optimal VVC in distribution network. The IEEE 13-bus three phase unbalanced distribution system is used to verify the effectiveness of the proposed algorithm. Simulation results demonstrate that the proposed method has excellent performance in voltage and reactive power regulation of a distribution network.

A Sufficient Descent Property to Improving a ThreeTerm Conjugate Gradient Algorithm

The nonlinear conjugate gradient (NLCGM) methods have received attention because due to their simplicity, low memory requirements, and global convergent property, which allows them to be used directly to solve large-scale nonlinear unconstrained optimization problems. We suggested a modification to the β KMAR k formula, applied with three-term conjugate gradient method that is both simple and effective, denoted by (TTKMAR), which has a sufficient descent property (SDP) and ensures global convergence (GCP) when we use any line search. The numerical efficiency of TTKMAR was assessed using a variety of standard test functions. TTCGM has been demonstrated to be more numerically efficient than two-term CG methods. This paper also quantifies the difference between TTCGM and two-term methods of performance. As a result, we compare our new modification to an efficient two-term and TTCGM in the numerical results. Finally, we conclude that our proposed modification.

Nested Decomposition Approach for Dispatch Optimization of Large-Scale, Integrated Electricity, Methane and Hydrogen Infrastructures

Energy system integration enables raising operational synergies by coupling the energy infrastructures for electricity, methane, and hydrogen. However, this coupling reinforces the infrastructure interdependencies, increasing the need for integrated modeling of these infrastructures. To analyze the cost-efficient, sustainable, and secure dispatch of applied, large-scale energy infrastructures, an extensive and non-linear optimization problem needs to be solved. This paper introduces a nested decomposition approach with three stages. The method enables an integrated and full-year consideration of large-scale multi-energy systems in hourly resolution, taking into account physical laws of power flows in electricity and gas transmission systems as boundary conditions. For this purpose, a zooming technique successively reduces the temporal scope while first increasing the spatial and last the technical resolution. A use case proves the applicability of the presented approach to large-scale energy systems. To this end, the model is applied to an integrated European energy system model with a detailed focus on Germany in a challenging transport situation. The use case demonstrates the temporal, regional, and cross-sectoral interdependencies in the dispatch of integrated energy infrastructures and thus the benefits of the introduced approach.

Application of physics-constrained data-driven reduced-order models to shape optimization

This study proposes a novel approach for building surrogate models, in the form of reduced-order models(ROMs), for partial differential equation constrained optimization. A physics-constrained data-driven framework is developed to transform large-scale nonlinear optimization problems into ROM-constrained optimization problems. Unlike conventional methods, which suffer from instability of the forward sensitivity function, the proposed approach maps optimization problems to system dynamics optimization problems in Hilbert space to improve stability, reduce memory requirements, and lower computational cost. The utility of this approach is demonstrated for aerodynamic optimization of an NACA 0012 airfoil at $Re = 1000$ . A drag reduction of 9.35 % is obtained at an effective angle of attack of eight degrees, with negligible impact on lift. Similarly, a drag reduction of 20 % is obtained for fully separated flow at an angle of attack of $25^{\circ }$ . Results from these two optimization problems also reveal relationships between optimization in physical space and optimization of dynamical behaviours in Hilbert space.

Tuning successive linear programming to solve AC optimal power flow problem for large networks

Successive linear programming (SLP) is a practical approach for solving large-scale nonlinear optimization problems. Alternating current optimal power flow (ACOPF) is no exception, particularly the large size of real-world networks. However, in order to achieve tractability, it is essential to tune the SLP algorithm presented in the literature. This paper presents a modified SLP algorithm to solve the ACOPF problem, specified by the U.S. Department of Energy’s (DOE) Grid Optimization (GO) Competition Challenge 1, within strict time limits. The algorithm first finds a near-optimal solution for the relaxed problem (i.e., Stage 1). Then, it finds a feasible solution in the proximity of the near-optimal solution (i.e., Stage 2 and Stage 3). The numerical experiments on test cases ranging from 500-bus to 30,000-bus systems show that the algorithm is tractable. The results show that our proposed algorithm is tractable and can solve more than 80% of test cases faster than the well-known Interior Point Method while significantly reduce the number of iterations required to solve ACOPF. The number of iterations is considered an important factor in the examination of tractability which can drastically reduce the computational time required within each iteration.

Enhanced Tunicate Swarm Algorithm for Solving Large-Scale Nonlinear Optimization Problems

Nowadays optimization problems become difficult and complex, traditional methods become inefficient to reach global optimal solutions. Meanwhile, a huge number of meta-heuristic algorithms have been suggested to overcome the shortcomings of traditional methods. Tunicate Swarm Algorithm (TSA) is a new biologically inspired meta-heuristic optimization algorithm which mimics jet propulsion and swarm intelligence during the searching for a food source. In this paper, we suggested an enhancement to TSA, named Enhanced Tunicate Swarm Algorithm (ETSA), based on a novel searching strategy to improve the exploration and exploitation abilities. The proposed ETSA is applied to 20 unimodal, multimodal and fixed dimensional benchmark test functions and compared with other algorithms. The statistical measures, error analysis and the Wilcoxon test have affirmed the robustness and effectiveness of the ETSA. Furthermore, the scalability of the ETSA is confirmed using high dimensions and results exhibited that the ETSA is least affected by increasing the dimensions. Additionally, the CPU time of the proposed algorithms are obtained, the ETSA provides less CPU time than the others for most functions. Finally, the proposed algorithm is applied at one of the important electrical applications, Economic Dispatch Problem, and the results affirmed its applicability to deal with practical optimization tasks.

Thallo – Scheduling for High-Performance Large-Scale Non-Linear Least-Squares Solvers

Large-scale optimization problems at the core of many graphics, vision, and imaging applications are often implemented by hand in tedious and error-prone processes in order to achieve high performance (in particular on GPUs), despite recent developments in libraries and DSLs. At the same time, these hand-crafted solver implementations reveal that the key for high performance is a problem-specific schedule that enables efficient usage of the underlying hardware. In this work, we incorporate this insight into Thallo, a domain-specific language for large-scale non-linear least squares optimization problems. We observe various code reorganizations performed by implementers of high-performance solvers in the literature, and then define a set of basic operations that span these scheduling choices, thereby defining a large scheduling space. Users can either specify code transformations in a scheduling language or use an autoscheduler. Thallo takes as input a compact, shader-like representation of an energy function and a (potentially auto-generated) schedule, translating the combination into high-performance GPU solvers. Since Thallo can generate solvers from a large scheduling space, it can handle a large set of large-scale non-linear and non-smooth problems with various degrees of non-locality and compute-to-memory ratios, including diverse applications such as bundle adjustment, face blendshape fitting, and spatially-varying Poisson deconvolution, as seen in Figure 1. Abstracting schedules from the optimization, we outperform state-of-the-art GPU-based optimization DSLs by an average of 16× across all applications introduced in this work, and even some published hand-written GPU solvers by 30%+.

Piecewise linear approximations for hydropower production function applied on the hydrothermal unit commitment problem

In a centralized-based dispatch market, the network-constrained hydrothermal unit commitment (NCHTUC) determines generation decisions that minimize the expected operating cost, usually in a day-ahead planning horizon. In this context, hydropower offers unique flexibility features, and an adequate representation is crucial for overcoming operation challenges. However, due to computational burden, simplifications on hydropower production function (HPF) modeling are necessary since NCHTUC is a large-scale nonlinear discrete optimization problem. The plant-based HPF piecewise linear approach is the most common simplification presented for real-life cases. In this case, the generating units (GUs) of a plant are aggregated and represented by a single equivalent generator. Although this approach reduces the size and the complexity of the NCHTUC significantly, several operating issues are not considered adequately, especially the forbidden zones and the nonlinearities of the GUs. In this paper, we present a new approach that considers the nonlinearities and forbidden zones of the HPF via aggregation of the GUs and piecewise mixed-integer linear approximation in an innovative way. Experiments are performed in a modified version of the IEEE 118-bus system to verify the approach’s effectiveness, analyzed in terms of the relation between computation burden and the HPF approximation quality.

Good Characteristics of The New Spectral Conjugate Gradient Method for Unconstrained Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, we propose a new SCG method in which performance is numerically analyzed. We established the descent property and global convergence conditions based on assumptions through the strongWolfe-Powell line search. Numerical results were performed using benchmark functions widely used in many conventional functions to evaluate the efficiency of the proposed method.Subject Classification: 90C30, 90C06, 65K05, 65K10.

Global Convergence Condition for a New Spectral Conjugate Gradient Method for Large-Scale Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, a new spectral conjugate gradient method is proposed with a strong Wolfe-Powell line search (SWP). The idea of the new one is using the βBZA formula which is proposed by Baluch and et al., with suitable parameter φ denoted by (SCGBZA). Under the usual assumptions, the descent properties and overall global convergence of the proposed method (SCGBZA) are proved. The proposed method is numerically proven to be effective.

An Estimation of Distribution Algorithm With Filtering and Learning

Estimation of distribution algorithm (EDA) is an efficient population-based stochastic search technique. Since it was proposed, many attempts have been made to improve its performance in the context of nonlinear continuous optimization. However, the success of EDA depends on the accuracy of modeling, the effectiveness of sampling, and the ability of exploration. An effective EDA often needs to take some measures to adjust the model and to guide sampling. In this article, we propose a novel EDA which applies the idea of Kalman filtering to revise the modeling data and a learning strategy to improve sampling. The filtering scheme modifies the modeling data set using an estimation error matrix based on historic solution data. During the sampling process, the learning strategy determines the region to sample next based on the sampling outcomes so far, instead of completely random sampling. The proposed EDA also employs a multivariate probabilistic model based on copula function and can quickly reach the promising area in which the optimal solution is likely to be located. A collection of general benchmark functions are used to test the performance of the proposed algorithm. Computational experiments show that the EDA is effective. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In many process industries, there exist black-box operation optimization problems and large-scale nonlinear optimization problems with variable coupling. For these problems, it is difficult to establish mechanism models between input and output. However, real-time data can be measured from the system through sensors. We can utilize this process information to optimize the system so as to attain the desired objective. In this article, we propose a novel estimation of distribution algorithm (EDA) which applies a filtering scheme to revise the modeling data and a learning strategy to improve sampling, which can solve the problems with the characteristics of nonlinearity, variable coupling, and large scale. Computational experiments show that the EDA is effective. In the future, the proposed algorithm can be applied to some practical optimization problems such as operation optimization in blast furnace, which is considered as a continuous production process with variable coupling. The algorithm has the potential to help optimizing the process control parameters.

Solving Multi-Area Economic Dispatch with Multiple Fuels Using Hybrid Optimization Technique (Dept. E )

window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag("js", new Date()); gtag("config", "UA-72322659-1");

Lower bound limit and shakedown analysis of orthotropic material

We present in this study a new approach for predicting the plastic and shakedown limits of structures composed of orthotropic materials. In this approach, the Hill yield criterion is introduced to Melan’s theorem. By formulating the problem by means of the finite element method and solving the resulting large-scale nonlinear optimization problem we successfully predict the plastic and shakedown limits of structures having complex geometries made from multi-orthotropic materials. Several numerical examples are elaborated in this study for evaluating the accuracy, general applicability, as well as the efficiency of the established numerical scheme. Overall, the study confirms that the direct method can be extended and adopted as a viable means for design and analysis of structures made of orthotropic materials.

Nature-inspired algorithms for feed-forward neural network classifiers: A survey of one decade of research

Recently, an explosive growth in the potential use of natural metaphors in modelling and solving large-scale non-linear optimization problems. Artificial neural network (ANN) is a potent gadget broadly utilized in many data classification tasks. Fundamentally, nature-inspired algorithms have demonstrated their effectiveness and ability over traditional algorithms for generating the optimal ANN parameters, rules and topology that provide the best classification performance with regarding to the quality of the solution, computational cost and avoiding local minima. The literature is vast and growing. This study provides a review on the basic theories and main recent algorithms for optimizing the ANN. Different types of nature-inspired meta-heuristic algorithms are presented; outlining the concepts and components that are used in order to give a summary and ease of the state-of-the-arts to find suitable methods in real world applications for the readers. Additionally, this survey covers the most used type of neural networks, feed-forward neural network (FFNN) in several optimized applications. The performances of FFNNs designed by nature-inspired algorithms have been explored in single and multi-dimensional optimization space; highlighting their models, features, objectives, constraints, etc. to analyse their differences and similarities. A comprehensive survey of the earliest works and recent modified in the last decade in addition to expect approaches has been investigated in details.

Inexact Sequential Quadratic Optimization with Penalty Parameter Updates within the QP Solver

This paper focuses on the design of sequential quadratic optimization (commonly known as SQP) methods for solving large-scale nonlinear optimization problems. The most computationally demanding asp...