Local Optimization Techniques Research Articles

Improving the Generalization Abilities of Constructed Neural Networks with the Addition of Local Optimization Techniques

Constructed neural networks with the assistance of grammatical evolution have been widely used in a series of classification and data-fitting problems recently. Application areas of this innovative machine learning technique include solving differential equations, autism screening, and measuring motor function in Parkinson’s disease. Although this technique has given excellent results, in many cases, it is trapped in local minimum and cannot perform satisfactorily in many problems. For this purpose, it is considered necessary to find techniques to avoid local minima, and one technique is the periodic application of local minimization techniques that will adjust the parameters of the constructed artificial neural network while maintaining the already existing architecture created by grammatical evolution. The periodic application of local minimization techniques has shown a significant reduction in both classification and data-fitting problems found in the relevant literature.

Optimizing energy management strategies for microgrids through chaotic local search and particle swarm optimization techniques

The advent of multi-Microgrid (MG) energy systems necessitates the optimization of management strategies to curtail operational costs. This paper introduces an innovative MG energy management strategy that integrates Chaotic Local Search (CLS) with Particle Swarm Optimization (PSO) to fulfill this requirement. Our approach leverages PSO for extensive global exploration and subsequently employs CLS to refine local searches, thereby ensuring the attainment of optimal global outcomes. To further enhance performance, we have crafted a PSO algorithmic framework underpinned by chaotic local search principles, aimed at circumventing regions of local optima. The study presents a comprehensive MG energy system model that encompasses a photovoltaic generation unit, battery energy storage, and a micro gas turbine. The experimental data corroborates that our proposed algorithm secures optimal solutions within a range of 48.2–51.7, outperforming others in achieving these optimal resolutions. When juxtaposed with Scenario 1, there is a significant reduction in both operational and primary energy conversion costs by 24.22 % and 31.39 %, respectively. In comparison to Scenario 2, these figures are reduced by an additional 3.08 % and 6.05 %, respectively. The research findings underscore the strategy's exceptional performance in optimization tasks, as illustrated by the simulation outcomes. The methodology's application to a micro-energy network substantiates its practical relevance. Collectively, this research offers a holistic solution for the optimization of MG energy systems, effectively merging theoretical progress with tangible practical applications.

Bayesian analysis of 1D [formula omitted]H-NMR spectra

Extracting spin system parameters from 1D high resolution 1H-NMR spectra can be an intricate task requiring sophisticate methods. With a few exceptions methods to perform such a total line shape analysis commonly rely on local optimization techniques which for increasing complexity of the underlying spin system tend to reveal local solutions. In this work we propose a full Bayesian modeling approach based on a quantum mechanical model of the spin system. The Bayesian formalism provides a global optimization strategy which allows to efficiently include prior knowledge about the spin system or to incorporate additional constraints concerning the parameters of interest. The proposed algorithm has been tested on synthetic and real 1D 1H-NMR data for various spin systems with increasing complexity. The results show that the Bayesian algorithm provides accurate estimates even for complex spectra with many overlapping regions, and that it can cope with symmetry induced local minima. By providing an unbiased estimate of the model evidence the proposed algorithm furthermore offers a way to discriminate between different spin system candidates.

Data-driven modeling of unsteady flow based on deep operator network

Time-dependent flow fields are typically generated by a computational fluid dynamics method, which is an extremely time-consuming process. However, the latent relationship between the flow fields is governed by the Navier–Stokes equations and can be described by an operator. We therefore train a deep operator network (DeepONet) to learn the temporal evolution between flow snapshots. Once properly trained, given a few consecutive snapshots as input, the network has a great potential to generate the next snapshot accurately and quickly. Using the output as a new input, the network iterates the process, generating a series of successive snapshots with little wall time. Specifically, we consider two-dimensional flow around a circular cylinder at Reynolds number 1000 and prepare a set of high-fidelity data using a high-order spectral/hp element method as ground truth. Although the flow fields are periodic, there are many small-scale features in the wake flow that are difficult to generate accurately. Furthermore, any discrepancy between the prediction and the ground truth for the first snapshots can easily accumulate during the iterative process, which eventually amplifies the overall deviations. Therefore, we propose two alternative techniques to improve the training of DeepONet. The first one enhances the feature extraction of the network by harnessing the “multi-head non-local block.” The second one refines the network parameters by leveraging the local smooth optimization technique. Both techniques prove to be highly effective in reducing the cumulative errors, and our results outperform those of the dynamic mode decomposition method.

Optimisation-based strategies to identification of material parameters of hygro-thermo-mechanical problems

Applications involving hygro-thermo-mechanical models are found in engineering fields as diverse as electronic packaging, concrete structures, composite materials and wood structures. Within the framework of wood processing, hygro-thermo-mechanical problems account for the cross-dependence of moisture content, temperature and dimensional changes of wooden components. Numerical simulation plays an important role in predicting the behaviour of wooden structures and, therefore, use of appropriate process and material parameters is essential for a successful prediction. The present work is inserted in this context and addresses application of optimisation techniques to identification of heat transfer, moisture diffusion and moisture-dependent swelling/shrinkage parameters. The direct problem is formulated based on a fully coupled transient solution of the energy, moisture and momentum transfer conservation laws. A global–local hybrid optimisation technique is proposed combining swarm intelligence and deterministic approaches, respectively based on particle swarm optimisation and the Nelder–Mead (NM) technique. The first stage aims at reducing the search space, so that the NM algorithm is able to determine the global minimum with acceptable accuracy. The strategy is both robust and efficient, being able to avoid local minima with reduced number of fitness computations. An illustrative example featuring a moisture-gaining process is also discussed, in which special attention is placed on the elastic and material swelling coupled effects caused by moisture transfer.

High capacity reversible data hiding in radiographic images with optimal bit allocation

This paper extends and improves the performance of a digital reversible watermarking algorithm based on histogram shifting presented in previous works. The considered algorithm exploits the property of image histograms of some kinds of medical images which present many contiguous 0-runs, i.e., a comb structure in the gray level frequencies. In particular, radiographic images exhibit this structure after contrast enhancement during the acquisition process. The previous work suggested performing gray-level histogram shifting according to a local optimization technique. In this paper, we apply combinatorial optimization techniques to entire blocks of contiguous 0-runs using a non-linear objective function transformed to fit a linear optimization algorithm. The obtained results show a meaningful improvement in the payload capacity of the original data-hiding method. A mild Peak Signal-to-Noise Ratio (PSNR) reduction is still acceptable for a qualitative preview of the images, which can be completely restored to their original cover form thanks to the reversibility of the method.

Computationally efficient parameter estimation for high-dimensional ocean biogeochemical models

Abstract. Biogeochemical (BGC) models are widely used in ocean simulations for a range of applications but typically include parameters that are determined based on a combination of empiricism and convention. Here, we describe and demonstrate an optimization-based parameter estimation method for high-dimensional (in parameter space) BGC ocean models. Our computationally efficient method combines the respective benefits of global and local optimization techniques and enables simultaneous parameter estimation at multiple ocean locations using multiple state variables. We demonstrate the method for a 17-state-variable BGC model with 51 uncertain parameters, where a one-dimensional (in space) physical model is used to represent vertical mixing. We perform a twin-simulation experiment to test the accuracy of the method in recovering known parameters. We then use the method to simultaneously match multi-variable observational data collected at sites in the subtropical North Atlantic and Pacific. We examine the effects of different objective functions, sometimes referred to as cost functions, which quantify the disagreement between model and observational data. We further examine increasing levels of data sparsity and the choice of state variables used during the optimization. We end with a discussion of how the method can be applied to other BGC models, ocean locations, and mixing representations.

Schrödinger's Red Beyond 65,000Pixel-Per-Inch by Multipolar Interaction in Freeform Meta-Atom through Efficient Neural Optimizer.

Freeform nanostructures have the potential to support complex resonances and their interactions, which are crucial for achieving desired spectral responses. However, the design optimization of such structures is nontrivial and computationally intensive. Furthermore, the current "black box" design approaches for freeform nanostructures often neglect the underlying physics. Here, a hybrid data-efficient neural optimizer for resonant nanostructures by combining a reinforcement learning algorithm and Powell's local optimization technique is presented. As a case study, silicon nanostructures with a highly-saturated red color are designed and experimentally demonstrated. Specifically, color coordinates of (0.677, 0.304) in the International Commission on Illumination (CIE) chromaticity diagram- close to the ideal Schrödinger's red, with polarization independence, high reflectance (>85%), and a large viewing angle (i.e., up to ± 25°) is achieved. The remarkable performance is attributed to underlying generalized multipolar interferences within each nanostructure rather than the collective array effects. Based on that, pixel size down to ≈400nm, corresponding to a printing resolution of 65000 pixels per inch is demonstrated. Moreover, the proposed design model requires only ≈300 iterations to effectively search a thirteen-dimensional (13D) design space - an order of magnitude more efficient than the previously reported approaches. The work significantly extends the free-form optical design toolbox for high-performance flat-optical components and metadevices.

Explainable Artificial Intelligence Using Expressive Boolean Formulas

We propose and implement an interpretable machine learning classification model for Explainable AI (XAI) based on expressive Boolean formulas. Potential applications include credit scoring and diagnosis of medical conditions. The Boolean formula defines a rule with tunable complexity (or interpretability) according to which input data are classified. Such a formula can include any operator that can be applied to one or more Boolean variables, thus providing higher expressivity compared to more rigid rule- and tree-based approaches. The classifier is trained using native local optimization techniques, efficiently searching the space of feasible formulas. Shallow rules can be determined by fast Integer Linear Programming (ILP) or Quadratic Unconstrained Binary Optimization (QUBO) solvers, potentially powered by special-purpose hardware or quantum devices. We combine the expressivity and efficiency of the native local optimizer with the fast operation of these devices by executing non-local moves that optimize over the subtrees of the full Boolean formula. We provide extensive numerical benchmarking results featuring several baselines on well-known public datasets. Based on the results, we find that the native local rule classifier is generally competitive with the other classifiers. The addition of non-local moves achieves similar results with fewer iterations. Therefore, using specialized or quantum hardware could lead to a significant speedup through the rapid proposal of non-local moves.

Optimization technique for optimal location selection based on medical image watermarking on healthcare system

The optimization algorithms mimic the process of natural evolution. In watermarking, appropriate positions to insert the watermark is identified by the image that covers. These positions represent the populations of genetic algorithms. The major drawback in genetic algorithm are that it may get stuck-up at a local optimum while moving towards the best global solution and hence the result is poor when compared to other local optimization techniques. The proposed work based on Bandelet based biogeography firefly hybrid algorithms. The Number of pixels, Intensity of the pixel and contrast are considered for watermarking. The redundancy is reduced by Bandelet and used to determine the best location to embed the information into an image both locally and globally. Results of these techniques are compared based on coefficient correlation, index structural similarity, and noise ratio from peak signal.

Inner product preconditioned trust-region methods for frequency-domain full waveform inversion

Full waveform inversion is a seismic imaging method which requires solving a large-scale minimization problem, typically through local optimization techniques. Most local optimization methods can basically be built up from two choices: the update direction and the strategy to control its length. In the context of full waveform inversion, this strategy is very often a line search. We here propose to use instead a trust-region method, in combination with non-standard inner products which act as preconditioners. More specifically, a line search and several trust-region variants of the steepest descent, the limited memory BFGS algorithm and the inexact Newton method are presented and compared. A strong emphasis is given to the inner product choice. For example, its link with preconditioning the update direction and its implication in the trust-region constraint are highlighted. A first numerical test is performed on a 2D synthetic model then a second configuration, containing two close reflectors, is studied. The latter configuration is known to be challenging because of multiple reflections. Based on these two case studies, the importance of an appropriate inner product choice is highlighted and the best trust-region method is selected and compared to the line search method. In particular we were able to demonstrate that using an appropriate inner product greatly improves the convergence of all the presented methods and that inexact Newton methods should be combined with trust-region methods to increase their convergence speed.

Retrieving 3D distributions of atmospheric particles using Atmospheric Tomography with 3D Radiative Transfer – Part 1: Model description and Jacobian calculation

Abstract. Our global understanding of clouds and aerosols relies on the remote sensing of their optical, microphysical, and macrophysical properties using, in part, scattered solar radiation. These retrievals assume that clouds and aerosols form plane-parallel, homogeneous layers and utilize 1D radiative transfer (RT) models, limiting the detail that can be retrieved about the 3D variability in cloud and aerosol fields and inducing biases in the retrieved properties for highly heterogeneous structures such as cumulus clouds and smoke plumes. To overcome these limitations, we introduce and validate an algorithm for retrieving the 3D optical or microphysical properties of atmospheric particles using multi-angle, multi-pixel radiances and a 3D RT model. The retrieval software, which we have made publicly available, is called Atmospheric Tomography with 3D Radiative Transfer (AT3D). It uses an iterative, local optimization technique to solve a generalized least squares problem and thereby find a best-fitting atmospheric state. The iterative retrieval uses a fast, approximate Jacobian calculation, which we have extended from Levis et al. (2020) to accommodate open and periodic horizontal boundary conditions (BCs) and an improved treatment of non-black surfaces. We validated the accuracy of the approximate Jacobian calculation for derivatives with respect to both the 3D volume extinction coefficient and the parameters controlling the open horizontal boundary conditions across media with a range of optical depths and single-scattering properties and find that it is highly accurate for a majority of cloud and aerosol fields over oceanic surfaces. Relative root mean square errors in the approximate Jacobian for a 3D volume extinction coefficient in media with cloud-like single-scattering properties increase from 2 % to 12 % as the maximum optical depths (MODs) of the medium increase from 0.2 to 100.0 over surfaces with Lambertian albedos <0.2. Over surfaces with albedos of 0.7, these errors increase to 20 %. Errors in the approximate Jacobian for the optimization of open horizontal boundary conditions exceed 50 %, unless the plane-parallel media providing the boundary conditions are optically very thin (∼0.1). We use the theory of linear inverse RT to provide insight into the physical processes that control the cloud tomography problem and identify its limitations, supported by numerical experiments. We show that the Jacobian matrix becomes increasing ill-posed as the optical size of the medium increases and the forward-scattering peak of the phase function decreases. This suggests that tomographic retrievals of clouds will become increasingly difficult as clouds become optically thicker. Retrievals of asymptotically thick clouds will likely require other sources of information to be successful. In Loveridge et al. (2023a; hereafter Part 2), we examine how the accuracy of the retrieved 3D volume extinction coefficient varies as the optical size of the target medium increases using synthetic data. We do this to explore how the increasing error in the approximate Jacobian and the increasingly ill-posed nature of the inversion in the optically thick limit affect the retrieval. We also assess the accuracy of retrieved optical depths and compare them to retrievals using 1D radiative transfer.

NURBS-based form-finding algorithm for double-curvature cable domes

Cable domes are one of the most widely used types of cable structures due to their light weight, aesthetic view, and adaptable forms. However, feasibility and structural stability are key elements in designing new forms of cable domes. This paper uses, in a two-stage form-finding algorithm, two NURBS curves to develop a surface of translation (sweeping one curve over the other) or a surface of revolution (rotating one curve around z axis). The main aim of the algorithm is to control the curvature of the dome surface while searching for a feasible geometry, mainly in structures with unfeasible initial configurations, i.e., with zero self-stress states. Examples of these structures are the negative- and zero-Gaussian curvature cable domes of Geiger type. The algorithm starts with a global optimization technique that searches for an optimal feasible NURBS surface for the dome and followed by a local optimization technique that slightly calibrates the vertical locations of nodes to enhance the prestress equilibrium at free nodes. The NURBS surface, not only directs the form-finding process towards a smooth configuration by defining the appropriate degree of the surface, but also maintains the desired sag and rise of negative and positive curves, respectively, to the end of form-finding. The efficiency of the proposed algorithm is verified by applying it to a double-curvature cable dome from the literature. After that, a parametric study is carried out on the reference dome with multiple rise/sag ratios to investigate the effect of rise/sag on the feasibility and stability of such forms. To emphasize the significance of the algorithm in controlling the curvature of the surface, two new forms of cable domes are also developed; one is circular with zero-curvature and the other is elliptic with negative-curvature.

Performance of Heat Transfer in Micropolar Fluid with Isothermal and Isoflux Boundary Conditions Using Supervised Neural Networks

The current study delivers a numerical investigation on the performance of heat transfer and flow of micropolar fluid in porous Darcy structures with isothermal and isoflux walls (boundary conditions) of a stretching sheet. The dynamics and mechanism of such fluid flows are modelled by nonlinear partial differential equations that are reduced to a system of nonlinear ordinary differential equations by utilizing the porosity of medium and similarity functions. Generally, the explicit or analytical solutions for such nonlinear problems are hard to calculate. Therefore, we have designed a computer or artificial intelligence-based numerical technique. The reliability of neural networks using the machine learning (ML) approach is used with a local optimization technique to investigate the behaviours of different material parameters such as the Prandtl number, micropolar parameters, Reynolds number, heat index parameter, injection/suction parameter on the temperature profile, fluid speed, and spin/rotational behaviour of the microstructures. The approximate solutions determined by the efficient machine learning approach are compared with the classical Runge–Kutta fourth-order method and generalized finite difference approximation on a quasi-uniform mesh. The accuracy of the errors lies around 10−8 to 10−10 between the traditional analytical solutions and machine learning strategy. ML-based techniques solve different problems without discretization or computational work, and are not subject to the continuity or differentiability of the governing model. Moreover, the results are illustrated briefly to help implement microfluids in drug administering, elegans immobilization, and pH controlling processes.

Three-dimensional point cloud registration algorithm based on the correlation coefficient

Point cloud registration is an important research problem in machine vision. One of the most widely used methods for point set registration is the iterative closest point (ICP) algorithm. However, ICP is known to be susceptible to local minima because it is based on the local iterative optimization technique and its performance critically relies on the quality of the initialization. To reduce the possibility of the algorithm falling into local minima, we propose an iterative algorithm based on geometric distance and local feature weighting named weighted correlation coefficient iterative closest point algorithm (WCC_ICP). The algorithm first establishes the corresponding relationship of points using linear representation and constructs an optimization model with constraints. Subsequently, the constrained optimization model is converted into an unconstrained optimization model by Lagrangian multiplication. Finally, an iterative technique is used to solve the optimal rigid body transformation of the optimized model. In addition, in each iteration, the identification ability of the local feature operator of points is used to repair the corresponding relation of points, which reduces the risk of the iterative algorithm falling into the local minima. Numerical experiments show that the proposed WCC_ICP algorithm reduces the risk of the iterative algorithm falling into local minima.

Cycle-based formulations in Distance Geometry

The distance geometry problem asks to find a realization of a given simple edge-weighted graph in a Euclidean space of given dimension K, where the edges are realized as straight segments of lengths equal (or as close as possible) to the edge weights. The problem is often modelled as a mathematical programming formulation involving decision variables that determine the position of the vertices in the given Euclidean space. Solution algorithms are generally constructed using local or global nonlinear optimization techniques. We present a new modelling technique for this problem where, instead of deciding vertex positions, the formulations decide the length of the segments representing the edges in each cycle in the graph, projected in every dimension. We propose an exact formulation and a relaxation based on a Eulerian cycle. We then compare computational results from protein conformation instances obtained with stochastic global optimization techniques on the new cycle-based formulation and on the existing edge-based formulation. While edge-based formulations take less time to reach termination, cycle-based formulations are generally better on solution quality measures.

An Optimized Solution to Multi-Constraint Vehicle Routing Problem

A Vehicle Routing Problem (VRP) is a Non-Polynomial Hard Category (NP-hard) problem in which the best set of routes for a convoy of vehicles is traversed to deliver goods or services to a known set of customers. In VRP, some constraints are added to improve performance. Some variations of VRP are Capacitated Vehicle Routing Problem (CVRP), Vehicle Routing Problem with Stochastic Demands (VRPSD), Vehicle Routing Problem with Time Window (VRPTW), Dynamic Vehicle Routing Problem (DVRP), and Vehicle Routing Problem with Simultaneous Pickup and Delivery (VRPSPD) where vehicle and routes have multiple constraints. Swarm intelligence is a well-used approach to solve VRPs. Moreover, different hybrid combinations of global and local optimization techniques are also used to optimize the said problem. In this research, an attempt is made to solve CVRP with VRPSD by using two different hybridized population-based approaches, that is, the Cuckoo Search Algorithm (CSA) and Particle Swarm Optimization (PSO). The experiments showed the accuracy of the improved CVRP that is superior to one obtained by using other classical versions and better than the results achieved by comparable algorithms. Besides, this improved algorithm can also improve search efficiency.

Application of Combined Local and Global Optimization Algorithms in Joint Interpretation of Direct Current Resistivity and Seismic Refraction Data: A Case Study of Dammam Dome, Eastern Saudi Arabia.

The main geological structures in the Dammam Dome are defined by integrating geophysical measurements and applying new methodological approaches. Dammam Dome is characterized by a well-developed fracture/joints system; thus, high complexity of the subsurface is expected. Direct Current Resistivity (DCR) and Seismic Refraction (SR) geophysical survey aimed to map the Dammam Dome's near-surface features. The geophysical data were acquired along two profiles in the northern part of Dammam Dome. To maximize the results from conducting DCR and SR measurements over a complex area, a combined local and global optimization algorithm was used to obtain high-resolution near-surface images in resistivity and velocity models. The local optimization technique involves individual and joint inversion of the DCR and SR data incorporating appropriate regularization parameters, while the global optimization uses single and multi-objective genetic algorithms in model parameter estimation. The combined algorithm uses the output from the local optimization method to define a search space for the global optimization algorithm. The results show that the local optimization produces satisfactory inverted models, and that the global optimization algorithm improves the local optimization results. The joint inversion and processing of the acquired data identified two major faults and a deformed zone with an almost N-S direction that corresponds with an outcrop were mapped in profile one, while profile two shows similar anomalies in both the resistivity and velocity models with the main E-W direction. This study not only demonstrates the capability of using the combined local and global optimization multi-objectives techniques to estimate model parameters of large datasets (i.e., 2D DCR and SR data), but also provides high-resolution subsurface images that can be used to study structural features of the Dammam Dome.

Analysis of heat transmission in convective, radiative and moving rod with thermal conductivity using meta-heuristic-driven soft computing technique.

The present study analyzes the thermal attribute of conductive, convective, and radiative moving fin with thermal conductivity and constant velocity. The basic Darcy’s model is utilized to formulate the governing equation for the problem, which is further nondimensionalized using certain variables. Moreover, an effective soft computing paradigm based on the approximating ability of the feedforword artificial neural networks (FANN’s) and meta-heuristic approach of global and local search optimization techniques is developed to quantify the effect of variations in significant parameters such as ambient temperature, radiation-conduction number, Peclet number, nonconstant thermal conductivity, and initial temperature parameter on the temperature gradient of the rod. The results by the proposed FANN-AOA-SQP algorithm are compared with radial basis function approximation, Runge–Kutta–Fehlberg method and machine-learning algorithms. An extensive graphical and statistical analysis based on solution curves and errors such as absolute errors, mean square error, standard deviations in Nash–Sutcliffe efficiency, mean absolute deviations, and Theil’s inequality coefficient are performed to show the accuracy, ease of implementation, and robustness of the design scheme.Graphical

An unrefinement algorithm for planar THB-spline parameterizations

Unrefinement is a tool that allows to perform faster numerical simulations by controlling the level of precision in the specified area. We introduce an algorithm that creates a coarser geometry from an initial regular geometry, which is represented with respect to THB-splines, so that the coarser geometry approximates the initial geometry with a given level of the precision and is regular. The algorithm is based on the unrefinement of cells of the parameter domain, on which the geometry is defined, thus obtaining the new subdivision of the domain (the new subdomain hierarchy). The algorithm uses local linear projectors and optimization techniques in order to ensure the regularity and the approximation properties.