Adaptive Optimization Method Research Articles

Signal estimation and filtering from quantized observations via adaptive stochastic resonance.

Using a gradient-based algorithm, we investigate signal estimation and filtering in a large-scale summing network of single-bit quantizers. Besides adjusting weights, the proposed learning algorithm also adaptively updates the level of added noise components that are intentionally injected into quantizers. Experimental results show that minimization of the mean-squared error requires a nonzero optimal level of the added noise. The process adaptively achieves in this way a form of stochastic resonance or noise-aided signal processing. This adaptive optimization method of the level of added noise extends the application of adaptive stochastic resonance to some complex nonlinear signal processing tasks.

Intelligent Wavelet Based Pre-Filtering Method for Ultrasonic Sensor Fusion Inverse Algorithm Performance Optimization

Certain obstacle mapping applications require the live evaluation of the measured data to prevent collision with obstacles. The fusion of different or similar sensors usually has a high calculation demand, which increases significantly with the area to be evaluated and the number of sensors. In the present considerations, we propose a wavelet-based adaptive optimization method, which can greatly decrease the number of grid points to be evaluated, and thus the necessary computation time. The basis of the method is to use the fact that the areas to be evaluated mostly face a rather small number of obstacles, which cover a smaller percentage of the whole environment. The first step in a pre-filtering process is the determination of the zones where no obstacles are present. This step can already result in a considerable decrease in the computation time, however with the transformation to polar coordinates, the method will not only be more fitted to the problem to be solved, but the area of the evaluation can also be increased with the same number of grid points. As a last step, we applied wavelet transformation to identify the regions of interest, where the application of a refined raster is necessary, and thus further decreasing the number of grid points where the calculation has to be carried out. We used our previously developed probability-based ultrasonic sensor fusion inverse algorithm to demonstrate the efficiency of the proposed method.

Path planning of surgical needle: A new adaptive intelligent particle swarm optimization method

Percutaneous puncture interventional therapy is an important method for pathological examination, local anesthesia, and local drug delivery in modern clinics. Due to the existence of complex obstacles such as nerves, arteries, bones and so on in the puncture path, it is a challenging work to design the optimal path for surgical needle. In this paper, we propose a new path planning method based on the adaptive intelligent particle swarm optimization (PSO) algorithm with parameter adjustment mechanism. First, force and motion analysis are carried out on the bevel-tip flexible needle after piercing into human tissues, the motion model of the needle and the spatial transformation model of puncture route in three-dimensional space are obtained, respectively. Then, a multi-objective function is established, which includes puncture path length function, puncture error function and collision detection function. Finally, the optimal puncture path is obtained based on the adaptive intelligent PSO algorithm. The simulation results show that the newly proposed path planning method has higher efficiency, better adaptability to complex environments and higher accuracy than other path planning methods in literature.

Intelligent injection molding: Parameters self‐learning optimization using iterative gradient‐approximation adaptive method

AbstractIntelligent injection molding consists of three aspects: intelligent parameter optimization, process monitoring and control. The optimal process parameters are critical to guarantee product quality. Injection molding is a typical batch process and has the property that previous runs can provide feedback to optimize subsequent runs. This study proposes a self‐learning parameter optimization method named iterative gradient‐approximation adaptive optimization (IGAO) method, which adopts the batch‐to‐batch information to remove the need to establish optimization model with large numbers of experiments. The analysis and the optimization of the parameters can be performed simultaneously. The IGAO method approximates the gradient iteratively and assigns an adaptive step size to each parameter according to gradient accumulation. The experiments conducted in both simulation software and injection molding machine prove that the method has fast convergence speed. Standard product weight can be obtained within 11 runs from three different starting process parameters. Experiment results show that 25% less steps are needed compared with the traditional gradient descent method. The method also has good stability to resist disturbances during the optimization procedure. In general, the proposed IGAO method is fast, stable and robust, and it has good prospects for parameter optimization in batch processes.

Inverse Optimal Design of Direct Adaptive Fuzzy Controllers for Uncertain Nonlinear Systems

Optimized performance obtained from existing adaptive fuzzy optimal control methods comes at the cost of a intricate design procedure and a heavy computation of online parameter learning, and it is an under-explored problem on how to remove such a restriction. In this article, we tackle this problem and ensure the optimized performance using only one adaptive parameter. To this end, a direct adaptive fuzzy inverse approach is first proposed to design a switching-type inverse optimal controller and a one-parameter learning mechanism. It is proved that the proposed approach ensures the input-to-state stability of the control system and besides, the inverse optimality in regard to a meaningful cost functional is achieved. Illustrative examples verify the approach developed.

Simulation physics-informed deep neural network by adaptive Adam optimization method to perform a comparative study of the system

Taking the marvelous advantages of artificial intelligence (AI) in accelerating the procedure of finding a solution to different engineering analyses is the main motivation of this article to establish a non-model-based mechanism on the basics of fully connected deep neural networks (FC-DNN) to analyze the hygro-thermomechanical buckling response of the multiscale hybrid composite MHC doubly curved panel. First, the system's buckling response at its design points is obtained by applying DQM to motion equations developed based upon the refined-form of third-order shear deformation theory (TSDT). Then the obtained information would be transferred to DNN to acquire the regressor system. Finding the optimal values of weights and biases of the DNN is the key factor to provide an AI system with high-accuracy prediction. For this reason, the adaptive Adam optimization approach is chosen due to its phenomenal speed as well as lower computational costs.

Vague Adaptive Optimization Based Research on Western Guangdong Province Favorable Natural Rubber Species

Apply vague adaptive optimization method to find new natural rubber species. It is advisable to accelerate the promotion of wind-resistant and high-yield natural rubber species’ plantation in the dominant areas of western Guangdong Province, which are Reyan 7-33-97, Yunyan 77-2, Dafeng-95, Yunyan 77-4, Haiken 1, PR 107, RRIM 600, RRII 208 and other species.

An Adaptive Optimization Method Based on Learning Rate Schedule for Neural Networks

Artificial intelligence (AI) is achieved by optimizing the cost function constructed from learning data. Changing the parameters in the cost function is an AI learning process (or AI learning for convenience). If AI learning is well performed, then the value of the cost function is the global minimum. In order to obtain the well-learned AI learning, the parameter should be no change in the value of the cost function at the global minimum. One useful optimization method is the momentum method; however, the momentum method has difficulty stopping the parameter when the value of the cost function satisfies the global minimum (non-stop problem). The proposed method is based on the momentum method. In order to solve the non-stop problem of the momentum method, we use the value of the cost function to our method. Therefore, as the learning method processes, the mechanism in our method reduces the amount of change in the parameter by the effect of the value of the cost function. We verified the method through proof of convergence and numerical experiments with existing methods to ensure that the learning works well.

Multi-objective two-stage adaptive robust planning method for an integrated energy system considering load uncertainty

An integrated energy system (IES) is considered as an effective approach to reduce carbon emissions, lower energy supply costs and increase system flexibility. As numerous energy conversion and storage devices with various features have been developed, determining their types and capacity size and optimizing the synergic action of all selected energy devices under load uncertainty have become challenging issues during the planning and designing of a new IES. To address these issues, a load uncertainty model is constructed by integrating the regional equivalent standard building-integrated load prediction method, the robust uncertainty set method and the fuzzy C-means clustering algorithm. Based on this uncertainty model, this paper proposes a planning method that incorporates the fuzzy multi-objective decision and two-stage adaptive robust optimization methods for handling these challenging issues. The multi-objective decision method enables comprehensive optimization based on the system's economic performance, carbon emissions and grid integration level. The two-stage adaptive robust optimization method aims to optimize the single-objective problem transformed by the multi-objective decision method. The planning method is implemented in an industrial park in the Baoding City of China. The influence of load budget uncertainty, annual load growth rate and energy purchase price is then investigated. The results show that the load budget uncertainty and annual load growth rate only affect the optimal capacity size of the selected devices, whereas the energy purchase price influences both the optimal device types and their capacity size. The optimal capacity size remains constant when the load budget uncertainty exceeds 3. This study provides a methodology to select the suitable device types and determine their capacity size for IES under load uncertainty, which will benefit the design of a new IES.

An adaptive RBF neural network–based multi-objective optimization method for lightweight and crashworthiness design of cab floor rails using fuzzy subtractive clustering algorithm

To improve the computational cost and accuracy of approximate-based multi-objective optimization problems in engineering, an adaptive RBF neural network (ARBFNN) method integrating RBF neural network model, fuzzy subtractive clustering (FSC) sequence sampling method and non-dominated sorting genetic algorithm (NSGA-II) is proposed. To solve the problem that the number of new sample points is difficult to determine, FSC sequence sampling is proposed to select the clustering center points as newly added sample points. First, the ARBFNN method is verified by the five test functions. The results show that the ARBFNN method is significantly better than static RBFNN model based on multi-objective optimization (SRBFNN) in terms of global convergence and efficiency performance, and the other two adaptive approximate optimization methods in terms of overall efficiency. Based on the above, the accuracy and efficiency of ARBFNN method are very high. Finally, the method is applied to an engineering example: the lightweight and crashworthiness design of floor rails. The cab model coupled with implicit parameterized floor rails model is built using the SFE-CONCEPT software to achieve collaborative optimization design of shape-size-material. The optimization results show that the ARBFNN method can guarantee the error of the approximate optimal solution and the finite element solution (expensive solution) within 2%, so the accuracy of highly nonlinear (finite element analysis of collision conditions) approximate optimization is improved. Hence, the proposed ARBFNN method is feasible and effective in solving complex and expensive engineering optimization problems.

Crude palm oil price prediction using multilayer perceptron and long short-term memory

Crude Palm Oil is a leading commodity from Indonesia. Accurate prediction of Crude Palm Oil prices is very important to ensure future prices and help decision making. Study on crude palm oil prices is needed to anticipate fluctuations. In this study, prediction model was made using Multilayer Perceptron and Long Short-Term Memory. The optimization methods in this study are Stochastic Gradient Descent, Root Mean Square Propagation, and Adaptive Moment Estimation. The best model is selected based on Mean Square Error. Based on the results, Long Short-Term Memory model with Adaptive Moment Estimation optimization method is more optimal than Long Short-Term Memory with Stochastic Gradient Descent and Long Short-Term Memory with Root Mean Square Propagation. The prediction results using Long Short-Term Memory with Adam optimization show that the predicted value is not different from the actual value and Mean Absolute Percentage Error is 2.11%. This model has high forecasting accuracy because Mean Absolute Percentage Error is less than 10%.

A Two-stage Adaptive Robust Model for Residential Micro-CHP Expansion Planning

This paper addresses the planning problem of residential micro combined heat and power (micro-CHP) systems (including micro-generation units, auxiliary boilers, and thermal storage tanks) considering the associated technical and economic factors. Since the accurate values of the thermal and electrical loads of this system cannot be exactly predicted for the planning horizon, the thermal and electrical load uncertainties are modeled using a two-stage adaptive robust optimization method based on a polyhedral uncertainty set. A solution method, which is composed of column-and-constraint generation (C&CG) algorithm and block coordinate descent (BCD) method, is proposed to efficiently solve this adaptive robust optimization model. Numerical results from a practical case study show the effective performance of the proposed adaptive robust model for residential micro-CHP planning and its solution method.

An Adaptive Deep Learning Optimization Method Based on Radius of Curvature.

An adaptive clamping method (SGD-MS) based on the radius of curvature is designed to alleviate the local optimal oscillation problem in deep neural network, which combines the radius of curvature of the objective function and the gradient descent of the optimizer. The radius of curvature is considered as the threshold to separate the momentum term or the future gradient moving average term adaptively. In addition, on this basis, we propose an accelerated version (SGD-MA), which further improves the convergence speed by using the method of aggregated momentum. Experimental results on several datasets show that the proposed methods effectively alleviate the local optimal oscillation problem and greatly improve the convergence speed and accuracy. A novel parameter updating algorithm is also provided in this paper for deep neural network.

Robustness of Adaptive Neural Network Optimization Under Training Noise

Adaptive gradient methods such as adaptive moment estimation (Adam), RMSProp, and adaptive gradient (AdaGrad) use the temporal history of the gradient updates to improve the speed of convergence and reduce reliance on manual learning rate tuning, making them a popular choice for off-the-shelf Deep Neural Network (DNN) optimizers. In this article, we study the robustness of neural network optimizers in the presence of training perturbations. We show that popular adaptive optimization methods exhibit poor generalization while learning from noisy training data, compared to vanilla Stochastic Gradient Descent (SGD) and its variants, which manifest better implicit regularization properties. We construct an illustrative example of a family of two-class linearly separable toy-data such that models trained under noise using adaptive optimizers show only 52% test accuracy (random classifier), whereas SGD-based methods can achieve 100% test accuracy. We strengthen our hypothesis by empirical analysis using Convolutional Neural Networks (CNNs) on publicly available image datasets. For this purpose, our method trains neural network models with various optimizers on noisy training data, and we compute test accuracy on clean test data. Our results further highlight the robustness of SGD optimization against such noisy training data compared to its adaptive counterparts. Based on the results, our paper suggests a reconsideration of the extensive use of adaptive gradient methods for neural network optimization, especially when the training data is noisy.

Adaptive powered descent guidance based on multi-phase pseudospectral convex optimization

The real-time and fuel-optimal powered descent guidance is one of the critical technologies for planetary landing. This work focuses on the powered descent guidance algorithm based on pseudospectral convex optimization and aims to increase the accuracy and adaptability of the algorithm. The entire powered descent process is divided into phases according to the thrust switching times, and the pseudospectral method is adopted to discretize the optimization problem. Convexification process is performed to transform the optimization problem into a convex programming problem, and a successive solving process algorithm is proposed. For the established multi-phase pseudospectral convex optimization formulation, the indirect method is adopted to derive and determine the optimal phase division from switching function, and the adaptive multi-phase pseudospectral convex optimization method is proposed. Numerical examples demonstrate that the proposed method has excellent accuracy and adaptability.

Hybrid Design Optimization of Heating Furnace Temperature using ANFIS-PSO

-- Intelligent control design for industrial heating furnace temperature control is indispensable. PID, Fuzzy, and ANFIS controllers have been proven reliable and have been widely used. However, it is constrained in choosing a better gain controller. Then an approach method is given to determine the most appropriate controller gain value using the artificial intelligence tuning method. The artificial intelligence method used is a combination of the Adaptive Neuro Fuzzy Inference System and Particle Swarm Optimization (ANFIS-PSO) methods. As a comparison, several methods were used, namely; Conventional PID (PID-Konv), Matlab Auto tuning PID (PID-Auto), PSO tuned PID (PID-PSO), and Hybrid ANFIS-PSO. The ANFIS-PSO controller is the best choice compared to conventional single loop control systems, conventional PID, and matlab 2013a auto tuning methods to control this nonlinear process. The simulation results show that the ANFIS-PSO design is the best method with overshot = 0.0722, undershot 0.0085, and settling time at 18.8789 seconds which can produce a fast response with strong dynamic performance.

Adaptive Optimization Method in Digital Twin Conveyor Systems via Range-Inspection Control

The automated conveyor system, as the core component in the modern manufacturing world, has gained lots of attention from researchers. To optimize the operation of the conveyor system, range-inspection control (RIC) has been considered an efficient strategy to bring this conventional technology to an intelligent level. Various algorithms have been put into use to achieve optimal control. However, the current methodologies are only focusing on control optimization, not scaled into the smart manufacturing framework. The schema of alignment and corporation between the physical and virtual spaces for the system remains an important problem. Therefore, the work in this article aims for an effective framework of implementation between the physical and virtual stations in an automated conveyor system. Since increasingly more application scenarios rely on the digital twin (DT) technology to realize the integration of physical and virtual systems, we proposed the DT automated conveyor system (DT-ACS) that constructs the road map to implement the RIC-based conveyor system under the background of a smart factory. Besides, profit-sharing-based deep Q-networks (PDQNs) have been proposed to cope with the RIC optimization problem. The robustness and efficiency of the proposed PDQN were evaluated via sets of experiments. The discussion and conclusion are presented at last accordingly. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article aims to propose a strategy of control optimization for conveyor-based manufacturing systems under the digital twin (DT) framework. The conveyor system can be flexible to control the running flows to avoid overloading workstations. Due to the complex environment in the production line, the range that is able to be inspected and the capacity of the reserve area can be considerably diverse among the workstations. To maximally evaluate our framework, we set a comparatively complex environment for the experiments. Nevertheless, to obtain practically ideal performance under other circumstances, the parameters should be precisely tested and fine-tuned with simulation in advance.

Robust Transmission Expansion Planning Based on Adaptive Uncertainty Set Optimization Under High-Penetration Wind Power Generation

Previous robust transmission expansion planning (RTEP) studies have rarely considered the important question of how large the uncertainty set should be, and hence, how the uncertainty budget can be determined objectively. This study addresses this issue by proposing a novel adaptive optimization method that optimizes the value of the uncertainty budget to minimize the size of the uncertainty set while considering the underlying risk for wind power generation (WPG) fluctuations residing outside of the proposed uncertainty set. As such, the proposed method ensures a good tradeoff between the robustness and costs of RTEP solutions. In addition, the proposed method optimizes investment strategies under forecasted WPG scenarios, while providing a security guarantee under extreme WPG scenarios. The variable-limit integral terms introduced by the proposed method are addressed in the solution process by applying the piecewise linearization approximation method combined with the quadratic Newton-Gregory interpolating polynomial technique, which allows the solution process to be cast as a mixed integer linear programming problem. The good performance and effectiveness of the proposed adaptive uncertainty set optimization method are verified by numerical results.

Accelerating full waveform inversion through Adaptive Gradient Optimization methods and Dynamic Simultaneous Sources

Summary Full-Waveform Inversion (FWI) is a procedure based on the minimization of a misfit (or cost) function applied to the difference between synthetic waveforms and real seismic traces that derives high-resolution velocity models. This is achieved through the iterative adjustment of the velocity model and/or some other physical parameters of the Earth’s subsurface, which generally implies large computational effort. In order to minimize this cost function we explore the use of Adaptive Gradient Optimization (AGO), a variant of Stochastic Gradient Descent (SGD) methods, combining them with a dynamic simultaneous sources strategy that allow us to reduce the computational cost involved in this process. AGO methods are computationally efficient, have little memory requirements and have the capability of adapting the step-length according to the optimization process’ evolution. Since a precise calibration of the step-length is needed to ensure efficiency, the AGOs are well-suited for this task because they are able to adapt the step-length according to the optimization’s development. In this work, we propose a simple non-linear relationship that allows an adjustment of the step-length with respect to the frequencies used in the multiscale FWI, avoiding the line-search strategy’s high computational burden. Additionally, the application of this new step-length rule into the AGO methods with a dynamic simultaneous sources strategy, allow us to concurrently accelerate and significantly improve the FWI’s numerical performance and results. We compare the performance and final results of seven AGO methods, using two different FWI misfit functionals (based on L1 and L2 norms) applied to estimate the final velocity models of two benchmark acoustic models: the Marmousi and the Canadian overthrust BP velocity models.

Adaptive uncertainty sets‐based two‐stage robust optimisation for economic dispatch of microgrid with demand response

It is an effective way to regard the electric vehicles as the demand response for reducing the negative impact of large-scale introduction on the power system. Aiming at the microgrid with demand response, the adaptive uncertainty sets-based two-stage robust optimisation method is established in this study. The coordination of micro-gas turbine, energy storage, and demand response etc. are considered in the economic dispatch model. To effectively consider the uncertain variable contained in the microgrid, the concept of adaptive uncertainty sets is proposed in this study. The uncertainty sets are achieved by the long short-term memory network and modified fuzzy information granulation. To handle the adaptive uncertainty sets-based robust optimisation model, the column and constraint generation algorithm and strong duality theory are introduced to decompose the model into a master problem and a subproblem with mixed-integer linear structure. To verify the performance of the proposed adaptive uncertainty sets-based two-stage robust optimisation method, measured data from a plateau city of China are introduced in the simulation test. The simulation results demonstrate the effectiveness of the model and solution strategy.