Model-based Optimization Algorithm Research Articles

A novel deep learning model-based optimization algorithm for text message spam detection

A novel deep learning model-based optimization algorithm for text message spam detection

Enhancing clay content estimation through hybrid CatBoost-GP with model class selection

Excessive soil swelling, triggered by variations in moisture content, is recognized as one of the most severe geo-hazards. Swelling Potential (SP) is closely correlated with Clay Content (CC), Plasticity Index (PI), and Activity (A = PI/CC), which are crucial input features for SP prediction models. Traditional methods for determining CC, such as the hydrometer and pipette methods, are labor-intensive and time-consuming. Employing easily accessible soil properties to predict CC directly not only enhances the modeling efficiency of SP prediction models but also mitigates the impact of laboratory test errors on model performance. This study leveraged two databases encompassing a broad spectrum of soil classifications and developed three ensemble learning models (RF, XGBoost, and CatBoost) to predict CC efficiently. Sequential model-based optimization algorithms (SMBO) were employed to effectively identify the optimal hyperparameters for these models. To further refine the model, the Shapley additive explanation (SHAP) method was utilized to ascertain the most influential soil properties. The findings revealed that the CatBoost model, optimized via GP, achieved the highest level of predictive accuracy. Three easily measurable soil properties — the percentage of soil particles passing through 0.075 mm and 0.425 mm sieves, along with the plasticity index — were identified as key predictors for CC. The estimated CC values were instrumental in constructing various empirical and machine learning models to predict SP, all of which exhibited exceptional performance. This research advances the modeling efficiency of SP prediction models, making it particularly relevant for rapid-response scenarios in transportation infrastructure projects.

Surrogate model-based multi-objective optimization algorithm for train-ballast turnout system: Balancing safety and vibration damping in parameter design

In rail transportation structural parameter design, multi-objective optimal algorithms (MOOAs) have advanced significantly in safety, comfort, and aerodynamics domain. However, the vital factor of environment vibration remains neglected, impeding depot cover technology advancements. To address this challenge, with a focus on both safety and vibration damping. Firstly, the mechanistic models are constructed to establish a surrogate modeling database. Subsequently, a surrogate model is proposed, integrating stacked denoising autoencoders (SdAEs), self-attention (SA), and the gated recurrent unit (GRU). In addition, A novel MOOA, Dynamic Population-Multi-Objective Harris Hawks Optimization-Crowding Distance-Elitist Learning Strategy (D-MOHHO-CD-ELS), is proposed and its efficacy is validated against five existing MOOAs using the CTP1∼CTP7 test functions. Moreover, the train-ballast No.12 turnout system is considered as an engineering case. Importantly, the accuracy of both the mechanistic model and surrogate model is thoroughly validated. Among them, the performance of the data preprocessing and prediction algorithms is substantiated through a comparison of the proposed SdAEs-SA-GRU model against simulation results and eight other algorithms. Furthermore, the surrogate-based D-MOHHO-CD-ELS method, applied to tackle a multi-objective optimization problem with nine optimization parameters and four optimization objectives, demonstrates an efficiency improvement of 1 to 2 orders of magnitude compared to traditional methods. Consequently, following optimization, the wheel load reduction rate, derailment coefficient, capsizing coefficient and vibration damping effect are improved by 15.5%, 11.1%, 5.5% and 24.8% respectively, which reveals the effectiveness of enhancements in both the safety and damping efffect. the study offers a novel perspective for parameter design of TBT.

A three-dimensional Magnetic Resonance Spin Tomography in Time-domain protocol for high-resolution multiparametric quantitative magnetic resonance imaging.

Magnetic Resonance Spin TomogrAphy in Time-domain (MR-STAT) is a multiparametric quantitative MR framework, which allows for simultaneously acquiring quantitative tissue parameters such as T1, T2, and proton density from one single short scan. A typical two-dimensional (2D) MR-STAT acquisition uses a gradient-spoiled, gradient-echo sequence with a slowly varying RF flip-angle train and Cartesian readouts, and the quantitative tissue maps are reconstructed by an iterative, model-based optimization algorithm. In this work, we design a three-dimensional (3D) MR-STAT framework based on previous 2D work, in order to achieve better image signal-to-noise ratio, higher though-plane resolution, and better tissue characterization. Specifically, we design a 7-min, high-resolution 3D MR-STAT sequence, and the corresponding two-step reconstruction algorithm for the large-scale dataset. To reduce the long acquisition time, Cartesian undersampling strategies such as SENSE are adopted in our transient-state quantitative framework. To reduce the computational burden, a data-splitting scheme is designed for decoupling the 3D reconstruction problem into independent 2D reconstructions. The proposed 3D framework is validated by numerical simulations, phantom experiments, and in vivo experiments. High-quality knee quantitative maps with 0.8 × 0.8 × 1.5 mm3 resolution and bilateral lower leg maps with 1.6 mm isotropic resolution can be acquired using the proposed 7-min acquisition sequence and the 3-min-per-slice decoupled reconstruction algorithm. The proposed 3D MR-STAT framework could have wide clinical applications in the future.

A back analysis scheme for refined soil stratification based on integrating borehole and CPT data

Utilizing both borehole and Cone Penetration Testing (CPT) data in soil stratification helps to get more convincing soil stratification results. However, the soil classification results revealed by borehole (Unified Soil Classification System, USCS) and CPT tests (soil behavior type, SBT) are commonly not consistent. This study proposes a feasible solution to integrate the borehole and CPT data with the tree-based method. The tree-based method is naturally suitable for soil stratification tasks as it aims to divide the subsurface space into several clusters based on the similarities of the soil types. A novel boundary dictionary method is proposed to enhance the model performance on complex soil layer conditions. A probabilistic mapping matrix between the USCS-SBT system is built based on a collected municipal database with collocated borehole and CPT data. The optimal soil stratification results can be selected based on considering multiple borehole information and pruning the structure of trees. The structure of the trees can be optimized in a back analysis perspective with the Sequential Model-Based Global Optimization (SMBO) algorithm which aims to maximize the possibility of observing the borehole information based on the USCS-SBT probabilistic mapping matrix. The uncertainties of the optimal soil stratification results can be estimated based on a weighted Gini index method. The performance of the proposed method is validated based on a real case in New Zealand with a cross-validation method. The results indicate that the proposed method is robust and effective.

Economic evaluation of energy storage integrated with wind power

Energy storage can further reduce carbon emission when integrated into the renewable generation. The integrated system can produce additional revenue compared with wind-only generation. The challenge is how much the optimal capacity of energy storage system should be installed for a renewable generation. Electricity price arbitrage was considered as an effective way to generate benefits when connecting to wind generation and grid. This wind-storage coupled system can make benefits through a time-of-use (TOU) tariff. A proportion of electricity is stored from the wind power system at off-peak time (low price), and released to the customer at peak time (high price). Thus, extra benefits are added to the wind-storage system compared with wind-only system. A Particle Swarm Optimization (PSO) algorithm based optimization model was constructed for this integrated system including constraints of state-of-charge (SOC), maximum storage and release powers etc. The proposed optimization model was to obtain the optimal capacity of energy storage system and its operation control strategy of the storage-release processes, to maximize the revenue of the coupled system considering the arbitrage. Furthermore, the energy storage can provide reserve ancillary services for the grid, which generates benefits. The benefits of energy storage system through reserve ancillary services were also calculated. A case study was analyzed with respect to yearly wind generation and electricity price profiles. The benefit compared with no energy storage scenario was calculated. The impact of the energy storage efficiency, cost and lifetime was considered. The sensitivity and optimization capacity under various conditions were calculated. An optimization capacity of energy storage system to a certain wind farm was presented, which was a significant value for the development of energy storage system to integrate into a wind farm.

Adaptive Optimization Operation of Electric Vehicle Energy Replenishment Stations Considering the Degradation of Energy Storage Batteries

As the construction of supporting infrastructure for electric vehicles (EV) becomes more and more perfect, an energy replenishment station (ERS) involving photovoltaics (PV) that can provide charging and battery swapping services for electric vehicle owners comes into the vision of humanity. The operation optimization of each device in the ERS is conducive to improving the service capacity of the ERS, extending the service life of the energy storage batteries (ESB), and enhancing the economic benefits of the ERS. However, traditional model-based optimization algorithms cannot fully consider the stochastic nature of EV owners’ charging and battery swapping demands, the uncertainty of PV output, and the complex operating characteristics of ESB. Therefore, we propose a deep reinforcement learning-based adaptive optimal operation method for ERS considering ESB’s losses. Firstly, a mathematical model of each device in the ERS is established, and a refined energy storage model is introduced to describe ESB’s capacity degradation and efficiency decay. Secondly, to solve the dimensional disaster problem, the state space and action space selection method, and the charging strategy of batteries in the battery swapping station (BSS) are proposed to apply to the ERS, thus modeling the ERS optimization operation problem as a Markov decision process. Then, the solution is performed using a neural network-based proximal policy optimization (PPO) algorithm, consisting of a recurrent neural network that extracts information about the PV outflow trend and a deep neural network used to generate a control policy. Finally, the effectiveness of the proposed method is verified by simulation calculations, which not only enable adaptive decision-making under different PV output scenarios, but also consider the availability of EV battery swapping services, energy storage losses, and the economic benefits of the ERS.

From Model-Based Optimization Algorithms to Deep Learning Models for Clustering Hyperspectral Images

Hyperspectral images (HSIs), captured by different Earth observation airborne and space-borne systems, provide rich spectral information in hundreds of bands, enabling far better discrimination between ground materials that are often indistinguishable in visible and multi-spectral images. Clustering of HSIs, which aims to unveil class patterns in an unsupervised way, is highly important in the interpretation of HSI, especially when labelled data are not available. A number of HSI clustering methods have been proposed. Among them, model-based optimization algorithms, which learn the cluster structure of data by solving convex/non-convex optimization problems, have achieved the current state-of-the-art performance. Recent works extend the model-based algorithms to deep versions with deep neural networks, obtaining huge breakthroughs in clustering performance. However, a systematic survey on the topic is absent. This article provides a comprehensive overview of clustering methods of HSI and tracked the latest techniques and breakthroughs in the domain, including the traditional model-based optimization algorithms and the emerging deep learning based clustering methods. With a new taxonomy, we elaborated on the main ideas, technical details, advantages, and disadvantages of different types of clustering methods of HSIs. We provided a systematic performance comparison between different clustering methods by conducting extensive experiments on real HSIs. Unsolved problems and future research trends in the domain are pointed out. Moreover, we provided a toolbox that contains implementations of representative clustering algorithms to help researchers to develop their own models.

Model-based linear–quadratic–integral controller for simultaneous regulation of the current profile and normalized beta in NSTX-U

Achieving advanced scenarios that are characterized by steady-state operation, stable plasma confinement, and high-performance plasmas is one of the primary objectives of the National Spherical Tokamak eXperiment-Upgrade (NSTX-U). Active control of the plasma may be necessary to achieve these conditions. In particular, control algorithms that can simultaneously optimize the shapes and values of plasma profiles and scalars, respectively, may play a critical role in robustly achieving and sustaining these advanced scenarios. In this work, a model-based optimal control algorithm is developed for feedback control of the current profile in NSTX-U. The linear, finite-dimensional, control model is derived by discretizing and linearizing the magnetic diffusion equation in combination with empirical correlations for electron density, electron temperature, and noninductive current drives. The linear, time-variant model is then used for designing a linear–quadratic–integral (LQI) controller that is capable of regulating both the safety factor and the normalized beta around desired targets. The controller determines the neutral beam injection powers and the overall plasma current that are needed to achieve the desired current profile and normalized beta. The proposed controller is tested in higher-fidelity nonlinear simulations that employ 1D models for the evolutions of both current and temperature profiles using the Control Oriented Transport SIMulator (COTSIM). The closed-loop simulations show the effectiveness of the controller at shaping the safety factor in NSTX-U while achieving the desired normalized beta.

A data-driven approach towards fast economic dispatch in integrated electricity and natural gas system

Effective economic dispatch for the integrated energy system (IES) can improve energy efficiency and promote renewable energy accommodation. Tradition IES economic dispatch are based on model-based methods that rely on accurate system parameters and uncertainty prediction. This paper proposes a data-driven fast economic dispatch method based on deep reinforcement learning (DRL) in an integrated electricity and natural gas system (IEGS). Unlike other DRL-based studies that spend a lot of computation time to calculate power flow, we employ DNNs to learn the complex nonlinear relationship existing in the IEGS, namely IEGSNet. Then, the economic dispatch problem is formulated as a Markov decision process, and solved by the soft actor–critic​ (SAC) algorithm. Simulation results illustrate that the proposed method achieves the similar operation cost to the tradition DRL-based dispatch method, but takes almost one tenth of the training time. Additionally, the computation time of the proposed method for 10-day dataset is at least two orders of magnitudes shorter that the model-based optimization algorithm.

Genetic Algorithm Based Optimization Model for Time-Cost Trade-Off for Construction Project

Abstract: The main cause of gradual development of project management is the necessity of controlling and optimizing the construction project’s objectives. In planning phase of construction project management, some objectives of proposed project are required to be set as per stakeholder’s perspective. Time and cost are of paramount importance objectives of construction project, which vary due to variation in the resource utilization amount. High-cost resources and advance technologies reduce the project time but make the project cost higher. While low-cost resources and traditional technologies give lower project cost but increase the project time. Basically, a construction project is said to be successful if it is completed in minimum possible time and cost. Therefore, the two fundamental goals of any building project are to do it as quickly and inexpensively as possible. The development of time-cost trade-off models has received a lot of focus. However, in addition to a wide range of approaches for time-cost trade-off (TCT) models, this work offers a TCT model based on a genetic algorithm. This model was created in a way that makes it easier to find the best approaches to complete the project on time and for the lowest possible cost. The applicability of proposed TCT model is demonstrated through solving two practically existing construction project. Outcomes of proposed model were found satisfactory based on statistical analysis.

MoG-DS: model-guided deep convolutional network for joint denoising and super-resolution of a single-photon counting image

Single-photon counting (SPC) imaging has attracted considerable research attention in recent years due to its capability to detect targets under extremely low-light conditions. However, the spatial quality of SPC images is always unsatisfactory because they typically suffer from considerable effects of noise and their spatial resolution is low. Most traditional methods are dedicated to solving the noise problem while ignoring the improvement of spatial resolution. To address these challenging issues, we propose a novel model-guided deep convolutional network for joint denoising and super-resolution (SR) of SPC images. First, we introduce a model-based iterative optimization algorithm with deep regularizer to unify denoising and SR into one problem. Second, we construct a model-guided deep convolutional network by unfolding the aforementioned model-based iterative algorithm to achieve an optimal solution. All modules in the proposed network are interpretable due to the special model-guided design, and they enable good generalization in real situations. In addition, the deep regularizer and other parameters in the proposed network are jointly optimized in an end-to-end manner, which efficiently reduces the difficulty of parameter design. Extensive simulation and real experimental results are reported to demonstrate the superiority of the proposed method in terms of visual comparison and quantitative analysis, respectively.

Derivation of unit hydrograph using genetic algorithm-based optimization model

Derivation of unit hydrograph using genetic algorithm-based optimization model

Model-based three-material decomposition in dual-energy CT using the volume conservation constraint

Objective. We develop a model-based optimization algorithm for ‘one-step’ dual-energy (DE) CT decomposition of three materials directly from projection measurements. Approach. Since the three-material problem is inherently undetermined, we incorporate the volume conservation principle (VCP) as a pair of equality and nonnegativity constraints into the objective function of the recently reported model-based material decomposition (MBMD). An optimization algorithm (constrained MBMD, CMBMD) is derived that utilizes voxel-wise separability to partition the volume into a VCP-constrained region solved using interior-point iterations, and an unconstrained region (air surrounding the object, where VCP is violated) solved with conventional two-material MBMD. Constrained MBMD (CMBMD) is validated in simulations and experiments in application to bone composition measurements in the presence of metal hardware using DE cone-beam CT (CBCT). A kV-switching protocol with non-coinciding low- and high-energy (LE and HE) projections was assumed. CMBMD with decomposed base materials of cortical bone, fat, and metal (titanium, Ti) is compared to MBMD with (i) fat-bone and (ii) fat-Ti bases. Main results. Three-material CMBMD exhibits a substantial reduction in metal artifacts relative to the two-material MBMD implementations. The accuracies of cortical bone volume fraction estimates are markedly improved using CMBMD, with ∼5–10× lower normalized root mean squared error in simulations with anthropomorphic knee phantoms (depending on the complexity of the metal component) and ∼2–2.5× lower in an experimental test-bench study. Significance. In conclusion, we demonstrated one-step three-material decomposition of DE CT using volume conservation as an optimization constraint. The proposed method might be applicable to DE applications such as bone marrow edema imaging (fat-bone-water decomposition) or multi-contrast imaging, especially on CT/CBCT systems that do not provide coinciding LE and HE ray paths required for conventional projection-domain DE decomposition.

Performance-based data-driven optimal tracking control of shape memory alloy actuated manipulator through reinforcement learning

This article focuses on the continuous-time optimal tracking control problem of a shape memory alloy (SMA) actuated manipulator subject to prescribed error constraints and completely unknown nonlinear dynamics. Firstly, prespecified error constraints imposed by the prescribed performance function (PPF) are transformed into an equivalent unconstrained task by adopting a transformation function, where the newly designed PPF is irrelated to the initial condition. An unconstrained augmented system and a long-term discounted performance considering tracking errors and input cost are constructed for the convenience of designing the optimal controller. Then, a model-based optimal control algorithm is derived to solve the Hamilton–Jacobi–Bellman equation (HJBE). Next, data-driven reinforcement learning (RL) is employed to approximate the solution of the HJBE iteratively to obviate requiring an SMA manipulator model. Moreover, critic neural networks (NNs) and actor NNs are introduced to the RL-based optimal controller, and the least-squares method is used to find the parameters for the NNs-based RL algorithm. Rigorous theoretical analyses demonstrate that the proposed controller can stable the SMA manipulator system, and the tracking errors are always constrained within the prescribed region. Finally, experiments are conducted on the established SMA actuated manipulator platform, and the results illustrate that the proposed controller is feasible and effective.

HNO: High-Order Numerical Architecture for ODE-Inspired Deep Unfolding Networks

Recently, deep unfolding networks (DUNs) based on optimization algorithms have received increasing attention, and their high efficiency has been confirmed by many experimental and theoretical results. Since this type of networks combines model-based traditional optimization algorithms, they have high interpretability. In addition, ordinary differential equations (ODEs) are often used to explain deep neural networks, and provide some inspiration for designing innovative network models. In this paper, we transform DUNs into first-order ODE forms, and propose a high-order numerical architecture for ODE-inspired deep unfolding networks. To the best of our knowledge, this is the first work to establish the relationship between DUNs and ODEs. Moreover, we take two representative DUNs as examples, apply our architecture to them and design novel DUNs. In theory, we prove the existence, uniqueness of the solution and convergence of the proposed network, and also prove that our network obtains a fast linear convergence rate. Extensive experiments verify the effectiveness and advantages of our architecture.

Optimisation of Trailing Edge Flaps on the Base Cavity of a Vehicle for Improved Performance at Yaw

Regulations to reduce greenhouse gas emissions of passenger vehicles are becoming increasingly stringent. The aerodynamic drag is a major contributor to the vehicle’s total energy consumption where a large portion is attributed to the base wake. This paper optimises the angles of small trailing edge flaps on a base cavity of a full-scale sports utility vehicle placed in a wind tunnel. The trailing edge flaps are controlled using servos mounted inside the cavity. The flap angles are optimised using a surrogate model based optimisation algorithm with the objective of reducing the aerodynamic drag at different yaw angles and to create a yaw-insensitive geometry by considering several weighted yaw angles to form the driving cycle averaged drag. Low drag designs are further investigated using base pressures and wake measurements. The results show that the base pressures are symmetrised by reducing the crossflow in the wake. As the model is yawed the wake becomes increasingly downwash dominated by a large rotating windward structure which is reduced by the optimised flaps. The cycle averaged drag optimised design has a smaller increase in drag when yawed compared to a design optimised without considering yaw.

A Novel Resolution Scheme of Time-Energy Optimal Trajectory for Precise Acceleration Controlled Industrial Robot Using Neural Networks

The surging popularity of adopting industrial robots in smart manufacturing has led to an increasing trend in the simultaneous improvement of the energy costs and operational efficiency of motion trajectory. Motivated by this, multi-objective trajectory planning subject to kinematic and dynamic constraints at multiple levels has been considered as a promising paradigm to achieve the improvement. However, most existing model-based multi-objective optimization algorithms tend to come out with infeasible solutions, which results in non-zero initial and final acceleration. Popular commercial software toolkits applied to solve multi-objective optimization problems in actual situations are mostly based on the fussy conversion of the original objective and constraints into strict convex functions or linear functions, which could induce a failure of duality and obtain results exceeding limits. To address the problem, this paper proposes a time-energy optimization model in a phase plane based on the Riemann approximation method and a solution scheme using an iterative learning algorithm with neural networks. We present forward-substitution interpolation functions as basic functions to calculate indirect kinematic and dynamic expressions introduced in a discrete optimization model with coupled constraints. Moreover, we develop a solution scheme of the complex trajectory optimization problem based on artificial neural networks to generate candidate solutions for each iteration without any conversion into a strict convex function, until minimum optimization objectives are achieved. Experiments were carried out to verify the effectiveness of the proposed optimization solution scheme by comparing it with state-of-the-art trajectory optimization methods using Yalmip software. The proposed method was observed to improve the acceleration control performance of the solved robot trajectory by reducing accelerations exceeding values of joints 2, 3 and 5 by 3.277 rad/s2, 26.674 rad/s2, and 7.620 rad/s2, respectively.

Multi-agent DRL-based data-driven approach for PEVs charging/discharging scheduling in smart grid

This paper studies the charging/discharging scheduling problem of plug-in electric vehicles (PEVs) in smart grid, considering the users’ satisfaction with state of charge (SoC) and the degradation cost of batteries. The objective is to collectively determine the energy usage patterns of all participating PEVs so as to minimize the energy cost of all PEVs while ensuring the charging needs of PEV owners. The challenges herein are mainly in three folds: 1) the randomness of electricity price and PEVs’ commuting behavior; 2) the unknown dynamics model of SoC; and 3) a large solution space, which make it challenging to directly develop a model-based optimization algorithm. To this end, we first reformulate the above energy cost minimization problem as a Markov game with unknown transition probabilities. Then a multi-agent deep reinforcement learning (DRL)-based data-driven approach is developed to solve the Markov game. Specifically, the proposed approach consists of two networks: an extreme learning machine (ELM)-based feedforward neural network (NN) for uncertainty prediction of electricity price and PEVs’ commuting behavior and a Q network for optimal action-value function approximation. Finally, the comparison results with three benchmark solutions show that our proposed algorithm can not only adaptively decide the optimal charging/discharging policy by on-line learning process, but also yield a lower energy cost within an unknown market environment.

Bayesian optimization based dynamic ensemble for time series forecasting

Among various time series (TS) forecasting methods, ensemble forecast is extensively acknowledged as a promising ensemble approach achieving great success in research and industry. Due to the high diversification of individual model assumptions, heterogeneous information fusion contributes to generating effective and robust forecasts for Economics, Meteorology, and Transportation. This paper proposes a Bayesian optimization-based dynamic ensemble (BODE) that overcomes the single model-based methods limitation and provides a dynamic ensemble forecast combination for TS with time-varying underlying patterns. The proposed BODE method combines ten disparate model candidates, including statistical methods, machine learning (ML)-based models, and the latest deep neural networks (DNN). We take into consideration their prediction performance for the recent past to adjust their weights for combination and apply the model-based Bayesian optimization algorithm (BOA) for the combination hyperparameter (HP) tuning to endow our method with higher adaptability and better generalization performance. Besides, the frequency impact of TS data on the ensemble forecast methods is under-researched in the current literature. Therefore, four groups of distinct seasonal TS datasets are investigated in this paper. The empirical result demonstrates that our method performs robustly better performance with the main reasons analyzed in a detailed ablation study.