Integrated Learning Algorithm Research Articles

An online model-free adaptive learning control solution for robotic arms

This paper focuses on the online control of a class of nonlinear dynamical systems, specifically robotic manipulators. Solutions utilizing Proportional–Integral–Derivative (PID) control schemes are employed to control the joints of robotic manipulators. However, the existing control strategies utilize fixed gains, which do not fully account for the inherent nonlinearity of the dynamical structure or the dynamics of reference-tracking error. Additionally, the individual joint’s dynamic performance is optimized independently from the performance of other joints. This work introduces an adaptive integral Reinforcement Learning algorithm to control a four-DoF robotic arm in real time. This is done using a model-free Value Iteration process implemented in a continuous-time mode. The solution does not assume any knowledge of the dynamics of the robot arm and does not require any initial admissible control strategy to proceed with the adaptive learning solution. The self-learning algorithm provides adaptable strategies to control the turntable, forearm, bicep, and wrist joints of the robotic arm. The performance of the adaptive learning solution is compared with those of Proportional–Integral–Derivative and high-order model-free adaptive control schemes to highlight its effectiveness.

Integral reinforcement learning‐based optimal tracking control for uncertain nonlinear systems under input constraint and specified performance constraints

AbstractThis article addresses the optimal tracking control problem with prescribed performance for uncertain nonlinear systems subject to input constraint and unknown disturbances. First, a fixed‐time monotonic convergence function is introduced to restrain tracking error, and a nonlinear mapping technique is employed to transform the constrained error into an unconstrained variable, then the fixed‐time output tracking issue is boiled down to the boundedness problem of the transformed variable. With the aid of a nonquadratic cost function, the input constraint is encoded into the optimization problem. To solve the unknown disturbances, an auxiliary system and an auxiliary disturbance policy are constructed, and the optimal control problem is formulated as a two‐player zero‐sum game. Moreover, a Hamilton–Jacobi–Isaacs (HJI) equation associated with this nonquadratic zero‐sum game is established to give the optimal control and the worst‐case disturbance policy solution. Subsequently, to avoid using knowledge of the system dynamics, three neural network approximators, namely, actor, critic, and disturbance, which are tuned online and simultaneously for approximating the solution of HJI, are constructed based on the integral reinforcement learning algorithm. Theoretical analysis shows that the reconstructed error system states and the weight estimation errors are semi‐globally uniformly ultimately bounded. Finally, the simulation study further tests the availability of the proposed control strategy.

Adaptive Critic Learning-Based Optimal Bipartite Consensus for Multiagent Systems With Prescribed Performance.

Developing a distributed bipartite optimal consensus scheme while ensuring user-predefined performance is essential in practical applications. Existing approaches to this problem typically require a complex controller structure due to adopting an identifier-actor-critic framework and prescribed performance cannot be guaranteed. In this work, an adaptive critic learning (ACL)-based optimal bipartite consensus scheme is developed to bridge the gap. A newly designed error scaling function, which defines the user-predefined settling time and steady accuracy without relying on the initial conditions, is then integrated into a cost function. The backstepping framework combines the ACL and integral reinforcement learning (IRL) algorithm to develop the adaptive optimal bipartite consensus scheme, which contributes a critic-only controller structure by removing the identifier and actor networks in the existing methods. The adaptive law of the critic network is derived by the gradient descent algorithm and experience replay to minimize the IRL-based residual error. It is shown that a compute-saving learning mechanism can achieve the optimal consensus, and the error variables of the closed-loop system are uniformly ultimately bounded (UUB). Besides, in any bounded initial condition, the evolution of bipartite consensus is limited to a user-prescribed boundary under bounded initial conditions. The illustrative simulation results validate the efficacy of the approach.

Inversion of Soil Moisture Content in Cotton Fields Using GBR-RF Algorithm Combined with Sentinel-2 Satellite Spectral Data

Soil moisture content plays a vital role in agricultural production, significantly influencing crop growth, development, and yield. Thoroughly understanding the specific soil moisture content in cotton fields is crucial for enhancing agricultural efficiency and driving sustainable agricultural development. This study utilized the gradient-boosting regression–random forest (GBR-RF) algorithm and the GBR and RF algorithms separately, in conjunction with Sentinel-2 satellite images, to estimate cotton soil moisture content, focusing on the B1–B8 bands and in particular the sensitive B6, B7, and B8 bands. The soil data in the jujube orchard of the study area were collected using soil augers at a depth of 30 cm, with soil data collected from a depth of 20 to 30 cm. The findings revealed that the integrated learning algorithm GBR-RF demonstrated high accuracy, with R2, MAE, and MSE results of 0.8838, 1.0121, and 1.6168, respectively. In comparison, the results using just the GBR algorithm yielded R2, MAE, and MSE values of 0.8158, 1.1327, and 1.9645, respectively, while those obtained from the RF algorithm were 0.8415, 1.0680, and 1.8331, respectively. These results indicate that the algorithms exhibited strong generalization, robustness, and accuracy, with GBR-RF outperforming GBR and RF by 8.34% and 5.03%, respectively, in combination with using the B1–B8 bands for inversion. Furthermore, utilizing the full-band data resulted in R2 values that were up to 24.27% higher than those of the individual bands, affirming the efficacy of band combinations for improved accuracy. This study’s demonstration of the positive impact of integrated learning algorithms on estimating cotton soil moisture content underscores the advantages of multi-band data combinations over single-band data, highlighting their ability to enhance accuracy without significantly impacting errors. Importantly, this study’s findings, while not limited to a single experimental field, have broad applicability in cotton precision agriculture, offering valuable insights for research on yield enhancement and agricultural efficiency.

Modeling and prediction of tribological properties of copper/aluminum-graphite self-lubricating composites using machine learning algorithms

The tribological properties of self-lubricating composites are influenced by many variables and complex mechanisms. Data-driven methods, including machine learning (ML) algorithms, can yield a better comprehensive understanding of complex problems under the influence of multiple parameters, typically for how tribological performances and material properties correlate. Correlation of friction coefficients and wear rates of copper/aluminum-graphite (Cu/Al-graphite) self-lubricating composites with their inherent material properties (composition, lubricant content, particle size, processing process, and interfacial bonding strength) and the variables related to the testing method (normal load, sliding speed, and sliding distance) were analyzed using traditional approaches, followed by modeling and prediction of tribological properties through five different ML algorithms, namely support vector machine (SVM), K-Nearest neighbor (KNN), random forest (RF), eXtreme gradient boosting (XGBoost), and least-squares boosting (LSBoost), based on the tribology experimental data. Results demonstrated that ML models could satisfactorily predict friction coefficient and wear rate from the material properties and testing method variables data. Herein, the LSBoost model based on the integrated learning algorithm presented the best prediction performance for friction coefficients and wear rates, with R2 of 0.9219 and 0.9243, respectively. Feature importance analysis also revealed that the content of graphite and the hardness of the matrix have the greatest influence on the friction coefficients, and the normal load, the content of graphite, and the hardness of the matrix influence the wear rates the most.

Reinforcement learning based sliding mode control for two-time-scale multi-agent systems with malicious attacks

The optimal consensus problem for linear two-time-scale multi-agent systems under malicious attacks is studied in this paper. Firstly, an integral sliding mode function is devised to guide the system trajectory towards the sliding mode surface and the impact of attacks can be eliminated. Then, the optional consensus problem is reformulated as a zero-sum game problem between each agent and its neighbouring agents. Thus, the game algebraic Riccati equation with singularly perturbed parameter is formulated. Furthermore, to avoid the requirement of the system dynamics information, an integral reinforcement learning algorithm is presented to obtain the optimal control policy for multi-agent systems. Compared with existing learning methods, the obtained reinforcement learning algorithm is devoid of potential calculation error issues from singularly perturbed parameter. Meanwhile, the convergence of the proposed algorithm is verified. Finally, a simulation example is provided to demonstrate the efficacy of the proposed control method.

MINDG: a drug-target interaction prediction method based on an integrated learning algorithm.

Drug-target interaction (DTI) prediction refers to the prediction of whether a given drug molecule will bind to a specific target and thus exert a targeted therapeutic effect. Although intelligent computational approaches for drug target prediction have received much attention and made many advances, they are still a challenging task that requires further research. The main challenges are manifested as follows: (i) most graph neural network-based methods only consider the information of the first-order neighboring nodes (drug and target) in the graph, without learning deeper and richer structural features from the higher-order neighboring nodes. (ii) Existing methods do not consider both the sequence and structural features of drugs and targets, and each method is independent of each other, and cannot combine the advantages of sequence and structural features to improve the interactive learning effect. To address the above challenges, a Multi-view Integrated learning Network that integrates Deep learning and Graph Learning (MINDG) is proposed in this study, which consists of the following parts: (i) a mixed deep network is used to extract sequence features of drugs and targets, (ii) a higher-order graph attention convolutional network is proposed to better extract and capture structural features, and (iii) a multi-view adaptive integrated decision module is used to improve and complement the initial prediction results of the above two networks to enhance the prediction performance. We evaluate MINDG on two dataset and show it improved DTI prediction performance compared to state-of-the-art baselines. https://github.com/jnuaipr/MINDG.

An enhanced perception health state assessment method for subway sliding plug door transmission system

The monitoring data (current and rotational speed) of the subway sliding plug door transmission system changed positively and negatively with the acceleration and weight of the door. How to perceive the changes is a challenging problem in the health state assessment of transmission system. To address this problem, an enhanced perception health state assessment method was proposed for the transmission systems. In the method, firstly, the equivalent resistance force is calculated by monitoring the current and rotational speed data according to mechanical dynamic knowledge. Secondly, the sensitive features of normal and abnormal states are screened out from the enhanced dataset constructed by current, rotational speed data, and equivalent resistance force data. Finally, the health state of the transmission system is assessed using an integrated learning algorithm. The effectiveness of the method is verified by benchmark experimental data, and the results indicate that the method has a higher accuracy with four classifiers and a broader suitable range with varying door acceleration and weight.

Prediction of Biochar Yield and Specific Surface Area Based on Integrated Learning Algorithm

Biochar is a biomaterial obtained by pyrolysis with high porosity and high specific surface area (SSA), which is widely used in several fields. The yield of biochar has an important effect on production cost and utilization efficiency, while SSA plays a key role in adsorption, catalysis, and pollutant removal. The preparation of biochar materials with better SSA is currently one of the frontiers in this research field. However, traditional methods are time consuming and laborious, so this paper developed a machine learning model to predict and study the properties of biochar efficiently for engineering through cross-validation and hyper parameter tuning. This paper used 622 data samples to predict the yield and SSA of biochar and selected eXtreme Gradient Boosting (XGBoost) as the model due to its excellent performance in terms of performance (yield correlation coefficient R2 = 0.79 and SSA correlation coefficient R2 = 0.92) and analyzed it using Shapley Additive Explanation. Using the Pearson correlation coefficient matrix revealed the correlations between the input parameters and the biochar yield and SSA. Results showed the important features affecting biochar yield were temperature and biomass feedstock, while the important features affecting SSA were ash and retention time. The XGBoost model developed provides new application scenarios and ideas for predicting biochar yield and SSA in response to the characteristic input parameters of biochar.

Data-Based Optimal Synchronization of Heterogeneous Multiagent Systems in Graphical Games via Reinforcement Learning.

This article studies the optimal synchronization of linear heterogeneous multiagent systems (MASs) with partial unknown knowledge of the system dynamics. The object is to realize system synchronization as well as minimize the performance index of each agent. A framework of heterogeneous multiagent graphical games is formulated first. In the graphical games, it is proved that the optimal control policy relying on the solution of the Hamilton-Jacobian-Bellmen (HJB) equation is not only in Nash equilibrium, but also the best response to fixed control policies of its neighbors. To solve the optimal control policy and the minimum value of the performance index, a model-based policy iteration (PI) algorithm is proposed. Then, according to the model-based algorithm, a data-based off-policy integral reinforcement learning (IRL) algorithm is put forward to handle the partially unknown system dynamics. Furthermore, a single-critic neural network (NN) structure is used to implement the data-based algorithm. Based on the data collected by the behavior policy of the data-based off-policy algorithm, the gradient descent method is used to train NNs to approach the ideal weights. In addition, it is proved that all the proposed algorithms are convergent, and the weight-tuning law of the single-critic NNs can promote optimal synchronization. Finally, a numerical example is proposed to show the effectiveness of the theoretical analysis.

Model-free optimal robust tracking control for the joint actuated by pneumatic artificial muscles with saturated air pressure input

At present, positioning and tracking the joint angle of the PAM-actuated system is still a challenging issue, due to the inherent hysteresis, high nonlinearities, and time-varying characteristics. Considering the problem of uncertain parameters, external disturbance, and saturated input, a model-free optimal robust tracking control scheme is proposed for the PAMs system in this paper. Also, a nonquadratic cost function is introduced to address the saturation air pressure input. Moreover, an integral reinforcement learning(IRL) algorithm is developed to solve the H∞ tracking Hamilton–Jacobi–Isaacs(HJI) equation. And the IRL algorithm is implemented on a critic–actor-disturbance neural network (NN) structure. The presented control scheme avoids the complete system dynamic and simultaneously compensates for the uncertain parameters, and external disturbance. Then, the effectiveness of the optimal robust tracking controller is verified via computational simulation and keeps the system stable. In addition, the tracking experiment is implemented on a hardware platform, indicating that the presented approach achieves satisfactory tracking performance.

Study on the application of data visualization technology in human resource management and employee turnover prediction

Abstract This paper presents a method to process the dataset of departing employees. The clustering categories of separated employees are accurately determined by fuzzy c-mean clustering and improved clustering FCM algorithm, and new samples of separated employees are generated using SMOTE algorithm to reduce noisy data. The kernel function trick of SVM is used to achieve clustering oversampling, improving classification accuracy. For imbalanced data, this paper uses a new integrated learning algorithm for constructing evaluations, PIBoost, combined with a full-sample cost-aware weight algorithm to improve the generalization ability of the SVM classifier, which has better classification results for various data sets. In the model performance comparison, K-AFCM-SMOTE-SVM has the highest accuracy with a value of 0.89. In the ten-fold cross-validation accuracy comparison, the K-AFCM-SMOTE-SVM model has a better overall performance index than the other two, with an average cross-validation accuracy of 0.932.

Heart failure prediction based on multiple machine learning algorithms

Heart failure is a complex medical condition that arises due to the heart's inability to adequately circulate blood throughout the body, which is challenging to predict. This research aims to investigate three distinct models, namely logistic regression, random forest and decision tree generation algorithms. Logistic regression is essentially a logistic function applied to linear regression, and the loss function associated with linear regression is similar to the mean square error-like loss. In contrast, the loss function for logistic regression follows cross-entropy loss. while cross-entropy loss is often used in practice, it differs from mean square error loss. The derivative of cross-entropy loss is a difference that updates rapidly when the error is significant and slowly when the error is small, which is a desirable trait for the purposes. Decision tree generation algorithms utilize tree structures in which internal nodes represent judgments on attributes, branches represent outputs of judgments, and leaf nodes represent classification results. Random forest is an integrated learning algorithm that employs decision trees as the base learner. In classification models, multiple decision trees are processed for voting, while multiple decision tree results are processed for averaging in regression models. Experimental results indicate that random forest outperforms the other two models, albeit with a marginal difference. Further studies should incorporate additional models to identify a more suitable model for predicting heart failure.

Retrieval of Water Quality Parameters in Dianshan Lake Based on Sentinel-2 MSI Imagery and Machine Learning: Algorithm Evaluation and Spatiotemporal Change Research

According to current research, machine learning algorithms have been proven to be effective in detecting both optical and non-optical parameters of water quality. The use of satellite remote sensing is a valuable method for monitoring long-term changes in the quality of lake water. In this study, Sentinel-2 MSI images and in situ data from the Dianshan Lake area from 2017 to 2023 were used. Four machine learning methods were tested, and optimal detection models were determined for each water quality parameter. It was ultimately determined that these models could be applied to long-term images to analyze the spatiotemporal variations and distribution patterns of water quality in Dianshan Lake. Based on the research findings, integrated learning algorithms, especially CatBoost, have achieved good results in the retrieval of all water quality parameters. Spatiotemporal analysis reveals that the overall distribution of water quality parameters is uneven, with significant spatial variations. Permanganate index (CODMn), Total Nitrogen (TN), and Total Phosphorus (TP) show relatively small interannual differences, generally exhibiting a decreasing trend in concentrations. In contrast, chlorophyll-a (Chl-a), dissolved oxygen (DO), and Secchi Disk Depth (SDD) exhibit significant interannual and inter-year differences. Chl-a reached its peak in 2020, followed by a decrease, while DO and SDD showed the opposite trend. Further analysis indicated that the distribution of water quality parameters is significantly influenced by climatic factors and human activities such as agricultural expansion. Overall, there has been an improvement in the water quality of Dianshan Lake. The study demonstrates the feasibility of accurately monitoring water quality even without measured spectral data, using machine learning methods and satellite reflectance data. The research results presented in this paper can provide new insights into water quality monitoring and water resource management in Dianshan Lake.

A multilayer stacking method base on RFE-SHAP feature selection strategy for recognition of driver’s mental load and emotional state

The driver state monitoring is becoming one of the research hotspots in the field of traffic and vehicle safety, which can ensure driving safety by monitoring the driver’s state. Therefore, this work makes an attempt to recognize driver's mental load and emotional states. However, the reliability and accuracy of driver status detection largely depend on the extracted features and the detection algorithm. The existing methods mainly improve accuracy by increasing the number of features, but for the problem with limited training samples, the increase in features may lead to an increase in model variance, and resulting in overfitting. In addition, single-model based methods may fall into local optimal solutions when performing local search. To alleviate these, we firstly propose an RFE-SHAP algorithm that improves the recursive feature elimination algorithm by using the Shapley Additive exPlanning value, and paired it with the eXtreme Gradient Boosting (XGBoost) algorithm to screen out the subset of features that best represent the driver's mental load and emotional state. Secondly, we proposed a multilayer stacking ensemble learning method based on different optimal feature subsets to further improve the accuracy of the driver’s mental load and emotional state recognition. For each base models used for integration learning, the corresponding optimal feature subsets are selected using the RFE-SHAP algorithm, and then the different optimal feature subsets are combined with multilayer stacking ensemble learning. The validation via experimental data demonstrates that the XGBoost-RFE-SHAP algorithm achieves superior model performance with fewer feature combinations than the classic algorithm. The proposed driver state detection model based on multiple optimum feature subsets and multilayer stacking integration methods would more accurately identify the driver's mental load and emotional state than the single model, commonly used machine learning and integrated learning algorithms.

Composition analysis and identification of ancient glass objects based on AdaBoost and CART classification tree

Ancient glass is highly susceptible to weathering by the burial environment, and a series of chemical reactions will occur in the process, which leads to changes in the chemical composition of glass artifacts. In order to identify and classify the composition types of glass artifacts, this paper uses high potassium glass and lead-barium glass as target training models to derive CART stump (CART tree with only 2 layers) combinations as a way to analyze the classification laws. Then, we analyzed the sub-classification results of weathering, color, and ornamentation, and analyzed the classification rules according to their chemical composition, and came up with the classification method based on CART stumps. In order to identify the type of unknown types of glass, this paper uses an integrated learning algorithm model based on CART classification tree and AdaBoost to train a prediction model using all the samples, with the objective of artifact type, and then performs type prediction on the data. This study is important for the correct classification of glass types.

A novel learning-based data-driven [formula omitted] control strategy for vanadium redox flow battery in DC microgrids

Vanadium redox flow battery (VRB) is one of the most promising batteries at present. In order to enhance the stability and anti-interference ability of VRB in microgrids, a novel learning-based data-driven H∞ control approach is proposed for the VRB, which uses a new integral reinforcement learning algorithm to produce excellent steady-state and dynamic responses only by measurements. Compared to the model-based control methods, it is insensitive to model parameter variations. Furthermore, compared to most of the existing artificial intelligent control approaches that require large amounts of experimental data for offline neural network (NN) training, the proposed control strategy contributes to eliminate the offline training process and therefore, does not need the costly and tedious training data acquisition process. More importantly, the proposed control offers guaranteed closed-loop control stability, which cannot be achieved by nearly all the control methods that purely rely on the offline trained NNs. In this paper, the rigorous proof of stability is given, and the validity of the proposed method is verified by simulation results.

AdaBoost.Rt-LSTM Based Joint SOC and SOH Estimation Method for Retired Batteries

Achieving accurate retired battery state of health (SOH) and state of charge (SOC) estimation is a safe prerequisite for securing the battery secondary utilization and thus effectively improving the energy utilization efficiency. The data-driven approach is efficient and accurate, and does not rely on accurate battery models, which is a hot direction in battery state estimation research. However, the huge number of retired batteries and obvious consistency differences bring bottleneck problems such as long learning time and low model updating efficiency to the traditional data-driven algorithm. In view of this, this paper proposes an integrated learning algorithm based on AdaBoost. Rt-LSTM to realize the joint estimation of SOC and SOH of retired lithium batteries, which relies on the LSTM neural network model and completes the correlation adaption in the spatio-temporal dimension of the whole life cycle sample data. The LSTM model is used as the base learner to construct the AdaBoost. Rt-LSTM strong learning model. The LSTM weak predictor is combined with weights to form a strong predictor, which greatly solves the problem of low accuracy of state estimation due to the large number and variability of retired batteries. Simulation and experimental comparison show that the integrated algorithm proposed in this paper is suitable for improving the SOC and SOH prediction accuracy and the generalization performance of the model.

A learning-based approach to event-triggered guaranteed cost control for completely unknown nonlinear systems

This paper develops a novel guaranteed cost control (GCC) approach under the event-triggered mechanism for completely unknown systems using integral reinforcement learning (IRL) algorithm. First, based on the adaptive dynamic programming (ADP) method, the GCC problem is addressed by transforming it into the optimal control problem. Second, without using the accurate information of system dynamics, a model-free data-based GCC approach is designed via IRL algorithm. Moreover, for the purpose of reducing the waste of communication resources, a GCC algorithm is presented under the event-triggered mechanism for completely unknown system by utilizing the explorized IRL algorithm. The critic–actor–disturbance neural networks (NNs) are applied to approximate near optimal solution. In addition, the weight estimations of NNs are tuned synchronously according to the designed novel triggering condition. Furthermore, the stability analysis of the controlled system is given by utilizing the Lyapunov principle. Finally, the simulation results are presented to verify the feasibility of the designed approach.

Dynamic Event-Triggered-Based Integral Reinforcement Learning Algorithm for Frequency Control of Microgrid With Stochastic Uncertainty

The intermittency of renewable energy and the uncertainty of load put forward higher robustness requirements for the frequency recovery of microgrid (MG). The energy storage equipment provides an idea for frequency control because of its fast response and strong controllability. In this paper, an on-line adaptive frequency control method is proposed to control the governor and energy storage to realize the frequency recovery of MG under stochastic uncertainty. First, the MG system with external disturbances is constructed as a zero-sum differential game model to obtain a robust optimal control scheme. Then, considering the system parameter uncertainty, an improved integral reinforcement learning (IRL) algorithm is designed, in which the reinforcement signal contains a non-quadratic function to solve the energy storage control constraints. Furthermore, a novel dynamic event-triggered control (DETC) is developed to reduce control update times of energy storage. The dynamic variable in DETC coupled with static trigger includes not only the past triggering information but also the disturbances. DETC has a larger trigger threshold than static event-triggered control (SETC). Meanwhile, the proposed algorithm is implemented by an action-critic network structure, in which the action network of energy storage is updated aperiodically. Finally, simulation results show that the proposed control algorithm can realize the frequency recovery of MG, and it has good robustness compared with other algorithms.