Online Learning Ability Research Articles

Modeling optical filters based on serially coupled microring resonators using radial basis function neural network

In this research for the first time by using radial basis function neural network (RBFNN), filters based on serially coupled microring resonators have been modeled. Also, signal flow graph (SFG) method based on Mason’s rule has been used to simulate filters. It has been represented when RBFNN has been learned, model can extract the outputs same as what was simulated by SFG method. It has been proved that RBFNN model can properly obtain results in several cases in which some parameters of filter like the order of filter; MRRs radius; coupling coefficients; and propagation loss have been changed. In these cases to design filter by an analytical method like the SFG, we need to obtain new transfer function. Obtaining novel transfer function would make filter designating complicated in terms of calculation and simulation time while the RBFNN can match with any change as fast as possible. The RBFNN has advantages of optimization ability, straightforward topological architecture, stable generalization ability, appropriate tolerance against input noise, online learning ability, accuracy in dynamically nonlinear approximation, predictability and fast and easy learning algorithms. These properties of RBFNN make it suitable to model pliable optical systems.

Incremental model based online heuristic dynamic programming for nonlinear adaptive tracking control with partial observability

Heuristic dynamic programming is a class of reinforcement learning, which has been introduced to aerospace engineering to solve nonlinear, optimal adaptive control problems. However, it requires an off-line learning stage to train a global system model to represent the system dynamics. This paper uses an incremental model in heuristic dynamic programming to improve the online learning ability, which is incremental model based heuristic dynamic programming. The trait of the online identification of the incremental model makes this method an option for fault-tolerant control and partially observable control problems. This study, therefore, also extends this method to deal with partial observability. The presented method has been validated on two different online tracking problems: missile fault-tolerant control with full-state measurements and also spacecraft attitude control disturbed with liquid sloshing under partially observable conditions. The results reveal that the proposed method outperforms the conventional heuristic dynamic programming method in fault-tolerant control tasks, deals with partial observability, and is robust to internal uncertainties and external disturbances.

Neural Networks Based Learning Control for a Piezoelectric Nanopositioning System

In this article, approximation model-based control and neural networks-based adaptive control are investigated for obtaining the solution to the motion tracking of a piezoelectric nanopositioning system, respectively. In order to reduce the effect of unknown hysteresis nonlinearity, a disturbance observer is introduced to estimate it. By considering nominal parts of an unknown piezoelectric nanopositioning system, approximation model-based control is obtained. The unknown parts corresponding to nominal parts are dealt with by the online learning ability of neural networks, and an adaptive neural network control is proposed to improve control accuracy. Compared with existing works, a great benefit of the proposed control method is that the neural networks-based learning algorithm is developed to deal with uncertainty of a piezoelectric nanopositioning system in an online way such that the closed-loop system can be governed automatically, obtaining satisfactory motion tracking. With Lyapunov stability theory, it is proved that all error signals are semiglobally uniformly ultimately bounded. Experiment is carried out to verify the effectiveness of the proposed control.

A convolutional fuzzy min-max neural network

Data updation is a natural process in real life machine leaning problems. Online machine learning is a possible solution to tackle such new data and facilitate continuous updation of knowledge without hampering the already acquired one. This work proposes a convolutional fuzzy min-max neural network (CFMNN) for image classification with online learning ability. Image classification using convolutional neural network (CNN) is a well-known and established technique. The CNN based systems are trained in offline mode. To recognize new classes in addition to the learned ones, CNN requires a complete re-training based on earlier and new data samples. This training process of CNN is time consuming. The proposed CFMNN is designed to address this problem without the need of re-training. The CFMNN can learn new data classes in an online mode. In this work, the concept of hyperbox fuzzy set is introduced for CNN to add the online learning proficiency. Four different CNN architectures are considered to verify the robustness of CFMNN performance. The proposed method is evaluated on standard datasets MNIST, Caltech-101, Plant seedlings and CIFAR-100. Accuracy and training time are used as performance metrics. The experimental results show that CFMNN with online learning capability has tremendously reduced training time and it gives compatible or better accuracy than the existing methods.

Online Intrusion Scenario Discovery and Prediction Based on Hierarchical Temporal Memory (HTM)

With the development of intrusion detection, a number of the intelligence algorithms (e.g., artificial neural networks) are introduced to enhance the performance of the intrusion detection systems. However, many intelligence algorithms should be trained before being used, and retrained regularly, which is not applicable for continuous online learning and analyzing. In this paper, a new online intrusion scenario discovery framework is proposed and the intelligence algorithm HTM (Hierarchical Temporal Memory) is employed to improve the performance of the online learning ability of the system. The proposed framework can discover and model intrusion scenarios, and the constructed model keeps evolving with the variance of the data. Additionally, a series of data preprocessing methods are introduced to enhance its adaptability to the noisy and twisted data. The experimental results show that the framework is effective in intrusion scenario discovery, and the discovered scenario is more concise and accurate than our previous work.

Exploratory research of nursing distance education model under the epidemic situation of COVID-19

Objective To explore the nursing distance education model under the epidemic situation of corona virus disease 2019 (COVID-19), and challenges of teachers and students so as to bring forth a proposal for future nursing education model reform. Methods The mixed research was used with methods of questionnaire survey and interview. In February 2020, we investigated the distance learning conditions and needs among 922 nursing undergraduates in School of Nursing, Beijing University of Chinese Medicine. Qualitative research of phenomenology was used to carry out semi-structured and in-depth interview among 14 nursing teachers. Data analysis was implemented with the method of Colaizzi phenomenological analysis. Results Among 922 nursing undergraduates, 95.99% of them could learn on line normally on the public platform, but there were still some students in mountain area limited by network conditions; 97.18% of them accepted the nursing distance education model proposed by our school. Qualitative interview of teachers extracted two themes, the key element to implement nursing distance education and challenges, inspiration of epidemic situation to nursing education reform. Conclusions In distance education model, teachers face challenges of idea, technology and resource; students are influenced by study habits, online learning ability and objective environment. In future, we can make a reform in the aspects of teachers, educational content and form, educational resources and platforms, education management. Key words: Students, nursing; Corona virus disease 2019; Epidemic situation; Education, distance; Nursing; Mixed research

Research on parallel nonlinear control system of PD and RBF neural network based on U model

ABSTRACTThe modelling problem of nonlinear control system is studied, and a higher generality nonlinear U model is established. Based on the nonlinear U model, RBF neural network and PD parallel control algorithm are proposed. The difference between the control input value and the output value of the neural network is taken as the learning target by using the online learning ability of the neural network. The gradient descent method is used to adjust the PD output value, and ultimately track the ideal output. The Newton iterative algorithm is used to complete the transformation of the nonlinear model, and the nonlinear characteristic of the plant is reduced without loss of modelling precision, consequently, the control performance of the system is improved. The simulation results show that RBF neural network and PD parallel control system can control the nonlinear system. Moreover, the control system with Newton iteration can improve the control effect and anti-interference performance of the system.

MPC of Sewage Treatment Process

Sewage treatment is one of the main methods to promote the recycling of water resources. The control goal of sewage treatment process is to reduce energy consumption under the premise that the effluent quality reaches the standard. In recent years, model predictive control (MPC) has attracted some attention in sewage treatment. Neural network is widely used in control field because of its strong online learning ability. BP neural network is selected as the prediction layer and control layer of MPC and applied to sewage treatment plant to realize on-line control of dissolved oxygen and nitrate. The training index of traditional neural network usually only selects the error between measured value and set value as the variable, and now the change of control quantity is also taken as the training index variable of control layer to adjust the weight relation between them to get the best control effect. Considering that different weather conditions will have a greater impact on the water inflow, different coefficients of the two can be selected to achieve better results in different weather.

Multi-Kernel Online Reinforcement Learning for Path Tracking Control of Intelligent Vehicles

Path tracking control of intelligent vehicles has to deal with the difficulties of model uncertainties and nonlinearities. As a class of adaptive optimal control methods, reinforcement learning (RL) has received increasing attention in solving difficult control problems. However, feature representation and online learning ability are two major problems to be solved for learning control of uncertain dynamic systems. In this article, we propose a multi-kernel online RL approach for path tracking control of intelligent vehicles. In the proposed approach, a multiple kernel feature learning framework is designed for online learning control based on dual heuristic programming (DHP) and the new online learning control algorithm is called multi-kernel DHP (MKDHP). In MKDHP, instead of the expert knowledge for selecting and fine-tuning of a suitable kernel function, only a set of basic kernel functions is required to be predefined and the multi-kernel features can be learned for value function approximation in the critic. The simulation studies on path tracking control for intelligent vehicles have been conducted under <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$S$ </tex-math></inline-formula> -curve and urban road conditions. The results demonstrated that compared with other typical path tracking controllers for intelligent vehicles, such as the linear quadratic regulator (LQR), the pure pursuit controller and the ribbon-based controller, the proposed multi-kernel learning controller can achieve better performance in terms of tracking precision and smoothness.

Training high-performance and large-scale deep neural networks with full 8-bit integers

Deep neural network (DNN) quantization converting floating-point (FP) data in the network to integers (INT) is an effective way to shrink the model size for memory saving and simplify the operations for compute acceleration. Recently, researches on DNN quantization develop from inference to training, laying a foundation for the online training on accelerators. However, existing schemes leaving batch normalization (BN) untouched during training are mostly incomplete quantization that still adopts high precision FP in some parts of the data paths. Currently, there is no solution that can use only low bit-width INT data during the whole training process of large-scale DNNs with acceptable accuracy. In this work, through decomposing all the computation steps in DNNs and fusing three special quantization functions to satisfy the different precision requirements, we propose a unified complete quantization framework termed as “WAGEUBN” to quantize DNNs involving all data paths including W (Weights), A (Activation), G (Gradient), E (Error), U (Update), and BN. Moreover, the Momentum optimizer is also quantized to realize a completely quantized framework. Experiments on ResNet18/34/50 models demonstrate that WAGEUBN can achieve competitive accuracy on the ImageNet dataset. For the first time, the study of quantization in large-scale DNNs is advanced to the full 8-bit INT level. In this way, all the operations in the training and inference can be bit-wise operations, pushing towards faster processing speed, decreased memory cost, and higher energy efficiency. Our throughout quantization framework has great potential for future efficient portable devices with online learning ability.

Optimal Decision-Making Approach for Cyber Security Defense Using Game Theory and Intelligent Learning

Existing approaches of cyber attack-defense analysis based on stochastic game adopts the assumption of complete rationality, but in the actual cyber attack-defense, it is difficult for both sides of attacker and defender to meet the high requirement of complete rationality. For this aim, the influence of bounded rationality on attack-defense stochastic game is analyzed. We construct a stochastic game model. Aiming at the problem of state explosion when the number of network nodes increases, we design the attack-defense graph to compress the state space and extract network states and defense strategies. On this basis, the intelligent learning algorithm WoLF-PHC is introduced to carry out strategy learning and improvement. Then, the defense decision-making algorithm with online learning ability is designed, which helps to select the optimal defense strategy with the maximum payoff from the candidate strategy set. The obtained strategy is superior to previous evolutionary equilibrium strategy because it does not rely on prior data. By introducing eligibility trace to improve WoLF-PHC, the learning speed is further improved and the defense timeliness is significantly promoted.

Online detection of anomaly behaviors based on multidimensional trajectories

In the surveillance domain, timely detection of anomaly behaviors is very important and is a great challenge to human operators due to information overload, fatigue and inattention. Many anomaly detection algorithms based on trajectories have been proposed for this problem. However, these algorithms generally have problems such as complex parameter setting, unfaithful statistical model, not well-calibrated false alarm rate, poor ability of online learning and sequential anomaly detection, etc. The theory of conformal prediction was introduced to solve these problems by constructing the sequential Hausdorff nearest neighbor conformal anomaly detector. Yet, it only considers position information of the targets and is not sensitive to velocity and course anomaly behaviors. And the run times are increasing as the increase of the data size, which is not appropriate for early warning surveillance application. In order to solve these problems, sequential multi-factor Hausdorff nearest neighbor conformal anomaly detector (SMFHNNCAD) and sequential multi-factor Hausdorff nearest neighbor inductive conformal anomaly detector (SMFHNNICAD) based on multidimensional trajectories are proposed in this paper. Experiments in both simulated military scenario and realistic civilian scenario show the presented algorithm has a good performance to online detect anomaly behaviors and would have a wide prospect in early warning surveillance systems.

Cable joint fault detection for the ring main unit based on an adaptive TNPE algorithm

AbstractThe ring main unit (RMU) is the main equipment of the electrical power distribution network. The deterioration process of the RMU cable joint is multivariable and nonstationary over time. Existing fault detection models for the RMU cable joint are single‐variable and static over time. A novel algorithm of adaptive time neighborhood preserving embedding (ATNPE) is proposed in this paper, a multivariable time series statistical algorithm with online learning ability to improve the robustness and generalization ability of cable joint fault detection. This method combines an approximate linear dependence (ALD) condition with the time neighborhood preserving embedding (TNPE). New independent samples were determined by calculating the ALD value between new samples and modeling samples. In addition, the new independent samples were used to update the TNPE model. The offline training module was used to update RMU fault detection based on the TNPE model by training new samples with historical offline data. The online updating module trained the historical model to ensure the correct rate of online detection. The experimental results indicated that the method has the ability to adapt to new samples and the squared prediction error statistic limit responded to changes in statistical characteristics of monitoring data. The method can effectively reduce the false alarm rate and has the same fault detection rate compared to the existing model.This article is categorized under: Technologies &gt; Computational Intelligence Application Areas &gt; Business and Industry Technologies &gt; Machine Learning

Pedagogical learning supports based on human–systems inclusion applied to rail flow control

The paper presents the new concept on human–systems inclusion for designing, analyzing or evaluating human–machine systems and applied it to two case studies on rail flow control. Human–systems inclusion is related to the development of online learning abilities using pedagogical learning supports and aims at applying both automation-supported human and human-supported automation processes in which a human is supported by automation or automation is supported by a human, respectively. Different learning supports are proposed: accident reports, course supports, internet accesses, simulators, technical systems, etc. People are free to use or not to use them to learn alone, from the other or with the others. The first case study concerns automation-supported driving activities and focuses on the analysis of possible dissonances of use of two automated tools. The second case study consists of designing and validating a human-supported eco-driving assistance system.

Real-time stochastic optimal scheduling of large-scale electric vehicles: A multidimensional approximate dynamic programming approach

This paper studies the real-time charging optimization (RTCO) of large-scale electric vehicles (EVs), which is a multistage and multidimensional stochastic resource allocation problem. In order to handle the complex RTCO of large-scale EVs, a multidimensional approximate dynamic programming (ADP-RTCO) is designed for the sequent optimal decision makings. The hierarchy of ADP-RTCO contains two layers. In the upper layer, RTCO of large-scale EVs is formulated as a multidimensional energy storage problem by classifying EVs into several virtual EV clusters (EVCs). Then, temporal difference learning based policy iteration method is used to obtain the value function approximation for each EVC. In the lower layer, priority based reallocation algorithm is employed to obtain the detailed charging power for connected EVs. ADP-RTCO is designed with on-line learning ability to enhance its adaptability and robustness to ambiguous environment. Comprehensive simulation results demonstrate the optimality and robustness of ADP-RTCO in both cost-saving and load-flattening problems. Furthermore, ADP-RTCO can obtain higher-quality solution compared with other algorithms and is applicable for RTCO of large-scale EVs under uncertainties due to its high computation efficiency.

Multivariable System Identification Method Based on Continuous Action Reinforcement Learning Automata

In this work, a closed-loop identification method based on a reinforcement learning algorithm is proposed for multiple-input multiple-output (MIMO) systems. This method could be an attractive alternative solution to the problem that the current frequency-domain identification algorithms are usually dependent on the attenuation factor. With this method, after continuously interacting with the environment, the optimal attenuation factor can be identified by the continuous action reinforcement learning automata (CARLA), and then the corresponding parameters could be estimated in the end. Moreover, the proposed method could be applied to time-varying systems online due to its online learning ability. The simulation results suggest that the presented approach can meet the requirement of identification accuracy in both square and non-square systems.

DeepCascade-WR: a cascading deep architecture based on weak results for time series prediction

Noisy and nonstationary real-world time series predictions (TSPs) are challenging tasks. Confronted with these challenging tasks, the predictive power of traditional shallow models is commonly not satisfactory enough. While the research on deep learning (DL) has made milestone breakthrough in recent years, and DL paradigm has gradually become indispensable for accomplishing these complex tasks. In this work, a cascading deep architecture based on weak results (DeepCascade-WR) is established, which possesses deep models’ marked capability of feature representation learning based on complex data. In DeepCascade-WR, weak prediction results are defined, innovating the forecasting mode of traditional TSP. The original data will be properly reconstituted with prior knowledge, generating attribute vectors with valid predictive information. DeepCascade-WR possesses online learning ability and effectively avoids the retraining problem, owing to the property of OS-ELM, one base model of DeepCascade-WR. Besides, ELM is exploited as another base model of DeepCascade-WR, therefore, DeepCascade-WR naturally inherits some valuable virtues from ELM, including faster training speed, better generalization ability and the avoidance of being fallen into local optima. Ultimately, in the empirical results, DeepCascade-WR demonstrates its superior predictive performance on five benchmark financial datasets, i.e., ^DJI, ^GSK, ^HSI, JOUT, and S&P 500 Index, compared with its base learners and other state-of-the-art algorithms.

Autonomous Path Planning of AUV in Large-Scale Complex Marine Environment Based on Swarm Hyper-Heuristic Algorithm

Autonomous underwater vehicles (AUVs) as an efficient underwater exploration means have been used to perform various marine missions. However, limited by the technologies of underwater acoustic communications and intelligent autonomy, the most current and advanced AUVs only perform a limited number of tasks in the small-scale area and the known underwater environment. Therefore, in this paper, a one path planning model was proposed combining the global path planning and the local path planning for the large-scale complex marine environment. More specifically, the B-spline curve was used to represent the smooth path for the requirement of kinematic constraints of AUVs. After considering the various constraints, such as the energy/time consumption, the turning radius limitation, the marine environment, and the ocean current, the path planning was abstractly modeled as a multi-objective optimization model with the time cost, the curvature cost, the map cost, and the ocean current cost. The swarm hyper-heuristic algorithm (SHH) with the online learning ability was proposed to solve this model with real-time performance and stability. The results showed that the proposed online learning SHH algorithm had obvious advantages in terms of time efficiency, stability, and optimal performance compared with the results of two traditional heuristic algorithms, both particle swarm optimization (PSO) and firefly algorithm (FFA). The time efficiency of the online learning SHH algorithm improved at least 20% compared with PSO and FFA.

AA-HMM: An Anti-Adversarial Hidden Markov Model for Network-Based Intrusion Detection

In the field of network intrusion, malware usually evades anomaly detection by disguising malicious behavior as legitimate access. Therefore, detecting these attacks from network traffic has become a challenge in this an adversarial setting. In this paper, an enhanced Hidden Markov Model, called the Anti-Adversarial Hidden Markov Model (AA-HMM), is proposed to effectively detect evasion pattern, using the Dynamic Window and Threshold techniques to achieve adaptive, anti-adversarial, and online-learning abilities. In addition, a concept called Pattern Entropy is defined and acts as the foundation of AA-HMM. We evaluate the effectiveness of our approach employing two well-known benchmark data sets, NSL-KDD and CTU-13, in terms of the common performance metrics and the algorithm’s adaptation and anti-adversary abilities.

Multi-Step Model Predictive Control Based on Online Support Vector Regression Optimized by Multi-Agent Particle Swarm Optimization Algorithm

As optimization of parameters affects prediction accuracy and generalization ability of support vector regression (SVR) greatly and the predictive model often mismatches nonlinear system model predictive control, a multi-step model predictive control based on online SVR (OSVR) optimized by multi-agent particle swarm optimization algorithm (MAPSO) is put forward. By integrating the online learning ability of OSVR, the predictive model can self-correct and adapt to the dynamic changes in nonlinear process well.