Proposed Online Learning Algorithm Research Articles

Online Learning of Bayesian Classifiers with Nonstationary Data Streams

Article Online Learning of Bayesian Classifiers with Nonstationary Data Streams Peng Wu 1,2,*, and Ning Xiong 1 1 School of Innovation, Design and Engineering Mälardalen University, Västerås 721 23, Sweden 2 Department of Computer Engineering, Taiyuan Institute of Technology, Taiyuan 030008, China * Correspondence: 14112078@bjtu.edu.cn Received: 31 March 2023 Accepted: 13 July 2023 Published: 26 September 2023 Abstract: The advancement in Internet of things and sensor technologies has enabled data to be continuously generated with a high rate, i.e., data streams. It is practically infeasible to store streaming data in a hard disk, and apply a traditional batch learning method to extract a relevant knowledge model from these data. This paper studies online incremental learning with data streams, in which one sample is processed at each time to update the existing model. For the learning target, the Bayesian classifier is adopted which is a computationally economical model of easy deployment for online processing in edges or devices. By using the individual new example, we first present an online learning algorithm to incrementally update classifier parameters in a way equivalent to the offline learning counterpart. In order to adapt to concept drifts in nonstationary environments, the proposed online learning algorithm is improved to enable recent examples to be more impactful during the sequential learning procedure. Preliminary simulation tests reveal that the improved online learning algorithm can lead to faster model adaption than the unimproved online algorithm when the data drift occurs. In case of presumed stationary data streams without drifts, the improved online algorithm is proved to be competent by performing at least as good as (sometimes, even better than) the unimproved algorithm.

One-Pass Online Learning Based on Gradient Descent for Multilayer Spiking Neural Networks

Online supervised learning algorithms update synaptic weights in real-time during the running process of spiking neural networks (SNNs), which are important for modeling the behavior and cognitive process of the brain. This article proposes an online supervised learning algorithm based on gradient descent for multilayer feedforward SNNs, where precisely timed spike trains are used to represent neural information. The online learning rule is derived from the real-time error function and backpropagation mechanism. The synaptic weights are adjusted when an output neuron fires a spike. Results of spike train learning demonstrate that the proposed online learning algorithm can achieve higher learning accuracy and requires fewer learning epochs than the corresponding offline learning method and other typical supervised learning algorithms. Furthermore, the proposed algorithm is used for solving pattern classification problems, where the one-pass learning approach is employed for training SNNs. Results show that the proposed algorithm can obtain comparable classification accuracy compared with other state-of-the-art algorithms even in the case of only one iteration. It indicates that the proposed algorithm is effective for solving spatio-temporal pattern recognition problems.

Decentralized DNN Task Partitioning and Offloading Control in MEC Systems With Energy Harvesting Devices

In this paper, we study a decentralized deep neural network (DNN) task partitioning and offloading control problem for a multi-access edge computing (MEC) system with multiple wireless devices (WDs) powered by renewable energy sources. Instead of generic computational tasks, we focus on the case when the WDs and the MEC systems are computing DNN inferencing tasks. For each WD, we build a virtual local-computing CPU workload queue and an energy queue to model the DNN computation dynamics and the battery energy dynamics, respectively. We first introduce a new Lyapunov function based on the DNN tasks' cumulative execution latency and the total energy consumptions of the entire MEC system. Using a Lyapunov optimization approach, the DNN task partitioning and offloading control at the WDs are formulated as a Lyapunov drift minimization problem. To facilitate a distributed implementation, we exploit the linear structure of the cumulative latency and the independency of each WD's local-computing CPU rate, and we decompose the centralized Lyapunov drift minimization problem into multiple distributed subproblems, each of which is associated with a single WD. Next, we develop a parametric online learning algorithm such that each sub-Lyapunov drift minimization problem is solved locally at its associated WD. The proposed solution is completely decentralized in the sense that the sequential offloading and DNN task segmentation control at each WD is determined by its local battery energy queue state information (EQSI), the channel state information (CSI), and the DNN inferencing task workload queue state information (TQSI). Numerical results reveal that the proposed online learning algorithm can achieve significant performance gain over various state-of-the-art baselines.

Efficiently embedding dynamic knowledge graphs

Knowledge graph (KG) embedding encodes the entities and relations from a KG into low-dimensional vector spaces to support various applications such as KG completion, question answering, and recommender systems. In real world, knowledge graphs (KGs) are dynamic and evolve over time with addition or deletion of triples. However, most existing models focus on embedding static KGs while neglecting dynamics. To adapt to the changes in a KG, these models need to be retrained on the whole KG with a high time cost. In this paper, to tackle the aforementioned problem, we propose a new context-aware Dynamic Knowledge Graph Embedding (DKGE) method which supports the embedding learning in an online fashion. DKGE introduces two different representations (i.e., knowledge embedding and contextual element embedding) for each entity and each relation, in the joint modeling of entities and relations as well as their contexts, by employing two attentive graph convolutional networks, a gate strategy, and translation operations. This effectively helps limit the impacts of a KG update in certain regions, not in the entire graph, so that DKGE can rapidly acquire the updated KG embedding by a proposed online learning algorithm. Furthermore, DKGE can also learn KG embedding from scratch. Experiments on the tasks of link prediction and question answering in a dynamic environment demonstrate the effectiveness and efficiency of DKGE.

Learning-Based Prediction and Proactive Uplink Retransmission for Wireless Virtual Reality Network

Wireless Virtual Reality (VR) users are able to enjoy immersive experience from anywhere at anytime. However, providing full spherical VR video with high quality under limited VR interaction latency is challenging. If the viewpoint of the VR user can be predicted in advance, only the required viewpoint is needed to be rendered and delivered, which can reduce the VR interaction latency. Therefore, in this paper, we use offline and online learning algorithms to predict viewpoint of the VR user using real VR dataset. For the offline learning algorithm, the trained learning model is directly used to predict the viewpoint of VR users in continuous time slots. While for the online learning algorithm, based on the VR user&#x0027;s actual viewpoint delivered through uplink transmission, we compare it with the predicted viewpoint and update the parameters and input viewpoints of the online learning algorithm to further improve the prediction accuracy. To guarantee the reliability of the uplink transmission, we integrate the Proactive retransmission scheme into our proposed online learning algorithm. Simulation results show that our proposed online learning algorithm for uplink wireless VR network with the proactive retransmission scheme only exhibits about 5<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> prediction error.

An Online Learning Algorithm for Distributed Task Offloading in Multi-Access Edge Computing

This paper addresses the problem of distributed task offloading centred at individual user terminals in a cellular multi-access edge computing (MEC) system. We introduce an online learning-assisted algorithm based on distributed bandit optimization (DBO) to cope with time-varying cost and time-varying constraint functions with unknown statistics on-the-go. The proposed algorithm jointly exploits the projected dual gradient iterations and a greedy method as well as a single broadcast communicating the MEC states to the users at the end of each decision cycle to minimize task computing-communication delay in the long run at user terminals. To track the performance of the proposed online learning algorithm over time, we define a dynamic regret to assess the closeness of the underlying delay cost of the DBO to a clairvoyant dynamic optimum, and an aggregate violation metric to evaluate the asymptotic satisfaction of the constraints. We derive lower and upper bounds for dynamic regret as well as an upper-bound for the aggregate violation and show that the upper-bounds are sub-linear under sub-linear accumulated hindsight variations. The simulation results and comparisons confirm the effectiveness of the proposed algorithm in the long run.

Adaptive quasi-dynamic state estimation for MV and LV grids

State estimation in middle- (MV) and low-voltage (LV) electrical grids poses a number of challenges for the estimation method employed. A significant difference to high-voltage grids is the lack of measurements as the instrumentation with measurement equipment in MV and LV grids is very sparse due to economical reasons. Typically, pseudo-measurements are used as a replacement for actual measurements to this end. A recently proposed disturbance observer based on the extended Kalman filter uses a simplified dynamic model for the errors in the pseudo-measurements of bus power. The aim is then to estimate the errors in the pseudo-measurements and thereby improving the overall estimation result. Despite initial promising results of this so-called nodal load observer (NLO), the main disadvantage of this method is the need for a suitable dynamic model for the error of the pseudo-measurements. Therefore, we here propose a versatile dynamic model for the disturbance observer based on autoregressive processes (AR). We consider a recently proposed online learning algorithm for the prediction of the AR model parameters together with the extended Kalman filter disturbance observer. We demonstrate that this approach results in an efficient method for the dynamic state estimation for MV and LV grids than the original NLO method.

Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective

This paper studies the problem of non-cooperative radio resource scheduling in a vehicle-to-vehicle communication network. The technical challenges lie in high vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued packets. The frequency allocation at the beginning of each slot by the RSU is regulated following a sealed second-price auction. Each VUE-pair aims to optimize the expected long-term performance. Such interactions among VUE-pairs are modelled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the stochastic game into an equivalent game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium (MPE), which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE policies. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

Online Learning Algorithm for LSSVM Based Modeling with Time-varying Kernels

Online learning based Least Squares Support Vector Machine (LSSVM) can address the modeling problems of a time-varying process, which has a few advantages such as low training time and good general. Nevertheless, many of online learning algorithms cannot adapt the kernel parameters for the time-varying characteristic, so the inferred LSSVM models are low-accuracy. An online learning algorithm with time-varying kernels is proposed to improve online training accuracy of LSSVM model. The kernel parameters are optimized along with time-varying process using updating samples data. To achieve reliable performance during online optimization, we propose a controllable metaheuristic algorithm that adopts a contracted particle swarm optimization with an elaborate chaotic operator. The proposed modeling approach is utilized in the energy efficiency prediction of the electrical smelting process, and the experimental results show that the proposed online learning algorithm can both improve the accuracy of LSSVM model and ensure low online training time.

Resource Allocation Optimization for Delay-Sensitive Traffic in Fronthaul Constrained Cloud Radio Access Networks

The cloud radio access network (C-RAN) provides high spectral and energy efficiency performances, low expenditures and intelligent centralized system structures to operators, which has attracted intense interests in both academia and industry. In this paper, a hybrid coordinated multi-point transmission (H-CoMP) scheme is designed for the downlink transmission in C-RANs, which fulfills the flexible tradeoff between cooperation gain and fronthaul consumption. The queue-aware power and rate allocation with constraints of average fronthaul consumption for the delay-sensitive traffic are formulated as an infinite horizon constrained partially observed Markov decision process (POMDP), which takes both the urgent queue state information (QSI) and the imperfect channel state information at transmitters (CSIT) into account. To deal with the curse of dimensionality involved with the equivalent Bellman equation, the linear approximation of post-decision value functions is utilized. A stochastic gradient algorithm is presented to allocate the queue-aware power and transmission rate with H-CoMP, which is robust against unpredicted traffic arrivals and uncertainties caused by the imperfect CSIT. Furthermore, to substantially reduce the computing complexity, an online learning algorithm is proposed to estimate the per-queue post-decision value functions and update the Lagrange multipliers. The simulation results demonstrate performance gains of the proposed stochastic gradient algorithms, and confirm the asymptotical convergence of the proposed online learning algorithm.

An Adaptive Network-Based Reinforcement Learning Method for MPPT Control of PMSG Wind Energy Conversion Systems

This paper proposes an artificial neural network (ANN)-based reinforcement learning (RL) maximum power point tracking (MPPT) algorithm for permanent-magnet synchronous generator (PMSG)-based variable-speed wind energy conversion systems (WECSs). The proposed MPPT algorithm first learns the optimal relationship between the rotor speed and electrical power of the PMSG through a combination of the ANNs and the Q-learning method. The MPPT algorithm is switched from the online RL to the optimal relation-based online MPPT when the maximum power point is learned. The proposed online learning algorithm enables the WECS to behave like an intelligent agent with memory to learn from its own experience, thus improving the learning efficiency. The online RL process can be reactivated any time when the actual optimal relationship deviates from the learned one due to the aging of the system or a change in the environment. Simulation and experimental results are provided to validate the proposed ANN-based RL MPPT control algorithm for a 5-MW PMSG-based WECS and a small emulated PMSG-based WECS, respectively.

Denoising and Trend Terms Elimination Algorithm of Accelerometer Signals

Acceleration-based displacement measurement approach is often used to measure the polish rod displacement in the oilfield pumping well. Random noises and trend terms of the accelerometer signals are the main factors that affect the measuring accuracy. In this paper, an efficient online learning algorithm is proposed to improve the measurement precision of polish rod displacement in the oilfield pumping well. To remove the random noises and eliminate the trend term of accelerometer signals, the ARIMA model and its parameters are firstly derived by using the obtained data of time series of acceleration sensor signals. Secondly, the period of the accelerometer signals is estimated through the Rife-Jane frequency estimation approach based on Fast Fourier Transform. With the obtained model and parameters, the random noises are removed by employing the Kalman filtering algorithm. The quadratic integration of the period is calculated to obtain the polish rod displacement. Moreover, the windowed recursive least squares algorithm is implemented to eliminate the trend terms. The simulation results demonstrate that the proposed online learning algorithm is able to remove the random noises and trend terms effectively and greatly improves the measurement accuracy of the displacement.

Online Learning of Hierarchical Pitman-Yor Process Mixture of Generalized Dirichlet Distributions With Feature Selection.

In this paper, a novel statistical generative model based on hierarchical Pitman-Yor process and generalized Dirichlet distributions (GDs) is presented. The proposed model allows us to perform joint clustering and feature selection thanks to the interesting properties of the GD distribution. We develop an online variational inference algorithm, formulated in terms of the minimization of a Kullback-Leibler divergence, of our resulting model that tackles the problem of learning from high-dimensional examples. This variational Bayes formulation allows simultaneously estimating the parameters, determining the model's complexity, and selecting the appropriate relevant features for the clustering structure. Moreover, the proposed online learning algorithm allows data instances to be processed in a sequential manner, which is critical for large-scale and real-time applications. Experiments conducted using challenging applications, namely, scene recognition and video segmentation, where our approach is viewed as an unsupervised technique for visual learning in high-dimensional spaces, showed that the proposed approach is suitable and promising.

Double Updating Online Learning

In most kernel based online learning algorithms, when an incoming instance is misclassified, it will be added into the pool of support vectors and assigned with a weight, which often remains unchanged during the rest of the learning process. This is clearly insufficient since when a new support vector is added, we generally expect the weights of the other existing support vectors to be updated in order to reflect the influence of the added support vector. In this paper, we propose a new online learning method, termed Double Updating Online Learning, or DUOL for short, that explicitly addresses this problem. Instead of only assigning a fixed weight to the misclassified example received at the current trial, the proposed online learning algorithm also tries to update the weight for one of the existing support vectors. We show that the mistake bound can be improved by the proposed online learning method. We conduct an extensive set of empirical evaluations for both binary and multi-class online learning tasks. The experimental results show that the proposed technique is considerably more effective than the state-of-the-art online learning algorithms. The source code is available to public at http://www.cais.ntu.edu.sg/~chhoi/DUOL/.

An online supervised learning method for spiking neural networks with adaptive structure

A novel online learning algorithm for Spiking Neural Networks (SNNs) with dynamically adaptive structure is presented. The main contribution of this work lies in the fact that the proposed adaptive SNN is able to classify spike-based spatio-temporal inputs after just one presentation of the training set, i.e. in one pass only, and does not require the entire training set to be available at once. Both the structure and weights of the SNN are learned dynamically through a combination of unsupervised and supervised learning paradigms. The proposed feed-forward SNN consists of three layers of spiking neurons: an input layer which temporally encodes real valued features into spike-based spatio-temporal patterns, a hidden layer of dynamically grown and pruned neurons which perform spatio-temporal clustering, and an output layer for classification. An unsupervised spiking-based clustering algorithm is implemented by the hidden layer whose spiking neurons are trained to compute a temporal Radial Basis Function (RBF) where incoming inputs will selectively activate hidden neurons based on how close the inputs are to the preferred inputs of the hidden neurons. The centre of each hidden RBF spiking neuron is represented by its time to first spike. In addition, a growing and pruning strategy is proposed to adjust the structure of the hidden layer ‘on-the-fly’ as inputs are presented to the SNN. Both the weights and the centres of the hidden RBF neurons are learned in an unsupervised way and classification at the output layer is achieved through supervised learning where the learning windows proposed for STDP and anti-STDP are used to adjust the weights of the output neurons afferent connections. Competition at both the hidden and the output layers is achieved through the use of lateral inhibitory connections between the neurons of each layer. The proposed online learning algorithm is validated on several benchmark datasets. The evaluation results demonstrate that SNNs trained with the proposed approach require only one pass through the training set in order to classify the inputs with comparable accuracies to existing SNN-based approaches as well as traditional representative classifiers.