Online Learning Mechanism Research Articles

Data-Driven Configuration of Interference Coordination Parameters in HetNets

The heterogeneous networks (HetNets) performance is highly dependent on the interference coordination among cells since both macrocells and small cells share the same spectrum. This paper focuses on the configuration of eICIC parameters: almost blank subframe ratio and cell range expansion bias defined for LTE-A. We propose an online learning mechanism based on a big data-driven framework and multiarmed bandit algorithms that retrieves data from the network to learn efficient configurations of eICIC parameters without requiring any previous knowledge about the network (e.g., traffic load, topology, scheduling algorithm). Our numerical results show that our approach attains a significant improvement with respect to the state of the art of online learning algorithms in networks under stationary and variable conditions (e.g., number of small cells, traffic load).

Contextual Online Dictionary Learning for Hyperspectral Image Classification

Sparse representation (SR) has been successfully used in the classification of hyperspectral images (HSIs) by representing HSI pixels over a dictionary and yielding discriminative sparse coefficients. Most of SR-based classification methods construct the dictionary by directly using some labeled pixels as atoms. Such dictionary can lead to inefficient SR for large-sized HSIs, and may be incomplete when the number of labeled pixels is less than the number of spectral bands. This paper proposes a contextual online dictionary learning (DL) method for HSIs classification, which learns a dictionary over the whole image rather than few labeled pixels. The proposed method can effectively and efficiently improve the adaptive representation capability of different pixels with an online learning mechanism. Specifically, the contextual characteristics of the HSI are integrated with discriminative spectral information for online DL, i.e., pushing similar pixels in neighborhood to share similar sparse coefficients with respect to the well-learned dictionary. By this way, the obtained sparse coefficients are structured and discriminative. Finally, a traditional classifier, i.e., the linear support vector machine, is applied to the sparse coefficients, and the final classification results are obtained. Experimental results on real HSIs show the effectiveness of the proposed method.

Robust Adaptive Coverage Control for Robotic Sensor Networks

This paper presents a distributed control algorithm to drive a group of robots to spread out over an environment and provide adaptive sensor coverage of that environment. The robots use an online learning mechanism to estimate the areas in the environment that require more concentrated sensor coverage, while simultaneously providing this coverage. The algorithm differs from previous approaches in that both provable robustness is emphasized in the learning mechanism, and decentralization over a communication network is emphasized in the control. To achieve a provable bound on the learning error, the robots explore the environment to gather sufficient data. They then switch to a Voronoi-based coverage controller to position themselves for sensing. Robots coordinate their learning, control, and mode switching in a fully distributed fashion over their mesh network. It is proved that the robots approximate the weighting function with a known bounded error, and that they converge to locations that are locally optimal for sensing. Simulations and multiple experiments with six iRobot Create robots are presented to illustrate the performance of the method.

Optimal energy-delay tradeoff for opportunistic spectrum access in cognitive radio networks

Cognitive radio (CR) has been considered as a promising technology to enhance spectrum efficiency via opportunistic transmission at link level. Basic CR features allow secondary users (SUs) to transmit only when the licensed channel is not occupied by primary users (PUs). However, waiting for an idle time slot may lead to large packet delays and high energy consumption. We further consider that the SU may decide, at any moment, to use another dedicated way of communication (4G) in order to transmit his packets. Thus, we consider an Opportunistic Spectrum Access (OSA) mechanism that takes into account packet delay and energy consumption. We formulate the OSA problem as a Partially Observable Markov Decision Process (POMDP) by explicitly considering the energy consumption as well as packets’ delay, which are often ignored in existing OSA solutions. Specifically, we consider a POMDP with an average reward criterion. We derive structural properties of the value function and we show the existence of optimal strategies in the class of the threshold strategies. For implementation purposes, we propose online learning mechanisms that estimate the PU activity and determine the appropriate threshold strategy on the fly. In particular, numerical illustrations validate our theoretical findings.

Statistical Mechanics of On-Line Learning Using Correlated Examples and Its Optimal Scheduling

We theoretically study the generalization capability of on-line learning using several correlated input vectors in each update in a statistical-mechanical manner. We consider a model organized with...

SmartLA: Reinforcement learning-based link adaptation for high throughput wireless access networks

High throughput wireless standards based on IEEE 802.11n and IEEE 802.11ac have been developed and released within the last few years as new amendments over the commercially popular IEEE 802.11. IEEE 802.11n and IEEE 802.11ac support a large pool of parameter set such as increased number of spatial streams via multiple input multiple output (MIMO) communications, channel bonding, guard intervals, different modulation and coding schemes, several levels of frame aggregation, block acknowledgement etc. As a consequence, they boost up physical data rate in the order of Gigabits per second. However, all these enhancements have their internal trade-offs with the channel quality, as explored in the existing literature. For example, higher channel bonding levels result in poor performance under high bit error rate. In a free wireless environment, multiple heterogeneous stations share the wireless channel which is again a time-varying system. Consequently, none of these link level parameters provide an optimal performance for all channel quality instances. Therefore, to practically meet the theoretical high throughput, each wireless device should adapt its physical data transmission rate dynamically by an appropriate tuning of different link parameters. Otherwise, high transmission failure may arise. In this paper, we design an adaptive automated on-line learning mechanism, called “Smart Link Adaptation” (SmartLA), for dynamic selection of link parameters, motivated by “State-Action-Reward-State-Action” (SARSA) model, a variant of reinforcement learning. SmartLA can make a wireless station quite intelligent to cope up with various network conditions by exploiting the best suited data rate observed so far for various channel conditions from the past experience as well as by exploring different possible set of parameters. We analyze the performance of SmartLA in both from simulation analysis and over a 26 nodes IEEE 802.11ac testbed (6 access points and 20 client devices). We observe that the proposed link adaptation mechanism performs significantly better compared to other competing mechanisms mentioned in the literature.

Dynamic extreme learning machine for data stream classification

In our society, many fields have produced a large number of data streams. How to mining the interesting knowledge and patterns from continuous data stream becomes a problem which we have to solve. Different from conventional classification algorithms, data stream classification algorithms have to adjust their classification models with the change of data stream because of concept drift. However, conventional classification models will keep stable once models are trained. To solve the problem, a dynamic extreme learning machine for data stream classification (DELM) is proposed. DELM utilizes online learning mechanism to train ELM as basic classifier and trains a double hidden layer structure to improve the performance of ELM. When an alert about concept drift is set, more hidden layer nodes are added into ELM to improve the generalization ability of classifier. If the value measuring concept drift reaches the upper limit or the accuracy of ELM is in a low level, the current classifier will be deleted, and the algorithm will use new data to train a new classifier so as to learn new concept. The experimental results showed DELM could improve the accuracy of classification result, and can adapt to new concept in a short time.

Managing Uncertainty in Autonomic Cloud Elasticity Controllers

Elasticity allows a cloud system to maintain an optimal user experience by automatically acquiring and releasing resources. Autoscaling-adding or removing resources automatically on the fly-involves specifying threshold-based rules to implement elasticity policies. However, the elasticity rules must be specified through quantitative values, which requires cloud resource management knowledge and expertise. Furthermore, existing approaches don't explicitly deal with uncertainty in cloud-based software, where noise and unexpected events are common. The authors propose a control-theoretic approach that manages the behavior of a cloud environment as a dynamic system. They integrate a fuzzy cloud controller with an online learning mechanism, putting forward a framework that takes the human out of the dynamic adaptation loop and can cope with various sources of uncertainty in the cloud.

Modeling of Wind/Environment/Economic Dispatch in power system and solving via an online learning meta-heuristic method

This paper addresses the effect of the wind power units into the classical Environment/Economic Dispatch (EED) model which called hereafter as Wind/Environment/Economic Dispatch (WEED) problem. The optimal dispatch between thermal and wind units so that minimized the total generating costs are considered as multi objective model. Normally, the nature of the wind energy as a renewable energy sources has uncertainty in generation. Therefore, in this paper, use a practical model known as 2m-point to estimate the uncertainty of wind power. To solve the WEED problem, this paper proposed a new meta-heuristic optimization algorithm that uses online learning mechanism. Honey Bee Mating Optimization (HBMO), a moderately new population-based intelligence algorithm, shows fine performance on optimization problems. Unfortunately, it is usually convergence to local optima. Therefore, in the proposed Online Learning HBMO (OLHBMO), two neural networks are trained when reached to the predefined threshold by current and previous position of solutions and their fitness values. Moreover, Chaotic Local Search (CLS) operator is use to develop the local search ability and a new data sharing model determine the set of non-dominated optimal solutions and the set of non-dominated solutions to kept in the external memory. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) as a decision-making technique is employed to find the best solution from the set of Pareto solutions. The proposed model has been individually examined and applied on the IEEE 30-bus 6-unit, the IEEE 118-bus 14-unit, and 40-unit with valve point effect test systems. The robustness and effectiveness of this algorithm is shows by these test systems compared to other available algorithms.

The Development of Selective Attention Orienting is an Agent of Change in Learning and Memory Efficacy.

The present study examined whether the developmental transition from facilitation-based orienting mechanisms available very early in life to selective attention orienting (e.g., inhibition of return, IOR) promotes better learning and memory in infancy. We tested a single age group (4-month-olds) undergoing rapid development of attention orienting mechanisms. Infants completed a spatial cueing task designed to elicit IOR, in which cat or dog category exemplars consistently appeared in either the cued or noncued locations. Infants were subsequently tested on a visual paired comparison of exemplars from these cued and noncued animal categories. As expected, infants showed either facilitation-based orienting or the more mature IOR-based orienting during spatial cueing/encoding. Infants who demonstrated IOR-based orienting showed memory for both specific exemplars and broader category learning, whereas those who showed facilitation-based orienting showed weaker evidence of learning. Attention orienting also interacted with previous pet experience, such that the number of pets at home influenced learning only when infants engaged facilitation-based orienting during encoding. Learning in the context of IOR-based orienting was stable regardless of pet experience, suggesting that selective attention serves as an online learning mechanism during visual exploration that is less sensitive to prior experience.

Feature Adaptive Online Sequential Extreme Learning Machine for lifelong indoor localization

Wi-Fi-based indoor localization with high capability and feasibility needs to implement lifelong online learning mechanism. However, the characteristic of Wi-Fi is wide variability, which lies in not only the fluctuation of signal strength value, but also the increase or decrease in the number of access points (APs). The traditional algorithms are effective for signal fluctuation, but cannot handle the dimension-changing problem of features caused by increase and decrease in APs' number. To solve this problem, we propose a Feature Adaptive Online Sequential Extreme Learning Machine (FA-OSELM) algorithm. It can transfer the original model to a new one with a small number of data with new features, so as to make the new model suitable for the new feature dimension. The experiments show that the FA-OSELM can get higher accuracy with a small amount of new data, and it is an effective method to make lifelong indoor localization practical.

Multi-agent Gaussian Adaptive Resonance Theory Map for building energy control and thermal comfort management of UCLan's WestLakes Samuel Lindow Building

This paper presents a Gaussian Adaptive Resonance Theory Map (gARTMAP) incorporated to a multi-agent system (MAS) for building intelligent heat management system (IHMS), or the gARTMAP-MAS IHMS. The gARTMAP aims to maintain the desired space temperature defined by the building occupants (thermal comfort management) and improve energy efficiency by intelligently controlling the energy flow and usage in the building (building energy control) with minimal waste. The gARTMAP-MAS IHMS is a three-layer MAS, consisting of an agent in the Source Layer responsible for building energy control, agents in the User Layer for thermal comfort management, and an agent in the Mediator Layer which mediates the communication between the other two layers. Existing MAS typically uses a rule-based approach for its agents, and the rules are generally pre-defined. The incorporation of the gARTMAP to the agents provides additional adaptive capabilities via the use of its online learning mechanism. Simulation results demonstrated that the gARTMAP-MAS IHMS provides better (automated) energy control and thermal comfort management for a building environment in comparison to the existing rule-based MAS approach.

Research on Target Tracking Based on TLD Algorithm

This paper introduces the theoretical basis and the framework of improved algorithm of TLD(Tracking-Learning-Detection, TLD) algorithm. Difference between the improved TLD algorithm and traditional algorithm tracking were compared by experiments in the paper. When the improved algorithm has joined the online learning mechanism, it improves the effects of target tracking. Trough the experiments between the improved TLD algorithm and traditional algorithm, it turns out that TLD algorithm has good stability and robustness in target tracking.

Active Bucket Categorization for High Recall Video Retrieval

<?Pub Dtl=""?> There are large amounts of digital video available. High recall retrieval of these requires going beyond the ranked results, which is the common target in high precision retrieval. To aid high recall retrieval, we propose Active Bucket Categorization, which is a multicategory interactive learning strategy which extends MediaTable <citerefgrp xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><citeref refid="ref1"/> </citerefgrp> , our multimedia categorization tool. MediaTable allows users to place video shots into buckets: user-assigned subsets of the collection. Our Active Bucket Categorization approach augments this by unobtrusively expanding these buckets with related footage from the whole collection. In this paper, we propose an architecture for active bucket-based video retrieval, evaluate two different learning strategies, and show its use in video retrieval with an evaluation using three groups of nonexpert users. One baseline group uses only the categorization features of MediaTable such as sorting and filtering on concepts and fast grid preview, but no online learning mechanisms. One group uses on-demand passive buckets. The last group uses fully automatic active buckets which autonomously add content to buckets. Results indicate a significant increase in the number of relevant items found for the two groups of users using bucket expansions, yielding the best results with fully automatic bucket expansions, thereby aiding high recall video retrieval significantly.

Online Learning in BitTorrent Systems

We propose a BitTorrent-like protocol based on an online learning (reinforcement learning) mechanism, which can replace the peer selection mechanisms in the regular BitTorrent protocol. We model the peers' interactions in the BitTorrent-like network as a repeated stochastic game, where the strategic behaviors of the peers are explicitly considered. A peer that applies the reinforcement learning (RL)-based mechanism uses the observations on the associated peers' statistical reciprocal behaviors to determine its best responses and estimate the corresponding impact on its expected utility. The policy determines the peer's resource reciprocations such that the peer can maximize its long-term performance. We have implemented the proposed mechanism and incorporated it into an existing BitTorrent client. Our experiments performed on a controlled Planetlab testbed confirm that the proposed protocol 1) promotes fairness and provides incentives to contributed resources, i.e., high capacity peers improve their download completion time by up to 33 percent, 2) improves the system stability and robustness, i.e., reduces the peer selection fluctuations by 57 percent, and (3) discourages free-riding, i.e., peers reduce their uploads to free-riders by 64 percent as compared to the regular BitTorrent protocol.

CBR Based Collision Avoidance Approach to Reactive Control for Multi-Robot System

In this paper we discuss a Case-Based Reasoning (CBR) method which is an on-line learning mechanism for dynamic selection and modification of behavior assemblages for collision avoidance of multi-robot system. The CBR module is designed as an additional reactive control system which provide flexible performance in novel environments without extensive high-level reasoning that would slow the system down. The results by robot simulation software MissionLab show that the CBR are effective for making decisions to avoid the collision with static obstacles as well as moving robots in multi-robot system.

Utilization of Intelligent Software Agent Features for Improving E-Learning Efforts

E-learning is one of the most important learning approaches within which intelligent software agents can be efficiently used so as to automate and facilitate the process of learning. The aim of this paper is to illustrate a comprehensive categorization of intelligent software agent features, which is valuable for being deployed in the virtual world of learning and training. The characteristics of agents make them worthwhile for being used in the implementation and improvement of e-learning information systems, especially the web-based or web-enabled e-learning mechanisms. An extensive exploration of agent-related international papers has been conducted so as to cover a wide range of internationally accepted features as well as the classification of agents and their correspondent references. These agent characteristics enable the online learning mechanisms to act on behalf of the user in electronic learning and to respond to specific requirements in a more efficient and effective manner than the traditional learning systems. The implementation of agents according to the most useful features helps teachers transfer the related knowledge easier, faster, and more directed towards the targeted audience of the e-learning system for a better and deeper learning quality.Request access from your librarian to read this article's full text.

Utilization of Intelligent Software Agent Features for Improving E-Learning Efforts

E-learning is one of the most important learning approaches within which intelligent software agents can be efficiently used so as to automate and facilitate the process of learning. The aim of this paper is to illustrate a comprehensive categorization of intelligent software agent features, which is valuable for being deployed in the virtual world of learning and training. The characteristics of agents make them worthwhile for being used in the implementation and improvement of e-learning information systems, especially the web-based or web-enabled e-learning mechanisms. An extensive exploration of agent-related international papers has been conducted so as to cover a wide range of internationally accepted features as well as the classification of agents and their correspondent references. These agent characteristics enable the online learning mechanisms to act on behalf of the user in electronic learning and to respond to specific requirements in a more efficient and effective manner than the traditional learning systems. The implementation of agents according to the most useful features helps teachers transfer the related knowledge easier, faster, and more directed towards the targeted audience of the e-learning system for a better and deeper learning quality.

STABILIZED NONLINEAR DISCRETE-TIME ADAPTIVE CONTROLLER BASED ON FUZZY RULES EMULATED NETWORKS AND TIME VARYING LEARNING RATE

A direct adaptive controller based on fuzzy rules emulated network (FREN) for a class of unknown nonlinear discrete-time systems is addressed in this article. According to the fuzzy inference system inside FREN, the human knowledge about the unknown systems is transferred to be if–then rules for setting the network structure. All adjustable parameters are tuned by the on-line learning mechanism with time varying learning rate. This variation of learning rate is introduced by main theorem to improve the system performance and stabilization. Furthermore, the convergence of adjustable parameters is guaranteed through the on-line learning and membership functions properties. The theoretical validation is delineated by the experimental setup with the commercial omni-directional mobile robot system.

A Learning based Hierarchical Framework for Automatic Prostate Localization in CT Images.

Accurate localization of prostate in CT images plays an important role in image guided radiation therapy. However, it has two main challenges. The first challenge is due to the low contrast in CT images. The other challenge is due to the uncertainty of the existence of bowel gas. In this paper, a learning based hierarchical framework is proposed to address these two challenges. The main contributions of the proposed framework lie in the following aspects: (1) Anatomical features are extracted from input images, and the most salient features at distinctive image regions are selected to localize the prostate. Regions with salient features but irrelevant to prostate localization are also filtered out. (2) An image similarity measure function is explicitly defined and learnt to enforce the consistency between the distance of the learnt features and the underlying prostate alignment. (3) An online learning mechanism is used to adaptively integrate both the inter-patient and patient-specific information to localize the prostate. Based on the learnt image similarity measure function, the planning image of the underlying patient is aligned to the new treatment image for segmentation. The proposed method is evaluated on 163 3D prostate CT images of 10 patients, and promising experimental results are obtained.