Concurrent Learning Research Articles

Composite Learning Fuzzy Control of Uncertain Nonlinear Systems

Function approximation accuracy and computational cost are two major concerns in approximation-based adaptive fuzzy control. In this paper, a model reference composite learning fuzzy control strategy is proposed for a class of affine nonlinear systems with functional uncertainties. In the proposed approach, a modified modeling error that utilizes data recorded online is defined as a prediction error, a linear filter is applied to estimate time derivatives of plant states, and both the tracking error and the prediction error are exploited to update parametric estimates. It is proven that the closed-loop system achieves semiglobal practical exponential stability by an interval-excitation condition which is much weaker than a persistent-excitation condition. Compared with a concurrent learning approach that has the same aim as this study, the computational cost of the proposed approach is significantly reduced for the guarantee of accurate function approximation. An illustrative example of aircraft wing rock control has been provided to verify effectiveness of the proposed control strategy.

Direct Adaptive Limit and Control Margin Estimation with Concurrent Learning

In this paper, two vital signals to enable flight envelope protection, namely the onset to the flight envelope (limit margin) and the available control travel to reach the limit boundary (control margin), are estimated using improved noniterative adaptive neural-network-based approximate models. The adaptive elements use current and past information (concurrent learning) and have guaranteed signal bounds. Minimum singular value maximization is used to record data for concurrent learning. Results showed better convergence properties of the network weights compared with results in the literature in which only the current data is used for network weight updates. Two methods are introduced to calculate control margins from approximate models. None of the introduced methods require iteration and therefore remove previously introduced assumptions related to iteration convergence. A nonlinear fixed-wing aircraft model is used to show the effectiveness in simulation for estimating limit and control margins and avoiding the limit through artificial control saturation.

Naloxone impairs concurrent but not sequential flavor aversion: Resorting to a flexible/explicit learning.

The role of opiate systems has been extensively studied in relation to learning and memory. Naloxone (Nx), an opiate antagonist, was administrated in concurrent (Experiment 1) and sequential (Experiment 2) flavor aversion learning (FAL) tasks. The outcomes demonstrate that Nx impairs the acquisition of concurrent but not sequential FAL. In the concurrent learning (7 trials), both control (vehicle) and Nx2 (treated with Nx only on the first 2 days) groups learned the task. Furthermore, these 2 groups retained the learning in a discrimination test without drug administration (Day 8) but failed a reversal test (Day 9). In contrast, the Nx group (7 trials with Nx) showed no concurrent learning but correctly performed the discrimination test (Day 8) and, critically, the reversal test. These results suggest that Nx blocks concurrent (implicit) learning in these experiments but induces animals to resort to new strategies that are flexible, a characteristic of explicit learning.

Concurrent cognitive processes in rat serial pattern learning: II. Discrimination learning, rule learning, chunk length, and multiple-item memories.

The current experiment examined the factors that determine acquisition for elements of highly structured serial patterns. Three groups of rats were trained on three patterns with parallel rule-based hierarchical structure, but with 3-, 4-, or 5-element chunks, each with a final violation element. Once rats mastered their patterns, probe patterns were introduced to answer several questions. To assess the extent to which the learned response pattern depended on intrachamber location cues for anticipating different element types, Spatial Shift Probes shifted the starting lever of patterns to locations that positioned chunk boundaries where they had never been experienced during training. To assess the extent to which a phrasing cue is necessary for rats to perform a chunk-boundary response, a Cue Removal Probe tested whether rats would produce a chunk-boundary response in the correct serial position if the phrasing cue was omitted. To assess the extent to which cues from multiple trials leading up to the violation element are required to anticipate the violation element, Multiple-Item Memory Probes required rats to make an unexpected response on one of the elements in the last two chunks of the pattern prior to the violation element. The results indicated that rats used multiple concurrent learning and memory processes to master serial patterns, including discrimination learning, rule learning, encoding of chunk length, and multiple-item memories.

Model-based reinforcement learning for approximate optimal regulation

Reinforcement learning (RL)-based online approximate optimal control methods applied to deterministic systems typically require a restrictive persistence of excitation (PE) condition for convergence. This paper develops a concurrent learning (CL)-based implementation of model-based RL to solve approximate optimal regulation problems online under a PE-like rank condition. The development is based on the observation that, given a model of the system, RL can be implemented by evaluating the Bellman error at any number of desired points in the state space. In this result, a parametric system model is considered, and a CL-based parameter identifier is developed to compensate for uncertainty in the parameters. Uniformly ultimately bounded regulation of the system states to a neighborhood of the origin, and convergence of the developed policy to a neighborhood of the optimal policy are established using a Lyapunov-based analysis, and simulation results are presented to demonstrate the performance of the developed controller.

A robust approach to robot team learning

The paper achieves two outcomes. First, it summarizes previous work on concurrent Markov decision processes (CMDPs) currently demonstrated for use with multi-agent foraging problems. When using CMDPs, each agent models the environment using two Markov decision process (MDP). The two MDPs characterize a multi-agent foraging problem by modeling both a single-agent foraging problem, and multi-agent task allocation problem, for each agent. Second, the paper studies the effects of state uncertainty on a heterogeneous robot team that utilizes the aforementioned CMDP modelling approach. Furthermore, the paper presents a method to maintain performance despite state uncertainty. The resulting robust concurrent individual and social learning (RCISL) mechanism leads to an enhanced team learning behaviour despite state uncertainty. The paper analyzes the performance of the concurrent individual and social learning mechanism with and without a particle filter for a heterogeneous foraging scenario. The RCISL mechanism confers statistically significant performance improvements over the CISL mechanism.

Frequency selective learning model reference adaptive control

This study proposes a new frequency selective learning adaptive control approach for model reference adaptive control systems to improve their adaptation performance without depending on high learning rates. It is developed by inspiration from the philosophies of the Q-modification and concurrent learning approaches and retains the key advantages of both. The philosophy of this proposed approach is that the known part of the plant dynamics passes through multiple filters. By constraint of the governing dynamic equations, this result equals an expression containing the unknown parameters in a filtered version, which is used for augmentation of the update law. The use of multiple filters aims at increasing the rank of the update law based on online instantaneous information and for a sufficient number of filters at different bandwidths, exponential stability can be achieved in the presence of the structured matched uncertainty. Moreover, it can cope with a sudden change of the system configuration. As a result, it leads to an efficient adaptive control approach. Furthermore, a challenging roll control problem of an aircraft demonstrates the usefulness of this proposed approach.

Self-organizing fuzzy failure diagnosis of aircraft sensors

A novel scheme for diagnosing sensor failures in a flight control system is presented. In the proposed scheme, a set of self-organizing fuzzy systems named as extended sequential adaptive fuzzy inference systems (ESAFISs) are applied as the online approximators for recognizing the sensor outputs. ESAFIS is an online learning fuzzy system with concurrent structure and parameter learning. The rules of the ESAFIS are added or deleted based on the input data without predefining them by trial and error. From an analysis of the residual signals between the estimated states and the measurements of the sensors, the failure diagnosis by determining the failure detection, identification and accommodation can be achieved. The efficiency of the proposed scheme is demonstrated by simulation examples where hard and soft failures in the angular rate gyros are successfully diagnosed.

Learning User Real-Time Intent for Optimal Dynamic Web Page Transformation

Many e-commerce websites struggle to turn visitors into real buyers. Understanding online users’ real-time intent and dynamic shopping cart choices may have important implications in this realm. This study presents an individual-level, dynamic model with concurrent optimal page adaptation that learns users’ real-time, unobserved intent from their online cart choices, then immediately performs optimal Web page adaptation to enhance the conversion of users into buyers. To suggest optimal strategies for concurrent page adaptation, the model analyzes each individual user’s browsing behavior, tests the effectiveness of different marketing and Web stimuli, as well as comparison shopping activities at other sites, and performs optimal Web page transformation. Data from an online retailer and a laboratory experiment reveal that concurrent learning of the user’s unobserved purchase intent and real-time, intent-based optimal interventions greatly reduce shopping cart abandonment and increase purchase conversions. If the concurrent, intent-based optimal page transformation for the focal site starts after the first page view, shopping cart abandonment declines by 32.4% and purchase conversion improves by 6.9%. The optimal timing for the site to intervene is after three page views, to achieve efficient learning of users’ intent and early intervention simultaneously.

Ideomotor feedback control in a recurrent neural network.

The architecture of a neural network controlling an unknown environment is presented. It is based on a randomly connected recurrent neural network from which both perception and action are simultaneously read and fed back. There are two concurrent learning rules implementing a sort of ideomotor control: (i) perception is learned along the principle that the network should predict reliably its incoming stimuli; (ii) action is learned along the principle that the prediction of the network should match a target time series. The coherent behavior of the neural network in its environment is a consequence of the interaction between the two principles. Numerical simulations show a promising performance of the approach, which can be turned into a local and better "biologically plausible" algorithm.

Learning multiple lists at the same time in the Hebb repetition effect.

In short-term ordered recall, when 1 list of items is repeated over the course of the experiment, recall performance typically improves. This is known as the Hebb repetition effect (Hebb, 1961). In the present study, we contrasted the typical condition involving a single repeated sequence with concurrent learning of 2 repeated sequences. Participants performed a spatial recall task, in which sequences of 7 dots were presented in each trial. For a given participant, the location of the dots were the same for all trials. Presentation order of dots varied randomly, except for 1 or 2 particular series that were repeated every 4 trials. Results revealed a significant learning slope for both the single and dual list conditions and learning was as efficient in both conditions. The findings provide further evidence in support of models linking the Hebb repetition effect to word-form learning.

Modelling successful primary care for multimorbidity: a realist synthesis of successes and failures in concurrent learning and healthcare delivery.

BackgroundPeople are increasingly living for longer with multimorbidity. Medical education and healthcare delivery must be re-orientated to meet the societal and individual patient needs that multimorbidity confers. The impact of multimorbidity on the educational needs of doctors is little understood. There has been little critique of how learning alongside healthcare provision is negotiated by patients, general practitioners and trainee doctors. This study asked ‘what is known about how and why concurrent healthcare delivery and professional experiential learning interact to generate outcomes, valued by patients, general practitioners and trainees, for patients with multimorbidity in primary care?’MethodsThis realist synthesis is reported using RAMESES standards. Relationship-centred negotiation of needs-based learning and care was the primary outcome of interest. Healthcare, social science and educational literature were sought as evidence. Data extraction focused on context, mechanism and outcome configurations within studies and on data which might assist understanding and explain; i) these configurations; ii) the relationships between them and; iii) their role and place in evolving programme theories arising from data synthesis. Mind-mapping software and team meetings were used to aid interpretative analysis.ResultsThe final synthesis included 141 papers of which 34 contained models for workplace-based experiential learning and/or patient care. Models of experiential learning for practitioners and for patient engagement were congruent, frequently referencing theories of transformation and socio-cultural processes as mechanisms for improving clinical care. Key issues included the perceived impossibility of reconciling personalised concepts of success with measurability of clinical markers or adherence to guidelines, and the need for greater recognition of social dynamics between patients, GPs and trainees including the complexities of shared responsibilities. A model for considering the implications of concurrency for learning and healthcare delivery in the context of multimorbidity in primary care is proposed and supporting evidence is presented.ConclusionsThis study is novel in considering empirical evidence from patients, GPs and trainees engaged in concurrent learning and healthcare delivery. The findings should inform future interventions designed to produce a medical workforce equipped to provide multimorbidity care.Trial registrationPROSPERO International prospective register of systematic reviews CRD42013003862Electronic supplementary materialThe online version of this article (doi:10.1186/s12875-015-0234-9) contains supplementary material, which is available to authorized users.

Coordinated learning based on time-sharing tracking framework and Gaussian regression for continuous multi-agent systems

Applying multi-agent reinforcement learning (MARL) in continuous distributed control system is an attractive issue, because it entitles agents adaptively to construct a cooperative behavior, even if the dynamics of such distributed system is unknown a priori. However the implementation of MARL always suffers from dimension explosion, nonstationary learning, and generalization in continuous systems. This paper presents a continuous coordinated learning algorithm with time-sharing tracking framework (CCL-TT) to deal with these problems, in which the value function is dimension reduced to lighten dimension explosion, the time-sharing tracking framework (TTF) is developed to solve nonstationary learning, and Gaussian regression modeling is applied to realize generalization. With TTF, a macroscopic concurrent learning is set up to meet the requirements of temporal stationary condition in value learning and generalization. Finally the simulation illustrates how CCL-TT realizes cooperative learning without knowledge about the dynamics of the system, even with disturbance.

A ranking method for the concurrent learning of compounds with various activity profiles.

BackgroundIn this study, we present a SVM-based ranking algorithm for the concurrent learning of compounds with different activity profiles and their varying prioritization. To this end, a specific labeling of each compound was elaborated in order to infer virtual screening models against multiple targets. We compared the method with several state-of-the-art SVM classification techniques that are capable of inferring multi-target screening models on three chemical data sets (cytochrome P450s, dehydrogenases, and a trypsin-like protease data set) containing three different biological targets each.ResultsThe experiments show that ranking-based algorithms show an increased performance for single- and multi-target virtual screening. Moreover, compounds that do not completely fulfill the desired activity profile are still ranked higher than decoys or compounds with an entirely undesired profile, compared to other multi-target SVM methods.ConclusionsSVM-based ranking methods constitute a valuable approach for virtual screening in multi-target drug design. The utilization of such methods is most helpful when dealing with compounds with various activity profiles and the finding of many ligands with an already perfectly matching activity profile is not to be expected.Electronic supplementary materialThe online version of this article (doi:10.1186/s13321-014-0050-6) contains supplementary material, which is available to authorized users.

Concurrent learning: How firms develop multiple dynamic capabilities in parallel

Much is known about the importance of dynamic capabilities. Yet, surprisingly little is known about how multiple dynamic capabilities might be developed in parallel, since most existing work explores a particular dynamic capability in isolation. Using rich quantitative and qualitative data on Dow Chemical's acquisitions, joint ventures, and divestitures over the past 20 years, we seek to address this gap. Besides contributing by adding fresh insights about managing growth and the utility of distributed practice, and by shedding light on positive and negative experience transfer, our core contribution is an emergent theoretical framework that develops the concept of “concurrent learning. ” Copyright © 2014 John Wiley & Sons, Ltd.

Multirobot Cooperative Learning for Predator Avoidance

Multirobot collaboration has great potentials in tasks, such as reconnaissance and surveillance. In this paper, we propose a multirobot system that integrates reinforcement learning and flocking control to allow robots to learn collaboratively to avoid predator/enemy. Our system can conduct concurrent learning in a distributed fashion as well as generate efficient combination of high-level behaviors (discrete states and actions) and low-level behaviors (continuous states and actions) for multirobot cooperation. In addition, the combination of reinforcement learning and flocking control enables multirobot networks to learn how to avoid predators while maintaining network topology and connectivity. The convergence and scalability of the proposed system are investigated. Simulations and experiments are performed to demonstrate the effectiveness of the proposed system.

Visual context does not promote concurrent sequence learning

Concurrent sequence learning (CSL) of two or more sequences refers to the concurrent maintenance, in memory, of the two or more sequence representations. Research using the serial reaction time task has established that CSL is possible when the different sequences involve different dimensions (e.g., visuospatial locations versus manual keypresses). Recently some studies have suggested that visual context can promote CSL if the different sequences are embedded in different visual contexts. The results of these studies have been difficult to interpret because of various limitations. Addressing the limitations, the current study suggests that visual context does not promote CSL and that CSL may not be possible when the different sequences involve the same elements (i.e., the same target locations, response keys and effectors).

Reinforcement learning and neural networks for multi-agent nonzero-sum games of nonlinear constrained-input systems

This paper presents an online adaptive optimal control method based on reinforcement learning to solve the multi-agent nonzero-sum (NZS) differential games of nonlinear constrained-input continuous-time systems. A non-quadratic cost functional associated with each agent is employed to encode the saturation nonlinearity into the NZS game. The algorithm is implemented as a separate actor-critic neural network (NN) structure for every participant in the game, where adaptation of both NNs is performed simultaneously and continuously. The technique of concurrent learning is utilized to obtain novel update laws for the critic NN weights. That is, recorded data and current data are used concurrently for adaptation of the critic NN weights. This results in an algorithm where an easier and verifiable condition is sufficient for parameter convergence rather than the restrictive persistence of excitation (PE) condition. The stability of the closed-loop systems is guaranteed and the convergence to the Nash equilibrium solution of the game is shown. Simulation results show the effectiveness of the proposed method.

Concurrent Learning Enabled Adaptive Limit Detection for Active Pilot Cueing

Neural-network-based adaptive dynamic models are commonly used to estimate allowable control travel and the proximity to a limiting flight condition in the design of advanced envelope protection algorithms for fly-by-wire aircraft. In this paper, linear models are compensated with adaptive neural networks, which use instantaneous sensor data as well as past flight history information for concurrent learning. A law for collecting appropriate training data into the history stack is established. It is observed that using the proposed time history data for online neural network training provides more accurate dynamic trim and control limit predictions compared to using instantaneous sensor data only. Simulation results for a fixed-wing aircraft during maneuvers show comparisons between the different adaptation schemes.

Online concurrent reinforcement learning algorithm to solve two‐player zero‐sum games for partially unknown nonlinear continuous‐time systems

SummaryOnline adaptive optimal control methods based on reinforcement learning algorithms typically need to check for the persistence of excitation condition, which is necessary to be known a priori for convergence of the algorithm. However, this condition is often infeasible to implement or monitor online. This paper proposes an online concurrent reinforcement learning algorithm (CRLA) based on neural networks (NNs) to solve the H ∞ control problem of partially unknown continuous‐time systems, in which the need for persistence of excitation condition is relaxed by using the idea of concurrent learning. First, H ∞ control problem is formulated as a two‐player zero‐sum game, and then, online CRLA is employed to obtain the approximation of the optimal value and the Nash equilibrium of the game. The proposed algorithm is implemented on actor–critic–disturbance NN approximator structure to obtain the solution of the Hamilton–Jacobi–Isaacs equation online forward in time. During the implementation of the algorithm, the control input that acts as one player attempts to make the optimal control while the other player, that is, disturbance, tries to make the worst‐case possible disturbance. Novel update laws are derived for adaptation of the critic and actor NN weights. The stability of the closed‐loop system is guaranteed using Lyapunov technique, and the convergence to the Nash solution of the game is obtained. Simulation results show the effectiveness of the proposed method. Copyright © 2014 John Wiley & Sons, Ltd.