Continuous Task Research Articles

Personalized Federated Continual Learning for Task-Incremental Biometrics

In the age of Internet of Things where information is explosively growing, people pay more attention on personal privacy. In the real-world task-incremental scenario for biometrics, every edge device faces continuous task flows of private data without communication with others. Security and Performance are the primary concerns in identity authentication, and Federated Continual Learning (FCL) is a promising solution. In this paper, we design a personalized federated continual learning framework to solve the problem of sequentially identification in every distributed device. For each client, we create an adaptive continual meta-learning model called cTD-αMAML, aiming to align the gradients of previous and new tasks and to make the learning-rate model learnable. For central aggregation, the server gathers the meta-initialization from every local update and allocates the updated global meta-initialization to clients. We propose an extension of FedAvg to locally reserve the learnable learning-rate network to realize the personalization of clients. Results prove that in Continual Learning, our cTD-αMAML can learn to learn the seen tasks and avoid catastrophic forgetting. And in Federated Continual Learning, our personalized method realizes the knowledge transferring across clients, meanwhile improving the local performance and reducing the communication cost. In this way, the proposed personalized federated continual learning framework can obtain a biometric template that is able to learn the expression space for new tasks with rapid adaption.

A Trajectory Tracking and Local Path Planning Control Strategy for Unmanned Underwater Vehicles

The control strategy of an underdriven unmanned underwater vehicle (UUV) equipped with front sonar and actuator faults in a continuous task environment is investigated. Considering trajectory tracking and local path planning in complex-obstacle environments, we propose a task transition strategy under the event-triggered mechanism and design the corresponding state space and action space for the trajectory tracking task under the deep reinforcement learning framework. Meanwhile, a feed-forward compensation mechanism is designed to counteract the effects of external disturbances and actuator faults in combination with a reduced-order extended state observer. For the path planning task under the rapidly exploring random tree (RRT) framework, a reward component and angular factors are introduced to optimize the growth and exploration points of the extended tree under the consideration of the shortest distance, optimal energy consumption, and steering angle constraints. The effectiveness of the proposed method was verified through continuous task simulations of trajectory tracking and local path planning.

Invariant Representations Learning with Future Dynamics

High-dimension inputs limit the sample efficiency of deep reinforcement learning, increasing the difficulty of applying it to real-world continuous control tasks, especially in uncertain environments. A good state embedding is crucial for the performance of agents on downstream tasks. The bisimulation metric is an excellent representation learning method that can abstract task-relevant invariant latent embeddings of states based on behavior similarity. However, since only one time-step transition is considered, the features captured by this method we call short-term dynamics. We think long-term dynamics are also important for state representation learning. In this paper, we present Invariant Representations Learning with Future Dynamics (RLF), which uses graph neural networks to learn long-term dynamics and trains the representation network based on a new state metric inspired by bisimulation relation. We experimented with our method on continuous control tasks from DeepMind Control Suite and showed that the RLF can mine more stable embeddings than the state-of-the-art representation learning methods for both state and pixel inputs. The learned policy on top of the embeddings has higher sample efficiency and performance and generalizes well in different tasks.

Intrinsically Motivated Multi-Goal Reinforcement Learning Using Robotics Environment Integrated with OpenAI Gym

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabelling the goals. In open-ended and changing environments, agents face a wide range of potential tasks that might not come with associated reward functions. Such autonomous learning agents must set their own tasks and build their own curriculum through an intrinsically motivated exploration. Because some tasks might prove easy and some impossible, agents must actively select which task to practice at any given moment, to maximize their overall mastery on the set of learnable tasks. The purpose of this technical report is two-fold. First, it introduces a suite of challenging continuous control tasks (integrated with OpenAI Gym) based on currently existing robotics hardware. The tasks include pushing, sliding and pick & place with a Fetch robotic arm as well as in-hand object manipulation with a Shadow Dexterous Hand. All tasks have sparse binary rewards and follow a Multi-Goal Reinforcement Learning (RL) framework in which an agent is told what to do using an additional input. The second part of the paper presents a set of concrete research ideas for improving RL algorithms, most of which are related to Multi-Goal RL and Hindsight Experience Replay. The Fetch environments are based on the 7-DoF Fetch robotics arm,2 which has a two-fingered parallel gripper. Agents focus on achievable tasks first and focus back on tasks that are being forgotten. Experiments conducted in a new multi-task multi-goal robotic environment show that our algorithm benefits from these two ideas and demonstrate properties of robustness to distracting tasks, forgetting and changes in body properties

Romanian (subject-like) DPs attract more than bare nouns: Evidence from speeded continuations

This paper investigates whether agreement attraction is modulated by distributional properties determining subject-likelihood by examining the degree to which bare nouns and full determiner phrases (DPs) cause agreement attraction effects in Romanian. Romanian represents an ideal testing ground for this, given two distributional constraints making bare nouns less subject-like: Locative Determiner Omission, preventing locative prepositions from taking nouns with definite articles (unless modified by adjectives), and the Naked Noun Constraint, disallowing bare nouns as preverbal subjects. We predicted that bare nouns should be less likely to trigger agreement attraction than overt DPs. We conducted four speeded forced-choice sentence continuation tasks on Romanian native speakers to test this prediction. We observe that overt DPs cause significantly more attraction than bare nouns. We suggest that the results are consistent with a cue-based retrieval mechanism for forming agreement dependencies, where cues that determine subjecthood are used to reactivate elements in working memory upon processing a verb. These cues can be language specific, and in Romanian, this means that agreement attraction is sensitive to the morphophonological overtness of the determiner.

A mechanistic insight into sources of error of visual working memory in multiple sclerosis

Working memory (WM) is one of the most affected cognitive domains in multiple sclerosis (MS), which is mainly studied by the previously established binary model for information storage (slot model). However, recent observations based on the continuous reproduction paradigms have shown that assuming dynamic allocation of WM resources (resource model) instead of the binary hypothesis will give more accurate predictions in WM assessment. Moreover, continuous reproduction paradigms allow for assessing the distribution of error in recalling information, providing new insights into the organization of the WM system. Hence, by utilizing two continuous reproduction paradigms, memory-guided localization (MGL) and analog recall task with sequential presentation, we investigated WM dysfunction in MS. Our results demonstrated an overall increase in recall error and decreased recall precision in MS. While sequential paradigms were better in distinguishing healthy control from relapsing-remitting MS, MGL were more accurate in discriminating MS subtypes (relapsing-remitting from secondary progressive), providing evidence about the underlying mechanisms of WM deficit in progressive states of the disease. Furthermore, computational modeling of the results from the sequential paradigm determined that imprecision in decoding information and swap error (mistakenly reporting the feature of other presented items) was responsible for WM dysfunction in MS. Overall, this study offered a sensitive measure for assessing WM deficit and provided new insight into the organization of the WM system in MS population.

A mechanistic insight into sources of error of visual working memory in multiple sclerosis.

Working memory (WM) is one of the most affected cognitive domains in multiple sclerosis (MS), which is mainly studied by the previously established binary model for information storage (slot model). However, recent observations based on the continuous reproduction paradigms have shown that assuming dynamic allocation of WM resources (resource model) instead of the binary hypothesis will give more accurate predictions in WM assessment. Moreover, continuous reproduction paradigms allow for assessing the distribution of error in recalling information, providing new insights into the organization of the WM system. Hence, by utilizing two continuous reproduction paradigms, memory-guided localization (MGL) and analog recall task with sequential presentation, we investigated WM dysfunction in MS. Our results demonstrated an overall increase in recall error and decreased recall precision in MS. While sequential paradigms were better in distinguishing healthy control from relapsing-remitting MS, MGL were more accurate in discriminating MS subtypes (relapsing-remitting from secondary progressive), providing evidence about the underlying mechanisms of WM deficit in progressive states of the disease. Furthermore, computational modeling of the results from the sequential paradigm determined that imprecision in decoding information and swap error (mistakenly reporting the feature of other presented items) was responsible for WM dysfunction in MS. Overall, this study offered a sensitive measure for assessing WM deficit and provided new insight into the organization of the WM system in MS population.

The effect of multisensory semantic congruency on unisensory object recognition in schizophrenia.

Multisensory, as opposed to unisensory processing of stimuli, has been found to enhance the performance (e.g., reaction time, accuracy, and discrimination) of healthy individuals across various tasks. However, this enhancement is not as pronounced in patients with schizophrenia (SZ), indicating impaired multisensory integration (MSI) in these individuals. To the best of our knowledge, no study has yet investigated the impact of MSI deficits in the context of working memory, a domain highly reliant on multisensory processing and substantially impaired in schizophrenia. To address this research gap, we employed two adopted versions of the continuous object recognition task to investigate the effect of single-trail multisensory encoding on subsequent object recognition in 21 schizophrenia patients and 21 healthy controls (HC). Participants were tasked with discriminating between initial and repeated presentations. For the initial presentations, half of the stimuli were audiovisual pairings, while the other half were presented unimodal. The task-relevant stimuli were then presented a second time in a unisensory manner (either auditory stimuli in the auditory task or visual stimuli in the visual task). To explore the impact of semantic context on multisensory encoding, half of the audiovisual pairings were selected to be semantically congruent, while the remaining pairs were not semantically related to each other. Consistent with prior studies, our findings demonstrated that the impact of single-trial multisensory presentation during encoding remains discernible during subsequent object recognition. This influence could be distinguished based on the semantic congruity between the auditory and visual stimuli presented during the encoding. This effect was more robust in the auditory task. In the auditory task, when congruent multisensory pairings were encoded, both participant groups demonstrated a multisensory facilitation effect. This effect resulted in improved accuracy and RT performance. Regarding incongruent audiovisual encoding, as expected, HC did not demonstrate an evident multisensory facilitation effect on memory performance. In contrast, SZs exhibited an atypically accelerated reaction time during the subsequent auditory object recognition. Based on the predictive coding model we propose that this observed deviations indicate a reduced semantic modulatory effect and anomalous predictive errors signaling, particularly in the context of conflicting cross-modal sensory inputs in SZ.

Beta oscillation is an indicator for two patterns of sensorimotor synchronization.

Previous study indicates that there are two distinct behavioral patterns in the sensory-motor synchronization task with short stimulus onset asynchrony (SOA; 2-3 s) or long SOA (beyond 4 s). However, the underlying neural indicators and mechanisms have not been elucidated. The present study applied magnetoencephalography (MEG) technology to examine the functional role of several oscillations (beta, gamma, and mu) in sensorimotor synchronization with different SOAs to identify a reliable neural indicator. During MEG recording, participants underwent a listening task without motor response, a sound-motor synchronization task, and a motor-only continuation task. These tasks were used to explore whether and how the activity of oscillations changes across different behavioral patterns with different tempos. Results showed that during both the listening and the synchronization task, the beta oscillation changes with the tempo. Moreover, the event-related synchronization of beta oscillations was significantly correlated with motor timing during synchronization. In contrast, mu activity only changes with the tempo in the synchronization task, while the gamma activity remains unchanged. In summary, the current study indicates that beta oscillation could be an indicator of behavioral patterns between fast tempo and slow tempo in sensorimotor synchronization. Also, it is likely to be the potential mechanism of maintaining rhythmic continuous movements with short SOA, which is embedded within the 3 s time window.

Adaptive bias-variance trade-off in advantage estimator for actor–critic algorithms

Actor–critic methods are leading in many challenging continuous control tasks. Advantage estimators, the most common critics in the actor–critic framework, combine state values from bootstrapping value functions and sample returns. Different combinations balance the bias introduced by state values and the variance returned by samples to reduce estimation errors. The bias and variance constantly fluctuate throughout training, leading to different optimal combinations. However, existing advantage estimators usually use fixed combinations that fail to account for the trade-off between minimizing bias and variance to find the optimal estimate. Our previous work on adaptive advantage estimation (AAE) analyzed the sources of bias and variance and offered two indicators. This paper further explores the relationship between the indicators and their optimal combination through typical numerical experiments. These analyses develop a general form of adaptive combinations of state values and sample returns to achieve low estimation errors. Empirical results on simulated robotic locomotion tasks show that our proposed estimators achieve similar or superior performance compared to previous generalized advantage estimators (GAE).

Elastic Wave Mechanics in Damaged Metallic Plates

Human health monitoring (HHM) is essential for continued daily task execution, as is structural health monitoring (SHM) for structures to ensure the continual performance of their designed tasks with optimal efficiency. The existence of damage in a structure affects its optimal use through stiffness deterioration. Damage of different forms could occur in a structure but have the singular objective of material degradation, leading to its underuse for a task. Guided wave ultrasonics has shown strength in detecting sundry damage in structures, but most of the damage monitored and detected is unfilled with substances. However, some damage could trap and accumulate substances that could hasten material degradation through corrosion activities under favorable conditions, especially in the oil and gas industry. This study used the ultrasonic-guided waves’ pitch–catch inspection technique to identify damage filled with different materials. The assessment was based on the RMSD of the dominant Lamb wave mode’s average maximum amplitude and the response signals’ transmission coefficient (TC). A five-cycle tone burst of excitation signals of different frequencies was created to generate propagating Lamb waves in the structure. The fundamental antisymmetric mode was found to be more sensitive than the fundamental symmetric mode when detecting damage filled with various substances. At 80 kHz, the deviation of the current response signals from the baseline response signals due to different filled substances in the damage was distinct and decreased with increased fluid viscosity. Given that structures in the oil and gas sector are particularly susceptible to substance-induced damage, the outcomes of this study are paramount.

Learning inverse kinematics using neural computational primitives on neuromorphic hardware

Current low-latency neuromorphic processing systems hold great potential for developing autonomous artificial agents. However, the variable nature and low precision of the underlying hardware substrate pose severe challenges for robust and reliable performance. To address these challenges, we adopt hardware-friendly processing strategies based on brain-inspired computational primitives, such as triplet spike-timing dependent plasticity, basal ganglia-inspired disinhibition, and cooperative-competitive networks and apply them to motor control. We demonstrate this approach by presenting an example of robust online motor control using a hardware spiking neural network implemented on a mixed-signal neuromorphic processor, trained to learn the inverse kinematics of a two-joint robotic arm. The final system is able to perform low-latency control robustly and reliably using noisy silicon neurons. The spiking neural network, trained to control two joints of the iCub robot arm simulator, performs a continuous target-reaching task with 97.93% accuracy, 33.96 ms network latency, 102.1 ms system latency, and with an estimated power consumption of 26.92 μW during inference (control). This work provides insights into how specific computational primitives used by real neural systems can be applied to neuromorphic computing for solving real-world engineering tasks. It represents a milestone in the design of end-to-end spiking robotic control systems, relying on event-driven sensory encoding, neuromorphic processing, and spiking motor control.

Quantifying decision-making in dynamic, continuously evolving environments.

During perceptual decision-making tasks, centroparietal electroencephalographic (EEG) potentials report an evidence accumulation-to-bound process that is time locked to trial onset. However, decisions in real-world environments are rarely confined to discrete trials; they instead unfold continuously, with accumulation of time-varying evidence being recency-weighted towards its immediate past. The neural mechanisms supporting recency-weighted continuous decision-making remain unclear. Here, we use a novel continuous task design to study how the centroparietal positivity (CPP) adapts to different environments that place different constraints on evidence accumulation. We show that adaptations in evidence weighting to these different environments are reflected in changes in the CPP. The CPP becomes more sensitive to fluctuations in sensory evidence when large shifts in evidence are less frequent, and the potential is primarily sensitive to fluctuations in decision-relevant (not decision-irrelevant) sensory input. A complementary triphasic component over occipito-parietal cortex encodes the sum of recently accumulated sensory evidence, and its magnitude covaries with parameters describing how different individuals integrate sensory evidence over time. A computational model based on leaky evidence accumulation suggests that these findings can be accounted for by a shift in decision threshold between different environments, which is also reflected in the magnitude of pre-decision EEG activity. Our findings reveal how adaptations in EEG responses reflect flexibility in evidence accumulation to the statistics of dynamic sensory environments.

Lexical Alignment Effect of the Continuation Task on Interpreting Trainees’ English-to-Chinese Sight Translation Fluency

The present study explored the alignment effect of the sight translation continuation task (STCT) and its possible influence on sight translation (ST) fluency. Thirty-four third-year English Education majors at a Chinese university were divided into two groups in the study. The experimental group who conducted the STCT read the English source text of a speech and its translation in Chinese, while the control group only read the English source text. Afterwards, both groups sight translated the continued source text into Chinese. The results indicated that (a) the experimental group aligned with the pre-reading text at the lexical level and (b) the continuation task improved ST fluency to some extent as the experimental group produced significantly fewer self-repairs in their ST products. The study concludes by suggesting that the continuation task can be useful in ST instruction and, hence, should be more visible in the interpreting classroom.

Effects of input modality on alignment in continuation writing

Previous studies demonstrated that alignment entailed in the continuation task can facilitate language learning (e.g., Wang & Wang, 2015), but the investigation into how to maximize alignment to tap greater learning potential is still in its infancy. The present study extends this line of inquiry by examining whether interaction with multimodal presentations would bring about enhanced alignment effect in the reading-writing continuation task. Participants were three groups of Chinese senior English as a foreign language (EFL) learners who were exposed to each of the three input modalities: text modality, text+picture modality and text+video modality. Results showed that 1) text+video modality brought about stronger alignment at both linguistic and situational levels, more accurate and cohesive writing production; 2) text+picture modality only magnified alignment at situation models; 3) learners varied their perceptions towards the incorporation of pictorial and video input. Pedagogical implications of the study were thereafter discussed.

Visual Feedback Decoding During Bimanual Circle Drawing.

The between-hand interference during bimanual tasks is a consequence of the connection between the neural controllers of movement. Previous studies showed the existence of an asymmetric between-hand interference (caused by neural crosstalk) when different kinematics plans were to be executed by each hand or when only one was visually guided and received perturbed visual feedback. Here, in continuous bimanual circle drawing tasks, we investigated if the central nervous system (CNS) can benefit from visual composite feedback, i.e., a weighted sum of hands' positions presented for the visually-guided hand, to control the non-visible hand. Our results demonstrated improvement in the non-visible non-dominant hand (NDH) performance in the presence of the composite feedback. When NDH was visually guided, the dominant hand's (DH) performance during asymmetric drawing deteriorated, while its performance during symmetric drawing improved. This indicates that the CNS's ability to leverage composite feedback, which can be the result of decoding the non-visible hand positional information from the composite feedback, is task-dependent and can be asymmetric. Also, the non-visible hand's performance degraded when DH or NDH was visually guided with amplified error feedback. The results of the amplified feedback condition do not strongly support the asymmetry of the interference during asymmetric circle drawing. Comparing muscle activations in the asymmetric experiment, we concluded that the observed kinematic differences were not due to alternation in muscle co-contractions.

Reliability of on-line visual feedback influences learning of continuous motor task of healthy young adults.

A continuous task was used to determine how the reliability of on-line visual feedback during acquisition impacts motor learning. Participants performed a right hand pointing task of a repeated sequence with a visual cursor that was either reliable, moderately unreliable, or largely unreliable. Delayed retention tests were administered 24 h later, as well as intermanual transfer tests (performed with the left hand). A visuospatial transfer test was performed with the same targets' sequence (same visuospatial configuration) while a motor transfer test was performed with the visual mirror of the targets' sequence (same motor patterns). Results showed that pointing was slower and long-term learning disrupted in the largely unreliable visual cursor condition, compared with the reliable and moderately unreliable conditions. Also, analysis of transfers revealed classically better performance on visuospatial transfer than on motor transfer for the reliable condition. However, here we first show that such difference disappears when the cursor was moderately or largely unreliable. Interestingly, these results indicated a difference in the type of sequence coding, depending on the reliability of the on-line visual feedback. This recourse to mixed coding opens up interesting perspectives, as it is known to promote better learning of motor sequences.

Integrative Study of Continuation Task in High School based on Themetic Progression

The writing component of college entrance exams now includes a brand-new form of test called a continuation task. In high school, a well-defined form of thematic growth is a midwife to improving its coherence, articulation, and logic. The narrative essay's mode of thematic progression in this study is examined using the examples “My Amazing Naadam Experience” from Unit 1 Festival and Celebration, “Li Lan’s Travel Journal” from Unit 3 Diverse Culture, in PEP Compulsory Three, and “Seeing the True North via Rail: Vancouver and the Heart of Canada” from Unit 4 Journey Across, in Selective Compulsory Two. The concept and tips for the test question “Continuation Task” from the New Curriculum II Paper 2022 were examined in this study, using it as an example. The “six elements and four patterns” technique is offered, along with the evaluation criteria, hoping that this can give front-line teachers a reference for teaching the continuing assignment.

Diffusion theory of the antipodal "shadow" mode in continuous-outcome, coherent-motion decisions.

Continuous-outcome decisions, in which responses are made on continuous scales, are increasingly used to study perception and memory for stimulus attributes like color, orientation, and motion. This interest has led to the development of models of continuous-outcome decision processes like the circular diffusion model that predict joint distributions of decision outcomes and response times (RTs). We use the circular diffusion model and a new spherical generalization of it to model performance in a continuous-outcome version of the random-dot motion task. The task is a benchmark test of decision models because it yields bimodal distributions of decision outcomes: In addition to a peak or mode in the true direction of motion, there is a secondary, antipodal, mode at 180° to the true direction. Models like the circular diffusion model, in which evidence is accumulated by a single process, are thought to be unable to predict bimodality. We compared diffusion models for the continuous motion task in which evidence is accumulated in either a two-dimensional (2D) or a three-dimensional (3D) representational space. We found that performance was well described by a spherical (3D) diffusion model in which the drift rate encodes perceived motion direction and strength and the points on the bounding sphere representing the decision criterion are projected onto a 2D circle to make a response. A model with an antipodal component of drift rate and drift-rate variability successfully predicted bimodal distributions of decision outcomes and the joint distributions of decision outcomes and RT for individual participants. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Pinning down the interaction between animacy and syntactic function in the interpretation of German and Italian personal and demonstrative pronouns

ABSTRACT This article investigates how animacy in interaction with the syntactic function of a referent’s antecedent determines the interpretation of different types of pronouns and demonstratives in German and Italian. The results of a sentence continuation task conducted in both languages show that Italian null pronouns and German p-pronouns have a strong tendency to refer to a preceding subject, but only if its referent is animate. With inanimate subjects, both forms tend to refer to the animate referent in object position, showing that animacy enhances a referent’s accessibility more than the syntactic function of its previous mention. Demonstratives in German and Italian generally tend to refer to object antecedents, a tendency that is also influenced by the animacy of the referent, especially in German. The cross-linguistic comparison reveals that the effect of animacy is overall stronger in German than in Italian, suggesting that across languages, different forms may show a different sensitivity to syntactic function and animacy.