Sort by
Proximal Policy Optimization With Time-Varying Muscle Synergy for the Control of an Upper Limb Musculoskeletal System

Because of their unique adaptability, flexibility, and robustness, musculoskeletal robotic systems are regarded potentially as next-generation robots. However, motion learning and generation of such a robotic system are still challenging. This paper presents a neuromuscular control method, namely, TMS-PPO, based on time-varying muscle synergy (TMS) and proximal policy optimization (PPO). The electromyogram (EMG) activation signals of actual human motions are decomposed to obtain TMSs based on the temporal properties of the TMS. The weights of networks are trained to generate the scale and phase coefficients through the PPO. The coefficients modulate the TMSs to generate appropriate activation patterns to optimize motion learning of the musculoskeletal system. To verify the effectiveness of the proposed method, the TMSs are extracted from human upper limb muscle activation signals, and we compare TMS-PPO with PPO in the motion learning and generation process of an upper limb musculoskeletal system. The results show that TMS-PPO can complete the control tasks because the average errors of the joints are less than 0.05 rad. In the meantime, TMSs are used as motion primitives of the musculoskeletal system to simulate the process of the human CNS controlling muscles. It shows that TMS-PPO reduces the energy consumption and improves the learning rate significantly compared with the PPO. The learning episodes reduce from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">\(10^4\)</tex-math> </inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">\(10^3\)</tex-math> </inline-formula> , which indicates that TMS-PPO has a stronger learning ability and better physiological explanation. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Due to the superiorities of the musculoskeletal system, humanoid robots that imitate human driven mechanisms are vigorously carried out worldwide. Taking advantages of human-like characteristics, the musculoskeletal robot provides new opportunities to understand and validate the human mechanisms of muscle control and motion learning, to compare the performance of the robot to that of humans as well as work in real world, e.g., human interactive robots, amusement robots and medical training robots in the future. However, strong redundancy, coupling, and nonlinearity of the system also raises many challenges for the investigation of the control problem. Inspired by how the human CNS controls a musculoskeletal system and realize motion generalization, a novel muscle-synergies-based neuromuscular control that combines time-varying muscle synergy (TMS) and Proximal Policy Optimization (PPO), namely, TMS-PPO is proposed in this paper. The learning efficiency of PPO and the physiological interpretation of the control process are improved during the motion learning and generation processes of the musculoskeletal system. Preliminary simulation experiments suggest that this method is feasible in terms of control accuracy and efficiency. Moreover, the performance of the TMS-PPO is comparable to the PPO without significant improvement. To solve this problem, in future work, we will introduce the cerebellar model into the control method which plays the role of adjusting and correcting the motions of the limbs to achieve accurate and stable control in the actions process of humans.

Just Published
Relevant
ASSOCIATION OF INTRAVENOUS FLUORESCEIN ANGIOGRAPHY AND ADAPTIVE OPTICS IMAGING IN DIABETIC RETINOPATHY: A Prospective Case Series.

To our knowledge, we present the first case series investigating the relationship between adaptive optics (AO) imaging and intravenous fluorescein angiography (IVFA) parameters in patients with diabetic retinopathy. Consecutive patients with diabetic retinopathy older than age 18 years presenting to a single center in Toronto, Canada, from 2020 to 2021 were recruited. Adaptive optics was performed with the RTX1 camera (Imagine Eyes, Orsay, France) at retinal eccentricities of 2° and 4°. Intravenous fluorescein angiography was assessed with the artificial intelligence-based RETICAD system to extract blood flow, perfusion, and blood-retinal-barrier (BRB) permeability at the same retinal locations. Correlations between AO and IVFA parameters were calculated using Pearson's correlation coefficient. Across nine cases, a significant positive correlation existed between photoreceptor spacing on AO and BRB permeability (r = 0.303, P = 0.027), as well as perfusion (r = 0.272, P = 0.049) on IVFA. When stratified by location, a significant positive correlation between photoreceptor dispersion and both BRB permeability and perfusion (r = 0.770, P = 0.043; r = 0.846, P = 0.034, respectively) was observed. Cone density was also negatively correlated with BRB permeability (r = -0.819, P = 0.046). Photoreceptor spacing on AO was significantly correlated with BRB permeability and perfusion on IVFA in patients with diabetic retinopathy. Future studies with larger sample sizes are needed to understand the relationship between AO and IVFA parameters in diverse patient populations.

Just Published
Relevant
Optimal Tracking Control of Heterogeneous MASs Using Event-Driven Adaptive Observer and Reinforcement Learning.

This article considers the output tracking control problem of nonidentical linear multiagent systems (MASs) using a model-free reinforcement learning (RL) algorithm, where partial followers have no prior knowledge of the leader's information. To lower the communication and computing burden among agents, an event-driven adaptive distributed observer is proposed to predict the leader's system matrix and state, which consists of the estimated value of relative states governed by an edge-based predictor. Meanwhile, the integral input-based triggering condition is exploited to decide whether to transmit its private control input to its neighbors. Then, an RL-based state feedback controller for each agent is developed to solve the output tracking control problem, which is further converted into the optimal control problem by introducing a discounted performance function. Inhomogeneous algebraic Riccati equations (AREs) are derived to obtain the optimal solution of AREs. An off-policy RL algorithm is used to learn the solution of inhomogeneous AREs online without requiring any knowledge of the system dynamics. Rigorous analysis shows that under the proposed event-driven adaptive observer mechanism and RL algorithm, all followers are able to synchronize the leader's output asymptotically. Finally, a numerical simulation is demonstrated to verify the proposed approach in theory.

Just Published
Relevant
Movi: a fast and cache-efficient full-text pangenome index.

Efficient pangenome indexes are promising tools for many applications, including rapid classification of nanopore sequencing reads. Recently, a compressed-index data structure called the "move structure" was proposed as an alternative to other BWT-based indexes like the FM index and r-index. The move structure uniquely achieves both O(r) space and O(1)-time queries, where r is the number of runs in the pangenome BWT. We implemented Movi, an efficient tool for building and querying move-structure pangenome indexes. While the size of the Movi's index is larger than the r-index, it scales at a smaller rate for pangenome references, as its size is exactly proportional to r, the number of runs in the BWT of the reference. Movi can compute sophisticated matching queries needed for classification - such as pseudo-matching lengths and backward search - at least ten times faster than the fastest available methods, and in some cases more than 30-fold faster. Movi achieves this speed by leveraging the move structure's strong locality of reference, incurring close to the minimum possible number of cache misses for queries against large pangenomes. We achieve still further speed improvements by using memory prefetching to attain a degree of latency hiding that would be difficult with other index structures like the r-index. Movi's fast constant-time query loop makes it well suited to real-time applications like adaptive sampling for nanopore sequencing, where decisions must be made in a small and predictable time interval.

Open Access
Relevant
Cortical Plasticity is associated with Blood-Brain-Barrier Modulation

Brain microvessels possess the unique properties of a blood-brain barrier (BBB), tightly regulating the passage of molecules from the blood to the brain neuropil and vice versa. In models of brain injury, BBB dysfunction and the associated leakage of serum albumin to the neuropil have been shown to induce pathological plasticity, neuronal hyper-excitability, and seizures. The effect of neuronal activity on BBB function and whether it plays a role in plasticity in the healthy brain remain unclear. Here we show that neuronal activity induces modulation of microvascular permeability in the healthy brain and that it has a role in local network reorganization. Combining simultaneous electrophysiological recording and vascular imaging with transcriptomic analysis in rats, and functional and BBB-mapping MRI in human subjects we show that prolonged stimulation of the limb induces a focal increase in BBB permeability in the corresponding somatosensory cortex that is associated with long-term synaptic plasticity. We further show that the increased microvascular permeability depends on neuronal activity and involves caveolae-mediated transcytosis and transforming growth factor beta signaling. Our results reveal a role of BBB modulation in cortical plasticity in the healthy brain, highlighting the importance of neurovascular interactions for sensory experience and learning.

Open Access
Relevant
Oral health care and living environment for older people: a scoping review protocol.

This scoping review will map the literature on the extent of knowledge on living environment oral health care (LIVEOR) by examining its different terminologies, its description, the stakeholders involved, the implementation characteristics, and the outcomes reported by the authors. Older people want to receive oral health care in their current living environment. Although several authors have explored some aspects of LIVEOR for older people, there are still inconsistent findings regarding the extent of this model of care. This scoping review will include quantitative, qualitative, and mixed method studies, as well as any type of knowledge synthesis on LIVEOR involving people aged 60 years and over. The search will not be limited by language, time frame, geographic location, or publication date. We will use the JBI methodology for scoping reviews and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR). The search will include MEDLINE, CINAHL, Embase, Web of Science, and the Cochrane Library. A hand-search of the references of the included studies, and a gray literature search will also be conducted. Two independent reviewers will screen titles, abstracts, and full texts of selected studies, and perform data extraction. Findings are expected to explore what is known of LIVEOR targeting older people and to identify any knowledge gaps for future studies. We will disseminate our findings mostly through peer-reviewed publications. Open Science Framework https://osf.io/e7fm2.

Open Access
Relevant