Robotic Lower Limb Prosthesis Research Articles

Advancements and Challenges in the Development of Robotic Lower Limb Prostheses: A Systematic Review

Advancements and Challenges in the Development of Robotic Lower Limb Prostheses: A Systematic Review

Swing-phase detection of locomotive mode transitions for smooth multi-functional robotic lower-limb prosthesis control.

Robotic lower-limb prostheses, with their actively powered joints, may significantly improve amputee users' mobility and enable them to obtain healthy-like gait in various modes of locomotion in daily life. However, timely recognition of the amputee users' locomotive mode and mode transition still remains a major challenge in robotic lower-limb prosthesis control. In the paper, the authors present a new multi-dimensional dynamic time warping (mDTW)-based intent recognizer to provide high-accuracy recognition of the locomotion mode/mode transition sufficiently early in the swing phase, such that the prosthesis' joint-level motion controller can operate in the correct locomotive mode and assist the user to complete the desired (and often power-demanding) motion in the stance phase. To support the intent recognizer development, the authors conducted a multi-modal gait data collection study to obtain the related sensor signal data in various modes of locomotion. The collected data were then segmented into individual cycles, generating the templates used in the mDTW classifier. Considering the large number of sensor signals available, we conducted feature selection to identify the most useful sensor signals as the input to the mDTW classifier. We also augmented the standard mDTW algorithm with a voting mechanism to make full use of the data generated from the multiple subjects. To validate the proposed intent recognizer, we characterized its performance using the data cumulated at different percentages of progression into the gait cycle (starting from the beginning of the swing phase). It was shown that the mDTW classifier was able to recognize three locomotive mode/mode transitions (walking, walking to stair climbing, and walking to stair descending) with 99.08% accuracy at 30% progression into the gait cycle, well before the stance phase starts. With its high performance, low computational load, and easy personalization (through individual template generation), the proposed mDTW intent recognizer may become a highly useful building block of a prosthesis control system to facilitate the robotic prostheses' real-world use among lower-limb amputees.

Neural prosthesis control restores near-normative neuromechanics in standing postural control.

Current lower-limb prostheses do not provide active assistance in postural control tasks to maintain the user's balance, particularly in situations of perturbation. In this study, we aimed to address this missing function by enabling neural control of robotic lower-limb prostheses. Specifically, electromyographic (EMG) signals (amplified neural control signals) recorded from antagonistic residual ankle muscles were used to drive a robotic prosthetic ankle directly and continuously. Participants with transtibial amputation were recruited and trained in using the EMG-driven robotic ankle. We studied how using the EMG-controlled ankle affected the participants' anticipatory and compensatory postural control strategies and stability under expected perturbations compared with using their daily passive devices. We investigated the similarity of neuromuscular coordination (by analyzing motor modules) of the participants, using either device in a postural sway task, to that of able-bodied controls. Results showed that, compared with their passive prosthesis, the EMG-controlled prosthesis enabled participants to use near-normative postural control strategies, as evidenced by improved between-limb symmetry in intact-prosthetic center-of-pressure and joint angle excursions. Participants substantially improved postural stability, as evidenced by a reduction in steps or falls using the EMG-controlled prosthetic ankle. Furthermore, after relearning to use residual ankle muscles to drive the robotic ankle in postural control, nearly all participants' motor module structure shifted toward that observed in individuals without limb amputations. Here, we have demonstrated the potential benefit of direct EMG control of robotic lower limb prostheses to restore normative postural control strategies (both neural and biomechanical) toward enhancing standing postural stability in amputee users.

Vision-Based Recognition of Human Motion Intent during Staircase Approaching.

Walking in real-world environments involves constant decision-making, e.g., when approaching a staircase, an individual decides whether to engage (climbing the stairs) or avoid. For the control of assistive robots (e.g., robotic lower-limb prostheses), recognizing such motion intent is an important but challenging task, primarily due to the lack of available information. This paper presents a novel vision-based method to recognize an individual's motion intent when approaching a staircase before the potential transition of motion mode (walking to stair climbing) occurs. Leveraging the egocentric images from a head-mounted camera, the authors trained a YOLOv5 object detection model to detect staircases. Subsequently, an AdaBoost and gradient boost (GB) classifier was developed to recognize the individual's intention of engaging or avoiding the upcoming stairway. This novel method has been demonstrated to provide reliable (97.69%) recognition at least 2 steps before the potential mode transition, which is expected to provide ample time for the controller mode transition in an assistive robot in real-world use.

Learning Ergonomic Control in Human–Robot Symbiotic Walking

This article presents an imitation learning strategy for extracting ergonomically safe control policies in physical human–robot interaction scenarios. The presented approach seeks to proactively reduce the risk of injuries and musculoskeletal disorders by anticipating the ergonomic effects of a robot's actions on a human partner, e.g., how the ankle angle of a prosthesis affects future knee torques of the user. To this end, we extend ensemble Bayesian interaction primitives to enable the prediction of latent biomechanical variables. This methodology yields a reactive control strategy, which we evaluate in an assisted walking task with a robotic lower limb prosthesis. Building upon the learned interaction primitives, we also present a model-predictive control (MPC) strategy that actively steers the human–robot interaction toward ergonomic and safe movement regimes. We compare the introduced control strategies and highlight the framework's ability to generate ergonomic, biomechanically safe assistive prosthetic control. A rich analysis of constrained MPC shows a 20× reduction in the effects of large perturbations on prosthetic control system. We empirically demonstrate a 16% reduction in vertical knee reaction forces in real-world jumping experiments utilizing our control methodology and examine other optimal control strategies in simulated walking experiments.

Force-Moment Sensor for Prosthesis Structural Load Measurement

Measurement of prosthesis structural load, as an important way to quantify the interaction of the amputee user with the environment, may serve important purposes in the control of smart lower-limb prosthetic devices. However, the majority of existing force sensors used in protheses are developed based on strain measurement and thus may suffer from multiple issues such as weak signals and signal drifting. To address these limitations, this paper presents a novel Force-Moment Prosthesis Load Sensor (FM-PLS) to measure the axial force and bending moment in the structure of a lower-limb prosthesis. Unlike strain gauge-based force sensors, the FM-PLS is developed based on the magnetic sensing of small (millimeter-scale) deflection of an elastic element, and it may provide stronger signals that are more robust against interferences and drifting since such physical deflection is several orders of magnitude greater than the strain of a typical load-bearing structure. The design of the sensor incorporates uniquely curved supporting surfaces such that the measurement is sensitive to light load but the sensor structure is robust enough to withstand heavy load without damage. To validate the sensor performance, benchtop testing of the FM-PLS and walking experiments of a FM-PLS-embedded robotic lower-limb prosthesis were conducted. Benchtop testing results displayed good linearity and a good match to the numerical simulation results. Results from the prosthesis walking experiments showed that the sensor signals can be used to detect important gaits events such as heel strike and toe-off, facilitating the reliable motion control of lower-limb prostheses.

A Piecewise Monotonic Gait Phase Estimation Model for Controlling a Powered Transfemoral Prosthesis in Various Locomotion Modes

Gait phase-based control is a trending research topic for walking-aid robots, especially robotic lower-limb prostheses. Gait phase estimation is a challenge for gait phase-based control. Previous researches used the integration or the differential of the human's thigh angle to estimate the gait phase, but accumulative measurement errors and noises can affect the estimation results. In this letter, a more robust gait phase estimation method is proposed using a unified form of piecewise monotonic gait phase-thigh angle models for various locomotion modes. The gait phase is estimated from only the thigh angle, which is a stable variable and avoids phase drifting. A Kalman filter-based smoother is designed to further suppress the mutations of the estimated gait phase. Based on the proposed gait phase estimation method, a gait phase-based joint angle tracking controller is designed for a transfemoral prosthesis. The proposed gait estimation method, the gait phase smoother, and the controller are evaluated through offline analysis on walking data in various locomotion modes. And the real-time performance of the gait phase-based controller is validated in an experiment on the transfemoral prosthesis.

A Multimodal Sensory Apparatus for Robotic Prosthetic Feet Combining Optoelectronic Pressure Transducers and IMU

Timely and reliable identification of control phases is functional to the control of a powered robotic lower-limb prosthesis. This study presents a commercial energy-store-and-release foot prosthesis instrumented with a multimodal sensory system comprising optoelectronic pressure sensors (PS) and IMU. The performance was verified with eight healthy participants, comparing signals processed by two different algorithms, based on PS and IMU, respectively, for real-time detection of heel strike (HS) and toe-off (TO) events and an estimate of relevant biomechanical variables such as vertical ground reaction force () and center of pressure along the sagittal axis (). The performance of both algorithms was benchmarked against a force platform and a marker-based stereophotogrammetric motion capture system. HS and TO were estimated with a time error lower than 0.100 s for both the algorithms, sufficient for the control of a lower-limb robotic prosthesis. Finally, the computed from the PS showed a Pearson correlation coefficient of 0.97 (0.02) with the same variable computed through the force platform.

Small-Data-Driven Temporal Convolutional Capsule Network for Locomotion Mode Recognition of Robotic Prostheses.

Locomotion mode recognition has been shown to substantially contribute to the precise control of robotic lower-limb prostheses under different walking conditions. In this study, we proposed a temporal convolutional capsule network (TCCN) which integrates the spatial-temporal-based, dilation-convolution-based, dyna- mic routing and vector-based features for recognizing locomotion mode recognition with small data rather than big-data-based neural networks for robotic prostheses. TCCN proposed in this study has four characteristics, which extracts the (1) spatial-temporal information in the data and then makes (2) dilated convolution to deal with small data, and uses (3) dynamic routing, which produces some similarities to the human brain to process the data as a (4) vector, which is different from other scalar-based networks, such as convolutional neural network (CNN). By comparison with a traditional machine learning, e.g., support vector machine(SVM) and big-data-driven neural networks, e.g., CNN, recurrent neural network(RNN), temporal convolutional network(TCN) and capsule network(CN). The accuracy of TCCN is 4.1% higher than CNN under 5-fold cross-validation of three-locomotion-mode and 5.2% higher under the 5-fold cross-validation of five-locomotion modes. The main confusion we found appears in the transition state. The results indicate that TCCN may handle small data balancing global and local information which is closer to the way how the human brain works, and the capsule layer allows for better processing vector information and retains not only magnitude information, but also direction information.

Human Prediction and Robotic Lower-Limb Prosthesis Planning for Safe Perturbation Recovery During Motion

Human Prediction and Robotic Lower-Limb Prosthesis Planning for Safe Perturbation Recovery During Motion

A Classification Approach Based on Directed Acyclic Graph to Predict Locomotion Activities With One Inertial Sensor on the Thigh

Current state-of-the-art locomotion mode classifiers for controlling robotic lower-limb prostheses rely on multiple sensors to achieve high accuracy, prediction performance, and robustness to both speed changes and subject-specific gait patterns. However, multiple sensors placed on different body parts usually entail discomfort and poor usability for the user. This paper presents an intention detection method that relies on the features extracted from an inertial measurement unit worn on the thigh and an online phase estimator. The algorithm classifies the locomotion mode of the upcoming stride among the three modes of ground-level walking, stair ascent, and stair descent. A two-stage classification process first distinguishes between transient and steady-state strides and then classifies the locomotion mode of the impending stride based on directed acyclic graphs of binary classifiers. The classification is performed at 75% or 85% of the previous stride phase, respectively for steady-state and transient strides. Data were gathered from 10 healthy subjects and processed offline. Feature design and selection were based on the data of all subjects, while the classification performance was assessed by leave-one-subject-out cross-validation. Results presented a median recognition accuracy of 98.7% for steady-state strides and 95.6% for transitions, suggesting that the method was inherently robust to variations in gait cadence, since all of the features were phase-based and not dependent on fixed time intervals. These results inform the design of control strategies for active transfemoral prostheses able to predict the user’s locomotion intention during the next stride, using minimum sensors.

КОНЦЕПТУАЛЬНОЕ ПРОЕКТИРОВАНИЕ МЕХАТРОННОГО СТЕНДА ДЛЯ ПРЕДКЛИНИЧЕСКИХ ИСПЫТАНИЙ РОБОТИЗИРОВАННЫХ ПРОТЕЗОВ НИЖНИХ КОНЕЧНОСТЕЙ

Over the past several years, the first real prototypes of robotic prostheses have been developed in Russia, but they still lose competition to the best foreign designs and are practically not in demand in the domestic market of assistive devices. One of the reasons preventing this is the absence in Russia of stands for certification tests of robotic prostheses. In addition, the use of such stands at the design stage would significantly simplify the solution of the problem of synthesizing high-quality control systems and would not require the participation of disabled volunteers for their testing and adjustment. However, there are currently no analogues of such test benches in Russia. This paper describes a conceptual version of a mechatronic stand-simulator capable to simulate biosimilar movements of a human lower limb in various modes of physical activity under variable environmental parameters in order to adjust parameters, laboratory and preclinical tests of structures and control systems of active lower limbs prostheses and their modules. The stand can be easily upgraded for certification testing of robotic lower limb prostheses.

Dynamic optimization of Gait with a Generalized Lower-Limb Prosthesis Model.

Predictive simulation of gait is a promising tool for robotic lower limb prosthesis design, but has been limited in its application to models of existing design types. We propose a modeling approach to find optimal prosthesis dynamics in gait simulations without constraining the prosthesis to follow kinematics allowed by a specific joint mechanism. To accomplish this, we render a transtibial prosthetic device as the composition of its resultant forces and moments as they act upon the prosthetic foot and socket and allow3 degree-of-freedom planar motion. The model is implemented into a human musculoskeletal model and used to solve dynamic optimizations of muscle and prosthesis controls to minimize muscle effort and loading on the residual limb during walking. The emphasis on muscle effort vs. limb loading is varied in the minimization objective and the resulting optimal prosthesis dynamics are compared. We found that muscle effort and socket loading measures were reduced for our prosthesis model compared to a revolute joint prosthesis model. We interpret large displacements in the linear axes to transfer energy to the plantarflexion action before toe-off and reduce loading at the socket-limb interface. Our results suggest this approach could assist in the design of non-biomimetic prostheses but requires experimental validation to assess our modeling assumptions, as well as progress toward increased fidelity of predictive simulation approaches more generally.

Preliminary Design of an Environment Recognition System for Controlling Robotic Lower-Limb Prostheses and Exoskeletons.

Drawing inspiration from autonomous vehicles, using future environment information could improve the control of wearable biomechatronic devices for assisting human locomotion. To the authors knowledge, this research represents the first documented investigation using machine vision and deep convolutional neural networks for environment recognition to support the predictive control of robotic lower-limb prostheses and exoskeletons. One participant was instrumented with a battery-powered, chest-mounted RGB camera system. Approximately 10 hours of video footage were experimentally collected while ambulating throughout unknown outdoor and indoor environments. The sampled images were preprocessed and individually labelled. A deep convolutional neural network was developed and trained to automatically recognize three walking environments: level-ground, incline staircases, and decline staircases. The environment recognition system achieved 94.85% overall image classification accuracy. Extending these preliminary findings, future research should incorporate other environment classes (e.g., incline ramps) and integrate the environment recognition system with electromechanical sensors and/or surface electromyography for automated locomotion mode recognition. The challenges associated with implementing deep learning on wearable biomechatronic devices are discussed.

Online adaptive neural control of a robotic lower limb prosthesis

Objective. The purpose of this study was to develop and evaluate an adaptive intent recognition algorithm that continuously learns to incorporate a lower limb amputee’s neural information (acquired via electromyography (EMG)) as they ambulate with a robotic leg prosthesis. Approach. We present a powered lower limb prosthesis that was configured to acquire the user’s neural information and kinetic/kinematic information from embedded mechanical sensors, and identify and respond to the user’s intent. We conducted an experiment with eight transfemoral amputees over multiple days. EMG and mechanical sensor data were collected while subjects using a powered knee/ankle prosthesis completed various ambulation activities such as walking on level ground, stairs, and ramps. Our adaptive intent recognition algorithm automatically transitioned the prosthesis into the different locomotion modes and continuously updated the user’s model of neural data during ambulation. Main results. Our proposed algorithm accurately and consistently identified the user’s intent over multiple days, despite changing neural signals. The algorithm incorporated 96.31% [0.91%] (mean, [standard error]) of neural information across multiple experimental sessions, and outperformed non-adaptive versions of our algorithm—with a 6.66% [3.16%] relative decrease in error rate. Significance. This study demonstrates that our adaptive intent recognition algorithm enables incorporation of neural information over long periods of use, allowing assistive robotic devices to accurately respond to the user’s intent with low error rates.

User-Independent Intent Recognition for Lower Limb Prostheses Using Depth Sensing.

The intent recognizers of advanced lower limb prostheses utilize mechanical sensors on the prosthesis and/or electromyographic measurements from the residual limb. Besides the delay caused by these signals, such systems require user-specific databases to train the recognizers. In this paper, our objective is the development and validation of a user-independent intent recognition framework utilizing depth sensing. We collected a depth image dataset from 12 healthy subjects engaging in a variety of routine activities. After filtering the depth images, we extracted simple features employing a recursive strategy. The feature vectors were classified using a support vector machine. For robust activity mode switching, we implemented a voting filter scheme. The model selection showed that the support vector machine classifier with no dimension reduction has the highest classification accuracy. Specifically, it reached 94.1% accuracy on the testing data from four subjects. We also observed a positive trend in the accuracy of classifiers trained with data from increasing the number of subjects. Activity mode switching using a voting filter detected 732 out of 778 activity mode transitions of the four users while initiating 70 erroneous transitions during steady-state activities. The intent recognizer trained on multiple subjects can be used for any other subject, providing a promising solution for supervisory control of powered lower limb prostheses. A user-independent intent recognition framework has the potential to decrease or eliminate the time required for extensive data collection regiments for intent recognizer training. This could accelerate the introduction of robotic lower limb prostheses to the market.

A Robotic Lower Limb Prosthesis for Efficient Bicycling

A controller for a powered transfemoral prosthesis is presented, which can coordinate power delivery at the knee with the motion of the crankshaft on a bicycle. The controller continuously estimates the lengths of a four-bar linkage model through the application of a recursive least squares algorithm. The link lengths are used to estimate the angle of the bicycle crankshaft. With this measure, the delivery of knee torque is coordinated with the motion of the user. The controller is implemented on a prosthesis prototype and assessed on a transfemoral amputee subject ( N = 1). The subject exhibited a bilateral work asymmetry of 83.5% when cycling with his daily use prosthesis in a free-swing mode. When 60-N·m peak assistance was provided by the powered prosthesis, the bilateral work asymmetry was reduced to 11.4%. The subject's metabolic energy rate was measured for speed and power-matched cycling while the powered prosthesis provided zero assistance (i.e., turned of f and providing its back-drive resistance) or 30-N·m peak assistance. Subject's metabolic energy rate decreased by 16.5% when receiving powered assistance relative to the zero-assistance condition.

Combining Vibrotactile Feedback with Volitional Myoelectric Control for Robotic Transtibial Prostheses.

In recent years, the development of myoelectric control for robotic lower-limb prostheses makes it possible for amputee users to volitionally control prosthetic joints. However, the human-centered control loop is not closed due to the lack of sufficient feedback of prosthetic joint movement, and it may result in poor control performance. In this research, we propose a vibrotactile stimulation system to provide the feedback of ankle joint position, and validate the necessity of combining it with volitional myoelectric control to achieve improved control performance. The stimulation system is wearable and consists of six vibrators. Three of the vibrators are placed on the anterior side of the thigh and the other three on the posterior side of the thigh. To explore the potential of applying the proposed vibrotactile feedback system for prosthetic ankle control, eight able-bodied subjects and two transtibial amputee subjects (TT1 and TT2) were recruited in this research, and several experiments were designed to investigate subjects’ sensitivities to discrete and continuous vibration stimulations applied on the thigh. Then, we proposed a stimulation controller to produce different stimulation patterns according to current ankle angle. Amputee subjects were asked to control a virtual ankle displayed on the computer screen to reach different target ankle angles with a myoelectric controller, and control performances under different feedback conditions were compared. Experimental results indicated that subjects were more sensitive to stimulation position changes (identification accuracies were 96.39 ± 0.86, 91.11, and 93.89% for able-bodied subjects, TT1, and TT2, respectively) than stimulation amplitude changes (identification accuracies were 89.89 ± 2.40, 87.04, and 85.19% for able-bodied subjects, TT1, and TT2, respectively). Response times of able-bodied subjects, TT1, and TT2 to stimulation pattern changes were 0.47 ± 0.02 s, 0.53 s, and 0.48 s, respectively. Furthermore, for both TT1 and TT2, the absolute error of virtual ankle control reduced by about 50% with the addition of vibrotactile feedback. These results suggest that it is promising to apply the vibrotactile feedback system for the control of robotic transtibial prostheses.

Robotic lower limb prosthesis design through simultaneous computer optimizations of human and prosthesis costs.

Robotic lower limb prostheses can improve the quality of life for amputees. Development of such devices, currently dominated by long prototyping periods, could be sped up by predictive simulations. In contrast to some amputee simulations which track experimentally determined non-amputee walking kinematics, here, we explicitly model the human-prosthesis interaction to produce a prediction of the user’s walking kinematics. We obtain simulations of an amputee using an ankle-foot prosthesis by simultaneously optimizing human movements and prosthesis actuation, minimizing a weighted sum of human metabolic and prosthesis costs. The resulting Pareto optimal solutions predict that increasing prosthesis energy cost, decreasing prosthesis mass, and allowing asymmetric gaits all decrease human metabolic rate for a given speed and alter human kinematics. The metabolic rates increase monotonically with speed. Remarkably, by performing an analogous optimization for a non-amputee human, we predict that an amputee walking with an appropriately optimized robotic prosthesis can have a lower metabolic cost – even lower than assuming that the non-amputee’s ankle torques are cost-free.

高機能化が進む義足・下肢装具(解説 リハビリのメカトロニクス,<特集>リハビリ・介護とメカトロニクス)

高機能化が進む義足・下肢装具(解説 リハビリのメカトロニクス,<特集>リハビリ・介護とメカトロニクス)