Control Of Robotic Hand Research Articles

Concurrent Estimation of Finger Flexion and Extension Forces Using Motoneuron Discharge Information

A reliable neural-machine interface offers the possibility of controlling advanced robotic hands with high dexterity. The objective of this study was to develop a decoding method to estimate flexion and extension forces of individual fingers concurrently. First, motor units (MUs) firing information were identified through surface electromyogram (EMG) decomposition, and the MUs were further categorized into different pools for the flexion and extension of individual fingers via a refinement procedure. MU firing rate at the populational level was calculated, and the individual finger forces were then estimated via a bivariate linear regression model (neural-drive method). Conventional EMG amplitude-based method was used as a comparison. Our results showed that the neural-drive method had a significantly better performance (lower estimation error and higher correlation) compared with the conventional method. Our approach provides a reliable neural decoding method for dexterous finger movements. Further exploration of our method can potentially provide a robust neural-machine interface for intuitive control of robotic hands.

Development of a Real-Time Human-Robot Collaborative System Based on 1 kHz Visual Feedback Control and Its Application to a Peg-in-Hole Task.

In this research, we focused on Human-Robot collaboration. There were two goals: (1) to develop and evaluate a real-time Human-Robot collaborative system, and (2) to achieve concrete tasks such as collaborative peg-in-hole using the developed system. We proposed an algorithm for visual sensing and robot hand control to perform collaborative motion, and we analyzed the stability of the collaborative system and a so-called collaborative error caused by image processing and latency. We achieved collaborative motion using this developed system and evaluated the collaborative error on the basis of the analysis results. Moreover, we aimed to realize a collaborative peg-in-hole task that required a system with high speed and high accuracy. To achieve this goal, we analyzed the conditions required for performing the collaborative peg-in-hole task from the viewpoints of geometric, force and posture conditions. Finally, in this work, we show the experimental results and data of the collaborative peg-in-hole task, and we examine the effectiveness of our collaborative system.

Reinforcement learning control of a humanoid robotic hand actuated by shape memory alloy

Humanoid robotic hand actuated by shape memory alloy (SMA) represents a new emerging technology. SMA has a wide range of potential applications in many different fields, ranging from industrial assembly to biomedicine applications, due to the characteristic of high power-to-weight ratio, low driving voltages and noiselessness. However, nonlinearities of SMA and complex dynamic models of SMA-based robotic hands result in difficulties in controlling. In this paper, a humanoid SMA-based robotic hand composed of five fingers is presented with the ability of adaptive grasping. Reinforcement learning as a model-free control strategy can search for optimal control of systems with nonlinear and uncertainty. Therefore, an adaptive SA-Q-Learning (ASA-Q-learning) controller is proposed to control the humanoid robotic finger. The performance of ASA-Q-learning controller is compared with SA-Q-learning and PID controller through experimentation. Results have shown that ASA-Q-learning controller can control the humanoid SMA-based robotic hand effectively with faster convergence rate and higher control precision than SA-Q-learning and PID controller, and is feasible for implementation in a model-free system.

Object-Agnostic Dexterous Manipulation of Partially Constrained Trajectories

We address the problem of controlling a partially constrained trajectory of the manipulation frame–an arbitrary frame of reference rigidly attached to the object–as the desired motion about this frame is often underdefined. This may be apparent, for example, when the task requires control only about the translational dimensions of the manipulation frame, with disregard to the rotational dimensions. This scenario complicates the computation of the grasp frame trajectory, as the mobility of the mechanism is likely limited due to the constraints imposed by the closed kinematic chain. In this letter, we address this problem by combining a learned, object-agnostic manipulation model of the gripper with Model Predictive Control (MPC). This combination facilitates an approach to simple vision-based control of robotic hands with generalized models, enabling a single manipulation model to extend to different task requirements. By tracking the hand-object configuration through vision, the proposed framework is able to accurately control the trajectory of the manipulation frame along translational, rotational, or mixed trajectories. We provide experiments quantifying the utility of this framework, analyzing its ability to control different objects over varied horizon lengths and optimization iterations, and finally, we implement the controller on a physical system.

Dexterous Force Estimation during Finger Flexion and Extension Using Motor Unit Discharge Information.

With the development of advanced robotic hands, a reliable neural-machine interface is essential to take full advantage of the functional dexterity of the robots. In this preliminary study, we developed a novel method to estimate isometric forces of individual fingers continuously and concurrently during dexterous finger flexion and extension. Specifically, motor unit (MU) discharge activity was extracted from the surface high-density electromyogram (EMG) signals recorded from the finger extensors and flexors, respectively. The MU information was separated into different groups to be associated with the flexion or extension of individual fingers and was then used to predict individual finger forces during multi-finger flexion and extension tasks. Compared with the conventional EMG amplitude-based method, our method can obtain a better force estimation performance (a higher correlation and a smaller estimation error between the predicted and the measured force) when a linear regression model was used. Further exploration of our method can potentially provide a robust neural-machine interface for intuitive control of robotic hands.

영상기반 파지 동작 제어를 위한 딥러닝 기반 물체검출과 파지물체 선정

Hands perform various functions. There are many inconveniences in life without the use of hands. People without the use of hands wear prostheses. Recently, there have been many developments and studies about robotic prosthetic hands performing hand functions. Grasping motions of robotic prosthetic hands are integral in performing various functions. Grasping motions of robotic prosthetic hands are required recognition of grasping targets. A path toward using images to recognize grasping targets exists. In this study, object recognition in images for grasping motions are performed by using object detection based on deep-learning. A suitable model for the grasping motion was examined through three object detection models. Also, we present a method for selecting a grasping target when several objects are recognized. Additionally, it will be used for grasping control of robotic prosthetic hands in the future and possibly enable automatic control robotic prosthetic hands.

On-skin graphene electrodes for large area electrophysiological monitoring and human-machine interfaces

On-skin sensors capable of capturing human electrophysiological signals are valuable for a variety of areas, such as wearable health monitoring, prosthesis, human machine interfaces (HMI). Extensive research efforts have been devoted to developing on-skin electrodes and they offer potential advantages over conventional rigid electrodes. However, those approaches usually have limitations as they require sophisticated and costly fabrication processes to produce only a small number of electrodes. Here we report the reduced graphene oxide (rGO) electrodes prepared by simultaneous room temperature reduction and patterning process for scalable fabrication of a large number of on-skin electrodes. Due to the low thickness and adhesive substrate adopted, the rGO electrodes can form conformal contact with skin even during deformation, leading to reduced impedance in skin-electrode interface. The rGO electrodes have been successfully adopted for the measurements of electromyography (EMG), electrooculogram (EOG)and electroencephalogram (EEG) signals with high fidelity. And the implementation of a HMI task is demonstrated through the recognition of five typical human hand grasping gestures via a multi-channel rGO electrode array for the control of a dexterous robotic hand to mimic the gestures. The rGO electrodes presented in this study are promising for practical applications in healthcare and HMI.

Investigating the performance of an amplitude-independent algorithm for detecting the hand muscle activity of stroke survivors

To make robotic hand devices controlled by surface electromyography (sEMG) signals feasible and practical tools for assisting patients with hand impairments, the problems that prevent these devices from being widely used have to be overcome. The most significant problem is the involuntary amplitude variation of the sEMG signals due to the movement of electrodes during forearm motion. Moreover, for patients who have had a stroke or another neurological disease, the muscle activity of the impaired hand is weak and has a low signal-to-noise ratio (SNR). Thus, muscle activity detection methods intended for controlling robotic hand devices should not depend mainly on the amplitude characteristics of the sEMG signal in the detection process, and they need to be more reliable for sEMG signals that have a low SNR. Since amplitude-independent muscle activity detection methods meet these requirements, this paper investigates the performance of such a method on people who have had a stroke in terms of the detection of weak muscle activity and resistance to false alarms caused by the involuntary amplitude variation of sEMG signals; these two parameters are very important for achieving the reliable control of robotic hand devices intended for people with disabilities. A comparison between the performance of an amplitude-independent muscle activity detection algorithm and three amplitude-dependent algorithms was conducted by using sEMG signals recorded from six hemiparesis stroke survivors and from six healthy subjects. The results showed that the amplitude-independent algorithm performed better in terms of detecting weak muscle activity and resisting false alarms.

Grip Stabilization through Independent Finger Tactile Feedback Control

Grip force control during robotic in-hand manipulation is usually modeled as a monolithic task, where complex controllers consider the placement of all fingers and the contact states between each finger and the gripped object in order to compute the necessary forces to be applied by each finger. Such approaches normally rely on object and contact models and do not generalize well to novel manipulation tasks. Here, we propose a modular grip stabilization method based on a proposition that explains how humans achieve grasp stability. In this biomimetic approach, independent tactile grip stabilization controllers ensure that slip does not occur locally at the engaged robot fingers. Local slip is predicted from the tactile signals of each fingertip sensor i.e., BioTac and BioTac SP by Syntouch. We show that stable grasps emerge without any form of central communication when such independent controllers are engaged in the control of multi-digit robotic hands. The resulting grasps are resistant to external perturbations while ensuring stable grips on a wide variety of objects.

Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization.

Grasping force control is important for multi-fingered robotic hands to stabilize the grasped object. Humans are able to adjust their grasping force and react quickly to instabilities through tactile sensing. However, grasping force control through tactile sensing with robotic hands is still relatively unexplored. In this paper, we make use of tactile sensing for multi-fingered robot hands to adjust the grasping force to stabilize unknown objects without prior knowledge of their shape or physical properties. In particular, an online detection module based on Deep Neural Network (DNN) is designed to detect contact events and object material simultaneously from tactile data. In addition, a force estimation method based on Gaussian Mixture Model (GMM) is proposed to compute the contact information (i.e., contact force and contact location) from tactile data. According to the results of tactile sensing, an object stabilization controller is then employed for a robotic hand to adjust the contact configuration for object stabilization. The spatio-temporal property of tactile data is exploited during tactile sensing. Finally, the effectiveness of the proposed framework is evaluated in a real-world experiment with a five-fingered Shadow Dexterous Hand equipped with BioTac sensors.

Robotic Hand Control with a Remote Sensory Glove

Robotic Hand Control with a Remote Sensory Glove

Fuzzy Logic Based Force Control of Robot Hand with Haptic Feedback

Sunulan çalışmada; kullanıcı tarafından kontrol edilen bir robot el sistemi için çalışma yapılmıştır. Sistemin nesne kavrama kuvveti hassasiyetinin sağlanması amacıyla kavrama kuvvetinin kontrol sorunu üzerine durulmuştur. Bunun yanı sıra nesneye uygulanan kuvvetin kullanıcıya geri bildirimi için haptik geri bildirim sistemi oluşturulmuştur. Robot elin kullanıcı komutu doğrultusunda nesnelere uyguladığı kuvvetin kontrolünde, bulanık mantık önerilmiş ve uygulanmıştır. Kullanıcı bulanık mantık kontrolcüye tut, bırak, sık gibi farklı kavrama komutlarını gönderebilmektedir. Bulanık mantık kontrolcü ise kullanıcı komutu doğrultusunda objeye uygulanan kuvveti kontrol etmektedir. Ayrıca kuvvet sensöründen gelen veriler bulanık mantık kontrolcü ile değerlendirilerek haptik geri bildirim sisteminde bulunan titreşim motorlarının titreşim şiddeti ayarlanmaktadır. Böylelikle kullanıcı, hem objeleri ne kadar kuvvetle sıktığını hissedebilmekte hem de sadece bir kavrama komutu göndererek nesnenin uygun kuvvette kavranmasını sağlayabilmektedir. Oluşturulan sistem objelerin kavranması esnasında oluşan kavrama kuvvetinin kontrol sorunu ve his geri bildirim eksikliği sorunlarına bir çözüm niteliğindedir.

Reconstructing Degree of Forearm Rotation from Imagined movements for BCI-based Robot Hand Control.

Brain-computer interface (BCI) is an important tool for rehabilitation and control of an external device (e.g., robot arm or home appliances). Fully reconstruction of upper limb movement from brain signals is one of the critical issues for intuitive BCI. However, decoding of forearm rotation from imagined movements using electroencephalography (EEG) is difficult to decode degree of rotation accurately. In this paper, we reconstructed imagined forearm rotation from low- frequency (0.3-3 Hz) of EEG signals. We selected 20 EEG channel on motor cortex for analysis. Ten healthy subjects participated in our experiment. The subjects performed actual and imagined forearm rotation to reach different targets. We trained a reconstruction decoder which used the EEG signals measured from actual movements and the kinematic information only. Additionally, we applied a long short-term memory (LSTM) network to enhance decoding performances. As a result, we achieved the high correlation performance (Average: 0.67) to decode imagined forearm rotation angle. This result has demonstrated that the reconstruction decoder which is trained by the EEG data from actual movement has effective to decode robustly for the imagined forearm rotation angle.

Hierarchical Proximity Sensor for High-Speed and Intelligent Control of Robotic Hand

This study proposes a hierarchical proximity information processing system and a novel proximity sensor for realizing high-speed and robust grasp control of a robotic hand. The sensor requires both fast response and advanced situation judgment abilities. Therefore, a function that digitally samples individual reaction amounts of all detection elements is added to the net-structure proximity sensor (NSPS), which extracts the sum and center position of the distribution of reaction amounts of detection elements by high-speed analog computation on the sensor circuit. To integrate these two functions, we construct a circuit design method that enables the coexistence of a multichannel A/D converter circuit on the analog computing circuit of the NSPS without disturbing the current flow for sensing; the proposed sensor is called the “hierarchical proximity sensor.” An analysis of its characteristics indicates that the sensor can be used for feedback control of the fingertip position/posture and to estimate the curvature of objects. Through an experiment conducted using a robotic hand equipped with the proposed sensor, we confirmed that the fingertip can approach an object in 0.18 s based on the high-speed analog computation information, while the information for improving the motion can be obtained by comparing the temporal change in the finger joints with the digital sampling information of the process.

A Control Architecture for Grasp Strength Regulation in Myocontrolled Robotic Hands Using Vibrotactile Feedback: Preliminary Results.

Nowadays, electric-powered hand prostheses do not provide adequate sensory instrumentation and artificial feedback to allow users voluntarily and finely modulate the grasp strength applied to the objects. In this work, the design of a control architecture for a myocontrol-based regulation of the grasp strength for a robotic hand equipped with contact force sensors is presented. The goal of the study was to provide the user with the capability of modulating the grasping force according to target required levels by exploiting a vibrotactile feedback. In particular, the whole human-robot control system is concerned (i.e. myocontrol, robotic hand controller, vibrotactile feedback.) In order to evaluate the intuitiveness and force tracking performance provided by the proposed control architecture, an experiment was carried out involving four naïve able-bodied subjects in a grasping strength regulation task with a myocontrolled robotic hand (the University of Bologna Hand), requiring for grasping different objects with specific target force levels. The reported results show that the control architecture successfully allowed all subjects to achieve all grasping strength levels exploiting the vibrotactile feedback information. This preliminary demonstrates that, potentially, the proposed control interface can be profitably exploited in upper-limb prosthetic applications, as well as for non-rehabilitation uses, e.g. in ultra-light teleoperation for grasping devices.

Abstract 76: Motor and Sensory Control of Robotic Hands Through Fascicular Targeting

PURPOSE: 18-25,000 upper limb amputations occur in the US annually. We have developed fascicular targeting (FAST) as a surgical approach for implanting electrode interfaces within the individual component fascicular groups of the nerves in the residual limb of upper extremity amputees. Our hypothesis is that FAST, when combined with custom-developed technologies including longitudinal intrafascicular electrode (LIFE) interfaces, nerve stimulation and recording electronics, sensory stimulation patterns, and artificial intelligence (AI) motor decoding algorithms, will enable the restoration of naturalistic hand function in robotic hand protheses used by upper extremity amputees. METHODS: 5 upper extremity amputees were implanted with LIFE + cuff interfaces for durations ranging from 3 months up to 1 year. 3 subjects were partial-hand amputees with 2 FAST interfaces (2 ulnar nerve fascicles). 2 subjects were transradial amputees with 4 FAST interfaces (2 ulnar nerve and 2 median nerve fascicles). Weekly experimental sessions were performed: to assess stimulation parameters for sensory feedback, to record motor signals for robotic hand control, and to train and measure functional performance following restoration of naturalistic sensory feedback and motor control. Implants were removed at the completion of the trial. RESULTS: All implants were well-tolerated by subjects, with little to no functional morbidity caused by their participation in the trial. Sensory stimulation thresholds remained within a usable range for the duration of the trial. Stimulation patterns were used to elicit sensations perceived as either natural or non-natural, according to subjects’ preferences. Sensory discrimination and anatomic localization were also assessed. Motor signals were recorded using a novel microchip design with built-in artifact rejection circuitry. Acquisition of single-unit motor data permitted the use of AI-based signal decoding algorithms to establish independent, free-will control of all 5 digits of the robotic hand. Single-session decodes continued to work for over 3–4 weeks post-training. Functional performance using closed-loop sensorimotor control, with anatomically relevant sensory feedback and user controlled activation of individual digits of the robotic hand, will be discussed in detail. CONCLUSION: Fascicular targeting, when combined with specialized sensory stimulation and motor recording strategies, can enable true dexterity for upper extremity amputees using robotic hands. This holds promise for the development of full clinical systems to restore the hands of upper extremity amputees.

Caging-based grasping of deformable objects for geometry-based robotic manipulation

In this paper, we present a novel method for caging-based grasping of deformable objects. This method enables manipulators to grasp objects simply with geometric constraints by using position control of robotic hands, and not through force controls or mechanical analysis. Therefore, this method has cost benefits and algorithmic simplicity. In our previous studies, we mainly focused on caging-based grasping of rigid objects such as 2D/3D primitive-shaped objects. However, considering realistic objects, manipulation of deformable objects is also required frequently. Hence, this study is motivated to manipulate deformable objects, adopting a caging-based grasping approach. We formulate caging-based grasping of deformable objects, and target three types of deformable objects: a rigid object covered with a soft part, a closed-loop structure, and two rigid bodies connected with a string, which can be regarded as primitive shapes. We then derive concrete conditions for grasp synthesis and conduct experimental verification of our proposed method with an industrial manipulator.

A Systematic Approach to Evaluating and Benchmarking Robotic Hands—The FFP Index

The evaluation of robotic hands is a subjectively biased, complex process. The fields pertaining to robotic hands are human-centric in nature, making human hands a good standard for benchmark comparisons of robotic hands. To achieve this, we propose a new evaluation index, where we evaluate robotic hands on three fronts: their form, features and performance. An evaluation on how anthropomorphic robotic hands are in basic mobility, and appearance constitutes the “Form”, while features that can be read, changed and actuated for effective control of robotic hands constitutes the “Features”. We derived these key features from an extensive analysis of robotic hands in literature. Finally, the robotic hands carry out a series of tasks that evaluate their “Performance”. An individual score for each category is drawn and we carry out a three-pronged analysis. We also propose an additional feature in the form of price to provide context when analysing multiple hands.

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses.

This work describes a methodological framework that can be used to explicitly and implicitly characterize the sense of agency developed over the neural-machine interface (NMI) control of sensate virtual or robotic prosthetic hands. The formation of agency is fundamental in distinguishing the actions that we perform with our limbs as being our own. By striving to incorporate advanced upper-limb prostheses into these same perceptual mechanisms, we can begin to integrate an artificial limb more closely into the user's existing cognitive framework for limb control. This has important implications in promoting user acceptance, use, and effective control of advanced upper-limb prostheses. In this protocol, participants control a virtual prosthetic hand and receive kinesthetic sensory feedback through their preexisting NMIs. A series of virtual grasping tasks are performed and perturbations are systematically introduced to the kinesthetic feedback and virtual hand movements. Two separate measures of agency are employed: established psychophysical questionnaires (to capture the explicit experience of agency) and a time interval estimate task to capture the implicit sense of agency (intentional binding). Results of this protocol (questionnaire scores and time interval estimates) can be analyzed to quantify the extent of agency formation.

Synergy-Based Control of Anthropomorphic Robotic Hands with Contact Force Sensors

A study on the synergy-based control of an anthropomorphic hand is proposed in this paper. The robotic hand is equipped with three-axis optical force sensors placed on thumb, index, and middle fingertips in order to obtain a better grasping quality in terms of adaptability and contact forces with respect to the physical object properties. A fully-actuated hand is used in the experiments since this allows to select different postural synergies and explore how the hand behaves in case of decoupled finger motion. Therefore, the proposed method can be extended to two different application cases: purely synergy-based contact force control and synergy-based contact force control with decoupled fingers. While the first control strategy allows to simplify the grasping by selecting only the synergy weights in order to obtain the desired value of the mean contact forces among the fingers, the second allows both to control the contact force individually on each finger in order to better adapt the hand to the object and to preserve the simplicity in the grasp design provided by the synergy-based approach. Experimental tests have been performed in both the cases to show the performance of the proposed methods and to highlight the capability of the proposed control strategies to regulate the contact forces properly.