Finger Gaiting Research Articles

Continuous adaptive gaits manipulation for three-fingered robotic hands via bioinspired fingertip contact events

The remarkable skill of changing its grasp status and relocating its fingers to perform continuous in-hand manipulation is essential for a multifingered anthropomorphic hand. A commonly utilized method of manipulation involves a series of basic movements executed by a high-level controller. However, it remains unclear how these primitives evolve into sophisticated finger gaits during manipulation. Here, we propose an adaptive finger gait-based manipulation method that offers real-time regulation by dynamically changing the primitive interval to ensure the force/moment balance of the object. Successful manipulation relies on contact events that act as triggers for real-time online replanning of multifinger manipulation. We identify four basic motion primitives of finger gaits and create a heuristic finger gait that enables the continuous object rotation of a round cup. Our experimental results verify the effectiveness of the proposed method. Despite the constant breaking and reengaging of contact between the fingers and the object during manipulation, the robotic hand can reliably manipulate the object without failure. Even when the object is subjected to interfering forces, the proposed method demonstrates robustness in managing interference. This work has great potential for application to the dexterous operation of anthropomorphic multifingered hands.

Nonprehensile Manipulation for Rapid Object Spinning via Multisensory Learning from Demonstration.

Dexterous manipulation concerns the control of a robot hand to manipulate an object in a desired manner. While classical dexterous manipulation strategies are based on stable grasping (or force closure), many human-like manipulation tasks do not maintain grasp stability and often utilize the dynamics of the object rather than the closed form of kinematic relation between the object and the robotic hand. Such manipulation strategies are referred as nonprehensile or dynamic dexterous manipulation in the literature. Nonprehensile manipulation often involves fast and agile movements such as throwing and flipping. Due to the complexity of such motions and uncertainties associated with them, it has been challenging to realize nonprehensile manipulation tasks in a reliable way. In this paper, we propose a new control strategy to realize practical nonprehensile manipulation. First, we make explicit use of multiple modalities of sensory data for the design of control law. Specifically, force data are employed for feedforward control, while position data are used for feedback control. Secondly, control signals (both feedback and feedforward) are obtained through multisensory learning from demonstration (LfD) experiments designed and performed for specific nonprehensile manipulation tasks of concern. To prove the concept of the proposed control strategy, experimental tests were conducted for a dynamic spinning task using a sensory-rich, two-finger robotic hand. The control performance (i.e., the speed and accuracy of the spinning task) was also compared with that of classical dexterous manipulation based on force closure and finger gaiting.

Learning to Transfer In‐Hand Manipulations Using a Greedy Shape Curriculum

AbstractIn‐hand object manipulation is challenging to simulate due to complex contact dynamics, non‐repetitive finger gaits, and the need to indirectly control unactuated objects. Further adapting a successful manipulation skill to new objects with different shapes and physical properties is a similarly challenging problem. In this work, we show that natural and robust in‐hand manipulation of simple objects in a dynamic simulation can be learned from a high quality motion capture example via deep reinforcement learning with careful designs of the imitation learning problem. We apply our approach on both single‐handed and two‐handed dexterous manipulations of diverse object shapes and motions. We then demonstrate further adaptation of the example motion to a more complex shape through curriculum learning on intermediate shapes morphed between the source and target object. While a naive curriculum of progressive morphs often falls short, we propose a simple greedy curriculum search algorithm that can successfully apply to a range of objects such as a teapot, bunny, bottle, train, and elephant.

Complex In-Hand Manipulation Via Compliance-Enabled Finger Gaiting and Multi-Modal Planning

Constraining contacts to remain fixed on an object during manipulation limits the potential workspace size, as motion is subject to the hand’s kinematic topology. Finger gaiting is one way to alleviate such restraints. It allows contacts to be freely broken and remade so as to operate on different manipulation manifolds. This capability, however, has traditionally been difficult or impossible to practically realize. A finger gaiting system must simultaneously plan for and control forces on the object while maintaining stability during contact switching. This letter alleviates the traditional requirement by taking advantage of system compliance, allowing the hand to more easily switch contacts while maintaining a stable grasp. Our method achieves complete <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$SO(3)$</tex-math></inline-formula> finger gaiting control of grasped objects against gravity by developing a manipulation planner that operates via orthogonal safe modes of a compliant, underactuated hand absent of tactile sensors or joint encoders. During manipulation, a low-latency 6D pose object tracker provides feedback via vision, allowing the planner to update its plan online so as to adaptively recover from trajectory deviations. The efficacy of this method is showcased by manipulating both convex and non-convex objects on a real robot. Its robustness is evaluated via perturbation rejection and long trajectory goals. To the best of the authors’ knowledge, this is the first work that has autonomously achieved full <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$SO(3)$</tex-math></inline-formula> control of objects within-hand via finger gaiting and without a support surface, elucidating a valuable step towards realizing true robot in-hand manipulation capabilities.

Algorithmic Fingertip Repositioning for Enhanced In-Hand Manipulation of the Objects

Abstract This article focuses on a method to relocate the robotic fingertips on the surface of the object when the fingertips instantaneously hold the object under precision grasp. Precision grasp involves holding the object using fingertips. Finger gaiting involves repositioning the fingertips on the surface of the object and then manipulation of the object. During repositioning, one contact point leaves the object surface and recontacts at the other point. A metric is defined on the set of feasible grasp configurations to limit deviation from force closure during repositioning of the fingertips. Then, a manipulability-based metric is described to search for the optimal goal grasp states on the object’s surface. The manipulability-based metric is used to search the grasp state to relocate the contacts, such that the range of object motion is increased.

ManipNet

Natural hand manipulations exhibit complex finger maneuvers adaptive to object shapes and the tasks at hand. Learning dexterous manipulation from data in a brute force way would require a prohibitive amount of examples to effectively cover the combinatorial space of 3D shapes and activities. In this paper, we propose a hand-object spatial representation that can achieve generalization from limited data. Our representation combines the global object shape as voxel occupancies with local geometric details as samples of closest distances. This representation is used by a neural network to regress finger motions from input trajectories of wrists and objects. Specifically, we provide the network with the current finger pose, past and future trajectories, and the spatial representations extracted from these trajectories. The network then predicts a new finger pose for the next frame as an autoregressive model. With a carefully chosen hand-centric coordinate system, we can handle single-handed and two-handed motions in a unified framework. Learning from a small number of primitive shapes and kitchenware objects, the network is able to synthesize a variety of finger gaits for grasping, in-hand manipulation, and bimanual object handling on a rich set of novel shapes and functional tasks. We also demonstrate a live demo of manipulating virtual objects in real-time using a simple physical prop. Our system is useful for offline animation or real-time applications forgiving to a small delay.

Sensing form - finger gaiting as key totactile object exploration - a data glove analysis of a prototypical dailytask

BackgroundMotor hand skill and associated dexterity is important for meetingthe challenges of daily activity and an important resource post-stroke. In thiscontext, the present study investigated the finger movements of right-handedsubjects during tactile manipulation of a cuboid, a prototypical task underlyingtactile exploration. During one motor act, the thumb and fingers of one handsurround the cuboid in a continuous and regular manner. While the object ismoved by the guiding thumb, the opposed supporting fingers are replaced oncethey reach their joint limits by free fingers, a mechanism termed fingergaiting.MethodsFor both hands of 22 subjects, we acquired the time series ofconsecutive manipulations of a cuboid at a frequency of 1 Hz, using a digitaldata glove consisting of 29 sensors. Using principle component analysis, wedecomposed the short action into motor patterns related to successivemanipulations of the cuboid. The components purport to represent changing graspconfigurations involving the stabilizing fingers and guiding thumb. The temporalfeatures of the components permits testing whether the distinct configurationsoccur at the frequency of 1 Hz, i.e. within the time window of 1 s, and, thus,taxonomic classification of the manipulation as finger gaiting.ResultsThe fraction of variance described by the principal componentsindicated that three components described the salient features of the singlemotor acts for each hand. Striking in the finger patterns was the prominent andvarying roles of the MCP and PIP joints of the fingers, and the CMC joint of thethumb. An important aspect of the three components was their representation ofdistinct finger configurations within the same motor act. Principal componentand graph theory analysis confirmed modular, functionally synchronous action ofthe involved joints. The computation of finger trajectories in one subjectillustrated the workspace of the task, which differed for the right and lefthands.ConclusionIn this task one complex motor act of 1 s duration could bedescribed by three elementary and hierarchically ordered grasp configurationsoccurring at the prescribed frequency of 1 Hz. Therefore, these configurationsrepresent finger gaiting, described until now only in artificial systems, as theprincipal mechanism underlying this prototypical task.Trial registrationclinicaltrials.gov, NCT02865642, registered 12 August 2016.

Object-Level Impedance Control for Dexterous In-Hand Manipulation

This work presents a novel object-level control framework for the dexterous in-hand manipulation of objects with torque-controlled robotic hands. The proposed impedance-based controller realizes the compliant 6-DOF positioning of a grasped object. Enabled by the in-hand localization, the slippage of contacts is avoided by actively maintaining the desired grasp configuration. The internal forces on the object are determined by solving a quadratic optimization problem, which explicitly considers friction constraints on the contacts and allows to gradually shift the load between fingers. The proposed framework is capable of dealing with dynamic changes in the grasp configuration, which makes it applicable for the control of the object during grasp acquisition or the reconfiguration of fingers (i.e. finger gaiting). A nullspace controller avoids joint limits and singularities. Experiments with the DLR robot David demonstrate the efficiency and performance of the controller in various grasping scenarios.

Learning dexterous in-hand manipulation

We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system such as friction coefficients and an object’s appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM.

Dynamic regrasping by in-hand orienting of grasped objects using non-dexterous robotic grippers

A method for in-hand dynamic regrasping using a non-dexterous gripper.Stochastic motion planning algorithm with trajectory optimization.The algorithm optimizes the initial pose of the regrasp and control gains.Simulations and experiments validate the proposed algorithm. Almost any task on an object requires regrasping of the object prior to performing an intended task by varying between grasp configurations. The human hand uses many methods to perform regrasping manipulations such as in-hand sliding, finger gaiting, juggling, picking and placing, etc. The most complex and inspiring approach is the in-hand orienting dynamic regrasping where an object is released into mid air and regrasped in a different orientation. This manipulation is useful in industrial robotics for rapid manufacturing and reducing the number of robotic arms in production lines. In this work, we present a novel approach for performing in-hand orienting regrasping using computed torque control. To maintain an efficient and low-cost approach, the regrasping is performed using a non-dexterous two-jaw gripper and by utilizing the robotic arms dynamics. We focus on the motion planning for the motion and propose a novel stochastic algorithm for performing an optimal manipulation satisfying the kinematic and dynamic constraints. The algorithm optimizes the initial pose of the arm and the control gains. Statistical analysis shows the probability for the algorithm to find a solution if such exists. Simulations on a KUKA arm and demonstration on a planar 3R arm validate the feasibility of the proposed regrasping approach and planning algorithm.

Real-Time Finger Gaits Planning for Dexterous Manipulation

Dexterous manipulation has broad potential applications in assembly lines, warehouses and agriculture, and so on. To perform large-scale, complicated manipulation tasks, a multi-fingered robotic hand sometimes has to sequentially adjust its grasping status, i.e. the finger gaits, to deal with the workspace limits and object stability. However, realizing finger gaits planning in dexterous manipulation is challenging due to the involved hybrid dynamics, complicated grasp quality metrics, and uncertainties during the finger gaiting. In this paper, a dual-stage optimization based controller is proposed to handle these challenges. First, a velocity-level finger gaits planner is introduced by combining object grasp quality with hand kinematic limitations. The proposed finger gaits planner is computationally efficient and can be solved in real-time. Second, a manipulation controller using force optimization is presented. To deal with mass uncertainties and external disturbances, a modified impedance control is integrated into the manipulation controller. The dual-stage controller does not require the shape of the object, nor does it rely on expensive 3D/6D tactile sensors. Simulation results verify the efficacy of the proposed dual-stage controller.

Hierarchical Fingertip Space: A Unified Framework for Grasp Planning and In-Hand Grasp Adaptation

We present a unified framework for grasp planning and in-hand grasp adaptation using visual, tactile, and proprioceptive feedback. The main objective of the proposed framework is to enable fingertip grasping by addressing problems of changed weight of the object, slippage, and external disturbances. For this purpose we introduce the Hierarchical Fingertip Space as a representation enabling optimization for both efficient grasp synthesis and online finger gaiting. Grasp synthesis is followed by a grasp adaptation step that consists of both grasp force adaptation through impedance control and regrasping/finger gaiting when the former is not sufficient. Experimental evaluation is conducted on an Allegro hand mounted on a Kuka LWR arm.

Synthesis of detailed hand manipulations using contact sampling

Capturing human activities that involve both gross full-body motion and detailed hand manipulation of objects is challenging for standard motion capture systems. We introduce a new method for creating natural scenes with such human activities. The input to our method includes motions of the full-body and the objects acquired simultaneously by a standard motion capture system. Our method then automatically synthesizes detailed and physically plausible hand manipulation that can seamlessly integrate with the input motions. Instead of producing one "optimal" solution, our method presents a set of motions that exploit a wide variety of manipulation strategies. We propose a randomized sampling algorithm to search for as many as possible visually diverse solutions within the computational time budget. Our results highlight complex strategies human hands employ effortlessly and unconsciously, such as static, sliding, rolling, as well as finger gaits with discrete relocation of contact points.

Regrasping objects during manipulation tasks by combining genetic algorithms and finger gaiting

During a manipulation task with a mechanical hand, some particular situations occur frequently. For example, a finger can reach its joint limits or there can be a collision between two or more parts of the hand. In such situations, regrasping must be done without changing the position and the orientation of the object inside the hand. The objective is thus to insure the continuity of the manipulation task.

Sampling-based finger gaits planning for multifingered robotic hand

To perform large scale or complicated manipulation tasks, a multi-fingered robotic hand sometimes has to sequentially adjust its grasp status to overcome constraints of the manipulation, such as workspace limits, force balance requirement, etc. Such a strategy of changing grasping status is called a finger gait, which exhibits strong hybrid characteristics due to the discontinuity caused by relocating limited fingers and the continuity caused by manipulating objects. This paper aims to explore the complicated finger gaits planning problem and provide a method for robotic hands to autonomously generate feasible finger gaits to accomplish given tasks. Based on the hybrid automaton formulation of a popular finger gaiting primitive, finger substitution, we formulate the finger gait planning problem into a classic motion planning problem with a hybrid configuration space. Inspired by the rapidly-exploring random tree (RRT) techniques, we develop a finger gait planner to quickly search for a feasible manipulation strategy with finger substitution primitives. To increase the search performance of the planner, we further develop a refined sampling strategy, a novel hybrid distance and an efficient exploring strategy with the consideration of the problem's hybrid nature. Finally, we use a representative numerical example to verify the validity of our problem formulation and the performance of the RRT based finger gait planner.

A Kinematic Model of Finger Gaits by Multifingered Hand as Hybrid Automaton

Finger gait is necessarily needed in order to relocate fingers of the robotic hand when a dextrous manipulation task cannot be accomplished only by the rolling and sliding motions of the fingers. In this paper, a novel modeling framework for the multifingered manipulation with finger gaits is proposed. Through classifying the fingers into grasping fingers and free fingers, an alternate representation of a grasp is introduced. With the consideration of both its discrete and continuous characteristics, the kinematics model of a fingered manipulation with finger gaits is formulated into a hybrid automaton. Finally, simulation results verify the validity of the proposed modeling framework.

CPG Based Manipulation: Measurement of Human's Rhythmic Finger Gaits

A number of researches for the dextrous manipulation have been studied in robotics and physiologic fields independently. Multi-fingered manipulation for dextrous robot hands should be based on the analysis of human skills. The purpose of this study is to apply the CPG-based control to the dextrous manipulation. In this paper, CPG-based manipulation is proposed for the rotating manipulation. Manipulation patterns of fingers are investigated when a subject rotates a cylindrical object. Typical contact patterns during the rotating manipulation are measured by FSRs. Based on the observation, the neural circuit model which generates a similar contact pattern is constructed. The experimental results by the computer simulation suggest that rotating manipulation can be performed using the constructed CPG model.

Motion planning for kinematic stratified systems with application to quasi-static legged locomotion and finger gaiting

We present a general motion planning algorithm for robotic systems with a "stratified" configuration space. Such systems include quasi-static legged robots and kinematic models of object manipulation by finger repositioning. Our method is an extension of a nonlinear motion planning algorithm for smooth systems to the stratified case, where the relevant dynamics are not smooth. The method does not depend upon the number of legs or fingers; furthermore, it is not based on foot placement or finger placement concepts. Examples demonstrate the method.

Selected Papers INES2000

In engineering practice we often have to deal with complex systems, where the conventional approaches for understanding and predicting the behavior of the system can prove to be inadequate. Hence, the researchers try to put some intelligence into the system. The term intelligence in this context still more or less remains a mysterious phenomenon and can be characterized by different abilities of the system or machine, such as adaptation, decision-making, learning, recognition, diagnostics, autonomy, etc. Many of the new results related to this area are published in Journals and in International Conference Proceedings. One such conference is the "IEEE International Conference on Intelligent Engineering Systems". The fourth conference in this series (INES 2000) took place in Portoroz, Slovenia, on September 17-19,2000. There were around eighty participants from eighteen countries around the world. We are glad that so many authors have contributed to ideas related to the issues at the conference. Many of the papers were about applications and design, and others on more theoretical aspects of intelligent systems. This variety made the selection of papers for this special issue very difficult. Eight papers have been selected in the end, which cover different aspects of intelligent engineering systems. It should be pointed out that the respective authors were also kind to revise and update the presented papers for this special issue. The first paper deals with the manipulation problem where the motion changes depending on the state of the system as it is the case in the finger gaiting applications. To solve it the semi-stratified control theory using smooth motion planning is used. The proposed concept combines the stratified motion planning with the unconstrained finger allocations. In the second paper a special branch of Soft Computing developed for the control of mechanical devices is described. It reduces the number of free parameters and computational complexity. For illustration of the efficiency of the proposed adaptive control, a simulation of polishing with a 3 DOF robot is given. The next paper discusses the force control of redundant robots in an unstructured environment. A special attention is given to the decoupling of the task space and null space motion. For that the minimal null space approach is used. The proposed impedance controller assures good task space performances and minimizes the disturbances caused by obstacles. The performance of the proposed controllers has been evaluated by the simulation and by experiments on a real robot. The forth paper presents some advanced modeling approaches and methods. As one of the key issues a manufacturing process model fully associative with form feature based part model has been introduced. The motivation has been that the low level integration of design and manufacturing of mechanical parts, as identified by the authors, is still a main drawback of efficient application of expensive modeling systems. The proposed method allows for creating part model simultaneously with their analysis of machineability. The next paper discusses the design of fractal-order discrete-time controllers. Some approaches to implement fractal derivatives and integrals are analyzed. As the application of the theory of fractional calculus is rather new, many aspects remain to be investigated. The sixth paper demonstrates how to map classical dictionaries and similar structured data to a hypertext structure that is more suitable for the modern media. To achieve the new shape automatically, the HiLog language is used. The automated mapping is illustrated by an example based on Oxford Dictionary of Modern English. In the seventh paper a humanoid robotics shoulder is compared to the human shoulder. First, the capabilities of the robotics shoulder are analyzed and next, using the optical measurement system the human shoulder movements have been measured and analyzed. The last paper discusses the bias-variance tests on multi-layer perception. The performance of Bayesian neural networks is compared with the performance of neural networks trained with a gradient method. Additionally, it is analyzed if it is possible to use a number of networks in committee trained with gradient descent to achieve the performance of a Bayesian network.

Extensions of Nonlinear Control Concept to Object Manipulation with Finger Relocation

The paper provides a powerful basic concept for object manipulation with finger relocation and extends the previous approaches in the area of stratified manipulation. The algorithm gives also some improvements of the developed software for smooth and stratified motion planning (MP). The key element of the approach is based strongly on the theory of stratified control for nonlinear systems. The discussed manipulation concept exploits the information about the shape of the object and endeavors to discover its most important physical properties. In connection with the subject, the paper investigates the finger gaiting manipulation (based on finger relocation) in the context of stratified nonlinear control.