Fine Manipulation Research Articles

Intent-based Task-Oriented Shared Control for Intuitive Telemanipulation

The unique challenges in shared control for telemanipulation-beyond teleoperation-include physical discrepancy between human and robot hands and the fine manipulation constraints needed for task success. We present an intuitive shared-control strategy that generates robotic grasp poses better suited for human perception of success and feeling of control while ensuring a stable grasp for task success. The robot’s motion follows an arbitration between following the user’s motion constraints and accomplishing the inferred task. The arbitration adapts based on the physical discrepancy between the human and robot hands. We have conducted a user study with a telemanipulation scenario to analyze the effects of task predictability, following, and user preference. The results demonstrated that intent-based approaches provide advantages over direct motion mapping strategies.

Multi-parameter optimization of polarization gradient cooling for 87Rb atoms based on reinforcement learning.

Polarization gradient cooling (PGC) plays an important role in many cold atom applications including the formation of Bose-Einstein condensates (BECs) and cooling of single atoms. Traditional parameter optimization of PGC usually relies on subjective expertise, faces challenges in fine manipulation, and exhibits low optimization efficiency. Here, we propose a segmented control method that differs from the traditional PGC process by expanding the experiment parameters from 3 to 30. Subsequently, the conventional timing optimization problem is reformulated as a Markov decision process (MDP), and the experiment parameters are optimized using a reinforcement learning model. With proper settings of hyperparameters, the learning process exhibits good convergence and powerful parameter exploration capabilities. Finally, we capture ∼4.3 × 108 cold atoms, with a phase space density of ∼7.1 × 10-4 at a temperature of ∼3.7 µK in ∼18.8 min. Our work paves the way for the intelligent preparation of degenerate quantum gas.

Influence of breastfeeding type and gender on child development: differences in global and fine motor skills.

There are few studies that approach the subject of breastfeeding related to the development of motor skills. The study aimed to verify if there are differences in the global and fine motor skills, considering the variables gender and type of breastfeeding, in children from 18 to 44 months. We developed a quantitative correctional typology, being a cross-sectional study with a sample of 128 children of both genders. The instruments used in the study were the Peabody Developmental Motor Scales-2, and to assess the profile of the children a questionnaire was delivered to the parents. To test the normality of the sample, we applied the Kolmogorov-Smirnov Test and the Shapiro-Wilk Test. The Mann-Whitney U-Test was used for independent samples, also calculating the effect size. The boys obtained higher mean values in all motor skills, except for the fine motor skills and fine motricity, and in these, the girls obtainied the higher mean values. With respect to the inferential analysis, concerning the variable gender, there are statistically significant differences (P<0.00) in the variables: locomotor skills, object handling skills, global motricity and global motor quotient (P=0.017) the boys exhibiting the higher values, which means the better performances. We also verify that children who were not breastfed with breast milk, through feeding bottle with replacement milk, obtained higher mean values in all motor skills when compared to the ones who were breastfed with breast milk, and there is a statistically significant difference in the variable fine manipulation skills (P=0.024). Male children obtained better results in all motor skills, except for fine motricity, comparing with female children. Regarding breastfeeding, the children who were not breastfed with breast milk, show better results, mainly in Fine manipulation skills, which may be due to the previous need to handle the bottle on early age.

Triple Plexcitonic Nonreciprocity of Magnetochiral Plexcitons.

Nonreciprocal quantum devices, allowing different transmission efficiencies of light-matter polaritons along opposite directions, are key technologies for modern photonics, yet their miniaturization and fine manipulation remain an open challenge. Here, we report on magnetochiral plexcitons dressed with geometric-time double asymmetry in compact nonreciprocal hybrid metamaterials, leading to triple plexcitonic nonreciprocity with flexible controllability. A general magnetically dressed plexcitonic Born-Kuhn model is developed to reveal the hybrid optical nature and dynamic energy evolution of magnetochiral plexcitons, demonstrating a plexcitonic nonreciprocal mechanism originating from the strong coupling among photon, electron, and spin degrees of freedom. Moreover, we introduce the temperature-controlled knob/switch for magnetochiral plexcitons, achieving precise magnetochiral control and nonreciprocal transmission in a given system. We expect this mechanism and approach to open up a new route for the integration and fine control of on-chip nonreciprocal quantum devices.

A Dexterous and Compliant (DexCo) Hand Based on Soft Hydraulic Actuation for Human Inspired Fine In-Hand Manipulation

A Dexterous and Compliant (DexCo) Hand Based on Soft Hydraulic Actuation for Human Inspired Fine In-Hand Manipulation

A two-stage grasp detection method for sequential robotic grasping in stacking scenarios.

Dexterous grasping is essential for the fine manipulation tasks of intelligent robots; however, its application in stacking scenarios remains a challenge. In this study, we aimed to propose a two-phase approach for grasp detection of sequential robotic grasping, specifically for application in stacking scenarios. In the initial phase, a rotated-YOLOv3 (R-YOLOv3) model was designed to efficiently detect the category and position of the top-layer object, facilitating the detection of stacked objects. Subsequently, a stacked scenario dataset with only the top-level objects annotated was built for training and testing the R-YOLOv3 network. In the next phase, a G-ResNet50 model was developed to enhance grasping accuracy by finding the most suitable pose for grasping the uppermost object in various stacking scenarios. Ultimately, a robot was directed to successfully execute the task of sequentially grasping the stacked objects. The proposed methodology demonstrated the average grasping prediction success rate of 96.60% as observed in the Cornell grasping dataset. The results of the 280 real-world grasping experiments, conducted in stacked scenarios, revealed that the robot achieved a maximum grasping success rate of 95.00%, with an average handling grasping success rate of 83.93%. The experimental findings demonstrated the efficacy and competitiveness of the proposed approach in successfully executing grasping tasks within complex multi-object stacked environments.

A Mechanical Hand-Tracking System With Tactile Feedback Designed for Telemanipulation.

In this paper, we present a mechanical hand-tracking system with tactile feedback designed for fine manipulation in teleoperation scenarios. Alternative tracking methods based on artificial vision and data gloves have become an asset for virtual reality interaction. Yet, occlusions, lack of precision, and the absence of effective haptic feedback beyond vibrotactile still appear as a limit for teleoperation applications. In this work, we propose a methodology to design a linkage mechanism for hand pose tracking purposes, preserving complete finger mobility. Presentation of the method is followed by design and implementation of a working prototype, and by evaluation of the tracking accuracy using optical markers. Moreover, a teleoperation experiment involving a dexterous robotic arm and hand was proposed to ten participants. It investigated the effectiveness and repeatability of the hand tracking with combined haptic feedback during a proposed pick and place manipulation tasks.

Rapid Thermal Processing of Kesterite Thin Films

Multinary chalcogenides with Kesterite structure Cu2ZnSn(S,Se)4 (CZTSSe) are a prospective material base for the enhancement of the photovoltaics industry with abundant and environmentally friendly constituents and appropriate electro-physical properties for building highly efficient devices at a low cost with a short energy pay-back time. The actual record efficiency of 13.6%, which was reached recently, is far below the current isostructural chalcopyrite’s solar cells efficiency of near 24%. The main problems for future improvements are the defects in and stability of the Kesterite absorber itself and recombination losses at interfaces at the buffer and back contacts. Here, we present an investigation into the rapid thermal annealing (RTA) of as-electrodeposited thin films of Cu2ZnSnS4 (CZTS). The treatment was carried out in a cold wall tubular reactor in dynamic conditions with variations in the temperature, speed and time of the specific elements of the process. The effect of annealing was investigated by X-ray diffractometry, Raman scattering and Scanning Electron Microscopy (SEM). The phase composition of the films depending on treatment conditions was analyzed, showing that, in a slow, prolonged, high-temperature process, the low-temperature binaries react completely and only Kesterite and ZnS are left. In addition, structural investigations by XRD have shown a gradual decrease in crystallite sizes when the temperature level and duration of the high-temperature segment increases, and respectively increase in the strain due to the formation of the phases in non-equilibrium conditions. However, when the speed of dynamic segments in the process decreases, both the crystallite size and strain of the Kesterite non-monotonically decrease. The grain sizes of Kesterite, presented by SEM investigations, have been shown to increase when the temperature and the duration increase, while the speed decreases, except at higher temperatures of near 750 °C. The set of experiments, following a scrupulous analysis of Raman data, were shown to have the potential to elucidate a way to ensure the fine manipulation of the substitutional Cu/Zn defects in the structure of CZTS thin films, considering the dependences of the ratios of Q = I287/I303 and Q′ = I338/(I366 + I374) on the process variables. Qualitatively, it can be concluded that increases in the speed, duration and temperature of RTA lead to increases in the order of the structure, whereas, at higher temperatures of near 750 °C, these factors decrease.

Dual-Gain Mode of Head-Gaze Interaction Improves the Efficiency of Object Positioning in a 3D Virtual Environment

Head-gaze interaction is an integral mode of interaction in virtual reality (VR) applications, demonstrating high precision in fine manipulation tasks but low efficiency in large-scale object movements. To enhance the efficiency of head-gaze interaction, this study adjusted the control-display gain to compensate for the weaknesses of head-gaze interaction in a long-distance object-positioning task. We investigated the effect of the control-display gain of head-gaze interaction on movement time (MT) using a cohort of participants (n = 24) to perform experiments. The results showed that the MT first decreased as the gain increased from 1 to 1.5 and then increased afterwards. Further analysis showed that a high gain improved the interaction efficiency in the ballistic phase, but reduced the interaction efficiency in the corrective phase. To be able to obtain higher efficiency of interaction, we designed a dual-gain mode which set different gains in the ballistic and corrective phases. Evaluated using an additional experimental cohort (n = 24), our results showed that the dual-gain mode was more efficient than the mono-gain mode. Moreover, the dual-gain mode with optimal gains did not induce a more serious perception of inconsistency, confusion, nonacceptance and motion sickness, while it had a tendency to reduce the total workload compared to the interaction with normal gain. Our findings provide potential valuable design insights and guidance contributing to improving the efficiency of head-gaze interaction in virtual spaces.

Simultaneous hand/wrist motion recognition and continuous grasp force estimation based on nonlinear spectral sEMG features for transradial amputees

Simultaneously and accurately estimating both grasp motion classes and force based on surface electromyogram (EMG) of residual arms is essential for precise grasping with a dexterous prosthetic hand. However, accurate grasping-mode prediction and grasping-force estimation often contradict each other. When performing hand grasping motion, different grasping forces will generate different EMG patterns, which would decay the grasp motion prediction accuracy. In this study, we proposed a framework of fusing motion classification and continuous grasp force estimation based on a new set of nonlinear Mel-Frequency Spectrum features of EMG signals. The performance of the proposed method was tested on eight able bodied (AB) subjects and four transradial amputees (TR) who are the end-users of dexterous prostheses. The experimental results showed that the newly proposed features obtained a significant reduction in the average classification error rate compared to other well-known feature sets, achieving improvements of about 5 % to 28 % in the average classification accuracy across all subjects and force levels. Simultaneously, the proposed features obtained the best continuous grasp force estimation accuracies with R2 (coefficient of determination) of 0.96 and 0.83, and root mean square of maximum voluntary contractions of grasp force (std%GFMVC) of 6.2 % and 10.5 % for the AB and TR subjects, respectively, compared to the previous EMG features. This study suggested that the proposed method would be helpful for the stable and fine manipulation of multifunctional prosthetic hand.

Plasmon-Controlled Selective Emission Enhancement of Eu3+ with (Au Core)@(Y(V,P)O4:Eu) Nanostructures.

Plasmonic nanostructures have a capability to control the photoluminescence (PL) emission properties of optical species and thus can dramatically enhance the performances of diverse optical systems and devices. Lanthanide ions typically exhibit multiple PL emission lines. Systematic studies on the plasmon-enabled selective enhancement for the different emission lines of lanthanide ions are still highly desired in order to achieve the fine manipulation on the spectral profile and luminescence intensity ratio (LIR). Herein we report on the synthesis and PL emission properties of monodisperse spherical (Au core)@(Y(V,P)O4:Eu) nanostructures, which integrate the plasmonic and luminescent units into an individual core@shell structure. The localized surface plasmon resonance adjusted through control of the size of the Au nanosphere core enables the systematic modulation of the selective emission enhancement of Eu3+. As revealed by single-particle scattering and PL measurements, the five luminescence emission lines of Eu3+ originating from the 5D0,1 excitation states are affected by the localized plasmon resonance to different extents, which are dependent on both the dipole transition nature and the intrinsic quantum yield of the emission line. Based on the plasmon-enabled tunable LIR, high-level anticounterfeiting and optical temperature measurements for photothermal conversion are further demonstrated. Our architecture design and PL emission tuning results offer many possibilities for constructing multifunctional optical materials by integrating plasmonic and luminescent building blocks into hybrid nanostructures with different configurations.

The importance of oriented physical activity in the first 48 months: differences in motor skills

BackgroundThe importance of physical activity in the first months of age is well known, however, with the evolution of the urban environment, the excessive workload of parents and the excessive time in growing up in kindergartens has limited this same free practice and little has been studied about this issue. In Portugal, there are institutions that provide oriented physical activity for their children, however, this is optional, which may create disadvantages in children’s motor skills in these ages.ObjectiveThe objective of the study isto verify if there are differences in the development of motor skills (global and fine) comparing children between 12 and 48 months who practice oriented physical activity (OPA) and children who do not.MethodsParticipated in this study, 400 children of both genders (28.14 ± 7.23 months). Two groups were created (the group that had oriented physical activity (30 min long and 2 times a week) and the group that didn’t have oriented physical activity). For a better understanding they were divided into 3 age groups (12–23, 24–35 and 36–48 months). Motor skills were assessed using the PDMS-2 scales, for 6 months, following the instrument’s application standards.ResultsIn a first analysis, we found that the majority of children only start to practice oriented physical activity in institutions from 36 months of age, however, it is in the first months (from 12 to 35) that the greatest differences between the two groups can occur. The OPA group presented better results according to the mean values, in all motor skills. Differences between groups were most noticeable in Postural, locomotion and fine manipulation Skills (showing effect size: moderate and low).ConclusionsWe can conclude that a practice of oriented physical activity in the first 48 months is fundamental to the development of motor skills. It is in the first months (up to 36) that there are greater differences, but it is also where there are less children carrying out guided physical activity. This is an important factor, and is determinant to make institutions aware of this importance of this variable in child development.

Charge transfer engineering to achieve extraordinary power generation in GeTe-based thermoelectric materials.

By the fine manipulation of the exceptional long-range germanium-telluride (Ge─Te) bonding through charge transfer engineering, we have achieved exceptional thermoelectric (TE) and mechanical properties in lead-free GeTe. This chemical bonding mechanism along with a semiordered zigzag nanostructure generates a notable increase of the average zT to a record value of ~1.73 in the temperature range of 323 to 773 K with ultrahigh maximum zT~2.7. In addition, we significantly enhanced the Vickers microhardness numbers (Hv) to an extraordinarily high value of 247 Hv and effectively eliminated the thermal expansion fluctuation at the phase transition, which was problematic for application, by the present charge transfer engineering process and concomitant formation of microstructures. We further fabricated a single-leg TE generator and obtained a conversion efficiency of ~13.4% at the temperature difference of 463 K on a commercial instrument, which is located at the pinnacle of TE conversion.

Selectively Manipulating Interactions between Lanthanide Sublattices in Nanostructure toward Orthogonal Upconversion.

Smart control of ionic interactions is a key factor to manipulate the luminescence dynamics of lanthanides and tune their emission colors. However, it remains challenging to gain a deep insight into the physics involving the interactions between heavily doped lanthanide ions and in particular between the lanthanide sublattices for luminescent materials. Here we report a conceptual model to selectively manipulate the spatial interactions between erbium and ytterbium sublattices by designing a multilayer core-shell nanostructure. The interfacial cross-relaxation is found to be a leading process to quench the green emission of Er3+, and red-to-green color-switchable upconversion is realized by fine manipulation of the interfacial energy transfer on the nanoscale. Moreover, the temporal control of up-transition dynamics can also lead to an observation of green emission due to its fast rise time. Our results demonstrate a new strategy to achieve orthogonal upconversion, showing great promise in frontier photonic applications.

A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

In this work, we present a reconfigurable data glove design to capture different modes of human hand–object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks. To achieve various downstream tasks with distinct features, our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time. In the tactile-sensing mode, the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material; this design minimizes interference during complex hand movements. The virtual reality (VR) mode enables real-time interaction in a physically plausible fashion: A caging-based approach is devised to determine stable grasps by detecting collision events. Leveraging a state-of-the-art finite element method, the simulation mode collects data on fine-grained four-dimensional manipulation events comprising hand and object motions in three-dimensional space and how the object’s physical properties (e.g., stress and energy) change in accordance with manipulation over time. Notably, the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions. In a series of experiments, we characterize our data glove in terms of individual sensors and the overall system. More specifically, we evaluate the system’s three modes by ① recording hand gestures and associated forces, ② improving manipulation fluency in VR, and ③ producing realistic simulation effects of various tool uses, respectively. Based on these three modes, our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments, thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

Morphology, Connectivity, and Encoding Features of Tactile and Motor Representations of the Fingers in the Human Precentral and Postcentral Gyrus.

Despite the tight coupling between sensory and motor processing for fine manipulation in humans, it is not yet totally clear which specific properties of the fingers are mapped in the precentral and postcentral gyrus. We used fMRI to compare the morphology, connectivity, and encoding of the motor and tactile finger representations (FRs) in the precentral and postcentral gyrus of 25 5-fingered participants (8 females). Multivoxel pattern and structural and functional connectivity analyses demonstrated the existence of distinct motor and tactile FRs within both the precentral and postcentral gyrus, integrating finger-specific motor and tactile information. Using representational similarity analysis, we found that the motor and tactile FRs in the sensorimotor cortex were described by the perceived structure of the hand better than by the actual hand anatomy or other functional models (finger kinematics, muscles synergies). We then studied a polydactyly individual (i.e., with a congenital 6-fingered hand) showing superior manipulation abilities and divergent anatomic-functional hand properties. The perceived hand model was still the best model for tactile representations in the precentral and postcentral gyrus, while finger kinematics better described motor representations in the precentral gyrus. We suggest that, under normal conditions (i.e., in subjects with a standard hand anatomy), the sensorimotor representations of the 5 fingers in humans converge toward a model of perceived hand anatomy, deviating from the real hand structure, as the best synthesis between functional and structural features of the hand.SIGNIFICANCE STATEMENT Distinct motor and tactile finger representations exist in both the precentral and postcentral gyrus, supported by a finger-specific pattern of anatomic and functional connectivity across modalities. At the representational level, finger representations reflect the perceived structure of the hand, which might result from an adapting process harmonizing (i.e., uniformizing) the encoding of hand function and structure in the precentral and postcentral gyrus. The same analyses performed in an extremely rare polydactyly subject showed that the emergence of such representational geometry is also found in neuromechanical variants with different hand anatomy and function. However, the harmonization process across the precentral and postcentral gyrus might not be possible because of divergent functional-structural properties of the hand and associated superior manipulation abilities.

The StarCraft Multi-Agent Exploration Challenges: Learning Multi-Stage Tasks and Environmental Factors Without Precise Reward Functions

In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Exploration Challenges(SMAC-Exp), where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. SMAC-Exp, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate a total twelve MARL algorithms under both sequential and parallel episode settings of SMAC-Exp and observe that recent approaches perform well in similar settings to the previous challenge, but we discover that current multi-agent approaches place relatively less emphasis on exploration perspectives. To a limited extent, we observe that an enhanced exploration method has a positive effect on SMAC-Exp, however there is still gap that state-of-art algorithms cannot resolve the most challenging scenarios of SMAC-Exp. Consequently, we propose a new axis for future research into Multi-Agent Reinforcement Learning studies.

Orientation control strategies and adaptation to a visuomotor perturbation in rotational hand movements.

Computational approaches to biological motor control are used to discover the building blocks of human motor behaviour. Models explaining features of human hand movements have been studied thoroughly, yet only a few studies attempted to explain the control of the orientation of the hand; instead, they mainly focus on the control of hand translation, predominantly in a single plane. In this study, we present a new methodology to study the way humans control the orientation of their hands in three dimensions and demonstrate it in two sequential experiments. We developed a quaternion-based score that quantifies the geodicity of rotational hand movements and evaluated it experimentally. In the first experiment, participants performed a simple orientation-matching task with a robotic manipulator. We found that rotations are generally performed by following a geodesic in the quaternion hypersphere, which suggests that, similarly to translation, the orientation of the hand is centrally controlled, possibly by optimizing geometrical properties of the hand's rotation. This result established a baseline for the study of human response to perturbed visual feedback of the orientation of the hand. In the second experiment, we developed a novel visuomotor rotation task in which the rotation is applied on the hand's rotation, and studied the adaptation of participants to this rotation, and the transfer of the adaptation to a different initial orientation. We observed partial adaptation to the rotation. The patterns of the transfer of the adaptation to a different initial orientation were consistent with the representation of the orientation in extrinsic coordinates. The methodology that we developed allows for studying the control of a rigid body without reducing the dimensionality of the task. The results of the two experiments open questions for future studies regarding the mechanisms underlying the central control of hand orientation. These results can be of benefit for many applications that involve fine manipulation of rigid bodies, such as teleoperation and neurorehabilitation.

The Mathematical Development of Children with Down Syndrome: The Adapted Cuisenaire Material as a Learning Facilitator

With this study, we tried to understand how the adaptation of a manipulative material can facilitate the learning of the decomposition of natural numbers in a child with Down Syndrome. Many authors defend the use of manipulative materials to teach Mathematics to children with Down Syndrome and, also, recommend the Cuisenaire material to promote the learning of mathematical initiation concepts. However, we found that the child we study presents difficulties in terms of fine manipulation and abstraction. We promoted the adaptation of the Cuisenaire material to this student and developed an intervention program of twenty sessions, divided into three phases: manipulation of the adapted Cuisenaire; manipulation of conventional Cuisenaire and the performance of exercises. In all sessions, the child worked on the decomposition of natural numbers and the exploration of the train game. After analyzing the data collected, we found that the dimensions and textures of the adapted Cuisenaire allowed us to overcome some of the difficulties felt in terms of manipulation and abstraction and facilitated the learning of decomposing natural numbers to ten. However, the adapted Cuisenaire did not facilitate the manipulation of the conventional Cuisenaire.

Remotely Manipulated Peg-in-Hole Task Conducted by Cable-Driven Parallel Robots

This article presents a remotely manipulated peg-in-hole task performed by a cable-driven parallel robot (CDPR) to facilitate the development of robotic fine manipulation in wide workspaces. CDPRs have offered a wide workspace with low inertia effects of actuator but have not addressed remote fine manipulation, thus, far. To mitigate these challenges, we developed a novel teleoperation approach that involves using identical CDPR configurations and an interactive haptic control scheme composed of global and local motions. Owing to the identical configurations, an operator can intuitively control the slave CDPR through visual feedback and admittance control containing wrench force feeling. To validate our findings, we conducted a remote peg-in-hole task in an experiment through UDP/IP communication. The results indicate that the given task was successfully completed and the required force and torque of the proposed method was reduced to 61.3% and 45.3% of the without-wrench feedback, based on our third-party force and torque measurements, respectively. The task completion time was reduced and the operator significantly adapted to the given task. The proposed method can provide a unique solution for robotic remote fine manipulation in a wide workspace with minimum control effort of an operator.