Telemanipulation Research Articles

Brain functional connectivity under teleoperation latency: a fNIRS study

IntroductionLong-distance robot teleoperation faces high latencies that pose cognitive challenges to human operators. Latency between command, execution, and feedback in teleoperation can impair performance and affect operators’ mental state. The neural underpinnings of these effects are not well understood.MethodsThis study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. A human subject experiment (n = 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed.ResultsThe presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and best performance in no latency condition, which echoes the neural activity patterns.ConclusionThe study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.

Programmable Robotic Shape Shifting and Color Morphing Dynamics Through Magneto‐Mechano‐Chromic Coupling

AbstractRecreating natural organisms’ dynamic shape‐morphing and adaptive color‐changing capabilities in a compact structure poses significant challenges but unlocks unprecedented hybridized robotic‐visual applications. Overcoming programmability and predictability obstacles is key to achieving real‐time, responsive changes in appearance and functionality, enhancing robot‐environment‐user interactions in ways previously unattainable. Herein, a Soft Magneto‐Mechano‐Chromic (SoMMeC) structure comprising a magnetic actuating and a synthetic photonic film, mirroring the intricate color‐tuning mechanism of chameleons is devised. A model combining numerical simulation and a strain‐dependent color evolution map enables precise predictions and controllable shape‐color alterations across various geometrical and magnetization profiles. The SoMMeC exhibits rapid (0.1s), broad (full‐visible spectrum), tether‐free (remote magnetic manipulation), and programmable (model‐guided control) color transformations, surpassing traditional limitations with its real‐time response, broad and omnidirectional coloration for enhanced visibility, and robustness against external disturbances. The SoMMeC translates into dynamic advertising iridescence, adaptive camouflage, self‐sensing, and multi‐level encryption, and transcends traditional robotics by seamlessly blending dynamic movement with nuanced visual changes. It opens up a spectrum of applications that redefine robotic functionality through dynamic appearance modulation, making robotic systems more versatile, adaptive, and suitable for unexplored integrative functions.

Robust Shared Control With Stable Contact Servoing for Enhanced Object Transportation by Telerobotic Bimanual Mobile Manipulators

ABSTRACTBimanual mobile manipulation significantly enhances the capabilities of field robots that interact with diverse dynamic environments and expands the range of possible manipulation actions. This paper introduces a robust shared control architecture for remote manipulation of the Collaborative Dual‐arm Robot Manipulator (CURI), aimed at stable object transportation. Given that stable contact is pivotal for successful object transportation, our focus is on stabilizing the contact between the robot end‐effectors and the object using a contact servoing strategy. To maintain stable contact, it is essential that the contact wrenches consistently stay within the predefined friction cones throughout the task. We propose a novel shared control strategy to stabilize the contact using the contact parameterization model. This model formulates the contact stability manifold using a set of unconstrained reparameterized variables through a bijective mapping function, ensuring that the controller derived from these unconstrained reparameterized variables consistently maintains contact stability. The efficacy of the proposed approach is convincingly demonstrated through a series of experiments involving representative bimanual transportation tasks within a logistics context. Both theoretical analysis and experimental validations confirm that our proposed control strategy not only ensures the stable transportation of objects by CURI but also substantially enhances the overall robustness of the shared control system.

Design and implementation of corrosion-resistant multitasking cell stainers.

Compared with machine staining, traditional manual staining faces various problems, such as a low preparation success rate, low efficiency, and harm to the human body due to corrosive gases. Therefore, a stainer that is of low cost, has strong corrosion resistance, and is suitable for small-batch preparation should be developed. In this study, by choosing a rotary scheme as the structural basis, a reusable container cover and a master-slave manipulator cooperation scheme are developed, which greatly improve the space utilization rate. Through material selection and structural design, in the designed stainer, effective protection against strong acids with high volatility and permeability is realized, thereby eliminating the corrosion issue. Using the designed splitting-running algorithm for the dyeing procedure, simultaneous multistaining is realized, which significantly improves the staining efficiency. Compared with large-sized stainers, the cost of the proposed stainer is very low, which will help popularize early screening for cervical cancer in low- and middle-income countries.

Multifunctional Magnetic Muscles for Soft Robotics

Despite recent advancements, artificial muscles have not yet been able to strike the right balance between exceptional mechanical properties and dexterous actuation abilities that are found in biological systems. Here, we present an artificial magnetic muscle that exhibits multiple remarkable mechanical properties and demonstrates comprehensive actuating performance, surpassing those of biological muscles. This artificial muscle utilizes a composite configuration, integrating a phase-change polymer and ferromagnetic particles, enabling active control over mechanical properties and complex actuating motions through remote laser heating and magnetic field manipulation. Consequently, the magnetic composite muscle can dynamically adjust its stiffness as needed, achieving a switching ratio exceeding 2.7 × 10³. This remarkable adaptability facilitates substantial load-bearing capacity, with specific load capacities of up to 1000 and 3690 for tensile and compressive stresses, respectively. Moreover, it demonstrates reversible extension, contraction, bending, and twisting, with stretchability exceeding 800%. We leverage these distinctive attributes to showcase the versatility of this composite muscle as a soft continuum robotic manipulator. It adeptly executes various programmable responses and performs complex tasks while minimizing mechanical vibrations. Furthermore, we demonstrate that this composite muscle excels across multiple mechanical and actuation aspects compared to existing actuators.

Cybersecurity in Autonomous Vehicles—Are We Ready for the Challenge?

The rapid development and deployment of autonomous vehicles (AVs) present unprecedented opportunities and challenges in the transportation sector. While AVs promise enhanced safety, efficiency, and convenience, they also introduce significant cybersecurity vulnerabilities due to their reliance on advanced electronics, connectivity, and artificial intelligence (AI). This review examines the current state of cybersecurity in autonomous vehicles, identifying major threats such as remote hacking, sensor manipulation, data breaches, and denial of service (DoS) attacks. It also explores existing countermeasures including intrusion detection systems (IDSs), encryption, over-the-air (OTA) updates, and authentication protocols. Despite these efforts, numerous challenges remain, including the complexity of AV systems, lack of standardization, latency issues, and resource constraints. This review concludes by highlighting future directions in cybersecurity research and development, emphasizing the potential of AI and machine learning, blockchain technology, industry collaboration, and legislative measures to enhance the security of autonomous vehicles.

Plasmonic-Driven Regulation of Biomolecular Activity In Situ.

Selective and remote manipulation of activity for biomolecules, including protein, DNA, and lipids, is crucial to elucidate their molecular function and to develop biomedical applications. While advances in tool development, such as optogenetics, have significantly impacted these directions, the requirement for genetic modification significantly limits their therapeutic applications. Plasmonic nanoparticle heating has brought new opportunities to the field, as hot nanoparticles are unique point heat sources at the nanoscale. In this review, we summarize fundamental engineering problems such as plasmonic heating and the resulting biomolecular responses. We highlight the biological responses and applications of manipulating biomolecules and provide perspectives for future directions in the field.

Delivery and utilization of photo‐energy for temperature control using a light‐driven microfluidic control device at −40 °C

AbstractLow‐temperature energy harvest, delivery, and utilization pose significant challenges for thermal management in extreme environments owing to heat loss during transport and difficulty in temperature control. Herein, we propose a light‐driven photo‐energy delivery device with a series of photo‐responsive alkoxy‐grafted azobenzene‐based phase‐change materials (a‐g‐Azo PCMs). These a‐g‐Azo PCMs store and release crystallization and isomerization enthalpies, reaching a high energy density of 380.76 J/g even at a low temperature of −63.92 °C. On this basis, we fabricate a novel three‐branch light‐driven microfluidic control device for distributed energy recycling that achieves light absorption, energy storage, controlled movement, and selective release cyclically over a wide range of temperatures. The a‐g‐Azo PCMs move remote‐controllably in the microfluidic device at an average velocity of 0.11–0.53 cm/s owing to the asymmetric thermal expansion effect controlled by the temperature difference. During movement, the optically triggered heat release of a‐g‐Azo PCMs achieves a temperature difference of 6.6 °C even at a low temperature of −40 °C. These results provide a new technology for energy harvest, delivery, and utilization in low‐temperature environments via a remote manipulator.

Portable Mass Spectrometry for On-site Detection of Hazardous Volatile Organic Compounds via Robotic Extractive Sampling.

Various hazardous volatile organic compounds (VOCs) are frequently released into environments during accidental events that cause many hazards to ecosystems and humans. Therefore, rapid, sensitive, and on-site detection of hazardous VOCs is crucial to understand their compositions, characteristics, and distributions in complex environments. However, manual handling of hazardous VOCs remains a challenging task, because of the inaccessible environments and health risk. In this work, we designed a quadruped robotic sampler to reach different complex environments for capturing trace hazardous VOCs using a needle trap device (NTD) by remote manipulation. The captured samples were rapidly identified by portable mass spectrometry (MS) within minutes. Rapid detection of various hazardous VOCs including toxicants, chemical warfare agents, and burning materials from different environments was successfully achieved using this robot-MS system. On-site detection of 83 typical hazardous VOCs was examined. Acceptable analytical performances including low detection limits (at subng/mL level), good reproducibility (relative standard deviation (RSD) < 20%, n = 6), excellent quantitative ability (R2 > 0.99), and detection speed (within minutes) were also obtained. Our results show that the robot-MS system has excellent performance including safety, controllability, applicability, and robustness under dangerous chemical conditions.

Magoctopus: Bio-inspired small-scale magnetic soft octopus with programmable shape morphing and motions

Microscale magnetic soft robots, with their tetherless propulsion capability, non-invasive interaction with the environment, and high adaptability to confined and unstructured spaces, hold tremendous promise in the field of biomedicine. These robots can navigate flexibly within narrow internal environments, such as blood vessels and organs, under remote magnetic field manipulation. However, the lack grasping capabilities in most existing microscale magnetic soft robots, has restricted their utility in medical applications. Inspired by the movement and predatory behavior of octopuses in the ocean, we have developed a microscale biomimetic magnetic octopus (Magoctopus), capable of mimicking biological functions such as movement and object grasping in liquid environments. The Magoctopus, with a total length of 17.30 mm and weight of 0.24 g, features a polystyrene head and six tentacles made of VHB (Very hard bond) material embedded with SmFeN powder for magnetic programming. By utilizing horizontally alternating magnetic fields, the six tentacles of Magoctopus can exhibit pre-programmed bending deformations to mimic the motion patterns of octopus tentacles. Furthermore, the Magoctopus has demonstrated capabilities in cargo delivery, targeted drug delivery, and other potential application, making it suitable for transportation within complex environments, biomedical applications, and beyond.

Structural color control by remote electrical manipulation of chiral dopant molecules in a chiral nematic phase

Abstract 1,2-Bis((cholesteryloxy)octyloxy)-4-nitrobenzene was synthesized as a chiral dopant that exhibited electroresponsiveness in nematic liquid crystal (LC) phases. Even in a nonelectroresponsive host nematic LC, the helical pitch of the chiral nematic phase, generated by doping the chiral dopant molecules, could be altered in response to an external electric field, thus controlling the selective reflection color.

Group Control of Photo-Responsive Colloidal Motors with a Structured Light Field

Using structured light to drive colloidal motors, due to its advantages of remote manipulation, energy tunability, programmability, and the controllability of spatiotemporal distribution, has been attracting much attention in the fields of targeted drug delivery, environmental control, chemical agent detection, and smart device design. Here, we focus on studying the group control of colloidal motors made from a photo-responsive organic polymer molecule NO-COP (N,O-Covalent organic polymer). These colloidal motors mainly respond to light intensity patterns. Considering its merits of fast refreshing speed, good programmability, and high-power threshold, we chose a digital micromirror device (DMD) to modulate the structured light field shining on the sample. It was found that under ultraviolet or green light modulation, such colloidal motors exhibit various group behaviors including group spreading, group patterning, and group migration. A qualitative interpretation is also provided for these observations.

A comprehensive approach to characterize navigation instruments for magnetic guidance in biological systems

Achieving non-invasive spatiotemporal control over cellular functions, tissue organization, and behavior is a desirable aim for advanced therapies. Magnetic fields, due to their negligible interaction with biological matter, are promising for in vitro and in vivo applications, even in deep tissues. Particularly, the remote manipulation of paramagnetic (including superparamagnetic and ferromagnetic, all with a positive magnetic susceptibility) entities through magnetic instruments has emerged as a promising approach across various biological contexts. However, variations in the properties and descriptions of these instruments have led to a lack of reproducibility and comparability among studies. This article addresses the need for standardizing the characterization of magnetic instruments, with a specific focus on their ability to control the movement of paramagnetic objects within organisms. While it is well known that the force exerted on magnetic particles depends on the spatial variation (gradient) of the magnetic field, the magnitude of the field is often overlooked in the literature. Therefore, we comprehensively analyze and discuss both actors and propose a novel descriptor, termed ‘effective gradient’, which combines both dependencies. To illustrate the importance of both factors, we characterize different magnet systems and relate them to experiments involving superparamagnetic nanoparticles. This standardization effort aims to enhance the reproducibility and comparability of studies utilizing magnetic instruments for biological applications.

Hybrid Hydrophilic Nanofiber Membrane for Photodynamic Inactivation

We present a novel approach to creating nanofiber membranes with antibacterial, photocatalytic, and magnetic properties. Using electrospinning, we functionalized the membranes with tetraphenylporphyrin to produce singlet oxygen when exposed to visible light. We also added maghemite nanoparticles stabilized by polyethylenimine to enable remote manipulation of the membranes by a magnetic field. The resulting material demonstrated strong antibacterial effects against E. coli and improved photothermal properties upon short, 10-minute light irradiation, as well as the ability to produce hydrogen peroxide after irradiation. The magnetic properties of the membranes also make them suitable for use in hazardous or highly contaminated environments. These multifunctional nanofiber membranes have potential applications in fields such as biomedicine, environmental science, and industrial filtration.

Exploring Vulnerabilities and Attack Vectors Targeting Pacemaker Devices in Healthcare

This technical paper investigates the vulnerabilities and potential threats posed by emerging technologies, specifically Bluetoothenabled patient pacemakers. With the advancements in healthcare technology, pacemakers now utilize Bluetooth connectivity for real-time monitoring and data transmission, offering patients and healthcare providers an important convenience. However, this technology also introduces significant security risks, leaving these life-sustaining devices susceptible to malicious attacks. Through an in-depth analysis of existing research, real-life incidents, and vulnerabilities identified by experts in the field, this paper will underscore the critical vulnerabilities present in pacemaker systems. Examples, including findings from researchers such as Billy Rios, Jonathon Butts, and Marie Moe, demonstrate the potential severity of these vulnerabilities. From remote control manipulation to unauthorized access to sensitive medical data, the threats posed by these vulnerabilities are substantial and potentially life-threatening. Moreover, this paper outlines advanced mitigation strategies essential for protecting patient pacemakers against these security risks. Recommendations include end-to-end encryption, whitelist device pairing, intrusion detection systems, and regular firmware updates, highlight the collaborative efforts required from patients, healthcare providers, and manufacturers to mitigate these risks effectively. This paper’s findings underscore the urgent need for robust cybersecurity measures in the design, implementation, and maintenance of pacemaker systems. Addressing these vulnerabilities is key for ensuring patient safety, maintaining privacy, and building trust in healthcare technology. The implications of this research extend beyond pacemaker security, emphasizing the broader importance of cybersecurity in medical devices and the importance of ongoing research and regulatory initiatives to protect patient health.

Impact of Haptic Feedback in High Latency Teleoperation for Space Applications

Remote manipulation is a key enabler for upcoming space activities such as in-orbit servicing and manufacture (IOSM). However, due to the large distances involved, these systems encounter unavoidable signal delays which can lead to poor performance and users adopting a disjointed, ‘move-and-wait’ style of operation. We use a robot arm teleoperated with a haptic controller to test the impact of haptic feedback on delayed (up to 2.6 s: Earth-Moon communications) teleoperation performance for two example IOSM-style tasks. This user study showed that increased latency reduced performance in all of metrics recorded. In real-time teleoperation, haptic feedback showed improvements in success rate, accuracy, contact force, velocity, and trust, but, of these, only the improvements to contact forces and moving velocity were also seen at higher latencies. Accuracy and trust improvements were lost, or even reversed, at higher latencies. Results varied between the two tasks, highlighting the need for further research into the range of task types to be encountered in teleoperated space activities. This study also provides a framework by which to explore how features other than haptic feedback can impact both performance and trust in delayed teleoperation.

Magnetic-Actuated Jumping of Droplets on Superhydrophobic Grooved Surfaces: A Versatile Strategy for Three-Dimensional Droplet Transportation.

On-demand droplet transportation is of great significance for numerous applications. Although various strategies have been developed for droplet transportation, out-of-surface three-dimensional (3D) transportation of droplets remains challenging. Here, a versatile droplet transportation strategy based on magnetic-actuated jumping (MAJ) of droplets on superhydrophobic grooved surfaces (SHGSs) is presented, which enables 3D, remote, and precise manipulation of droplets even in enclosed narrow spaces. To trigger MAJ, an electromagnetic field is utilized to deform the droplet on the SHGS with the aid of an attached magnetic particle, thereby the droplet acquires excess surface energy. When the electromagnetic field is quickly removed, the excess surface energy is partly converted into kinetic energy, allowing the droplet to jump atop the surface. Through high-speed imaging and numerical simulation, the working mechanism and size matching effect of MAJ are unveiled. It is found that the MAJ behavior can only be observed if the sizes of the droplets and the superhydrophobic grooves are matched, otherwise unwanted entrapment or pinch-off effects would lead to failure of MAJ. A regime diagram which serves as a guideline to design SHGSs for MAJ is proposed. The droplet transportation capacities of MAJ, including in-surface and out-of-surface directional transportation, climbing stairs, and crossing obstacles, are also demonstrated. With the ability to remotely manipulate droplets in enclosed narrow spaces without using any mechanical moving parts, MAJ can be used to design miniaturized fluidic platforms, which exhibit great potential for applications in bioassays, microfluidics, droplet-based switches, and microreactions.

Reconfigurable Microlens Array Enables Tunable Imaging Based on Shape Memory Polymers.

Microlens arrays (MLAs) with a tunable imaging ability are core components of advanced micro-optical systems. Nevertheless, tunable MLAs generally suffer from high power consumption, an undeformable rigid body, large and complex systems, or limited focal length tunability. The combination of reconfigurable smart materials with MLAs may lead to distinct advantages including programmable deformation, remote manipulation, and multimodal tunability. However, unlike photopolymers that permit flexible structuring, the fabrication of tunable MLAs and compound eyes (CEs) based on transparent smart materials is still rare. In this work, we report reconfigurable MLAs that enable tunable imaging based on shape memory polymers (SMPs). The smart MLAs with closely packed 200 × 200 microlenses (40.0 μm in size) are fabricated via a combined technology that involves wet etching-assisted femtosecond laser direct writing of MLA templates on quartz, soft lithography for MLA duplication using SMPs, and the mechanical heat setting for programmable reconfiguration. By stretching or squeezing the shape memory MLAs at the transition temperature (80 °C), the size, profiles, and spatial distributions of the microlenses can be programmed. When the MLA is stretched from 0 to 120% (area ratio), the focal length is increased from 116 to 283 μm. As a proof of concept, reconfigurable MLAs and a 3D CE with a tunable field of view (FOV, 160-0°) have been demonstrated in which the thermally triggered shape memory deformation has been employed for tunable imaging. The reconfigurable MLAs and CEs with a tunable focal length and adjustable FOV may hold great promise for developing smart micro-optical systems.

Programming crack patterns with light in colloidal plasmonic films

Crack formation observed across diverse fields like geology, nanotechnology, arts, structural engineering or surface science, is a chaotic and undesirable phenomenon, resulting in random patterns of cracks generally leading to material failure. Limiting the formation of cracks or “programming” the path of cracks is a great technological challenge since it holds promise to enhance material durability or even to develop low cost patterning methods. Drawing inspiration from negative phototropism in plants, we demonstrate the capability to organize, guide, replicate, or arrest crack propagation in colloidal films through remote light manipulation. The key consists in using plasmonic photothermal absorbers to generate “virtual” defects enabling controlled deviation of cracks. We engineer a dip-coating process coupled with selective light irradiation enabling simultaneous deposition and light-directed crack patterning. This approach represents a rare example of a robust self-assembly process with long-range order that can be programmed in both space and time.

Droplet manipulation of smart ferrofluid on covalently grafted slippery surface

Remote manipulation of droplets plays an important role in fields such as energy, bioanalysis and microfluidics. Despite efforts to develop many surfaces that respond to external stimuli for droplet manipulation, not all surfaces are responsive in practical applications, making droplet manipulation difficult to achieve. Manipulating droplets on unresponsive surfaces presents a formidable challenge. Our study unveils a covalently grafted slippery surface (CGSS) that is capable of proficiently manipulating ferrofluid droplets in the presence of a magnetic field. By adjusting the strength of the magnetic field applied, a horizontal CGSS can achieve efficient, long-range, and high-speed droplet transportation. On the tilted CGSS, droplets can be transported and aggregated against gravity. Magnetic field guidance allows for droplets to be programmatically guided along a desired trajectory. This research offers feasible strategies for developing new methods to manipulate droplets, which are anticipated to have significant applications in fields like microfluidics, drug delivery, and intelligent robotics.