Capabilities Of Robots Research Articles

EEG-based action anticipation in human-robot interaction: a comparative pilot study.

As robots become integral to various sectors, improving human-robot collaboration is crucial, particularly in anticipating human actions to enhance safety and efficiency. Electroencephalographic (EEG) signals offer a promising solution, as they can detect brain activity preceding movement by over a second, enabling predictive capabilities in robots. This study explores how EEG can be used for action anticipation in human-robot interaction (HRI), leveraging its high temporal resolution and modern deep learning techniques. We evaluated multiple Deep Learning classification models on a motor imagery (MI) dataset, achieving up to 80.90% accuracy. These results were further validated in a pilot experiment, where actions were accurately predicted several hundred milliseconds before execution. This research demonstrates the potential of combining EEG with deep learning to enhance real-time collaborative tasks, paving the way for safer and more efficient human-robot interactions.

Electromagnetic Slime Sentinels: Transforming Drug Delivery, Object Extraction, and Healing Innovations

Magnetic small soft-bodied robots are perfect for targeted medication administration, micromanipulation, and minimally invasive surgery because they provide non-invasive access to confined locations. Presently available magnetically operated small soft robots are based on elastomers (silicone) and fluids, such as ferrofluid or liquid metal; however, they have certain drawbacks. Robots built on elastomers have trouble deforming, which makes it challenging for them to maneuver in extremely constrained spaces. Although they may deform more easily, fluidbased robots have unstable forms and limited environmental adaptation. The non-Newtonian fluid- based magnetically actuated slime robots shown in this work combine the notable deformation capabilities of fluid-based robots with the flexibility of elastomer-based robots. These slime robots can move on different surfaces in intricate surroundings and navigate via tiny channels as little as 1.5 mm in diameter. They can carry out various tasks, including transporting, ingesting, and gripping solid items, and also adapt to various surfaces. Magnetic slime robots, combining the properties of non-Newtonian fluids and elastomers, offer promising solutions for targeted drug delivery and minimally invasive surgeries. This review discusses the design, preparation, and applications of magnetic slime robots, highlighting their potential to revolutionize biomedical operations. It also highlights their stability among different atmospheric conditions, marking a new age of targeted drug delivery systems and predicting various innovations and concepts related to magnetic slime robots.

A human digital twin approach for fatigue-aware task planning in human-robot collaborative assembly

Human-robot collaboration (HRC) has emerged as a pivotal paradigm in manufacturing, integrating the strengths of both human and robot capabilities. Neglecting human physical fatigue may adversely affect worker health and, in extreme cases, may lead to musculoskeletal disorders. However, human fatigue has rarely been considered for decision-making in HRC manufacturing systems. Integrating adaptive decision-making to optimise human fatigue in HRC manufacturing systems is crucial. Nonetheless, real-time perception and estimation of human fatigue and decision-making informed by human fatigue face considerable challenges. To address these challenges, this paper introduces a human digital twin method, a bidirectional communication system for physical fatigue assessment and reduction in human-robot collaborative assembly tasks. The methodology encompasses an IK-BiLSTM-AM-based surrogate model, which consists of inverse kinematics analysis (IK), bidirectional long short-term memory (BiLSTM), and attention mechanism (AM), for real-time muscle force estimation integrated with a muscle force-fatigue model for muscle fatigue assessment. An And-Or graph and optimisation model-based HRC task planner is also developed to alleviate physical fatigue via task allocation. The efficacy of this approach has been validated through proof-of-concept assembly experiments involving multiple subjects. The results show that the IK-BiLSTM-AM model achieves a minimum of 8% greater accuracy in muscle force estimation than the baseline methods. The 12-subject assessment results indicate that the task planner effectively reduces the physical fatigue of workers while performing collaborative assembly tasks.

Towards Design and Development of an MRI Conditional Robot to Enable Curvilinear Transperineal Prostate Biopsy.

In-bore MRI prostate biopsy offers improved visualisation and detection of significant prostate cancer; however, it is not widely practiced in cancer diagnosis due to its associated costs. This work introduces the first prototype towards a 7-degrees-of-freedom (DOF) MRI-conditional piezoelectrically actuated robotic system for transperineal prostate biopsy. The robot enables needle insertions in the desired trajectories. Kinematic and static models of the active needle as well as automated control of the robot are presented. It is shown that the controller can force the needle to realize the reference sine and triangular bending angles with an accuracy of 1.78 and 1.88°, respectively, in air. The trajectory tracking capability of the system in free space is shown with an RMS error of 0.86mm and a standard deviation of 0.36mm. The robot's capability to steer the needle towards target inside a phantom and extract a sample was evaluated.

Adaptive Route Memory Sequences for Insect-Inspired Visual Route Navigation

Visual navigation is a key capability for robots and animals. Inspired by the navigational prowess of social insects, a family of insect-inspired route navigation algorithms—familiarity-based algorithms—have been developed that use stored panoramic images collected during a training route to subsequently derive directional information during route recapitulation. However, unlike the ants that inspire them, these algorithms ignore the sequence in which the training images are acquired so that all temporal information/correlation is lost. In this paper, the benefits of incorporating sequence information in familiarity-based algorithms are tested. To do this, instead of comparing a test view to all the training route images, a window of memories is used to restrict the number of comparisons that need to be made. As ants are able to visually navigate when odometric information is removed, the window position is updated via visual matching information only and not odometry. The performance of an algorithm without sequence information is compared to the performance of window methods with different fixed lengths as well as a method that adapts the window size dynamically. All algorithms were benchmarked on a simulation of an environment used for ant navigation experiments and showed that sequence information can boost performance and reduce computation. A detailed analysis of successes and failures highlights the interaction between the length of the route memory sequence and environment type and shows the benefits of an adaptive method.

Benchmarking neural radiance fields for autonomous robots: An overview

Neural Radiance Field (NeRF) has emerged as a powerful paradigm for scene representation, offering high-fidelity renderings and reconstructions from a set of sparse and unstructured sensor data. In the context of autonomous robotics, where perception and understanding of the environment are pivotal, NeRF holds immense promise for improving performance. However, few survey has discussed such a potential. To fill this gap, we have collected over 200 papers since the publication of original NeRF in 2020 and present a thorough analysis of how NeRF can be used to enhance the capabilities of autonomous robots. We especially focus on the perception, localization and navigation, and decision-making modules of autonomous robots and delve into tasks crucial for autonomous operation, including 3-dimensional reconstruction, segmentation, pose estimation, simultaneous localization and mapping, navigation and planning, and interaction. Our survey meticulously benchmarks existing NeRF-based methods, comparing their reported performance, and providing insights into their strengths and limitations. Moreover, we target the existing challenges of applying NeRF in autonomous robots, including real-time processing, sparse input views, and explore promising avenues for future research and development in this domain. We especially discuss potential of integrating advanced deep learning techniques like 3-dimensional Gaussian splatting, large language models, and generative artificial intelligence. This survey serves as a roadmap for researchers seeking to leverage NeRF to empower autonomous robots, paving the way for innovative solutions that can navigate and interact seamlessly in complex environments.

SURG-26. IMPROVEMENT IN PRECISION AND SURGICAL EFFICIENCY USING THE STEALTH AUTOGUIDE™

Abstract INTRODUCTION The Stealth Autoguide™ (Medtronic) automatically determines the path for guiding instruments based on the created surgical plan. It seamlessly integrates continuous real-time navigation with robotic alignment capabilities, enhancing intraoperative workflow efficiency and supporting precise positioning. Our institution has achieved favorable results with surgeries utilizing the Stealth Autoguide™, and we report these outcomes and our future prospects. METHODS We conducted 15 surgeries at our institution using the StealthStation S8 (Medtronic) linked with the Stealth Autoguide™. The average age was 56.2 years, with 10 males and 5 females, all under general anesthesia. RESULTS The cases included 11 gliomas, 2 primary central nervous system lymphomas, 1 germ cell tumor, and 1 brain abscess. Applications of the Stealth Autoguide™ included 7 biopsies (6 burr holes, 1 craniotomy), 6 placements of fenestrations, and 2 tubing placements. The biopsies allowed for quick and reliable tissue sampling. In fenestration cases, it was possible to position quickly and precisely at the target, placing it at the planned site with high accuracy. Additionally, by using a 3D endoscopic system, surgeons could view both the surgical field and navigation screens simultaneously without shifting their gaze, enhancing safety during procedures. In the case of an intraventricular germ cell tumor, tubing preceded by ViewSite™ allowed for a safe and clear surgical field. CONCLUSION Surgeries using the Stealth Autoguide™ were conducted safely and efficiently, regardless of the surgical technique, achieving high-precision positioning. Continuing to utilize this system is expected to improve the accuracy, safety, and reproducibility of surgeries, thereby reducing the burden on patients.

Robotic Edge Intelligence for Energy-Efficient Human–Robot Collaboration

Energy-efficient human–robot collaboration poses significant challenges to the sustainable operation of production systems. Therefore, our work proposes novel robotic edge intelligence to address the issue. First, robotic edge intelligence is proposed to fully utilize the embedded computing capabilities of edge robots, and the state transition diagrams are developed for jobs, humans, and robots, respectively. Second, a multi-objective model is designed for the energy-efficient human–robot scheduling problem to evaluate the production performance and energy efficiency as a whole. Third, a heuristic algorithm is developed to search for the optimal solutions based on an artificial plant community, which is lightweight enough to be run on edge robots. Finally, a benchmark data set is developed, and a series of benchmark experiments are implemented to test the proposed system. The results demonstrate that the proposed method can effectively enhance energy efficiency and production performance with satisfying solution performance.

Human Perception of Robot Failure and Explanation During a Pick-and-Place Task

In recent years, researchers have extensively used non-verbal gestures, such as head and arm movements, to express a robot's intentions and capabilities to humans. Inspired by past research, we investigated how different explanation modalities can aid human understanding and perception of how robots communicate failures and provide explanations during block pick-and-place tasks. Through an in-person, within-subjects experiment with 24 participants, we studied four modes of explanations across four types of failures. Some of these were chosen to mimic combinations from prior work in order to both extend and replicate past findings by the community. We found that speech explanations were preferred to non-verbal and visual cues in terms of similarity to humans. Additionally, projected images had a comparable effect on explanation as other non-verbal modules. We also found consistent results with a prior online study.

Robust Shared Control With Stable Contact Servoing for Enhanced Object Transportation by Telerobotic Bimanual Mobile Manipulators

ABSTRACTBimanual mobile manipulation significantly enhances the capabilities of field robots that interact with diverse dynamic environments and expands the range of possible manipulation actions. This paper introduces a robust shared control architecture for remote manipulation of the Collaborative Dual‐arm Robot Manipulator (CURI), aimed at stable object transportation. Given that stable contact is pivotal for successful object transportation, our focus is on stabilizing the contact between the robot end‐effectors and the object using a contact servoing strategy. To maintain stable contact, it is essential that the contact wrenches consistently stay within the predefined friction cones throughout the task. We propose a novel shared control strategy to stabilize the contact using the contact parameterization model. This model formulates the contact stability manifold using a set of unconstrained reparameterized variables through a bijective mapping function, ensuring that the controller derived from these unconstrained reparameterized variables consistently maintains contact stability. The efficacy of the proposed approach is convincingly demonstrated through a series of experiments involving representative bimanual transportation tasks within a logistics context. Both theoretical analysis and experimental validations confirm that our proposed control strategy not only ensures the stable transportation of objects by CURI but also substantially enhances the overall robustness of the shared control system.

Enhancing postural stability in musculoskeletal quadrupedal locomotion through tension feedback for CPG-based controller

A central pattern generator (CPG)-based controller enhances the adaptability of quadrupedal locomotion, for example, by controlling the trunk posture. The conventional CPG-based controllers with attitude control often utilized the posture angle as feedback information. However, if the robot's body is as soft as a musculoskeletal structure, it can detect the over-tilting of the trunk based on proprioceptive information of the muscles. In general, proprioceptive information such as muscle tension changes more rapidly than posture angle information. Therefore, a feedback loop based on proprioceptive information has great potential to respond to sudden disturbances that occur during locomotion over uneven terrain. In this research, we proposed a CPG-based controller utilizing the tension of soft pneumatic artificial muscles (PAMs). Musculoskeletal quadruped robots driven by PAMs are so soft, which prevents over-tilting of the trunk because the soft leg acts like a suspension. In addition, tension, one of the proprioceptive information of PAMs, exhibits high sensitivity to changes in trunk posture because the soft body's motion easily is affected by over-tilting of the trunk. To validate the efficacy of the proposed controller, we conducted numerical simulations with a simple quadruped model and experiments with a musculoskeletal quadruped robot. As a result, the tension feedback is not effective for posture stabilization on flat terrain whereas it is effective on uneven terrain. Moreover, the tension feedback improved the running velocity over uneven terrain. These results will enhance the locomotion capability of musculoskeletal quadruped robots, advancing their practical application.

Enhancing intention prediction and interpretability in service robots with LLM and KG

The rapid advancement of artificial intelligence has significantly expanded the role of service robots in everyday life. This expansion necessitates the accurate recognition and prediction of human intentions to provide timely and appropriate services. However, existing methods often struggle to perform effectively in complex and unstructured environments. To address this challenge, we propose the Large language model and Knowledge graph based Intention Recognition Framework (LKIRF), which combines large language model (LLM) with knowledge graphs (KG) to enhance the intention recognition capabilities of service robots. Our approach constructs an offline KG from human motion and environmental data and builds an online reasoning graph through real-time interaction, utilizing LLM for interpretation. Experimental results indicate that compared to traditional methods, LKIRF not only improves prediction accuracy across various scenarios but also enhances the transparency and interpretability of the intention reasoning process.

Survey of learning-based approaches for robotic in-hand manipulation.

Human dexterity is an invaluable capability for precise manipulation of objects in complex tasks. The capability of robots to similarly grasp and perform in-hand manipulation of objects is critical for their use in the ever changing human environment, and for their ability to replace manpower. In recent decades, significant effort has been put in order to enable in-hand manipulation capabilities to robotic systems. Initial robotic manipulators followed carefully programmed paths, while later attempts provided a solution based on analytical modeling of motion and contact. However, these have failed to provide practical solutions due to inability to cope with complex environments and uncertainties. Therefore, the effort has shifted to learning-based approaches where data is collected from the real world or through a simulation, during repeated attempts to complete various tasks. The vast majority of learning approaches focused on learning data-based models that describe the system to some extent or Reinforcement Learning (RL). RL, in particular, has seen growing interest due to the remarkable ability to generate solutions to problems with minimal human guidance. In this survey paper, we track the developments of learning approaches for in-hand manipulations and, explore the challenges and opportunities. This survey is designed both as an introduction for novices in the field with a glossary of terms as well as a guide of novel advances for advanced practitioners.

Humanoid Robot-Assisted Learning for Level 1 Autistic Children: Developing Test Plans to Enhance Number Recognition Skills

Autism Spectrum Disorder (ASD) impacts the social, communication, and cognitive development of affected individuals. Level 1 autism, the mildest form of ASD, presents unique challenges in the acquisition of crucial skills such as number recognition, which serves as a foundational aspect of children's cognitive development. Recent studies have shown that humanoid robots can offer effective intervention strategies for children with autism, fostering learning in a structured, engaging, and interactive environment. This study aims to develop and evaluate test plans using humanoid robots to enhance number recognition skills in Level 1 children with autism. Based on the findings, a test plan was designed to address the unique learning needs of Level 1 children with autism in number recognition. Participants were selected based on specific criteria, ensuring a diverse and representative sample. The humanoid robot's capabilities and features were detailed, emphasizing its role in facilitating the test plan. The research methodology outlined the test plan components, implementation procedure, data collection, and analysis techniques. Results demonstrated that the humanoid robot-assisted test plan was effective in enhancing number recognition skills among Level 1 children with autism. The participants exhibited significant improvements in their abilities to identify and manipulate numbers, as well as increased engagement and motivation in learning tasks. The study also revealed that the interactive and supportive nature of the humanoid robot fostered a positive learning environment, which contributed to the observed improvements in number recognition skills. The implications of these findings extend to the broader field of autism intervention, highlighting the potential benefits of incorporating humanoid robots into educational programs for children with autism.

Artificial Intelligences in Industrial Robots: A Framework Based on Gardner’s Multiple Intelligences

Industrial robots, both in manufacturing and non-manufacturing sectors, are evolving rapidly, driven by advancements in artificial intelligence (AI). This paper presents a comprehensive survey of industrial robots, framed through the lens of Howard Gardner’s theory of multiple intelligences. By categorising various AI capabilities in industrial robots—such as visual recognition, decision-making, and collaborative interaction—based on Gardner’s intelligence framework, we provide a novel taxonomy that bridges human cognitive abilities and artificial systems. The survey explores the historical development of industrial robots, the current state of AI implementation, and future trends in robotics. Additionally, we discuss the implications of these advancements for industries and their workforce, as well as the ethical considerations surrounding the growing autonomy of AI systems. This paper aims to serve as a reference point for researchers and professionals seeking to understand the intersection of cognitive science and industrial AI, highlighting the potential and challenges of integrating AI into robotic systems.

Structural design and motion characteristics analysis of dual-mode active swing arm tracked robot

Focusing on the demand for complex terrain traversal capability of mobile robots, this paper presents a mobile and creep dual-mode tracked robot configuration with capability of autonomous extrication, in response to solving the problem of tracked robots trapped due to insufficient obstacle-crossing capability. Based on the geometrical characteristics of parallelogram and its dynamic computation on the ground, a topological configuration of mobile tracked mechanism with the ability of creep is studied. An integrated structural design method of mobile chassis that integrates the swing arm and the tracked mechanism is proposed and a dual-mode active swing arm tracked robot structural model is established. The mobile and creep characteristics of the dual-mode active swing arm tracked robot are revealed through comparative simulations in multiple working conditions; Through the prototype tests in such typical working conditions as climbing, turning and obstacle-crossing, the feasibility of creep for the robot is verified and it is proved that the obstacle-crossing ability can be effectively improved. The research results provide theoretical support for improving the adaptability and possibility of mobile robots in complex terrain.

Robotics in Disaster Management: AI-Driven Rescue, Aerial Mapping, and Emerging IoT Applications for Effective Response

This review paper explores the growing role of robotics in disaster management, focusing on the integration of artificial intelligence (AI), Internet of Things (IoT) applications, and advanced simulation environments. Highlighting lessons from events like the Fukushima nuclear disaster and Wenchuan earthquake, it examines the deployment of aerial, ground, and underwater robots in tasks such as data collection, search and rescue, and hazard monitoring. AIoT frameworks enable these robotic systems to classify objects and make decisions on-site, supporting timely and efficient responses in hazardous conditions. Virtual platforms like the DARPA Virtual Robotics Challenge, using simulators such as Gazebo and CloudSim, facilitate realistic testing of robotic capabilities in disaster scenarios, advancing operational readiness through virtual training. This review synthesizes recent progress in AI-driven disaster robotics, identifies challenges, and discusses potential improvements for future applications.

Study of Collaborative Space in Rebar Tying Robotic Systems

Abstract In the process of rebar tying operations conducted by multiple robots, the flexibility and operability of the robot’s end effector directly influence the overall performance of the rebar tying system. This reflects the crucial role of the coordinated working capability of multiple robots in the trajectory planning of the rebar tying system. Utilizing an eight degrees of freedom rebar tying system composed of two collaborative robots and a truss structure, the D-H (Denavit-Hartenberg) dynamic model for each robotic arm is established. The working area of the system is ascertained via Monte Carlo methods. Flexibility and operability are adopted as metrics to evaluate the system’s performance, subsequently leading to the formulation of a mathematical model for assessing the system’s flexibility. MATLAB is employed to delineate the robotic working range and simulate analysis of flexibility and operability when operating on the rebar tying plane, yielding an intuitive distribution of system flexibility. By analysing the distribution of flexibility and operability across the working plane at varying robot spacing intervals, an optimal robot movement distance can be determined. This ensures that during operational tasks, the rebar tying system not only possesses a larger total working area but also maintains a high level of flexibility and operability within the collaborative region, thereby fulfilling the objective of enhancing workspace utilization and improving rebar tying efficiency.

Toward a Generic Framework for Mission Planning and Execution with a Heterogeneous Multi-Robot System.

This paper presents a comprehensive framework for mission planning and execution with a heterogeneous multi-robot system, specifically designed to coordinate unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) in dynamic and unstructured environments. The proposed architecture evaluates the mission requirements, allocates tasks, and optimizes resource usage based on the capabilities of the available robots. It then executes the mission utilizing a decentralized control strategy that enables the robots to adapt to environmental changes and maintain formation stability in both 2D and 3D spaces. The framework's architecture supports loose coupling between its components, enhancing system scalability and maintainability. Key features include a robust task allocation algorithm, and a dynamic formation control mechanism, using a ROS 2 communication protocol that ensures reliable information exchange among robots. The effectiveness of this framework is demonstrated through a case study involving coordinated exploration and data collection tasks, showcasing its ability to manage missions while optimizing robot collaboration. This work advances the field of heterogeneous robotic systems by providing a scalable and adaptable solution for multi-robot coordination in challenging environments.

Robot learning on the job: Human-in-the-loop autonomy and learning during deployment

With the rapid growth of computing powers and recent advances in deep learning, we have witnessed impressive demonstrations of novel robot capabilities in research settings. Nonetheless, these learning systems exhibit brittle generalization and require excessive training data for practical tasks. To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work. In this framework, partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably; meanwhile, human operators monitor the process and intervene in challenging situations. Such a human–robot team ensures safe deployments in complex tasks. Further, we introduce a new learning algorithm to improve the policy’s performance on the data collected from the task executions. The core idea is re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning. We evaluate Sirius in simulation and on real hardware, showing that Sirius consistently outperforms baselines over a collection of contact-rich manipulation tasks, achieving an 8% boost in simulation and 27% on real hardware than the state-of-the-art methods in policy success rate, with twice faster convergence and 85% memory size reduction. Videos and more details are available at https://ut-austin-rpl.github.io/sirius/ .