Simulated Robot Research Articles

Different mechanisms of contextual inference govern associatively learned and sensory-evoked postural responses

Human standing balance relies on the continuous monitoring and integration of sensory signals to infer our body's motion and orientation within the environment. However, when sensory information is no longer contextually relevant to balancing the body (e.g., when sensory and motor signals are incongruent), sensory-evoked balance responses are rapidly suppressed, much earlier than any conscious perception of changes in balance control. Here, we used a robotic balance simulator to assess whether associatively learned postural responses are similarly modulated by sensorimotor incongruence and contextual relevance to postural control. Twenty-nine participants in three groups were classically conditioned to generate postural responses to whole-body perturbations when presented with an initially neutral sound cue. During catch and extinction trials, participants received only the auditory stimulus but in different sensorimotor states corresponding to their group: 1) during normal active balance, 2) while immobilized, and 3) throughout periods where the computer subtly removed active control over balance. In the balancing and immobilized states, conditioned responses were either evoked or suppressed, respectively, according to the (in)ability to control movement. Following the immobilized state, conditioned responses were renewed when balance was restored, indicating that conditioning was retained but only expressed when contextually relevant. In contrast, conditioned responses persisted in the computer-controlled state even though there was no causal relationship between motor and sensory signals. These findings suggest that mechanisms responsible for sensory-evoked and conditioned postural responses do not share a single, central contextual inference and assessment of their relevance to postural control, and may instead operate in parallel.

Novel Robotic Esophagogastric Anastomosis Simulation Model for Skill Development and Training

BackgroundEsophagogastric anastomosis is a critical step of esophagectomy. We aimed to develop a novel robotic esophagectomy simulator with high rates of fidelity and educational value for trainee surgeons to advance these skills in a low-risk setting. MethodsA porcine esophagus-stomach block was secured on a platform resembling the anatomy during an esophagectomy, and a da Vinci Xi (Intuitive Surgical) robotic system was docked above it. Participants completed 5 key steps (creating the gastric conduit, transecting the esophagus, making the gastrotomy and esophagotomy, creating the anastomosis, and sewing the common enterotomy). The model was assessed through surveys under domains of fidelity (surgical field, reality of materials, anatomy, and experience) and value as a training tool on a scale of 1 to 5 (strongly disagree to strongly agree). ResultsOf 14 participants, 8 (57.1%) were women, 9 (64.3%) were integrated cardiothoracic surgery residents, 1 (7.1%) was a thoracic-track resident, and 10 (71.4%) were in postgraduate year 4 or higher. Participants thought most aspects of the model had high fidelity, including the anatomy of conduit (4.8 ± 0.4) and proximal esophagus (4.9 ± 0.4), realism of the stomach (4.9 ± 0.4) and esophagus (4.9 ± 0.4), stapling (4.7 ± 0.6), suturing (4.8 ± 0.4), and tissue handling (4.4 ± 0.6). Participants rated the model highly overall (4.7 ± 0.5) and as a training tool (4.9 ± 0.4), with strong interrater reliability (0.69). ConclusionsThe robotic esophagogastric simulation model demonstrated high fidelity and value as a training tool, suggesting its potential effectiveness for surgeons with limited experience. However, it warrants further refinement to address limitations and to optimize its value as a training tool.

A Novel Recursive Algorithm for the Implementation of Adaptive Robot Controllers

In this paper, a novel recursive and efficient algorithm for real-time implementation of the adaptive and passivity-based controllers in model-based control of robot manipulators is proposed. Many of the previous methods on these topics involve the computation of the regressor matrix explicitly or non-recursive computations, which remains as the main challenge in practical applications. The proposed method achieves a compact and fully recursive reformulation without computing the regressor matrix or its elements. This paper is based on a comprehensive literature review of the previously proposed methods, presented in a unified mathematical framework suitable for understanding the fundamentals and making comparisons. The considered methods are implemented on several processors and their performances are compared in terms of real-time computational efficiency. Computational results show that the proposed Adaptive Newton-Euler Algorithm significantly reduces the computation time of the control law per cycle time in the implementation of adaptive control laws. In addition, using the dynamic simulation of an industrial robot with 6-DoF, trajectory tracking performances of the adaptive controllers are compared with those of non-adaptive control methods where dynamic parameters are assumed to be known.

Dynamic modeling and simulation of a torque-controlled spatial quadruped robot

AbstractEvolution has shown that legged locomotion is most adequate for tasks requiring versatile movement on land, allowing animals to traverse a wide variety of environments ranging from natural terrain to artificial, man-made landscapes with great ease. By employing well-designed control schemes, this ability could be replicated for legged robots, enabling them to be used in critical situations that still pose great danger to human integrity, such as search and rescue missions, inspection of hazardous areas, and even space exploration. This work characterizes the quadruped robot and contact dynamics that will compose our in-house simulator to be used for prototyping locomotion control schemes applied to quadruped robots. The proposed simulator computes the robot dynamics using the Recursive Newton-Euler and Composite-Rigid-Body algorithms with a few modifications to make certain aspects relevant for contact detection and control more easily accessible; furthermore, a compliant contact force method alongside stick-slip friction modeled the contact dynamics. To allow the robot to move, a simple PD-independent joint controller was implemented to track a desired leg trajectory. With the same robot and controller implemented using the MuJoCo simulation software, this work evaluates the proposed simulator by comparing characteristic locomotion signals such as the trunk pose and the ground reaction forces. Results showed similar behavior for both simulators, especially with regard to the contact detection, despite the significantly different contact models. Lastly, final remarks to enhance our simulator’s performance are suggested to be explored in future works.

A Novel Robotic Pushing and Grasping Method Based on Vision Transformer and Convolution.

Robotic grasping techniques have been widely studied in recent years. However, it is always a challenging problem for robots to grasp in cluttered scenes. In this issue, objects are placed close to each other, and there is no space around for the robot to place the gripper, making it difficult to find a suitable grasping position. To solve this problem, this article proposes to use the combination of pushing and grasping (PG) actions to help grasp pose detection and robot grasping. We propose a pushing-grasping combined grasping network (GN), PG method based on transformer and convolution (PGTC). For the pushing action, we propose a vision transformer (ViT)-based object position prediction network pushing transformer network (PTNet), which can well capture the global and temporal features and can better predict the position of objects after pushing. To perform the grasping detection, we propose a cross dense fusion network (CDFNet), which can make full use of the RGB image and depth image, and fuse and refine them several times. Compared with previous networks, CDFNet is able to detect the optimal grasping position more accurately. Finally, we use the network for both simulation and actual UR3 robot grasping experiments and achieve SOTA performance. Video and dataset are available at https://youtu.be/Q58YE-Cc250.

Design of a Robotic 3D Printing System

Abstract Additive manufacturing enables the production of complex shapes with a large degree of freedom compared to conventional manufacturing techniques such as injection molding, thermoforming or pressing. In particular, the creation of moving parts, complex structures and large-scale printing remains underexplored and research is constantly expanding. Due to design limitations related to the construction of currently available printers, 3D printing technology is increasingly being combined with robotics. Using a robot to 3D print products eliminates the need to design complex printer structures, which ultimately should simplify the process of manufacturing large-scale products and shorten the time required for preparation and production. The paper deals with the description of the designed 3D printing system and the procedure of the steps of creating a robotic simulation model with the generation of a control program for the FDM 3D printing process by an industrial robot.

Semi-decentralized Lyapunov-based formation control of multiple omnidirectional mobile robots

<p>This paper introduces an advanced formation control algorithm based on a Lyapunov approach for coordinating multiple omnidirectional mobile robots in collaborative object transport tasks. The semi-decentralized strategy ensures that the robots maintain a predefined geometric formation, crucial for stability during material transportation, and dynamically adapt to avoid collisions using onboard sensors. Experimental with a physical robot simulator demonstrates successful maintenance of line and triangle formations achieving an average side length maintenance of 1.00 meters with minimal deviation. Quantitative analysis across 30 experimental runs reveals consistent performance, with a maximum side length fluctuation of only 2 centimeters, validating the effectiveness of maintaining formation within a multi-robot system (MRS) framework. The Lyapunov-based approach proves to be an efficient method for cooperative object transport, achieving consistent performance with minimal deviation.</p>

Experimental Study on the Longitudinal Motion Performance of a Spherical Robot Rolling on Sandy Terrain

To provide the necessary theoretical models of sphere–soil interaction for the structural design, motion control, and simulation of spherical robots, this paper derives analytical expressions for traction force and driving torque when spherical robots slide and sink into sandy terrain, based on terramechanics and multibody dynamics. Furthermore, orthogonal experimental analysis identifies the load, joint angular acceleration, and maximum joint angular velocity of spherical robots as influencing factors, highlighting that the load significantly affects their longitudinal motion performance. Experimental results indicate that rolling friction and additional resistance on sandy terrain cannot be ignored. The corrected theoretical model effectively replicates the temporal variation of driving torque exerted by spherical robots on sandy terrain. Numerical computations and experimental analyses demonstrate that increasing the radius of the sphere shell, the load, and the slip ratio all lead to increased traction force and driving torque. However, traction force and driving torque begin to decrease once the slip ratio reaches approximately 0.5. Therefore, in the design of spherical robot structures and control laws, appropriate parameters such as load and slip ratio should be chosen based on the established sphere–soil interaction theoretical model to achieve high-quality longitudinal motion performance on sandy terrain.

Effects of the Human Presence among Robots in the ARIAC 2023 Industrial Automation Competition

ARIAC is a robotic simulation competition promoted by NIST annually since 2017, aiming to present competitors’ with contemporary industry problems to be solved using agile robotics. For the 2023 competition, ARIAC competitors must perform assembly and kitting tasks by controlling four autonomous ground vehicles (AGVs), one floor-based robot, and one ceiling-based (Gantry) robot in an attempt to overcome a range of agility challenges in the supplied simulated environment, itself based on the Robot Operating System (ROS 2) and Gazebo. The 2023 competition also included a “human” agility challenge, comprising a (simulated) human operator working among robots on the factory floor. This development was motivated by the fact that, while robots and automation play an increasingly significant role in modern manufacturing, there still remains a close relationship between machines and humans. They should complement each other’s strengths and cover each other’s limitations while also observing any required safety rules. For example, the ISO standard “Robots and Robotic Devices – Collaborative robots” (ISO 15066:2016) prescribes the distances required between humans and robots. Within the ARIAC simulation environment, each human operator is controlled using autonomous Belief-Desire-Intention (BDI) agents. At the same time, competitors can monitor the position of each human operator at any time by subscribing to the relevant ROS topic. In this article, we analyse the effects of this (simulated) human presence in the 2023 ARIAC competition and perform a detailed analysis of how the three different human personalities that were implemented affect the assembly tasks undertaken at the four different locations of the assembly stations. Given how the system is currently implemented, it appears that the influence of each encoded personality on the competitors is not as predictable as anticipated. We expand on why this may be a problem when addressing real collaborative spaces involving humans and industrial robots and the improvements that can be undertaken to mitigate the ensuing problems.

A novel affordable user interface for robotic surgery training: design, development and usability study.

The use of robotic systems in the surgical domain has become groundbreaking for patients and surgeons in the last decades. While the annual number of robotic surgical procedures continues to increase rapidly, it is essential to provide the surgeon with innovative training courses along with the standard specialization path. To this end, simulators play a fundamental role. Currently, the high cost of the leading VR simulators limits their accessibility to educational institutions. The challenge lies in balancing high-fidelity simulation with cost-effectiveness; however, few cost-effective options exist for robotic surgery training. This paper proposes the design, development and user-centered usability study of an affordable user interface to control a surgical robot simulator. It consists of a cart equipped with two haptic interfaces, a VR visor and two pedals. The simulations were created using Unity, which offers versatility for expanding the simulator to more complex scenes. An intuitive teleoperation control of the simulated robotic instruments is achieved through a high-level control strategy. Its affordability and resemblance to real surgeon consoles make it ideal for implementing robotic surgery training programs in medical schools, enhancing accessibility to a broader audience. This is demonstrated by the results of an usability study involving expert surgeons who use surgical robots regularly, expert surgeons without robotic surgery experience, and a control group. The results of the study, which was based on a traditional Peg-board exercise and Camera Control task, demonstrate the simulator's high usability and intuitive control across diverse user groups, including those with limited experience. This offers evidence that this affordable system is a promising solution for expanding robotic surgery training.

Simulation of street landscape design based on machine learning and entertainment design robots: An interactive entertainment design experience

Traditional design methods usually present design ideas through plans and models, but this method can not truly simulate the design effect. Therefore, this research aims to use computer game simulation and virtual robotics technology to provide a more interactive and realistic design experience. The study creates a virtual street environment with elements such as streets, buildings and vegetation through computer game simulation. Participants can move and interact in this environment through virtual robots, experience the process of street landscape design, adjust the street layout, add landscape elements, and observe the design effect in real time according to their own design intentions. Compared with traditional design methods, the experimental results show that the design experience using computer game simulation and virtual robot technology is more vivid, creative and interactive. Participants can better perceive the design effect and adjust the design scheme in real time. This interactive entertainment design experience method can help designers and participants better understand and evaluate the design scheme, and improve the design effect and participation.

On a dynamic and decentralized energy-aware technique for multi-robot task allocation

In the real world, multi-robot systems need to deal with on-the-fly (runtime) arrivals of new sets of tasks. This entails repeated adjustments of their current task allocations to include the newer ones while also ensuring that the overall performance does not degrade. This paper proposes a decentralized and distributed dynamic task allocation algorithm to handle this issue in a multi-robot scenario. The proposed work provides a conflict-free allocation of a set of tasks constituting a job to robots and minimizes the total execution time. These jobs can comprise multiple independent and/or dependent tasks or a combination thereof, which are injected on-the-fly into a network of robots. The dependent tasks of a job are related by precedence constraints that specify the ordering or dependencies between pairs of tasks. The work also describes a decentralized adaptive energy threshold mechanism for determining whether or not a robot needs to visit a battery stockpile after the execution of a task. Conflicting task selections among the robots in this decentralized set-up are resolved using mobile agents during runtime. Apart from allocating tasks to the robots, these mobile agents exploit the benefits of centralized and decentralized systems and provide an advantage over auction-based task allocation algorithms. The proposed algorithm takes into consideration the energy requirements, both during the task allocation process and actual execution. The proposed algorithm also caters to strategies to deal with delays caused by obstacles and congestion during the actual execution of the tasks. Experiments conducted using Webots, an open-source robot simulator, and Tartarus, a multi-agent platform, authenticate the efficacy of the proposed algorithm compared to other prominent task allocation algorithms in terms of minimization of average waiting time, total task allocation time, total job allocation time, and total execution time of an experiment.

A Trajectory Planning Method for Capture Operation of Space Robotic Arm Based on Deep Reinforcement Learning

Abstract In order to deal with the complex dynamics and control problems involved in space debris removal, a trajectory planning technique for a spatial robotic arm based on twin delayed DDPG (TD3) in deep reinforcement learning is proposed, and it can accomplish an end-to-end control effect comparable to that of human hand gripping objects. The trajectory planning method for capturing space debris by a floating-base space robotic arm is realized using a space robotic arm task simulation platform built on MuJoCo and using trajectory planners, trajectory trackers, and joint and end-effector control strategies formulated with seven different weighted reward functions. This makes it easier to complete spacecraft in-orbit servicing and maintenance missions. The experiment results demonstrate that the capture strategy can maintain a capture success rate of more than 99%, and debris capture can be mostly finished in three stages when taking the stability of the floating base into consideration by continuously modifying the trajectory.

Interactive Design of Stylized Walking Gaits for Robotic Characters

Procedural animation has seen widespread use in the design of expressive walking gaits for virtual characters. While similar tools could breathe life into robotic characters, existing techniques are largely unaware of the kinematic and dynamic constraints imposed by physical robots. In this paper, we propose a system for the artist-directed authoring of stylized bipedal walking gaits, tailored for execution on robotic characters. The artist interfaces with an interactive editing tool that generates the desired character motion in realtime, either on the physical or simulated robot, using a model-based control stack. Each walking style is encoded as a set of sample parameters which are translated into whole-body reference trajectories using the proposed procedural animation technique. In order to generalize the stylized gait over a continuous range of input velocities, we employ a phase-space blending strategy that interpolates a set of example walk cycles authored by the animator while preserving contact constraints. To demonstrate the utility of our approach, we animate gaits for a custom, free-walking robotic character, and show, with two additional in-simulation examples, how our procedural animation technique generalizes to bipeds with different degrees of freedom, proportions, and mass distributions.

Interactive Path Editing and Simulation System for Motion Planning and Control of a Collaborative Robot

Robots in hazardous environments demand precise and advanced motion control, making extensive simulations crucial for verifying the safety of motion planning. This paper presents a simulation system that enables interactive path editing, allowing for motion planning in a simulated collaborative robot environment and its real-world application. The system includes a simulation host, a control board, and a robot. Unity 3D on a Windows platform provides the simulation environment, while a virtual Linux environment runs ROS2 for execution. Unity sends edited motion paths to ROS2 using the Unity ROS TCP Connector package. The ROS2 MoveIt framework generates trajectories, which are synchronized back to Unity for simulation and real-world validation. To control the six-axis Indy7 collaborative robot, we used the MIO5272 embedded board as an EtherCAT master. Verified trajectories are sent to the target board, synchronizing the robot with the simulation in position and speed. Data are relayed from the host to the MIO5272 using ROS2 and the Data Distribution Service (DDS) to control the robot via EtherCAT communication. The system enables direct simulation and control of various trajectories for robots in hazardous environments. It represents a major advancement by providing safe and optimized trajectories through efficient motion planning and repeated simulations, offering a clear improvement over traditional time-consuming and error-prone teach pendant methods.

Decentralized control mechanism for limb steering in quadruped robot walking

Quadrupedal mammals adaptively change their direction, and their behavior is remarkable when they initiate such turns. They flexibly alter their interlimb coordination patterns when transitioning from forward to turning motion. However, the turning performance of quadruped robots is low compared to that of animals. We attempted to understand the control mechanism underlying the animal's flexible turning behavior by developing a simple robot model. We hypothesize that animals achieve lateral acceleration during transitions by adjusting the position of their limbs sideways at ground contact. On this basis, we proposed a decentralized control model for limb steering in quadrupedal turning behavior. The results obtained from the robot simulation demonstrated that either a turn initiated by the medial forelimb or the lateral forelimb occurred depending on the timing of the turn command. The fact that the proposed control algorithm could reproduce the behavior observed in animals suggests that a mechanism similar to the algorithm may exist in animals. Biological verification is expected in the future.

Robotic hepaticojejunostomy training in novices using robotic simulation and dry-lab suturing (ROSIM): randomized controlled crossover trial

BackgroundRobotic suturing training is in increasing demand and can be done using suture-pads or robotic simulation training. Robotic simulation is less cumbersome, whereas a robotic suture-pad approach could be more effective but is more costly. A training curriculum with crossover between both approaches may be a practical solution. However, studies assessing the impact of starting with robotic simulation or suture-pads in robotic suturing training are lacking.MethodsThis was a randomized controlled crossover trial conducted with 20 robotic novices from 3 countries who underwent robotic suturing training using an Intuitive Surgical® X and Xi system with the SimNow (robotic simulation) and suture-pads (dry-lab). Participants were randomized to start with robotic simulation (intervention group, n = 10) or suture-pads (control group, n = 10). After the first and second training, all participants completed a robotic hepaticojejunostomy (HJ) in biotissue. Primary endpoint was the objective structured assessment of technical skill (OSATS) score during HJ, scored by two blinded raters. Secondary endpoints were force measurements and a qualitative analysis. After training, participants were surveyed regarding their preferences.ResultsOverall, 20 robotic novices completed both training sessions and performed 40 robotic HJs. After both trainings, OSATS was scored higher in the robotic simulation-first group (3.3 ± 0.9 vs 2.5 ± 0.8; p = 0.049), whereas the median maximum force (N) (5.0 [3.2–8.0] vs 3.8 [2.3–12.8]; p = 0.739) did not differ significantly between the groups. In the survey, 17/20 (85%) participants recommended to include robotic simulation training, 14/20 (70%) participants preferred to start with robotic simulation, and 20/20 (100%) to include suture-pad training.ConclusionSurgical performance during robotic HJ in robotic novices was significantly better after robotic simulation-first training followed by suture-pad training. A robotic suturing curriculum including both robotic simulation and dry-lab suturing should ideally start with robotic simulation.

772 Assessing the Impact of Gaming Skills on Robotic Surgical Skill Acquisition and Learning Curves

Abstract Robotic surgery has rapidly advanced across various medical specialties over the past decade. Extensive research has confirmed the benefits of robotic surgery, leading to improved surgical outcomes and patient satisfaction. Historically, only highly experienced consultants in laparoscopic and open surgery received training in robotic surgery to the detriment of junior surgical trainees. Virtual reality simulators have introduced gamification to enhance surgical skills. Studies have shown that individuals with gaming experience, musical aptitude, and sports backgrounds exhibit improved psychomotor skills relevant to surgery, including hand-eye coordination, reaction time, and visuospatial awareness. Despite this, there is a lack of research on how gaming and musical abilities impact robotic surgical skill acquisition and learning curves. This prospective, observational, and comparative study involved 15 novice surgeons with no prior robotic experience recruited from various UK medical institutions. Participants were divided into two groups based on their video game experience. While statistical significance was not reached (p 0.26), an observed trend favoured the gaming cohort's performance (M 56.29, SD 57.85) on the robotic surgical simulator compared to the non-gamer group (M 38.41, SD 46.77) possibly due to the limited sample size. Gamers exhibited notably superior reaction time scores (p 0.037) Additionally, gamers demonstrated enhanced overall performance in spatial reasoning and the Beat Saber game, assessing hand-eye coordination, though these differences did not reach statistical significance. This study suggests that video gaming experience may provide novice surgeons with certain advantages when beginning training on the high-fidelity dVSS robotic surgical simulator. Further research with larger participant groups may confirm these findings.

A self-adaptive agent for flexible posture planning in robotic milling system

In recent years, industrial robots are widely used in the milling processes of the fundamental components such as aeronautic parts, aerospace cabins and blades on the marine propellers of the worships, because of their flexible structure, large operating space, and robust capacity of rapid reconfiguration. Along with the robotic machining process, multiple types of geometrical and physical constraints display the time-varying characteristics, which are difficult to be considered simultaneously by the traditional static planning methods on the robotic milling posture. Therefore, a deep reinforcement learning (DRL) enhanced virtual repulsive potential field flexible model is proposed for the robotic milling posture dynamic planning. The preliminary planning of static milling posture is realized based on the fixed setting of the virtual modal parameters in the posture planning model, which is treated as a theory preparation for training the deep reinforcement learning agent. To develop the static planning model, multiple constraints are established by considering the performances of geometric, kinematic and machining precision simultaneously. Furthermore, a reward function is constructed for the dynamic agent, comprehensively combining the long and short reward benefits in nonlinear correlation with virtual repulsive forces. By taking the virtual modal parameters as the action space of the dynamic agent, a DRL enhanced agent VRPF-SAC is created and trained under the dynamic milling environment from virtual repulsive planning field. Four tasks of large size robotic milling simulations and experiments are designed and developed to validate the effectiveness of the flexible planning model. In comparison with the static model, the flexible planning model displays the excellent global effects covering the kinematic, and precision performance, with 65.78 %, 72.04 %, and 70.25 % reduction in velocity, acceleration, and jerk of robotic joints, and ensuring the effective precision performance from −0.365 to 0.283 mm of large size workpiece with large curvature changes. The proposed flexible posture planning architecture can provide a basic for robotic dynamic milling of core parts with large size, narrow space and sharp curvatures of the aeronautic parts, aerospace cabins and worship propellers.

Beyond simulation: Unlocking the frontiers of humanoid robot capability and intelligence with Pepper's open-source digital twin

This research paper presents a high-fidelity, open-source digital-twin of the Pepper robot developed within the framework of the Robot Operating System 2 (ROS 2) for better simulation realism in complex tasks of machine learning. We developed a dedicated, custom ROS 2 package with modern simulation tools, such as Gazebo Sim, MoveIt 2, Rviz2, that brings complete, realistic environments in line with the exact behaviors and interactions of robots in reality. Better accuracy of the physical movement of Pepper robot's simulation was shown on the digital twin, validated by the Choregraphe software and real robot performance, to be a strong platform of collaboration and further research by the community. This development greatly pushes the envelope of human-like humanoid robotics further by offering a scaled, flexible, and plausible training environment conducive to integrating complex algorithms of robot learning and interaction capabilities.