Remote Robotic System Research Articles

State Reconstruction of Remote Robotic System Using Delayed Output and Torque Estimation

State Reconstruction of Remote Robotic System Using Delayed Output and Torque Estimation

Clinical efficacy and performance evaluation of a bendable remote robot system for a bone tumour surgery: A pilot animal study.

Traditional open surgery for bone tumours sometimes has as a consequence an excessive removal of healthy bone tissue because of the limitations of rigid surgical instruments, increasing infection risk and recovery time. We propose a remote robot with a 4.5-mm diameter bendable end-effector, offering four degrees of freedom for accessing the inside of the bone and performing tumour debridement. The preclinical studies evaluated the effectiveness, clinical scenario, and usability across 12 total surgeries-six phantom surgeries and six bovine bone surgeries. Evaluation criteria included skin incision size, bone window size, surgical time, removal rate, and conversion to open surgery. Preclinical studies demonstrated that the robotic approach requires significantly smaller incision size and procedure times than traditional open curettage. This study validated the performance of the proposed system by assessing its preclinical effectiveness and optimising surgical methods using human phantom and bovine bone tumour models.

Effects of communication delay in the dual cockpit remote robotic surgery system

PurposeTo evaluate the impact of dual cockpit telesurgery on proctors and operators, and acceptable levels of processing delay for video compression and restoration.MethodsEight medical advisors and eight trainee surgeons, one highly skilled per group, performed gastrectomy, rectal resection, cholecystectomy, and bleeding tasks on pigs. Using the Medicaroid surgical robot hinotori™, simulated delay times (0 ms, 50 ms, 100 ms, 150 ms, and 200 ms) were inserted mid-surgery to evaluate the tolerance level. Operative times and dual cockpit switching times were measured subjectively using 5-point scale questionnaires (mSUS [modified System Usability Scale], and Robot Usability Score).ResultsNo significant difference was observed in operative times between proctors and operators (proctor: p = 0.247, operator: p = 0.608) nor in switching times to the dual cockpit mode (p = 0.248). For each survey setting, proctors tended to give lower ratings to delays of ≥ 150 ms. No marked difference was observed in the operator evaluations. On the postoperative questionnaires, there were no marked differences in the mSUS or Robot Usability Score between the proctors and operators (mSUS: p = 0.779, Robot Usability Score: p = 0.261).ConclusionTelesurgery using a dual cockpit with hinotori™ is practical and has little impact on surgical procedures.

Wearable teleoperation controller with 2-DoF robotic arm and haptic feedback for enhanced interaction in virtual reality.

Teleoperation is an essential component in fields such as medicine and manufacturing, enabling human operators to interact with remote robotic systems. A wearable device has been designed and manufactured to enhance sensitivity, wearability, and synchronization, providing users with the ability to experience the sensation of grasping virtual objects. The device incorporates a 2-DoF robotic arm, haptic sensors for finger gestures, and a Unity-powered virtual scene system. Its effectiveness was evaluated through user experiments, where participants were asked to rank the weights of three virtual balls and identify the direction of force applied to a virtual ball in separate tests. Additionally, the device's ability to render various shapes was also examined. The experiments showed that 73.3% of participants accurately ranked the balls by mass, and an overall correctness rate of 87.3% was achieved for force direction identification. For shape rendering, the device yielded more accurate results for simple objects like spheres, whereas rendering more complex objects such as cups and cones was challenging. The findings indicate that this wearable device has potential applications in haptic feedback and virtual reality contexts. Despite the challenges with complex shape rendering, the device shows promising capability in enhancing the user's immersive experience in virtual environments.

Remote robotic system for 3D measurement of concrete damage in tunnel with ground vehicle and manipulator

AbstractTo ensure the safety of a concrete structure, periodic and accurate inspection for damage is necessary. However, there is a high risk of inaccurate diagnosis of the structure condition because the manual inspection method is dependent on operator experience and subjective judgment. Additionally, in future society, it may become increasingly difficult to maintain high‐quality infrastructure, as the number of inspection personnel will also decrease with a decrease in population. For this purpose, the demand for accurate and automated inspection technology is increasing. To meet these issues, in this study, an objective and automated inspection method is proposed by developing an inspection robot and management system. The inspection robot comprises an artificial intelligence sensor, a manipulator, and a ground vehicle; it can autonomously drive inside a tunnel and inspect the damage on concrete surfaces using stereo vision. The management system remotely controls the robot through wireless communication and monitors the inspection status in real time. In addition, damage is automatically restored in three dimensions based on the images, and the physical size of the damage is analyzed with an average relative error of 0.39%. This method is expected to be used as a core technology for unmanned and automated tunnel maintenance in the future.

Manipulation Tasks in Hazardous Environments Using a Teleoperated Robot: A Case Study at CERN.

Remote robotic systems are employed in the CERN accelerator complex to perform different tasks, such as the safe handling of cables and their connectors. Without dedicated control, these kinds of actions are difficult and require the operators' intervention, which is subjected to dangerous external agents. In this paper, two novel modules of the CERNTAURO framework are presented to provide a safe and usable solution for managing optical fibres and their connectors. The first module is used to detect touch and slippage, while the second one is used to regulate the grasping force and contrast slippage. The force reference was obtained with a combination of object recognition and a look-up table method. The proposed strategy was validated with tests in the CERN laboratory, and the preliminary experimental results demonstrated statistically significant increases in time-based efficiency and in the overall relative efficiency of the tasks.

P67 Short-Term Cost-Effectiveness of a Remote Robotic Mechanical Thrombectomy System

Mechanical thrombectomy (MT) is a treatment for acute ischemic stroke (AIS). The clinical effect of MT for AIS patients suffering a large-vessel occlusion, is highly time-dependent. Patients admitted to a primary stroke center (PSC) currently must be transferred to a comprehensive stroke center (CSC) for a MT. A remote robotic MT system has the potential to disrupt the acute stroke care pathway by allowing the performance of MT at a PSC by an interventional radiologist physically located in a CSC.

Paper] Effects of Local and Global Adaptive Δ-Causality Control on Cooperative Work between Remote Robot Systems with Force Feedback

Paper] Effects of Local and Global Adaptive Δ-Causality Control on Cooperative Work between Remote Robot Systems with Force Feedback

Exploring Effects of Information Filtering With a VR Interface for Multi-Robot Supervision

Supervising and controlling remote robot systems currently requires many specialised operators to have knowledge of the internal state of the system in addition to the environment. For applications such as remote maintenance of future nuclear fusion reactors, the number of robots (and hence supervisors) required to maintain or decommission a facility is too large to be financially feasible. To address this issue, this work explores the idea of intelligently filtering information so that a single user can supervise multiple robots safely. We gathered feedback from participants using five methods for teleoperating a semi-autonomous multi-robot system via Virtual Reality (VR). We present a novel 3D interaction method to filter the displayed information to allow the user to read information from the environment without being overwhelmed. The novelty of the interface design is the link between Semantic and Spatial filtering and the hierarchical information contained within the multi robot system. We conducted a user study including a cohort of expert robot teleoperators comparing these methods; highlighting the significant effects of 3D interface design on the performance and perceived workload of a user teleoperating many robot agents in complex environments. The results from this experiment and subjective user feedback will inform future investigations that build upon this initial work.

Tele-operated robotic ultrasound system for medical diagnosis

Ultrasound scanning is a commonly used diagnostic method because of its high efficiency and limited side effects. However, unlike computed tomography (CT) and magnetic resonance imaging (MRI), ultrasound scanning requires professional doctors. In China, experienced ultrasound doctors are concentrated in big hospitals. Ultrasound treatment cannot be performed effectively in most rural areas. To provide equal ultrasound treatments to those areas, a novel remote robotic ultrasound scanning system is introduced in this work. The system can be divided into two parts: doctor-side and patient-side. Using our system, doctors could control the robotic arm and ultrasound machine remotely. Integrated position and force control are applied in our system to recur the hand movements of the doctor as well as maintaining flexible contact with the patient body. Real-time audio and video transmission is also realized in our system. The system can be used with the wired network, 4G, and 5G wireless network. The system was tested on 95 subjects in clinical trials. The results show that the remote ultrasound scanning task can be accomplished using our system. The ultrasound images obtained by our system have a high medical diagnostic value.

Neurobehavioral assessment of force feedback simulation in industrial robotic teleoperation

Telerobotic operation, i.e., a human operator to manipulate remote robotic systems at a distance, has started to gain its popularity in the construction industry. It is expected to help tackle operational challenges in dynamic construction workplaces. The success of telerobotic operation builds on the effective design of the human-robot interface to provide human operators with necessary senses about the remote workplaces, involving multimodal sensory cues, such as visual, audio and haptic feedback. Especially the force feedback design in telerobotic control interface is of central interest and is becoming the main feature of the bilateral control system for teleoperation, as it helps provide feedback about heavy physical interactions and processes in typical construction operations. Nonetheless, how force feedback simulation solutions affect the human operator's perceptional and behavioral reactions is less understood. This paper investigates the neurobehavioral performance of operators with a bilateral control system in a typical industrial valve operation experiment (n = 21). The experiment tested two force feedback conditions: Realistic (the system replicates the exact same feeling of the torque in valve manipulation operations) and Mediated (the simulation reduces the force on the human operator end by 50% to enable more flexible controls). The performance of the participants was evaluated via various metrics, including task performance, human performance and operational velocity uniformity. Data was collected with eye-tracking, neuroimaging (functional near-infrared spectroscopy, fNIRS), motion analysis, and NASA TLX surveys. The results indicated that the mediated force feedback in bilateral telerobotic operation helped more accurate operation, increased dual tasking, reduced cognitive load and more efficient neural functions; yet it encouraged participants to engage in more irregular actions, showing as dramatic changes in valve rotating speeds. The findings suggest that the force feedback design of telerobotic systems should be more carefully thought through to balance the advantages and disadvantages.

Robot Position Control Using Force Information for Cooperative Work in Remote Robot Systems with Force Feedback

This paper proposes robot position control using force information for cooperative work between two remote robot systems with force feedback in each of which a user operates a remote robot by using a haptic interface device while observing work of the robot with a video camera. We also investigate the effect of the proposed control by experiment. As cooperative work, we deal with work in which two robots carry an object together. The robot position control using force information finely adjusts the position of the robot arm to reduce the force applied to the object. Thus, the purpose of the control is to avoid large force so that the object is not broken. In our experiment, we make a comparison among the following three cases in order to clarify how to carry out the control effectively. In the first case, the two robots are operated manually by a user with his/her both hands. In the second case, one robot is operated manually by a user, and the other robot is moved automatically under the proposed control. In the last case, the object is carried directly by a human instead of the robot which is operated by the user in the second case. As a result, experimental results demonstrate that the control can help each system operated manually by the user to carry the object smoothly.

Remote control robot system based on predictive algorithm

Remote control robot system based on predictive algorithm

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.

Cooperative and Multimodal Capabilities Enhancement in the CERNTAURO Human–Robot Interface for Hazardous and Underwater Scenarios

The use of remote robotic systems for inspection and maintenance in hazardous environments is a priority for all tasks potentially dangerous for humans. However, currently available robotic systems lack that level of usability which would allow inexperienced operators to accomplish complex tasks. Moreover, the task’s complexity increases drastically when a single operator is required to control multiple remote agents (for example, when picking up and transporting big objects). In this paper, a system allowing an operator to prepare and configure cooperative behaviours for multiple remote agents is presented. The system is part of a human–robot interface that was designed at CERN, the European Center for Nuclear Research, to perform remote interventions in its particle accelerator complex, as part of the CERNTAURO project. In this paper, the modalities of interaction with the remote robots are presented in detail. The multimodal user interface enables the user to activate assisted cooperative behaviours according to a mission plan. The multi-robot interface has been validated at CERN in its Large Hadron Collider (LHC) mockup using a team of two mobile robotic platforms, each one equipped with a robotic manipulator. Moreover, great similarities were identified between the CERNTAURO and the TWINBOT projects, which aim to create usable robotic systems for underwater manipulations. Therefore, the cooperative behaviours were validated within a multi-robot pipe transport scenario in a simulated underwater environment, experimenting more advanced vision techniques. The cooperative teleoperation can be coupled with additional assisted tools such as vision-based tracking and grasping determination of metallic objects, and communication protocols design. The results show that the cooperative behaviours enable a single user to face a robotic intervention with more than one robot in a safer way.

Efficiency of Cooperation between Remote Robot Systems with Force Feedback-Comparison with Cooperation between User and Remote Robot System

Efficiency of Cooperation between Remote Robot Systems with Force Feedback-Comparison with Cooperation between User and Remote Robot System

Human Perception of Force in Cooperation between Remote Robot Systems with Force Feedback

Human Perception of Force in Cooperation between Remote Robot Systems with Force Feedback

Partnering with robotic technology in electrophysiology: have we arrived at a tipping point?

Bring readers up to date on the rationale for, current state of, and promising innovations in remote and robotic technology in electrophysiology. There is a growing peer-reviewed literature regarding existing nontraditional technology for mapping and ablation. There also is accelerated innovation under early evaluation that promises significant impact. The development and adoption of remote technologies in electrophysiology has faced considerable challenges yet holds tremendous promise for our patients. First principles must include benefit for patients in both safety and effectiveness, optimization of the process for providers, and sound economic and clinical justification for integration into healthcare systems. The limitations of traditional methods and tools that dominate current practice are discussed as a rationale for considering remote robotic systems. The growing library of published outcomes as well as the emergence of promising new technology merits fresh consideration.

Comparison of Stabilization Control in Cooperation between Remote Robot Systems with Force Feedback

Comparison of Stabilization Control in Cooperation between Remote Robot Systems with Force Feedback

Effect of Stabilization Control on Cooperative Work between Remote Robot Systems with Force Feedback

Effect of Stabilization Control on Cooperative Work between Remote Robot Systems with Force Feedback