Robot Operating Research Articles

A portable EEG signal acquisition system and a limited-electrode channel classification network for SSVEP

Brain-computer interfaces (BCIs) have garnered significant research attention, yet their complexity has hindered widespread adoption in daily life. Most current electroencephalography (EEG) systems rely on wet electrodes and numerous electrodes to enhance signal quality, making them impractical for everyday use. Portable and wearable devices offer a promising solution, but the limited number of electrodes in specific regions can lead to missing channels and reduced BCI performance. To overcome these challenges and enable better integration of BCI systems with external devices, this study developed an EEG signal acquisition platform (Gaitech BCI) based on the Robot Operating System (ROS) using a 10-channel dry electrode EEG device. Additionally, a multi-scale channel attention selection network based on the Squeeze-and-Excitation (SE) module (SEMSCS) is proposed to improve the classification performance of portable BCI devices with limited channels. Steady-state visual evoked potential (SSVEP) data were collected using the developed BCI system to evaluate both the system and network performance. Offline data from ten subjects were analyzed using within-subject and cross-subject experiments, along with ablation studies. The results demonstrated that the SEMSCS model achieved better classification performance than the comparative reference model, even with a limited number of channels. Additionally, the implementation of online experiments offers a rational solution for controlling external devices via BCI.

Terrain Traversability via Sensed Data for Robots Operating Inside Heterogeneous, Highly Unstructured Spaces

This paper presents a comprehensive approach to evaluating the ability of multi-legged robots to traverse confined and geometrically complex unstructured environments. The proposed approach utilizes advanced point cloud processing techniques integrating voxel-filtered cloud, boundary and mesh generation, and dynamic traversability analysis to enhance the robot’s terrain perception and navigation. The proposed framework was validated through rigorous simulation and experimental testing with humanoid robots, showcasing the potential of the proposed approach for use in applications/environments characterized by complex environmental features (navigation inside collapsed buildings). The results demonstrate that the proposed framework provides the robot with an enhanced capability to perceive and interpret its environment and adapt to dynamic environment changes. This paper contributes to the advancement of robotic navigation and path-planning systems by providing a scalable and efficient framework for environment analysis. The integration of various point cloud processing techniques into a single architecture not only improves computational efficiency but also enhances the robot’s interaction with its environment, making it more capable of operating in complex, hazardous, unstructured settings.

Comparative Study of Digital Twin Developed in Unity and Gazebo

Digital twin (DT) technology has become a cornerstone in the simulation and analysis of real-world systems, offering unparalleled insights into the lifecycle management of physical assets. By providing a real-time synchronized replica of the physical entity, DTs enable predictive maintenance, performance optimization, and lifecycle extension, which are pivotal for industries aiming for digital transformation. This paper presents a comprehensive comparative study of DT development of a robotic arm using two prominent simulation platforms: Unity and Gazebo. Unity, with its roots in the gaming industry, offers robust real-time rendering and a user-friendly interface, making it a versatile choice for various industries. Gazebo, traditionally used in robotics, provides detailed physics simulations and sensor data emulation, which is ideal for precise engineering applications. We explored the performance of both platforms in creating accurate and dynamic digital replicas. Through qualitative and quantitative analyses, this study evaluates each platform’s strengths and limitations. The study assesses these platforms across key performance metrics such as accuracy, latency, graphic quality, and integration with the Robot Operating System (ROS). The DTs were developed using a consistent physical setup and communication layer to ensure fair comparisons. The results indicate that Unity performed better in terms of accurately mimicking the robotic arm with lower latency, making it ideal for applications requiring high-fidelity visualizations and real-time responsiveness. However, Gazebo excels in its ease of ROS integration and cost-effectiveness, making it a suitable choice for smaller robotics and automation projects. This study conducts an empirical comparison of these platforms in terms of their performance in creating DTs of robotic arms which is not readily available. This paper aims to guide developers and organizations in selecting the appropriate platform for their DT initiatives, ensuring efficient resource utilization and optimal outcomes.

Are YouTube Videos Useful in Robot-Assisted Segmentectomy Education?

Segmentectomy operation became a preferable operation for small lesions due to the importance of saving lung parenchyma. Using robotic technology has too many advantages for segmentectomy operations. Websites such as YouTube have become educational tools for surgical trainees. The aim of our study is to analyze YouTube videos for accurate and up-to-date information about robotic segmentectomy operations. The videos on www.youtube.com, which were reached on July 11, 2024, by using the keywords 'Robot segmentectomy' and 'Robotic segmentectomy lung', were evaluated in this research. The videos were evaluated by using the Journal of the American Medical Association (JAMA) scoring system, critical view of safety (CVS), and LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS). Eighty-one videos were included. Almost half of the videos (n = 42) were affiliated with university hospitals. Preoperative imaging was seen in 49% of all videos; however, the rates were 32% and 20.9% for patients' demographics and preoperative assessment information, respectively. Only 29.6% of the videos presented the placement of trocars during the presentation. It has become possible to record high-quality videos easily with developing technology. However, our results showed that many of the videos don't include the parameters, especially related to education. Therefore, we believe that those videos are inadequate for trainees.

Adaptive Robot Motion Planning for Smart Manufacturing Based on Digital Twin and Bayesian Optimization-Enhanced Reinforcement Learning

Abstract Advanced motion planning is crucial for safe and efficient robotic operations in various scenarios of smart manufacturing, such as assembling, packaging and palletizing. Compared to traditional motion planning methods, Reinforcement Learning (RL) shows better adaptability to complex and dynamic working environments. However, the training of RL models is often time-consuming and the determination of well-behaved reward function parameters is challenging. To tackle these issues, we propose an adaptive robot motion planning approach based on digital twin and reinforcement learning. The core idea is to adaptively select geometry-based or RL-based method for robot motion planning through a real-time distance detection mechanism, which can reduce the complexity of RL model training and accelerate the training process. In addition, we integrate Bayesian Optimization within RL training to refine the reward function parameters. We validate the approach with a Digital Twin enabled robot system through five kinds of tasks (Pick and Place, Drawer Open, Light Switch, Button Press, Cube Push) in dynamic environments. Experiment results show that our approach outperforms traditional RL-based method with improved training speed and guaranteed task performance. This work contributes to the practical deployment of adaptive robot motion planning in smart manufacturing.

Integrating humanoid robots with human musicians for synchronized musical performances

Entertainment robotics has garnered significant attention in recent years, with researchers focusing on developing robots capable of performing a variety of tasks, including magic, drawing, dancing, and music. This article presents our research on forming a musical band that includes both humanoid robots and human musicians, with the goal of achieving natural synchronization and collaboration during musical performances. We utilized two of our humanoid robots for this project: Polaris, a mid-sized humanoid robot, as the drummer, and Oscar, a Robotis-OP3 humanoid robot, as the keyboardist. The technical implementation incorporated essential components such as visual servoing, human-robot interaction, and Robot Operating System (ROS), enabling seamless communication and coordination between the humanoid robots and the human musicians. The success of this collaborative effort can be both seen and heard through the following YouTube link: https://youtu.be/pFOyt1KKCfY?feature=shared.

Efficacy of Minimally Invasive Techniques versus Open Surgery for Recurrent Inguinal Hernias: A Meta-Analysis

Background: Recurrent inguinal hernias present difficult tasks in surgical approaches to address the issue and enhance patient prognosis. Laparoscopic and other forms of obsolete, less invasive surgery are increasingly being preferred to traditional open surgery. Objective: This meta-analysis aims to synthesize the effectiveness of minimal approach strategies compared with the open approach for recurrent inguinal hernia repair. Methods: The present study adhered to the PRISMA flowchart and databases search was conducted in PubMed, Cochrane, Embase, and Scopus. A total of 29 articles were used from the literature published between 2000 and 2024, which compared minimally invasive techniques and open surgery. Meaningfully, assessment parameters drawn from the study were recurrence rates, post-surgery pain, lengths of hospital stay, complications, and, satisfaction. Structural analysis was performed and combined ordinal data were analyzed for comparison of relative effectiveness. Results: It can be seen that, over the different parameters, minimally invasive approaches provided better results. Repeat rate was smaller (6.06% versus 11.14%), early postoperative pain (3.1 versus 5.4 days); shorter hospital stay (2.5 versus 4.8 days). In the following analysis we observed that the complication rate was significantly less in the minimally invasive group (4.2% vs 8.9%) and patient satisfaction index was slightly higher (89.4 vs 76.3). Further, cross-sectional analysis demonstrated cost effectiveness of laparoscopic techniques and precision in complicated cases offered by robotic operations. Conclusion: The results of the present study showed that minimally invasive approaches are superior to open surgery in treating recurrent inguinal hernias in terms of postoperative recurrence, recovery, and patient satisfaction. Patient and surgery based tailored treatment planning should be vital for promoting best treatment results.

基于改进RRT算法的葡萄采摘机械臂路径规划研究

The robot's operation in a grape orchard environment is often disrupted by obstacles such as vines and leaves, resulting in low fruit picking efficiency. To achieve stable obstacle avoidance, an improved RRT algorithm based on global adaptive step size and target-biased sampling was developed. First, the kinematic equations of the grape-picking robotic arm were established using the PoE method, and both forward and inverse kinematics calculations were performed to determine the robot's workspace. Then, to address the issues of lack of target orientation and other shortcomings in the traditional RRT algorithm when planning collision-free paths, dynamic updating and global adaptive step size strategies were proposed. Simulation experiments conducted using MATLAB software demonstrated that our improved RRT algorithm, compared to the RRT, RRT_informed, and RRT_star algorithms, offered advantages in terms of lower planning time, fewer sampling points, and shorter path lengths in both 2D and 3D map scenarios. Finally, grape-picking experiments were conducted in both a laboratory setting and a real orchard. The results demonstrated that the average path planning time using the proposed algorithm was shorter compared to baseline algorithms, effectively validating the efficiency and practicality of the algorithm.

Robotics and artificial intelligence

The general objective of the research was to determine the advances related to the robotics and artificial intelligence. The specific objectives of the research are to determine the countries and organizations that successfully manufacture robots, identify the characteristics of the most representative robots and the software used in their design and operation. Methodology, in this research, 61 documents have been selected, carried out in the period 2018 - 2024; including: scientific articles, review articles and information from websites of recognized organizations. Results, the software used in the design and operation of the robots are: Robot operating systems (ros), industrial robot programming software, simulation software, robot vision software, autonomous navigation software, collaborative robot (cobot) software. Conclusions, artificial intelligence has been evolving and has managed to position itself as an important technology to improve various systems. The leading robotics producers are in Germany, Japan, and Switzerland. The organizations that successfully manufacture robots are ABB Group (Switzerland), Adept Technology Inc (United States) and Apex Automation and Robotics (Australia). And the software used in the design and operation of the robots are: Robot operating systems (ros), industrial robot programming software, simulation software, robot vision software, autonomous navigation software, collaborative robot (cobot) software.

Enhancing Robot Autonomy: Path Planning via Hand-Drawn Sketches and Computer Vision Integration

This investigation delves into an advanced method for enhancing robot autonomy by integrating hand-drawn sketches with computer vision systems to optimize route planning effectively. The procedure starts with the capture of image data, which is then processed through multiple stages until the sketches are turned into exact coordinates that guide a robot. Canny edge detection, a specific computer vision technique, is employed to detect the edges of the drawn lines, facilitating precise and dependable navigation for autonomous robot operations. Initially, the image is subjected to noise reduction and edge enhancement before converting the sketch lines into Cartesian coordinates. This step is succeeded by point filtering and ordering to ascertain accurate path adherence by the robot, with no coordinates overlooked. Scale adjustments are also made to match digital image coordinates with the real-world setting, thereby improving the robot’s interaction based on the received visual inputs. The program has been tested in Robotic Operating System 2 (ROS) using MoveIt2 and also in RoboDK software, where it was used to verify the results. It is important to note that path planning, which focuses on determining an optimal route from start to finish, is a subset of motion planning, which also considers the dynamics and kinematics of the robot's movement along that route.

REMOTE INTELLIGENT ROBOT MANIPULATOR CONTROL SUBSYSTEM

The issue of expanding the functionality of an intelligent software and hardware subsystem for capturing an object by a robot manipulator was considered. It is shown that the task of expanding the functionality of the subsystem for free-form object capture by a robot manipulator with technical vision based on the operations of building a depth map from the obtained stereo image is insufficiently developed and can be solved by integrating a remote robot cntrol subsystem connected to a local network or the Internet into the robotic complex. The subsystem was developed using a Raspberry Pi microcomputer, a Stereo Pi board, a board for connecting eight Adafruit PCA9865 servos, Raspberry PI cameras, and a camera module. The structure and electrical diagram of the subsystem, a description of the software, and the main code snippet of the web application program are presented. Based on the analysis of the research results, it is established that the proposed remote control subsystem provides an extension of the functionality of the intelligent software and hardware subsystem for capturing an object by a robot manipulator, since in addition to the functions of automatic object capture, due to remote adjustment of the accuracy and manufacturability of object capture, it provides the functions of high-precision positioning of objects relative to each other and especially the effort on the robot manipulator to perform precise operations. The reliability of the object capture is also increased due to the fact that the subsystem allows for the additional involvement of a team of competent users. The proposed system increases the degree of automation, provides the possibility of moderation, collective administration, and user feedback during robot operation.

Biotissue Curriculum Translates to Performance in the Operating Room for Gastrojejunostomy and Hepaticojejunostomy in Robotic Pancreaticoduodenectomy.

Robotic simulation training curricula aim to aid surgeons in attaining robotic operating room proficiency, but the crossover success remains to be examined. A retrospective cohort study grading robotic biotissue training models and intraoperative anastomotic videos. The curriculum included deliberate practice of inanimate drills of a hepaticojejunostomy (HJ) and gastrojejunostomy (GJ). Videos were blindly reviewed, and performance was evaluated by time, errors, and Objective Structured Assessment of Technical Skills (OSATS). Spearman's correlation coefficients (ρ) were calculated for prior experience, biotissue performance, and intraoperative performance. University of Pittsburgh Medical Center from 2014 to 2018. Thirty-one surgical oncology fellows participated in the 5-step proficiency-based robotic training curriculum for robotic pancreaticoduodenectomy. Fellows completed an average of 5.1 ± 3.7 HJ and 4.3 ± 3.3 GJ on biotissue. More practice on biotissue correlated with greater improvement on both times to complete an anastomosis (ρ = -0.51) and errors (ρ = -0.45). Average errors on biotissue GJ and longer time on the last attempt correlated with lower average intraoperative GJ OSATS (ρ = -0.64; ρ = -0.66). More errors on the last biotissue GJ correlated with longer average intraoperative GJ time (ρ = 0.58). Errors on the first and average biotissue HJ errors correlated with lower OSATS for the intraoperative HJ (ρ = -0.74; ρ = -0.80). Performance on biotissue correlated with intraoperative performance. Results suggest the importance deliberate practice to achieve surgical proficiency.

The safety and feasibility of robotic pancreaticoduodenectomy: A multicenter retrospective assessment of 425 patients in Japan.

The Japanese public health insurance system has recently covered robotic pancreaticoduodenectomy (RPD). This study aimed to review the results of RPD during the introductory period and elucidate its safety and feasibility in Japan. Consecutive data of 425 patients who underwent RPD were retrospectively collected from 10 high-volume centers in Japan between April 2020 and September 2022. The primary endpoints were the rates of conversion to open surgery and completion of minimally invasive surgery (MIS), defined as the use of a totally robotic operation or combined robotic and laparoscopic procedures. Secondary endpoints were operative time, blood loss, complication rate, and 30- and 90-day mortality rates. Study comprised 222 males and 203 females, with a median age of 70 (p10-p90; 50-81) years. The conversion to open surgery and completion rates of MIS were 3.8% (16 patients) and 91.1% (387 patients), respectively. The median operative time was 617 min (p10-p90; 456-834 min), and the median volume of blood loss was 160 g (p10-p90; 30-558 g). The complication rate (Clavien-Dindo classification grade ≥ IIIa) was 20.5%. The 30- and 90-day mortality rates were 0.2% and 0.5%, respectively. Our results indicate that RPD can be introduced successfully and is a promising approach for pancreaticoduodenectomy.

Single-Sheet Separation from Paper Stack Based on Friction Uncertainty Using High-Speed Robot Hand

The successful separation of a single sheet from a stack of paper is considered a paper-handling goal when using a robot hand. Under the condition of uncertain friction coefficients, a stochastic algorithm introducing randomness is formulated, which converges a paper stack to a state of single-sheet separation through the repetition of simple robot operations. This formulation is based on the proposed motion strategy for a robotic hand, which introduces a state of a partially separated paper bundle to temporarily allow the simultaneous separation of multiple sheets and a return operation to return the paper to the original paper bundle. The experimental results indicate that a single sheet can be completely separated from a vertically standing stack of business-card-sized papers by shifting the paper in a high-speed translational movement using two fingers of the robot hand that grasp the paper from both sides.

The Luna Analog Facility testbeds (ESA, EAC): contemporary characterization work of highland (lunar) and mare (EAC-1) lunar regolith simulants

The Luna Analog Facility, a joint ESA-DLR endeavour, consists of three components and spans an area of 1,000 m2, providing testbeds of simulated lunar environments. The main sections within the facility are a large area filled with lunar mare regolith simulant resembling mare regions and a smaller, individual “Dust Chamber”. The latter replicates highland conditions and contains approximately 20 tons of material, specifically simulating the fine-particle lunar regolith portion up to 250 µm. The Dust Chamber serves as a platform for testing various technologies, such as mechanical tools, robotic operations, in-situ resource utilization activities, and astronaut attire, as well as different procedures including rover and astronaut tasks. This work represents the geotechnical, geochemical and mineralogical characterization of the Lumina Sustainable Materials Ltd. 2023 batch highland simulants, from which Lunar250 is intended for use in the Luna Dust Chamber. Additionally, this work provides new results for ESA’s mare simulant, EAC-1. We provide data on particle size distribution, particle shape, abrasivity, density, water content, major and trace element geochemistry and modal mineralogy. As the simulants in the Luna Facility will be constantly overseen, this work organized by the Vulcan Facility (ESA) intends to support the monitoring of the geotechnical property variations of the simulants over time. Ultimately, we analysed several properties with different tools to emphasize how different methods and instruments affect the variability and reliability of the results.

IoRT-Based Middleware for Heterogeneous Multi-Robot Systems

The concurrence of social robots with different functionalities and cyber-physical systems in indoor environments has recently been increasing in many fields, such as medicine, education, and industry. In such scenarios, the collaboration of such heterogeneous robots demands effective communication for task completion. The concept of the Internet of Robotic Things (IoRT) is introduced as a potential solution, leveraging technologies like Artificial Intelligence, Cloud Computing, and Mesh Networks. This paper proposes an IoRT-based middleware that allows the communication of different types of robot operating systems in dynamic environments, using a cloud-based protocol. This middleware facilitates task assignment, training, and planning for heterogeneous robots, while enabling distributed communication via WiFi. The system operates in two control modes: local and cloud-based, for flexible communication and information distribution. This work highlights the challenges of current communication methods, particularly in ensuring information reach, agility, and handling diverse robots. To demonstrate the middleware suitability and applicability, an implementation of a proof-of-concept is shown in a touristic scenario where several guide robots can collaborate by effectively sharing information gathered from their heterogeneous sensor systems, with the aid of cloud processing or even internal communication processes. Results show that the performance of the middleware allows real-time applications for heterogeneous multi-robot systems in different domains.

Collaborative Robot Teleoperation in Mixed Reality Environment for Inspection Tasks

Abstract Collaborative robots are increasing their presence in the industrial production context due to their flexibility in the tasks they can perform. However, programming and teleoperating them could be a difficult task from the perspective of an unqualified worker. In cases where the worker has to collaborate remotely with a robotic arm, the probability of collisions between the robotic arm and the environment increase. In this work, we describe the design and implementation of a mixed-reality multimodal interface for a worker to teleoperate a robotic arm by interacting with its holographic digital twin, while remaining aware of surrounding obstacles to avoid any collisions. To teleoperate the robot, we have developed a multimodal interface with which the user can communicate naturally with voice (through a semantics-based task-oriented dialogue system), and by performing manual actions on holographic objects. The application also provides audiovisual information to the user about the status of the collaborative robot as well as the actions commanded. We compute and command the behavior of the robotic arm with ROS (Robot Operating System) and “MoveIt!” (motion planning framework), and we represent computed trajectories and the real-time pose of the robot in the mixed reality environment.

Research on Real-Time Multi-Robot Task Allocation Method Based on Monte Carlo Tree Search

Task allocation is an important problem in multi-robot systems, particularly in dynamic and unpredictable environments such as offshore oil platforms, large-scale factories, or disaster response scenarios, where high change rates, uncertain state transitions, and varying task demands challenge the predictability and stability of robot operations. Traditional static task allocation strategies often struggle to meet the efficiency and responsiveness demands of these complex settings, while optimization heuristics, though improving planning time, exhibit limited scalability. To address these limitations, this paper proposes a task allocation method based on the Monte Carlo Tree Search (MCTS) algorithm, which leverages the anytime property of MCTS to achieve a balance between fast response and continuous optimization. Firstly, the centralized adaptive MCTS algorithm generates preliminary solutions and monitors the state of the robots in minimal time. It utilizes dynamic Upper Confidence Bounds for Trees (UCT) values to accommodate varying task dimensions, outperforming the heuristic Multi-Robot Goal Assignment (MRGA) method in both planning time and overall task completion time. Furthermore, the parallelized distributed MCTS algorithm reduces algorithmic complexity and enhances computational efficiency through importance sampling and parallel processing. Experimental results demonstrate that the proposed method significantly reduces computation time while maintaining task allocation performance, decreasing the variance of planning results and improving algorithmic stability. Our approach enables more flexible and efficient task allocation in dynamically evolving and complex environments, providing robust support for the deployment of multi-robot systems.

EEG-Based Mobile Robot Control Using Deep Learning and ROS Integration

Efficient BCIs (Brain-Computer Interfaces) harnessing EEG (Electroencephalography) have shown potential in controlling mobile robots, also presenting new possibilities for assistive technologies. This study explores the integration of advanced deep learning models—ASTGCN, EEGNetv4, and a combined CNN-LSTM architecture—with ROS (Robot Operating System) to control a two-wheeled mobile robot. The models were trained using a published EEG dataset, which includes signals from subjects performing thought-based tasks. Each model was evaluated based on its accuracy, F1-score, and latency. The CNN-LSTM architecture model exhibited the best performance on the cross-subject strategy with an accuracy of 88.5%, demonstrating significant potential for real-time applications. Integration with ROS was facilitated through a custom middleware, enabling seamless translation of neural commands into robot movements. The findings indicate that the CNN-LSTM model not only outperforms existing EEG-based systems in terms of accuracy but also underscores the practical feasibility of implementing such systems in real-world scenarios. Considering its efficacy, CNN-LSTM shows a great potential for assistive technology in the future. This research contributes to the development of a more intuitive and accessible robotic control system, potentially enhancing the quality of life for individuals with mobility impairments.

Optimal dimensioning of elastic-link manipulators regarding lifetime estimation

Resourceful operation and design of robots is key for sustainable industrial automation. This will be enabled by lightweight design along with time and energy optimal control of robotic manipulators. Design and control of such systems is intertwined as the control must take into account inherent mechanical compliance while the design must accommodate the dynamic requirements demanded by the control. As basis for such design optimization, a method for estimating the lifetime of elastic link robotic manipulators is presented. This is applied to the geometry optimization of flexible serial manipulators performing pick-and-place operations, where the optimization objective is a combination of overall weight and vibration amplitudes. The lifetime estimation draws from a fatigue analysis combining the rainflow counting algorithm and the method of critical cutting plane. Tresca hypothesis is used to formulate an equivalent stress, and linear damage accumulation is assumed. The final robot geometry is selected from a Pareto front as a tradeoff of lifetime and vibration characteristic. The method is illustrated for a three degrees of freedom articulated robotic manipulator.