Robot Experiments Research Articles

BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks.

Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.

A hierarchical long short term safety framework for efficient robot manipulation under uncertainty

Safe and efficient robot manipulation in uncertain clustered environments has been recognized to be a key element of future intelligent industrial robots. Unlike traditional robots that work in structured and deterministic environments, intelligent industrial robots need to operate in dynamically changing and stochastic environments with limited computation resources. This paper proposed a hierarchical long short term safety system (HLSTS), where the upper layer contains a long term planner for global reference trajectory generation and the lower layer contains a short term planner for real-time emergent safety maneuvers. Additionally, a hierarchical coordinator is proposed to enable smooth coordination of the two layers by compensating the communication delay through trajectory modification. The theoretical results verify that the long term planner can always find a feasible trajectory (feasibility guarantee); and the short term planner can guarantee safety in the probabilistic sense. The proposed architecture is validated in industrial settings in both simulations and real robot experiments, where the robot is interacting with randomly moving obstacles while performing a goal reaching task. Experimental results demonstrate that the proposed HLSTS framework not only guarantees safety but also improves task efficiency.

Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Mobile robot autonomous navigation in large-scale environments with crowded dynamic objects and static obstacles is still an essential yet challenging task. Recent works have demonstrated the potential of using deep reinforcement learning to enable autonomous navigation in crowds. However, only considering the human-robot interactions results in short-sighted and unsafe behaviors, and they typically use hand-crafted features and assume the global observation range, leading to large performance declines in large-scale crowded environments. Recent advances have shown the power of graph neural networks to learn local interactions among surrounding objects. In this paper, we consider autonomous navigation task in large-scale environments with crowded static and dynamic objects (such as humans). Particularly, local interactions among dynamic objects are learned for better-understanding their moving tendency and relational graph learning is introduced for aggregating both the object-object interactions and object-robot interactions. In addition, local observations are transformed into graphical inputs to achieve the scalability to various number of surrounding dynamic objects and various static obstacle patterns, and the globally guided reinforcement learning strategy is introduced to achieve the fixed-sized learning model even in large-scale complex environments. Simulation results validate our generalizability to various environments and advanced performance compared with existing works in large-scale crowded environments. In particular, our method with only local observations performs better than the benchmarks with global complete observability. Finally, physical robotic experiments demonstrate our effectiveness and practical applicability in real scenarios.

Smart Service Dimensions: Exploring Customer Experience and Participation using Stimulus-Organism-Response Model

Today, the emergence of smart service technologies has an impact on customer experience and customer participation in the service industry. Artificial intelligence (AI) appears likely to influence marketing strategies, including business models, sales processes, and customer service options, as well as customer behaviors. Technology has been regarded as a game changer in the service context. As consumers become more accustomed to smart technology and robotic services, technology will continue to play an essential role in the service industry to better serve customers. While using smart technology, customers themselves will be involved in the service process. They will participate in the service with no or limited involvement with the staff. Therefore, it is very crucial to emphasis the technology-enhanced experience from a customers’ perspective to have better understanding of the actual content and operational application of smart service and robotic experience by examining the specific dimensions of the smart service. Therefore, this proposed study aims to provide an intense discussion and comprehensive conceptualization of the smart service dimensions that may stimulate customer experience and participation by using Stimulus-Organism-Response (S-O-R) Model.

NIMS-OS: an automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science

NIMS-OS: an automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science

Robust approximate constraint following control for SCARA robots system with uncertainty and experimental validation

In the context of mechanical systems characterized by nonlinearity, uncertainty, and unknown external disturbances, this paper presents a novel and practical robust control algorithm based on the Udwadia–Kalaba (UK) equation that incorporates the ideal system trajectory as a servo constraint. Firstly, the constraint force of the system under servo constraints can be written in analytic form, which is the first part of the proposed control algorithm. The remaining parts deal with the initial condition incompatibility and the uncertainty within the system, respectively. Secondly, it is proved that the controlled system with uniform boundedness (UB) and uniform ultimate boundedness (UUB) by using the Lyapunov method. Finally, based on the SCARA robot platform, numerical simulations and experiments are performed using the rapid control prototype CSPACE. The results indicate that the proposed robust control method provides excellent dynamic performance under the constraints of the ideal trajectory. The effectiveness and practicality of the control method are further demonstrated.

High Utility Teleoperation Framework for Legged Manipulators Through Leveraging Whole-Body Control

Legged manipulators are a prime candidate for reducing risk to human lives through completing tasks in hazardous environments. However, controlling these systems in real-world applications requires a highly functional teleoperation framework, capable of leveraging all utility of the robot to complete tasks. In this work, such a teleoperation framework is presented, where a wearable whole-body motion capture suit is integrated with a whole-body controller specialised for teleoperation and a set of teleoperation strategies that enable the control of all main frames of the robot along with additional functions. Within the whole-body controller, all tasks and constraints can be configured dynamically due to their modularity, hence enabling seamless transitions between each teleoperation strategy. As a result, this not only enables the realisation of trajectories outside the workspace without the whole-body controller but also the ability to complete tasks that would require an additional manipulator if just the gripper frames of the robot were controllable. To validate the presented framework, a set of real robot experiments have been completed to demonstrate all teleoperation strategies and analyse their proficiency.

The Hierarchical Newton’s Method for Numerically Stable Prioritized Dynamic Control

This work links optimization approaches from hierarchical least-squares programming to instantaneous prioritized whole-body robot control. Concretely, we formulate the hierarchical Newton’s method which solves prioritized nonlinear least-squares problems in a numerically stable fashion even in the presence of kinematic and algorithmic singularities of the approximated kinematic constraints. These results are then transferred to control problems which exhibit the additional variability of time. This is necessary to formulate acceleration-based controllers and to incorporate the second-order dynamics. However, we show that the Newton’s method without complicated adaptations is not appropriate in the acceleration domain. We therefore formulate a velocity-based controller which exhibits second-order proportional derivative (PD) convergence characteristics. Our developments are verified in toy robot control scenarios as well as in complex robot experiments which stress the importance of prioritized control and its singularity resolution.

EMExplorer: an episodic memory enhanced autonomous exploration strategy with Voronoi domain conversion and invalid action masking

Autonomous exploration is a critical technology to realize robotic intelligence as it allows unsupervised preparation for future tasks and facilitates flexible deployment. In this paper, a novel Deep Reinforcement Learning (DRL) based autonomous exploration strategy is proposed to efficiently reduce the unknown area of the workspace and provide accurate 2D map construction for mobile robots. Different from existing human-designed exploration techniques that usually make strong assumptions about the scenarios and the tasks, we utilize a model-free method to directly learn an exploration strategy through trial-and-error interactions with complex environments. To be specific, the Generalized Voronoi Diagram (GVD) is first utilized for domain conversion to obtain a high-dimensional Topological Environmental Representation (TER). Then, the Generalized Voronoi Networks (GVN) with spatial awareness and episodic memory is designed to learn autonomous exploration policies interactively online. For complete and efficient exploration, Invalid Action Masking (IAM) is employed to reshape the configuration space of exploration tasks to cope with the explosion of action space and observation space caused by the expansion of the exploration range. Furthermore, a well-designed reward function is leveraged to guide the learning of policies. Extensive baseline tests and comparative simulations show that our strategy outperforms the state-of-the-art strategies in terms of map quality and exploration speed. Sufficient ablation studies and mobile robot experiments demonstrate the effectiveness and superiority of our strategy.

Implementation of a structured robotic colorectal curriculum for general surgery residents.

There is increasing demand for colorectal robotic training for general surgery residents. We implemented a robotic colorectal surgery curriculum expecting that it would increase resident exposure to the robotic platform and would increase the number of graduating general surgery residents obtaining a robotic equivalency certificate. The aim of this study is to describe the components of the curriculum and characterize the immediate impact of the implementation or residents. Our curriculum started in 2019 and consists of didactics, simulation, and clinical performance. Objectives are specified for both junior residents (post-graduate years [PGY]1-2) and senior residents (PGY3-5). The robotic colorectal surgical experience was characterized by comparing robotic to non-robotic operations, differences in robotic operations across post-graduate year, and percentage of graduates achieving an equivalency certificate. Robotic operations are tracked using case log annotation. From 2017 to 2021, 25 residents logged 681 major operations on the colorectal service (PGY1 mean = 7.6 ± 4.6, PGY4 mean = 29.7 ± 14.4, PGY5 mean = 29.8 ± 14.8). Robotic colorectal operations made up 24% of PGY1 (49% laparoscopic, 27% open), 35% of PGY4 (35% laparoscopic, 29% open), and 41% of PGY5 (44% laparoscopic, 15% open) major colorectal operations. Robotic bedside experience is primarily during PGY1 (PGY1 mean 2.0 ± 2.0 bedside operations vs 1.4 ± 1.6 and 0.2 ± 0.4 for PGY4 and 5, respectively). Most PGY4 and 5 robotic experience is on the console (PGY4 mean 9.1 ± 7.7 console operations, PGY5 mean 12.0 ± 4.8 console operations). Rates of robotic certification for graduating chief residents increased from 0% for E-2013 to 100% for E-2018. Our robotic colorectal curriculum for general surgery residents has facilitated earlier and increased robotic exposure for residents and increased robotic certification for our graduates.

Disentangling Reafferent Effects by Doing Nothing

An agent's ability to distinguish between sensory effects that are self-caused, and those that are not, is instrumental in the achievement of its goals. This ability is thought to be central to a variety of functions in biological organisms, from perceptual stabilisation and accurate motor control, to higher level cognitive functions such as planning, mirroring and the sense of agency. Although many of these functions are well studied in AI, this important distinction is rarely made explicit and the focus tends to be on the associational relationship between action and sensory effect or success. Toward the development of more general agents, we develop a framework that enables agents to disentangle self-caused and externally-caused sensory effects. Informed by relevant models and experiments in robotics, and in the biological and cognitive sciences, we demonstrate the general applicability of this framework through an extensive experimental evaluation over three different environments.

Surgical treatment of benign prostatic hyperplasia: Thulium enucleation versus single-port transvesical robotic simple prostatectomy.

The objective of this work is to compare our outcomes using thulium laser enucleation of prostate (ThuLEP) to the single-port robot-assisted simple prostatectomy (SP RASP) in the surgical management of benign prostatic hyperplasia (BPH). A retrospective cohort study was conducted from January 2017 through December 2021 of men who underwent SP RASP and ThuLEP performed by a single surgeon with an enucleation experience of >300 cases and extensive robotic experience. The primary outcome was changed in International Prostate Symptom Score (IPSS) postoperatively. Secondary outcomes were operative time, length of stay (LOS), change in post-void residuals (PVR), de novo stress- or urge-urinary incontinence (SUI, UUI), and rate of complications. One hundred two patients underwent surgery during the study period: 33 RASP and 69 ThuLEP. There was no difference in preoperative characteristics, including age and body mass index, between both groups. Changes in IPSS scores postoperatively were not significant between SP RASP versus ThuLEP (-17 vs. -14, p = 0.2956). SP RASP had a longer operative time (180 vs. 90 min, p < 0.0001). There was no difference in LOS (0 vs. 0days, p = 0.2904). There was no difference in change in PVR (-96 vs. -91 mL, p = 0.8504). SP RASP patients had significantly less postoperative SUI than ThuLEP (0 vs. 13 patients, p = 0.0083), while there was no difference in UUI between both groups (4 vs. 2 patients, p = 0.0843). There was no difference in 30-day complication rate (21.2% vs. 21.7%, p = 0.9517), although there were three ThuLEP patients with Clavien-Dindo Class III or higher complication. There was no difference in change in IPSS scores between the two groups. ThuLEP is associated with shorter postoperative catheter days and decreased operative times. Hospital LOS was equivalent. SP RASP demonstrates significantly improved continence rates. Though SP RASP is within the initial learning curve at our institution, early results demonstrate the role for this modality alongside ThuLEP in the treatment of large gland BPH.

Human Joint Axis Estimation Algorithm Using a Single Sensor to Recognize the Intention of the Wearer of a Wearable Robot

Wearable robots have been developed to aid or substitute human gait locomotion. To assist gait locomotion based on the intention of the wearer, gait pattern analysis is required with a wearable sensor that measures body information, such as the joint angular velocity. However, it is difficult to measure the joint angular velocity precisely because of the attachment position of a sensor that has a curvature and an anatomical human joint axis that is invisible. Therefore, a sensor calibration algorithm that aligns the sensor axis with an anatomical human joint axis is required to provide appropriate assistance to the wearer. Hence, in this study, a new and simple sensor calibration algorithm that uses a single sensor to simplify the robot system and reduce its weight is proposed. The attachment position of the sensor may change because the wearer shakes the body or collides with the ground when walking. Thus, a continuous sensor-compensation algorithm is proposed to correct for the changes in the location of the sensor. The effectiveness of the proposed algorithm is demonstrated through gait locomotion experiments on various paths. In addition to the gait locomotion experiments, the proposed algorithm is applied to a real wearable robot system. An ankle-assist wearable robot experiment reveals that the proposed algorithm can enhance the sensing and recognition performance of the robot system.

Robot versus human barista: Comparison of volatile compounds and consumers’ acceptance, sensory profile, and emotional response of brewed coffee

The increasing trend of integrating robots into the food industry has sparked debates regarding their potential influence on consumer attitudes toward food technology. This study investigated volatile compound profiles via gas chromatography–mass spectrometry (GC–MS), consumer acceptability, sensory profiling, and emotional responses of consumers toward coffee samples brewed by robot and human baristas. Moreover, the effect of the robot experience on food technology neophobia (FTN) was investigated. The principal component analysis of the volatile compound profiles revealed that the samples by the robot barista exhibited a higher degree of similarity compared to those prepared by the human barista. The range of relative standard deviations of volatile compounds from the robot barista brewed coffee was 1.4–83.1% and the variation was smaller than that of the human barista, which was 5.0–118.3%. Participants had a significant decrease in FTN scores after evaluating the robot-brewed coffee (p < 0.05), but there was no significant difference in FTN scores before and after evaluating the coffee brewed by the human barista (p > 0.05). Sensory evaluation studies revealed no significant differences in acceptability ratings and purchase intentions between the two groups (p > 0.05). However, emotional responses to the coffee samples significantly varied, with the robot-brewed coffee inducing more dynamic and positive emotions and the human-brewed coffee inducing more static and positive emotions (p < 0.05). Overall, this study provides valuable insights into consumer attitudes toward food robot service to humans and indicates that consumer’s experience with food robots may significantly reduce FTN (p < 0.001).

Self-Organized Patchy Target Searching and Collecting with Heterogeneous Swarm Robots Based on Density Interactions

The issue of searching and collecting targets with patchy distribution in an unknown environment is a challenging task for multiple or swarm robots because the targets are unevenly dispersed in space, which makes the traditional solutions based on the idea of path planning and full spatial coverage very inefficient and time consuming. In this paper, by employing a novel framework of spatial-density-field-based interactions, a collective searching and collecting algorithm for heterogeneous swarm robots is proposed to solve the challenging issue in a self-organized manner. In our robotic system, two types of swarm robots, i.e., the searching robots and the collecting robots, are included. To start with, the searching robots conduct an environment exploration by means of formation movement with Levy flights; when the targets are detected by the searching robots, they spontaneously form a ring-shaped envelope to estimate the spatial distribution of targets. Then, a single robot is selected from the group to enter the patch and locates at the patch’s center to act as a guiding beacon. Subsequently, the collecting robots are recruited by the guiding beacon to gather the patch targets; they first form a ring-shaped envelope around the target patch and then push the scattered targets inward by using a spiral shrinking strategy; in this way, all targets eventually are stacked near the center of the target patch. With the cooperation of the searching robots and the collecting robots, our heterogeneous robotic system can operate autonomously as a coordinated group to complete the task of collecting targets in an unknown environment. Numerical simulations and real swarm robot experiments (up to 20 robots are used) show that the proposed algorithm is feasible and effective, and it can be extended to search and collect different types of targets with patchy distribution.

Minimizing running buffers for tabletop object rearrangement: Complexity, fast algorithms, and applications

For rearranging objects on tabletops with overhand grasps, temporarily relocating objects to some buffer space may be necessary. This raises the natural question of how many simultaneous storage spaces, or “running buffers,” are required so that certain classes of tabletop rearrangement problems are feasible. In this work, we examine the problem for both labeled and unlabeled settings. On the structural side, we observe that finding the minimum number of running buffers (MRB) can be carried out on a dependency graph abstracted from a problem instance and show that computing MRB is NP-hard. We then prove that under both labeled and unlabeled settings, even for uniform cylindrical objects, the number of required running buffers may grow unbounded as the number of objects to be rearranged increases. We further show that the bound for the unlabeled case is tight. On the algorithmic side, we develop effective exact algorithms for finding MRB for both labeled and unlabeled tabletop rearrangement problems, scalable to over a hundred objects under very high object density. More importantly, our algorithms also compute a sequence witnessing the computed MRB that can be used for solving object rearrangement tasks. Employing these algorithms, empirical evaluations reveal that random labeled and unlabeled instances, which more closely mimic real-world setups generally have fairly small MRBs. Using real robot experiments, we demonstrate that the running buffer abstraction leads to state-of-the-art solutions for the in-place rearrangement of many objects in a tight, bounded workspace.

Is Robotic Console Time a Surrogate for Resident Operative Autonomy?

Robotic-assisted surgery is an increasing part of general surgery training, but resident autonomy on the robotic platform can be hard to quantify. Robotic console time (RCT), the percentage of time the resident controls the console, may be an appropriate measure of resident operative autonomy. This study aims to characterize the correlation between objective resident RCT and subjectively scored operative autonomy. Using a validated resident performance evaluation instrument, we collected resident operative autonomy ratings from residents and attendings performing robotic cholecystectomy (RC) and robotic inguinal hernia repair (IH) at a university-based general surgery program between 9/2020-6/2021. We then extracted RCT data from the Intuitive surgical system. Descriptive statistics, t-tests and ANOVA were performed. A total of 31 robotic operations (13 RC, 18 IH) performed by 4 attending surgeons and 8 residents (4 junior, 4 senior) were matched and included. 83.9% of cases were scored by both attending and resident. The average RCT per case was 35.6%(95% CI 13.0%,58.3%) for junior residents (PGY 2-3) and 59.7%(CI 51.1%,68.3%) for senior residents (PGY 4-5). The mean autonomy evaluated by residents was 3.29(CI 2.85,3.73) out of a maximum score of 5, while the mean autonomy evaluated by attendings was 4.12(CI 3.68,4.55). RCT significantly correlated with subjective evaluations of resident autonomy (r=0.61, p=0.0003). RCT also moderately correlated with resident training level (r=0.5306, p<0.0001). Neither attending robotic experience nor operation type significantly correlated with RCT or autonomy evaluation scores. Our study suggests that resident console time is a valid surrogate for resident operative autonomy in robotic cholecystectomy and inguinal hernia repair. RCT may be a valuable measure in objective assessment of residents' operative autonomy and training efficiency. Future investigation into how RCT correlates with subjective and objective autonomy metrics such as verbal guidance or distinguishing critical operative steps is needed to validate the study findings further.

Development and Experiment of Clamp Type Submarine Cable Inspection Robot

Relying on the research and development project of an auxiliary device for a submarine cable to cross a steel pipeline, with regard to a long-distance submarine cable crossing a pipeline, combined with a pipe-climbing robot and the laying of the submarine cable, this paper developed a detection robot that walks along the outer wall of the cable inside the submarine casing. The non-enclosed four-bar linkage mechanism is adopted, a stepper motor is used to drive a roller to walk on the submarine cable, the diameter change of the submarine cable is recorded in real-time, the damage of the submarine cable is detected when it is moved along the submarine cable, and the walking experiment is carried out. The submarine cable diameter measurement verification experiment showed that the outer wall detection robot of the submarine cable could stably travel on the submarine cable, and at the same time, could measure the real-time diameter of the submarine cable and record the actual condition of the submarine cable through video.

Improved multi-objective artificial bee colony algorithm-based path planning for mobile robots.

Mobile robots are widely used in various fields, including cosmic exploration, logistics delivery, and emergency rescue and so on. Path planning of mobile robots is essential for completing their tasks. Therefore, Path planning algorithms capable of finding their best path are needed. To address this challenge, we thus develop improved multi-objective artificial bee colony algorithm (IMOABC), a Bio-inspired algorithm-based approach for path planning. The IMOABC algorithm is based on multi-objective artificial bee colony algorithm (MOABC) with four strategies, including external archive pruning strategy, non-dominated ranking strategy, crowding distance strategy, and search strategy. IMOABC is tested on six standard test functions. Results show that IMOABC algorithm outperforms the other algorithms in solving complex multi-objective optimization problems. We then apply the IMOABC algorithm to path planning in the simulation experiment of mobile robots. IMOABC algorithm consistently outperforms existing algorithms (the MOABC algorithm and the ABC algorithm). IMOABC algorithm should be broadly useful for path planning of mobile robots.

Adaptive Asymptotic Tracking Control for Flexible-Joint Robots With Prescribed Performance: Design and Experiments

This study reports the adaptive asymptotic tracking control problem for flexible-joint (FJ) robot systems, the output tracking error can be kept within the prescribed range in the initial stage of system operation, as time approaches infinity, the asymptotic tracking result can be obtained. The prescribed performance function and the positive integrable time-varying function are introduced simultaneously in the control design of FJ robot systems for the first time. The control scheme is designed under the frame of the adaptive backstepping method and command filtered technique, which successfully avoids the problem of complexity explosion. The radial basis function neural networks are used to deal with unknown uncertainties and the adaptive laws are designed to approximate the norms of weight vectors and approximation errors. Finally, the feasibility of the proposed scheme is proved by the simulation and the experiment of the 2-link FJ robot on the Quanser platform.