Robotic System Research Articles

Body Movement Mirroring and Synchrony in Human–Robot Interaction

This review article provides an overview of papers that have studied body movement mirroring and synchrony within the field of human-robot interaction. The papers included in this review cover system studies, which focus on evaluating the technical aspects of mirroring and synchrony robotic systems, and user studies, which focus on measuring particular interaction outcomes or attitudes towards robots expressing mirroring and synchrony behaviors. We review the papers in terms of the employed robotic platforms and the focus on parts of the body, the techniques used to sense and react to human motion, the evaluation methods, the intended applications of the human-robot interaction systems and the scenarios utilized in user studies. Finally, challenges and possible future directions are considered and discussed.

Hinotori™ robotic esophagectomy: a feasibility cadaver study.

This preclinical feasibility study investigates the potential of utilizing the hinotori™ robot system for esophagectomy. In three human cadaver models, the esophagus was successfully mobilized and resected using the hinotori™ system, with a mean thoracic procedure time of 57minutes. The system allowed for precise dissection and radical lymphadenectomy without arm collision, attributed to its versatile design and docking-free trocars. Standard robot-specific patient positioning, including a 35° left lateral inclination, and trocar placement in a posterior axillary line configuration were employed. Notably, trocars suitable for both laparoscopy and the hinotori™ robot were utilized, providing flexibility in trocar selection. Unique features, such as the ergonomic console and pointer-based pivot point identification system, contributed to procedural success. While these findings highlight the promising potential of the hinotori™ system in advancing esophageal surgery, further clinical studies are warranted to validate its reproducibility and clinical utility. Additionally, enhancements to the pivot point identification system and evaluation of the arm base's features may further optimize surgical outcomes.

Accuracy of Zygomatic Implant Placement Using Task-Autonomous Robotic System or Dynamic Navigation: An InVitro Study.

To evaluate and compare the accuracy of task-autonomous robot-assisted implant surgery (RAIS) and dynamic computer-assisted implant surgery (dCAIS) for zygomatic implant placement. Ten atrophic edentulous maxilla models requiring zygomatic implant (ZI) placement were randomly divided into the RAIS and dCAIS groups. Osteotomies and implant placement were performed under the guidance of a task-autonomous robotic system or dynamic navigation system. A total of 20 ZIs were analyzed. The angular, coronal, lateral coronal, coronal depth, apical, lateral apical, and apical depth deviations were measured and analyzed between the two groups. The primary outcome parameters were the angular deviations between the planned and the placed ZIs. Data was subjected to descriptive and comparative statistical analysis. The significance of inter-group differences for continuous variables was assessed with Student's two-sample t-tests, Welch two-sample t-tests, and Mann-Whitney U tests according to the distribution normality and variance homogeneity. ZI placement deviations were compared between the RAIS and dCAIS groups, showing a mean angular deviation of 0.92 ± 0.40° versus 2.03 ± 0.53° (p < 0.001), a mean (±SD) coronal deviation of 0.48 ± 0.25 mm versus 1.29 ± 0.46 mm (p < 0.001), and a mean apical deviation of 0.88 ± 0.28 mm versus 1.96 ± 0.46 mm (p < 0.001). For computer-guided ZI placement, task-autonomous RAIS was superior to dCAIS in terms of accuracy.

A Real-Time System Status Evaluation Method for Passive UHF RFID Robots in Dynamic Scenarios

In dynamic scenarios, the status of a Radio Frequency Identification (RFID) system fluctuates with environmental changes. The key to improving system efficiency lies in the real-time monitoring and evaluation of the system status, along with adaptive adjustments to the system parameters and read algorithms. This paper focuses on the status changes in RFID systems in dynamic scenarios, aiming to enhance system robustness and reading performance, ensuring high link quality, reasonable resource scheduling, and real-time status evaluation under varying conditions. This paper comprehensively considers the system parameter settings in dynamic scenarios, integrating the interaction model between readers and tags. The system’s real-time status is evaluated from both the physical layer and the Medium Access Control (MAC) layer perspectives. For the physical layer, a link quality evaluation model based on Uniform Manifold Approximation and Projection (UMAP) and K-Means clustering is proposed from the link quality. For the MAC layer, a multi-criteria decision-making evaluation model based on combined weighting and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is proposed, which comprehensively considers both subjective and objective factors, utilizing the TOPSIS algorithm for an accurate evaluation of the MAC layer system status. For the RFID system, this paper proposes a real-time status evaluation model based on the Classification and Regression Tree (CART), which synthesizes the evaluation results of the physical layer and MAC layer. Finally, engineering tests and verification were conducted on the RFID robot system in mobile scenarios. The results showed that the clustering average silhouette coefficient of the physical layer link quality evaluation model based on K-Means was 0.70184, indicating a relatively good clustering effect. The system status evaluation model of the MAC layer, based on the combined weighting-TOPSIS method, demonstrated good flexibility and generalization. The real-time status evaluation model of the RFID system, based on CART, achieved a classification accuracy of 98.3%, with an algorithm runtime of 0.003 s. Compared with other algorithms, it had a higher classification accuracy and shorter runtime, making it well suited for the real-time evaluation of the RFID robot system’s status in dynamic scenarios.

Enhancing Grapevine Node Detection to Support Pruning Automation: Leveraging State-of-the-Art YOLO Detection Models for 2D Image Analysis.

Automating pruning tasks entails overcoming several challenges, encompassing not only robotic manipulation but also environment perception and detection. To achieve efficient pruning, robotic systems must accurately identify the correct cutting points. A possible method to define these points is to choose the cutting location based on the number of nodes present on the targeted cane. For this purpose, in grapevine pruning, it is required to correctly identify the nodes present on the primary canes of the grapevines. In this paper, a novel method of node detection in grapevines is proposed with four distinct state-of-the-art versions of the YOLO detection model: YOLOv7, YOLOv8, YOLOv9 and YOLOv10. These models were trained on a public dataset with images containing artificial backgrounds and afterwards validated on different cultivars of grapevines from two distinct Portuguese viticulture regions with cluttered backgrounds. This allowed us to evaluate the robustness of the algorithms on the detection of nodes in diverse environments, compare the performance of the YOLO models used, as well as create a publicly available dataset of grapevines obtained in Portuguese vineyards for node detection. Overall, all used models were capable of achieving correct node detection in images of grapevines from the three distinct datasets. Considering the trade-off between accuracy and inference speed, the YOLOv7 model demonstrated to be the most robust in detecting nodes in 2D images of grapevines, achieving F1-Score values between 70% and 86.5% with inference times of around 89 ms for an input size of 1280 × 1280 px. Considering these results, this work contributes with an efficient approach for real-time node detection for further implementation on an autonomous robotic pruning system.

Empirical study of robotic systems implementation to corporate performance in manufacturing sector

PurposeThis study aims to investigate the impact of implementing various robotic systems in logistics and supply chain management on corporate performance in Jordanian manufacturing companies, focusing on environmental, operational, economic, and social dimensions.Design/methodology/approachUsing a quantitative approach, data was collected through a survey questionnaire to measure the relationship between robotic systems and several performance dimensions. Various established constructs were employed, and the structural relationships were analyzed using partial least squares structural equation modeling (PLS-SEM) to assess the complex interdependencies within the model.FindingsThe results of this study indicate that the adoption of robotic systems has a positive influence on the environmental, operational, economic, and social performance of Jordanian manufacturing companies. In contrast to prior research that revealed inconsistencies in the social dimension, our findings align with previous studies highlighting the benefits of robotics in logistics and supply chain management. However, it’s worth noting that this study did not uncover similar inconsistencies, particularly in terms of the impact on social performance.Practical implicationsThe study provides valuable insights for manufacturing companies considering the implementation of robotic systems, highlighting the need to evaluate the environmental, operational, social, and economic consequences. This understanding can help organizations make informed decisions to leverage the benefits of robotics for sustainable growth.Originality/valueThis study contributes to the growing literature on robotics in logistics and supply chain management, specifically focusing on the unique context of Jordanian manufacturing companies. By examining the multifaceted impact of robotic systems, this study extends the understanding of the role of technology in enhancing corporate performance.

Avian-inspired embodied perception in biohybrid flapping-wing robotics

Avian feather intricate adaptable architecture to wing deformations has catalyzed interest in feathered flapping-wing aircraft with high maneuverability, agility, and stealth. Yet, to mimic avian integrated somatic sensation within stringent weight constraints, remains challenging. Here, we propose an avian-inspired embodied perception approach for biohybrid flapping-wing robots. Our feather-piezoelectric mechanoreceptor leverages feather-based vibration structures and flexible piezoelectric materials to refine and augment mechanoreception via coupled oscillator interactions and robust microstructure adhesion. Utilizing convolutional neural networks with the grey wolf optimizer, we develop tactile perception of airflow velocity and wing flapping frequency proprioception. This method also senses pitch angle via airflow direction and detects wing morphology through feather collisions. Our low-weight, accurate perception of flapping-wing robot flight states is validated by motion capture. This investigation constructs a biomechanically integrated embodied perception system in flapping-wing robots, which holds significant promise in reflex-based control of complex flight maneuvers and natural bird flight surveillance.

A Computationally Efficient Neuronal Model for Collision Detection with Contrast Polarity-Specific Feed-Forward Inhibition.

Animals utilize their well-evolved dynamic vision systems to perceive and evade collision threats. Driven by biological research, bio-inspired models based on lobula giant movement detectors (LGMDs) address certain gaps in constructing artificial collision-detecting vision systems with robust selectivity, offering reliable, low-cost, and miniaturized collision sensors across various scenes. Recent progress in neuroscience has revealed the energetic advantages of dendritic arrangements presynaptic to the LGMDs, which receive contrast polarity-specific signals on separate dendritic fields. Specifically, feed-forward inhibitory inputs arise from parallel ON/OFF pathways interacting with excitation. However, none of the previous research has investigated the evolution of a computational LGMD model with feed-forward inhibition (FFI) separated by opposite polarity. This study fills this vacancy by presenting an optimized neuronal model where FFI is divided into ON/OFF channels, each with distinct synaptic connections. To align with the energy efficiency of biological systems, we introduce an activation function associated with neural computation of FFI and interactions between local excitation and lateral inhibition within ON/OFF channels, ignoring non-active signal processing. This approach significantly improves the time efficiency of the LGMD model, focusing only on substantial luminance changes in image streams. The proposed neuronal model not only accelerates visual processing in relatively stationary scenes but also maintains robust selectivity to ON/OFF-contrast looming stimuli. Additionally, it can suppress translational motion to a moderate extent. Comparative testing with state-of-the-art based on ON/OFF channels was conducted systematically using a range of visual stimuli, including indoor structured and complex outdoor scenes. The results demonstrated significant time savings in silico while retaining original collision selectivity. Furthermore, the optimized model was implemented in the embedded vision system of a micro-mobile robot, achieving the highest success ratio of collision avoidance at 97.51% while nearly halving the processing time compared with previous models. This highlights a robust and parsimonious collision-sensing mode that effectively addresses real-world challenges.

Current status analysis of the prevalence and regional disparities of robot-assisted laparoscopic prostatectomy in Japan using diagnosis procedure combination data

The introduction of the “da Vinci S HD Surgical System” marked a significant shift towards robotic surgeries in Japan. However, initial high costs and lack of efficacy data posed barriers to its widespread adoption. By 2023, more than 570 da Vinci units were operational in Japan, highlighting the growing acceptance of robotic surgery despite these challenges. This study aimed to investigate the prevalence and regional disparities in the adoption of robot-assisted laparoscopic prostatectomy (RALP) across Japan using diagnosis procedure combination data. This retrospective observational study analyzed data from 2857 urban and 4184 regional hospitals across 47 prefectures in Japan. The study focused on the number of RALP procedures, da Vinci systems, and certified urological surgery proctors. Multiple regression analysis was performed to identify significant factors influencing RALP adoption. Urban areas demonstrated a higher prevalence of RALP procedures and more da Vinci systems compared to regional areas, with urban hospitals performing an average of 937 RALP procedures compared to 195.5 in regional hospitals. The number of certified urological surgeons also showed significant urban-regional disparities, contributing to the overall imbalance. Our findings highlight substantial regional disparities in access to robot-assisted surgery in Japan, with urban areas benefiting from better access to advanced medical technologies and specialist training. Addressing these disparities will require targeted policies to improve the dissemination of robotic surgery systems and enhance training opportunities in regional cities.

Three-Dimensional Space Chaotic Scrolls Generated by Memristive Hopfield Neural Network and Its Application in Robot Path Planning

Abstract In this paper, a novel three-dimensional space-scroll chaotic attractor is constructed by introducing piece-wise linear memristors into a three-neurons-based Hopfield neural network to simulate the electromagnetic influence on neural dynamics. The dynamical behaviors of the novel memristive Hopfield neural network are discussed by means of numerical methods including the Lyapunov exponent spectrum, bifurcation diagrams, phase plots and dynamic maps. Furthermore, we also investigated different neurons subject to the memristive electromagnetic influence, where the two-dimensional grid-scroll chaotic attractors can be generated by one neuron subjects to the electromagnetic influence and three-dimensional space-scroll chaotic attractors can be generated by all of the neurons subject to the memristive electromagnetic influence. Moreover, we present a three-dimensional space-scroll scroll attractor-based chaotic mobile robot system. Superiority of the proposed space-scroll-based chaotic mobile robot system compared to existing methods has been demonstrated through numerical simulations.

Towards Transforming Neurorehabilitation: The Impact of Artificial Intelligence on Diagnosis and Treatment of Neurological Disorders.

Background and Objectives: Neurological disorders like stroke, spinal cord injury (SCI), and Parkinson's disease (PD) significantly affect global health, requiring accurate diagnosis and long-term neurorehabilitation. Artificial intelligence (AI), such as machine learning (ML), may enhance early diagnosis, personalize treatment, and optimize rehabilitation through predictive analytics, robotic systems, and brain-computer interfaces, improving outcomes for patients. This systematic review examines how AI and ML systems influence diagnosis and treatment in neurorehabilitation among neurological disorders. Materials and Methods: Studies were identified from an online search of PubMed, Web of Science, and Scopus databases with a search time range from 2014 to 2024. This review has been registered on Open OSF (n) EH9PT. Results: Recent advancements in AI and ML are revolutionizing motor rehabilitation and diagnosis for conditions like stroke, SCI, and PD, offering new opportunities for personalized care and improved outcomes. These technologies enhance clinical assessments, therapy personalization, and remote monitoring, providing more precise interventions and better long-term management. Conclusions: AI is revolutionizing neurorehabilitation, offering personalized, data-driven treatments that enhance recovery in neurological disorders. Future efforts should focus on large-scale validation, ethical considerations, and expanding access to advanced, home-based care.

Hierarchically‐Interlocked, Three‐Axis soft Iontronic Sensor for Omnidirectional Shear and Normal Forces

AbstractArtificial tactile sensing capable of measuring shear and normal forces is crucial for the diverse human‐machine interactions and dexterous robotic manipulations. However, existing soft multi‐axis force sensors usually suffer from limited detectable directions and complex coupling mechanisms, limiting their applications in realistic wearable and robotic systems. Here, a hierarchically‐interlocked, three‐axis soft iontronic sensor is presented with an asymmetrical electrode pattern that leverages the mortise‐and‐tenon structure and ultracapacitive principle to detect omnidirectional shear and normal forces. The designed sensor facilitates the decoupling process and achieves enhanced sensing performances including high accuracy, fast response ability, and mechanical robustness. Prototypical integration and application of the sensor are demonstrated to perform wearable telecontrol of virtual platforms and assist robot gripper through closed‐loop force feedback. A sensing array is further developed to construct a touch panel and identify handwriting based on the continuously measured directions of applied forces. This work may offer a potentially promising solution for the next‐generation intelligent electronic skins requiring multimodal tactile sensing information.

Ground-LIO: enhanced LiDAR-inertial odometry for ground robots based on ground optimization

Abstract In the field of LiDAR-based Simultaneous Localization and Mapping, the potential of ground point clouds to enhance pose estimation in mobile robots has yet to be fully realized. This paper focuses on leveraging ground point clouds to improve the performance of LiDAR-Inertial Odometry (LIO) systems for ground-based mobile robots. We begin by analyzing the characteristics of ground point clouds and the typical types of noise that affect their extraction and utilization. Ground point clouds are then extracted from denoised data. Given the generally local planar nature of ground point clouds, we propose a segmentation-and-refitting approach to process them. This method reduces the computational burden of residual calculation in pose estimation by avoiding redundant plane fitting. Additionally, we introduce a data structure designed for the efficient management and utilization of ground point clouds obtained through segmentation and refitting. This structure is particularly suited to the ground point cloud data produced by our method, enabling efficient access and registration through the continuous maintenance and consolidation of local plane parameters. Our method has been integrated into advanced LIO systems (Bai et al 2022 IEEE Robot. Autom. Lett. 7 4861–8), and experimental results on challenging datasets demonstrate its promising performance.

Hierarchical prescribed-time ESO-based fault-tolerant controller synthesis for error-constrained coordination of networked robotic systems

Hierarchical prescribed-time ESO-based fault-tolerant controller synthesis for error-constrained coordination of networked robotic systems

Aerosol Jet and Inkjet Printing Methods for Die-Level Packaging of Custom MEMS Devices

Abstract In this study, we propose an alternative, low-cost, packaging method using both inkjet and aerosol jet printing to fabricate functional interconnects for custom die-level packages assembled on Printed Circuit Boards (PCB). Our process involves the manufacturing of a stacked structure with insulating and conducting layers, fabricated using inkjet and aerosol jet printing, respectively. In the first part of the study, we characterized UV adhesive deposition on the standard resin-coated PCB to acquire the optimal parameters necessary for the fabrication of the insulating layer with thicknesses around 100 μm. In the second part of our study, we developed a precise method for printing silver conducting lines using an aerosol jet printer, enabling the realization of structures with features as small as 50 microns on non-planar surfaces. The insulating part of the MEMS/PCB structure serves as a ramp ensuring continuity of the aerosol jet printed connection between the PCB and the Si die. A 1 cm x 1 cm Si chip with a custom MEMS microrobot was used to demonstrate the feasibility and flexibility of our approach in practice. After fabrication, we evaluated the interconnects for conductivity and repeatability. The packaging and inspection process was carried out using Nexus, a unique robotic system integrating additive manufacturing, robotic transport, and metrology. The results show that we successfully fabricated printed interconnects between copper PCB pads and Si die cleanroom fabricated gold pads, while electrical characterization revealed resistances in the range of 1 Ω - 10 Ω. Our approach can be utilized in the manufacturing of electrical interconnects for custom devices on different substrates, including traditional and flexible PCBs. Furthermore, the applied printing techniques enable the use of other insulating or conducting inks and the formation of structures of custom geometries across a wide range of scales - 20 μm to 1 mm.

Perturbation defense ultra high-speed weak target recognition

Ultra high-speed target recognition in complex electromagnetic environments is a critical and fundamental machine perception issue. It is difficult to ensure privacy protection and intelligent confrontation with centralized training in many practical situations. In this paper, an efficient ultra high-speed target recognition approach, named InVision, for robot systems is proposed using deep trustworthy federated learning (DTFL). InVision is accurate, safe, fast, and robust. Particularly, geometric component transformer (GCT) is presented to significantly promote neural element's complex representation description ability. And an ambiguity-aware cooperative learning (AACL) scheme is developed to relieve the noisy label problem. Moreover, decentralized federated training (DFT) is designed to mitigate the intractable privacy protection problem, to efficiently search for similarities and reduce the representation redundancy. Furthermore, to promote the running speed of the system in real-world environments, a lightweight deep architecture, called Mobile-XB, is developed. Extensive quantitative and qualitative experiments are carried out, and the results demonstrate that InVision greatly outperforms the outstanding comparison methods, establishing efficient connections and extraction, and providing security guarantees.

Integrating reinforcement learning and supervisory control theory for optimal directed control of discrete-event systems

Directed control is crucial for implementing controllers on programmable logic controllers. A directed controller is one that actively selects at most one controllable event (control command) to execute at any instant. This study investigates the numerical optimization problem of directed control for discrete-event systems, such as traffic systems and robotic systems. By integrating supervisory control theory and reinforcement learning, an algorithmic procedure is designed to synthesize an optimal directed controller, which minimizes the total cost associated with event execution while ensuring the safety and liveness of the controlled system. Two improved Q-learning algorithms incorporating dynamic parameter adaptation strategies are developed to enhance the global search ability of basic Q-learning. To demonstrate applicability and effectiveness, we apply the proposed method to control a guideway system and a multi-train traffic system, respectively. The experimental results indicate the superiority of the proposed method over the comparison methods.

Adaptive Fault-Tolerant Control of Mobile Robots with Fractional-Order Exponential Super-Twisting Sliding Mode

Industrial mobile robots easily experience actuator loss of some effectiveness and additive bias faults due to the working scenarios, resulting in unexpected performance degradation. This article proposes a novel adaptive fault-tolerant control (FTC) strategy for nonholonomic mobile robot systems subject to simultaneous actuator lock-in-place (LIP) and partial loss-of-effectiveness (LOE) faults. First, a nominal fractional-order sliding mode controller based on the designed exponential super-twisting reaching law is investigated to reduce the reaching phase time and eliminate the chattering. To address the time-varying LIP faults and uncertainties, a novel barrier function (BF)-based gain is explored to assist the super-twisting law. An estimator is designed to estimate the lower bound of the time-varying partial LOE fault coefficients, thus without requiring the boundary information of faults that is commonly requested in traditional FTC schemes. Combined with the nominal controller clubbed with BF and estimator-based LOE fault compensation term, the fault-tolerant controller is finally constructed. The proposed FTC scheme achieves fast convergence and the sliding variables can be confined in a predetermined neighborhood of the sliding manifold under actuator faults. The results show that the proposed controller has superior tracking performance under faulty conditions compared with other state-of-the-art adaptive FTC approaches.

MoMa: An assistive mobile manipulator with a webcam-based gaze control system

Mobile Manipulators (MoMa) is a category of mobile robots designed to assist people with motor disabilities to perform object retrieval tasks using a webcam-based gaze control system. Using off-the-shelf components such as reproducible acrylic and 3D-printed plates, and a webcam for eye tracking, MoMa serves as an inexpensive, open-source, and customizable solution in assistive robotics. The robotic system consists of a mobile base that can move forward and backward, as well as turn in place; and a 2-axis cartesian arm equipped with a claw gripper that opens and closes. The simple movement of the robot also allows for a simple control method and graphical user interface (GUI). The user receives information about what is in front of the robot through a mounted camera, and, by looking at parts of the screen that correspond to controls, has their gaze predicted by a convolutional neural network and sends commands wirelessly. The performance of the entire system has been validated through testing of the gaze prediction model, the integration of the control system, as well as its task completion capabilities. All the design, construction and software files are freely available online under the CC BY 4.0 license at https://doi.org/10.17632/k7yfn6wdv7.2.

Deep learning based approach for actinidia flower detection and gender assessment

Pollination is critical for crop development, especially those essential for subsistence. This study addresses the pollination challenges faced by Actinidia, a dioecious plant characterized by female and male flowers on separate plants. Despite the high protein content of pollen, the absence of nectar in kiwifruit flowers poses difficulties in attracting pollinators. Consequently, there is a growing interest in using artificial intelligence and robotic solutions to enable pollination even in unfavourable conditions. These robotic solutions must be able to accurately detect flowers and discern their genders for precise pollination operations. Specifically, upon identifying female Actinidia flowers, the robotic system should approach the stigma to release pollen, while male Actinidia flowers should target the anthers to collect pollen. We identified two primary research gaps: (1) the lack of gender-based flower detection methods and (2) the underutilisation of contemporary deep learning models in this domain. To address these gaps, we evaluated the performance of four pretrained models (YOLOv8, YOLOv5, RT-DETR and DETR) in detecting and determining the gender of Actinidia flowers. We outlined a comprehensive methodology and developed a dataset of manually annotated flowers categorized into two classes based on gender. Our evaluation utilised k-fold cross-validation to rigorously test model performance across diverse subsets of the dataset, addressing the limitations of conventional data splitting methods. DETR provided the most balanced overall performance, achieving precision, recall, F1 score and mAP of 89%, 97%, 93% and 94%, respectively, highlighting its robustness in managing complex detection tasks under varying conditions. These findings underscore the potential of deep learning models for effective gender-specific detection of Actinidia flowers, paving the way for advanced robotic pollination systems.