Robot In Environment Research Articles

Hyper CLS-Data-Based Robotic Interface and Its Application to Intelligent Peg-in-Hole Task Robot Incorporating a CNN Model for Defect Detection

Various types of numerical control (NC) machine tools can be standardly operated and controlled based on NC data that can be easily generated using widespread CAD/CAM systems. On the other hand, the operation environments of industrial robots still depend on conventional teaching and playback systems provided by the makers, so it seems that they have not been standardized and unified like NC machine tools yet. Additionally, robotic functional extensions, e.g., the easy implementation of a machine learning model, such as a convolutional neural network (CNN), a visual feedback controller, cooperative control for multiple robots, and so on, has not been sufficiently realized yet. In this paper, a hyper cutter location source (HCLS)-data-based robotic interface is proposed to cope with the issues. Due to the HCLS-data-based robot interface, the robotic control sequence can be visually and unifiedly described as NC codes. In addition, a VGG19-based CNN model for defect detection, whose classification accuracy is over 99% and average time for forward calculation is 70 ms, can be systematically incorporated into a robotic control application that handles multiple robots. The effectiveness and validity of the proposed system are demonstrated through a cooperative pick and place task using three small-sized industrial robot MG400s and a peg-in-hole task while checking undesirable defects in workpieces with a CNN model without using any programmable logic controller (PLC). The specifications of the PC used for the experiments are CPU: Intel(R) Core(TM) i9-10850K CPU 3.60 GHz, GPU: NVIDIA GeForce RTX 3090, Main memory: 64 GB.

Analysis of Robot–Environment Interaction Modes in Anguilliform Locomotion of a New Soft Eel Robot

Bio-inspired robots with elongated anatomy, like eels, are studied to discover anguilliform swimming principles and improve the robots’ locomotion accordingly. Soft continuum robots replicate animal–environment physics better than noncompliant, rigid, multi-body eel robots. In this study, a slender soft robot was designed and tested in an actual swimming experiment in a still-water tank. The robot employs soft pneumatic muscles laterally connected to a flexible backbone and activated with a rhythmic input. The position of seven markers mounted on the robot’s backbone was recorded using QualiSys® Tracking Manager (QTM) 1.6.0.1. The system was modeled as a fully coupled fluid–solid interaction (FSI) system using COMSOL Multiphysics® 6.1. Further data postprocessing and analysis were conducted, proposing a new mode decomposition algorithm using simulation data. Experiments show the success of swimming with a velocity of 28 mm/s and at a frequency of 0.9 Hz. The mode analysis allowed the modeling and explanation of the fluctuation. Results disclose the presence of traveling waves related to anguilliform waves obtained by the superposition of two main modes. The similarities of the results with natural anguilliform locomotion are discussed. It is concluded that soft robot undulation is ruled by dynamic modes induced by robot–environment interaction.

Cost-Effective Localization of Mobile Robots Using Ultrasound Beacons and Differential Time-of-Flight Measurement

This paper presents an innovative and cost-effective solution for the absolute localization of mobile robots using ultrasound beacons. The proposed system addresses the challenge of precise positioning within a controlled environment by employing Differential Time-of-Flight (ToF) measurements to determine the relative distances between the robot and optimally placed beacons. Unlike other ToF methods that require synchronization pulses, the proposed approach eliminates this requirement, significantly simplifying the setup and reducing system complexity. Furthermore, the system achieves a higher sampling rate than conventional synchronization-based systems, enhancing real-time performance. Detailed analysis and simulation demonstrate the system’s ability to provide accurate and reliable localization. The results highlight the potential for broad application in various robotic environments, offering a robust solution for absolute positioning without complex synchronization strategies. This work underscores the advantages of using ToF measurements with ultrasound beacons and contributes to the ongoing development of efficient and cost-effective robotic localization systems.

Ontology for BIM-Based Robotic Navigation and Inspection Tasks

The availability of inspection robots in the construction and operation phases of buildings has led to expanding the scope of applications and increasing technological challenges. Furthermore, the building information modeling (BIM)-based approach for robotic inspection is expected to improve the inspection process as the BIM models contain accurate geometry and relevant information at different phases of the lifecycle of a building. Several studies have used BIM for navigation purposes. Also, some studies focused on developing a knowledge-based ontology to perform activities in a robotic environment (e.g., CRAM). However, the research in this area is still limited and fragmented, and there is a need to develop an integrated ontology to be used as a first step towards logic-based inspection. This paper aims to develop an ontology for BIM-based robotic navigation and inspection tasks (OBRNIT). This ontology can help system engineers involved in developing robotic inspection systems by identifying the different concepts and relationships between robotic inspection and navigation tasks based on BIM information. The developed ontology covers four main types of concepts: (1) robot concepts, (2) building concepts, (3) navigation task concepts, and (4) inspection task concepts. The ontology is developed using Protégé. The following steps are taken to reach the objectives: (1) the available literature is reviewed to identify the concepts, (2) the steps for developing OBRNIT are identified, (3) the basic components of the ontology are developed, and (4) the evaluation process is performed for the developed ontology. The semantic representation of OBRNIT was evaluated through a case study and a survey. The evaluation confirms that OBRNIT covers the domain’s concepts and relationships, and can be applied to develop robotic inspection systems. In a case study conducted in a building at Concordia University, OBRNIT was used to support an inspection robot in navigating to identify a ceiling leakage. Survey results from 33 experts indicate that 28.13% strongly agreed and 65.63% agreed on the usage of OBRNIT for the development of robotic navigation and inspection systems. This highlights its potential in enhancing inspection reliability and repeatability, addressing the complexity of interactions within the inspection environment, and supporting the development of more autonomous and efficient robotic inspection systems.

Interactive Path Editing and Simulation System for Motion Planning and Control of a Collaborative Robot

Robots in hazardous environments demand precise and advanced motion control, making extensive simulations crucial for verifying the safety of motion planning. This paper presents a simulation system that enables interactive path editing, allowing for motion planning in a simulated collaborative robot environment and its real-world application. The system includes a simulation host, a control board, and a robot. Unity 3D on a Windows platform provides the simulation environment, while a virtual Linux environment runs ROS2 for execution. Unity sends edited motion paths to ROS2 using the Unity ROS TCP Connector package. The ROS2 MoveIt framework generates trajectories, which are synchronized back to Unity for simulation and real-world validation. To control the six-axis Indy7 collaborative robot, we used the MIO5272 embedded board as an EtherCAT master. Verified trajectories are sent to the target board, synchronizing the robot with the simulation in position and speed. Data are relayed from the host to the MIO5272 using ROS2 and the Data Distribution Service (DDS) to control the robot via EtherCAT communication. The system enables direct simulation and control of various trajectories for robots in hazardous environments. It represents a major advancement by providing safe and optimized trajectories through efficient motion planning and repeated simulations, offering a clear improvement over traditional time-consuming and error-prone teach pendant methods.

Enhancing Autonomous Driving Robot Systems with Edge Computing and LDM Platforms

The efficient operation and interaction of autonomous robots play crucial roles in various fields, e.g., security, environmental monitoring, and disaster response. For these purposes, processing large volumes of sensor data and sharing data between robots is essential; however, processing such large data in an on-device environment for robots results in substantial computational resource demands, causing high battery consumption and heat issues. Thus, this study addresses challenges related to processing large volumes of sensor data and the lack of dynamic object information sharing among autonomous robots and other mobility systems. To this end, we propose an Edge-Driving Robotics Platform (EDRP) and a Local Dynamic Map Platform (LDMP) based on 5G mobile edge computing and Kubernetes. The proposed EDRP implements the functions of autonomous robots based on a microservice architecture and offloads these functions to an edge cloud computing environment. The LDMP collects and shares information about dynamic objects based on the ETSI TR 103 324 standard, ensuring cooperation among robots in a cluster and compatibility with various Cooperative-Intelligent Transport System (C-ITS) components. The feasibility of operating a large-scale autonomous robot offloading system was verified in experimental scenarios involving robot autonomy, dynamic object collection, and distribution by integrating real-world robots with an edge computing–based offloading platform. Experimental results confirmed the potential of dynamic object collection and dynamic object information sharing with C-ITS environment components based on LDMP.

Indoor Quadruped Robot Navigation Algorithm Based on ORB-SLAM

In recent years, the development of artificial intelligence, big data, and the Internet of Things technologies has revealed unprecedented potential and value in mobile robots across various sectors of automation and intelligence. Among these, quadruped robots have shown unique applications in the domain of indoor security and inspection, as they can navigate multi-storey buildings by ascending and descending stairs. To enhance the stability of visual navigation in indoor inspection environments for quadruped robots, this paper builds on the ORB-SLAM framework. It introduces the AGC algorithm to address the issue of reduced feature point extraction during night patrols, thereby increasing the number of feature points extracted. Additionally, to tackle the problem of high oscillation intensity during the movement of quadruped robots, the EKF algorithm has been incorporated for sensor fusion with IMU, enhancing the robustness of visual navigation. This has been validated through physical experiments.

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Reinforcement learning (RL) has achieved remarkable advancements in navigation tasks in recent years. However, tackling multi-goal navigation tasks with sparse rewards remains a complex and challenging problem due to the long-sequence decision-making involved. Such multi-goal navigation tasks inherently incorporate a hybrid action space, where the robot needs to select a navigation endpoint first before executing primitive actions. To address the problem of multi-goal navigation with sparse rewards, we introduce a novel hierarchical RL framework named Hierarchical RL with Multi-Goal (HRL-MG). The main idea of HRL-MG is to divide and conquer the hybrid action space, splitting long-sequence decisions into short-sequence decisions. The HRL-MG framework is composed of two main modules: a selector and an actuator. The selector employs a temporal abstraction hierarchical architecture designed to specify a desired end goal based on the discrete action space. Conversely, the actuator utilizes a continuous goal-oriented hierarchical architecture developed to enact continuous action sequences to reach the desired end goal specified by the selector. In addition, we incorporate a dynamic goal detection mechanism, grounded in hindsight experience replay, to mitigate the challenges posed by sparse reward landscapes. We validated the algorithm’s efficacy on both the discrete environment Maze_2D and the continuous robotic environment MuJoCo ‘Ant’. The results indicate that HRL-MG significantly outperforms other methods in multi-goal navigation tasks with sparse rewards.

YS-SLAM: YOLACT++ based semantic visual SLAM for autonomous adaptation to dynamic environments of mobile robots

Aiming at the problem of poor autonomous adaptability of mobile robots to dynamic environments, this paper propose a YOLACT++ based semantic visual SLAM for autonomous adaptation to dynamic environments of mobile robots. First, a light-weight YOLACT++ is utilized to detect and segment potential dynamic objects, and Mahalanobis distance is combined to remove feature points on active dynamic objects, also, epipolar constraint and clustering are employed to eliminate feature points on passive dynamic objects. Then, in terms of the semantic labels of dynamic and static components, the global semantic map is divided into three parts for construction. The semantic overlap and uniform motion model are chose to track moving objects and the dynamic components are added to the background map. Finally, a 3D semantic octree map is constructed that is consistent with the real environment and updated in real time. A series of simulations and experiments demonstrated the feasibility and effectiveness of the proposed approach.

Remote intelligent perception system for multi-object detection.

During the last few years, a heightened interest has been shown in classifying scene images depicting diverse robotic environments. The surge in interest can be attributed to significant improvements in visual sensor technology, which has enhanced image analysis capabilities. Advances in vision technology have a major impact on the areas of multiple object detection and scene understanding. These tasks are an integral part of a variety of technologies, including integrating scenes in augmented reality, facilitating robot navigation, enabling autonomous driving systems, and improving applications in tourist information. Despite significant strides in visual interpretation, numerous challenges persist, encompassing semantic understanding, occlusion, orientation, insufficient availability of labeled data, uneven illumination including shadows and lighting, variation in direction, and object size and changing background. To overcome these challenges, we proposed an innovative scene recognition framework, which proved to be highly effective and yielded remarkable results. First, we perform preprocessing using kernel convolution on scene data. Second, we perform semantic segmentation using UNet segmentation. Then, we extract features from these segmented data using discrete wavelet transform (DWT), Sobel and Laplacian, and textual (local binary pattern analysis). To recognize the object, we have used deep belief network and then find the object-to-object relation. Finally, AlexNet is used to assign the relevant labels to the scene based on recognized objects in the image. The performance of the proposed system was validated using three standard datasets: PASCALVOC-12, Cityscapes, and Caltech 101. The accuracy attained on the PASCALVOC-12 dataset exceeds 96% while achieving a rate of 95.90% on the Cityscapes dataset. Furthermore, the model demonstrates a commendable accuracy of 92.2% on the Caltech 101 dataset. This model showcases noteworthy advancements beyond the capabilities of current models.

Overcoming boundaries: Interdisciplinary challenges and opportunities in cognitive neuroscience

Cognitive neuroscience has considerable untapped potential to translate our understanding of brain function into applications that maintain, restore, or enhance human cognition. Complex, real-world phenomena encountered in daily life, professional contexts, and in the arts, can also be a rich source of information for better understanding cognition, which in turn can lead to advances in knowledge and health outcomes. Interdisciplinary work is needed for these bi-directional benefits to be realized. Our cognitive neuroscience team has been collaborating on several interdisciplinary projects: hardware and software development for brain stimulation, measuring human operator state in safety-critical robotics environments, and exploring emotional regulation in actors who perform traumatic narratives. Our approach is to study research questions of mutual interest in the contexts of domain-specific applications, using (and sometimes improving) the experimental tools and techniques of cognitive neuroscience. These interdisciplinary attempts are described as case studies in the present work to illustrate non-trivial challenges that come from working across traditional disciplinary boundaries. We reflect on how obstacles to interdisciplinary work can be overcome, with the goals of enriching our understanding of human cognition and amplifying the positive effects cognitive neuroscientists have on society and innovation.

A novel joint depth sensor calibration method without fixture for mobile robots’ navigation

AbstractCommercial mobile robots are usually equipped with multiple depth sensors that can measure the point cloud information around the robot's environment. The installation process of these sensors contains assembly error and sensor measurement error, so it is necessary to calibrate each sensor to align the point cloud. In order to obtain the sensor calibration results of commercial robots under normal working conditions, this study proposes a fixture free multi depth sensor joint calibration method that can be deployed on low‐cost embedded computing units, which efficiently aligns the point clouds of each sensor. During the calibration process, the robot is placed in the center of three upright thin plates perpendicular to the ground. 2D LIDAR depicts high‐precision contours of the upright thin plates. In the calibration process of each depth sensor, the roll angle and pitch angle of the sensor point cloud are first calibrated to make it perpendicular to the ground, and then the yaw angle and position of the point cloud are calibrated to fit the high‐precision contour of the upright thin plate. The results show that this method can be deployed on low‐cost embedded computing units, with real‐time and accurate calibration results. The convergence of calibration results can be achieved through up to 5 iterations, and the average running time is less than 120 ms. This research achievement provides a reference for multi‐sensor calibration of commercial robots.

Curiosity model policy optimization for robotic manipulator tracking control with input saturation in uncertain environment.

In uncertain environments with robot input saturation, both model-based reinforcement learning (MBRL) and traditional controllers struggle to perform control tasks optimally. In this study, an algorithmic framework of Curiosity Model Policy Optimization (CMPO) is proposed by combining curiosity and model-based approach, where tracking errors are reduced via training agents on control gains for traditional model-free controllers. To begin with, a metric for judging positive and negative curiosity is proposed. Constrained optimization is employed to update the curiosity ratio, which improves the efficiency of agent training. Next, the novelty distance buffer ratio is defined to reduce bias between the environment and the model. Finally, CMPO is simulated with traditional controllers and baseline MBRL algorithms in the robotic environment designed with non-linear rewards. The experimental results illustrate that the algorithm achieves superior tracking performance and generalization capabilities.

6-DoF grasp estimation method that fuses RGB-D data based on external attention

6-DoF grasp estimation methods based on point clouds have long been a challenge in robotics due to the limitations of single data input, which hinder the robot’s perception of real-world scenarios, thus reducing its robustness. In this work, we propose a 6-DoF grasp pose estimation method based on RGB-D data, which leverages ResNet to extract color image features, utilizes the PointNet++ network to extract geometric information features, and employs an external attention mechanism to fuse both features. Our method is an end-to-end design, and we validate its performance through benchmark tests on a large-scale dataset and evaluations in a simulated robot environment. Our method outperforms previous state-of-the-art methods on public datasets, achieving 47.75mAP and 40.08mAP for seen and unseen objects, respectively. We also test our grasp pose estimation method on multiple objects in a simulated robot environment, demonstrating that our approach exhibits higher grasp accuracy and robustness than previous methods.

Enhancing human–robot communication with a comprehensive language-conditioned imitation policy for embodied robots in smart cities

Integrating Embodied Robots into a smart city’s networked system can significantly enhance the city’s operational efficiency. These robots can be connected to the city’s network, receiving and transmitting data in real time for enhancing human–robot communications. Language-conditioned robot behavior plays a vital role in executing complex tasks by associating human commands or instructions with perception and actions. However, most language-conditioned policy-related research is limited to specific datasets and cannot generalize across different environments. In this study, we propose a novel imitation learning framework tailored for language-conditioned robotic tasks. Our framework includes specialized encoders designed for various benchmarks and utilizes two distinct models: the Transformer and Diffusion models. We rigorously evaluate this framework in three different robotic environments. Our findings indicate that the framework consistently delivers superior performance across multiple domains. Notably, we observe that the Transformer model is particularly effective in managing tasks with long trajectories, whereas the Diffusion model demonstrates enhanced proficiency in generating trajectories from limited training datasets. Our approach showcases remarkable generalization capabilities across a range of tasks and achieves significantly higher success rates in task completion.

Implementations of Digital Transformation and Digital Twins: Exploring the Factory of the Future

In the era of rapid technological advancement and evolving industrial landscapes, embracing the concept of the factory of the future (FoF) is crucial for companies seeking to optimize efficiency, enhance productivity, and stay sustainable. This case study explores the concept of the FoF and its role in driving the energy transition and digital transformation within the automotive sector. By embracing advancements in technology and innovation, these factories aim to establish a smart, sustainable, inclusive, and resilient growth framework. The shift towards hybrid and electric vehicles necessitates significant adjustments in vehicle components and production processes. To achieve this, the adoption of lighter materials becomes imperative, and new technologies such as additive manufacturing (AM) and artificial intelligence (AI) are being adopted, facilitating enhanced efficiency and innovation within the factory environment. An important aspect of this paradigm involves the development and utilization of a modular, affordable, safe human–robot interaction and highly performant intelligent robot. The introduction of this intelligent robot aims to foster a higher degree of automation and efficiency through collaborative human–robot environments on the factory floor and production lines, specifically tailored to the automotive sector. By combining the strengths of human and robotic capabilities, the future factory aims to revolutionize manufacturing processes, ultimately driving the automotive industry towards a more sustainable and technologically advanced future. This study explores the implementation of automation and the initial strides toward transitioning from Industry 4.0 to 5.0, focusing on three recognized, large, and automotive companies operating in the north of Portugal.

Novel Adaptive Control for Flexible-Joint Robots With Unknown Measurement Sensitivity

For the existing tracking control schemes of flexible-joint robots, precise sensor measurement is an implicit premise. However, idealized sensors are difficult to achieve due to manufacturing technology or other external factors. To this end, this paper further investigates the tracking control problem for flexible-joint robots with unknown measurement sensitivity. Specifically, for such multi-input multi-output Euler-Lagrange systems with completely unknown system dynamics, a novel measurement values-based adaptive control method is proposed by fusing sensitivity information and system variables into Lyapunov function candidates, where the restriction on system states in other the approximation lemma-based results is removed, since unknown nonlinearities are scaled by the structural characteristics of system variables. Above all, even if there are measurement errors, satisfactory tracking performance can be obtained by adjusting the design parameters, which is proved by rigorous theoretical analysis. Finally, hardware experiments further verify the effectiveness of the proposed method. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work is motivated by the trajectory tracking control problem for flexible-joint robots under imprecise sensor measurements. Due to manufacturing technology limitations and component aging, there is inevitably a deviation between the measured values of sensors and real values, and this problem may become more prominent as the working environment of flexible-joint robots tends to become more complex. To our knowledge, most of the existing solutions for flexible-joint robots are developed based on precise sensor measurements, and they may fail to achieve satisfactory performance when real state information is not available. Moreover, the prior knowledge about model parameters and measurement sensitivity is difficult or impossible to exactly obtain in practice, which seriously hinders the further application of control methods that are dependent on system dynamics. To this end, this paper proposes a novel tracking control scheme based on measurement information for flexible-joint robots with unknown measurement sensitivity, where the dependence on model information is eliminated with the elaborately constructed Lyapunov function candidates, and the real tracking error is still adjusted to an acceptable range even if there are measurement errors. Preliminary experiments on a flexible-joint robot developed by Quanser company demonstrate the feasibility and effectiveness of the proposed method. In future studies, designing an effective scheme to achieve direct preset tracking control is the focus of the work.

Dynamic path planning via Dueling Double Deep Q-Network (D3QN) with prioritized experience replay

Path planning is a key requirement for mobile robots employed for different tasks such as rescue or transport missions. Conventional methods such as A* or Dijkstra to tackle path planning problem need a premise map of the robot's environment. Nowadays, dynamic path planning is a popular research topic, which drives mobile robots without prior static requirements. Deep reinforcement learning (DRL), which is another popular research area, is being harnessed to solve dynamic path planning problem by the researchers. In this study, Deep Q-Networks, which is a subdomain of DRL are opted to solve dynamic path planning problem. We first employ well known techniques Double Deep Q-Networks (D2QN) and Dueling Double Deep Q-Networks (D3QN) to train a model which can drive a mobile robot in environments with static and dynamic obstacles within 3 different configurations. Then we propose D3QN with Prioritized Experience Replay (PER) extension in order to further optimize the DRL model. We created a test bed to measure the performance of the DRL models against 99 randomly generated goal locations. According to our experiments, D3QN-PER method performs better than D2QN and D3QN in terms of path length and travel time to the goal without any collisions. Robot Operating System and Gazebo simulation environment is utilized to realize the training and testing environments, thus, the trained DRL models can be deployed to any ROS compatible robot seamlessly.

Robot skill learning and the data dilemma it faces: a systematic review

PurposeCompared with traditional methods relying on manual teaching or system modeling, data-driven learning methods, such as deep reinforcement learning and imitation learning, show more promising potential to cope with the challenges brought by increasingly complex tasks and environments, which have become the hot research topic in the field of robot skill learning. However, the contradiction between the difficulty of collecting robot–environment interaction data and the low data efficiency causes all these methods to face a serious data dilemma, which has become one of the key issues restricting their development. Therefore, this paper aims to comprehensively sort out and analyze the cause and solutions for the data dilemma in robot skill learning.Design/methodology/approachFirst, this review analyzes the causes of the data dilemma based on the classification and comparison of data-driven methods for robot skill learning; Then, the existing methods used to solve the data dilemma are introduced in detail. Finally, this review discusses the remaining open challenges and promising research topics for solving the data dilemma in the future.FindingsThis review shows that simulation–reality combination, state representation learning and knowledge sharing are crucial for overcoming the data dilemma of robot skill learning.Originality/valueTo the best of the authors’ knowledge, there are no surveys that systematically and comprehensively sort out and analyze the data dilemma in robot skill learning in the existing literature. It is hoped that this review can be helpful to better address the data dilemma in robot skill learning in the future.

Graph-based meta-learning for context-aware sensor management in nonlinear safety-critical environments

This study introduces a novel framework for optimizing energy efficiency and computational load in safety-critical robotic systems operating in nonlinear domains. Leveraging Graph Attention Networks for state awareness and decision-making, the framework employs adaptive sensor and filter toggling strategies to dynamically manage system resources through real-time inferential processes. Our framework maintains continuous robot operation in the presence of sensor noise and environmental disturbances by activating additional sensors, thus preventing system shutdowns or stalls. Few-shot meta-learning techniques further augment the model's adaptability, allowing it to generalize and make real-time decisions across varying operational conditions. An extensive evaluation reveals a reduced average energy consumption, compared to ‘always-on’ configurations, by 13.71% and CPU utilization by 29.07%, without compromising system performance and safety. We also introduce Matching Networks and Siamese Networks with different loss functions to assess the system's capability to adapt to different levels of criticality. Our experiments demonstrate that the system prioritizes performance and safety in high-critical scenarios while maximizing energy efficiency in less critical situations. The framework's real-time decision-making capability is particularly crucial in human–robot environments and holds significant implications for future applications in nonlinear control systems and resilient robotic systems.