Autonomous Mobile Robot System Research Articles

B-Splined Trajectory Modified Generation to Maximize Speed of the Nonholonomic AMR Robot

Abstract: Trajectory path generation is critical for the Autonomous Mobile Robot (AMR) when moving frequently in the working environment in the shop floor to transport loads from one work station to another continuously. Traditionally, the AMR moves from one point to stop at the next point or turn which is inefficient and consumes much energy. This paper proposes the new concept of AMR trajectory path planning with curvature driven by maximizing speed control of the differential drive at each curve to move smoothly. B-splined is commonly applied to CAD and CAM machining effectively for tool path trajectory. Therefore, the B-splined curvature is studied and validated by simulation together with energy consumption. The simulation is on Matlab Simulink with numerical model. It is investigated that the B-splined trajectory is efficient in animating the AMR’s actual system. The velocity can obtain both linear and angular velocity of the AMR movements on forward and backward directions as well as the acceleration. The trajectory can be selected based on the degree of closeness and used to generate speed and velocity control for the AMR system.

Complete Visibility Algorithms of Luminous Robots With Two‐Color Lights on Grid

ABSTRACTAn autonomous mobile robot system is a distributed system consisting of multiple mobile computational entities, called robots, which autonomously and repeatedly perform three fundamental operations: look, compute, and move. Various challenges in such systems, including gathering, pattern formation, and flocking, have been extensively studied to explore the relationship between the robots' capabilities and the feasibility of solving these problems (i.e., solvability). In this study, we focus on the complete visibility problem, which aims to relocate all robots on an infinite grid plane so that every robot is visible to every other robot (i.e., complete visibility). We assume that each robot is a luminous robot (i.e., has a light with a constant number of colors) and opaque (non‐transparent). This paper primarily examines the number of light colors required for each robot to solve the complete visibility problem. Specifically, we investigate the question: “how many colors can we reduce while still achieving complete visibility?” (even under certain stronger assumptions). As an answer to the above question, we show the existence of a deterministic algorithm to achieve complete visibility (i.e., every robot can observe all the other robots) using only two colors of light, if the robots agree on the directions and orientations of both axes. The proposed algorithm correctly works even if robots operate asynchronously and have no knowledge of the total number of robots. Moreover, its spatial complexity (i.e., the area of the smallest enclosed rectangle that includes all robots in the final configuration) ensures the optimal one, , where is the number of robots.

Mapping and Localization of Autonomous Mobile Robots in Simulated Indoor Environments

Autonomously making a map, localizing within it, and planning with it are fundamental problems in mobile robotics. Every autonomous mobile robot system must include a solution to all three problems. These three problems are interconnected, with simultaneous localization and mapping (SLAM) being a well-known issue. However, there is indeed a growing and developing realization in the research field that path planning how a robot goes about mapping and finding an environment (and then operating in the environment such as starting to the destination point) can avoid degenerate conditions and greatly reduce SLAM complexity. In this paper, the implementation of an autonomous mobile robot system for indoor environments using open-source ROS packages and a combination of cartography algorithm and adaptive Monte Carlo localization (AMCL) algorithms has been implemented. The system addresses the challenge of developing three components such as mapping, localization, and path planning systems for indoor autonomous mobile robots. The mapping module creates a global map using the cartography ROS package and SLAM algorithm. The localization module estimates the robot's pose using the AMCL approach. The planning module generates collision-free trajectories and control commands using the moving base ROS package. The experimental results demonstrate the effectiveness of this approach and its valuable contribution to the robotics field. The cartography algorithm mapping algorithm generates accurate and reliable maps, while the localization algorithm successfully determines the robot's position with good performance. Additionally, the path planning algorithm effectively avoids both static and dynamic obstacles, ensuring smooth navigation in the environment.

A new CNN-BASED object detection system for autonomous mobile robots based on real-world vehicle datasets

Recently, autonomous mobile robots (AMRs) have begun to be used in the delivery of goods, but one of the biggest challenges faced in this field is the navigation system that guides a robot to its destination. The navigation system must be able to identify objects in the robot's path and take evasive actions to avoid them. Developing an object detection system for an AMR requires a deep learning model that is able to achieve a high level of accuracy, with fast inference times, and a model with a compact size that can be run on embedded control systems. Consequently, object recognition requires a convolutional neural network (CNN)-based model that can yield high object classification accuracy and process data quickly. This paper introduces a new CNN-based object detection system for an AMR that employs real-world vehicle datasets. First, we create original real-world datasets of images from Banda Aceh city. We then develop a new CNN-based object identification system that is capable of identifying cars, motorcycles, people, and rickshaws under morning, afternoon, and evening lighting conditions. An SSD Mobilenetv2 FPN Lite 320 × 320 architecture is employed for retraining using these real-world datasets. Quantitative and qualitative performance indicators are then applied to evaluate the CNN model. Training the pre-trained SSD Mobilenetv2 FPN Lite 320 × 320 model improves its classification and detection accuracy, as indicated by its performance results. We conclude that the proposed CNN-based object detection system has the potential for use in an AMR.

Order sequencing, tote scheduling, and robot routing optimization in multi-tote storage and retrieval autonomous mobile robot systems

This study explores a novel multi-tote storage and retrieval autonomous mobile robot system, where multi-tote autonomous mobile robots transport totes (or ‘SKU bins’) for order picking. We investigate the joint optimisation problem of order sequencing, tote scheduling, and robot routing in a single workstation equipped with the capacity to accommodate multiple order bins and parallel totes. The inbound and outbound streams of SKUs stored in totes are crucial for order picking at the workstation, as they jointly affect the picking performance efficiency through synchronization with the order sequence. To address this synchronization problem, we formulate a Mixed-Integer Linear Programming (MILP) model and propose a two-stage hybrid heuristic combining a variable neighbourhood search (VNS) algorithm and a refinement model. This model further improves the VNS solution with partially fixed SKU inbound stream and problem-tailored inequalities. Our numerical studies highlight the superior performance of the proposed hybrid heuristic, where the VNS solution surpasses the benchmark VNS-Simplified by a significant margin of 14%. The rearrangement process enhances the VNS performance by reducing 33.8% of SKU revisits. We also offer managerial insights, suggesting that the maximum number of order bins and parallel totes, exhibit boundary points where adding capacity yields limited marginal improvements.

Design of a Mobile Industrial Robot Platform based on Fixed Cameras

This study presents a design for an autonomous mobile robot system that utilizes environmental cameras for navigation and obstacle avoidance. Operating within a known environment, the robot employs these cameras to facilitate path planning and to circumvent obstacles. The hardware setup includes fixed cameras strategically placed in the environment to pinpoint the robot’s location and to identify obstacles amidst varying conditions. A computer system is essential for processing the data captured by the cameras, and a wireless transmission system communicates operational instructions to the robot.For navigation, the study explores various algorithms for path planning, selecting the most efficient one through comparative analysis. Once the path is established, the robot’s instructions are computed under the designed control rate, employing the vehicle’s PD control method. This method is informed by the external camera’s simulation of the vehicle, ensuring accurate and responsive navigation.The project’s goal is to create a mobile robot platform that leverages an external camera as its primary sensor. This design promises efficient autonomous navigation and the capability to avoid static obstacles. The robot’s design is conceptualized using SolidWorks and brought to life through implementation in the Python and MATLAB programming languages, demonstrating the system’s practicality and adaptability in real-world scenarios. Additionally, the system’s design considers ease of integration and scalability to accommodate future technological advancements and new application requirements.

Design an omnidirectional autonomous mobile robot based on non‐linear optimal control to track a specified path

AbstractThis paper explores two non‐linear control techniques for designing an effective control system for an omnidirectional autonomous mobile robot with four Mecanum wheels. Due to the unique wheel structure and four separate wheels, the robot has non‐linear dynamics, multiple inputs and outputs. The first technique uses the state‐dependent Riccati equation (SDRE) to address optimal non‐linear control while considering energy and time constraints. The second technique, using an intermediate variable , has expanded the Hamilton‐Jacobi‐Belman equation in terms of the power series. Consequently, these equations are reduced to a set of recursive Lyapunov algebraic equations, leading to a closed‐form solution for solving the non‐linear optimal control problem. Finally, the maneuverability and path‐tracking capability of the robot are examined by highlighting the non‐linear term through numerical simulation.

Development of Autonomous Mobile Robot with 3DLidar Self-Localization Function Using Layout Map

In recent years, there has been growing interest in autonomous mobile robots equipped with Simultaneous Localization and Mapping (SLAM) technology as a solution to labour shortages in production and distribution settings. SLAM allows these robots to create maps of their environment using devices such as Lidar, radar, and sonar sensors, enabling them to navigate and track routes without prior knowledge of the environment. However, the manual operation of these robots for map construction can be labour-intensive. To address this issue, this research aims to develop a 3D SLAM autonomous mobile robot system that eliminates the need for manual map construction by utilizing existing layout maps. The system includes a PC for self-position estimation, 3DLidar, a camera for verification, a touch panel display, and the mobile robot itself. The proposed SLAM method extracts stable wall point cloud information from 3DLidar, matches it with the wall surface information in the layout map, and uses a particle filter to estimate the robot’s position. The system also includes features such as route creation, tracking, and obstacle detection for autonomous movement. Experiments were conducted to compare the proposed system with conventional 3D SLAM methods. The results showed that the proposed system significantly reduced errors in self-positioning and enabled accurate autonomous movement on specified routes, even in the presence of slight differences in layout maps and obstacles. Ultimately, this research demonstrates the effectiveness of a system that can transport goods without the need for manual environment mapping, addressing labour shortages in such environments.

Learning Selective Sensor Fusion for State Estimation.

Autonomous vehicles and mobile robotic systems are typically equipped with multiple sensors to provide redundancy. By integrating the observations from different sensors, these mobile agents are able to perceive the environment and estimate system states, e.g., locations and orientations. Although deep learning (DL) approaches for multimodal odometry estimation and localization have gained traction, they rarely focus on the issue of robust sensor fusion--a necessary consideration to deal with noisy or incomplete sensor observations in the real world. Moreover, current deep odometry models suffer from a lack of interpretability. To this extent, we propose SelectFusion, an end-to-end selective sensor fusion module that can be applied to useful pairs of sensor modalities, such as monocular images and inertial measurements, depth images, and light detection and ranging (LIDAR) point clouds. Our model is a uniform framework that is not restricted to specific modality or task. During prediction, the network is able to assess the reliability of the latent features from different sensor modalities and to estimate trajectory at both scale and global pose. In particular, we propose two fusion modules--a deterministic soft fusion and a stochastic hard fusion--and offer a comprehensive study of the new strategies compared with trivial direct fusion. We extensively evaluate all fusion strategies both on public datasets and on progressively degraded datasets that present synthetic occlusions, noisy and missing data, and time misalignment between sensors, and we investigate the effectiveness of the different fusion strategies in attending the most reliable features, which in itself provides insights into the operation of the various models.

Motion planning and control of an autonomous mobile robot

The application of autonomous mobile robots (AMRs) has gradually become crucial in smart factories due to the advantages of improving production efficiency and reducing labour costs. Motion planning has been a key part of AMR control development. This paper presents motion planning and position tracking control systems of an omnidirectional wheel AMR powered by a hybrid fuel cell and battery power source. First, the kinematical and dynamic models of the AMR are introduced. The navigation system comprises three loops, with the first loop being motor control, the second loop being position tracking control, and a motion planning layer. The position data of the AMR for feedback control is obtained through sensor fusion of data from the inertial measurement unit (IMU) sensor, encoder sensor, and ranging sensor with simultaneous localisation and mapping (SLAM) algorithm. The motion planning is then applied to obtain an optimal path with the shortest distance and collision-free movement. In addition, the tracking algorithm is designed to drive the AMR to follow the optimal path and achieve high accuracy. The experimental results show a 30% improvement in tracking accuracy compared to traditional approaches and 8 hours of continuous working, which is promising for industrial applications, and the results are satisfactory in terms of both accuracy and efficiency requirements.

An Industry 5.0 Perspective on Feeding Production Lines

The emerging concept of Industry 5.0 is fostering companies to consider the three pillars of human-centricity, sustainability, and resilience. How such a new perspective can be effectively declined and practically guide the introduction of new technologies is a challenge to be addressed. This study proposes a framework to support companies when introducing new solutions to feed production lines by adopting an Industry 5.0 perspective. For each fundamental pillar, critical points to focus on have been highlighted and operational checklists have been developed to effectively support the analysis and implementation of new solutions. The application of the framework and related operational checklists to a case study regarding the integration of an Autonomous Mobile Robot system has proved its validity. Following the human-centricity checklist, full acceptance of the new technology by workers has been gained, together with a safer workplace. Energy savings for material handling and recycling have been supported about the sustainability pillar, while redundancy and backup systems have increased the resilience of the feeding system.

Developing a dynamic obstacle avoidance system for autonomous mobile robots using Bayesian optimization and object tracking: Implementation and testing

This study addresses the challenges faced by autonomous mobile robots in dynamic environments where moving obstacles such as people, cars, and other mobile robots are prevalent. Current motion planning methods for mobile robots focus on stationary obstacles or equivalent static ones at each control moment, resulting in collisions with moving objects or the need for longer distances to avoid them. To address this issue, we propose a system that incorporates dynamic information about moving obstacles into the motion planning decision-making process. We developed an object tracking technique using 2D LiDARs to gather this information and optimized the relevant parameters using Bayesian optimization. The dynamic information of moving obstacles is saved in a costmap representation at each instant, allowing the robot to yield or ignore confronting obstacles based on the prediction of who shall pass first. Simulation and real-world testing demonstrate that our approach saves paths for mobile robots in dynamic environments and prevents them from colliding with dynamic obstacles, improving the practicality of mobile robots working alongside moving humans.

Challenges and Solutions for Autonomous Ground Robot Scene Understanding and Navigation in Unstructured Outdoor Environments: A Review

The capabilities of autonomous mobile robotic systems have been steadily improving due to recent advancements in computer science, engineering, and related disciplines such as cognitive science. In controlled environments, robots have achieved relatively high levels of autonomy. In more unstructured environments, however, the development of fully autonomous mobile robots remains challenging due to the complexity of understanding these environments. Many autonomous mobile robots use classical, learning-based or hybrid approaches for navigation. More recent learning-based methods may replace the complete navigation pipeline or selected stages of the classical approach. For effective deployment, autonomous robots must understand their external environments at a sophisticated level according to their intended applications. Therefore, in addition to robot perception, scene analysis and higher-level scene understanding (e.g., traversable/non-traversable, rough or smooth terrain, etc.) are required for autonomous robot navigation in unstructured outdoor environments. This paper provides a comprehensive review and critical analysis of these methods in the context of their applications to the problems of robot perception and scene understanding in unstructured environments and the related problems of localisation, environment mapping and path planning. State-of-the-art sensor fusion methods and multimodal scene understanding approaches are also discussed and evaluated within this context. The paper concludes with an in-depth discussion regarding the current state of the autonomous ground robot navigation challenge in unstructured outdoor environments and the most promising future research directions to overcome these challenges.

Learning Observation Model for Factor Graph Based-State Estimation Using Intrinsic Sensors

The navigation system of autonomous mobile robots has appeared challenging when using exteroceptive sensors such as cameras, LiDARs, and radars in textureless and structureless environments. This paper presents a robust state estimation system for holonomic mobile robots using intrinsic sensors based on adaptive factor graph optimization in the degradation scenarios. In particular, the neural networks are employed to learn the observation and noise model using only IMU sensor and wheel encoder data. Investigating the learning model for the holonomic mobile robot is discussed with various neural network architectures. We also explore the neural networks that are far more powerful and have cheaper computing power when using the inertial-wheel encoder sensors. Furthermore, we employ an industrial holonomic robot platform equipped with multiple LiDARs, cameras, IMU, and wheel encoders to conduct the experiments and create the ground truth without a bulky motion capture system. The collected datasets are then implemented to train the neural networks. Finally, the experimental evaluation presents that our solution provides better accuracy and real-time performance than other solutions. <i>Note to Practitioners</i>&#x2014;Autonomous mobile robots need to serve in challenging environments robustly that deny extrinsic sensors such as cameras, LiDARs, and radars. In order to operate in this situation, the navigation system shall rely on the intrinsic sensor as inertial sensor and wheel encoders. Existing conventional methods have combined the intrinsic sensors in the form of the recursive Bayesian filtering technique without adapting these models. Besides, deep learning-based solutions have adopted extensive networks like LSTM or CNN to handle the estimation problem. This work aims to develop a state estimation subsystem of the navigation system for the holonomic mobile robots that utilize the intrinsic sensors in adaptive factor graph optimization. In particular, we present how to join the factor graph efficiently with the learning observation model of IMU and wheel encoder factor. Moreover, the neural networks are introduced to learn the observation model with an IMU and wheel encoder data inputs. We recognize that lightweight neural networks can achieve more power than deep learning techniques using the IMU sensor and wheel encoders. Finally, the neural networks are embedded in a factor graph to handle the smoothing state estimation. The proposed system could operate with high accuracy in real time.

Design and Implementation of an Artificial Intelligence of Things-Based Autonomous Mobile Robot System for Pitaya Harvesting

Since aging in agriculture has currently become a problem in many advanced countries worldwide, in response to the growing shortage of agricultural labor, many of these countries have invested in the development of agricultural robots to solve the agricultural labor shortage problem. Automatic equipment to support agricultural harvesting can reduce the labor demand. As a result, this article proposes an artificial intelligence of things (AIoT)-based autonomous mobile robot (AMR) system for pitaya harvesting. The proposed system uses an artificial intelligence (AI) edge computing-based development board (NVIDIA Jetson Nano development board) and combines a 2-D simultaneous localization and mapping (SLAM) algorithm and an AI object recognition module. The SLAM algorithm is used for environmental detection in an unknown environment, and surrounding environment information can be detected by the sensor for map construction to facilitate robot navigation in pitaya orchards. In addition, this article describes an AI object recognition module used for pitaya recognition to facilitate pitaya harvesting. The accuracy of the proposed pitaya recognition model can reach 96.7% on the adopted NVIDIA Jetson Nano development board. A pitaya orchard is simulated in the experimental environment discussed in this article. In the simulated experimental environment, the proposed AMR system can achieve efficient pitaya harvesting and realize intelligent farming.

Model-Predictive Control for Omnidirectional Mobile Robots in Logistic Environments Based on Object Detection Using CNNs.

Object detection is an essential component of autonomous mobile robotic systems, enabling robots to understand and interact with the environment. Object detection and recognition have made significant progress using convolutional neural networks (CNNs). Widely used in autonomous mobile robot applications, CNNs can quickly identify complicated image patterns, such as objects in a logistic environment. Integration of environment perception algorithms and motion control algorithms is a topic subjected to significant research. On the one hand, this paper presents an object detector to better understand the robot environment and the newly acquired dataset. The model was optimized to run on the mobile platform already on the robot. On the other hand, the paper introduces a model-based predictive controller to guide an omnidirectional robot to a particular position in a logistic environment based on an object map obtained from a custom-trained CNN detector and LIDAR data. Object detection contributes to a safe, optimal, and efficient path for the omnidirectional mobile robot. In a practical scenario, we deploy a custom-trained and optimized CNN model to detect specific objects in the warehouse environment. Then we evaluate, through simulation, a predictive control approach based on the detected objects using CNNs. Results are obtained in object detection using a custom-trained CNN with an in-house acquired data set on a mobile platform and in the optimal control for the omnidirectional mobile robot.

Design and Implementation of an Artificial Intelligence of Things-Based Autonomous Mobile Robot System for Cleaning Garbage

This article proposes an autonomous mobile robot (AMR) system based on the artificial intelligence of things (AIoT) for collecting garbage. The proposed system consists of an AMR subsystem, a robot operating system (ROS) robot subsystem, an AI garbage recognition subsystem, solar power supply subsystem, and a smart trash can subsystem. The AMR subsystem uses a motor drive, which is built on an aluminum alloy metal base and is connected to an AI edge computing module via an app, to control the robot. The ROS subsystem is built on an edge computing module, which is coupled with the motor drive module to control the robot arm and the robot gripper, and is connected to the AI edge computing module to control the robot arm through the app. The AI garbage recognition subsystem recognizes the types of waste in real time through an image sensor (camera). This subsystem adopts multiple sensors that can monitor the level of waste in the trash can. Moreover, a map on a web platform locates all trash cans. Servomotors enable automatic opening and closing of the lid to prevent the unpleasant smell of garbage from wafting. A user connects to the AI edge computing module through an app to monitor the robot and control the movement of the robot to any location on the map.

Path Planning of Autonomous Mobile Robots Based on an Improved Slime Mould Algorithm

Path planning is a crucial component of autonomous mobile robot (AMR) systems. The slime mould algorithm (SMA), as one of the most popular path-planning approaches, shows excellent performance in the AMR field. Despite its advantages, there is still room for SMA to improve due to the lack of a mechanism for jumping out of local optimization. This means that there is still room for improvement in the path planning of ARM based on this method. To find shorter and more stable paths, an improved SMA, called the Lévy flight-rotation SMA (LRSMA), is proposed. LRSMA utilizes variable neighborhood Lévy flight and an individual rotation perturbation and variation mechanism to enhance the local optimization ability and prevent falling into local optimization. Experiments in varying environments demonstrate that the proposed algorithm can generate the ideal collision-free path with the shortest length, higher accuracy, and robust stability.

Learning Type-2 Fuzzy Logic for Factor Graph Based-Robust Pose Estimation With Multi-Sensor Fusion

Although a wide variety of high-performance state estimation techniques have been introduced recently, the robustness and extension to actual conditions of the estimation systems have been challenging. This paper presents a robust adaptive state estimation framework based on the Type-2 fuzzy inference system and factor graph optimization for autonomous mobile robots. We use the hybrid solution to connect the advantages of the tightly and loosely coupled technique by providing an inertial sensor and other extrinsic sensors such as LiDARs and cameras. In order to tackle the uncertainty input covariance and sensor failures problems, a learnable observation model is introduced by joining the Type-2 FIS and factor graph optimization. In particular, the use of Type-2 Takagi-Sugeno FIS can learn the uncertainty by using particle swarm optimization before adding the observation model to the factor graph. The proposed design consists of four parts: sensor odometry, up-sampling, FIS based-learning observation model, and factor graph-based smoothing. We evaluate our system by using a mobile robot platform equipped with a sensor setup of multiple stereo cameras, an IMU, and a LiDAR sensor. We imitate the LiDAR odometry in structure environments without needing other bulky motion capture systems to learn the observation model of the visual-inertial estimators. The experimental results are deployed in real-world environments to present the accuracy and robustness of the algorithm.

Construction of a mathematical model of the dynamics of an autonomous mobile robot of variable configuration

This paper considers the construction of a mathematical model of the movement of an autonomous mobile robot (AMR) in variable configuration, taking into account the relationship of the dynamic parameters of a mechanical system. As an example, the design of AMR with a manipulator is considered. The object of this study is the dynamics of AMR with a manipulator. The peculiarities of the dynamics of AMR with the manipulator are due to the change in the position of the center of mass of the system with the relative movement of the manipulator and the commensurate non-diagonal and diagonal elements of the inertia tensor calculated relative to the axes of the base coordinate system. The construction of the mathematical model was carried out according to the Nyton-Euler method. The resulting mathematical model contains: – an equation of motion of the center of mass of the AMR system of variable configuration along the trajectory in the inertial coordinate system; – an equation of angular motion of AMR in variable configuration in the inertial coordinate system; – an equation of motion of the manipulator with respect to AMR. In a general case, the center of mass of the AMR platform moves in a horizontal plane. Establishing the relationship of dynamic parameters of the mechanical system will make it possible to maintain functionality and ensure the orientation of AMR in vertical planes despite the movement of the manipulator. As an object of control, AMR with a manipulator is a multi-connected system with a cross-internal connection of control channels, which is formed by the dynamic parameters of a mechanical system. Based on the results of mathematical modeling using the proposed model, it is possible to develop algorithms for adaptive control using cross-connection of channels. This will make it possible to identify reserves to reduce energy consumption, increase stability, improve the efficiency and survivability of AMR in variable configuration during autonomous work under extreme conditions.