Mobile Robot System Research Articles

Localization of Mobile Robots Based on Depth Camera

In scenarios of indoor localization of mobile robots, Global Positioning System (GPS) signals are prone to loss due to interference from urban building environments and cannot meet the needs of robot localization. On the other hand, traditional indoor localization methods based on wireless signals such as Bluetooth and WiFi often require the deployment of multiple devices in advance, and these methods can only obtain distance information and are unable to obtain the attitude of the positioning target in space. This paper proposes a method for the indoor localization of mobile robots based on a depth camera. Firstly, we extracted ORB feature points from images captured by a depth camera and performed homogenization processing. Then, we performed feature matching between adjacent two frames of images, and the mismatched points are eliminated to improve the accuracy of feature matching. Finally, we used the Iterative Closest Point (ICP) algorithm to estimate the camera’s pose, thus achieving the localization of mobile robots in indoor environments. In addition, an experimental evaluation was conducted on the TUM dataset of the Technical University of Munich to validate the feasibility of the proposed depth-camera-based indoor localization system for mobile robots. The experimental results show that the average localization accuracy of our algorithm on three datasets is 0.027 m, which can meet the needs of indoor localization for mobile robots.

Motion-Tracking Control of Mobile Manipulation Robotic Systems Using Artificial Neural Networks for Manufacturing Applications

Robotic systems have experienced exponential growth in their utilization for manufacturing applications over recent decades. Control systems responsible for executing desired robot motion planning face increasingly stringent performance requirements. These demands encompass high precision, efficiency, stability, robustness, ease of use, and simplicity of the user interface. Furthermore, diverse modern manufacturing applications primarily employ robotic systems within disturbed operating scenarios. This paper presents a novel neural motion-tracking control scheme for mobile manipulation robotic systems. Dynamic position output error feedback and B–Spline artificial neural networks are integrated in the design process of the introduced adaptive robust control strategy to perform efficient and robust tracking of motion-planning trajectories in robotic systems. Integration of artificial neural networks demonstrates performance improvements in the control scheme while effectively addressing common issues encountered in manufacturing environments. Parametric uncertainty, unmodeled dynamics, and unknown disturbance torque terms represent some adverse influences to be compensated for by the robust control scheme. Several case studies prove the robustness of the adaptive neural control scheme in highly coupled nonlinear six-degree-of-freedom mobile manipulation robotic systems. Case studies provide valuable insights and validate the efficacy of the proposed adaptive multivariable control scheme in manufacturing applications.

Adaptive neural network control for nonholonomic systems with partial/full or without state constraints

In this paper, we present a unifying neural networks-based adaptive control framework for nonholonomic systems with partial/full state constraints or without constraint requirements. By considering the tan-type barrier Lyapunov function, constraint requirements are handled for the x0-subsystem. A novel nonlinear state-dependent function is constructed to meet asymmetric time-varying state constraints for the x-subsystem. Combined with backstepping method, a coordinate transformation is designed to transform the original constrained system into unconstrained system. Then, we put forward the dynamic surface control to eliminate feasibility conditions of virtual controllers. In addition, neural networks can be introduced to compensate for the uncertainties of nonholonomic systems. It is demonstrated that without modifying control structure the adaptive neural networks controller guarantees the stability of nonholonomic systems, while asymmetric time-varying state constraints can be satisfied. Simulation examples of mobile robot systems further validate the efficacy of the devised algorithm.

Adaptive sliding mode control for uncertain wheel mobile robot

<span lang="EN-US">In this paper a simple adaptive sliding mode controller is proposed for tracking control of the wheel mobile robot (WMR) systems. The WMR are complicated systems with kinematic and dynamic model so the error dynamic model is built to simplify the mathematical model. The sliding mode control then is designed for this error model with the adaptive law to compensate for the mismatched. The proposed control scheme in this work contains only one control loop so it is simple in both implementation and mathematical calculation. Moreover, the requirement of upper bounds of disturbance that is popular in the sliding mode control is cancelled, so it is convenient for real world applications. Finally, the effectiveness of the presented algorithm is verified through mathematical proof and simulations. The comparison with the existing work is also executed to evaluate the correction of the introduced adaptive sliding mode controller. Thoroughly, the settling time, the peak value, the integral square error of the proposed control scheme reduced about 50% in comparison with the compared disturbance observer based sliding mode control.</span>

Smooth and collision-free path planning for holonomic mobile robot based RRT-Connect

Robot navigation is a crucial aspect of any mobile robot system. Many path planners generate a path that is zigzag in nature and contains many sharp or angular turns. Such courses are not adequate for a mobile robot to follow as it has to slow down and then speed up at these sharp turns. Thus, the integration of the smoothing function is a vital aspect to generate smooth and continuous paths. Moreover, a collision check after path smoothing needs to be accounted for a possible collision with obstacles surrounding. Herein, a path pruning algorithm is utilized to reduce the total path waypoints. Then a piecewise cubic B-spline smoothing function is applied to ensure the path is smooth and continuous. In addition, a post-smoothing collision checking and improvement algorithm based on mid-point insertion for the collide segments is proposed. Experiments and comparisons with the geometric RRT-Connect path planner have shown that the inclusion of a path pruning algorithm, smoothing function and post-smoothing collision check generates better results with reduced path length.

Hybrid motion control for multiple mobile robot systems

Hybrid motion control for multiple mobile robot systems

Learning Observation Model for Factor Graph Based-State Estimation Using Intrinsic Sensors

The navigation system of autonomous mobile robots has appeared challenging when using exteroceptive sensors such as cameras, LiDARs, and radars in textureless and structureless environments. This paper presents a robust state estimation system for holonomic mobile robots using intrinsic sensors based on adaptive factor graph optimization in the degradation scenarios. In particular, the neural networks are employed to learn the observation and noise model using only IMU sensor and wheel encoder data. Investigating the learning model for the holonomic mobile robot is discussed with various neural network architectures. We also explore the neural networks that are far more powerful and have cheaper computing power when using the inertial-wheel encoder sensors. Furthermore, we employ an industrial holonomic robot platform equipped with multiple LiDARs, cameras, IMU, and wheel encoders to conduct the experiments and create the ground truth without a bulky motion capture system. The collected datasets are then implemented to train the neural networks. Finally, the experimental evaluation presents that our solution provides better accuracy and real-time performance than other solutions. <i>Note to Practitioners</i>&#x2014;Autonomous mobile robots need to serve in challenging environments robustly that deny extrinsic sensors such as cameras, LiDARs, and radars. In order to operate in this situation, the navigation system shall rely on the intrinsic sensor as inertial sensor and wheel encoders. Existing conventional methods have combined the intrinsic sensors in the form of the recursive Bayesian filtering technique without adapting these models. Besides, deep learning-based solutions have adopted extensive networks like LSTM or CNN to handle the estimation problem. This work aims to develop a state estimation subsystem of the navigation system for the holonomic mobile robots that utilize the intrinsic sensors in adaptive factor graph optimization. In particular, we present how to join the factor graph efficiently with the learning observation model of IMU and wheel encoder factor. Moreover, the neural networks are introduced to learn the observation model with an IMU and wheel encoder data inputs. We recognize that lightweight neural networks can achieve more power than deep learning techniques using the IMU sensor and wheel encoders. Finally, the neural networks are embedded in a factor graph to handle the smoothing state estimation. The proposed system could operate with high accuracy in real time.

Particle Swarm Algorithm Path-Planning Method for Mobile Robots Based on Artificial Potential Fields

Path planning is an important part of the navigation control system of mobile robots since it plays a decisive role in whether mobile robots can realize autonomy and intelligence. The particle swarm algorithm can effectively solve the path-planning problem of a mobile robot, but the traditional particle swarm algorithm has the problems of a too-long path, poor global search ability, and local development ability. Moreover, the existence of obstacles makes the actual environment more complex, thus putting forward more stringent requirements on the environmental adaptation ability, path-planning accuracy, and path-planning efficiency of mobile robots. In this study, an artificial potential field-based particle swarm algorithm (apfrPSO) was proposed. First, the method generates robot planning paths by adjusting the inertia weight parameter and ranking the position vector of particles (rPSO), and second, the artificial potential field method is introduced. Through comparative numerical experiments with other state-of-the-art algorithms, the results show that the algorithm proposed was very competitive.

Mobile Robot System for Selective Asparagus Harvesting

Asparagus harvesting presents unique challenges, due to the variability in spear growth, which makes large-scale automated harvesting difficult. This paper describes the development of an asparagus harvesting robot system. The system consists of a delta robot mounted on a mobile track-based platform. It employs a real-time asparagus detection algorithm and a sensory system to determine optimal harvesting points. Low-level control and high-level control are separated in the robot control. The performance of the system was evaluated in a laboratory field mock-up and in the open field, using asparagus spears of various shapes. The results demonstrate that the system detected and harvested 88% of the ready-to-harvest spears, with an average harvesting cycle cost of 3.44s±0.14s. In addition, outdoor testing in an open field demonstrated a 77% success rate in identifying and harvesting asparagus spears.

Autonomous Navigation of Robots: Optimization with DQN

In the field of artificial intelligence, control systems for mobile robots have undergone significant advancements, particularly within the realm of autonomous learning. However, previous studies have primarily focused on predefined paths, neglecting real-time obstacle avoidance and trajectory reconfiguration. This research introduces a novel algorithm that integrates reinforcement learning with the Deep Q-Network (DQN) to empower an agent with the ability to execute actions, gather information from a simulated environment in Gazebo, and maximize rewards. Through a series of carefully designed experiments, the algorithm’s parameters were meticulously configured, and its performance was rigorously validated. Unlike conventional navigation systems, our approach embraces the exploration of the environment, facilitating effective trajectory planning based on acquired knowledge. By leveraging randomized training conditions within a simulated environment, the DQN network exhibits superior capabilities in computing complex functions compared to traditional methods. This breakthrough underscores the potential of our algorithm to significantly enhance the autonomous learning capacities of mobile robots.

Design and Implementation of an Artificial Intelligence of Things-Based Autonomous Mobile Robot System for Pitaya Harvesting

Since aging in agriculture has currently become a problem in many advanced countries worldwide, in response to the growing shortage of agricultural labor, many of these countries have invested in the development of agricultural robots to solve the agricultural labor shortage problem. Automatic equipment to support agricultural harvesting can reduce the labor demand. As a result, this article proposes an artificial intelligence of things (AIoT)-based autonomous mobile robot (AMR) system for pitaya harvesting. The proposed system uses an artificial intelligence (AI) edge computing-based development board (NVIDIA Jetson Nano development board) and combines a 2-D simultaneous localization and mapping (SLAM) algorithm and an AI object recognition module. The SLAM algorithm is used for environmental detection in an unknown environment, and surrounding environment information can be detected by the sensor for map construction to facilitate robot navigation in pitaya orchards. In addition, this article describes an AI object recognition module used for pitaya recognition to facilitate pitaya harvesting. The accuracy of the proposed pitaya recognition model can reach 96.7% on the adopted NVIDIA Jetson Nano development board. A pitaya orchard is simulated in the experimental environment discussed in this article. In the simulated experimental environment, the proposed AMR system can achieve efficient pitaya harvesting and realize intelligent farming.

Dynamic event-driven neural network-based adaptive fault-attack-tolerant control for wheeled mobile robot system

In this article, the fault-attack control problem is investigated for wheeled mobile robot (WMR) systems subjected to actuator faults, disturbances, communication attacks, and limited communication resources. An event-observer based compensation controller is presented. With the help of the observer estimation values and the attack sleep/active instant trigger information, the tracking control performance is realized for the robot system with the assistance of the neural network approximation technology. Concretely, first, the robot system dynamic model with actuator faults, disturbances, and attacks is established. Then, an event-based proportional–integral observer (PIO) is established. In the observer framework, a state estimator, an actuator fault efficiency estimator, and a disturbance estimator are embedded. Based on the observer outputs, a second-order adaptive sliding mode fault-compensation reliable controller is presented. In this controller framework, the fault compensation, disturbance attenuation, and the attack sleep/active time instant information are contained to guarantee the reliability and performance recoverability of the robot system. Furthermore, a dynamic even condition and an adaptive trigger scheme are constructed in the sensor and actuator channel to achieve the communication-efficient purpose. Finally, two cases of the robot system are performed to verify the system recoverability of the presented approach.

Application of supervisor synthesis methods in designing a discrete-event group control system for mobile robots

This paper discusses the features of methods used to synthesize a discrete-event supervisor for a group control system of mobile robots. We consider the upper levels of a group control system as a discrete-event system. Su-pervisor synthesis methods for such a system and the evaluation of their computational complexity are presented. The paper highlights the limitations of the best method in terms of computational complexity and proposes a modi-fication that allows to remove these limitations without increasing the computational complexity. The application of the modified method is demonstrated on the example of synthesizing a supervisor for the group action of a fire-fighting operation by three robots.

Walk-Burrow-Tug: Legged Anchoring Analysis Using RFT-Based Granular Limit Surfaces

We develop a new resistive force theory based granular limit surface (RFT-GLS) method to predict and guide behaviors of forceful ground robots. As a case study, we harness a small mobile robotic system - MiniRQuad (296g) - to ‘walk-burrow-tug;’ it actively exploits ground anchoring by burrowing its legs to tug loads. RFT-GLS informs the selection of efficient strategies to transport sleds with varying masses. The granular limit surface (GLS), a wrench boundary that separates stationary and kinetic behavior, is computed using 3D resistive force theory (RFT) for a given body and set of motion twists. This limit surface is then used to predict the quasi-static trajectory of the robot when it fails to withstand an external load. We find that the RFT-GLS enables accurate force and motion predictions in laboratory tests. For control applications, a pre-composed state space map of the twist-wrench pairs enables computationally efficient simulations to improve robotic anchoring strategies.

RFID-enabled localization system for mobile robot in the workshop

The development of RFID enabled intelligent localization system in the workshop is of great importance for reducing the operation cost, increasing production efficiency and improving management capabilities. From the aspects of feature extraction of RF fingerprint and localization algorithm, the scheme of mobile target tracking based on the fusion of inertial navigation and fingerprinting is explored. The deep neural network is used to establish the nonlinear relationship between fingerprint and coordinates. After the initial position of the mobile robot is obtained, Kalman filter is used to fuse the data collected by IMU and wheel encoder. Experimental results show that the proposed method is feasible and be able to track the mobile robot accurately.

Model-Predictive Control for Omnidirectional Mobile Robots in Logistic Environments Based on Object Detection Using CNNs.

Object detection is an essential component of autonomous mobile robotic systems, enabling robots to understand and interact with the environment. Object detection and recognition have made significant progress using convolutional neural networks (CNNs). Widely used in autonomous mobile robot applications, CNNs can quickly identify complicated image patterns, such as objects in a logistic environment. Integration of environment perception algorithms and motion control algorithms is a topic subjected to significant research. On the one hand, this paper presents an object detector to better understand the robot environment and the newly acquired dataset. The model was optimized to run on the mobile platform already on the robot. On the other hand, the paper introduces a model-based predictive controller to guide an omnidirectional robot to a particular position in a logistic environment based on an object map obtained from a custom-trained CNN detector and LIDAR data. Object detection contributes to a safe, optimal, and efficient path for the omnidirectional mobile robot. In a practical scenario, we deploy a custom-trained and optimized CNN model to detect specific objects in the warehouse environment. Then we evaluate, through simulation, a predictive control approach based on the detected objects using CNNs. Results are obtained in object detection using a custom-trained CNN with an in-house acquired data set on a mobile platform and in the optimal control for the omnidirectional mobile robot.

A practical type-3 Fuzzy control for mobile robots: predictive and Boltzmann-based learning

This study presents an innovative path-following scheme using a new intelligent type-3 fuzzy system for mobile robots. By designing a non-singleton FS and incorporating error measurement signals, this system is able to handle natural disturbances and dynamics uncertainties. To further enhance accuracy, a Boltzmann machine (BM) models tracking errors and predicts compensators. A parallel supervisor is also included in the central controller to ensure robustness. The BM model is trained using contrastive divergence, while adaptive rules extracted from a stability theorem train the NT3FS. Simulation results using chaotic reference signals show that the proposed scheme is accurate and robust, even in the face of unknown dynamics and disturbances. Moreover, a practical implementation on a real-world robot proves the feasibility of the designed controller. To watch a short video of the scheme in action, visit shorturl.at/imoCH.

Research on Environment Perception System of Quadruped Robots Based on LiDAR and Vision

Due to the high stability and adaptability, quadruped robots are currently highly discussed in the robotics field. To overcome the complicated environment indoor or outdoor, the quadruped robots should be configured with an environment perception system, which mostly contain LiDAR or a vision sensor, and SLAM (Simultaneous Localization and Mapping) is deployed. In this paper, the comparative experimental platforms, including a quadruped robot and a vehicle, with LiDAR and a vision sensor are established firstly. Secondly, a single sensor SLAM, including LiDAR SLAM and Visual SLAM, are investigated separately to highlight their advantages and disadvantages. Then, multi-sensor SLAM based on LiDAR and vision are addressed to improve the environmental perception performance. Thirdly, the improved YOLOv5 (You Only Look Once) by adding ASFF (adaptive spatial feature fusion) is employed to do the image processing of gesture recognition and achieve the human–machine interaction. Finally, the challenge of environment perception system for mobile robot based on comparison between wheeled and legged robots is discussed. This research provides an insight for the environment perception of legged robots.

Vector model-based robot-assisted control system for a wheeled mobile robot

ABSTRACT A robot-assisted control system based on the vector model is proposed for a wheeled mobile robot. According to the closed-loop control structure, the system is constituted of two parts – localization and path planning. The localization algorithm, which is enhanced from Monte Carlo localization, is more effective, stable, and robust than the traditional algorithm because of using many strengthening mechanisms, such as using a vector model, re-initialization, and reverse convergence. The path planning algorithm includes three stages to obtain a path with a motion plan. Firstly, a path from the current position to the goal is planned by an enhanced A* algorithm. Secondly, a smooth mechanism is applied to the path to obtain the continuity of orientation. Finally, a motion design based on the trapezoidal-curve velocity profile is implemented to the smoothed path in both linear and angular velocities to obtain the estimated moving time, position schedule, and velocity schedule. With the assisted control system, the robot knows its current position, the path with motion planning to the destination and its estimated arrival time. If the robot deviates from the move plan, the system will reschedule based on the current state. The experimental results show the great performance of our proposed method.

Environment Perception with Chameleon-Inspired Active Vision Based on Shifty Behavior for WMRs

To improve the environment perception ability of wheeled mobile robots (WMRs), the visual behavior mechanism of the negative-correlation motion of chameleons is introduced into the binocular vision system of WMRs, and a shifty-behavior-based environment perception model with chameleon-inspired active vision for WMRs is established, where vision–motor coordination is achieved. First, a target search sub-model with chameleon-inspired binocular negative-correlation motion is built. The relationship between the rotation angles of two cameras and the neck and the camera’s field of view (FOV), overlapping angle, region of interest, etc., is analyzed to highlight the binocular negative-correlation motion compared with binocular synchronous motion. The search efficiency of the negative-correlation motion is doubled compared with binocular synchronous motion, and the search range is also greatly improved. Second, the FOV model of chameleon-inspired vision perception based on a shifty-behavior mode is set up. According to the different functional requirements of target searching and tracking stages, the shift of the robot visual behavior is analyzed from two aspects, measuring range and accuracy. Finally, a chameleon-inspired active-vision-based environment perception strategy for mobile robots is constructed based on the shifty-behavior mode, and experimental verification is deployed, which achieves the reproduction of the visual behavior of chameleons in the vision system of mobile robots with satisfactory results.