Omnidirectional Mobile Manipulator Research Articles

Neural Dynamics for Cooperative Motion Control of Omnidirectional Mobile Manipulators in the Presence of Noises: A Distributed Approach

Neural Dynamics for Cooperative Motion Control of Omnidirectional Mobile Manipulators in the Presence of Noises: A Distributed Approach

Robust Hybrid Visual Servoing of Omnidirectional Mobile Manipulator With Kinematic Uncertainties Using a Single Camera.

A robust hybrid visual servoing (HVS) method for a single camera-mounted omnidirectional mobile manipulator (OMM) with kinematic uncertainties caused by slipping is proposed herein. Most existing studies pertaining to the visual servoing of mobile manipulators do not consider the kinematic uncertainties and the singularity of the manipulator that can occur during actual operations; furthermore, they required external sensors other than a single camera. In this study, the kinematics of an OMM are modeled considering kinematic uncertainties. Accordingly, an integral sliding-mode observer (ISMO) is designed to estimate the kinematic uncertainties. Subsequently, an integral sliding-mode control (ISMC) law is proposed to achieve robust visual servoing using the estimates of the ISMO. Additionally, an ISMO-ISMC-based HVS method is proposed to address the singularity issue of the manipulator; this method guarantees both robustness and finite-time stability in the presence of kinematic uncertainties. Overall, the entire visual servoing task is performed using only a single camera attached to the end effector without any other external sensors, unlike in previous studies. The stability and performance of the proposed method are verified numerically and experimentally in a slippery environment that generates kinematic uncertainties.

Bimanual Telemanipulation Framework Utilising Multiple Optically Localised Cooperative Mobile Manipulators

Bimanual manipulation is valuable for its potential to provide robots in the field with increased capabilities when interacting with environments, as well as broadening the number of possible manipulation actions available. However, for a robot to perform bimanual manipulation, the system must have a capable control framework to localise and generate trajectories and commands for each sub-system to allow for successful cooperative manipulation as well as sufficient control over each individual sub-system. The proposed method suggests using multiple mobile manipulator platforms coupled through the use of an optical tracking localisation method to act as a single bimanual manipulation system. The framework’s performance relies on the accuracy of the localisation. As commands are primarily high-level, it is possible to use any number and combination of mobile manipulators and fixed manipulators within this framework. We demonstrate the functionality of this system through tests in a Pybullet simulation environment using two different omnidirectional mobile manipulators, as well a real-life experiment using two quadrupedal manipulators.

Nonconvex Activation Noise-Suppressing Neural Network for Time-Varying Quadratic Programming: Application to Omnidirectional Mobile Manipulator

This paper proposes an improved general zeroing neural network (ZNN) model to suppress noise and to enhance the real-time performance of solving time-varying quadratic programming (TVQP) problems. The proposed model allows nonconvex activation functions (AFs) and has noise suppression characteristics, i.e., the nonconvex constrained noise suppressed ZNN (NCNSZNN) model. Theoretical analyses show that the developed NCNSZNN model converges globally to an accurate solution to the TVQP problem and is robust in the case of measurement noise (MN). Illustrative examples and comparisons are supplied to verify the validity and superiority of the proposed model for online solving TVQP constrained by equalities and inequalities (EAI) with MN.

Optimal Trajectory Planning of an Omni-directional Mobile Manipulator for Obstacle Avoidance

Optimal Trajectory Planning of an Omni-directional Mobile Manipulator for Obstacle Avoidance

Nonconvex Noise-Tolerant Neural Model for Repetitive Motion of Omnidirectional Mobile Manipulators

Nonconvex Noise-Tolerant Neural Model for Repetitive Motion of Omnidirectional Mobile Manipulators

Practical issues in data-driven model-free adaptive control for an omnidirectional mobile manipulator

This article focuses on addressing three practical issues encountered when applying a data-driven model-free adaptive control (MFAC) approach to mobile robots. The first practical issue lies in a common assumption in MFAC schemes that the sign of all elements in pseudo-partial derivative (PPD) should be constant, while it cannot be satisfied if omnidirectional mobile manipulators (OMMs) move with platform rotation. To solve this problem, a new coordinate frame is introduced, which is crucial for applying MFAC to any mobile robots with rotation. The second one is that the initial value setting method for estimation of PPD is unclear. Improper settings may easily lead to control system instability. An initial value setting method for estimation of PPD is proposed with explicit physical interpretation. Lastly, applying the typical MFAC scheme directly to OMM fails to converge well to the desired trajectory. To tackle this, a new data-driven MFAC controller is proposed by incorporating a sliding mode control. Finally, experimental tests on an OMM are carried out to verify the effectiveness of the proposed control scheme. To the best of our knowledge, this is the first MFAC scheme that has been experimentally verified on a prototype mobile robot with rotation.

Implementation of a robust data-driven control approach for an ommi-directional mobile manipulator based on koopman operator

The dynamic modeling and control of omni-directional mobile manipulators (OMM) are challenging since they are highly nonlinear, strongly coupled, and multi-input multi-output uncertainty systems. Koopman operator theory can provide an explicit linear dynamic model for OMM, only based on input-output data. However, the derived dynamic model usually has modeling errors and cannot capture external unknown disturbances. In this paper, a robust data-driven control scheme is designed and implemented for an OMM, based on Koopman operator and a disturbance observer. Firstly, a data-driven Koopman model of OMM is constructed, solely using the collected input-output data. Then a disturbance observer is designed to online estimate the inherent modeling error of the Koopman model as well as external disturbances. The controller is designed by combining linear MPC with feedback compensation of the estimated disturbances. Finally, both simulations and experimental tests are conducted to verify the effectiveness and robustness of the proposed control scheme.

Koopman operator based model predictive control for trajectory tracking of an omnidirectional mobile manipulator

Omnidirectional mobile manipulators (OMMs) have been widely used due to their high mobility and operating flexibility. However, since OMMs are complex nonlinear systems with uncertainties, the dynamic modeling and control are always challenging problems. Koopman operator theory provides a data-driven modeling method to construct explicit linear dynamic models for the original nonlinear systems, using only input-output data. It then allows to design control system based on well-established model-based linear control methods. This paper designs a Koopman operator based model predictive control (MPC) scheme for trajectory tracking control of an OMM. Firstly, using Koopman operator and extended dynamic mode decomposition method, an approximate high-dimensional linear dynamic explicit expression for the OMM system is obtained. Then MPC is employed to achieve tracking control based on the derived linear Koopman model. Finally, to show modeling accuracy for the OMM, the Koopman model is evaluated via both simulation and experimental tests. The control performances of the Koopman operator based MPC design are also verified in the simulation and experimental results.

A GNN for repetitive motion generation of four-wheel omnidirectional mobile manipulator with nonconvex bound constraints

This paper proposes a gradient neural network (GNN) to solve the repetitive motion generation scheme of the omnidirectional four-wheel mobile manipulator. The overall kinematics model of the omnidirectional mobile platform and the manipulator fixed on omnidirectional platform are established. First, the analysis of the current repetitive movement generation (RMG) scheme for the kinematic control of the manipulator can find that the position error does not theoretically converge to zero and fluctuates. This paper analyzes the phenomenon from a theoretical viewpoint and reveals that the current RMG scheme has position errors associated with joint errors. Then, to solve the shortcomings of the current solution, an orthogonal projection repetitive motion generation (OPRMG) method is proposed, which theoretically eliminates position errors and decouples joint space and Cartesian space. Using the gradient descent method to establish the corresponding GNN aided with the speed compensation, and provide theoretical analysis to reflect the stability. Moreover, the joint speed limit in the RMG scheme is extended to nonconvex constraints. The advantages of the OPRMG scheme are demonstrated by the simulation results of the omnidirectional mobile manipulator (OMM) synthesized by the current GNNRMG and the proposed GNNOPRMG. In addition, by adjusting the feedback coefficient, the high performance of the OPRMG scheme can be verified by simulation and comparison of the position error (PE) and joint error (JE) of the OMM.

A Holistic Approach to Reactive Mobile Manipulation

We present the design and implementation of a taskable reactive mobile manipulation system. In contrary to related work, we treat the arm and base degrees of freedom as a holistic structure which greatly improves the speed and fluidity of the resulting motion. At the core of this approach is a robust and reactive motion controller which can achieve a desired end-effector pose, while avoiding joint position and velocity limits, and ensuring the mobile manipulator is manoeuvrable throughout the trajectory. This can support sensor-based behaviours such as closed-loop visual grasping. As no planning is involved in our approach, the robot is never stationary thinking about what to do next. We show the versatility of our holistic motion controller by implementing a pick and place system using behaviour trees and demonstrate this task on a 9-degree-of-freedom mobile manipulator. Additionally, we provide an open-source implementation of our motion controller for both non-holonomic and omnidirectional mobile manipulators available at jhavl.github.io/holistic.

전방향 모바일 매니퓰레이터의 강인 영상기반 비주얼 서보잉

This paper proposes robust image based visual servoing of an omnidirectional mobile manipulator using integral sliding mode control to compensate for the uncertainty that may occur while performing the image-based visual servoing. The stability of the overall omnidirecitonal mobile manipulator system using the proposed method is proven by the Lyapunov stability analysis by showing that the error stays on the sliding surface always and asymptotically converges to zero. In order to solve the singularity problem that can occur when controlling the omnidirectional mobile robot and the manipulator at the same time, a switching technique is proposed in such a way that the omnidirectional mobile robot and manipulator operate sequentially to prevent the abnormal operation of the manipulator and solve the singularity problem. The validity of the proposed robust image based visual servoing method is verified through simulation results.

Non-singular terminal sliding mode control of an omnidirectional mobile manipulator based on extended state observer

Non-singular terminal sliding mode control of an omnidirectional mobile manipulator based on extended state observer

An admittance-controlled wheeled mobile manipulator for mobility assistance: Human–robot interaction estimation and redundancy resolution for enhanced force exertion ability

In this paper, a novel robotic assistive system (RAS) for mobility assistance of elderly adults is developed based on admittance control with force exertion ability enhancement (FEAE) of a wheeled mobile manipulator (WMM). The RAS can provide supportive force in the vertical direction and be guided by the user with a limited horizontal plane ability. An admittance controller is adopted to realize compliant behaviour between the end-effector and the user, enabling different admittance performances in different directions without requiring the complex system dynamics. The end-effector force needed by the controller is estimated by employing a nonlinear disturbance observer, avoiding the need for pricey force/torque (F/T) sensor. Meanwhile, with consideration of the limited joint torque output, the FEAE approach is implemented in the null-space of the WMM, which can make the system exert more Cartesian force in a given direction with consideration of Cartesian stiffness requirement using the same joint torque limitation through kinematic reconfiguration. Thus, it improves the capability of the system in realizing the desired admittance requirement. The advantages and effectiveness of the proposed approach are experimentally demonstrated with a 4-wheel omnidirectional mobile manipulator.

Robotized Additive Manufacturing of Funicular Architectural Geometries Based on Building Materials

Additive manufacturing (AM) technology has been identified as one of the major digital innovations that has revolutionized not only the field of the industry but also the construction. From a research side, AM is a multidisciplinary domain, combining between materials science, mechatronics engineering, and architectural design. The AM concept needs to consider the geometry and the shape of printed objects regarding material properties and robot kinematics which are adaptable to the object size. In this article, we present an integrated design and control of an omnidirectional mobile manipulator robot (MMR), capable of extruding concrete and clay materials, to print complex and funicular architectural geometries. The studied robot allows printing building pieces with different shapes and sizes by additive deposit which can later be assembled on-site. The main issue concerns the control of continuous material deposit, with specified accuracy and respecting the desired shape. This problem involves to solve, in real time, the nonlinear kinematic model of a ten-degrees-of-freedom heterogeneous robot. The kinematic model and the control of utilized MMR are discussed, and an optimization method based on quadratic programming is used for handling the robot redundancy. Finally, we present the experimental results to illustrate the efficiency of AM on the developed concept.

Towards a Collaborative Omnidirectional Mobile Robot in a Smart Cyber-Physical Environment

This paper presents current progress in a three-fold research project, focusing on increasing the movement flexibility in smart cyber-physical environments, which are becoming more prevalent with industry 4.0. The objective is to design and develop a new research platform in the form of a highly flexible omnidirectional mobile manipulator with a full kinematics model. The research platform entails an omnidirectional autonomous mobile platform with an integrated collaborative manipulator, making it a single dynamic system. This task is split into three parts: 1) designing and constructing a flexible low cost omnidirectional mobile platform, 2) the addition of a collaborative robot arm to add task execution capabilities, 3) design the 9 DoF model for the system. This paper presents the first step towards the overall goal of collaborative omnidirectional mobile manipulators. The constructed mobile robot was able to generate and navigate an obstacle free path, using sensor data, from an initial pose to a goal pose.

Visually Guided Picking Control of an Omnidirectional Mobile Manipulator Based on End-to-End Multi-Task Imitation Learning

In this paper, a novel deep convolutional neural network (CNN) based high-level multi-task control architecture is proposed to address the visual guide-and-pick control problem of an omnidirectional mobile manipulator platform based on deep learning technology. The proposed mobile manipulator control system only uses a stereo camera as a sensing device to accomplish the visual guide-and-pick control task. After the stereo camera captures the stereo image of the scene, the proposed CNN-based high-level multi-task controller can directly predict the best motion guidance and picking action of the omnidirectional mobile manipulator by using the captured stereo image. In order to collect the training dataset, we manually controlled the mobile manipulator to navigate in an indoor environment for approaching and picking up an object-of-interest (OOI). In the meantime, we recorded all of the captured stereo images and the corresponding control commands of the robot during the manual teaching stage. In the training stage, we employed the end-to-end multi-task imitation learning technique to train the proposed CNN model by learning the desired motion and picking control strategies from prior expert demonstrations for visually guiding the mobile platform and then visually picking up the OOI. Experimental results show that the proposed visually guided picking control system achieves a picking success rate of about 78.2% on average.

Inverse kinematics of mobile manipulators based on differential evolution

The solution of the inverse kinematics of mobile manipulators is a fundamental capability to solve problems such as path planning, visual-guided motion, object grasping, and so on. In this article, we present a metaheuristic approach to solve the inverse kinematic problem of mobile manipulators. In this approach, we represent the robot kinematics using the Denavit–Hartenberg model. The algorithm is able to solve the inverse kinematic problem taking into account the mobile platform. The proposed approach is able to avoid singularities configurations, since it does not require the inversion of a Jacobian matrix. Those are two of the main drawbacks to solve inverse kinematics through traditional approaches. Applicability of the proposed approach is illustrated using simulation results as well as experimental ones using an omnidirectional mobile manipulator.

Development of a Dexterous Dual-Arm Omnidirectional Mobile Manipulator

Abstract This paper presents an experimental mobile manipulator composed of a dual-arm torso with a human-like structure assembled on an omnidirectional platform. The dual-arm system integrates several commercial devices: two arms UR5 with 6 degrees of freedom, each one equipped with an Allegro Hand with four fingers and a total of 16 degrees of freedom, and each fingertip provided with a tactile sensor. The omnidirectional platform has a circular shape and three wheels, with an original special design, whose coordinated movements allow omnidirectional displacements of the platform. It is equipped with laser-range sensors, a radio positioning system and an RGB-D camera to detect potential obstacles and navigate safely. The paper describes the main aspects of the mechanical structure of the mobile manipulator and the software framework developed to control the whole device.

ワイヤを用いた3台の全方向移動マニピュレータにおける協調搬送

ワイヤを用いた3台の全方向移動マニピュレータにおける協調搬送