Aerial Manipulation Research Articles

Autonomous control design of an unmanned aerial manipulator for contact inspection

AbstractIn recent years, autonomous control based on contact inspections in unknown environments is a new hot and difficult point in robotics research. This paper presents a new control law for unmanned aerial manipulator (UAM) to perform contact inspection tasks on vertical surfaces. The selected circular image feature decouples the position and attitude of UAM, so an image-based impedance control is proposed to control the position and track the contact force. The developed controller uses geometric methods to control the attitude. In addition, the designed aerial manipulator decouples the roll and pitch of the UAV from the UAV, which improves the system’s stability. Experiments have been carried out to demonstrate the feasibility of this method.

A Trajectory Tracking Approach for Aerial Manipulators Using Nonsingular Global Fast Terminal Sliding Mode and an RBF Neural Network

An unmanned aerial manipulator (UAM) is a novel flying robot consisting of an unmanned aerial vehicle (UAV) and a multi-degree-of-freedom (DoF) robotic arm. It can actively interact with the environment to conduct dangerous or inaccessible tasks for humans. Owing to the underactuated characteristics of UAVs and the coupling generated by the rigid connection with the manipulator, robustness and a high-precision controller are critical for UAMs. In this paper, we propose a nonsingular global fast terminal sliding mode (NGFTSM) controller for UAMs to track the expected trajectory under the influence of disturbances based on a reasonably simplified UAM system dynamics model. To achieve active anti-disturbance and high tracking accuracy in a UAM system, we incorporate an RBF neural network into the controller to estimate lumped disturbances, including internal coupling and external disturbances. The controller and neural network are derived according to Lyapunov theory to ensure the system’s stability. In addition, we propose a set of illustrative metrics to evaluate the performance of the designed controller and compare it with other controllers by simulations. The results show that the proposed controller can effectively enhance the robustness and accuracy of a UAM system with satisfactory convergence. The experimental results also verify the effectiveness of the proposed controller.

Design and Implementation of a Fully-Actuated Integrated Aerial Platform Based on Geometric Model Predictive Control

Unlike individual unmanned aerial vehicles (UAVs), integrated aerial platforms (IAPs) containing multiple UAVs do not suffer from underactuation and can move omnidirectionally in six dimensions, providing a basis for constructing aerial manipulation platforms. Compared to single UAVs, multi-UAV IAPs are also advantageous in terms of payload and fault-tolerance capacity, making them promising candidates as platforms with integrated-response, observation, and strike capabilities. Herein, an IAP structure design containing three sub-UAVs connected in a star-like configuration is presented. This form of integration enables the IAP, as a whole, to simultaneously adjust its position and attitude in six dimensions. The dynamics of the overall system of the IAP are modeled. On this basis, an overall system controller is designed. To simplify control, based on stability of cascaded system, the rotational motion of the sub-UAVs is treated as a inner-loop subsystem, whereas the overall motion of the IAP is seen as a outer-loop subsystem. Because the configuration space of the sub-UAVs is non-Euclidean, a controller is designed for the outer-loop subsystem based on model predictive control on the manifold. Subsequently, the stability of the closed-loop system is demonstrated. Fieldbus technology is employed to design a real-time, scalable communication architecture for multiple sub-UAVs, followed by the development of a principle prototype of the multi-UAV IAP that consists of hardware and software systems. The effectiveness of the IAP design and control method is validated through simulation and real-world prototype-based tests. In the simulation and real-world tests, the proposed methodology can make the IAP system converge to the desired configuration at the presence of large initial configuration error. The same test scenario cannot be finished by a baseline PID controller. The advantage of the proposed control scheme in dealing with state and input constraints is shown via such tests.

One-shot learning for autonomous aerial manipulation.

This paper is concerned with learning transferable contact models for aerial manipulation tasks. We investigate a contact-based approach for enabling unmanned aerial vehicles with cable-suspended passive grippers to compute the attach points on novel payloads for aerial transportation. This is the first time that the problem of autonomously generating contact points for such tasks has been investigated. Our approach builds on the underpinning idea that we can learn a probability density of contacts over objects’ surfaces from a single demonstration. We enhance this formulation for encoding aerial transportation tasks while maintaining the one-shot learning paradigm without handcrafting task-dependent features or employing ad-hoc heuristics; the only prior is extrapolated directly from a single demonstration. Our models only rely on the geometrical properties of the payloads computed from a point cloud, and they are robust to partial views. The effectiveness of our approach is evaluated in simulation, in which one or three quadcopters are requested to transport previously unseen payloads along a desired trajectory. The contact points and the quadcopters configurations are computed on-the-fly for each test by our approach and compared with a baseline method, a modified grasp learning algorithm from the literature. Empirical experiments show that the contacts generated by our approach yield a better controllability of the payload for a transportation task. We conclude this paper with a discussion on the strengths and limitations of the presented idea, and our suggested future research directions.

A Planning-and-Control Framework for Aerial Manipulation of Articulated Objects

While the variety of applications for Aerial Manipulators (AMs) has increased over the last years, they are mostly limited to push-and-slide tasks. More complex manipulations of dynamic environments are poorly addressed and still require handcrafted designs of hardware, control, and trajectory planning. In this letter we focus on the active manipulation of articulated objects with AMs. We present a novel planning and control approach that allows the AM to execute complex interaction maneuvers with as little as possible priors given by the operator. Our framework combines sampling-based predictive control to generate pose trajectories with an impedance controller for compliant behaviours, applied to a fully-actuated flying platform. The framework leverages a physics engine to simulate the dynamics of the platform and the environment in order to find optimal motions to execute manipulation tasks. Experiments on two selected examples of pulling open a door and of turning a valve show the feasibility of the proposed approach.

Public Opinion About the Benefit, Risk, and Acceptance of Aerial Manipulation Systems

Aerial manipulation systems are an emerging subclass of unmanned aerial vehicles (UAVs or “drones”) that use a mobile arm to manipulate their environment. Public opinion is an important consideration for drones, but past public opinion polls have focused on drones without attached arms and have relied on text-based surveys. Study 1 ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> = 190) assessed participants’ perceived benefit, risk, and acceptance of aerial manipulation systems across five applications, using an animation-based online public opinion survey. Study 2 ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> = 194) assessed the influence of stimulus sampling on public opinion of aerial manipulation systems by replicating Study 1 using an alternative set of animations. Study 3 ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> = 396) assessed the influence of using animations versus a text control on perceptions of the poll and aerial manipulation systems, as well as the influence of survey platform (YouGov Direct versus Mechanical Turk). Results show that delivery applications are perceived as more beneficial than several other applications; that people’s opinions and imagining of drones were different if they watched animations versus read text; and that stimulus and population sampling influence the results of public opinion polls about drones.

An Aerial Parallel Manipulator With Shared Compliance

Accessing and interacting with difficult to reach surfaces at various orientations is of interest within a variety of industrial contexts. Thus far, the predominant robotic solution to such a problem has been to leverage the maneuverability of a fully actuated, omnidirectional aerial manipulator. Such an approach, however, requires a specialised system with a high relative degree of complexity, thus reducing platform endurance and real-world applicability. The work here presents a new aerial system composed of a parallel manipulator and conventional, underactuated multirotor flying base to demonstrate interaction with vertical and non-vertical surfaces. Our solution enables compliance to external disturbance on both subsystems, the manipulator and flying base, independently with the goal of improved overall system performance when interacting with surfaces. To achieve this behaviour, an admittance control strategy is implemented on various layers of the flying base's dynamics together with torque limits imposed on the manipulator actuators. Experimental evaluations show that the proposed system is compliant to external perturbations while allowing for differing interaction behaviours as compliance parameters of each subsystem are altered. Such capabilities enable an adjustable form of dexterity in completing sensor installation, inspection and aerial physical interaction tasks.

Dry Coupled Ultrasonic Non-Destructive Evaluation Using an Over-Actuated Unmanned Aerial Vehicle

Unmanned aerial vehicles (UAVs) are seeing increasing adoption to automated remote and in situ inspection of industrial assets, removing the need for hazardous manned access. Aerial manipulator architectures supporting pose-decoupled exertion of force and torque would further enable UAV deployment of contact-based transducers for sub-surface structural health assessment. Herein, for the first time, we introduce an over-actuated multirotor deploying a dry-coupled ultrasonic wheel probe as a novel means of wall thickness mapping. Using bi-axial tilting propellers in a unique tricopter layout, this system performs direct thrust vectoring for efficient omnidirectional flight and application of interaction forces. In laboratory testing, we demonstrate stable and repeatable probe deployment in a variety of representative asset inspection operations. We obtain a mean absolute error (MAE) in measured thickness of under 0.10 mm when measuring an aluminum sample with varying wall thickness. This is maintained over repeated exit and reentry of surface contact and when the sample is mounted vertically or on the underside of a 45° overhang. Furthermore, when rolling the probe dynamically across the sample surface in an area scanning modality, an MAE in wall thickness below 0.28 mm is recorded. Multi-modal operational confidence bounds of the system are thereby quantitatively defined. Note to Practitioners—Motivation for this article stems from the desire to enhance the speed and level of insight into structural health currently offered through remote aerial inspection processes. We approach this by integration of a thrust vectoring multirotor platform with a dry-coupling wheel probe for aerial ultrasonic thickness measurement. This system reliably presses the probe into the target surface and obtains point measurements across various surface orientations without a stabilizing frame. This broadens applicability and permits novel inspections where couplant gel would otherwise contaminate the surface and require manual cleaning. We profile thickness along a scanned linear section, a mode suited to corrosion mapping of large surface areas such as petrochemical storage tanks, pipework, or similar assets. We also make detailed consideration toward measurement accuracy, repeatability, and localization, an aspect commonly overlooked in literature. Future work to characterize variable friction effects currently limiting rolling scan speed and measurement coverage density may be beneficial. Quantitative study of the effectiveness of dry-couplant in the presence of any uncommon surface contaminants specific to a desired use-case is also advised.

Modeling and control for aerial manipulation based on center of inertia on SE(3)

Modeling and control for aerial manipulation based on center of inertia on SE(3)

Versatile articulated aerial robot DRAGON: Aerial manipulation and grasping by vectorable thrust control

Various state-of-the-art works have achieved aerial manipulation and grasping by attaching additional manipulator to aerial robots. However, such a coupled platform has limitations with respect to the interaction force and mobility. In this paper, we present the successful implementation of aerial manipulation and grasping by a novel articulated aerial robot called DRAGON, in which a vectorable rotor unit is embedded in each link. The key to performing stable manipulation and grasping in the air is the usage of rotor vectoring apparatus having two degrees-of-freedom. First, a comprehensive flight control methodology for aerial transformation using the vectorable thrust force is developed with the consideration of the dynamics of vectoring actuators. This proposed control method can suppress the oscillation due to the dynamics of vectoring actuators and also allow the integration with external and internal wrenches for object manipulation and grasping. Second, an online thrust-level planning method for bimanual object grasping using the two ends of this articulated model is presented. The proposed grasping style is unique in that the vectorable thrust force is used as the internal wrench instead of the joint torque. Finally, we show the experimental results of evaluation on the proposed control and planning methods for object manipulation and grasping.

Aerial Continuum Manipulation: A New Platform for Compliant Aerial Manipulation.

Traditional aerial manipulation systems were usually composed of rigid-link manipulators attached to an aerial platform, arising several rigidity-related issues such as difficulties of reach, compliant motion, adaptability to object’s shape and pose uncertainties, and safety of human-manipulator interactions, especially in unstructured and confined environments. To address these issues, partially compliant manipulators, composed of rigid links and compliant/flexible joints, were proposed; however, they still suffer from insufficient dexterity and maneuverability. In this article, a new set of compliant aerial manipulators is suggested. For this purpose, the concept of aerial continuum manipulation system (ACMS) is introduced, several conceptual configurations are proposed, and the functionalities of ACMSs for different applications are discussed. Then, the performances of proposed aerial manipulators are compared with conventional aerial manipulators by implementing available benchmarks in the literature. To enhance the comparison, new features with related benchmarks are presented and used for evaluation purposes. In this study, the advantages of ACMSs over their rigid-link counterparts are illustrated and the potential applications of ACMSs are suggested. The open problems such as those related to dynamic coupling and control of ACMSs are also highlighted.

DDPG-Based Adaptive Robust Tracking Control for Aerial Manipulators With Decoupling Approach.

Aerial manipulators have the potential to perform various tasks with high agility and mobility, but the requirement of system parameters and the complicated dynamic model impede the implementation in practice. To deal with uncertain parameters and complexity of the coupled dynamic model, a decoupling approach is presented in this article by utilizing the adaptive/robust techniques and reinforcement learning approach for the tracking control of quadrotors with position control on the robotic arm. A reinforcement learning approach is proposed to control the robotic arm ensuring minimal effect on the quadrotor dynamics while following the desired trajectory. With the design of nominal inputs, the dynamic uncertainties from the quadrotor, robotic arm, and payload are coped with by utilizing the proposed adaptive algorithms. In addition, the residue of interactive force/torque after the use of DDPG is compensated by robust controllers so that the stability and tracking performance are guaranteed. Numerical examples and experiments are illustrated to demonstrate the efficacy of the presented aerial manipulator control structure and algorithms.

Extended Kalman Filter State Estimation for Aerial Continuum Manipulation Systems

The primary goal of this letter is to address the state estimation problem for dual-arm tendon-driven aerial continuum manipulation systems (ACMSs). While the state estimation problem for conventional rigid aerial manipulation systems (AMSs) has been addressed, parameter estimation remains a significant challenge for the recently introduced ACMS platform. Compared to AMSs that utilize arms’ encoder data, ACMSs with flexible arms are not equipped with such sensors. As a result of the requirement for external sensors such as vision systems, measurement challenges may arise in ACMSs cases. Additionally, the dynamics of ACMSs are substantially more complicated, coupled, and nonlinear, posing additional barriers to tackling the estimating problem at hand. This letter proposes integrating deep neural networks with the extended Kalman filter (EKF) technique to enable real-time applications of the method. Simulation results demonstrate the performance of the suggested learning-based EKF approach.

A Hybrid Modelling Approach for Aerial Manipulators

Aerial manipulators (AM) exhibit particularly challenging, non-linear dynamics; the UAV and its manipulator form a tightly coupled dynamic system, mutually impacting each other. The mathematical model describing these dynamics forms the core of many solutions in non-linear control and deep reinforcement learning. Traditionally, the formulation of the dynamics involves Euler angle parametrization in the Lagrangian framework or quaternion parametrization in the Newton-Euler framework. The former has the disadvantage of giving birth to singularities and the latter being algorithmically complex. This work presents a hybrid solution, combining the benefits of both, namely a quaternion approach leveraging the Lagrangian framework, connecting the singularity-free parameterization with the algorithmic simplicity of the Lagrangian approach. We do so by offering detailed insights into the kinematic modeling process and the formulation of the dynamics of a general aerial manipulator. The obtained dynamics model is validated experimentally against a real-time physics engine. A practical application of the obtained dynamics model is shown in the context of a computed torque feedback controller (feedback linearization), where we analyze its real-time capability with increasingly complex AM models.

On the optimal energy efficiency of the multi-rotor UAVs of an aerial work platform based on an aerial cable towed robot

Aerial Cable Towed Robots (ACTRs) have been proposed by several researchers for aerial manipulation in recent years. However, the low energy efficiency of the multi-rotor Unmanned Aerial Vehicles (UAVs) of ACTRs prevents ACTR-based aerial work platforms for aerial manipulation from a wider range of applications. To provide a guideline for the design of an ACTR-based aerial work platform with an improved energy efficiency of UAVs, this paper investigates the effects of the position of UAVs, the tension distribution of cables driven by UAVs, and the number of UAVs on the energy efficiency of the UAVs of an ACTR-based aerial work platform. A power consumption model is developed for UAVs, assuming that the UAVs are towing cables to stably generate a certain resultant wrench or force on the moving platform. Based on the developed power consumption model, the position of UAVs and the tension distribution of cables to obtain the minimum total power consumption of UAVs are determined. Then, this paper studies whether an additional UAV will inevitably increase the minimum total power consumption of UAVs. The achievements of the energy efficiency of UAVs in theory are explained in case studies.

Low-Complexity Prescribed Performance Control for Unmanned Aerial Manipulator Robot System Under Model Uncertainty and Unknown Disturbances

This article presents a trajectory tracking control method for the unmanned aerial manipulator robot system (UAMRS) under model uncertainty and unknown disturbances. More specifically, a low-complexity prescribed performance controller is proposed to effectively reduce the design complexity and achieve the prescribed transient and steady-state performance. First, the dynamics model of the UAMRS is analyzed and modeled, where the unmeasured internal interaction generated by the coupling effect and the random environmental disturbances are considered simultaneously. Then, utilizing the property of prescribed performance, the UAMRS with model uncertainty and external disturbances can guarantee preferable trajectory tracking responses, where the nonlinear disturbance observer is used to estimate and compensate uncertainties and external disturbances. Moreover, the proposed controller defined by simple expressions does not require accurate knowledge of the UAMRS, which is of low complexity and can effectively reduce the amount of calculation. The stability of the proposed controller is analyzed. Finally, the performances of the proposed scheme are demonstrated by the numerical simulation comparisons and real-world experiments, where a quadrotor with a 3-DOF onboard active manipulator is adopted in outdoor experimental validations.

Fully-Actuated Aerial Manipulator for Infrastructure Contact Inspection: Design, Modeling, Localization, and Control

Fully-Actuated Aerial Manipulator for Infrastructure Contact Inspection: Design, Modeling, Localization, and Control

An Immersion and Invariance Controller for Aerial Manipulation

For aerial manipulators, a steady flight must be guaranteed to perform safe interaction with the surrounding environment. This paper focuses on the development of a position control algorithm for an aerial manipulator system (AMS). The position control algorithm is based on the Immersion and Invariance (I&amp;I) theory. The proposed controller maintains the position of the aerial manipulator at the desired point under external and internal disturbances. The control architecture uses a Visual-SLAM technique using on-board sensors for AMS positioning. A series of outdoor experimental tests are performed to demonstrate the effectiveness of the proposed control strategy.

Centimeter-Level Aerial Assembly Achieved With Manipulating Condition Inference and Compliance

This work develops a compliant control strategy for the aerial manipulation of high-precision, which is easy to deploy and transplant between the simulation and the real system. To facilitate this development, a nominal dynamic model of the aerial manipulation system is first formulated by taking the attitude as a low-level control loop. On this basis, all lumped uncertainty is treated as a possible dynamic variation of undetermined term. This undetermined dynamic variation is then iteratively inferred from the deviation between nominal model prediction and state measurement in real-time. A finite impulse response filter is also proposed to attenuate measurement noises while avoiding undesired phase lags. With inference results, a compensation controller is designed accordingly for the airframe to establish a stable base-platform. Subsequently, a kinematic compensation, handling possibly residual fluctuation of the airframe, is further implemented for the manipulator to enhance the performance. Simulations and experiments are carried out at last, and the controller is illustrated to be quickly deployable with a standalone onboard computer mounted on a common airframe. Results demonstrate that aerial manipulator can reliably fetch and assemble a concerned part from different orientations in holes of 0.015 m radius, i.e., the precision of the end-effector is in centimeter-level, which outperforms the state-of-the-art.

Adaptive NN motion control and predictive coordinate planning for aerial manipulators

The aerial manipulator has attracted great interest since it can interact with the environment through its end-effector. This paper divides the end-effector position tracking control task for the aerial manipulator into two subtasks. The first is motion control, and the second is coordinate planning so that the end-effector of the aerial manipulator can precisely track the given trajectory. The motion control of the aerial manipulator is solved by a partially coupled approach and divided into a multirotor controller and a manipulator controller. The motion controller is designed by the adaptive neural network control. By resorting to radial basis function neural networks with adaptive weight estimation laws, the dynamic couplings between the multirotor and the manipulator can be compensated in real time. With the support of Lyapunov stability criteria, it is proved that the desired trajectories can be boundedly tracked by the multirotor under the proposed controller. Then, a novel predictive coordinate planning method is proposed based on the move blocking method. This method ensures that the solution satisfies physical limits of the aerial manipulator and can be executed in real time. Simulations and comparisons demonstrate effectiveness and performance of the proposed methods.