Motion Control Tasks Research Articles

Motion control of autonomous underwater vehicle based on physics-informed offline reinforcement learning

Online reinforcement learning (RL) methods for autonomous underwater vehicles (AUV) are time-consuming and unsafe due to the need for real-world interaction. Offline RL methods can improve efficiency and safety by training with dynamic models, but an accurate model for AUV is difficult to obtain due to its highly nonlinear dynamics. These limit the application of RL methods in AUV control. To solve this issue, we propose physics-informed model-based conservative offline policy optimization (PICOPO). It offers the advantages of small dataset, strong generalizability and high safety by combining the physics-informed dynamic modelling method and the offline RL technique. First, the PICOPO constructs a physics-informed model based on a small offline dataset to serve as the digital twins (DT) of the actual AUV. This DT can forecast the long-term motion states of AUV with high-precision. The RL-based controller is then trained offline within this DT, eliminating the need for real-world interaction and allowing direct deployment to the AUV without fine-tuning. In this paper, simulations and field tests are carried out to evaluate the proposed method. Our results demonstrate that PICOPO achieves accurate motion control with just 2000 samples and enables zero-shot sim-to-real transfer, showcasing strong generalizability across various motion control tasks.

Energy-Selected Iterative Learning Control: A Novel Perspective to Analyze Precision Motion Control Tasks

Energy-Selected Iterative Learning Control: A Novel Perspective to Analyze Precision Motion Control Tasks

Augmented Finite Control Set MPC Design Technique for Wide Speed Range Control of Permanent Magnet Synchronous Motor Drives

The effective motion control task is highly influencing the final drive performance in wide-range motor drives applications such as electric vehicles (EVs), washing machines, compressors, elevators and lathe machines. Recently, permanent magnet synchronous motor (PMSM) drive has been widely employed in these applications to achieve desired performances such as high efficiency, high torque requirements with minimum size “occupancy”. The PMSM drive features combining with these modern drive applications are highly nonlinear and the desired operation requirements demand high-performance control schemes with fast system response and robustness features. Linear control schemes are usually implemented to drive the PMSM that entail trade-offs between drive performance and simplicity in control implementation. Therefore, this paper presents the nonlinear control technique based on augmented finite control set (AFCS) model predictive control (MPC) scheme (AFCS-MPC). The proposed AFCS-MPC scheme enhances the standard MPC design by augmenting the nonlinear characteristics of the PMSM drive to achieve wide speed range drive performance under both steady-state and dynamic operating conditions. Comparative performance studies have been conducted and presented to confirm the efficacy of the proposed scheme.

Methods of automated detection of travel points when training a collaborative robot

An algorithm has been developed and implemented in this paper, which allows automating the process of forming and controlling scenarios for the movement of a collaborative robot (“Cobot”) through a database of points without specific interfaces, services, and software tools characteristic of each Cobot model. The unification of the developed single graphical interface is achieved by automating the work with Cobot controllers through specialised structured file formats and Robot Operation System (ROS), and by automatically detecting marks as movement points in the image received from the stereo camera using neural network-based models and image processing techniques. Research based on a series of experiments ensured the selection of the most effective image processing method and neural network model in terms of accuracy, speed, resource consumption. The approach formalised in the paper and the graphical interface allowed to implement a classical set of industrial tasks of Cobot motion control.

Reinforcement Learning in the Task of Spherical Robot Motion Control

The output control algorithm is a pair of trained neural networks — actor and critic, where the actor-network is used to obtain the control torques applied to the robot wheel and the critic-network is only involved in the learning process. The results of the training are shown below, namely how the robot performs the motion along ten arbitrary trajectories, where the main quality functional is the average error magnitude over the trajectory length scale. The algorithm is implemented using the PyTorch machine learning library.

A cooperative car-following control model combining deep optical flow estimation and deep reinforcement learning for hybrid electric vehicles

Deep reinforcement learning (DRL) based car-following control (CFC) models are widely applied in the longitudinal motion control tasks of automated vehicles by self-learning for the optimal control policy. However, DRL algorithms easily produce unsafe commands and have low robustness, especially in complex car-following scenarios. To improve the DRL-based CFC model, this paper combines the deep deterministic policy gradient (DDPG) based CFC model with the deep optical flow estimation (DOFE) based CFC model that can overcome the shortcomings of DDPG-based one which is denoted as cooperative car-following model (DDPGoF). The DDPG-based CFC model utilizes prioritized experience replay which can intrinsically accelerate the learning speed. Meanwhile, the proposed DOFE-based CFC model employs the recurrent all-pairs field transforms algorithm (RAFT) and EfficientNet to perceive the motion variation of the surrounding vehicles, motorcycles, etc. The real vehicle driving data sets are applied to calibrate and validate the proposed DDPGoF-based CFC model while several assessment criteria are established to evaluate its overall performance. As a result, the DDPGoF-based CFC model is superior to DDPG-based one in avoiding crashes, improving car-following stability, riding comfort, and fuel economy of HEV.

A general motion control framework for an autonomous underwater vehicle through deep reinforcement learning and disturbance observers

This paper investigates the application of deep reinforcement learning (RL) in the motion control for an autonomous underwater vehicle (AUV), and proposes a novel general motion control framework which separates training and deployment. Firstly, the state space, action space, and reward function are customized under the condition of ensuring generality for various motion control tasks. Next, in order to efficiently learn the optimal motion control policy in the case that the AUV model is imprecise and there are unknown external disturbances, a virtual AUV model composed of the known and determined items of an actual AUV is put forward and a simulation training method is developed on this basis. Then, in the given deployment method, three independent extended state observers (ESOs) are designed to deal with the unknown items in different directions, and the final controller is obtained by compensating the estimated value of ESOs into the output of the optimal motion control policy obtained through simulation training. Finally, soft actor-critic is chosen as deep RL algorithm of the framework, and the generality and effectiveness of the proposed method are verified in four different AUV motion control tasks.

Accuracy of Relative Navigation Using Time-of-Flight Cameras

Three-dimensional time-of-flight cameras are being used increasingly widely, and they have found applications in robotics for information support of motion control tasks. They are also considered promising for spacecraft control at short distances from other spacecraft. One such application is relative navigation in the problem of docking with a non-cooperative spacecraft. In this case, the values of the relative coordinates’ measurement errors that the time-of-flight camera can provide are very important. This paper proposes a method for estimating the magnitude of errors in measuring relative coordinates, which depend on the shape of the passive spacecraft, and the distance, observation angle, and parameters of the time-of-flight camera. The lower Cramer–Rao bound is used as an estimate.

From Simulation to Reality: A Learning Framework for Fish-Like Robots to Perform Control Tasks

The fish-like robot is one of the typical underwater robots, which has the advantage of high maneuverability with low noise due to its bioinspired structure and biomimetic locomotion. However, it is challenging to efficiently design motion controllers for such robots to achieve satisfactory performance on specific control tasks in the real underwater environment, since the complex fluid-structure interaction exists during their swimming and exact dynamic models are absent. In this article, we propose a learning framework, incorporating a simulation system and a training methodology, to autonomously and fast train in simulation to create control policies that are capable of directly applying to a type of physical fish-like robots to perform motion control tasks. First, we construct a simulation system combining a data-driven environment and a computational fluid dynamics (CFD)-based environment, thus well balancing the simulation accuracy and the calculation speed. Second, we design a training methodology to train deep reinforcement learning (DRL)-based policies for the robot in our constructed simulation system to perform a specific control task. Then, we use two typical motion control tasks to verify our proposed framework. One is the path-following control task, which is a one-objective problem with dense rewards, while the other is the pose control task which is a two-objective problem with sparse rewards. For each task, the DRL-based control policy trained by our learning framework is directly deployed on the physical fish-like robot to perform the task in the real world. Experimental results show that the policies trained in simulation still work well in the real world, and perform even better in terms of control accuracy and stability compared with the traditional control methods, thus demonstrating the effectiveness of our learning framework.

A general motion controller based on deep reinforcement learning for an autonomous underwater vehicle with unknown disturbances

This paper studies the application of deep Reinforcement Learning (RL) in the motion control of an underactuated autonomous underwater vehicle (AUV) with unknown disturbances. Firstly, a general state space, action space and reward function are designed for motion control problems rather than each specific motion control task, which ensures the generality of our method. Furthermore, a virtual AUV model with partial random disturbances is established, and on this basis, a simulation training method is developed to solve the problems of extremely high risk and extremely low efficiency caused by training in actual experiments. Then, in order to directly deploy the optimal control policy obtained through simulation training to an actual AUV, we employ Extended State Observers (ESOs) to estimate the unknown disturbances in five degrees of freedom, and give a deployment method using the estimated values as the disturbance state vector and compensation vector. Combining the above training method and deployment method, a novel general motion controller is proposed. Finally, four different AUV motion control simulations are carried out, and the results confirm the generality and effectiveness of our proposed controller.

A general motion control architecture for an autonomous underwater vehicle with actuator faults and unknown disturbances through deep reinforcement learning

This paper studies the application of deep reinforcement learning (RL) in the motion control tasks of an underactuated AUV with actuator faults and unknown disturbances. Firstly, a general state space and action space that can be applied to a variety of motion control tasks are customized. Furthermore, a universal reward function is designed to minimize the energy consumption of the AUV under the condition that the learned optimal control policy can achieve fast and high-precision control. Then, a simulation method with random reference values and simulated random disturbances is given to train the deep RL algorithm. In order to deploy the optimal control policy obtained from the simulation training directly to an actual AUV, we design five extended state observers (ESOs) to estimate the unknown disturbances in different directions, and take the estimated values as the disturbance states required to obtain the optimal control action. Combined with these ideas, a general AUV motion control architecture with simulation training and deployment process is developed. Finally, four different AUV motion control experiments are carried out, and the results confirm the generality and effectiveness of our architecture.

Fully body visual self-modeling of robot morphologies.

Internal computational models of physical bodies are fundamental to the ability of robots and animals alike to plan and control their actions. These "self-models" allow robots to consider outcomes of multiple possible future actions without trying them out in physical reality. Recent progress in fully data-driven self-modeling has enabled machines to learn their own forward kinematics directly from task-agnostic interaction data. However, forward kinematic models can only predict limited aspects of the morphology, such as the position of end effectors or velocity of joints and masses. A key challenge is to model the entire morphology and kinematics without prior knowledge of what aspects of the morphology will be relevant to future tasks. Here, we propose that instead of directly modeling forward kinematics, a more useful form of self-modeling is one that could answer space occupancy queries, conditioned on the robot's state. Such query-driven self-models are continuous in the spatial domain, memory efficient, fully differentiable, and kinematic aware and can be used across a broader range of tasks. In physical experiments, we demonstrate how a visual self-model is accurate to about 1% of the workspace, enabling the robot to perform various motion planning and control tasks. Visual self-modeling can also allow the robot to detect, localize, and recover from real-world damage, leading to improved machine resiliency.

An L₁-Norm Based Optimization Method for Sparse Redundancy Resolution of Robotic Manipulators

For targeted motion control tasks of manipulators, it is frequently necessary to make use of full levels of joint actuation to guarantee successful motion planning and path tracking. Such way of motion planning and control may keep the joint actuation in a non-sparse manner during motion control process. In order to improve sparsity of joint actuation for manipulator systems, a novel motion planning scheme which can optimally and sparsely adopt joint actuation is proposed in this brief. The proposed motion planning strategy is formulated as a constrained <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> norm optimization problem, and an equivalent enhanced optimization solution dealing with bounded joint velocity is proposed as well. A new primal dual neural network with a new solution set division is further proposed and applied to solve such bounded optimization which can sparsely adopt joint actuation for motion control. Simulation and experiment results demonstrate the efficiency, accuracy and superiority of the proposed method for optimally and sparsely adopting joint actuation. The average sparsity (i.e., - <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\|\dot \theta \|_{p}$ </tex-math></inline-formula> where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\theta $ </tex-math></inline-formula> denotes the joint angle) of the joint motion of the manipulator can be increased by 39.22% and 51.30% for path tracking tasks in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$X - Y$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$X - Z$ </tex-math></inline-formula> planes respectively, indicating that the sparsity of joint actuation can be enhanced.

A Semi-Supervised Progressive Learning Algorithm for Brain-Computer Interface.

Brain-computer interface (BCI) usually suffers from the problem of low recognition accuracy and large calibration time, especially when identifying motor imagery tasks for subjects with indistinct features and classifying fine grained motion control tasks by electroencephalogram (EEG)-electromyogram (EMG) fusion analysis. To fill the research gap, this paper presents an end-to-end semi-supervised learning framework for EEG classification and EEG-EMG fusion analysis. Benefiting from the proposed metric learning based label estimation strategy, sampling criterion and progressive learning scheme, the proposed framework efficiently extracts distinctive feature embedding from the unlabeled EEG samples and achieves a 5.40% improvement on BCI Competition IV Dataset IIa with 80% unlabeled samples and an average 3.35% improvement on two public BCI datasets. By employing synchronous EMG features as pseudo labels for the unlabeled EEG samples, the proposed framework further extracts deep level features of the synergistic complementarity between the EEG signals and EMG features based on the deep encoders, which improves the performance of hybrid BCI (with a 5.53% improvement for the Upper Limb Motion Dataset and an average 4.34% improvement on two hybrid datasets). Moreover, the ablation experiments show that the proposed framework can substantially improve the performance of the deep encoders (with an average 5.53% improvement). The proposed framework not only largely improves the performance of deep networks in the BCI system, but also significantly reduces the calibration time for EEG-EMG fusion analysis, which shows great potential for building an efficient and high-performance hybrid BCI for the motor rehabilitation process.

A COMPUTATIONAL FRAMEWORK OF GOAL DIRECTED VOLUNTARY MOTION GENERATION AND CONTROL LOOP IN HUMANOID ROBOTS

In this paper, it is aimed to construct a computational framework related to bio-inspired motion generation and control systems for humanoid robots. To acquire natural motion patterns in humanoid robots, behaviors observed from biological motor systems in humans and other mammals should be analyzed in detail. Computational mechanisms are mainly placed on the bio-physical plausible neural structures embodied in different dynamics. The main components of the system are composed of the limbic system, neocortex, cerebellum, brainstem, and spinal cord modules. Internal dynamics of these modules include a nonlinear estimator (e.g. chaotic attractor), memory formation, learning (neural plasticity) procedure. While the proposed novel neuro-cognitive framework is performing goal-directed voluntary motion generation and control tasks, also it estimates the amount of motion errors and computes motion correction signals. By this study, some motion-based central nervous system lesions (e.g. epilepsy, Parkinson, etc.) can be computationally modeled so that impairments of motor control commands are detected. Thus motion disorders can be reconstructed not only in humanoid robots but also in humans via some locomotion equipment.

Soccer‐Assisted Training Robot Based on Image Recognition Omnidirectional Movement

With the continuous emergence and innovation of computer technology, mobile robots are a relatively hot topic in the field of artificial intelligence. It is an important research area of more and more scholars. The core of mobile robots is to be able to realize real‐time perception of the surrounding environment and self‐positioning and to conduct self‐navigation through this information. It is the key to the robot’s autonomous movement and has strategic research significance. Among them, the goal recognition ability of the soccer robot vision system is the basis of robot path planning, motion control, and collaborative task completion. The main recognition task in the vision system is the omnidirectional vision system. Therefore, how to improve the accuracy of target recognition and the light adaptive ability of the robot omnidirectional vision system is the key issue of this paper. Completed the system construction and program debugging of the omnidirectional mobile robot platform, and tested its omnidirectional mobile function, positioning and map construction capabilities in the corridor and indoor environment, global navigation function in the indoor environment, and local obstacle avoidance function. How to use the local visual information of the robot more perfectly to obtain more available information, so that the “eyes” of the robot can be greatly improved by relying on image recognition technology, so that the robot can obtain more accurate environmental information by itself has always been domestic and foreign one of the goals of the joint efforts of scholars. Research shows that the standard error of the experimental group’s shooting and dribbling test scores before and the experimental group’s shooting and dribbling test results after the standard error level is 0.004, which is less than 0.05, which proves the use of soccer‐assisted robot‐assisted training. On the one hand, we tested the positioning and navigation functions of the omnidirectional mobile robot, and on the other hand, we verified the feasibility of positioning and navigation algorithms and multisensor fusion algorithms.

Simulation of Quadcopter's Complex Spatial Motion in Virtual Environment Systems

The paper considers the task of simulation quadcopter motion control in vir-tual environment systems. The proposed solution of this task is based on the application of a mathematical model of quadcopter dynamics, in which quaternion and axis-angle representation of the attitude are used. In this work, a feedback linearization method for the equations of quadcopter and its elec-tric motors dynamics is proposed to control the quadcopter. With the application of this approach, quadcopter motion is achieved by the obtained expressions for linear acceleration and the desired quaternion. The proposed methods and approaches for quadcopter simulation were implemented in the software package of virtual environment and tested using examples of the motion of a quadcopter model along a given path.

A Sparse Optimization-Based Control Method for Manipulator With Simultaneous Potential Energy Minimization

Manipulators may often endure large-scale potential energy variations during kinematic control, producing unsafe oscillations of posture-hold effects. In this brief, for the first time, a sparse-optimization-based control method is proposed to simultaneously guarantee task accuracy and reduce potential energy variations for manipulators. The proposed control approach is formulated as an $L_{p}$ -norm sparse optimization paradigm with the variation of the potential energy as the constraint. Simulation results on the $n$ -link planar manipulator and experiment results on a 6-DoF manipulator system verify the proposed control method for accurate motion control tasks with potential energy minimization, with the average potential energy variations of the manipulators largely diminished with the sparse kinematic resolutions.

Dynamics Parametrization and Calibration of Flexible-Joint Collaborative Industrial Robot Manipulators

Many collaborative robots use strain-wave-type transmissions due to their desirable characteristics of high torque capacity and low weight. However, their inherent complex and nonlinear behavior introduces significant errors and uncertainties in the robot dynamics calibration, resulting in decreased performance for motion and force control tasks and lead-through programming applications. This paper presents a new method for calibrating the dynamic model of collaborative robots. The method combines the known inverse dynamics identification model with the weighted least squares (IDIM-WLS) method for rigid robot dynamics with complex nonlinear expressions for the rotor-side dynamics to obtain increased calibration accuracy by reducing the modeling errors. The method relies on two angular position measurements per robot joint, one at each side of the strain-wave transmission, to effectively compensate the rotor inertial torques and nonlinear dynamic friction that were identified in our previous works. The calibrated dynamic model is cross-validated and its accuracy is compared to a model with parameters obtained from a CAD model. Relative improvements are in the range of 16.5% to 28.5% depending on the trajectory.

NAVIGATION AND MOTION CONTROL SYSTEMS OF THE AUTONOMOUS UNDERWATER VEHICLE

Autonomous underwater vehicles (AUVs) are widely used and have proven their effectiveness in tasks such as transportation safety, area monitoring and seafloor mapping. When developing AUV’s navigation and control systems, the engineers have to ensure the required levels of accuracy and reliability for solving navigation and motion control tasks in autonomous underwater operation under restrictions on the overall dimensions and power consumption of the AUV. The main purpose of this paper is to present preliminary results of AUV navigation and motion control systems development. The AUV’s navigation system is built around strapdown inertial navigation system (SINS) designed specifically for this AUV. When surfaced, position and angular SINS correction is performed using data from dual-antenna GNSS receiver and doppler velocity log (DVL). When underwater, SINS position and velocity correction is performed using acoustic navigation system (ANS) and DVL data. AUV’s control system provides manual and automatic control. Manual control is carried out in real-time by operator via fiber-optic cable using a joystick. Automatic control allows AUV to move independently along a specified trajectory at a given depth and speed. The AUV also has a collision avoidance system that utilizes readings from a forward-facing acoustic rangefinder to estimate time before impact based on AUV’s analytic model. If possible collision is detected, information is transmitted to the control system so that a further appropriate action can be taken. Computer simulation utilizing the analytic AUV model was used in order to check the performance characteristics of the designed control and navigation algorithms. After confirming the operability of the developed algorithms, preliminary tests of the AUV were carried out. During the tests, AUV’s on-board equipment and navigation system readings were recorded and compared to the readings of the reference system, which was also installed on the AUV. During the tests, the dynamic characteristics of the AUV were evaluated. AUV’s characteristics obtained during simulation and testing will be used as a reference during future development