Motion Control Problem Research Articles

Longitudinal and Lateral Cooperative Control of Preview Intelligent Vehicles with Stabilization

Aiming at the highly nonlinear and coupled longitudinal and transverse motion control problems of intelligent vehicle path tracking control, a new method of preview MPC path tracking control is proposed by combining visual preview model and model predictive control theory. The method first locally linearizes the nonlinear prediction model and transforms the MPC optimization solution problem into a quadratic programming problem. The longitudinal vehicle speed and the preview distance are then treated as known in each control time domain, an exponential model is applied to characterize the longitudinal vehicle speed, and a transverse angular velocity-center of mass lateral deflection (r-β\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$r-\\beta $$\\end{document}) phase plane plot is applied to characterize the vehicle stability. Finally, proportional integral differential stability controller is designed to improve vehicle stability. Simulation and test results show that: Compared with the MPC tracking control method, the preview MPC control method proposed in this paper can optimize the vehicle tracking performance on different adhesion coefficient road surfaces and improve the tracking accuracy, and this enhancement effect is more obvious at high speeds.

Intelligent vehicle lateral control strategy research based on feedforward + predictive LQR algorithm with GA optimisation and PID compensation

Targeting the lateral motion control problem in the intelligent vehicle autopilot structural system, this paper proposes a feedforward + predictive LQR algorithm for lateral motion control based on Genetic Algorithm (GA) parameter optimisation and PID steering angle compensation. Firstly, based on the vehicle dynamics tracking error model, the intelligent vehicle LQR lateral motion controller as well as the feedforward controller are designed, and upon which the predictive controller is added to eliminate the system lag.Subsequently, exploiting the advantage that the PID algorithm is not model-based, a PID steering angle compensation controller that can directly control and correct the lateral error is designed. Second, a LQR controller based on path tracking deviation is designed by using the parameter rectification method of genetic algorithm (GA), which optimizes the control parameters of the lateral motion controller and improves the adaptivity of the control accuracy. Finally, Based on the Carsim-Simulink co-simulation platform, the simulation validation and analysis of double lane change (DLC) test and circular condition test (CCT) are carried out, and the results indicate that compared with the other two LQR controllers, the optimised controllers improved more than 50% in lateral error and heading error control, and the vehicle sideslip angle and vehicle yaw rate are in the range of −0.05° to 0.05° and − 0.15 rad/s to 0.10 rad/s, and it showed improved performance in tracking accuracy and satisfied vehicle stability constrains.

Study of control allocation algorithm for over-actuated underwater robot motion control using azimuth thrusters

The paper considers a problem of a mobile over-actuated underwater reconfigurable robot motion control using azimuth thrusters. To solve the control allocation problem, a configurator has been introduced into the control sys-tem. This makes it possible to synthesize the control law regardless of the number, position and type of the propul-sion-steering complex components. A control allocation algorithm is studied to determine both the control and the orientations of the azimuth thrusters.

RBF Neural Network for Chaotic Motion Control of Collision Vibration System

Aiming at the control problem of chaotic motion of a kind of collision vibration system with gap, a chaotic motion control of collision vibration system based on RBF neural network is proposed. Firstly, the system mechanical model and chaotic motion are introduced, and then a chaotic controller based on RBFNN is designed to control and simulate the chaotic attractor. The model information of the system is not used in the control method. In this paper, the model of the system is used only to generate the input / output data of the system, and it is not used for the design of the controller. The parameters of AHGSA algorithm are set as follows: the population size is 30, the maximum number of iterations is 100,G0, a = 18, and the proportional coefficient P = 0.96. Small disturbances are applied to the controllable parameter ωof the system to suppress the chaotic motion of the system and make the system tend to stable periodic motion. In order to more clearly show the control effect of chaotic motion, the chaotic motion is controlled when the system iterates 400 times. The results show that the chaotic motion can be quickly controlled to periodic 1-1 motion, the phase diagram is a closed curve, and there is a peak in the spectrum diagram. Chaotic motion can be quickly controlled as periodic 2-2 motion, the phase diagram is two closed curves, and two obvious peaks appear in the spectrum diagram. The proposed method can effectively control the chaotic motion of the system, and the expected target can be not only the fixed point of period 1, but also other periodic orbits.

Under-Actuated Motion Control of Haidou-1 ARV Using Data-Driven, Model-Free Adaptive Sliding Mode Control Method.

We propose a data-driven, model-free adaptive sliding mode control (MFASMC) approach to address the Haidou-1 ARV under-actuated motion control problem with uncertainties, including external disturbances and parameter perturbations. Firstly, we analyzed the two main difficulties in the motion control of Haidou-1 ARV. Secondly, in order to address these problems, a MFASMC control method was introduced. It is combined by a model-free adaptive control (MFAC) method and a sliding mode control (SMC) method. The main advantage of the MFAC method is that it relies only on the real-time measurement data of an ARV instead of any mathematical modeling information, and the SMC method guarantees the MFAC method's fast convergence and low overshooting. The proposed MFASMC control method can maneuver Haidou-1 ARV cruising at the desired forward speed, heading, and depth, even when the dynamic parameters of the ARV vary widely and external disturbances exist. It also addresses the problem of under-actuated motion control for the Haidou-1 ARV. Finally, the simulation results, including comparisons with a PID method and the MFAC method, demonstrate the effectiveness of our proposed method.

Distributed prescribed-time coordinated control of spacecraft formation flying under input saturation

This paper studies the prescribed-time relative motion control problem of spacecraft formation flying under input saturation. Using prescribed-time theory, a prescribed-time sliding mode is designed such that the states on the sliding mode converge to the equilibrium in the prescribed time. Based on the prescribed-time sliding mode, a prescribed-time relative motion tracking controller is developed, which guarantees fast formation maneuvers with feasible fuel consumption and strong robustness under input saturation. Furthermore, a simulation example is carried out to verify the effectiveness of the proposed controller.

Motion Coordination of Multiple Autonomous Mobile Robots under Hard and Soft Constraints

This paper presents a distributed approach to the motion control problem for a platoon of unicycle robots moving through an unknown environment filled with static obstacles under multiple hard and soft operational constraints. Each robot has an onboard camera to determine its relative position in relation to its predecessor and proximity sensors to detect and avoid nearby obstascles. Moreover, no robot apart from the leader can independently localize itself within the given workspace. To overcome this limitation, we propose a novel distributed control protocol for each robot of the fleet, utilizing the Adaptive Performance Control (APC) methodology. By utilizing the APC approach to address input constraints via the on-line modification of the error specifications, we ensure that each follower effectively tracks its predecessor without encountering collisions with obstacles, while simultaneously maintaining visual contact with its preceding robot, thus ensuring the inter-robot visual connectivity. Finally, extensive simulation results are presented to demonstrate the effectiveness of the presented control system along with a real-time experiment conducted on an actual robotic system to validate the feasibility of the proposed approach in real-world scenarios.

Control of a three-wheeled omnidirectional mobile robot via a mixed FTB/H∞\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal H_\\infty $$\\end{document} approach

In this paper, the problem of guidance and motion control of mobile robots is addressed and solved within the novel framework of the mixed finite-time/H∞\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal H_\\infty $$\\end{document} control theory of nonlinear quadratic systems (NLQSs). Starting from a NLQS describing the dynamics of omnidirectional mobile platforms, the main tasks performed for controlling in closed loop the motion of omnidirectional robots can be conveniently formulated as a mixed finite-time/H∞\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal H_\\infty $$\\end{document} control problem. A robust motion controller, which can effectively rejects disturbances deviating the robot platform from a planned path, can be designed after choosing a linear state-feedback structure for the controller. The synthesis problem is solved through some sufficient conditions contemplating both norm-bounded disturbances and sets constraining initial and terminal conditions, together with a finite-time bound on the output transient. Therefore, for all the allowable uncertainties, in presence of nonzero initial conditions and exogenous disturbance inputs which are possible within an unstructured environment, the motion control tasks can be accomplished through optimal H∞\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal H_\\infty $$\\end{document} performance by simultaneously guaranteeing that the NLQS, which governs in closed loop the robot platform, is finite-time bounded. Finally, the applicability and control performance of the design approach have been evaluated through numerical simulations.

Local experience of laboratory activities in a BS physical therapy course: integrating sEMG and kinematics technology with active learning across six cohorts.

Integrating technology and active learning methods into Laboratory activities would be a transformative educational experience to familiarize physical therapy (PT) students with STEM backgrounds and STEM-based new technologies. However, PT students struggle with technology and feel comfortable memorizing under expositive lectures. Thus, we described the difficulties, uncertainties, and advances observed by faculties on students and the perceptions about learning, satisfaction, and grades of students after implementing laboratory activities in a PT undergraduate course, which integrated surface-electromyography (sEMG) and kinematic technology combined with active learning methods. Six cohorts of PT students (n = 482) of a second-year PT course were included. The course had expositive lectures and seven laboratory activities. Students interpreted the evidence and addressed different motor control problems related to daily life movements. The difficulties, uncertainties, and advances observed by faculties on students, as well as the students' perceptions about learning, satisfaction with the course activities, and grades of students, were described. The number of students indicating that the methodology was "always" or "almost always," promoting creative, analytical, or critical thinking was 70.5% [61.0-88.0%]. Satisfaction with the whole course was 97.0% [93.0-98.0%]. Laboratory grades were linearly associated to course grades with a regression coefficient of 0.53 and 0.43 R-squared (p < 0.001). Integrating sEMG and kinematics technology with active learning into laboratory activities enhances students' engagement and understanding of human movement. This approach holds promises to improve teaching-learning processes, which were observed consistently across the cohorts of students.

Data‐driven model‐free predictive control for microstage with coupling and hysteresis nonlinearities

SummaryThe precision motion control problem is investigated in this paper for microstages with cross‐axial coupling and hysteresis. Cross‐axis coupling generally results in stress‐stiffening effects, thereby causing time‐varying dynamics in the microstages. Additionally, when a microstage is driven by piezoelectric actuators (PEAs), the hysteresis effect of the actuator itself must also be considered. Modeling the microstages becomes complicated when both nonlinear characteristics, coupling and hysteresis, coexist. To address this challenge without the need for modeling, a novel data‐driven model‐free predictive control scheme called first‐order tensor‐vector product polynomial approximation based model‐free predictive control (TPPA ‐MFPC is proposed. TPPA ‐MFPC solely relies on the sampling input/output (I/O) data of the systems. The main concept behind TPPA ‐MFPC is to derive a linear approximation model using the I/O data collected during operation. This linear approximation model then serves as a nominal model in a predictive controller, enabling the control of the microstages. Finally, the effectiveness of the proposed TPPA ‐MFPC scheme and the performance improvement over existing model‐free schemes, for example, proportion integration differentiation control (PID), model‐free adaptive control (MFAC), model‐free adaptive predictive control (MFAPC), data‐dependent LMI (DDLMI), and data‐enabled predictive control (DeePC) are demonstrated in the simulation examples with a 2‐degree of freedom (DOF) multileaf spring‐based microstage driven by PEA.

Sequential control barrier functions for mobile robots with dynamic temporal logic specifications

We address a motion planning and control problem for mobile robots to satisfy rich, time-varying tasks expressed as Signal Temporal Logic (STL) specifications. The specifications may include tasks with nested temporal operators or time-conflicting requirements (e.g., achieving periodic tasks or tasks defined within the same time interval). Moreover, the tasks can be defined in locations changing with time (i.e., dynamic targets), and their future motions are not known a priori. This unpredictability requires an online control approach which motivates us to investigate the use of control barrier functions (CBFs). The proposed CBFs take into account the actuation limits of the robots and a feasible sequence of STL tasks. They define time-varying feasible sets of states the system must always stay inside. We show the feasible sequence generation process that even includes the decomposition of periodic tasks and alternative scenarios due to disjunction operators. The sequence is used to define CBFs, ensuring STL satisfaction. We also show some theoretical results on the correctness of the proposed method. We illustrate the benefits of the proposed method and analyze its performance via simulations and experiments with aerial robots.

Motion planning and control of planar serial chain articulating locomotion system

This paper addresses two important problems in motion planning and control of serial chain planar articulating locomotion systems moving on a flat surface. Locomotion systems generally use closed curve cyclic gaits to perform maneuvers in the environment. The geometric mechanics tools are used to identify the cyclic gaits. Firstly, a motion control approach is presented using a single cyclic gait parametrized with variable gait parameters. The parameterized gait is used to drive the system along a continuous trajectory with varying curvature. On the other hand, the proposed approach presents how the gait parameters can be controlled to smoothly track a continuous trajectory. Secondly, this paper presents a joint coordination approach to generalize the height function based approach to design cyclic gaits for the system having joint space with more than two degrees of freedom. Using joint coordination approach, cyclic gaits can be designed for different motion behaviors such as longitudinal motion, sideways motion and rotation in place. The joint coordination approach also helps generalizing the proposed motion control approach to planar serial chain articulating robots with any even number of joints.

Adaptive Safe Reinforcement Learning With Full-State Constraints and Constrained Adaptation for Autonomous Vehicles.

High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

Switching Longitudinal and Lateral Semi-decoupled Active Disturbance Rejection Control for Unmanned Ground Vehicles

In this paper, a switching longitudinal and lateral semi-decoupled active disturbance rejection control (ADRC) strategy is proposed for motion control of unmanned ground vehicles (UGVs). To characterize the coupling property between longitudinal and lateral dynamics, a switching longitudinal and lateral semi-coupled model is first developed. By utilizing the switching method, a so-called “two-stage ADRC” strategy is presented. It has two forms before and after the transition instant such that the boundedness premise for the derivative of the total disturbance can be removed. With the designed two-stage ADRC, a longitudinal and lateral semi-decoupled controller is then suggested to solve the motion control problem of UGVs. Finally, both stability analysis and comparison results are given to demonstrate the effectiveness and robustness of the proposed control strategy.

Dimensionless Policies Based on the Buckingham π Theorem: Is This a Good Way to Generalize Numerical Results?

The answer to the question posed in the title is yes if the context (the list of variables defining the motion control problem) is dimensionally similar. This article explores the use of the Buckingham π theorem as a tool to encode the control policies of physical systems into a more generic form of knowledge that can be reused in various situations. This approach can be interpreted as enforcing invariance to the scaling of the fundamental units in an algorithm learning a control policy. First, we show, by restating the solution to a motion control problem using dimensionless variables, that (1) the policy mapping involves a reduced number of parameters and (2) control policies generated numerically for a specific system can be transferred exactly to a subset of dimensionally similar systems by scaling the input and output variables appropriately. Those two generic theoretical results are then demonstrated, with numerically generated optimal controllers, for the classic motion control problem of swinging up a torque-limited inverted pendulum and positioning a vehicle in slippery conditions. We also discuss the concept of regime, a region in the space of context variables, that can help to relax the similarity condition. Furthermore, we discuss how applying dimensional scaling of the input and output of a context-specific black-box policy is equivalent to substituting new system parameters in an analytical equation under some conditions, using a linear quadratic regulator (LQR) and a computed torque controller as examples. It remains to be seen how practical this approach can be to generalize policies for more complex high-dimensional problems, but the early results show that it is a promising transfer learning tool for numerical approaches like dynamic programming and reinforcement learning.

A leap into the future: Towards an augmented reality learning environment in ski-jumping

Introduction&#x0D; Professional sports are fiercely competitive. In ski jumping, for example, even small changes in take-off and flight can make a decisive difference between victory and defeat (Elfmark et al., 2022). Within the short time of a jump, athletes must learn to solve complex motor control problems while being exposed to harsh environmental conditions, e.g., wind, snow, and low temperatures. The actual take-off occurs within the blink of an eye (~300 ms) and an aerodynamically favourable and stable flight position should be attained immediately. Fine control of the centre of gravity in the in-run favours high speeds to generate optimum momentum during take-off (Müller, 2008). In flight, athletes can voluntarily influence aerodynamics by changing their body position. However, non-optimal flight positions occur unintentionally or due to incorrect behaviour. Furthermore, as a non-cyclical sport, ski jumping suffers from low repetition rates, which impairs the effectiveness of training. Thus, increasing the learning rate for each jump is a key success factor. Biofeedback methods have been shown to accelerate motor learning in athletes (Mulder &amp; Hulstijn, 1985). Current sensor technologies in ski jumping do not meet the requirements for a truly wearable system, which must be energy-efficient, unobtrusive and barely noticeable (so as not to interfere with natural movement behaviour and jumping technique) and, in particular, must be equipped with a wireless link (for real-time data analysis, e.g. on the trainer tower; Schulthess et al., 2023).&#x0D; Methods&#x0D; The proposed system consists of two multi-sensor nodes: One node is hidden in a modified ski jumping boot, integrating three force-sensing resistor sensors to measure the pressure distribution on the foot soles of ski jumpers. The second sensor node is located in the ski goggles and contains RGB LEDs that provide visual biofeedback in the peripheral vision.&#x0D; Results&#x0D; We have calculated the total power consumption of our systems to be 2.52 mW, meeting requirements for multi-day operation between battery recharges. Our on-device body position classification model achieves an accuracy of 92.7% in recognising body positions from data recorded in the laboratory.&#x0D; Discussion/Conclusion&#x0D; This is the first truly wearable training system in ski jumping, offering professional athletes a new augmented experience, aimed at accelerating motor learning. In addition, the real-time data transmission of biomechanically relevant characteristics facilitates the work of the training team and could in the future enable more informative and entertaining television broadcasts.&#x0D; References&#x0D; Elfmark, O., Ettema, G., &amp; Gilgien, M. (2022). Assessment of the steady glide phase in ski jumping. Journal of Biomechanics, 139, 111139. https://doi.org/10.1016/j.jbiomech.2022.111139&#x0D; Mulder, T., &amp; Hulstijn, W. (1985). Sensory feedback in the learning of a novel motor task. Journal of Motor Behavior, 17(1), 110–128. https://doi.org/10.1080/00222895.1985.10735340&#x0D; Müller, W. (2008). Performance factors in ski jumping. In H. Nørstrud (Ed.), Sport Aerodynamics (pp. 139– 160). Springer. https://doi.org/10.1007/978-3-211-89297-8_8&#x0D; Schulthess, L., Ingolfsson, T. M., Nölke, M., Magno, M., Benini, L., &amp; Leitner, C. (2023). Skilog: A smart sensor system for performance analysis and biofeedback in ski jumping. https://doi.org/10.48550/arXiv.2309.14455

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles With Optimized Backstepping.

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. Safety-critical systems in general, require safe performance even during the reinforcement learning (RL) period. To address this issue, a Barrier Lyapunov Function-based safe RL (BLF-SRL) algorithm is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges and incorporates the BLF items into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning. Wherein, thus, the optimal virtual/actual control in every backstepping subsystem is decomposed with BLF items and also with an adaptive uncertain item to be learned, which achieves safe exploration during the learning process. Then, the principle of Bellman optimality of continuous-time Hamilton-Jacobi-Bellman equation in every backstepping subsystem is satisfied with independently approximated actor and critic under the framework of actor-critic through the designed iterative updating. Eventually, the overall system control is optimized with the proposed BLF-SRL method. It is furthermore noteworthy that the variance of the attained control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with two motion control problems for autonomous vehicles through appropriate comparison simulations.

Multitask control of aerial manipulator robots with dynamic compensation based on numerical methods

This paper presents a control scheme for aerial manipulators which allows to solve different motion problems: end-effector position control, end-effector trajectory tracking control and path-following control. The scheme has two cascaded controllers: i) the first controller is a minimum norm controller based on numerical methods, it solves the three motion control problems just by modifying the controller references. Also, since the aerial manipulator robot is a redundant system, i.e., it has extra degrees of freedom to accomplish the task, it is possible to set other control objectives in a hierarchical order. As a secondary objective of the control it is proposed to maintain a desired configuration for the robotic arm during the task. ii) The second cascade controller is designed to compensate the dynamics of the system which main objective is to drive the velocity errors to zero. The coupled dynamic model of the robotic system (hexarotor and robotic arm) is presented. This model is usually developed as a function of the forces and torques. However, in this work, it is written as a function of reference velocities which are usual references for these vehicles. The proposed control algorithms are given with the corresponding stability and robustness analysis. Finally, to validate the control scheme, experimental tests are performed in a partially structured environment with an aerial manipulator conformed by an aerial platform and a 3DOF robotic arm.

Disturbance Observer-Based Model Predictive Control for an Unmanned Underwater Vehicle

This work addresses the motion control problem for a 4-degree-of-freedom unmanned underwater vehicle (UUV) in the presence of nonlinear dynamics, parametric uncertainties, system constraints, and time-varying external disturbances. A disturbance observer-based control scheme is proposed, which is structured around the model predictive control (MPC) method integrated with an extended active observer (EAOB). Compared to the conventional disturbance observer, the developed EAOB has the ability to handle both external disturbances and system/measurement noises simultaneously. The EAOB leverages a combination of sensor measurements and a system dynamic model to estimate disturbances in real-time, which allows continuous estimation and compensation of time-varying disturbances back to the controller. The proposed disturbance observer-based MPC is implemented by feeding the estimated disturbances back into the MPC’s prediction model, which forms an effective adaptive controller with a parameter-varying model. The proposed control strategy is validated through simulations in a Gazebo and robot operating system environment. The results show that the proposed method can effectively reject unpredictable disturbances and improve the UUV’s control performance.

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.