Quadruped Robot Locomotion Research Articles

Dual-Layer Reinforcement Learning for Quadruped Robot Locomotion and Speed Control in Complex Environments

Dual-Layer Reinforcement Learning for Quadruped Robot Locomotion and Speed Control in Complex Environments

Adaptive control algorithm for quadruped robots in unknown high-slope terrain

The capability to navigate high-slope terrains is crucial for quadruped robots engaged in field operations. We propose a novel terrain estimation and adaptation strategy tailored for facilitating quadruped robot locomotion on complex slope terrains. Our approach involves predicting terrain slopes by analyzing foot positions and IMU data, subsequently adjusting the robot’s body orientation and height in real-time to accommodate varying slope conditions. To effectively control the motion of quadruped robots on such challenging terrains, we introduce a multimodal motion control algorithm that integrates Model Predictive Control (MPC) with Quadratic Programming (QP) torque control. This combined approach ensures stable and efficient locomotion even on steep slopes. Furthermore, we present a state estimation method based on Kalman filtering, enabling accurate self-assessment by the robot without heavy reliance on visual sensors. In simulation validation, our quadruped robot achieved notable performance metrics. It achieved a forward speed of 0.7 m/s on slopes with angles up to 43∘ and demonstrated stable rotational capability at a speed of 2 rad/s on a 32∘ slope. When subjected to external force interference, the robot exhibited resilience, withstanding constant external forces of up to 60Nm and external torques of up to 35Nm on flat ground. Moreover, on a 30∘ slope, the robot maintained stable locomotion in the face of impulses reaching 64Nm ⋅ s along the x and y directions of its body coordinate system and 7Nm ⋅ s along the z-axis. This comprehensive strategy and experimental validation highlight the efficacy and robustness of our adaptive control algorithm, paving the way for enhanced performance of quadruped robots navigating high-slope terrains with unpredictable characteristics.

Contact detection with multi-information fusion for quadruped robot locomotion under unstructured terrain

Contact detection with multi-information fusion for quadruped robot locomotion under unstructured terrain

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

This paper investigates a model-free reinforcement learning-based approach that enables the quadruped robot to manipulate objects while maintaining its balance and dynamic stability during walking. At first, the dynamics of quadruped robots in two sub-spaces, position control space and force control space, are developed. Then, a new long-term performance index is introduced, and a radial basis function neural network as a critic network is presented to estimate the unobtainable long-term performance index. Based on the exported reinforcement signal, the actor neural network is introduced to generate the feedforward compensation term to cope with the nonlinear dynamics and the system uncertainties. The robustness of the actor-critic reinforcement learning algorithm is enhanced by using a fractional-order sliding-mode controller in the closed-loop system. The online adaptive laws for both the critic and actor-network weights are obtained using the Lyapunov stability theory. As a result, the uniformly ultimately boundedness of the position and the force tracking errors are proven. Finally, numerical simulations are conducted to illustrate the feasibility and effectiveness of the proposed adaptive actor-critic learning-based control scheme.

Unknown Payload Adaptive Control for Quadruped Locomotion With Proprioceptive Linear Legs

Quadruped robots manifest great potential to traverse rough terrains with payload. Current model-based controllers, which are extensively adopted in quadruped robot locomotion control, rely on accurate estimation of parameters and will significantly deteriorate in severe disturbance, e.g., adding heavy payload. This article introduces an online identification method, which is named as adaptive control for quadruped locomotion, to address model uncertainties. Meanwhile, a concurrent adaptive controller is also indispensable to accomplish identification and locomotion. Therefore, this article presents an adaptive control algorithm based on the online payload identification for the high payload capacity (the ratio between payload and robot’s self-weight) quadruped locomotion. The newly proposed algorithm could achieve estimating and compensating the external disturbances induced by the payload online. The tracking accuracy of the robot’s center of mass and orientation trajectories for the identification task is highly improved. The locomotion task can be incorporated in inverse-dynamics-based quadratic programming, realizing a trotting gait. The proposed method is verified in a real quadruped robot platform. Experiments prove the estimation efficacy for the payload weighing from 20 to 75 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm kg$</tex-math></inline-formula> and loaded at different locations of the robot’s torso.

PSTO: Learning Energy-Efficient Locomotion for Quadruped Robots

Energy efficiency is critical for the locomotion of quadruped robots. However, energy efficiency values found in simulations do not transfer adequately to the real world. To address this issue, we present a novel method, named Policy Search Transfer Optimization (PSTO), which combines deep reinforcement learning and optimization to create energy-efficient locomotion for quadruped robots in the real world. The deep reinforcement learning and policy search process are performed by the TD3 algorithm and the policy is transferred to the open-loop control trajectory further optimized by numerical methods, and conducted on the robot in the real world. In order to ensure the high uniformity of the simulation results and the behavior of the hardware platform, we introduce and validate the accurate model in simulation including consistent size and fine-tuning parameters. We then validate those results with real-world experiments on the quadruped robot Ant by executing dynamic walking gaits with different leg lengths and numbers of amplifications. We analyze the results and show that our methods can outperform the control method provided by the state-of-the-art policy search algorithm TD3 and sinusoid function on both energy efficiency and speed.

Attitude control in the Mini Cheetah robot via MPC and reward-based feed-forward controller

In this paper, a model-free approach, based on a generalization of unsupervised Self Organizing Feature Maps, is introduced to compensate for attitude control errors in a simulated mini cheetah quadruped platform. Traditional techniques mainly exploit the potentialities of convex model predictive control (MPC) to efficiently regulate the robot attitude while moving in unstructured environments. However, they are based on the knowledge of the analytical model of the robot. If the robot structure undergoes significant modifications due, for example to an unbalancing of the robot weight caused by a load charged of the robot center of mass, the added value of a model-free, adaptive unsupervised nonlinear controller, acting as a feed-forward error compensator, shows its real effect when integrated with the model-based approach. Moreover, the proposed solution, acting as a self-learning feedforward controller, contributes to the control action only when needed, preserving the basic performance of the ongoing MPC action. The design of the control scheme and simulation results will be reported, showing the impact of the introduced model-free compensation on the quadruped robot locomotion.

Optimized design of 5R planar parallel mechanism aimed at gait-cycle of quadruped robots

In quadruped robot locomotion using parallel mechanisms, researchers have used equal link lengths as legs for walking. However, force requirements are not the same in the forward and return strokes. An unsymmetrical parallel mechanism can be considered to accommodate such requirements. This work presents optimized dimensions of a 5R planar parallel mechanism (5R-PPM) with two degrees of freedom (DoF). Optimized dimensions are determined by formulating an optimization problem using kinematics and dynamics equations for the 5R-PPM. Genetic algorithm is considered to obtain solutions for the optimization problem formulated in this study. The constraint condition expressed here for optimization will attempt to minimize the peak torque essential to displace the links in the mechanism for the given height of the robot body and the path to be traced by the end-effector. After analysing all the four possible working modes for the same end-effector movement, the best working mode is selected for the quadruped legs. The equations are formulated and solved in MATLAB, and validated in the MATLAB Simscape Multibody toolbox.

Fast and Slow Adaptations of Interlimb Coordination via Reflex and Learning During Split-Belt Treadmill Walking of a Quadruped Robot.

Interlimb coordination plays an important role in adaptive locomotion of humans and animals. This has been investigated using a split-belt treadmill, which imposes different speeds on the two sides of the body. Two types of adaptation have been identified, namely fast and slow adaptations. Fast adaptation induces asymmetric interlimb coordination soon after a change of the treadmill speed condition from same speed for both belts to different speeds. In contrast, slow adaptation slowly reduces the asymmetry after fast adaptation. It has been suggested that these adaptations are primarily achieved by the spinal reflex and cerebellar learning. However, these adaptation mechanisms remain unclear due to the complicated dynamics of locomotion. In our previous work, we developed a locomotion control system for a biped robot based on the spinal reflex and cerebellar learning. We reproduced the fast and slow adaptations observed in humans during split-belt treadmill walking of the biped robot and clarified the adaptation mechanisms from a dynamic viewpoint by focusing on the changes in the relative positions between the center of mass and foot stance induced by reflex and learning. In this study, we modified the control system for application to a quadruped robot. We demonstrate that even though the basic gait pattern of our robot is different from that of general quadrupeds (due to limitations of the robot experiment), fast and slow adaptations that are similar to those of quadrupeds appear during split-belt treadmill walking of the quadruped robot. Furthermore, we clarify these adaptation mechanisms from a dynamic viewpoint, as done in our previous work. These results will increase the understanding of how fast and slow adaptations are generated in quadrupedal locomotion on a split-belt treadmill through body dynamics and sensorimotor integration via the spinal reflex and cerebellar learning and help the development of control strategies for adaptive locomotion of quadruped robots.

Model Predictive Control With Environment Adaptation for Legged Locomotion

Re-planning in legged locomotion is crucial to track the desired user velocity while adapting to the terrain and rejecting external disturbances. In this work, we propose and test in experiments a real-time Nonlinear Model Predictive Control (NMPC) tailored to a legged robot for achieving dynamic locomotion on a variety of terrains. We introduce a mobility-based criterion to define an NMPC cost that enhances the locomotion of quadruped robots while maximizing leg mobility and improves adaptation to the terrain features. Our NMPC is based on the real-time iteration scheme that allows us to re-plan online at $25\,\mathrm{Hz}$ with a prediction horizon of $2$ seconds. We use the single rigid body dynamic model defined in the center of mass frame in order to increase the computational efficiency. In simulations, the NMPC is tested to traverse a set of pallets of different sizes, to walk into a V-shaped chimney,and to locomote over rough terrain. In real experiments, we demonstrate the effectiveness of our NMPC with the mobility feature that allowed IIT's $87\, \mathrm{kg}$ quadruped robot HyQ to achieve an omni-directional walk on flat terrain, to traverse a static pallet, and to adapt to a repositioned pallet during a walk.

Multi-expert learning of adaptive legged locomotion.

Achieving versatile robot locomotion requires motor skills that can adapt to previously unseen situations. We propose a multi-expert learning architecture (MELA) that learns to generate adaptive skills from a group of representative expert skills. During training, MELA is first initialized by a distinct set of pretrained experts, each in a separate deep neural network (DNN). Then, by learning the combination of these DNNs using a gating neural network (GNN), MELA can acquire more specialized experts and transitional skills across various locomotion modes. During runtime, MELA constantly blends multiple DNNs and dynamically synthesizes a new DNN to produce adaptive behaviors in response to changing situations. This approach leverages the advantages of trained expert skills and the fast online synthesis of adaptive policies to generate responsive motor skills during the changing tasks. Using one unified MELA framework, we demonstrated successful multiskill locomotion on a real quadruped robot that performed coherent trotting, steering, and fall recovery autonomously and showed the merit of multi-expert learning generating behaviors that can adapt to unseen scenarios.

Generalization of movements in quadruped robot locomotion by learning specialized motion data

Machines that are sensitive to environmental fluctuations, such as autonomous and pet robots, are currently in demand, rendering the ability to control huge and complex systems crucial. However, controlling such a system in its entirety using only one control device is difficult; for this purpose, a system must be both diverse and flexible. Herein, we derive and analyze the feature values of robot sensor and actuator data, thereby investigating the role that each feature value plays in robot locomotion. We conduct experiments using a developed quadruped robot from which we acquire multi-point motion information as the movement data; we extract the features of these movement data using an autoencoder. Next, we decompose the movement data into three features and extract various gait patterns. Despite learning only the “walking” movement, the movement patterns of trotting and bounding are also extracted herein, which suggests that movement data obtained via hardware contain various gait patterns. Although the present robot cannot locomote with these movements, this research suggests the possibility of generating unlearned movements.

Virtual Model Control for Quadruped Robots

Virtual model control is a motion control framework that uses virtual components to create virtual forces/torques, which are actually generated by joint actuators when the virtual components interact with robot systems. Firstly, this paper employs virtual model control to do a dynamic balance control of whole body of quadruped robots' trot gait in a bottom controller. In each leg, there exists a designed swing phase virtual model control and a stance phase counterparts. In the whole body, virtual model control is utilized to achieve a attitude control containing roll, pitch and yaw. In the attitude control, a forces/torques distribution method between two stance legs is pre-investigated. In a high-level implemented controller, an intuitive velocity control approach proposed by Raibert is applied for the locomotion of quadruped robots. Secondly, an anti-disturbance control, which contains compensating gravity, adjusting step length, adjusting swing trajectory, adjusting attitude, and adjusting virtual forces/torques, is investigated to improve the robustness, terrain adaptability, and dynamic balance performance of quadrupedal locomotion. Thirdly, a trajectory tracking control method based on an intuitive velocity control is addressed through considering four factors: terrain complexity index, curvature radius of given trajectory, distance to terminal, and maximum velocity of quadruped robots. Finally, simulations validate the effectiveness of proposed controllers.

Creeping Gait Analysis and Simulation of a Quadruped Robot

A quadruped (four-legged) robot locomotion has the potential ability for using in different applications such as walking over soft and rough terrains and to grantee the mobility and flexibility. In general, quadruped robots have three main periodic gaits: creeping gait, running gait and galloping gait. The main problem of the quadruped robot during walking is the needing to be statically stable for slow gaits such as creeping gait. The statically stable walking as a condition depends on the stability margins that calculated particularly for this gait. In this paper, the creeping gait sequence analysis of each leg step during the swing and fixed phases has been carried out. The calculation of the minimum stability margins depends upon the forward and inverse kinematic models for each 3-DOF leg and depends on vertical geometrical projection during walking. Simulation and results verify the stability insurance after calculation the minimum margins which indicate clearly the robot COG (Center of Gravity) inside the supporting polygon resulted from the leg-tips.

Autonomous gait transition and galloping over unperceived obstacles of a quadruped robot with CPG modulated by vestibular feedback

The aim of this paper is to demonstrate, based on robot results, the effectiveness and practicability of vestibular feedback to central pattern generators (CPG) employed for the locomotion of quadruped robots. We build a new quadruped robot with mechanisms enabling walking to running and apply CPGs modulated by simple vestibular sensory feedback (a body tilt multiplied by a fixed gain). As a result, the robot safely locomotes at a variety of speeds by autonomously changing the gait from walking to trotting to galloping according to speed, despite the fact that the walk and gallop are not preprogrammed. In addition, as this paper’s major contribution, we discover and demonstrate that the robot robustly runs with an emergent gallop while stepping on and over several types of unperceived obstacles, while being suddenly pulled forward, and while the physical balance is changed (i.e., a weight is put forward on the robot), by autonomously modifying the phase differences between the four legs from the basic gallop. To our knowledge, no other galloping robots have been reported that can adapt to an unperceived obstacle. We conclude that CPGs modulated by vestibular feedback is effective and practical as a gait generator for bio-inspired robots that are expected to have both the abilities of “speed-based autonomous gait transition” and “autonomous robust running”.

Leg Control for Quadruped Locomotion Robot using straight-fiber-type pneumatic artificial muscle

Leg Control for Quadruped Locomotion Robot using straight-fiber-type pneumatic artificial muscle

Sequential Linear Quadratic Optimal Control for Nonlinear Switched Systems

In this contribution, we introduce an efficient method for solving the optimal control problem for an unconstrained nonlinear switched system with an arbitrary cost function. We assume that the sequence of the switching modes are given but the switching time in between consecutive modes remains to be optimized. The proposed method uses a two-stage approach as introduced by Xu and Antsaklis (2004) where the original optimal control problem is transcribed into an equivalent problem parametrized by the switching times and the optimal control policy is obtained based on the solution of a two-point boundary value differential equation. The main contribution of this paper is to use a Sequential Linear Quadratic approach to synthesize the optimal controller instead of solving a boundary value problem. The proposed method is numerically more efficient and scales very well to the high dimensional problems. In order to evaluate its performance, we use two numerical examples as benchmarks to compare against the baseline algorithm. In the third numerical example, we apply the proposed algorithm to the Center of Mass control problem in a quadruped robot locomotion task.

A sensory-driven controller for quadruped locomotion

Locomotion of quadruped robots has not yet achieved the harmony, flexibility, efficiency and robustness of its biological counterparts. Biological research showed that spinal reflexes are crucial for a successful locomotion in the most varied terrains. In this context, the development of bio-inspired controllers seems to be a good way to move toward an efficient and robust robotic locomotion, by mimicking their biological counterparts. This contribution presents a sensory-driven controller designed for the simulated Oncilla quadruped robot. In the proposed reflex controller, movement is generated through the robot's interactions with the environment, and therefore, the controller is solely dependent on sensory information. The results show that the reflex controller is capable of producing stable quadruped locomotion with a regular stepping pattern. Furthermore, it is capable of dealing with slopes without changing the parameters and with small obstacles, overcoming them successfully. Finally, system robustness was verified by adding noise to sensors and actuators and also delays.

Spinal joint compliance and actuation in a simulated bounding quadruped robot

Spine movements play an important role in quadrupedal locomotion, yet their potential benefits in locomotion of quadruped robots have not been systematically explored. In this work, we investigate ...

Combining central pattern generators and reflexes

Locomotion of quadruped robots has not yet achieved the robustness, harmony, efficiency and flexibility of its biological counterparts. Biological evidences showed that there is a two-way interaction between the Central Pattern Generators (CPGs) and the body in the locomotion process of animals. Therefore, the development of bio-inspired controllers seems to be a good and robust way to obtain an efficient and robust robotic locomotion. This contribution presents an innovative hybrid controller that generates locomotion through the combination of CPGs and reflexes. The results show that the hybrid controller is capable of producing stable quadruped locomotion with a regular stepping pattern. Furthermore, it proved to be able to deal with slopes without changing the parameters and with small obstacles, overcoming them successfully.