Autonomous Racing Research Articles

Reinforcement Learning in Autonomous Racing

Reinforcement Learning in Autonomous Racing

ForzaETH Race Stack—Scaled Autonomous Head‐to‐Head Racing on Fully Commercial Off‐the‐Shelf Hardware

ABSTRACTAutonomous racing in robotics combines high‐speed dynamics with the necessity for reliability and real‐time decision‐making. While such racing pushes software and hardware to their limits, many existing full‐system solutions necessitate complex, custom hardware and software, and usually focus on Time‐TrIals rather than full unrestricted Head‐to‐head racing, due to financial and safety constraints. This limits their reproducibility, making advancements and replication feasible mostly for well‐resourced laboratories with comprehensive expertise in mechanical, electrical, and robotics fields. Researchers interested in the autonomy domain but with only partial experience in one of these fields, need to spend significant time with familiarization and integration. The ForzaETH Race Stack addresses this gap by providing an autonomous racing software platform designed for F1TENTH, a 1:10 scaled Head‐to‐Head autonomous racing competition, which simplifies replication by using commercial off‐the‐shelf hardware. This approach enhances the competitive aspect of autonomous racing and provides an accessible platform for research and development in the field. The ForzaETH Race Stack is designed with modularity and operational ease of use in mind, allowing customization and adaptability to various environmental conditions, such as track friction and layout, which is exemplified by the various modularly implemented state estimation and control systems. Capable of handling both Time‐Trials and Head‐to‐Head racing, the stack has demonstrated its effectiveness, robustness, and adaptability in the field by winning the official F1TENTH international competition multiple times. Furthermore, the stack demonstrated its reliability and performance at unprecedented scales, up to over on tracks up to 150 m in length.

Three-player autonomous racing by Stackelberg game methods

This paper considers the autonomous racing of three cars, including a team of two cars and an opponent car. To handle it, the competition is modeled as a two-leader one-follower Stackelberg game to obtain the optimal strategies for each car. In the sequential game, all the cars maximise their progress while avoiding collisions. In the blocking game, blocking behaviours are taken into account by adding a reward to the payoff function. Through successful collaboration, actions are more aggressive in the second game. Given that the Stackelberg equilibrium is not unique in both games, the cost functions are designed for the players to cope with multiple strategies that have the same progress. The competitions are performed in a receding horizon fashion, and the aim of the research is to study the effects of cooperation and make the team have successful blocking behaviours under constraints as well as speed disadvantages.

Localization robustness improvement for an autonomous race car using multiple extended Kalman filters

In this paper, we introduce a vehicle localization method designed for the SZEnergy race car, which competes in the Shell Eco-marathon. The proposed method comprises four different extended Kalman filter-based localization algorithms and a selection algorithm that determines the most suitable one based on vehicle speed, GNSS availability, and signal quality. The low-speed Kalman filters are based on a kinematic vehicle model while the high-speed variants are based on a dynamic vehicle model. Several measurements were performed during test maneuvers to evaluate the performance of the filters. The proposed method succesfully handles sensor miscalibration and GNSS outages.

Inherently Robust Suboptimal MPC for Autonomous Racing With Anytime Feasible SQP

Inherently Robust Suboptimal MPC for Autonomous Racing With Anytime Feasible SQP

Gaussian Processes for Vehicle Dynamics Learning in Autonomous Racing

Gaussian Processes for Vehicle Dynamics Learning in Autonomous Racing

Deep Dynamics: Vehicle Dynamics Modeling With a Physics-Constrained Neural Network for Autonomous Racing

Deep Dynamics: Vehicle Dynamics Modeling With a Physics-Constrained Neural Network for Autonomous Racing

Enhancing the Safety of Autonomous Vehicles in Adverse Weather by Deep Learning-Based Object Detection

Recognizing and categorizing items in weather-adverse environments poses significant challenges for autonomous vehicles. To improve the robustness of object-detection systems, this paper introduces an innovative approach for detecting objects at different levels by leveraging sensors and deep learning-based solutions within a traffic circle. The suggested approach improves the effectiveness of single-stage object detectors, aiming to advance the performance in perceiving autonomous racing environments and minimizing instances of false detection and low recognition rates. The improved framework is based on the one-stage object-detection model, incorporating multiple lightweight backbones. Additionally, attention mechanisms are integrated to refine the object-detection process further. Our proposed model demonstrates superior performance compared to the state-of-the-art method on the DAWN dataset, achieving a mean average precision (mAP) of 99.1%, surpassing the previous result of 84.7%.

Learning autonomous race driving with action mapping reinforcement learning

Autonomous race driving poses a complex control challenge as vehicles must be operated at the edge of their handling limits to reduce lap times while respecting physical and safety constraints. This paper presents a novel reinforcement learning (RL)-based approach, incorporating the action mapping (AM) mechanism to manage state-dependent input constraints arising from limited tire-road friction. A numerical approximation method is proposed to implement AM, addressing the complex dynamics associated with the friction constraints. The AM mechanism also allows the learned driving policy to be generalized to different friction conditions. Experimental results in our developed race simulator demonstrate that the proposed AM-RL approach achieves superior lap times and better success rates compared to the conventional RL-based approaches. The generalization capability of driving policy with AM is also validated in the experiments.

Memory-based Distribution Shift Detection for Learning Enabled Cyber-Physical Systems with Statistical Guarantees

Incorporating learning based components in the current state-of-the-art cyber-physical systems (CPS) has been a challenge due to the brittleness of the underlying deep neural networks. On the bright side, if executed correctly with safety guarantees, this has the ability to revolutionize domains like autonomous systems, medicine, and other safety-critical domains. This is because it would allow system designers to use high-dimensional outputs from sensors like camera and LiDAR. The trepidation in deploying systems with vision and LiDAR components comes from incidents of catastrophic failures in the real world. Recent reports of self-driving cars running into difficult to handle scenarios is ingrained in the software components which handle such sensor inputs. The ability to handle such high-dimensional signals is due to the explosion of algorithms which use deep neural networks. Sadly, the reason behind the safety issues is also due to deep neural networks themselves. The pitfalls occur due to possible over-fitting and lack of awareness about the blind spots induced by the training distribution. Ideally, system designers would wish to cover as many scenarios during training as possible. However, achieving a meaningful coverage is impossible. This naturally leads to the following question: is it feasible to flag out-of-distribution (OOD) samples without causing too many false alarms? Such an OOD detector should be executable in a fashion that is computationally efficient. This is because OOD detectors often are executed as frequently as the sensors are sampled. Our aim in this article is to build an effective anomaly detector. To this end, we propose the idea of a memory bank to cache data samples which are representative enough to cover most of the in-distribution data. The similarity with respect to such samples can be a measure of familiarity of the test input. This is made possible by an appropriate choice of distance function tailored to the type of sensor we are interested in. Additionally, we adapt conformal anomaly detection framework to capture the distribution shifts with a guarantee of false alarm rate. We report the performance of our technique on two challenging scenarios: a self-driving car setting implemented inside the simulator CARLA with image inputs and autonomous racing car navigation setting with LiDAR inputs. From the experiments, it is clear that a deviation from the in-distribution setting can potentially lead to unsafe behavior. It should be noted that not all OOD inputs lead to precarious situations in practice, but staying in-distribution is akin to staying within a safety bubble and predictable behavior. An added benefit of our memory-based approach is that the OOD detector produces interpretable feedback for a human designer. This is of utmost importance since it recommends a potential fix for the situation as well. In other competing approaches, such feedback is difficult to obtain due to reliance on techniques which use variational autoencoders.

Learning-Based Control of Autonomous Vehicles Using an Adaptive Neuro-Fuzzy Inference System and the Linear Matrix Inequality Approach.

This paper proposes a learning-based control approach for autonomous vehicles. An explicit Takagi-Sugeno (TS) controller is learned using input and output data from a preexisting controller, employing the Adaptive Neuro-Fuzzy Inference System (ANFIS) algorithm. At the same time, the vehicle model is identified in the TS model form for closed-loop stability assessment using Lyapunov theory and LMIs. The proposed approach is applied to learn the control law from an MPC controller, thus avoiding the use of online optimization. This reduces the computational burden of the control loop and facilitates real-time implementation. Finally, the proposed approach is assessed through simulation using a small-scale autonomous racing car.

A Survey of Vehicle Dynamics Modeling Methods for Autonomous Racing: Theoretical Models, Physical/Virtual Platforms, and Perspectives

A Survey of Vehicle Dynamics Modeling Methods for Autonomous Racing: Theoretical Models, Physical/Virtual Platforms, and Perspectives

Autonomous Robot Racing Competitions: Truly Multivehicle Autonomous Racing Competitions [Competitions

Autonomous Robot Racing Competitions: Truly Multivehicle Autonomous Racing Competitions [Competitions

Deep Reinforcement Learning for Autonomous Driving in Amazon Web Services DeepRacer

The development of autonomous driving models through reinforcement learning has gained significant traction. However, developing obstacle avoidance systems remains a challenge. Specifically, optimising path completion times while navigating obstacles is an underexplored research area. Amazon Web Services (AWS) DeepRacer emerges as a powerful infrastructure for engineering and analysing autonomous models, providing a robust foundation for addressing these complexities. This research investigates the feasibility of training end-to-end self-driving models focused on obstacle avoidance using reinforcement learning on the AWS DeepRacer autonomous race car platform. A comprehensive literature review of autonomous driving methodologies and machine learning model architectures is conducted, with a particular focus on object avoidance, followed by hands-on experimentation and the analysis of training data. Furthermore, the impact of sensor choice, reward function, action spaces, and training time on the autonomous obstacle avoidance task are compared. The results of the best configuration experiment demonstrate a significant improvement in obstacle avoidance performance compared to the baseline configuration, with a 95.8% decrease in collision rate, while taking about 79% less time to complete the trial circuit.

Editorial: Special Issue on Opportunities and Challenges with Autonomous Racing

In the ever-evolving landscape of autonomous vehicles, competition and research of high-speed autonomous racing emerged as a captivating frontier, pushing the limits of perception, planning, and control. Autonomous racing presents a setup where the intersection of cutting-edge software and hardware development sparks unprecedented opportunities and confronts unique challenges. The motorsport axiom, “If everything seems under control, then you are not going fast enough,” resonates in this special issue, underscoring the demand for algorithms and hardware that can navigate at the cutting edge of control, traction, and agility. In pursuing autonomy at high speeds, the racing environment becomes a crucible, pushing autonomous vehicles to execute split-second decisions with high precision. Autonomous racing, we believe, offers a litmus test for the true capabilities of self-driving software. Just as racing has historically served as a proving ground for automotive technology, autonomous racing now presents itself as the crucible for testing self-driving algorithms. While routine driving situations dominate much of the autonomous vehicle operations, focusing on extreme situations and environments is crucial to support investigation into safety benefits. The urgency of advancing highspeed autonomy is palpable in burgeoning autonomous racing competitions like Formula Student Driverless, F1TENTH autonomous racing, Roborace, and the Indy Autonomous Challenge. These arenas provide a literal testbed for testing perception, planning, and control algorithms and symbolize the accelerating traction of autonomous racing as a proving ground for agile and safe autonomy. Our special issue focuses on cutting-edge research into software and hardware solutions for highspeed autonomous racing. We sought contributions from the robotics and autonomy communities that delve into the intricacies of head-to-head multi-agent racing: modeling vehicle dynamics at high speeds, developing advanced perception, planning, and control algorithms, as well as the demonstration of algorithms, in simulation and in real-world vehicles. While presenting recent developments for autonomous racing, we believe these special issue papers will also create an impact in the broader realm of autonomous vehicles.

Fast and Modular Autonomy Software for Autonomous Racing Vehicles

Autonomous motorsports aim to replicate the human race-car driver with software and sensors. As in traditional motorsports, Autonomous Racing Vehicles (ARVs) are pushed to their handling limits in multiagent scenarios at extremely high ( 150 mph) speeds. This Operational Design Domain (ODD) presents unique challenges across the autonomy stack. The Indy Autonomous Challenge (IAC) is an international competition aiming to advance autonomous vehicle development through ARV competitions. While far from challenging what a human race-car driver can do, the IAC is pushing the state of the art by facilitating full-sized ARV competitions. This paper details the MIT-Pitt-RW Team’s approach to autonomous racing in the IAC. In this work, we present our modular and fast approach to agent detection, motion planning and controls to create an autonomy stack. We also provide analysis of the performance of the software stack in single and multiagent scenarios for rapid deployment in a fast-paced competition environment. We also cover what did and did not work when deployed on a physical system (the Dallara AV-21 platform) and potential improvements to address these shortcomings. Finally, we convey lessons learned and discuss limitations and future directions for improvement.

Hierarchical Control for Head-to-Head Autonomous Racing

We develop a hierarchical controller for head-to-head autonomous racing. We first introduce a formulation of a racing game with realistic safety and fairness rules. A high-level planner approximates the original formulation as a discrete game with simplified state, control, and dynamics to easily encode the complex safety and fairness rules and calculates a series of target waypoints. The low-level controller takes the resulting waypoints as a reference trajectory and computes high-resolution control inputs by solving an alternative approximation formulation with simplified objectives and constraints. We consider two approaches for the low-level planner, constructing two hierarchical controllers. One approach uses multi-agent reinforcement learning (MARL), and the other solves a linear-quadratic Nash game (LQNG) to produce control inputs. The controllers are compared against three baselines: an end-to-end MARL controller, a MARL controller tracking a fixed racing line, and an LQNG controller tracking a fixed racing line. Quantitative results show that the proposed hierarchical methods outperform their respective baseline methods in terms of head-tohead race wins and abiding by the rules. The hierarchical controller using MARL for low-level control consistently outperformed all other methods by winning over 90% of head-to-head races and more consistently adhered to the complex racing rules. Qualitatively, we observe the proposed controllers mimicking actions performed by expert human drivers such as shielding/blocking, overtaking, and long-term planning for delayed advantages. We show that hierarchical planning for game-theoretic reasoning produces competitive behavior even when challenged with complex rules and constraints.

Integrating Large Language Models and Metaverse in Autonomous Racing: An Education-Oriented Perspective

Integrating Large Language Models and Metaverse in Autonomous Racing: An Education-Oriented Perspective

MPTree: A Sampling-based Vehicle Motion Planner for Real-time Obstacle Avoidance

This paper presents a novel and modular framework, named MPTree, for real-time vehicle motion planning with dynamic obstacle avoidance. MPTree adopts a sampling-based algorithm, to explore a tree of Motion Primitives (MPs) and return near-optimal trajectories. Specifically, MPTree builds motion primitives to connect the tree nodes (waypoints), sampled in a mesh of waylines on the local planning horizon. The tree exploration is based on a semi-structured RRTa, with an application-specific cost function (e.g., minimum-jerk or minimum-time) and high-level behavioral policy. We show examples of MPTREE’s specialization for urban environments and autonomous racing, using fast-to-evaluate motion primitives to accelerate the tree exploration phase. A prototype implementation is tested in a closed-loop simulation environment. Our preliminary results show that MPTree provides feasible collision-free trajectories while ensuring low computational times.

Active Exploration in Iterative Gaussian Process Regression for Uncertainty Modeling in Autonomous Racing

Active Exploration in Iterative Gaussian Process Regression for Uncertainty Modeling in Autonomous Racing