Autonomous Racing Research Articles

Online Verification Enabling Approval of Driving Functions—Implementation for a Planner of an Autonomous Race Vehicle

Safety guarantees and regulatory approval for autonomous vehicles remain an ongoing challenge. In particular, software that is frequently adapted or contains complex, non-transparent components, such as artificial intelligence, is exceeding the limits of safety standards. This paper presents a detailed implementation of an online verification module – the Supervisor – that copes with these challenges. The presented implementation focuses on autonomous race vehicles without loss of generality. Following an identified holistic list of safety-relevant requirements for a trajectory, metrics are developed to monitor whether the trajectory can safely be executed. To evaluate safety with respect to dynamic objects in a semi-structured and highly dynamic racing environment, rule-based reachable sets are presented. As a result, the pure reachable set is further constrained by applicable regulations. Real-time capability and effectiveness are demonstrated in fault-injected scenario-based tests and on real-world run data. The implemented Supervisor will be publicly available on GitHub.

Real-Time Adaptive Velocity Optimization for Autonomous Electric Cars at the Limits of Handling

With the evolution of self-driving cars, autonomous racing series like Roborace and the Indy Autonomous Challenge are rapidly attracting growing attention. Researchers participating in these competitions hope to subsequently transfer their developed functionality to passenger vehicles, in order to improve self-driving technology for reasons of safety, and due to environmental and social benefits. The race track has the advantage of being a safe environment where challenging situations for the algorithms are permanently created. To achieve minimum lap times on the race track, it is important to gather and process information about external influences including, e.g., the position of other cars and the friction potential between the road and the tires. Furthermore, the predicted behavior of the ego-car's propulsion system is crucial for leveraging the available energy as efficiently as possible. In this paper, we therefore present an optimization-based velocity planner, mathematically formulated as a multi-parametric Sequential Quadratic Problem (mpSQP). This planner can handle a spatially and temporally varying friction coefficient, and transfer a race Energy Strategy (ES) to the road. It further handles the velocity-profile-generation task for performance and emergency trajectories in real time on the vehicle's Electronic Control Unit (ECU).

End-to-End Velocity Estimation for Autonomous Racing

Velocity estimation plays a central role in driverless vehicles, but standard, and affordable methods struggle to cope with extreme scenarios like aggressive maneuvers due to the presence of high sideslip. To solve this, autonomous race cars are usually equipped with expensive external velocity sensors. In this letter, we present an end-to-end recurrent neural network that takes available raw sensors as input (IMU, wheel odometry, and motor currents), and outputs velocity estimates. The results are compared to two state-of-the-art Kalman filters, which respectively include, and exclude expensive velocity sensors. All methods have been extensively tested on a formula student driverless race car with very high sideslip (10° at the rear axle), and slip ratio (≈20%), operating close to the limits of handling. The proposed network is able to estimate lateral velocity up to 15x better than the Kalman filter with the equivalent sensor input, and matches (0.06 m/s RMSE) the Kalman filter with the expensive velocity sensor setup.

Autonomous Drone Racing with an Opponent: A First Approach

Drone racing is a popular sport where human pilots control their drones via radio frequency to fly at high speed through complex race tracks. The latter has motivated to pose the question: could an autonomous drone beat a human in a drone race. Thus, some works have dealt with the autonomous drone racing challenge; however, very few works have dealt with the case of a race when an opponent is present in the track. In this work, we present an initial approach to address the problem of autonomous navigation while considering the presence of an opponent in the race track. To address this problem, we present a compact Convolutional Neural Network that predicts flight commands from a set of camera images captured on the fly. This network has been trained by imitation; this is, the network learns from examples generated by a human pilot. It is important to highlight that there is no explicit gate detection or trajectory planning/tracking in our approach. We have carried out experiments in the Gazebo simulator in a racetrack where the autonomous drone will face an opponent on its way to the gate. Our results show that the network manages to pilot the drone to evade the opponent, and after the evasion, the drone gets on-track towards the gate.

AMZ Driverless: The full autonomous racing system

AbstractThis paper presents the algorithms and system architecture of an autonomous racecar. The introduced vehicle is powered by a software stack designed for robustness, reliability, and extensibility. To autonomously race around a previously unknown track, the proposed solution combines state of the art techniques from different fields of robotics. Specifically, perception, estimation, and control are incorporated into one high‐performance autonomous racecar. This complex robotic system, developed by AMZ Driverless and ETH Zürich, finished first overall at each competition we attended: Formula Student Germany 2017, Formula Student Italy 2018 and Formula Student Germany 2018. We discuss the findings and learnings from these competitions and present an experimental evaluation of each module of our solution.

Learning at the Racetrack: Data-Driven Methods to Improve Racing Performance Over Multiple Laps

Autonomous vehicles will generate data from the variety of sensors they employ to track the surrounding environment. This data is inherently valuable, as it gives algorithm designers the potential to leverage prior experience in order to improve driving performance over time. This paper uses the lens of autonomous racing to provide an example of how data from previous iterations of driving can be used to improve quantitative metrics of performance. Two complementary algorithms are demonstrated in this paper. The first algorithm uses iterative learning control (ILC) to simultaneously improve lateral and longitudinal tracking of the desired racing trajectory over multiple laps, while the second algorithm is focused on altering the trajectory itself using a search method. When driven experimentally at the limits of handling, the result is a reduction in lap time of nearly 1.4 seconds, a major improvement.

LiDAR-Based GNSS Denied Localization for Autonomous Racing Cars.

Self driving vehicles promise to bring one of the greatest technological and social revolutions of the next decade for their potential to drastically change human mobility and goods transportation, in particular regarding efficiency and safety. Autonomous racing provides very similar technological issues while allowing for more extreme conditions in a safe human environment. While the software stack driving the racing car consists of several modules, in this paper we focus on the localization problem, which provides as output the estimated pose of the vehicle needed by the planning and control modules. When driving near the friction limits, localization accuracy is critical as small errors can induce large errors in control due to the nonlinearities of the vehicle’s dynamic model. In this paper, we present a localization architecture for a racing car that does not rely on Global Navigation Satellite Systems (GNSS). It consists of two multi-rate Extended Kalman Filters and an extension of a state-of-the-art laser-based Monte Carlo localization approach that exploits some a priori knowledge of the environment and context. We first compare the proposed method with a solution based on a widely employed state-of-the-art implementation, outlining its strengths and limitations within our experimental scenario. The architecture is then tested both in simulation and experimentally on a full-scale autonomous electric racing car during an event of Roborace Season Alpha. The results show its robustness in avoiding the robot kidnapping problem typical of particle filters localization methods, while providing a smooth and high rate pose estimate. The pose error distribution depends on the car velocity, and spans on average from 0.1 m (at 60 km/h) to 1.48 m (at 200 km/h) laterally and from 1.9 m (at 100 km/h) to 4.92 m (at 200 km/h) longitudinally.

Improving the efficiency of neural networks with virtual training data

At Széchenyi István University, an autonomous racing car for the Shell Eco-marathon is being developed. One of the main tasks is to create a neural network which segments the road surface, protective barriers and other components of the racing track. The difficulty with this task is that no suitable dataset for special objects, e.g. protective barriers, exists. Only a dataset limited in terms of its size is available, therefore, computer-generated virtual images from a virtual city environment are used to expand this dataset. In this work, the effect of computer-generated virtual images on the efficiency of different neural network architectures is examined. In the training process, real images and computer-generated virtual images are mixed in several ways. Subsequently, three different neural network architectures for road surfaces and the detection of protective barriers are trained. Past experiences determine how to mix datasets and how they can improve efficiency.

Visual model‐predictive localization for computationally efficient autonomous racing of a 72‐g drone

AbstractDrone racing is becoming a popular e‐sport all over the world, and beating the best human drone race pilots has quickly become a new major challenge for artificial intelligence and robotics. In this paper, we propose a novel sensor fusion method called visual model‐predictive localization (VML). Within a small time window, VML approximates the error between the model prediction position and the visual measurements as a linear function. Once the parameters of the function are estimated by the RANSAC algorithm, this error model can be used to compensate the prediction in the future. In this way, outliers can be handled efficiently and the vision delay can also be compensated efficiently. Theoretical analysis and simulation results show the clear advantage compared with Kalman filtering when dealing with the occasional large outliers and vision delays that occur in fast drone racing. Flight tests are performed on a tiny racing quadrotor named “Trashcan,” which was equipped with a Jevois smart camera for a total of 72 g. An average speed of 2 m/s is achieved while the maximum speed is 2.6 m/s. To the best of our knowledge, this flying platform is currently the smallest autonomous racing drone in the world, while still being one of the fastest autonomous racing drones.

Health-aware control design based on remaining useful life estimation for autonomous racing vehicle

The accurate estimation of the State of Charge (SOC) and an acceptable prediction of the Remaining Useful Life (RUL) of batteries in autonomous vehicles are essential for safe and lifetime optimized operation. The estimation of the expected RUL is quite helpful to reduce maintenance cost, safety hazards, and operational downtime. This paper proposes an innovative health-aware control approach for autonomous racing vehicles to simultaneously control it to the driving limits and to follow the desired path based on maximization of the battery RUL. To deal with the non-linear behavior of the vehicle, a Linear Parameter Varying (LPV) model is developed. Based on this model, a robust controller is designed and synthesized by means of the Linear Matrix Inequality (LMI) approach, where the general objective is to maximize progress on the track subject to win racing and saving energy. The main contribution of the paper consists in preserving the lifetime of battery and optimizing a lap time to achieve the best path of a racing vehicle. The control design is divided into two layers with different time scale, path planner and controller. The first optimization problem is related to the path planner where the objective is to optimize the lap time and to maximize the battery RUL to obtain the best trajectory under the constraints of the circuit. The proposed approach is formulated as an optimal on-line robust LMI based Model Predictive Control (MPC) that steered from Lyapunov stability. The second part is focused on a controller gain synthesis solved by LPV based on Linear Quadratic Regulator (LPV-LQR) problem in LMI formulation with integral action for tracking the trajectory. The proposed approach is evaluated in simulation and results show the effectiveness of the proposed planner for optimizing the lap time and especially for maximizing the battery RUL.

A reproducible educational plan to teach mini autonomous race car programming

As autonomous cars and complex features of them grow in popularity, ensuring that analyses and capabilities are reproducible and repeatable has taken on importance in education plans too. This paper describes a reproducible research plan on mini autonomous race car programming. This educational plan is designed and implemented as part of a summer internship program at Kocaeli University and it consists of theoretical courses and laboratory assignments. A literate programming approach with the Python language is used for programming the race car. To assess the educational program’s impact on the learning process and to evaluate the acceptance and satisfaction level of students, they answered an electronic questionnaire after finishing the program. According to students’ feedback, the reproducible educational program is useful for learning and consolidating new concepts of mini autonomous car programming.

Optimizing Deep-Neural-Network-Driven Autonomous Race Car Using Image Scaling

In this work we propose scaling down the image resolution of an autonomous vehicle and measuring the performance difference using pre-determined metrics. We formulated a testing strategy and provided suitable testing metrics for RC driven autonomous vehicles. Our goal is to measure and prove that scaling down an image will result in faster response time and higher speeds. Our model shows an increase in response rate of the neural models, improving safety and results in the car having higher speeds.

NMPC for Racing Using a Singularity-Free Path-Parametric Model with Obstacle Avoidance

This work presents the real-time control of 1:43 scale autonomous race cars using nonlinear model predictive control based on a singularity-free prediction model. This model allows the car to drive at both low and high speeds and in stop-and-go maneuvers. Additional constraints are imposed in the optimal control problem to ensure the validity of the model assumptions. Moreover, the control scheme is capable of avoiding obstacles online. The experimental results show that the proposed method converges to nearly time-optimal behavior by maximizing the progress on the track and achieves competitive lap time results.

Design and Simulation of a Machine-learning and Model Predictive Control Approach to Autonomous Race Driving for the F1/10 Platform

This paper addresses the challenges of developing an embedded non-linear model predictive control (NMPC) solution for the optimal driving of miniature scale autonomous vehicles (AVs). The NMPC approach lends itself perfectly to driving applications, provided that a system for localization and tracking of the vehicle is available. An important challenge in the implementation results from the need to accurately steer the vehicle at high speeds, which requires fast actuation. In this paper we present a solution to this problem, which employs an artificial neural network (ANN) controller trained with rigorous NMPC input-output data. We discuss the development process, from modelling until the realization of the ANN controller within the operating system of the AV. The procedure is demonstrated within the virtual environment of the popular F1/10 race car, an AV platform widely used in AI and autonomous driving challenges. The results contain both NMPC and ANN-based simulations for different race tracks and for different driving strategies. The main focus of this work lies in the formulation of the optimal driving control problem and the training method of the ANN. Our approach uses a standardization of the driving problem, which enables us to abstractize optimal driving and to simplify it for the learning process. We show how driving patterns can be learned accurately on a reduced set of training data and that they can subsequently be extended to new and more challenging driving situations.

Time-optimal trajectory planning for a race car considering variable tyre-road friction coefficients

This paper shows the planning of time-optimal trajectories, which allows an autonomous race car to drive at the handling limits, taking into account locally changing road friction values. For this purpose, the minimum lap time problem is described as an optimal control problem, converted to a nonlinear programme using direct orthogonal Gauss-Legendre collocation and then solved by the interior-point method IPOPT. Reduced computing times are achieved using a curvilinear abscissa approach for track description, algorithmic differentiation using the software framework CasADi, and a smoothing of the track input data by approximate spline regression. The vehicle's behaviour is approximated as a single track and double track model with quasi-steady state tyre load simplification and nonlinear tyre model. The results are used to evaluate which vehicle physics are important for the calculation of the time-optimal trajectory. The novelty of this work is the consideration of wheel-specific tyre-road friction coefficients along the racetrack using a track friction map. It is shown that variable friction coefficients have a significant impact on the trajectory, and therefore significantly improve lap times on inhomogenous racetracks. The proposed trajectory planning has proven its practical suitability in first tests on an autonomous race car and will be used in the coming racing season in the Roborace competition.

Autonomous racing using Linear Parameter Varying-Model Predictive Control (LPV-MPC)

This article presents an innovative control approach for autonomous racing vehicles. Linear Parameter Varying (LPV) theory is used to model the dynamics of the vehicle and implement an LPV-Model Predictive Controller (LPV-MPC) that can be computed online with reduced computational cost. The optimal time problem is solved by an optimal off-line trajectory planner that calculates the best trajectory under the constraints of the circuit. An identification of the system model based on optimization is also carried out. The planning and control scheme is validated in simulation and experimentally in a real platform where the effectiveness of the proposed LPV-MPC is demonstrated.

LPV-MP planning for autonomous racing vehicles considering obstacles

In this paper, we present an effective online planning solution for autonomous vehicles that aims at improving the computational load while preserving high levels of performance in racing scenarios. The method follows the structure of the model predictive (MP) optimal strategy where the main objective is to maximize the velocity while smoothing the dynamic behavior and fulfilling varying constraints. We focus on reformulating the non-linear original problem into a pseudo-linear problem by convexifying the objective function and reformulating the non-linear vehicle equations to be expressed in a Linear Parameter Varying (LPV) form. In addition, the ability of avoiding obstacles is introduced in a simple way and with reduced computational cost.We test and compare the performance of the proposed strategy against its non-linear approach through simulations. We focus on testing the performance of the trajectory planning approach in a racing scenario. First, the case of free obstacles track and afterwards a scenario including static obstacles. Simulation results show the effectiveness of the proposed strategy by reducing the algorithm elapsed time while finding appropriate trajectories under several input/state constraints.

Autonomous Driving—A Crash Explained in Detail

Since 2017, a research team from the Technical University of Munich has developed a software stack for autonomous driving. The software was used to participate in the Roborace Season Alpha Championship. The championship aims to achieve autonomous race cars competing with different software stacks against each other. In May 2019, during a software test in Modena, Italy, the greatest danger in autonomous driving became reality: A minor change in environmental influences led an extensively tested software to crash into a barrier at speed. Crashes with autonomous vehicles have happened before but a detailed explanation of why software failed and what part of the software was not working correctly is missing in research articles. In this paper we present a general method that can be used to display an autonomous vehicle disengagement to explain in detail what happened. This method is then used to display and explain the crash from Modena. Firstly a brief introduction into the modular software stack that was used in the Modena event, consisting of three individual parts—perception, planning, and control—is given. Furthermore, the circumstances causing the crash are elaborated in detail.By presented and explaining in detail which software part failed and contributed to the crash we can discuss further software improvements. As a result, we present necessary functions that need to be integrated in an autonomous driving software stack to prevent such a vehicle behavior causing a fatal crash. In addition we suggest an enhancement of the current disengagement reports for autonomous driving regarding a detailed explanation of the software part that was causing the disengagement. In the outlook of this paper we present two additional software functions for assessing the tire and control performance of the vehicle to enhance the autonomous.

Learning How to Autonomously Race a Car: A Predictive Control Approach

We present a learning model predictive controller (LMPC) for autonomous racing. We model the autonomous racing problem as a minimum time iterative control task, where an iteration corresponds to a lap. The system trajectory and input sequence of each lap are stored and used to systematically update the controller for the next lap. In the proposed approach, the race time does not increase at each iteration. The first contribution is to propose a local LMPC which reduces the computational burden associated with existing LMPC strategies. In particular, we show how to construct a local safe set and approximation to the value function, using a subset of the stored data. The second contribution is to present a system identification strategy for the autonomous racing iterative control task. We use data from previous iterations and the vehicle’s kinematic equations of motion to build an affine time-varying prediction model. The effectiveness of the proposed strategy is demonstrated by experimental results on the Berkeley Autonomous Race Car (BARC) platform.

Cautious Model Predictive Control Using Gaussian Process Regression

Gaussian process (GP) regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in function estimation. In the context of control, it is seeing increasing use for modeling of nonlinear dynamical systems from data, as it allows the direct assessment of residual model uncertainty. We present a model predictive control (MPC) approach that integrates a nominal system with an additive nonlinear part of the dynamics modeled as a GP. Approximation techniques for propagating the state distribution are reviewed and we describe a principled way of formulating the chance constrained MPC problem, which takes into account residual uncertainties provided by the GP model to enable cautious control. Using additional approximations for efficient computation, we finally demonstrate the approach in a simulation example, as well as in a hardware implementation for autonomous racing of remote controlled race cars, highlighting improvements with regard to both performance and safety over a nominal controller.