As the bottom layer of the autonomous vehicle, path tracking control is a crucial element that provides accurate control command to the X-by-wire chassis and guarantees the vehicle safety. To overcome the deterioration of control performance for autonomous vehicle path-tracking controllers caused by modeling errors and parameter perturbation, an adaptive robust control framework is proposed in this paper. Firstly, the 2-DOF vehicle dynamic model is established and the non-singular fast terminal sliding mode control algorithm is adopted to formulate the control law. The unmeasured model disturbance and parameter perturbation is regarded as the system uncertainty. To enhance the control accuracy, the radial basis forward neural network is introduced to estimate such uncertainty in real time. Then, the dynamic model of an active front steering system is established. The model reference control algorithm is applied for the steering torque control considering model uncertainty brought by the dissipation of manufacturing and mechanical wear. Finally, the Simulink–CarSim co-simulation platform is used and the proposed control framework is validated in two test scenarios. The simulation results demonstrate the proposed adaptive robust control algorithm has satisfactory control performance and good robustness against the system uncertainty.