This letter studies the trajectory tracking problem for an autonomous bicycle that is a non-minimum phase, strongly nonlinear system. As compared with most existing methods dealing only with approximate trajectory tracking, this letter proposes a constructive design to achieve asymptotic trajectory tracking. More specifically, under the assumption that the desired trajectory is generated by a virtual bicycle, a novel asymptotic trajectory tracking controller design scheme is presented. Firstly, the nonlinear dynamics of the autonomous bicycle is established. Secondly, it is decomposed into two interconnected subsystems: a tracking subsystem and a balancing subsystem. For the tracking subsystem, the popular backstepping approach is applied to determine the propulsive force of the bicycle. For the balancing subsystem, optimal control is applied to determine the steering angular velocity of the handlebar in order to balance the bicycle and align the bicycle with the desired yaw angle. Thirdly, to tackle the strong coupling between the tracking and the balancing systems, the small-gain technique is applied for the first time to prove the asymptotic stability of the closed-loop bicycle system. Finally, the efficacy of the proposed exact trajectory tracking control methodology is validated by numerical simulations.