This paper investigates the leader-following formation control problem of nonholonomic agents with a persistently exciting leader based on bearing measurements. A follower needs to maintain a desired relative position to the leader in its local coordinate frame using bearing measurements. An integrated method consisting of a bearing-based estimator and a tracking controller is proposed to achieve the leader-following control objective. The estimator is designed to estimate the relative position to the leader in the follower's local coordinate frame. Based on the estimated relative position, the controller is proposed to drive the follower to achieve the desired formation. Different from existing integrated methods, our method does not require any conditions on the relative bearing between the leader and the follower in the global frame, which are difficult to guarantee. We characterize the conditions locally on the leader only, namely, conditions on the leader's linear and angular velocities with persistently exciting steering speed, such that the overall system is asymptotically stable with our integrated method. Besides, the proposed method is extended to the multi-agent formation control problem. A reasonable solution is proposed to the multi-agent case with a minimal leader follower structure. Simulations prove the effectiveness of the proposed methods in both cases.