Safety-critical control is crucial but difficult in the applications of autonomous underwater vehicles (AUVs). This article proposes a novel hierarchical safety-critical framework for the control of AUVs consisting of waypoint-based optimal trajectory generation and tracking. The objective is to generate a smooth and feasible trajectory to conform to the dynamics of the AUV and then control the AUV to operate in a safe region. To this end, the offline trajectory generator and the online controller are designed. In the offline step, an improved minimum snap polynomial trajectory is generated as the tracking reference. Specifically, the trajectory is optimized by explicitly considering the practical constraints of the AUV's actuator, which alleviates the complexity of the controller design and computational burden compared with the real-time nonlinear optimization. In the online step, the time- and state-dependent high-order control barrier functions are incorporated into optimization through quadratic programming (QP) that modifies the backstepping-based nominal controller in a minimally invasive way. As a supplement and adjustment to the offline planning, the online step ensures that the system states are maintained in the safety set, thus ensuring obstacle avoidance. The online computational efficient QP structure guarantees convenience and scalability in practical implementation. Both the online static and dynamic obstacle avoidance simulations demonstrate the adherence to the safety constraints. Experimental results validate the effectiveness of the proposed method.