The effectiveness of Autonomous Underwater Vehicles (AUVs) in diverse underwater tasks is heavily reliant on their ability to perform accurate trajectory tracking. However, due to uncertainties in AUVs modeling and the complex underwater environment disturbances, designing effective trajectory tracking controllers and disturbance observers for AUVs is still a major challenge. To address these uncertainties and enable faster convergence of tracking errors, a trajectory-tracking controller based on fixed-time sliding mode control (FTSMC) and a Radial Basis function neural network (RBFNN) observer are used in this paper. In most cases, the AUV platform carries limited computational resources. In most cases, AUV platforms carry limited computational resources, which restricts the practical use of online neural network methods, and it is particularly important to reduce the complexity of computational neural networks and enhance the real-time performance of the observer. Therefore, we adopted a fast online weight update strategy based on a single parameter. Considering actuator faults and input saturation, passive fault-tolerant control (PFTC) is used in this scheme to further reduce the computational burden. Furthermore, the Lyapunov method is used to demonstrate the fixed-time stability of the individual signals of the system. Finally, simulation results and theoretical analysis demonstrate the superiority and effectiveness of the proposed method.