In this paper, the attitude tracking control problem is investigated for rigid spacecraft subjected to external disturbance, inertia uncertainties, and actuator faults. The rotation matrix is used to model the spacecraft attitude control system which can get rid of the unwinding problem. A nonlinear observer is developed to achieve a high accuracy estimation for the lumped disturbance in fixedtime without the awareness of the boundary of disturbance. To restrain measurement noise, a tracking differentiator is used in the observer to construct signals instead of using the polluted measurement values. Based on the estimation provided by the disturbance observer, a fixed-time attitude tracking controller is proposed by backstepping technique to stabilize attitude tracking errors in fixed time regardless of initial values. The singularity problem in traditional fixed-time controllers based on signal function is eliminated with the help of the arctangent function in the proposed controller. The feature of the designed fixed-time attitude controller is that it can suppress actuator faults, measurement noises and has no unwinding problem. Theoretical analysis based on Lyapunov theory and numerical simulations are presented to illustrate the robustness and effectiveness of the proposed control strategy.