Substantial damage may occur when a rotary actuator fails during operation. Therefore, effective fault diagnosis of a rotary actuator is crucial to ensuring the safety of the device. However, only a few studies on fault detection, fault isolation, and performance assessment have focused on rotary actuators. In this study, fault detection and fault isolation processes were implemented by designing two observers based on a neural network, and a method that assesses the performance of the rotary actuator is proposed. First, two observers are established according to the structure of the rotary actuator. Data in their normal state are used to train the neural networks. Second, a radial basis function (RBF) neural network is employed to estimate the expected output of the system to generate residuals, and self-adaptive thresholds are obtained through another RBF neural network in each observer. The information on the observers is applied for fault isolation. Third, the residual is input into the self-organizing mapping neural network trained by the residual values in their normal state to normalize the performance of the rotary actuator into confidence values between 0 and 1. Finally, the detection and assessment of two typical faults in a rotary actuator were simulated. The results demonstrate that the proposed method is able to assess the performance of rotary actuator and detect faults suitably.