In this paper, we investigate the performance of a full-duplex (FD) relay system where multi-antennas are exploited at source and destination. Unlike previous works, the impacts of imperfect channel state information (I-CSI), transceiver hardware noise (THN), and residual self-interference (RSI) are taken into account. We mathematically derive the exact closed-form expressions of the outage probability (OP), symbol error rate (SER), and ergodic capacity (EC) of the FD relay system with I-CSI, THN, and RSI over Nakagami- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> fading channels. From the derived expressions, the performance of the considered system under the effects of three negative factors (I-CSI, THN, and RSI) is compared with that system in the case of all ideal factors (perfect channel state information (P-CSI), perfect transceiver hardware (P-TH) and perfect self-interference cancellation (P-SIC)), two ideal factors (P-CSI and P-TH, P-CSI and P-SIC, P-TH and P-SIC), or one ideal factor (P-CSI or P-TH or P-SIC). Numerical results show a strong impact of three negative factors on the OP, SER, and EC of the considered FD relay system, especially when the data transmission rate of the system and signal-to-noise ratio (SNR) are high. In particular, OP, SER, and EC go to the floors in the high SNR regime due to the three negative factors. Therefore, when I-CSI, THNs, and RSI exist in the FD relay system, we should use suitable source and relay transmission power to obtain excellent performance while saving energy consumption. Moreover, when two of the three negative factors are large enough, the remaining factor’s impact becomes weaker and may be neglected in certain circumstances.