In-band full duplex (IBFD) allows simultaneous communication in the same frequency band, thus significantly improving spectral efficiency. However, self-interference (SI) at the local transmitter affects decoding at the receiver, and SI intensity increases when the channel is time-varying. Existing techniques concentrate on dealing with digital self-interference (DSI) in static channels, which are difficult to adapt to time-varying environments. For this purpose, firstly, the recursive least squares (RLS) method is utilized to counteract linear DSI. Secondly, a new feedforward neural network based on the multi-head self-attention mechanism (MHSAM-FFNN) is designed to eliminate dynamically changing nonlinear DSI. RLS can track signals and update linear coefficients. The self-attention mechanism computes the attention between any two signals in a time-varying sequence and aggregates the temporal relations by the obtained attention weights. MHSAM-FFNN utilizes multiple self-attention modules in parallel, which enhances the acquisition of different effective information in the time-varying sequence to better learn the feature relationships between the current signal and other signals. It ultimately reconstructing the nonlinearity and memory effects of DSI. As tested in different time-varying environments, results show that this paper's scheme successfully reduces DSI to the noise floor. In addition, this paper considers two scenarios where both the power amplifier (PA) and the low noise amplifier (LNA) are nonlinear and where PA is nonlinear while LNA is linear. Results demonstrate that our scheme can achieve higher interference cancellation in different nonlinear scenarios.