When vibrations generated by marine machinery propagate through a ship’s hull into the ocean, they produce low-frequency radiated noise with distinct “acoustic fingerprint” characteristics. This noise, characterized by stable and concentrated energy, long transmission distances, and difficulty in elimination, becomes the primary target for enemy sonar detection. Active vibration isolation serves as a critical method for reducing low-frequency vibrations in ships and enhancing their acoustic stealth performance. However, control challenges persist, including multi-frequency excitation, frequency fluctuation, multi-channel coupling, and slow convergence speed. To address these issues, this paper introduced an innovative multi-channel decentralized decoupling filtered-x least mean square (DMFxLMS) algorithm. Firstly, a recursive least squares identification algorithm with a forgetting factor was proposed, taking into account the characteristics of single-input, multi-output and multi-input, and multi-output control systems, effectively enhancing the algorithm’s convergence speed and control accuracy. Secondly, based on the decentralized decoupling control concept, the multi-channel control system was simplified into parallel single-channel control loops. The control weight coefficient updates were only related to adjacent error signals, significantly reducing the algorithm’s computational complexity. Thirdly, an anti-impact link was designed to improve the algorithm’s robustness, considering the interference caused by other mechanical equipment during the control process. The influence of abnormal error signals in the control weight coefficient correction term was suppressed, and a percentage function was introduced to limit the output signal. Finally, the feasibility and effectiveness of the DMFxLMS algorithm were verified through simulations and experiments. The results demonstrated that the DMFxLMS algorithm achieved significant control effects for both constant frequency line spectrum excitation and frequency fluctuating line spectrum excitation, fulfilling the objective of reducing base vibration. The DMFxLMS algorithm exhibited fast convergence and excellent robustness, making it suitable for practical engineering applications.