Frequency-domain-independent component analysis (F-ICA) algorithm is proposed as demultiplexing method for mode-division multiplexing (MDM) system. We simulate an MDM system and testify the feasibility of F-ICA as demultiplexing algorithm. In addition, the symbol error rate (SER) performance of F-ICA and frequency-domain least mean square algorithm is compared. Simulation results show that the proposed algorithm can implement mode demultiplexing successfully. Meanwhile, it has a better SER performance under the same optical signal-to-noise ratio and coupling factor. We also conduct theoretical analysis of the two algorithms' computational complexity. It is demonstrated that with F-ICA, complexity of DSP is effectively reduced.