We experimentally obtain a lower frequency shift based on mode conversion by using a mode-selective coupler (MSC) and an acoustically induced fiber grating (AIFG) for optical heterodyne micro-vibration measurement. Through efficient conversion between LP11 and LP01 core modes in a few-mode fiber, a mode conversion frequency shifter (MCFS) can be achieved and the magnitude of frequency shift is several hundred kHz, which is generally challenging for traditional bulk acousto-optic frequency shifter. Moreover, the signal-to-signal beating interference (SSBI) can be eliminated by using the MCFS without requirement of guard band. The scheme of optical heterodyne micro-vibration measurement based on the MCFS has a high-quality signal-to-noise ratio (SNR) over 85 dB. An all-fiber Bragg grating (FBG) scheme is also designed to improve the measurement sensitivity of heterodyne detection, which has a measurable vibration frequency range of 1 Hz to 300 kHz and minimal detectable amplitude of 0.019 nm. The all-fiber MCFS has the advantages of compactness, easy integration and can be applied directly to satellite ground station and laser feedback tomography.