Magnetoelectric (ME) sensors have enormous potential for detecting weak magnetic fields because of their high sensitivity, low power consumption, compact size and, low cost. However, inevitable vibration interference limits their application in practical environments, especially in the case of mobile platform mounting. Here, we propose a differential ME sensor, consisting of PZT macro-fiber composites (MFCs) and Metglas laminates. The differential ME sensor has two output terminals with weak mutual mechanical coupling and works in longitudinal vibration mode. MFC cores are polarized in parallel mode to guarantee their consistency of electric characteristics and reversed bias field is provided by attached magnets. Experimental results show that the differential-mode response amplitudes have a gain of −17.6 dB for low-frequency vibration at 2 Hz and ∼6.2 dB for an applied magnetic field at 3 Hz, in comparison with the single-ended mode. In addition, our proposed ME sensor also has a low inherent equivalent magnetic noise of 18.3 pT/√Hz at 1 Hz. Finally, a target detection experiment in the presence of heavy lab noise and strong vibration interference is conducted and the improved detection performance of the proposed differential ME sensor is proved.