Direct interference between the orthogonal electric and magnetic modes in a hybrid silicon-gold nanocavity is demonstrated to induce a pronounced asymmetric magnetic-based Fano resonance in the total scattering spectrum at near-infrared frequencies. Differing from the previously reported magnetic-based Fano resonances in metal nanoparticle clusters, the narrow discrete mode provided by the silicon magnetic dipole resonance can be directly excited by external illumination, and greatly enhanced electric and magnetic fields are simultaneously obtained at the Fano dip.