The microstructure and magnetic properties of Co-based metallic microfibers before and after Joule annealing were comprehensively investigated to enhance the magnetoimpedance (MI) effect of metallic microfibers and facilitate the development of magnetic sensor applications. In particular, high-resolution transmission electron microscopy was used to investigate the microstructural changes induced by Joule annealing and to, further elucidate the associated mechanism. The experimental results show that the microfibers have a smooth and homogeneous surface, with no apparent macro/micro defects, as well as high glass-forming ability and thermal stability. Additionally, the 90 mA-annealed microfibers exhibit good magnetic properties, and their M s , M r , H c , and μ m are 81.40 emu/g, 15.01 emu/g, 33.63 Oe, and 0.1471 nm, respectively. The 90 mA-annealed microfibers also exhibit excellent MI performance. Accordingly, the [Δ Z/Z max ] max and ξ max are 213.08% and 41.76%/Oe as f = 1 MHz, respectively. Therefore, the Joule annealing treatment can significantly improve the MI properties of Co-based metallic microfibers, providing technical support for the development of sensitive materials for high-performance MI sensors. The transient temperature rising characteristic of Nb-doped Co-based metallic microfibers during Joule annealing can be numerically calculated based on a non-stationary heat conduction model, further to obtain the optimum parameters of practical annealing process. Accordingly, the MI properties of metallic microfibers were enhanced especially at the current intensity of 90 mA. Furthermore, Joule annealing could effectively eliminate the residual inner stress and improve the atomic short-range ordering arrangement, and the current could generate a stably toroidal magnetic field, which further regulates the toroidal distribution of magnetic domain structure, thereby improves the circumferential permeability, magnetic anisotropy field and MI properties.