High purity AlN fiber is a promising thermal conductive material. In this work, AlN fibers were prepared using solution blow spinning followed by nitridation under N2 or NH3 atmosphere. Soluble polymer, such as polyaluminoxane, and allyl-functional novolac resin were adopted as raw materials to form homogeneous distribution of Al2O3 and C nanoparticles within the fibers, which could inhibit the growth of alumina crystal and promote their nitridation process. The effect of nitriding atmosphere on the fiber morphology was investigated. XRD results showed that complete nitridation was achieved at 1300 °C in the NH3 or at 1500 °C in the N2 atmosphere. Hollowed fiber structure was observed when fiber was nitrided in N2 at high temperature, which was caused by gaseous Al gas diffusion, and this phenomenon was eliminated in NH3 atmosphere. The nitridation mechanisms in different atmosphere were analyzed in detail. It was demonstrated that the nitridation of Al2O3 fibers in the NH3 atmosphere offered the favored AlN morphology and chemical quality. Flexible AlN fiber with O content of 0.7 wt% was achieved after nitriding in NH3 at 1400 °C. The high quality AlN can be used in thermal conductive composite materials.