The unique characteristics, such as high magnetic moment, high Curie temperature, large magnetic crystalline anisotropy, and low cost, make the τ-phase MnAl a promising candidate as the market alternatives which could fill the gap between the rare earth magnets and ferrite magnets. Because at low temperatures the metal will become more brittle, the cryomilling technology may allow the τ-phase MnAl to be broken quickly and introduce few crystal defects, and then obtain better permanent magnetic properties. In this study, the cryomilling technology was used to grind the τ-phase MnAl, and the structural and magnetic properties of the obtained powder sample were investigated. It was found that compared to room-temperature ball milling, the morphology of cryomilled powders is granular and the powder agglomeration is dramatically suppressed at a low temperature, and as a result the larger particles tend to be broken down into smaller ones rather than being agglomerated to larger thin layers. The grain size D reduced continuously from 145 nm to 36 nm when the sample was milled for 120 minutes. The coercivity increased with increasing the cryo-milling time, while the saturation magnetization decreased. The maximum coercivity of up to 4.9 kOe was obtained by cryo-milling for 100 minutes. By the neutron diffraction analysis, it was confirmed that the decrease of the saturation magnetization with increasing milling time is mainly due to the migration of Mn atoms from 1a site to the 1d site and the decrease of Mn atomic magnetic moment.