PurposeMagnetic fluid has excellent function used as lubricants in bearings and mechanical seals, and the purpose of this study is to investigate the sealing performance in a spiral groove mechanical seal lubricated by magnetic fluid.Design/methodology/approachThe sealing characteristic parameters of the lubricating film between the end faces of two sealing rings were calculated based on the Muijderman narrow groove theory for a spiral groove mechanical seal lubricated by magnetic fluid. The film thickness was determined according to the balanced forces on the rotating ring, and the effects of operating conditions, intensity of the magnetic field and diameter of nanoparticles on the sealing characteristics were investigated.FindingsIt has been found that the intensity of magnetic field has a great effect on the viscosity of magnetic fluid, film thickness and friction torque while has a little effect on the mass flux of magnetic fluid. The film thickness, mass flux of magnetic fluid and friction torque increase with the increasing volume fraction, rotating speed and diameter of magnetic nanoparticles in magnetic fluid. The mass flux of magnetic fluid decrease with the increasing closing force, and the friction torque decreases with the increase of media pressure.Originality/valueThe change of intensity of magnetic field can affect the viscosity of magnetic fluid and then changes the sealing performance in a mechanical seal lubricated by magnetic fluid. To reduce the mass flux of magnetic fluid and friction torque, the volume fraction, diameter of solid magnetic particles and film thickness should be 5%–7%, 8–10 nm and 2–9.3 µm, respectively.Peer reviewThe peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-02-2023-0032/