This work investigates the thermal expansion behaviors of Fe30Co30Ni30Cr10-xMnx High Entropy Alloys (HEAs) (x = 0, 5, 10 at.%) from 400 K to 1200 K. Interestingly, comparing to that of Cantor alloy, a decrease in Cr and Mn in Co–Cr–Fe–Mn–Ni system could significantly decrease the thermal expansion coefficient by 45.2%; Fe30Co30Ni30Cr10-xMnx (x = 0, 5, 10) also showed an abrupt change in thermal expansion behaviors similar to that of Invar alloys. Experimental and theoretical analysis suggest the abrupt change in thermal expansion behaviors of these HEAs were associated with the transition from ferromagnetism to paramagnetism, and these alloys appear to exhibit the dimensional stability of the Invar effect. Since the Invar effect is related to ferromagnetic properties, the amount of Cr and Mn in the HEAs would influence the suppression of thermal expansion due to different intensity of anti-ferromagnetic coupling effect. This research contributes to the understanding of the thermal expansion behaviors of Co–Cr–Fe–Mn–Ni high entropy alloys and the effects of antiferromagnetic elements.