CoCrFeNi high entropy alloys (HEAs) with yttrium (Y) additions (1 and 4 at. %) were nanostructured by mechanical alloying process and annealed at various temperatures between 500 °C and 1100 °C. The structure, grain growth, and hardness were studied as a function of solute addition and annealing temperature using X-ray diffraction (XRD), focused ion beam (FIB), and scanning transmission electron microscope (S/TEM) techniques, and hardness test. The thermo-physical calculations were utilized to discuss the phase evolution after mechanical alloying and annealing with respect to added solutes. The results showed that Y additions did not affect the main crystal structure of the base CoCrFeNi HEA as the solid solution with a single face-centered cubic (fcc) crystal structure was maintained even after 1 h annealing at 1100 °C. The as-milled nanocrystalline grain size of CoCrFeNi HEA yielded extensive grain growth with the temperature exposures reaching 291 nm and 1.4 μm after annealing at 900 °C and 1100 °C, respectively. However, Y additions retarded the grain growth and decreased the average grain size upon annealing as compared to the base HEA. That is, 1 and 4 at. % Y additions stabilized the grain size around 88 nm and 95 nm (both determined by TEM) after annealing at 900 °C and 1100 °C, respectively. Accordingly, the as-milled hardness of CoCrFeNi HEA dropped from 475 HV to 220 HV after annealing at 1100 °C, while the reduction in hardness was relatively gradual with Y additions and retained around 435 HV with 4 at. % Y addition even after annealing at 1100 °C. Such thermal stability may facilitate the use of HEAs at high temperatures and enable the consolidation routes of powders into dense nanocrystalline compact HEAs.