Grain growth is an unavoidable phenomenon occurring during processing of multicomponent high-entropy alloys (HEAs) and medium-entropy alloys (MEAs), which significantly affects the properties and performance of these compositionally complex alloys during service. Accordingly, the present overview article is dedicated to summarize the state of the art of the grain coarsening behavior and its kinetics for HEAs and propose opportunities and scope for future research, with special attention to the Cantor-based compositions. Firstly, the primary recrystallization and grain growth of equiatomic and non-equiatomic CrMnFeCoNi alloys with face-centered cubic (FCC) crystal structure in the single-phase regime and with the presence of intermetallics such as the Cr-rich sigma (σ) phase is considered, which was accompanied by the powerful utilization of the solute drag effect, Zener pinning, and grain boundary segregation engineering via introducing carbon, nitrogen, boron, titanium, and niobium. Subsequently, the quaternary and ternary alloys based on Cr, Mn, Fe, Ni, and Co are taken into account. In these systems, the novel concept of grain boundary high-entropy effect (via the addition of molybdenum, niobium, and zirconium), the formation of pinning insoluble oxide and thermally stable carbide particles during mechanical alloying, the simultaneous solute drag and Zener pinning effects via Mo and C addition, and the formation of Cu-rich phase and the strong segregation effect of Cu atoms to grain boundaries in the FeCoNiCu HEA are critically discussed. Afterward, the thermal stability of AlxFeCoCrNi(Mn) HEAs is treated, where the formation of the Ni–Al-rich B2 phase and the corresponding Zener pinning effect, the influence of aluminum addition on the temperature range for the stability of the σ phase, and cluster drag mechanism of Al atoms are explained. What is more, the grain growth of refractory HEAs with body-centered cubic (BCC) crystal structure is also summarized, where the high grain growth activation energy is attributed to the presence of elements with very low self-diffusivity such as tantalum and niobium. Moreover, the possible effects of short-range ordering on the mobility of grain boundaries are discussed. Finally, the prospects for future research work are presented, including the development of precipitation-temperature-time (PTT) diagrams, systematic investigation of cold rolling and annealing for prior grain refinement, studying abnormal grain growth (secondary recrystallization), and assessment of grain growth behavior of HEAs manufactured by advanced methods such as additive manufacturing (3D printing).