The Al–Si–Fe system has drawn the attention of the scientific community due to its capacity to replace parts in several manufacturing industries, as this alloy system is very sensitive to small additions of transition metals. Therefore, the aim of this work is to study the effect of Cr, Ti, and Mn additions in the Al–20Si–5Fe (wt. %) alloy and to study the modification of the iron intermetallic and the microstructural refinement through the formation of secondary phases. Al–20Si–5Fe–X (X = Cr, Mn and Ti at 1.0, 3.0, and 5.0 wt. %) alloy ingots were prepared by arc melting furnace. The elemental chemical analysis was performed by X-ray fluorescence spectrometry (XRF). The microstructure of all samples was investigated by scanning electron microscopy and X-ray diffraction. Finally, microhardness was measured in order correlate the hardness with the formation of the different compounds. The highest hardness was found for the alloy with the 5 wt. % Cr. The addition of Ti and Mn raised the hardness by ~35 HVN (Vickers microhardness) when compared to that of AlSiFe master alloy. Important changes were also observed in the microstructure. Depending on the Cr, Ti, and Mn additions, the resulting microstructure was dendritic (CrFe), acicular (Ti5Si3), and “bone like” (Mn0.2Fe0.8), respectively.