The product quality problem of high Al steel is closely related to the agglomeration behavior of inclusions. The present study firstly established the stability diagram of inclusion in the Fe–Al–O–N system, thereby evaluating the formation behavior and control condition of various inclusions can be obtained. Based on that, laboratory-scale experiments with different Al contents were designed to obtain the inclusion evolution process. Furthermore, the calculated agglomeration forces were used to explain the difference of evolution of various inclusions. The cavity bridge force increases with the increase of diameter of inclusions and decrease of distance between inclusions, and the van deer Waals force decreases with the increase of diameter and distance. Due to the cavity bridge force is larger than van deer Waals force, the inclusions firstly approached each other by van deer Waals force and then agglomerated by cavity bridge force. In addition, the cavity bridge force of Al2O3 is larger than that of AlN at the same condition because of the larger contact angle. However, the increase of Al content makes this phenomenon converse. When the Al content is > 3 wt. %, the cavity bridge force of AlN is larger than that of Al2O3. When the Al content is > 7 wt. %, the cavity bridge force of Al2O3 disappears. The study can provide theoretical support for inclusion control in high Al steel.