Ultrafine gangue minerals of calcite and chlorite seriously affect the flotation of valuable mineral, and its inhibition is difficult to achieve using conventional depressants. This study finds the Al-starch colloidal depressants can inhibit the flotation of ultrafine calcite and chlorite, which significantly increases the grade of valuable minerals in concentrates of actual ores. When the chemisorption of Al-starch is occurred on mineral surface, the zeta potential of calcite decreases whilst that of chlorite increases. With the treatment of Al-starch, the initial functional group peaks of Fourier transform infrared spectroscopy measurement on the surface of calcite and chlorite are shifted, and the new O-Al functional group is generated. Meanwhile, the X-ray photoelectron spectroscopy measurement peaks of O1 s atom is changed, and the Al2p atom peak is generated. Furthermore, the optimal structure of Al-starch is O1—Al—O6, and the optimal adsorptive configuration on calcite (104) surface, hydroxyl-terminated and silica-terminated chlorite (001) surface are (Og1 Og2)—Al—(Om1 Om2), (Og1 Og2)—Al—(Oh1 Oh2 Oh3) and Og2—Al—(Om1 Om2 Om3 Og4) with adsorptive energy of − 678.50 kJ/mol, − 353.49 kJ/mol and − 475.05 kJ/mol, respectively. During the adsorption, the 3 s and 3p orbitals of Al on Al-starch accept the 2 s and 2p electrons of O atom on mineral surface.