The structure of starch often influences its interaction mode and affinity with polyphenols, subsequently impacting the functional properties of starch-polyphenol complexes. This study delved into the multiscale structure, physicochemical properties, and digestive performance of complexes formed by tannic acid (TA) with starches of different polymorphisms (corn starch (CS), potato starch (PS), and pea starch (ES)). The results revealed that TA induced notable changes in the surface morphology and aggregation behavior of starch pastes, with the most pronounced effects observed in ES. Elevated gelatinization enthalpy change and decreased thermal stability were observed in ES-TA complexes, contrary to complexes formed with CS and PS. TA enhanced the crystallinity of three pre-gelatinized starches, facilitating the formation of long-range ordered structures but disrupting short-range ordered structures, with effects more pronounced and TA-concentration-dependent in CS and ES rather than PS. The viscoelasticity of PS gel was strengthened, while that of CS and ES gels was reduced by TA. Digestive outcomes demonstrated that TA increased the resistance to digestion in all three starches, with CS exhibiting the most significant effect, followed by ES and PS. This might be linked to the binding modes of TA with different crystalline starches. The observed variations in regulatory effects on structure and function of starches from different sources combined with TA enhanced our understanding of TA-starch interactions, providing insights into the interaction between polyphenols and starch of different crystalline types.