Starch-lipid-protein complexes are attracting increasing attention due to their unique structure and low enzymatic digestibility. However, the mechanisms underlying the formation of these ternary complexes, especially those with monoglycerides as the lipid component, remain unclear. In the present study, potato starch or octenyl succinic anhydride (OSA)-modified potato starch (OSAPS), various monoglycerides (MGs), and beta-lactoglobulin (βLG) were used in model systems to characterize the formation, structure, and in vitro digestibility of the respective ternary complexes. Colorimetry and live/dead staining assays demonstrated that the OSAPS had good biocompatibility. Experimental data and molecular dynamics simulations showed that both unmodified potato starch and OSAPS formed starch-lipid-protein complexes with MGs and βLG. Of the two types of starch, OSA formed a greater amount of the more stable type II V-crystallites in complexes, which had greater resistance to in vitro enzymic digestion. This study demonstrated for the first time that starch can interact with MGs and βLG to form ternary complexes and that OSA esterification of starch promoted the formation of more complexes than unmodified starch.