The structural, physicochemical, gel textural, rheological, and in vitro digestibility properties as well as their relationships of non-waxy proso millet starch (NPMS) and waxy proso millet starch (WPMS) were evaluated by taking normal corn starch (CS) and potato starch (PS) as controls. Proso millet starch was mostly polygonal or spherical, with an A-type crystalline structure. Proso millet starch contained more short-branched chains (DP 6–24) compared with CS and PS. WPMS possessed higher crystallinity and more short-range ordered structures than NPMS. NPMS displayed higher pasting temperature, retrogradation rate and shear thinning degree, and lower gelatinization temperature and enthalpy than WPMS. The hardness and chewiness of starch gel formed by NPMS were higher than those of WPMS. All starch samples exhibited shear thinning behavior in the steady-flow test and typical elastic solid behavior in the dynamic rheological test. Moreover, NPMS was considered a potential formula for functional foods, with its lower rapidly digestible starch (RDS) and higher resistant starch (RS) contents than WPMS, CS, and PS. This paper revealed the influence of amylose content and structure on the physicochemical properties of different proso millet starch.