Vitamin A deficiency (VAD) has serious public health consequences including morbidity and mortality for populations in low and middle-income countries (LMICs), especially for children under 5 years and pregnant women. LMICs are at greater risk of VAD, in part due to low levels of consumption of vitamin A-rich foods most of which are plant-based, such as orange-fleshed sweet potatoes (OFSP), with lower bioavailability than animal sources of the vitamin A. Food-based approaches such as biofortification of OFSP, including promoting the consumption of vitamin A-rich biofortified staple crops, has been shown to be potentially effective in improving the status of vitamin A and other micronutrients. This study examined vitamin A-rich food consumption and its predictors among women of reproductive age from OFSP-growing households in two regions of Uganda. A cross-sectional survey was conducted among 617 OFSP growing households, focusing on women in the reproductive age group from the northern and eastern regions of Uganda. Households were not receiving any VAD-related intervention at the time of the survey. Quantitative data included vitamin A-rich food consumption, knowledge on vitamin A, and rich food sources dietary intake, using a 7-day food frequency questionnaire. Vitamin A consumption and risk of deficiency were estimated using the Hellen Keller International guide. The majority of women in this study were either pregnant (80%) or lactating (17%). More than 70% of the study population had a weighted vitamin A rich food consumption mean score of <6 days per week, indicating a high risk of VAD. Knowledge about vitamin A [b (SE) = -0.18 (0.50), p < 0.001] was significantly and inversely associated with vitamin A rich food consumption. Components of food insecurity such as availability, affordability, utilization, and changing food preferences may contribute to the unexpected inverse relationship between knowledge and consumption of vitamin A rich foods. Scaling up biofortified food initiatives, including OFSP, can improve consumption of vitamin A rich foods with effective strategies to comprehensively address consumption barriers such as lack of nutrition education, cooking skills, and storage facilities, as well as low production levels and perceived contamination of biofortified foods.