There appears a steep increase in the prevalence of food allergy worldwide in the past few decades. It is believed that, rather than genetic factors, the recently altered dietary and environmental factors are the driving forces behind the rapid increase of this disease. Accumulating evidence has implied that external exposures that occurred in prenatal and postnatal periods could affect the development of oral tolerance in later life. Understanding the potential risk factors for food allergy would greatly benefit the progress of intervention and therapy. In this review, we present updated knowledge on the dietary and environmental risk factors in early life that have been shown to impact the development of food allergy. These predominantly include dietary habits, microbial exposures, allergen exposure routes, environmental pollutants, and so on. The key evidence, conflicts, and potential research topics of each theory are discussed, and associated interventional strategies to prevent the disease development and ameliorate treatment burden are included. Accumulating evidence has supported the causative role of certain dietary and environmental factors in the establishment of oral tolerance in early life, especially the time of introducing allergenic foods, skin barrier function, and microbial exposures. In addition to certain immunomodulatory factors, increasing interest is raised toward modern dietary patterns, where adequately powered studies are required to identify contributions of those modifiable risk factors. This review broadens our understanding of the connections between diet, environment, and early-life immunity, thus benefiting the progress of intervention and therapy of food allergy.