Background Cow’s milk protein allergy (CMPA) is the most common type of food allergy. The prevalence is increasing in both developing and developed countries. The clinical presentations are variable in intensity and may involve many different organ systems. The diagnosis depends on positive response to an elimination diet, with recurrence of the symptoms upon re-challenge. CMPA usually resolves during early childhood, but some factors may delay the acquisition of tolerance. Aim The aim was to investigate the possible risk factors that may be associated with delayed tolerance to cow’s milk in patients with CMPA. Patients and methods The study included 95 patients aged 1 month to 3 years with proved CMPA attending the Gastroenterology Clinic at Alexandria University Children Hospital. Cases were recruited for 6 months, starting from January 1, 2018, to the end of June 2018. The demographic data, age at onset, clinical presentations, nutritional history, and family history of atopy were reported. The diagnosis was confirmed through elimination-re-challenge trial. All patients were treated by a milk-free diet with the use of appropriate milk substitutes according to the age. After a 6–9-month period of elimination, gradual re-introduction of milk was attempted, and the development of tolerance to milk was evaluated. Factors associated with delayed tolerance were compared in both the tolerant and intolerant groups. Results At their initial visit, the mean age of the studied patients was 8.5±4.9 months, and 17.9% were underweight. The mean age at onset of symptoms was 2.2±2.2 months, whereas the mean duration before diagnosis was 3.3±2.8 months. Overall, 85.3% were born via cesarean section. Chronic or delayed symptoms were reported in 93.7%. All patients had gastrointestinal symptoms. Chronic diarrhea was the commonest symptoms (90.53%) followed by vomiting (42%). In addition, 28.4% had cutaneous symptoms. Moreover, 81% were formula fed. Food protein-induced proctocolitis was the most common syndrome among patients with chronic symptoms. Overall, 85.26% of them were born via cesarean section. Early introduction of dairy products was encountered in 37.9%. In addition, 10% of the patients had cross-reaction to soy and 14.5% had multiple food allergy (MFA). Additionally, 56.8% had a family history of atopy. Overall, 68.42% tolerated cow milk after gradual re-introduction. Delayed diagnosis of CMPA symptoms, delayed introduction of complementary food, rural residence, family history of atopy, cross-reactivity to soya or MFA were associated with delayed tolerance to cow’s milk. Patients with immediate symptoms [immunoglobulin (Ig)E mediated] or food protein-induced enterocolitis syndrome or those with CMPA-associated eosinophilic gastrointestinal disorders showed delayed tolerance. Tolerance was achieved more frequently in patients with delayed (non-IgE mediated) symptoms, in patients with exclusive gastrointestinal symptoms, and in patients with food protein-induced proctocolitis. Conclusion The tolerance rate after 6–9 months of elimination was 63.2%, which is comparable to the international rates. IgE-mediated CMPA, food protein-induced enterocolitis syndrome, CMPA-associated eosinophilic gastrointestinal disorders, and MFA have delayed tolerance.