Organic agriculture (OA) is often regarded as a sustainable agricultural pathway for smallholder farmers in Sub-Saharan Africa, and an increasing number of initiatives promoting OA were initiated over the last decades. However, holistic empirical evidence on the effects of such initiatives on the sustainability of smallholder farmers is still scanty. We analyzed the effects of five initiatives promoting OA on farm-level sustainability. We selected farmers exposed to the initiatives (n = 678) and control farms (n = 957) in five different case studies, two implemented in Ghana and three in Kenya. We used a farm-level multi-criteria assessment tool that evaluates to what extent the environmental, social, economic, and governance sustainability goals formulated in the FAO-SAFA Guidelines are addressed by farmers. We found that the initiatives had limited effects on reducing farmers reliance on chemical inputs use (pesticides and synthetic fertilizers) and uptake of organic or agro-ecological practices. Nevertheless, the results show that the initiatives were able to trigger significant (p-value < 0.05) positive effects mainly for the environmental sustainability goals. In contrast, the goals within the economic, social and good governance sustainability dimensions were rarely affected. Moreover, certified initiatives had more frequently a positive sustainability effect compared to uncertified initiatives.