The mortality of cacao fruits caused by early fruit abortion or insect and pathogen attacks was investigated in differently managed agroforestry systems in Central Sulawesi, Indonesia. Nine agroforestry systems shaded by three different types of tree stands were selected, which represented a decrease in structural heterogeneity: forest remnants, diverse planted trees and one or two species of planted leguminose trees. After standardized manual cross-pollination, the development of 600 fruits on 54 trees (6 trees per agroforest) was followed during 18 weeks of fruit development. In total, 432 of all fruits were lost before maturity, which seriously undermined yields. The proportion of harvested fruits per tree (overall average: 27 ± 4%) was not affected by canopy type. Although shade cover did not have a significant effect, losses due to fruit abortion were most likely under forest shade, where nitrogen-fixing leguminose shade trees were absent. Fruit losses due to pathogenic infections and insect attacks increased with the homogenization of the agroforests, supporting the hypothesis that agricultural homogenization increases risks of pest outbreaks. In conclusion, shade management may be improved to increase yields from cacao using highly diversified natural shade agroforestry systems.