Fleshy-fruited plants rely on animal frugivores to disperse their seeds, and seed removal by frugivores may leave an imprint on seedling recruitment. However, to what extent plant–frugivore interactions are related to seedling recruitment has rarely been quantified at the community level, especially in species-rich tropical forests. In this study, we tested the effect of different plant traits on fruit removal by frugivores and tested the relative importance of fruit removal, plant traits and abiotic factors for seedling recruitment. We quantified plant–frugivore interactions of 22 fleshy-fruited plant species consumed by 56 diurnal frugivore species, and counted the number of seedlings that emerged along an elevational gradient in the Colombian Andes. We measured a set of plant traits (i.e., crop size; fruit size; seed load and mass; fruit nutritional contents), estimated the density of adult plants and recorded relevant abiotic factors (light, temperature and humidity). We found that fruit removal by frugivores was positively associated with crop size, but negatively associated with fruit length and unrelated to seed load and fruit nutritional content. Seedling densities were positively related to the density of adult plants, seed mass and fruit removal by animals. We found no relationship between abiotic factors and seedling recruitment. Our results indicate that fruit abundance and morphology are important determinants of fruit removal and that fruit removal is positively associated with seedling recruitment accounting for effects of species abundance and plant traits. We conclude that plant traits shape fruit removal and seedling recruitment at the community level, while these two crucial processes of forest regeneration are directly linked by seed dispersal of animals.