Compared to shade coffee plantations, sun-grown coffee is not considered very biodiversity-friendly, but little is known about how the management practices for this crop and landscape characteristics affect species diversity. Some predatory staphylinids (rove beetles) are important biological control agents in many agricultural systems, but have not received much attention in tropical agroecosystems or in coffee plantations. After collecting 2074 individuals belonging to 126 species of staphylinid collected from 27 sampling sites in a region dominated by sun-grown coffee in the Colombian Andes, we evaluated how local and landscape heterogeneity affect the diversity, abundance and composition of staphylinids. We found that both local and landscape variables had effects in different directions and of different magnitudes depending on the community attribute evaluated. At the local scale, rove beetle diversity increased with the density of herbaceous plants and coffee plant cover. At the landscape scale, species richness decreased with increasing forest cover, while the number of forest patches had a positive effect. We also found a high degree of species turnover within and between coffee plantations, and this was explained by farmland habitat structural differences between the coffee plantations and also the characteristics of the surrounding landscape. Our results indicate that the Andean sun-grown coffee plantations still host diverse and taxonomically dissimilar staphylinid communities. Therefore, properly managed local vegetation structure and the preservation of forest remnants on the landscape contribute to maintaining high diversity in this coffee production system. Promoting biodiversity conservation and the provision of ecosystem services by agroecosystems—even in intense production systems—requires that environmental heterogeneity on several scales be taken into account.