Globally, the coffee berry borer (CBB) Hypothenemus hampei (Ferreri) is the most severe pest of the commercial coffee species arabica coffee (Coffea arabica). Heterogenous landscapes characterize many coffee-growing areas. We studied the phenology of coffee and the abundance, and development of infestation by CBB within the context an altitudinal gradient in the coffee-growing region of Caldas, Colombia. This was between 1200 m and 1800 m (a.s.l.) from September 2019 to July 2020 covering two seasons of coffee flowering. Coffee berry phenology did not differ over the altitudinal gradient studied, but CBB abundance and berry infestation differed in quantity and timings, with lower altitudes having higher abundances and proportions of infestation than higher altitudes. In the context of CBB management, October through to December is vital for coffee farms <1500 m as CBB abundance and infestation both spike, and March to June is key for farms between 1500-1700 m for the same reasons. In conclusion, coffee farmers in highly heterogeneous landscapes will have to adjust their CBB management according to the altitude gradients of their farm.