AbstractMadagascar frogs (Mantellidae) contain species with derived mating behaviour and femoral macroglands secreting volatile pheromones. Here, we screen 28 mantellid species to show that volatile compounds are present in at least five genera with femoral glands, while no such compounds were found in genera devoid of these glands. The nasal cavity of volatile‐producing species of four genera showed a derived state, with partial or almost complete obstruction of the connection between main olfactory organ (MOO) and vomeronasal organ (VNO). Experimental evidence suggests that the pheromone of one mantelline, Mantidactylus betsileanus, distinctly stimulates sensory neurons in the MOO but not in the VNO. These data support an evolutionary connection of derived olfactory anatomy with chemical communication in these frogs, although the role of the anatomically separated VNO remains unclarified.