For small n , the known compact hyperbolic n -orbifolds of minimal volume are intimately related to Coxeter groups of smallest rank. For n=2 and 3 , these Coxeter groups are given by the triangle group [7,3] and the tetrahedral group [3,5,3] , and they are also distinguished by the fact that they have minimal growth rate among all cocompact hyperbolic Coxeter groups in \operatorname{Isom}\mathbb H^n , respectively. In this work, we consider the cocompact Coxeter simplex group G_4 with Coxeter symbol [5,3,3,3] in \operatorname{Isom}\mathbb H^4 and the cocompact Coxeter prism group G_5 based on [5,3,3,3,3] in \operatorname{Isom}\mathbb H^5 . Both groups are arithmetic and related to the fundamental group of the minimal volume arithmetic compact hyperbolic n -orbifold for n=4 and 5 , respectively. Here, we prove that the group G_n is distinguished by having smallest growth rate among all Coxeter groups acting cocompactly on \mathbb H^n for n=4 and 5 , respectively. The proof is based on combinatorial properties of compact hyperbolic Coxeter polyhedra, some partial classification results and certain monotonicity properties of growth rates of the associated Coxeter groups.