We study transience and recurrence of simple random walks on percolation clusters in the hierarchical group of order N, which is an ultrametric space. The connection probability on the hierarchical group for two points separated by distance k is of the form \(c_k/N^{k(1+\delta )}, \delta >0\), with \(c_k=C_0+C_1\log k+C_2k^\alpha \), non-negative constants \(C_0, C_1, C_2\), and \(\alpha >0\). Percolation occurs for \(\delta 0\) and sufficiently large \(C_2\). We show that in the case \(\delta 0,\alpha >0\) there exists a critical \(\alpha _\mathrm{c}\in (0,\infty )\) such that the walk is recurrent for \(\alpha \alpha _\mathrm{c}\). The proofs involve ultrametric random graphs, graph diameters, path lengths, and electric circuit theory. Some comparisons are made with behaviours of simple random walks on long-range percolation clusters in the one-dimensional Euclidean lattice.