A vertex triple (u,v,w) of a graph is called a 2-geodesic if v is adjacent to both u and w and u is not adjacent to w. A graph is said to be 2-geodesic transitive if its automorphism group is transitive on the set of 2-geodesics. In this paper, a complete classification of 2-geodesic transitive graphs of order pn is given for each prime p and n≤3. It turns out that all such graphs consist of three small graphs: the complete bipartite graph K4,4 of order 8, the Schläfli graph of order 27 and its complement, and fourteen infinite families: the cycles Cp,Cp2 and Cp3, the complete graphs Kp,Kp2 and Kp3, the complete multipartite graphs Kp[p], Kp[p2] and Kp2[p], the Hamming graph H(2,p) and its complement, the Hamming graph H(3,p), and two infinite families of normal Cayley graphs on the extraspecial group of order p3 and exponent p.