Blockage in millimeter wave (mmWave) channels can severely degrade the efficiency of air-to-ground (A2G) communications between unmanned aerial vehicles (UAVs) and base stations (BSs). Although relay-aided transmission has proved to be effective in mitigating the negative impact of blockage on mmWave communications, how to efficiently select the optimal relaying UAV for A2G communications remains an open problem. To enable optimal relay selection, the UAV under blockage should probe neighboring UAVs for the quality of their A2G links, which incurs non-negligible beam training overheads. Moreover, most existing studies ignore the impacts of UAV orientations as well as the finite sizes of beam codebooks on the performance of relay selection. In view of these challenges, we study the design of relay probing strategies for UAV mmWave A2G communications to maximize the expected achievable throughput between the UAV under blockage and the BS. Different from existing work, we jointly consider beam training overhead, the limited number of neighboring UAVs, the finite sizes of UAV codebooks, and the orientation of neighboring UAVs in our schematic design. Besides, we derive the closed-form expression for the distribution of achievable throughput when a specific neighboring UAV is adopted as the relay, in order to facilitate the development of the optimal relay probing strategy. Extensive simulation results demonstrate the accuracy of our analysis and the superiority of the obtained relay probing strategy.