Directional depinning dynamics of a two-dimensional (2D) dusty plasma solid modulated by a 2D square periodic substrate are investigated using Langevin dynamical simulations. We observe prominent directional locking effects when the direction of the external driving force is varied relative to the underlying square substrate. These locking steps appear when the direction of the driving force is close to the symmetry direction of the substrate, corresponding to the different dynamical flow patterns and the structures. In the conditions between the adjacent locking steps, moving ordered states are observed. Although the discontinuous transitions often occur between the locking steps and the nonlocking portion, the continuous transitions are also found around the locking step associated with the disordered plastic flow close to its termini. Our results show that directional locking also occurs for underdamped systems, which could be tested experimentally in dusty plasmas modulated by 2D substrates.