Two-dimensional Dirac semimetals have attracted much attention because of their linear energy dispersion and non-trivial Berry phase. Graphene-like 2D Dirac materials are gapless only within certain approximations, e.g., if spin-orbit coupling (SOC) is neglected. It has recently been reported that materials with nonsymmorphic crystal lattice possess symmetry-enforced Dirac-like band dispersion around certain high-symmetry momenta even in the presence of SOC. Here we calculate the optical absorption coefficient of nonsymmorphic semimetals, such as $\alpha$-bismuthene, which hosts two anisotropic Dirac cones with different Fermi velocities along $x$ and $y$ directions.We find that the optical absorption coefficient depends strongly on the anisotropy factor and the photon polarization. When a magnetic field is applied perpendicular to the plane of the material, the absorption coefficient also depends on an internal parameter we termed the mixing angle of the band structure. We further find that an in-plane magnetic field, while leaving the system gapless, can induce a Van-Hove singularity in the joint density of states: this causes a significant enhancement of the optical absorption at the frequency of the singularity for one direction of polarization but not for the orthogonal one, making the optical properties even more strongly dependent on polarization. Due to the anisotropy present in our model, the Dirac cones at two high-symmetry momenta in the Brillouin zone contribute very differently to the optical absorbance. Consequently, it might be possible to preferentially populate one valley or the other by varying photon polarization and frequency. These results suggest that nonsymmorphic 2D Dirac semimetals are excellent candidate materials for tunable magneto-optic devices.