We study Dirac fermionic dark matter (DM, $\chi^0$) and confront it with recent data. To evade the stringent direct search limits from PandaX-II, XENON1T and LUX experiments, the quantum numbers of the Dirac DM are taken to be $I_3=Y=0$ to remove the tree-level $Z$-exchange diagram. Loop amplitudes can contribute to the elastic scattering cross section. We find that there are cancellations in the one-loop diagrams, which largely reduce the cross section and make the Dirac DM viable in the direct search. For a generic isospin $I$, we survey the Dirac DM mass constrained by the latest results of PandaX-II, XENON1T and LUX experiments, the observed DM relic density, and the H.E.S.S. and the Fermi-LAT astrophysical observations. Sommerfeld enhancement effects on DM annihilation processes are investigated. We find that the cross section of $\chi^0\bar\chi^0$ annihilating to the standard model (SM) gauge bosons are in general significantly enhanced, and the Fermi-LAT, the H.E.S.S. upper limits on $\langle\sigma v\rangle({W^+W^-,\gamma\gamma})$ and the observed relic density become serious constraints on the Dirac DM mass. The $I<4$ cases are ruled out and for $I\geq 4$, the lower bound on Dirac DM mass are forced to be $\gtrsim$ 60 TeV. The elastic scattering cross section for $m_\chi$ of few tens TeV with a generic $I$ is found to be $\sigma^{SI}\simeq I^2(I+1)^2\times7\times10^{-49}$~cm$^2$. The predicted $\langle\sigma (\chi^0\bar\chi^0\to Z^0Z^0, Z^0\gamma, \gamma\gamma)v\rangle$ and $\sigma^{SI}$ are sizable and they will be useful to search for DM in astrophysical observation and in direct search in near future.