We report infrared magneto-spectroscopy studies on thin crystals of an emerging Dirac material ZrTe5 near the intrinsic limit. The observed structure of the Landau level transitions and zero-field infrared absorption indicate a two-dimensional Dirac-like electronic structure, similar to that in graphene but with a small relativistic mass corresponding to a 9.4 meV energy gap. Measurements with circularly polarized light reveal a significant electron-hole asymmetry, which leads to splitting of the Landau level transitions at high magnetic fields. Our model, based on the Bernevig-Hughes-Zhang effective Hamiltonian, quantitatively explains all observed transitions, determining the values of the Fermi velocity, Dirac mass (or gap), electron-hole asymmetry, and electron and hole g-factors.