Boltzmann solvers face significant difficulty in simulating rarefied flows at high Knudsen numbers. In this flow regime, the gas distribution function is widely scattered and highly concentrated with a very steep slope in the particle velocity space. In order to capture the feature of such a flow, the Boltzmann solvers such as the Discrete Unified Gas Kinetic Scheme (DUGKS) discretize the particle velocity space with a very fine mesh (many discrete particle velocities) using the Discrete Velocity Method (DVM) due to which the load for computation becomes unendurable. In this paper, a Reduced Order Modeling (ROM) method is used to generate a reduced discrete velocity space for the DUGKS. More specifically, the discrete empirical interpolation method [S. Chaturantabut and D. C. Sorensen, SIAM J. Sci. Comput. 32, 2737–2764 (2010)] is used to select the dominant nodes in the original discrete velocity space to form a reduced discrete velocity space, which represents important dynamical characteristics. In this way, most grid points in the discrete velocity space, which are of negligible importance on the integration, are removed in practical computation, which yields a significant improvement in computational efficiency. The proposed ROM approach is not limited to a specific DVM-based solver. For illustration, in this paper, we developed the Reduced Order Modeling-based Discrete Unified Gas Kinetic Scheme (ROM-DUGKS) by applying the reduced velocity space to the conventional DUGKS. Validations are performed in both low-speed and hypersonic rarefied flows at various Knudsen numbers. The results show that the ROM-DUGKS is much more efficient than the original DUGKS while still maintaining high accuracy. This significant improvement in computational efficiency will unleash the potential of the DVM-based solvers such as the DUGKS for practical applications to rarefied flow problems.