Millimeter wave (mmwave) communication has attracted increasing attention owing to its abundant spectrum resource. The short wavelength at mmwave frequencies facilitates placing a large number of antennas in a small space, and the mmwave channels are likely to be sparse in the directions. These two new features promise enhanced security by directional precoding. To explore this potential, we investigate the design of directional hybrid digital and analog precoding for the multiuser mmwave communication system with multiple eavesdroppers. Particularly, we consider two cost-efficient sub-connected hybrid architectures, i.e., multi-subarray architecture and switched-phased-array architecture, and optimize the hybrid precoding under per-antenna constant envelope (CE) constraints. The goal of our design is to guarantee the receive quality of the legitimate users while minimizing the power leaked to the eavesdroppers, so as to realize a directional transmission for a general mmwave channel. The resulting problems are very challenging due to the nonlinear CE constraints and binary integer constraint from antenna selection. To address them, we leverage exact penalty function methods to find efficient solutions to the CE hybrid directional precoding. Our analysis shows that the proposed algorithm is able to converge to a stationary point under some mild conditions. Simulation results are finally provided to confirm the effectiveness of the proposed schemes and their superiority over the existing schemes under both single-path and multi-path mmwave channels.