The hybrid beamforming is a promising technology for the millimeter wave MIMO system, which provides high spectrum efficiency, high data rate transmission, and a good balance between transmission performance and hardware complexity. The most existing beamforming systems transmit multiple streams by formulating multiple orthogonal beams. However, the Neural network Hybrid Beamforming (NHB) adopts a totally different strategy, which combines multiple streams into one and transmits by employing a high-order non-orthogonal modulation strategy. Driven by the Deep Learning (DL) hybrid beamforming, in this work, we propose a DL-driven non-orthogonal hybrid beamforming for the single-user multiple streams scenario. We first analyze the beamforming strategy of NHB and prove it with better Bit Error Rate (BER) performance than the orthogonal hybrid beamforming even with the optimal power allocation. Inspired by the NHB, we propose a new DL-driven beamforming scheme to simulate the NHB behavior, which avoids time-consuming neural network training and achieves better BERs than traditional hybrid beamforming. Moreover, our simulation results demonstrate that the DL-driven non-orthogonal beamforming outperforms its traditional orthogonal beamforming counterpart in the presence of sub-connected schemes and imperfect Channel State Information (CSI).