Conventional direction-of-arrival (DOA) estimation methods for multiple-input multiple-output (MIMO) joint sensing and communication system normally pursue high estimation accuracy and resolution by imposing orthogonal waveforms. However, such operation results in a deterioration of communication performance. In this paper, we propose a nonorthogonal waveform assisted DOA estimation algorithm, where an augmented virtual array is derived by exploiting the nonorthogonal MIMO communication waveforms, while a high communication rate can still be maintained. To estimate the round-trip sensing channels of each subcarrier, we utilize the transmitted symbols as pilot symbols, and obtain all the channel coefficients with a minimum mean square error solver. A virtual channel matrix can be formulated with these channel coefficients, which can be regarded as the samples of an augmented virtual array. Based on that, the subspace processing can be conducted for DOA estimation with fully nonorthogonal waveforms. Furthermore, the rank deficiency property of the equivalent signal matrix of the virtual array is analyzed when the distance of targets are identical. To address the problem, a Toeplitz reconstruction method is proposed to restore the rank of the rank-deficient equivalent signal matrix for DOA estimation. Simulations show that the proposed nonorthogonal waveform assisted DOA estimation algorithm outperforms the conventional methods in terms of resolution and accuracy, while maintaining a satisfactory computational efficiency.