The channel estimation is crucial in the “millimeter Wave (mmWave) Massive Multiple-Input MultipleOutput (MIMO) and Non-Orthogonal Multiple Access (NOMA)” devices. Hybrid beamforming techniques are employed nowadays to minimize the complexity and equipment price. However, the absence of digital beam forming in mmWave affects the dynamic range and accuracy of the channel estimation. Previous research has concentrated mainly on predicting narrow-band mmWave channels using deep learning networks, as the wideband channels of mmWave create a considerable range and noise issue. Accurate channel estimation in the MIMO system is challenging because of the increased number of antennas and radio frequency (RF) chains. MIMO system communications using mmWave are frequently chosen because of their massive spectrum resources. Therefore, it is essential to tackle the obstacles obtained in the standard channel estimation framework by developing a MIMO-NOMA network with the help of deep learning methods Hence, this paper proposes an efficient hybrid deep learning model for channel estimation in MIMO-NOMA for mmWave systems. At first, the channel estimation is carried out using the Adaptive Hybrid Deep Learning (AHDL) model, where it combines both Autoencoder and Recurrent Neural Network (RNN). Here, the parameters are optimized using the Improved Red-tailed Hawk Algorithm (IRHA). Later the hardware cost and system complexity are reduced by performing the hybrid beam-forming process. Numerical results show that the proposed channel estimation and pilot estimation process outperforms the state-of-the-art approaches.