Language identification has become a critical challenge in NLP, particularly in multilingual countries like India. This study addresses the identification of closely related Indo-Aryan languages, proposing a robust deep-learning ensemble model with data augmentation. Pre-trained models (Indic BERT, Indic Transformer, DevBERT, L3Cube-HindBERT) and various embedding techniques (word2vec, Indic transformer, Keras embedding) are explored with LSTM, Bi-LSTM, GRU, Bi-GRU, and CNN-Bi-LSTM deep learning models. Keras embedding performs better as compared to other embedding methods. The four best deep learning models are selected, and ensemble methods (soft voting, hard voting, stacking with meta models Random Forest and SVM classifier) are employed. When utilizing data augmentation with the training data and fitting the models Bi-LSTM, GRU, Bi-GRU, and CNN-Bi-LSTM with Keras embedding, the best performance, a 93% F1-score, was achieved by their soft voting classifier as compared to the other models. The performance of the proposed model was compared with the existing baseline models on the same dataset. For the comparison of model performance, the macro F1 score, accuracy, recall, and precision were used.