In recent years biometric identification and verification of human beings based on their physiological or behavioral characteristics have become a trend in security applications. The existing multi-modal biometric system (MBS) uses machine learning models for multi class classification, but they fail in providing maximum confidentiality, security, and accuracy. The proposed work adopts a Deep Learning Convolutional Neural Network (DLCNN) approach to implement the robust MBS. The proposed system is well trained and tested using four different datasets, including biometric images of the face, ear, palm, and finger. Initially, noise from the datasets is eliminated using Gabor filtering. Feature extraction and dimensionality reduction operations are implemented by using the Improved Principal Component Analysis (IPCA) technique. Here, IPCA is used for the fusion of four unique biometric features with the matching score, and feature ranking is performed using the Extended Borda-Count method. Further, DLCNN-based classification is used for the recognition of all biometric modalities, which improves the robustness and accuracy of the system. The proposed DLCNN-Gabor based IPCA technique improves the biometric recognition rate compared to existing methods.