Nowadays, there is tremendous growth in biometric authentication and cybersecurity applications. Thus, the efficient way of storing and securing personal biometric patterns is mandatory in most governmental and private sectors. Therefore, designing and implementing robust security algorithms for users’ biometrics is still a hot research area to be investigated. This work presents a powerful biometric security system (BSS) to protect different biometric modalities such as faces, iris, and fingerprints. The proposed BSS model is based on hybridizing auto-encoder (AE) network and a chaos-based ciphering algorithm to cipher the details of the stored biometric patterns and ensures their secrecy. The employed AE network is unsupervised deep learning (DL) structure used in the proposed BSS model to extract main biometric features. These obtained features are utilized to generate two random chaos matrices. The first random chaos matrix is used to permute the pixels of biometric images. In contrast, the second random matrix is used to further cipher and confuse the resulting permuted biometric pixels using a two-dimensional (2D) chaotic logistic map (CLM) algorithm. To assess the efficiency of the proposed BSS, (1) different standardized color and grayscale images of the examined fingerprint, faces, and iris biometrics were used (2) comprehensive security and recognition evaluation metrics were measured. The assessment results have proven the authentication and robustness superiority of the proposed BSS model compared to other existing BSS models. For example, the proposed BSS succeeds in getting a high area under the receiver operating characteristic (AROC) value that reached 99.97% and low rates of 0.00137, 0.00148, and 0.00157 for equal error rate (EER), false reject rate (FRR), and a false accept rate (FAR), respectively.