The tremendous growth of internet-connected devices can lead to network congestion, making it essential to incorporate new technologies to optimize performance. Thus, Device to Device(D2D) communication has been deemed as an emerging technology that can be used to communicate efficiently. However, establishing a secure and reliable mechanism for Device-to-Device (D2D) communication that ensures security, reliability, and availability poses a significant challenge. To tackle these issues a novel deep learning-based security-reliability-availability aware multiple D2D environment (SERAV Deep-MAD) has been proposed for Secure D-2-D Communication in a Fog environment. The proposed method utilizes a Fully Homomorphic Quantum Diffie Hellman Encryption (FHQDHE) to secure the data while transmitting. The proposed method utilizes the novel secure Quantum Diffie Hellman key exchange (QDHKE) technique for sharing the key to transform the plain data into cipher data, then applies FH operations on the encrypted data before transferring it to the Fog environment. Whenever a client requests data from the Fog, the Fog provider verifies the client's access rights. Moreover, the Attention-based Bi-GRU model is utilized to categorize whether the device is non-attack or attack, if the data is attacked then the proposed model classifies multiple attacks. The NS2 Simulator is used to verify and validate the proposed SERAV Deep-MAD model, and resiliency analysis is done to assess performance. The proposed techniques attain a higher accuracy of 98.18% which is 1.58%, 2.02%, and 2.41% better than MECC, MIMO, andAAKA-D2D respectively.