In the era of digital transformation, marked by the widespread adoption of cloud-based technologies such as cloud computing and storage, ensuring data security and protecting user privacy have emerged as critical industry concerns. Fully Homomorphic Encryption (FHE) addresses these issues by allowing various computations to be performed on ciphertexts without the need for decryption, thus safeguarding data security and privacy. This paper explores the mathematical and algorithmic foundations essential for understanding FHE. It provides a detailed analysis of the advancements in FHE schemes over the past decade, focusing on various mathematical problems, including ideal lattices and integers. The discussion extends to three practical applications: cloud computing, machine learning, and electronic voting, highlighting the progress and exploring potential future research avenues and developmental strategies in the field of FHE. This comprehensive review not only underscores the significance of FHE in enhancing data security but also charts a path for its future exploration and integration into emerging technologies.