Electrical power systems are the lifeblood of modern civilization, providing the essential energy infrastructure that powers our homes, industries, and technologies. As our world increasingly relies on electricity, and modern power systems incorporate renewable energy sources, the challenges have become more complex, necessitating advanced forecasting and optimization to ensure effective operation and sustainability. This review paper provides a comprehensive overview of electrical power systems and delves into the crucial roles that forecasting and optimization play in ensuring future sustainability. The paper examines various forecasting methodologies from traditional statistical approaches to advanced machine learning techniques, and it explores the challenges and importance of renewable energy forecasting. Additionally, the paper offers an in-depth look at various optimization problems in power systems including economic dispatch, unit commitment, optimal power flow, and network reconfiguration. Classical optimization methods and newer approaches such as meta-heuristic algorithms and artificial intelligence-based techniques are discussed. Furthermore, the review paper examines the integration of forecasting and optimization, demonstrating how accurate forecasts can enhance the effectiveness of optimization algorithms. This review serves as a reference for electrical engineers developing sophisticated forecasting and optimization techniques, leading to changing consumer behaviors, addressing environmental concerns, and ensuring a reliable, efficient, and sustainable energy future.