This paper presents the development of a U-Net model using four basic optical bands and SRTM data to analyze changes in mangrove forests from 1990 to 2024, with an emphasis on the impact of restoration programs. The model, which employed supervised learning for binary classification by fusing multi-temporal Landsat 8 and Sentinel-2 imagery, achieved a superior accuracy of 99.73% for the 2020 image classification. It was applied to predict the long-term mangrove maps in Wunbaik Mangrove Forest (WMF) and to detect the changes at five-year intervals. The change detection results revealed significant changes in the mangrove forests, with 29.3% deforestation, 5.75% reforestation, and −224.52 ha/yr of annual rate of changes over 34 years. The large areas of mangrove forests have increased since 2010, primarily due to naturally recovered and artificially planted mangroves. Approximately 30% of the increased mangroves from 2015 to 2024 were attributed to mangrove plantations implemented by the government. This study contributes to developing a deep learning model with multi-temporal and multi-source imagery for long-term mangrove monitoring by providing accurate performance and valuable information for effective conservation strategies and restoration programs.