Incorporation of feature engineering and attention mechanisms into deep learning models to develop an early warning system for harmful algal blooms

Abstract

The accurate forecasting of harmful algal blooms (HABs) is hindered due to insufficient monitoring frequency, numerous interacting factors, and large temporal variability in cyanobacteria abundance. In this study, the combined use of multiple feature engineering and attention mechanisms in deep learning (DL) models was explored to improve the performance, temporal resolution, and explainability of HAB forecasts at varying forecast horizons (i.e., 1, 7, and 14 days). Incorporation of feature engineering and attention mechanism into the DL models enabled daily forecasts of HABs without requiring any manual data manipulation. The hybrid DL models were applied to a site in the lower Nakdong River, South Korea, where HABs continue to be a significant water quality problem. Various input features pertaining to meteorological, hydrological, environmental, and biological factors were used to forecast cyanobacteria abundance. Regardless of the feature engineering and attention mechanisms, 1-day forecasts (root mean square error [RMSE] = 0.316–0.405, R2 = 0.985–0.991, mean absolute error [MAE] = 0.241–0.295, symmetric mean absolute percentage error [SMAPE] = 21.998%–22.581%) were much more accurate than 7-day and 14-day forecasts (RMSE = 2.335–2.669, R2 = 0.347–0.500, MAE = 1.830–2.170, SMAPE = 30.256%–34.348%). Among the two feature engineering mechanisms, bidirectional recurrent imputation for time series slightly but consistently outperformed unidirectional recurrent imputation for time series, with the performance difference increasing with an increase in the forecast horizon. Among the two attention mechanisms, although the reverse time attention mechanism (RMSE = 0.316–2.669, R2 = 0.347–0.991, MAE = 0.241–2.156, SMAPE = 22.429%–32.780%) and dual-stage attention-based recurrent neural network (RMSE = 0.382–2.622, R2 = 0.370–0.987, MAE = 0.288–2.170, SMAPE = 21.998%–34.348%) exhibited similar performance, explanations derived by the former had clearer distinctions regarding the relative importance of different input features and time steps. The results of this study reveal that explanations can differ considerably between different attention mechanisms, which necessitate further investigation to ensure the credibility of attention-based DL models.