Filter Transfer Learning Algorithm for Missing Data Imputation in Wastewater Treatment Process

Abstract

Missing data imputation is a critical data processing procedure in wastewater treatment process. However, the existing imputation methods cannot stand the missing data with high proportions that frequently happens due to unmaintained instruments or detection failures. Transfer Learning aims to learn much reliable information for the target domain with previous learned knowledge from a source domain, which provides a framework for solving such problem. This paper proposes a filter transfer learning algorithm (FTLA) for missing data imputation with high proportions. First, a knowledge acquisition strategy is developed to extract the source knowledge, including independent knowledge from historical datasets and parallel knowledge in terms of related datasets. The missing data is then interpreted through source knowledge comprehensively. Second, a filter transfer learning algorithm is designed to achieve target knowledge that mimics the tendency of the missing data. This algorithm can avoid serious negative transfer by using the extended Kalman filter to filtrate source knowledge. Third, a knowledge rolling mechanism is established to perform the imputation online with target knowledge, which can maintain the reliable imputation for missing data with high proportions. Finally, several comparative experiments of wastewater data are provided to demonstrate the merits of missing data imputation with FTLA.