Infilling Missing Monthly Maximum and Minimum Temperature Dataset by EM Algorithm Followed by Distribution Based Statistical Assessment Using Eight Absolute Homogeneity Tests

Abstract

In the current research, missing value analysis and tests for the presence of homogeneity were applied to the temperature records obtained from seven meteorological observatories spread throughout the state of Kerala. The monthly mean maximum and mean minimum temperature datasets of observatories managed by the Indian Meteorological Department (IMD) for the period 1969–2015 were considered. During analysis, every observatory was studied independently and those observatories which are having missing values for five continuous years, and more were rejected. The missing records were estimated using the Expectation Maximization Algorithm (EMA). The infilled dataset needs to be hydrologically as well as statistically stable for later hydrological and meteorological assessments. The reliability of datasets was tested using eight statistical absolute homogeneity tests. Before applying the homogeneity tests on the datasets, their assumptions must be fulfilled; one predominant assumption is regarding the normal distribution of the dataset. Thereby, the datasets are checked for normality behaviour by four statistical tests, namely Skewness z-ratio, Kurtosis z-ratio, Kolmogorov–Smirnov (KS) and Shapiro–Wilk (SW) test. Out of 14 datasets consisting of seven mean monthly maximum and seven mean monthly minimum temperature, ten datasets were found to be normal at 95% level of confidence (LOC) and the rest four were highly skewed and kurtotic. Following the normality tests, eight statistical absolute homogeneity tests were applied individually on the annual scale by which non-homogeneous stations were detected and eliminated. The datasets which resembled normal behaviour were tested using parametric homogeneity tests such as Linear Regression, Student’s t-test, Cumulative Deviation and Worsley Likelihood Ratio test. Out of normally distributed datasets, only three datasets were statistically homogenous at 99% LOC, and seven datasets failed to clear the homogeneity test. Remaining four non-normally distributed datasets was checked for homogeneity by applying non-parametric tests, such as Distribution-Free CUSUM, Rank-Sum, Median Crossing and Turning Points test. Out of four datasets, three were homogenous at 99% LOC and one failed homogeneity test.