Detecting components in censored and truncated meteorological data

Abstract

Many meteorological datasets are mixtures in which components correspond to particular physical phenomena, the accurate identification of which are important from a meteorological standpoint. In particular, rainfall is generated by at least two processes—one convection, the other frontal systems—each characterised by its own distribution of rain rates and durations. The breakpoint data format, in which the timings of rain-rate changes and the steady rates between changes are recorded, captures the information required to parameterise these phenomena. Rainfall data has only recently become available in breakpoint format, which is both more compact and contains more information than older sources such as the fixed amount and fixed interval representation commonly used. Techniques such as the EM algorithm can be used to decompose the breakpoint data into its components. However, the quality of the currently available breakpoint data is poor for low rates and short durations and these portions of the data need to be discarded, or screened out, and the EM algorithm modified. In this paper, the EM algorithm is extended to deal with datasets in which data screening has taken place. The unified approach adopted appears new and, although tailored to a particular and important application, the method should have much wider application. Furthermore, in this paper the extension is applied to a large scale breakpoint dataset of about 56,000 observations with univariate and bivariate normal mixtures being fitted after censoring or truncation below a point or line respectively. The procedure was also applied to simulated breakpoint data which showed that the procedure was relatively robust and gave excellent results in the majority of cases. For the actual data, the results at low truncation agreed with applications of the EM algorithm to non-truncated data, but a different picture arose at moderate truncation. An analysis of the dry times between periods of precipitation is also given as an example of censoring. Overall, four components were required to adequately represent the wet data and another four for the dry data, giving a total of 34 parameters to model the 56,000 breakpoints. Copyright © 1998 John Wiley & Sons, Ltd.