Persisting with persistence: The verification of Radio 4 weather forecasts

Abstract

There has been an ongoing debate in the literature since the end of the last century as to the accuracy of weather forecasts. In 1885 the Met. Office claimed that its storm warnings were 79.3 per cent accurate, rising to 93.0 per cent in 1894 (Gaster 1896). Unfortunately, the verification scheme was not published. It seems that weather forecasts have always been considered to be about 85 per cent accurate (the average of the ten annual forecasts 188594 = 85 per cent) since the Met. Office was formed! Is this level of accuracy a deliberately created perception rather than a scientific truth? It could be argued that the verification statistics are manipulated in order to achieve about an 85 per cent accuracy. The public would not believe figures over 90 per cent and figures in the 70 per cents start to feel unsatisfactory. Obviously to test this hypothesis we need to know how the verification statistics are calculated. Today the verification of the accuracy of weather forecasts is still shrouded in mystery. Thousands of weather forecasts are issued around the world every day and yet there are no regularly published statistics on their quality and accuracy in the UK. Several authors (e.g. Brettle 1998; Ewins 1997; Jolliffe and Jolliffe 1997; Lynagh 1990; Marks 1997; Murphy 1991; Thornes 1996) have discussed the need and means for objective, and in some cases independent, verification but there is still a reluctance from the weather service providers, both public and private, to publish any statistics. It could be argued that information on commercial weather forecast quality and accuracy is commercially sensitive, but surely information on public weather forecasts issued by the Met. Office, through the media, should be made available? The Met. Office gives some information each year in its Annual Report but it is not clear how the few statistics are calculated. Also, their key performance indicators and targets for public weather forecasts have changed three times in the last three years. The Met. Office Annual Report and Accounts (Met. Office 1996) refers to a target accuracy of 84 per cent for the Radio 4 1755 h forecasts. However, according to the Met. Office (1997) the public weather forecasts were assessed using a Customer Satisfaction Index (CSI) which must reach 0.80 or greater. In fact, the CSI for services on BBC TV and Radio 4 achieved 0.82. How was this calculated? We shall never know because the Met. Office (1 998) has now introduced the Service Quality Index (SQIX) which buries the public forecast verification in a host of other statistics: