Abstract
This paper introduces a new loss function and Usefulness measure for evaluating early warning systems (EWSs) that incorporate policymakers’ preferences between issuing false alarms and missing crises, and individual observations. The novelty derives from three enhancements: (i) accounting for unconditional probabilities of the classes, (ii) computing the proportion of available Usefulness that the model captures, and (iii) weighting observations by their importance for the policymaker. The proposed measures are model free such that they can be used to assess early warning signals issued by any type of EWS, and flexible for any type of crisis. Applications to two renowned EWSs, and comparisons to two common evaluation measures, illustrate the importance of an objective criterion for choosing a final specification and threshold value, and for models to be useful, the need to be more concerned about the rare class and the importance of correctly classifying observations of most relevant entities.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.