Paying attention to astronomical transients: introducing the time-series transformer for photometric classification

Abstract

Abstract Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the challenge of astronomical transient classification, with ever improving success. Transformers are a recently developed deep learning architecture, first proposed for natural language processing, that have shown a great deal of recent success. In this work, we develop a new transformer architecture, which uses multihead self-attention at its core, for general multivariate time-series data. Furthermore, the proposed time-series transformer architecture supports the inclusion of an arbitrary number of additional features, while also offering interpretability. We apply the time-series transformer to the task of photometric classification, minimizing the reliance of expert domain knowledge for feature selection, while achieving results comparable with state-of-the-art photometric classification methods. We achieve a logarithmic-loss of 0.507 on imbalanced data in a representative setting using data from the Photometric LSST Astronomical Time-Series Classification Challenge. Moreover, we achieve a micro-averaged receiver-operating-characteristic area under curve of 0.98 and micro-averaged precision–recall area under curve of 0.87.