Abstract
AbstractNeural networks have become increasingly prevalent within the geosciences, although a common limitation of their usage has been a lack of methods to interpret what the networks learn and how they make decisions. As such, neural networks have often been used within the geosciences to most accurately identify a desired output given a set of inputs, with the interpretation of what the network learns used as a secondary metric to ensure the network is making the right decision for the right reason. Neural network interpretation techniques have become more advanced in recent years, however, and we therefore propose that the ultimate objective of using a neural network can also be the interpretation of what the network has learned rather than the output itself. We show that the interpretation of neural networks can enable the discovery of scientifically meaningful connections within geoscientific data. In particular, we use two methods for neural network interpretation called backward optimization and layerwise relevance propagation, both of which project the decision pathways of a network back onto the original input dimensions. To the best of our knowledge, LRP has not yet been applied to geoscientific research, and we believe it has great potential in this area. We show how these interpretation techniques can be used to reliably infer scientifically meaningful information from neural networks by applying them to common climate patterns. These results suggest that combining interpretable neural networks with novel scientific hypotheses will open the door to many new avenues in neural network‐related geoscience research.
Highlights
Machine learning methods are emerging as a powerful tool in scientific applications across all areas of geoscience (e.g. Gil et al, 2018; Kapartne et al, 2018; Rolnick et al, 2019), including marine science (e.g. Malde et al, 2019), solid earth science (e.g. Bergen et al, 2019), and atmospheric science (e.g. Barnes et al, 2019; Boukabara et al, 2019; Lopatka, 2019; McGovern et al, 2017, Reichstein et al, 2019)
Ham et al (2019) used a neural network to predict the evolution of the El Nino Southern Oscillation (ENSO), and used interpretation techniques to show that ENSO precursors exist within the South Pacific and Indian Oceans
The rules that we present have been developed to work best with the Rectified Linear Unit (ReLU) activation function, since they test whether a node has been “activated” or not (Bach et al, 2015; Montavon et al, 2017)
Summary
Machine learning methods are emerging as a powerful tool in scientific applications across all areas of geoscience (e.g. Gil et al, 2018; Kapartne et al, 2018; Rolnick et al, 2019), including marine science (e.g. Malde et al, 2019), solid earth science (e.g. Bergen et al, 2019), and atmospheric science (e.g. Barnes et al, 2019; Boukabara et al, 2019; Lopatka, 2019; McGovern et al, 2017, Reichstein et al, 2019). In another case, Ham et al (2019) used a neural network to predict the evolution of the El Nino Southern Oscillation (ENSO), and used interpretation techniques to show that ENSO precursors exist within the South Pacific and Indian Oceans Even in these cases, the primary objective was to construct a neural network that most accurately predicted its output, with the interpretation being used to ensure the network attained high accuracy using reasoning consistent with physical theory. This theme is common throughout geoscientific applications of neural. We analyze two commonly studied climate phenomena, the El Nino Southern Oscillation and its relationship to seasonal prediction, so that we can first ensure the interpretation methods capture known patterns of geophysical variability before extending into the unknown
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
