State-dependent decadal predictability identified with explainable machine learning

Variation in climate from one year to another or even one decade to another can be substantial, including severe winters followed by mild ones and alternations between dry and rainy summers. Efforts to predict such interannual to decadal variations have been a subject of intensive research but with mixed results, suggesting that the predictability of climate variations is not high in general but there may be particular cases in which useful multi-year predictions can be made. For example recent work suggests that multi-year warming and cooling of North Atlantic sea surface temperature (SST) can be anticipated when Atlantic ocean heat transport is unusually strong or weak.Research supported under this award uses machine learning techniques in combination with climate model simulations to identify climate states that lead to enhanced predictability, and understand why climate predictability is enhanced for these states. The work uses Controlled Abstention Networks (CANs), a variant of neural networks developed by the lead Principal Investigator (PI) and others in which the neural network is able to overlook data from the training set in which there are no identifiable relationships between predictors (like ocean heat transport) and predictands (like North Atlantic SST). In effect the CAN says "I don't know" when confronted with ambiguous training data, thereby concentrating on those portions of the training dataset which contain strong, predictable signals. The CAN is ideally suited to the search for state-dependent climate predictability given its underlying assumption that predictable relationships are the exception rather than the norm.The CAN-based search for state-dependent predictability is accompanied by analysis seeking to explain why climate fluctuations evolve more predictably from some climate states than from others. Applications of neural networks to climate science are hampered by the "black box" nature of the networks, which may have uncanny predictive power yet lack credibility because there is no accounting for why a particular set of inputs produces a given result. The PIs address this shortcoming through an explainable artificial intelligence (XAI) technique called layerwise relevance propagation (LRP, developed by the lead PI), which generates "relevance heat maps" showing the spatial patterns of data that are the most influential in producing the predictive relationships found by CAN or other neural networks. For example LRP applied to a neural network predictive scheme for surface temperatures in the Pacific Northwest shows that most of the predictive skill comes from precursor SST patterns along the Kuroshio current and in the northwest Pacific, both regions associated with known modes of decadal Pacific climate variability.A further novelty of the work is the use of climate model output rather than observations. Machine learning methods require large amounts of training data, thus the few decades of the observational record are insufficient for the development of decadal prediction schemes. The PIs take advantage of the 100-member ensemble of simulations from the second version of the Community Earth System Model (CESM2), covering the period 1850 to 2100, along with similar simulations from the Coupled Model Intercomparison Project (CMIP), to provide adequate sample size. A further advantage of the climate model simulations is that they allow examination of changes in decadal predictability as a consequence of anthropogenic climate change.The work is of societal as well as scientific interest due to the potentially severe impacts of climate variability. The Dust Bowl drought of the 1930s is a prime example of decadal climate variability and its societal consequences, which were made worse by the agricultural practices of the era. The work also develops the techniques of XAI, which are relevant to the ethical use of artifical intelligence technology. In addition to the societal value of the research products the project has broader impacts through its partnership with two minority-serving institutions, Metropolitan State University of Denver (MSU) and North Carolina Agricultural and Technical University (NCA&T). The PIs work with collaborators Sam Ng (MSU) and Ademe Mekonnen (NCA&T) to incorporate machine learning methods into undergraduate courses, covering topics including what "machine learning" actually means and why overfitting is bad. The award provides funding for students from MSU and NCA&T to participate in the Reseach Experiences for Undergraduates (REU) program at Colorado State University, where they will spend 10 weeks working on research related to this project. The project also provides support and training to two graduate students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.