PERTURB-c: Correlation Aware Perturbation Explainability for Regression Techniques to Understand Retrieval Black-boxes

Abstract

In this paper we introduce PERTURB-c, a correlation-aware framework for interpreting black box regression models with one-dimensional structured inputs. We demonstrate this framework on a simulated case study with machine learning based transit spectroscopy retrievals of exoplanet WASP-107b. Characterising many exoplanet atmospheres can answer important questions about planetary populations, but traditional retrievals are very resource intensive; machine learning based methods offer a fast alternative however (i) they require high volumes data (only obtainable through simulations) to train and (ii) their complexity renders them black-boxes. Better understanding how they reach predictions can allow us to inspect for biases, which is especially important with simulated data, and verify that predictions are made on the basis of physically plausible features. This ultimately improves the ease of adoption of machine learning techniques. The most used methods to explain machine learning model predictions (such as SHAP and other methods that rely on stochastic sample generation) suffer from high computational complexity and struggle to account for interactions between inputs. PERTURB-c addresses these issues by leveraging physical knowledge of the known spectral correlation. For visualisation of this analysis, we propose a heat-map-based representation which is better suited to large numbers of input features along a single dimension, and that is more intuitive to those who are already familiar with retrieval methods. Note that while we chose this exoplanet retrieval context to demonstrate our methodologies, the PERTURB-c framework is model agnostic and in a broader context has potential value across a plethora of adjacent regression problems.