Scalable extensions to given-data Sobol' index estimators

Abstract

Given-data methods for variance-based sensitivity analysis have significantly advanced the feasibility of Sobol' index computation for computationally expensive models and models with many inputs. However, the limitations of existing methods still preclude their application to models with an extremely large number of inputs. In this work, we present practical and theoretical extensions to the existing given-data Sobol' index method, which allow variance-based sensitivity analysis to be efficiently performed on large models such as neural networks, which have $>10^4$ inputs. For models of this size, holding all input-output evaluations simultaneously in memory---as required by existing methods---can quickly become impractical. Our extensions include a general definition of the given-data Sobol' index estimator with arbitrary partition, a streaming algorithm to process input-output samples in batches, and an asymptotic analysis of the new estimator that motivates a practical screening heuristic for small indices. We show that the equiprobable partition employed in existing given-data methods can introduce significant bias into Sobol' index estimates even at large sample sizes and provide numerical analyses that demonstrate why this can occur. We also show that the streaming algorithm can achieve comparable accuracy and runtime while substantially reducing memory requirements, enabling sensitivity analysis of models with much larger input dimension. We demonstrate our novel developments on two application problems in neural network modeling.