Why we should condition denoising diffusion generative models on windows of past observations
Data assimilation (DA) is, traditionally, a cycling process that relies on time-dependent priors to propagate information from past observations to future cycles. Using denoising diffusion generative modeling for DA is challenging because standard approaches use a fixed training data set, which in turn leads to a static prior that ignores information from past observations. Because past observations are ignored, DA systems with static priors lead to larger posterior errors than cycling DA systems. Incorporating time-dependent priors into generative models, however, requires expensive and frequent retraining. Motivated by linear systems theory - where the dependence of a prediction of a Kalman filter on past observations decays exponentially - we condition diffusion models on short windows of past observations. Specifically, we describe training procedures for two frameworks: a diffusion DA system predicting the current state given a set of past observations, and a diffusion ``direct observation prediction'' (DOP) system, predicting future observations given a set of past observations. Using a canonical linear system, we show that both systems can achieve the minimal posterior error characteristic of a fully-cycled DA/DOP system, without re-training, provided the time windows are long enough. The linear setup ensures analytical tractability, avoids confounding neural network training errors, and confirms that conditioning on windows of past observations is required for efficient and accurate diffusion-based DA or DOP.