CuAD: Source-to-Source Automatic Differentiation of CUDA Kernels for Discrete Adjoints in Computational Fluid Dynamics

Abstract

Discrete adjoint methods give the sensitivities that aerodynamic design and data assimilation require, but they remain hard to obtain for flow solvers that run on graphics processors. This paper presents CuAD, a source-to-source automatic differentiation tool that reads CUDA kernels and writes their tangent and adjoint versions as ordinary CUDA source. Its reverse mode is organised around the GPU thread and exploits the structure of flow-solver kernels: pure accumulations need no tape, the remaining intermediate values are recomputed rather than stored, and adjoint updates are atomic only where several threads reach the same memory. We evaluate CuAD on four kernels from simple flow models: a viscous Burgers residual, a two-dimensional advection-diffusion residual, a quasi-one-dimensional Euler residual and a Biot-Savart summation over point vortices. All generated derivatives agree with finite differences and pass adjoint dot-product tests to round-off, while the adjoint kernels cost two to six times the primal kernels. Compared with the compiler-based tool Clad under the same compiler, CuAD gives identical results at identical cost on the stencil kernels and is 16 to 470 times faster on the summation kernel because it stores no tape. A steady Burgers study shows that in single precision the adjoint gradient of a finite-difference Jacobian reaches errors of 45 to 75 per cent once the condition number grows to $2.5\times10^{5}$, whereas the CuAD Jacobian keeps the error below $5\times10^{-5}$. Four applications complete the evaluation: adjoint fields of the Burgers shock, the sensitivity of a scalar sensor in a cellular vortex, the inverse design of a transonic nozzle whose area is recovered to $2\times10^{-8}$, and an unsteady adjoint of the roll-up of a vortex sheet over 400 time steps.