arXiv subjects

Max Sagebaum

Publications and source records attributed to Max Sagebaum.

10 recordsLinked to original sources

arXiv2026

Exploring the Boundaries of Differentiable Radiation Transport and Detector Simulation

We present an application of automatic differentiation for particle transport through matter using a Geant4-like radiation transport simulation with a full electromagnetic physics model. When differentiating this step-based transport, we observe exploding gradients driven by rare but extreme sensitivities at material boundaries, which propagate through subsequent transport and shower development. To obtain usable derivatives for optimization, we introduce a targeted mitigation strategy that stops gradient propagation through boundary-crossing operations under identifiable unstable conditions while leaving the forward (primal) simulation unchanged. We demonstrate that this enables stable, optimization-ready gradients in a detector-design problem.

physics.ins-det↗

arXiv2026

Algorithmic differentiation for domain specific languages in C++ with expression templates

The application of operator overloading algorithmic differentiation (AD) to computer programs in order to compute the derivative is quite common. But, the replacement of the underlying computational floating point type with the specialized type of an AD tool has two problems. First, the memory structure of the program is changed and floating-point data is interleaved with identifiers from AD. This prevents the compiler from performing optimizations such as SIMD optimizations. Second, the AD tool does not see any domain-specific operations, e.,g. linear algebra operations, that the program uses. This prevents the AD tool from using specialized algorithms in such places. We propose a new AD tool that is tailored to such situations. The memory structure of the primal data is retained by associating an identifier with each entity, e.,g. matrix, and not with each floating point value, e.,g. element of the matrix. Operations on such entities can then be annotated and a generator is used to create the AD overloads. We demonstrate that this approach provides performance comparable to that of other specializations. In addition, the run-time factor is below the theoretical 4.5 of reverse AD for programs that are written purely with linear algebra entities and operations.

cs.MS↗

arXiv2025

Adding complex numbers to expression template algorithmic differentiation tools

Operator overloading algorithmic differentiation (AD) tools are usually only developed for floating-point values. Algorithmic optimization for, e.g., linear systems solvers or matrix-matrix multiplications are often introduced via external functions or manual function specializations. Complex numbers can be viewed as aggregates of two floating-point values on which specialized operations are applied. Typically, these operations can be handled by the regular floating-point operations from the AD tool. Nevertheless, adding the complex number operations to the expression template framework of modern operator overloading AD tools has several benefits. The internal computations of a complex number operation are hidden, and the complex operations do not decompose into single operations. This leads to a smaller memory footprint of the recorded tape and faster gradient computation times. We will discuss these problems, analyze how complex numbers can be integrated into modern operator overloading AD tools, demonstrate an implementation in CoDiPack, and show performance results on a synthetic test case.

cs.MS↗

arXiv2022

Reverse-Mode Automatic Differentiation of Compiled Programs

Tools for algorithmic differentiation (AD) provide accurate derivatives of computer-implemented functions for use in, e. g., optimization and machine learning (ML). However, they often require the source code of the function to be available in a restricted set of programming languages. As a step towards making AD accessible for code bases with cross-language or closed-source components, we recently presented the forward-mode AD tool Derivgrind. It inserts forward-mode AD logic into the machine code of a compiled program using the Valgrind dynamic binary instrumentation framework. This work extends Derivgrind, adding the capability to record the real-arithmetic evaluation tree, and thus enabling operator overloading style reverse-mode AD for compiled programs. We maintain the high level of correctness reported for Derivgrind's forward mode, failing the same few testcases in an extensive test suite for the same well-understood reasons. Runtime-wise, the recording slows down the execution of a compiled 64-bit benchmark program by a factor of about 180.

cs.MS↗

arXiv2022

Forward-Mode Automatic Differentiation of Compiled Programs

Algorithmic differentiation (AD) is a set of techniques that provide partial derivatives of computer-implemented functions. Such a function can be supplied to state-of-the-art AD tools via its source code, or via an intermediate representation produced while compiling its source code. We present the novel AD tool Derivgrind, which augments the machine code of compiled programs with forward-mode AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Access to the source code is required at most for the files in which input and output variables are defined. Derivgrind's versatility comes at the price of scaling the run-time by a factor between 30 and 75, measured on a benchmark based on a numerical solver for a partial differential equation. Results of our extensive regression test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, nearly all of them are academic counterexamples or originate from highly optimized math libraries. As long as differentiating those is avoided, Derivgrind can be applied to an unprecedentedly wide range of cross-language or partially closed-source software with little integration efforts.

cs.MS↗

arXiv2022

Exploration of Differentiability in a Proton Computed Tomography Simulation Framework

Objective. Algorithmic differentiation (AD) can be a useful technique to numerically optimize design and algorithmic parameters by, and quantify uncertainties in, computer simulations. However, the effectiveness of AD depends on how "well-linearizable" the software is. In this study, we assess how promising derivative information of a typical proton computed tomography (pCT) scan computer simulation is for the aforementioned applications. Approach. This study is mainly based on numerical experiments, in which we repeatedly evaluate three representative computational steps with perturbed input values. We support our observations with a review of the algorithmic steps and arithmetic operations performed by the software, using debugging techniques. Main results. The model-based iterative reconstruction (MBIR) subprocedure (at the end of the software pipeline) and the Monte Carlo (MC) simulation (at the beginning) were piecewise differentiable. Jumps in the MBIR function arose from the discrete computation of the set of voxels intersected by a proton path. Jumps in the MC function likely arose from changes in the control flow that affect the amount of consumed random numbers. The tracking algorithm solves an inherently non-differentiable problem. Significance. The MC and MBIR codes are ready for the integration of AD, and further research on surrogate models for the tracking subprocedure is necessary.

physics.med-ph↗

arXiv2021

Event-Based Automatic Differentiation of OpenMP with OpDiLib

We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.

cs.MS↗

arXiv2020

Assign optimization for algorithmic differentiation reuse index management strategies

The identification of primal variables and adjoint variables is usually done via indices in operator overloading algorithmic differentiation tools. One approach is a linear management scheme, which is easy to implement and supports memory optimization for copy statements. An alternative approach performs a reuse of indices, which requires more implementation effort but results in much smaller adjoint vectors. Therefore, the vector mode of algorithmic differentiation scales better with the reuse management scheme. In this paper, we present a novel approach that reuses the indices and allows the copy optimization, thus combining the advantages of the two aforementioned schemes. The new approach is compared to the known approaches on a simple synthetic test case and a real-world example using the computational fluid dynamics solver SU2.

cs.MS↗

arXiv2018

Algorithmic Differentiation for Domain Specific Languages

Algorithmic Differentiation (AD) can be used to automate the generation of derivatives in arbitrary software projects. This will generate maintainable derivatives, that are always consistent with the computation of the software. If a domain specific language (DSL) is used in a software the state of the art approach is to differentiate the DSL library with the same AD tool. The drawback of this solution is the reduced performance since the compiler is no longer able to optimize the e.g. SIMD operations. The new approach in this paper integrates the types and operations of the DSL into the AD tool. It will be an operator overloading tool that is generated from an abstract definition of a DSL. This approach enables the compiler to optimize again e.g. for SIMD operation since all calculations are still performed with the original data types. This will also reduce the required memory for AD since the statements inside the DLS implementation are no longer seen by the AD tool. The implementation is presented in the paper and first results for the performance of the solution are presented.

cs.MS↗

arXiv2017

High-Performance Derivative Computations using CoDiPack

There are several AD tools available, which all implement different strategies for the reverse mode of AD. The major strategies are primal value taping (implemented e.g. by ADOL-c) and Jacobi taping (implemented e.g. by adept and dco/c++). Especially for Jacobi taping, recent advances by using expression templates make this approach very attractive for large scale software. The current implementations are either closed source or miss essential features and flexibility. Therefore, we present the new AD tool CoDiPack (Code Differentiation Package) in this paper. It is specifically designed for a minimal memory consumption and optimal runtime, such that it can be used for the differentiation of large scale software. An essential part of the design of CoDiPack is the modular layout and the recursive data structures, which do not only allow the efficient implementation of the Jacobi taping approach, but will also enable other approaches like the primal value taping or new research ideas. We will also present the performance value of CoDiPack on a generic PDE example and on the SU2 code.

cs.MS↗