arXiv subjects

Eric Phipps

Publications and source records attributed to Eric Phipps.

9 recordsLinked to original sources

arXiv2025

Performance Portable Gradient Computations Using Source Transformation

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and nonlinear solvers. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, resulting in widespread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures such as GPUs, which have limited memory capabilities and require massive thread-level concurrency. Portable scientific codes rely on domain specific programming models such as Kokkos making AD for such codes even more complex. In this paper, we will investigate source transformation-based automatic differentiation using Clad to automatically generate portable and efficient gradient computations of Kokkos-based code. We discuss the modifications of Clad required to differentiate Kokkos abstractions. We will illustrate the feasibility of our proposed strategy by comparing the wall-clock time of the generated gradient code with the wall-clock time of the input function on different cutting edge GPU architectures such as NVIDIA H100, AMD MI250x, and Intel Ponte Vecchio GPU. For these three architectures and for the considered example, evaluating up to 10 000 entries of the gradient only took up to 2.17x the wall-clock time of evaluating the input function.

cs.MS↗

arXiv2025

Trilinos: Enabling Scientific Computing Across Diverse Hardware Architectures at Scale

Trilinos is a community-developed, open-source software framework that facilitates building large-scale, complex, multiscale, multiphysics simulation code bases for scientific and engineering problems. Since the Trilinos framework has undergone substantial changes to support new applications and new hardware architectures, this document is an update to ``An Overview of the Trilinos project'' by Heroux et al. (ACM Transactions on Mathematical Software, 31(3):397-423, 2005). It describes the design of Trilinos, introduces its new organization in product areas, and highlights established and new features available in Trilinos. Particular focus is put on the modernized software stack based on the Kokkos ecosystem to deliver performance portability across heterogeneous hardware architectures. This paper also outlines the organization of the Trilinos community and the contribution model to help onboard interested users and contributors.

cs.MS↗

arXiv2021

Streaming Generalized Canonical Polyadic Tensor Decompositions

In this paper, we develop a method which we call OnlineGCP for computing the Generalized Canonical Polyadic (GCP) tensor decomposition of streaming data. GCP differs from traditional canonical polyadic (CP) tensor decompositions as it allows for arbitrary objective functions which the CP model attempts to minimize. This approach can provide better fits and more interpretable models when the observed tensor data is strongly non-Gaussian. In the streaming case, tensor data is gradually observed over time and the algorithm must incrementally update a GCP factorization with limited access to prior data. In this work, we extend the GCP formalism to the streaming context by deriving a GCP optimization problem to be solved as new tensor data is observed, formulate a tunable history term to balance reconstruction of recently observed data with data observed in the past, develop a scalable solution strategy based on segregated solves using stochastic gradient descent methods, describe a software implementation that provides performance and portability to contemporary CPU and GPU architectures and integrates with Matlab for enhanced useability, and demonstrate the utility and performance of the approach and software on several synthetic and real tensor data sets.

math.NA↗

arXiv2019

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

In this paper, we develop software for decomposing sparse tensors that is portable to and performant on a variety of multicore, manycore, and GPU computing architectures. The result is a single code whose performance matches optimized architecture-specific implementations. The key to a portable approach is to determine multiple levels of parallelism that can be mapped in different ways to different architectures, and we explain how to do this for the matricized tensor times Khatri-Rao product (MTTKRP) which is the key kernel in canonical polyadic tensor decomposition. Our implementation leverages the Kokkos framework, which enables a single code to achieve high performance across multiple architectures that differ in how they approach fine-grained parallelism. We also introduce a new construct for portable thread-local arrays, which we call compile-time polymorphic arrays. Not only are the specifics of our approaches and implementation interesting for tuning tensor computations, but they also provide a roadmap for developing other portable high-performance codes. As a last step in optimizing performance, we modify the MTTKRP algorithm itself to do a permuted traversal of tensor nonzeros to reduce atomic-write contention. We test the performance of our implementation on 16- and 68-core Intel CPUs and the K80 and P100 NVIDIA GPUs, showing that we are competitive with state-of-the-art architecture-specific codes while having the advantage of being able to run on a variety of architectures.

cs.MS↗

arXiv2017

Surrogate-based Ensemble Grouping Strategies for Embedded Sampling-based Uncertainty Quantification

The embedded ensemble propagation approach introduced in [49] has been demonstrated to be a powerful means of reducing the computational cost of sampling-based uncertainty quantification methods, particularly on emerging computational architectures. A substantial challenge with this method however is ensemble-divergence, whereby different samples within an ensemble choose different code paths. This can reduce the effectiveness of the method and increase computational cost. Therefore grouping samples together to minimize this divergence is paramount in making the method effective for challenging computational simulations. In this work, a new grouping approach based on a surrogate for computational cost built up during the uncertainty propagation is developed and applied to model diffusion problems where computational cost is driven by the number of (preconditioned) linear solver iterations. The approach is developed within the context of locally adaptive stochastic collocation methods, where a surrogate for the number of linear solver iterations, generated from previous levels of the adaptive grid generation, is used to predict iterations for subsequent samples, and group them based on similar numbers of iterations. The effectiveness of the method is demonstrated by applying it to highly anisotropic diffusion problems with a wide variation in solver iterations from sample to sample. It extends the parameter-based grouping approach developed in [17] to more general problems without requiring detailed knowledge of how the uncertain parameters affect the simulation's cost, and is also less intrusive to the simulation code.

stat.CO↗

arXiv2012

Reduced chaos expansions with random coefficients in reduced-dimensional stochastic modeling of coupled problems

Coupled problems with various combinations of multiple physics, scales, and domains can be found in numerous areas of science and engineering. A key challenge in the formulation and implementation of corresponding coupled models is to facilitate communication of information across physics, scale, and domain interfaces, as well as between iterations of solvers used for response computations. In a probabilistic context, any information that is to be communicated between subproblems or iterations should be characterized by an appropriate probabilistic representation. In this work, we consider stochastic coupled problems whose subproblems involve only uncertainties that are statistically independent of one another; for these problems, we present a characterization of the exchanged information by using a reduced chaos expansion with random coefficients. This expansion provides a reduced-dimensional representation of the exchanged information, while maintaining segregation between sources of uncertainty that stem from different subproblems. Further, we present a measure transformation that allows stochastic expansion methods to exploit this dimension reduction to obtain an efficient solution of subproblems in a reduced-dimensional space. We show that owing to the uncertainty source segregation, requisite orthonormal polynomials and quadrature rules can be readily obtained by tensorization. Finally, the proposed methodology is demonstrated by applying it to a multiphysics problem in nuclear engineering.

math.AP↗

arXiv2012

Efficient Expression Templates for Operator Overloading-based Automatic Differentiation

Expression templates are a well-known set of techniques for improving the efficiency of operator overloading-based forward mode automatic differentiation schemes in the C++ programming language by translating the differentiation from individual operators to whole expressions. However standard expression template approaches result in a large amount of duplicate computation, particularly for large expression trees, degrading their performance. In this paper we describe several techniques for improving the efficiency of expression templates and their implementation in the automatic differentiation package Sacado. We demonstrate their improved efficiency through test functions as well as their application to differentiation of a large-scale fluid dynamics simulation code.

cs.MS↗

arXiv2012

Dimension reduction in stochastic modeling of coupled problems

Coupled problems with various combinations of multiple physics, scales, and domains are found in numerous areas of science and engineering. A key challenge in the formulation and implementation of corresponding coupled numerical models is to facilitate the communication of information across physics, scale, and domain interfaces, as well as between the iterations of solvers used for response computations. In a probabilistic context, any information that is to be communicated between subproblems or iterations should be characterized by an appropriate probabilistic representation. Although the number of sources of uncertainty can be expected to be large in most coupled problems, our contention is that exchanged probabilistic information often resides in a considerably lower dimensional space than the sources themselves. This work thus presents an investigation into the characterization of the exchanged information by a reduced-dimensional representation and, in particular, by an adaptation of the Karhunen-Loeve decomposition. The effectiveness of the proposed dimension-reduction methodology is analyzed and demonstrated through a multiphysics problem relevant to nuclear engineering.

math.AP↗

arXiv2012

Measure transformation and efficient quadrature in reduced-dimensional stochastic modeling of coupled problems

Coupled problems with various combinations of multiple physics, scales, and domains are found in numerous areas of science and engineering. A key challenge in the formulation and implementation of corresponding coupled numerical models is to facilitate the communication of information across physics, scale, and domain interfaces, as well as between the iterations of solvers used for response computations. In a probabilistic context, any information that is to be communicated between subproblems or iterations should be characterized by an appropriate probabilistic representation. Although the number of sources of uncertainty can be expected to be large in most coupled problems, our contention is that exchanged probabilistic information often resides in a considerably lower dimensional space than the sources themselves. In this work, we thus use a dimension-reduction technique for obtaining the representation of the exchanged information. The main subject of this work is the investigation of a measure-transformation technique that allows implementations to exploit this dimension reduction to achieve computational gains. The effectiveness of the proposed dimension-reduction and measure-transformation methodology is demonstrated through a multiphysics problem relevant to nuclear engineering.

math.AP↗