arXiv subjects

Carol S. Woodward

Publications and source records attributed to Carol S. Woodward.

At least 19 recordsLinked to original sources

arXiv2026

Accelerating phase-field simulations on exascale computing systems for faster-than-real-time precipitate aging predictions

Three-dimensional phase-field simulations are a gold-standard for microstructure prediction for materials, but their computational cost often limits application-relevant calculations to modest domain sizes and timescales. Here, we present a holistic approach for accelerating Fourier pseudospectral phase-field simulations by combining performance-portable GPU computing, large-scale distributed-memory parallelism, and high-order implicit-explicit time integration within the MEUMAPPS C++ framework. We demonstrate this approach with a 1.5-billion-grid-point simulation of the growth and coarsening of 1,920 gamma'' precipitates in a Ni-Nb-Fe alloy. The simulated seven-hour heat treatment is completed in 5.5 hours, making the calculation faster than real time and an estimated 217-501x faster than a CPU-only, first-order baseline. This performance makes three-dimensional simulations of thousands of interacting precipitates tractable, enabling quantitative studies ocollective microstructural phenomena, large simulation ensembles, and real-time integration into controlling and interpreting experiments. Benchmarking shows single-node GPU speedups of up to 17.5x relative to comparable CPU resources, near-ideal strong scaling to 4096 GPUs for a 12-billion-grid-point problem, and a further 2.6-2.9x acceleration from fourth-order time integration at scientifically relevant error tolerances. These results establish a performance-portable strategy for exploiting leadership-scale GPU systems that is applicable to a broad class of Fourier pseudospectral simulations beyond phase-field models such as fluid dynamics and crystal plasticity simulations.

cond-mat.mtrl-sci↗

arXiv2026

Leveraging higher-order time integration methods for improved computational efficiency in a rainshaft model

Cloud and precipitation microphysics packages in atmospheric general circulation models typically use first-order time integration methods with a large time step, requiring ad hoc limiters and substepping of the sedimentation scheme to prevent numerical instability. We investigate alternative methods for rain microphysical processes in the current Energy Exascale Earth System Model (E3SMv3), provided by the Predicted Particle Properties (P3) scheme. Using an offline rainshaft model containing P3's rain microphysics as a proxy for E3SMv3, we find that rain microphysics is underresolved in time at E3SMv3's default 300s time step. Accurately resolving the rainshaft model processes in time requires an over 100x reduction in time step, increasing wall clock time by 40x. However, higher-order time integrators based on Runge-Kutta methods offer improved solution accuracy for a given computational cost. Adaptive time stepping is key to obtaining computationally efficient microphysics results, eliminating the need for specialized substepping procedures in the sedimentation process. An adaptive second-order Runge-Kutta method approximates a high-temporal-resolution reference solution at only 2.6x the cost of the default P3 scheme; this method achieves high accuracy greater than 15x more efficiently than simply reducing the time step of P3 in the rainshaft model. We also analyze the timescales of the rain processes to obtain insight about the maximum time step each process is able to take while maintaining stability and accuracy, and about how individual processes should be grouped together for most efficient results.

physics.ao-ph↗

arXiv2026

Computing nuclear response functions with time-dependent coupled-cluster theory

We compute nuclear response functions by solving the time-dependent A-body Schrödinger equation, recording the time-dependent transition moment and extracting spectral information via Fourier transforms. The solution of the time-dependent many-body problem accounts for correlations on top of the mean field by taking advantage of a time-dependent formulation of coupled-cluster theory. As a validation, we focus on electric dipole transitions in $^4$He and $^{16}$O and compare moments of the response function distribution to the results of an equivalent static framework, finding negligible discrepancies. We investigate how proton and neutron densities evolve in time, and we see the traditional picture of soft and giant dipole resonances as collective oscillations of protons and neutrons emerging from our calculations in $^{16}$O and $^{24}$O. This method also allows us to investigate the behavior of the nucleus in the presence of a strong electric field. In that regime, the behavior of the system becomes chaotic. Qualitatively, the spectral information obtained in this limit is in line with previous time-dependent mean-field results.

nucl-th↗

arXiv2026

New Time Integrators and Capabilities in SUNDIALS Versions 6.2.0-7.4.0

SUNDIALS is a well-established numerical library that provides robust and efficient time integrators and nonlinear solvers. This paper overviews several significant improvements and new features added over the last three years to support scientific simulations run on high-performance computing systems. Notably, three new classes of one-step methods have been implemented: low storage Runge-Kutta, symplectic partitioned Runge-Kutta, and operator splitting. In addition, we describe new time step adaptivity support for multirate methods, adjoint sensitivity analysis capabilities for explicit Runge-Kutta methods, additional options for Anderson acceleration in nonlinear solvers, and improved error handling and logging.

cs.MS↗

arXiv2025

Adaptive Time Stepping for the Two-Time Integro-Differential Kadanoff-Baym Equations

The non-equilibrium Green's function gives access to one-body observables for quantum systems. Of particular interest are quantities such as density, currents, and absorption spectra which are important for interpreting experimental results in quantum transport and spectroscopy. We present an integration scheme for the Green's function's equations of motion, the Kadanoff-Baym equations (KBE), which is both adaptive in the time integrator step size and method order as well as the history integration order. We analyze the importance of solving the KBE self-consistently and show that adapting the order of history integral evaluation is important for obtaining accurate results. To examine the efficiency of our method, we compare runtimes to a state of the art fixed time step integrator for several test systems and show an order of magnitude speedup at similar levels of accuracy.

physics.comp-ph↗

arXiv2024

Advancing Simulations of Coupled Electron and Phonon Nonequilibrium Dynamics Using Adaptive and Multirate Time Integration

Electronic structure calculations in the time domain provide a deeper understanding of nonequilibrium dynamics in materials. The real-time Boltzmann equation (rt-BTE), used in conjunction with accurate interactions computed from first principles, has enabled reliable predictions of coupled electron and lattice dynamics. However, the timescales and system sizes accessible with this approach are still limited, with two main challenges being the different timescales of electron and phonon interactions and the cost of computing collision integrals. As a result, only a few examples of these calculations exist, mainly for two-dimensional (2D) materials. Here we leverage adaptive and multirate time integration methods to achieve a major step forward in solving the coupled rt-BTEs for electrons and phonons. Relative to conventional (non-adaptive) time-stepping, our approach achieves a 10x speedup for a given target accuracy, or greater accuracy by 3-6 orders of magnitude for the same computational cost, enabling efficient calculations in both 2D and bulk materials. This efficiency is showcased by computing the coupled electron and lattice dynamics in graphene up to $\sim$100 ps, as well as modeling ultrafast lattice dynamics and thermal diffuse scattering maps in a bulk material (silicon). These results open new opportunities for quantitative studies of nonequilibrium physics in materials.

cond-mat.mtrl-sci↗

arXiv2024

Towards the Use of Anderson Acceleration in Coupled Transport-Gyrokinetic Turbulence Simulations

Predicting the behavior of a magnetically confined fusion plasma over long time periods requires methods that can bridge the difference between transport and turbulent time scales. The nonlinear transport solver, Tango, enables simulations of very long times, in particular to steady state, by advancing each process independently with different time step sizes and couples them through a relaxed iteration scheme. We examine the use of Anderson Acceleration (AA) to reduce the total number of coupling iterations required by interfacing Tango with the AA implementation, including several extensions to AA, provided by the KINSOL nonlinear solver package in SUNDIALS. The ability to easily enable and adjust algorithmic options through KINSOL allows for rapid experimentation to evaluate different approaches with minimal effort. Additionally, we leverage the GPTune library to automate the optimization of algorithmic parameters within KINSOL. We show that AA can enable faster convergence in stiff and very stiff tests cases without noise present and in all cases, including with noisy fluxes, increases robustness and reduces sensitivity to the choice of relaxation strength.

math.NA↗

arXiv2024

SUNDIALS Time Integrators for Exascale Applications with Many Independent ODE Systems

Many complex systems can be accurately modeled as a set of coupled time-dependent partial differential equations (PDEs). However, solving such equations can be prohibitively expensive, easily taxing the world's largest supercomputers. One pragmatic strategy for attacking such problems is to split the PDEs into components that can more easily be solved in isolation. This operator splitting approach is used ubiquitously across scientific domains, and in many cases leads to a set of ordinary differential equations (ODEs) that need to be solved as part of a larger "outer-loop" time-stepping approach. The SUNDIALS library provides a plethora of robust time integration algorithms for solving ODEs, and the U.S. Department of Energy Exascale Computing Project (ECP) has supported its extension to applications on exascale-capable computing hardware. In this paper, we highlight some SUNDIALS capabilities and its deployment in combustion and cosmology application codes (Pele and Nyx, respectively) where operator splitting gives rise to numerous, small ODE systems that must be solved concurrently.

math.NA↗

arXiv2024

Numerical coupling of aerosol emissions, dry removal, and turbulent mixing in the E3SM Atmosphere Model version 1 (EAMv1), part II: a semi-discrete error analysis framework for assessing coupling schemes

This paper complements the empirical justification of the revised scheme in Part I of this work with a mathematical justification leveraging a semi-discrete analysis framework for assessing the splitting error of process coupling methods. The novelty of the framework is that splitting error is distinguished from the process time integration errors, i.e., the errors caused by discrete time integration of individual processes, leading to expressions that are more easily interpreted utilizing existing physical understanding of the processes that the terms represent. This application of this framework to dust life cycle in EAMv1 showcases such an interpretation, using the leading-order splitting error that results from the framework to confirm (i) that the original EAMv1 scheme artificially strengthens the effect of dry removal processes, and (ii) that the revised splitting reduces that artificial strengthening. While the error analysis framework is presented in the context of the dust life cycle in EAMv1, the framework can be broadly leveraged to evaluate process coupling schemes, both in other physical problems and for any number of processes. This framework will be particularly powerful when the various process implementations support a variety of time integration approaches. Whereas traditional local truncation error approaches require separate consideration of each combination of time integration methods, this framework enables evaluation of coupling schemes independent of particular time integration approaches for each process while still allowing for the incorporation of these specific time integration errors if so desired. The framework also explains how the splitting error terms result from (i) the integration of individual processes in isolation from other processes, and (ii) the choices of input state and timestep size for the isolated integration of processes.

math.NA↗

arXiv2023

Numerical coupling of aerosol emissions, dry removal, and turbulent mixing in the E3SM Atmosphere Model version 1 (EAMv1), part I: dust budget analyses and the impacts of a revised coupling scheme

An earlier study evaluating the dust life cycle in the Energy Exascale Earth System Model (E3SM) Atmosphere Model version 1 (EAMv1) has revealed that the simulated global mean dust lifetime is substantially shorter when higher vertical resolution is used, primarily due to significant strengthening of dust dry removal in source regions. This paper demonstrates that the sequential splitting of aerosol emissions, dry removal, and turbulent mixing in the model's time integration loop, especially the calculation of dry removal after surface emissions and before turbulent mixing, is the primary reason for the vertical resolution sensitivity reported in that earlier study. Based on this reasoning, we propose a simple revision to the numerical process coupling scheme, which moves the application of the surface emissions to after dry removal and before turbulent mixing. The revised scheme allows newly emitted particles to be transported aloft by turbulence before being removed from the atmosphere, and hence better resembles the dust life cycle in the real world. Sensitivity experiments are conducted and analyzed to evaluate the impact of the revised coupling on the simulated aerosol climatology in EAMv1.

physics.comp-ph↗

arXiv2022

ARKODE: a flexible IVP solver infrastructure for one-step methods

We describe the ARKODE library of one-step time integration methods for ordinary differential equation (ODE) initial-value problems (IVPs). In addition to providing standard explicit and diagonally implicit Runge--Kutta methods, ARKODE also supports one-step methods designed to treat additive splittings of the IVP, including implicit-explicit (ImEx) additive Runge--Kutta methods and multirate infinitesimal (MRI) methods. We present the role of ARKODE within the SUNDIALS suite of time integration and nonlinear solver libraries, the core ARKODE infrastructure for utilities common to large classes of one-step methods, as well as its use of ``time stepper'' modules enabling easy incorporation of novel algorithms into the library. Numerical results show example problems of increasing complexity, highlighting the algorithmic flexibility afforded through this infrastructure, and include a larger multiphysics application leveraging multiple algorithmic features from ARKODE and SUNDIALS.

cs.MS↗

arXiv2022

Performance of explicit and IMEX MRI multirate methods on complex reactive flow problems within modern parallel adaptive structured grid frameworks

Large-scale multiphysics simulations are computationally challenging due to the coupling of multiple processes with widely disparate time scales. The advent of exascale computing systems exacerbates these challenges, since these enable ever increasing size and complexity. Recently, there has been renewed interest in developing multirate methods as a means to handle the large range of time scales, as these methods may afford greater accuracy and efficiency than more traditional approaches of using IMEX and low-order operator splitting schemes. However, there have been few performance studies that compare different classes of multirate integrators on complex application problems. We study the performance of several newly developed multirate infinitesimal (MRI) methods, implemented in the SUNDIALS solver package, on two reacting flow model problems built on structured mesh frameworks. The first model revisits the work of Emmet et al. (2014) on a compressible reacting flow problem with complex chemistry that is implemented using BoxLib but where we now include comparisons between a new explicit MRI scheme with the multirate spectral deferred correction (SDC) methods in the original paper. The second problem uses the same complex chemistry as the first problem, combined with a simplified flow model, but run at a large spatial scale where explicit methods become infeasible due to stability constraints. Two recently developed implicit-explicit MRI multirate methods are tested. These methods rely on advanced features of the AMReX framework on which the model is built, such as multilevel grids and multilevel preconditioners. The results from these two problems show that MRI multirate methods can offer significant performance benefits on complex multiphysics application problems and that these methods may be combined with advanced spatial discretization to compound the advantages of both.

math.NA↗

arXiv2021

Enabling GPU Accelerated Computing in the SUNDIALS Time Integration Library

As part of the Exascale Computing Project (ECP), a recent focus of development efforts for the SUite of Nonlinear and DIfferential/ALgebraic equation Solvers (SUNDIALS) has been to enable GPU-accelerated time integration in scientific applications at extreme scales. This effort has resulted in several new GPU-enabled implementations of core SUNDIALS data structures, support for programming paradigms which are aware of the heterogeneous architectures, and the introduction of utilities to provide new points of flexibility. In this paper, we discuss our considerations, both internal and external, when designing these new features and present the features themselves. We also present performance results for several of the features on the Summit supercomputer and early access hardware for the Frontier supercomputer, which demonstrate negligible performance overhead resulting from the additional infrastructure and significant speedups when using both NVIDIA and AMD GPUs.

cs.DC↗

arXiv2021

Performance of Low Synchronization Orthogonalization Methods in Anderson Accelerated Fixed Point Solvers

Anderson Acceleration (AA) is a method to accelerate the convergence of fixed point iterations for nonlinear, algebraic systems of equations. Due to the requirement of solving a least squares problem at each iteration and a reliance on modified Gram-Schmidt for updating the iteration space, AA requires extra costly synchronization steps for global reductions. Moreover, the number of reductions in each iteration depends on the size of the iteration space. In this work, we introduce three low synchronization orthogonalization algorithms into AA within SUNDIALS that reduce the total number of global reductions per iteration to a constant of 2 or 3, independent of the size of the iteration space. A performance study demonstrates the reduced time required by the new algorithms at large processor counts with CPUs and demonstrates the predicted performance on multi-GPU architectures. Most importantly, we provide convergence and timing data for multiple numerical experiments to demonstrate reliability of the algorithms within AA and improved performance at parallel strong-scaling limits.

math.NA↗

arXiv2021

Enabling New Flexibility in the SUNDIALS Suite of Nonlinear and Differential/Algebraic Equation Solvers

In recent years, the SUite of Nonlinear and DIfferential/ALgebraic equation Solvers (SUNDIALS) has been redesigned to better enable the use of application-specific and third-party algebraic solvers and data structures. Throughout this work, we have adhered to specific guiding principles that minimized the impact to current users while providing maximum flexibility for later evolution of solvers and data structures. The redesign was done through the addition of new linear and nonlinear solvers classes, enhancements to the vector class, and the creation of modern Fortran interfaces. The vast majority of this work has been performed "behind-the-scenes," with minimal changes to the user interface and no reduction in solver capabilities or performance. These changes allow SUNDIALS users to more easily utilize external solver libraries and create highly customized solvers, enabling greater flexibility on extreme-scale, heterogeneous computational architectures.

cs.MS↗

arXiv2020

Implicit multirate GARK methods

This work considers multirate generalized-structure additively partitioned Runge-Kutta (MrGARK) methods for solving stiff systems of ordinary differential equations (ODEs) with multiple time scales. These methods treat different partitions of the system with different timesteps for a more targeted and efficient solution compared to monolithic single rate approaches. With implicit methods used across all partitions, methods must find a balance between stability and the cost of solving nonlinear equations for the stages. In order to characterize this important trade-off, we explore multirate coupling strategies, problems for assessing linear stability, and techniques to efficiently implement Newton iterations for stage equations. Unlike much of the existing multirate stability analysis which is limited in scope to particular methods, we present general statements on stability and describe fundamental limitations for certain types of multirate schemes. New implicit multirate methods up to fourth order are derived, and their accuracy and efficiency properties are verified with numerical tests.

math.NA↗

arXiv2019

Evaluation of Implicit-Explicit Additive Runge-Kutta Integrators for the HOMME-NH Dynamical Core

The nonhydrostatic High Order Method Modeling Environment (HOMME-NH) atmospheric dynamical core supports acoustic waves that propagate significantly faster than the advective wind speed, thus greatly limiting the timestep size that can be used with standard explicit time-integration methods. Resolving acoustic waves is unnecessary for accurate climate and weather prediction. This numerical stiffness is addressed herein by considering implicit-explicit additive Runge-Kutta (ARK IMEX) methods that can treat the acoustic waves in a stable manner without requiring implicit treatment of non-stiff modes. Various ARK IMEX methods are evaluated for their efficiency in producing accurate solutions, ability to take large timestep sizes, and sensitivity to grid cell length ratio. Both the Gravity Wave test and Baroclinic Instability test from the 2012 Dynamical Core Model Intercomparison Project (DCMIP) are used to recommend 5 of the 27 ARK IMEX methods for use in HOMME-NH.

math.NA↗

arXiv2019

SUNDIALS Multiphysics+MPIManyVector Performance Testing

In this report we document performance test results on a SUNDIALS-based multiphysics demonstration application. We aim to assess the large-scale parallel performance of new capabilities that have been added to the SUNDIALS suite of time integrators and nonlinear solvers in recent years under funding from both the Exascale Computing Project (ECP) and the Scientific Discovery through Advanced Scientific (SciDAC) program, specifically: (a) SUNDIALS' new MPIManyVector module, that allows extreme flexibility in how a solution "vector" is staged on computational resources, (b) ARKode's new multirate integration module, MRIStep, allowing high-order accurate calculations that subcycle "fast" processes within "slow" ones, (c) SUNDIALS' new flexible linear solver interfaces, that allow streamlined specification of problem-specific linear solvers, and (d) SUNDIALS' new N_Vector additions of "fused" vector operations (to increase arithmetic intensity) and separation of reduction operations into "local" and "global" versions (to reduce latency by combining multiple reductions into a single MPI_Allreduce call). We anticipate that subsequent reports will extend this work to investigate a variety of other new features, including SUNDIALS' generic SUNNonlinearSolver interface and accelerator-enabled N_Vector modules, and upcoming MRIStep extensions to support custom "fast" integrators (that leverage problem structure) and IMEX integration of the "slow" time scale (to add diffusion).

cs.DC↗