Searcharxiv⌕ Search

arXiv subjects

Kyle E. Niemeyer

Publications and source records attributed to Kyle E. Niemeyer.

At least 19 recordsLinked to original sources

Uncertainty quantification of reacting fluids interacting with porous media using a hybrid physics-based and data-driven approach

Accurately simulating coupled physical processes under uncertainty is essential for reliable modeling and design in performance-critical applications such as combustion systems. Ablative heat shield design, as a specific example of this class, involves modeling multi-physics interactions between reacting flows and a porous material. Repeatedly evaluating these models to quantify parametric uncertainties would be prohibitively computationally expensive. In this work, we combine physics-based modeling using a single-domain approach with data-driven reduced-order modeling to quantify uncertainty via the operator inference method. The detailed physics-based simulations reproduce the measured surface temperature of an object exposed to high-enthalpy flow in a plasma wind tunnel experiment within 5%. We further use the model to demonstrate the effect of complex flow situations on the dynamic interactions between the porous heat shield material and the surrounding gas. The parametric reduced-order model, built on physics-based simulation data, successfully captures variations in quantities of interest resulting from changes in the permeability and heat transfer coefficient of the porous material in two separate studies: solid-fuel combustion with emission of buoyant reacting plumes in quiescent air, and ablation in a wind tunnel.

physics.comp-ph↗

Hybrid Delta Tracking Schemes Using a Track-Length Estimator

In Monte Carlo radiation transport calculations, Woodcock-delta tracking is a common alternative to the more popular surface tracking technique. In this work we introduce a delta-tracking algorithm that tallies fluxes to a structured rectilinear mesh using the track-length estimator. This development also enables hybrid surface-delta tracking algorithms, because the track-length tally can be used everywhere for scalar flux estimation regardless of which tracking algorithm is employed. We use this tallying technique to develop a novel hybrid-in-energy method. We also implement a hybrid-in-material method, like what is implemented in Serpent2. We demonstrate that these delta tracking algorithms can be used in conjunction with continuously moving surfaces. We compare these methods showing figures of merit on four time-dependent problems (multi-group and continuous energy) solved with CPU- and GPU-based computers. Our implementation of delta tracking with a track length tally modestly improves figures of merit compared to standard delta tracking with a collision estimator and surface tracking with a track length estimator (1.5X 2.5X) for a problem with significant void regions. For both multi-group and continuous energy pressurized water reactor benchmarks, standard delta tracking with a collision estimator performs best. Hybrid-in-energy methods show significant improvements (7X-11X) for a continuous energy reactor benchmark problem.

physics.comp-ph↗

The Monte Carlo Computational Summit -- October 25 & 26, 2023 -- Notre Dame, Indiana, USA

The Monte Carlo Computational Summit was held on the campus of the University of Notre Dame in South Bend, Indiana, USA on 25--26 October 2023. The goals of the summit were to discuss algorithmic and software alterations required for successfully porting respective code bases to exascale-class computing hardware, compare software engineering techniques used by various code teams, and consider the adoption of industry-standard benchmark problems to better facilitate code-to-code performance comparisons. A large portion of the meeting included candid discussions of direct experiences with approaches that have and have not worked. Participants reported that identifying and implementing suitable Monte Carlo algorithms for GPUs continues to be a sticking point. They also report significant difficulty porting existing algorithms between GPU APIs (specifically Nvidia CUDA to AMD ROCm). To better compare code-to-code performance, participants decided to design a C5G7-like benchmark problem with a defined figure of merit, with the expectation of adding more benchmarks in the future. Problem specifications and results will eventually be hosted in a public repository and will be open to submissions by all Monte Carlo transport codes capable of running the benchmark problem. The participants also identified the need to explore the intermediate and long-term future of the Monte Carlo neutron transport community and how best to modernize and contextualize Monte Carlo as a useful tool in modern industry. Overall the summit was considered to be a success by the organizers and participants, and the group shared a strong desire for future, potentially larger, Monte Carlo summits.

physics.comp-ph↗

Comparing the Performance of MC/DC's on-GPU Event-based Processing Methods in Multigroup and Continuous-energy Problems

Monte Carlo / Dynamic Code (MC/DC) is a portable Monte Carlo neutron transport package for rapid numerical methods exploration in heterogeneous and HPC contexts, developed under the auspices of the Center for Exascale Monte Carlo Neutron Transport (CEMeNT). To support execution on GPUs, MC/DC delegates resource and execution management to Harmonize (another CEMeNT software project). In this paper, we describe and compare the performance of the two methods that Harmonize currently provides: a stack-based method and a distributed, asynchronous method. As part of this investigation, we analyze the performance of both methods under the 3D C5G7 k-eigenvalue benchmark problem and a continuous-energy infinite pin cell problem, as run across 4 NVIDIA Tesla V100s. We find that the asynchronous method exhibits stronger early scaling compared to the stack-based method in the 3D C5G7 benchmark. We also found that the asynchronous method exhibits mixed performance relative to the stack-based method in the continuous-energy problem, depending upon tally resolution, particle count, and transport loop decomposition.

physics.comp-ph↗

Performance Portable Monte Carlo Neutron Transport in MCDC via Numba

Finding a software engineering approach that allows for portability, rapid development, and open collaboration for high-performance computing on GPUs and CPUs is a challenge. We implement a portability scheme using the Numba compiler for Python in Monte Carlo / Dynamic Code (MC/DC), a new neutron transport application for rapidly developing Monte Carlo. Using this scheme, we have built MC/DC as an application that can run as a pure Python, compiled CPU, or compiled GPU solver. In GPU mode, we use Numba paired with an asynchronous GPU scheduler called Harmonize to increase GPU performance. We present performance results (including weak scaling up to 256 nodes) for a time-dependent problem on both CPUs and GPUs and compare favorably to a production C++ code.

physics.comp-ph↗

Enabling GPU Portability into the Numba-JITed Monte Carlo Particle Transport Code MC/DC

The Center for Exascale Monte Carlo Neutron Transport is developing Monte Carlo / Dynamic Code (MC/DC) as a portable Monte Carlo neutron transport package for rapid numerical methods exploration on CPU- and GPU-based high-performance computers. In this paper, we describe MC/DC's current event-based GPU algorithm as well as the just-in-time (JIT) compilation scheme we use to enable GPU operability on Nvidia and AMD GPUs from MC/DC's Python source. To analyze performance, we conduct runtime tests of the C5G7 k-eigenvalue benchmark problem and a continuous-energy infinite pin cell on Nvidia Tesla V100 GPU, AMD MI250X GPU, and the AMD MI300A APU and make comparison to a dual-socket Intel Xeon Sapphire Rapid CPU node. We found that for the multi-group C5G7 benchmark problem, we respectively see a 15$\times$, 0.7$\times$, 12$\times$ speedup on a V100, MI250X, and MI300A over 112 Intel Xeon CPU cores. For the continuous-energy infinite pin-cell benchmark, we found speedups of 5$\times$, 3$\times$, 4$\times$ on a V100, MI250X, and MI300A, respectively, over the same CPU node.

physics.comp-ph↗

One-Cell Inversion for Solving Higher-Order Time-Dependent Radiation Transport on GPUs

To find deterministic solutions to the transient discrete-ordinates neutron-transport equation, source iterations (SI) are typically used to lag the scattering (and fission) source terms from subsequent iterations. For Cartesian geometries in one dimension, SI is parallel over the number of angles but not spatial cells; this is a disadvantage for many-core compute architectures like graphics processing units. One-cell inversion (OCI) is a class of alternative iterative methods that allow space-parallel radiation transport on heterogeneous compute architectures. For OCI, previous studies have shown that, in steady-state computations, spectral radius tends to unity when cells are optically thin regardless of the scattering ratio. In this work, we analyze how the convergence rate of an OCI scheme behaves when used for time-dependent neutron transport computations. We derive a second-order space-time discretization method from the simple corner balance and multiple balance time discretization schemes and show via Fourier analysis that it is unconditionally stable through time. Then, we derive and numerically solve the Fourier systems for both OCI and SI splittings of our discretization, showing that small mean-free times improve the spectral radius of OCI more than SI, and that spectral radius for OCI tends to zero as mean free time gets smaller. We extend both solvers to be energy dependent (using the multigroup assumption) and implement on an AMD MI250X using vendor-supplied batched LAPACK solvers. Smaller time steps improve the relative performance of OCI over SI, and, even when OCI requires more iterations to converge a problem, those iterations can be done much faster on a GPU. This leads to OCI performing better overall than SI on GPUs.

physics.comp-ph↗

A single-domain approach for modeling flow in and around porous media applied to buoyant reacting plume formation and ignition

Many processes involve mixed porous-solid fluid domains where fluid flow, heat transfer, and chemical reactions interact over disparate length scales, such as the combustion of multi-species solid fuels. Although many studies have concentrated on detailed physics within the fluid or porous phase, few consider both phases, in part due to the challenge in determining suitable boundary conditions between the regions, particularly in turbulent flows where eddies might penetrate the pores. Here, we apply a single-domain approach that eliminates the need for boundary conditions at the interface, and simulate scenarios involving porous solids and a surrounding fluid. Similar to large eddy simulation, the method averages properties over a small spatial volume -- but over the entire domain. We focus on ignition and related interfacial phenomena. After verifying and validating the model, we examine the emission of buoyant reacting plumes from the surface of a heated solid and the near-field flow dynamics. The results indicate flow instabilities similar to Rayleigh--Taylor and Kelvin--Helmholtz phenomena. A combination of viscous and baroclinic torques triggers vorticity generation near the interface and its growth in the surrounding fluid region. Furthermore, we explore the effect of interface morphology, finding that geometrical characteristics such as asymmetry or gap size can alter ignition time and location, or even suppress it. Asymmetry-induced oscillations initially cause negative heat fluxes, which prevent the temperature from reaching the critical level necessary to trigger ignition. These behaviors could significantly influence the mixing of oxidizer and fuel, ignition processes, and fire propagation.

physics.flu-dyn↗

A Structured Framework for Predicting Sustainable Aviation Fuel Properties using Liquid-Phase FTIR and Machine Learning

Sustainable aviation fuels have the potential for reducing emissions and environmental impact. To help identify viable sustainable aviation fuels and accelerate research, several machine learning models have been developed to predict relevant physiochemical properties. However, many of the models have limited applicability, leverage data from complex analytical techniques with confined spectral ranges, or use feature decomposition methods that have limited interpretability. Using liquid-phase Fourier Transform Infrared (FTIR) spectra, this study presents a structured method for creating accurate and interpretable property prediction models for neat molecules, aviation fuels, and blends. Liquid-phase FTIR spectra measurements can be collected quickly and consistently, offering high reliability, sensitivity, and component specificity using less than 2 mL of sample. The method first decomposes FTIR spectra into fundamental building blocks using Non-negative Matrix Factorization (NMF) to enable scientific analysis of FTIR spectra attributes and fuel properties. The NMF features are then used to create five ensemble models for predicting final boiling point, flash point, freezing point, density at 15C, and kinematic viscosity at -20C. All models were trained using experimental property data from neat molecules, aviation fuels, and blends. The models accurately predict properties while enabling interpretation of relationships between compositional elements of a fuel, such as functional groups or chemical classes, and its properties. To support sustainable aviation fuel research and development, the models and data are available on an interactive web tool.

physics.chem-ph↗

A Fully Explicit Integrator for Modeling Astrophysical Reactive Flows

Simulating complex astrophysical reacting flows is computationally expensive -- reactions are stiff and typically require implicit integration methods. The reaction update is often the most expensive part of a simulation, which motivates the exploration of more economical methods. In this research note, we investigate how the explicit Runge--Kutta--Chebyshev (RKC) method performs compared to an implicit method when applied to astrophysical reactive flows. These integrators are applied to simulations of X-ray bursts arising from unstable thermonuclear burning of accreted fuel on the surface of neutron stars. We show that the RKC method performs with similar accuracy to our traditional implicit integrator, but is more computationally efficient when run on CPUs.

astro-ph.IM↗

Exploring One-Cell Inversion Method for Transient Transport on GPU

To find deterministic solutions to the transient $S_N$ neutron transport equation, iterative schemes are typically used to treat the scattering (and fission) source terms. We explore the one-cell inversion iteration scheme to do this on the GPU and make comparisons to a source iteration scheme. We examine convergence behavior, through the analysis of spectral radii, of both one-cell inversion and source iterations. To further boost the GPU parallel efficiency, we derive a higher-order discretization method, simple corner balance (in space) and multiple balance (in time), to add more work to the threads and gain accuracy. Fourier analysis on this higher-order numerical method shows that it is unconditionally stable, but it can produce negative flux alterations that are critically damped through time. We explore a whole-problem (in all angle and all cell) sparse linear algebra framework, for both iterative schemes, to quickly produce performant code for GPUs. Despite one-cell inversion requiring additional iterations to convergence, those iterations can be done faster to provide a significant speedup over source iteration in quadrature sets at or below $S_{128}$. Going forward we will produce a two-dimensional implementation of this code to experiment with memory and performance impacts of a whole-problem framework including methods of synthetic acceleration and pre-conditioners for this scheme, then we will begin making direct comparisons to traditionally implemented source iteration in production code.

physics.comp-ph↗

Hybrid-Delta Tracking on a Structured Mesh in MCATK

Monte Carlo Application Toolkit (MCATK) commonly uses surface tracking on a structured mesh to compute scalar fluxes. In this mode, higher fidelity requires more mesh cells and isotopes and thus more computational overhead -- since every time a particle changes cells, new cross-sections must be found for all materials in a given cell -- even if no collision occurs in that cell. We implement a hybrid version of Woodcock (delta) tracking on this imposed mesh to alleviate the number of cross-section lookups. This algorithm computes an energy-dependent microscopic majorant cross section is computed for the problem. Each time a particle enters a new cell, rather than computing a true macroscopic cross-section over all isotopes in the cell, the microscopic majorant cross-section is simply multiplied by the total number density of the cell to obtain a macroscopic majorant cross-section for the cell. Delta tracking is then performed within that single cell. This increases performance with minimal code changes, speeding up the solve time by a factor of 1.5 -- 1.75 for k-eigenvalue simulations and 1.2 -- 1.6 for fixed source simulations in a series of materially complex criticality benchmarks.

physics.comp-ph↗

Development of MC/DC: a performant, scalable, and portable Python-based Monte Carlo neutron transport code

We discuss the current development of MC/DC (Monte Carlo Dynamic Code). MC/DC is primarily designed to serve as an exploratory Python-based MC transport code. However, it seeks to offer improved performance, massive scalability, and backend portability by leveraging Python code-generation libraries and implementing an innovative abstraction strategy and compilation scheme. Here, we verify MC/DC capabilities and perform an initial performance assessment. We found that MC/DC can run hundreds of times faster than its pure Python mode and about 2.5 times slower, but with comparable parallel scaling, than the high-performance MC code Shift for simple problems. Finally, to further exercise MC/DC's time-dependent MC transport capabilities, we propose a challenge problem based on the C5G7-TD benchmark model.

physics.comp-ph↗

Accelerating reactive-flow simulations using vectorized chemistry integration

The high cost of chemistry integration is a significant computational bottleneck for realistic reactive-flow simulations using operator splitting. Here we present a methodology to accelerate the solution of the chemical kinetic ordinary differential equations using single-instruction, multiple-data vector processing on CPUs using the OpenCL framework. First, we compared several vectorized integration algorithms using chemical kinetic source terms and analytical Jacobians from the pyJac software against a widely used integration code, CVODEs. Next, we extended the OpenFOAM computational fluid dynamics library to incorporate the vectorized solvers, and we compared the accuracy of a fourth-order linearly implicit integrator -- both in vectorized form and a corresponding method native to OpenFOAM -- with the community standard chemical kinetics library Cantera. We then applied our methodology to a variety of chemical kinetic models, turbulent intensities, and simulation scales to examine a range of engineering and scientific scale problems, including (pseudo) steady-state as well as time-dependent Reynolds-averaged Navier--Stokes simulations of the Sandia flame D and the Volvo Flygmotor bluff-body stabilized, premixed flame. Subsequently, we compared the performance of the vectorized and native OpenFOAM integrators over the studied models and simulations and found that our vectorized approach performs up to 33--35x faster than the native OpenFOAM solver with high accuracy.

physics.comp-ph↗

Smoldering combustion in cellulose and hemicellulose mixtures: Examining the roles of density, fuel composition, oxygen concentration, and moisture content

Smoldering combustion plays a key role in wildfires in forests, grasslands, and peatlands due to its common occurrence in porous fuels like peat and duff. As a consequence, understanding smoldering behavior in these fuels is crucial. Such fuels are generally composed of cellulose, hemicellulose, and lignin. Here we present an updated computational model for simulating smoldering combustion in cellulose and hemicellulose mixtures. We used this model to examine changes in smoldering propagation speed and peak temperatures with varying fuel composition and density. For a given fuel composition, increases in density decrease the propagation speed and increase mean peak temperature; for a given density, increases in hemicellulose content increase both propagation speed and peak temperature. We also examined the role of natural fuel expansion with the addition of water. Without expansion, addition of moisture content reduces the propagation speed primarily due to increasing (wet) fuel density. However, with fuel expansion similar to that observed in peat, the propagation speed increases due to the overall drop in fuel density. Finally, we studied the influence of fuel composition on critical moisture content of ignition and extinction: mixtures dominated by hemicellulose have 10% higher critical moisture content due to the increase in peak temperature.

physics.bio-ph↗

Assessing diffusion model impacts on enstrophy and flame structure in turbulent lean premixed flames

Diffusive transport of mass occurs at small scales in turbulent premixed flames. As a result, multicomponent mass diffusion, which is often neglected in direct numerical simulations (DNS) of premixed combustion, has the potential to impact both turbulence and flame characteristics at small scales. In this study, we evaluate these impacts by examining enstrophy dynamics and the internal structure of the flame for lean premixed hydrogen-air combustion, neglecting secondary Soret and Dufour effects. We performed three-dimensional DNS of these flames by implementing the Stefan-Maxwell equations in the code NGA to represent multicomponent mass transport, and we simulated statistically planar lean premixed hydrogen-air flames using both mixture-averaged and multicomponent models. The mixture-averaged model underpredicts the peak enstrophy by up to 13% in the flame front. Comparing the enstrophy budgets of these flames, the multicomponent simulation yields larger peak magnitudes compared to the mixture-averaged simulation in the reaction zone, showing differences of 17% and 14% in the normalized stretching and viscous effects terms. In the super-adiabatic regions of the flame, the mixture-averaged model overpredicts the viscous effects by up to 13%. To assess the effect of these differences on flame structure, we reconstructed the average local internal structure of the turbulent flame through statistical analysis of the scalar gradient field. Based on this analysis, we show that large differences in viscous effects contribute to significant differences in the average local flame structure between the two models.

physics.flu-dyn↗

The Two-Dimensional Swept Rule Applied on Heterogeneous Architectures

The partial differential equations describing compressible fluid flows can be notoriously difficult to resolve on a pragmatic scale and often require the use of high performance computing systems and/or accelerators. However, these systems face scaling issues such as latency, the fixed cost of communicating information between devices in the system. The swept rule is a technique designed to minimize these costs by obtaining a solution to unsteady equations at as many possible spatial locations and times prior to communicating. In this study, we implemented and tested the swept rule for solving two-dimensional problems on heterogeneous computing systems across two distinct systems. Our solver showed a speedup range of 0.22-2.71 for the heat diffusion equation and 0.52-1.46 for the compressible Euler equations. We can conclude from this study that the swept rule offers both potential for speedups and slowdowns and that care should be taken when designing such a solver to maximize benefits. These results can help make decisions to maximize these benefits and inform designs.

cs.DC↗

Assessing the impact of multicomponent diffusion in direct numerical simulations of premixed, high-Karlovitz, turbulent flames

Implementing multicomponent diffusion models in numerical combustion studies is computationally expensive; to reduce cost, numerical simulations commonly use mixture-averaged diffusion treatments or simpler models. However, the accuracy and appropriateness of mixture-averaged diffusion has not been verified for three-dimensional, turbulent, premixed flames. In this study we evaluated the role of multicomponent mass diffusion in premixed, three-dimensional high Karlovitz-number hydrogen, n-heptane, and toluene flames, representing a range of fuel Lewis numbers. We also studied a premixed, unstable two-dimensional hydrogen flame due to the importance of diffusion effects in such cases. Our comparison of diffusion flux vectors revealed differences of 10-20% on average between the mixture-averaged and multicomponent diffusion models, and greater than 40% in regions of high flame curvature. Overall, however, the mixture-averaged model produces small differences in diffusion flux compared with global turbulent flame statistics. To evaluate the impact of these differences between the two models, we compared normalized turbulent flame speeds and conditional means of species mass fraction and source term. We found differences of 5-20% in the mean normalized turbulent flame speeds, which seem to correspond to differences of 5-10% in the peak fuel source terms. Our results motivate further study into whether the mixture-averaged diffusion model is always appropriate for DNS of premixed turbulent flames.

physics.flu-dyn↗