SearcharxivSearch

arXiv subjects

Misun Min

Publications and source records attributed to Misun Min.

At least 19 recordsLinked to original sources

Super-Resolution Initialization of High-Fidelity CFD Simulations for Pebble-Bed Reactors

High-order CFD simulations provide detailed resolution of the heterogeneous interstitial flow in pebble-bed reactors, but their computational cost is high, especially during the initial flow-development period required to reach statistically stationary conditions. This work investigates the use of a Super-Resolution Graph Neural Network (SR-GNN) to improve the initialization of high-order NekRS simulations. Lower-order P = 2 velocity fields are used as inputs to reconstruct higher-order representations, which are then used as initial conditions for P = 7 restart simulations. The approach is evaluated using a 146-pebble bed at Re = 1000, Re = 2500, and Re = 5000, with pressure-drop convergence used as the main figure of merit. The SR-GNN models were trained using paired low- and high-order snapshots and were first evaluated through qualitative inference comparisons. High-order restart simulations showed that, for Re = 1000 and Re = 2500, the SR-GNN initialized cases produced pressure-drop histories similar to direct restarts from true P = 2 fields. For Re = 5000, however, the super-resolved field restart approached the statistically stationary P = 7 pressure-drop range faster than both the direct P = 2 restart and the reference P = 7 simulation initialized from a uniform velocity field. The trained Re = 5000 model was also applied to a larger 1568-pebble bed, demonstrating qualitative applicability of the workflow to a significantly larger packed-bed geometry. These results indicate that SR-GNN-based initialization is a promising strategy for reducing high-order flow-development cost, while also motivating further work on broader Reynolds-number and geometry generalization.

physics.flu-dyn

A flux bounce-back scheme for the filtered Spectral Element Lattice Boltzmann Method

We develop a spectral element lattice Boltzmann method (SELBM) with the flux bounce-back (FBB) scheme, to enable accurate simulations of single-phase fluid dynamics in unstructured mesh. We adopt an Eulerian description of the streaming process in place of the perfect shift in the regular LBM. The spectral element method is used to spatially discretize the convective term, while the strong stability-preserving Runge-Kutta (SSPRK) method is used for time integration. To increase stability, we investigate the use of an explicit filter, particularly in the context of the sensitive double shear layer problem. The results indicate that by using the high-order polynomial, we can effectively eliminate the small vortices around the neck region. We introduce the flux bounce-back scheme to enable the current scheme to handle complex boundaries. The proposed scheme and flux boundary method are validated through benchmark simulations, including the unsteady Couette flow and the planar Poiseuille flow. Further validation is provided through the Taylor-Green vortex problem, demonstrating the accuracy and convergence of the scheme for isotropic turbulence. Finally, we consider a fully developed turbulent flow within a cylindrical pipe and correctly predict the turbulent boundary layer profile.

physics.flu-dyn

Deciphering boundary layer dynamics in high-Rayleigh-number convection using 3360 GPUs and a high-scaling in-situ workflow

Turbulent heat and momentum transfer processes due to thermal convection cover many scales and are of great importance for several natural and technical flows. One consequence is that a fully resolved three-dimensional analysis of these turbulent transfers at high Rayleigh numbers, which includes the boundary layers, is possible only using supercomputers. The visualization of these dynamics poses an additional hurdle since the thermal and viscous boundary layers in thermal convection fluctuate strongly. In order to track these fluctuations continuously, data must be tapped at high frequency for visualization, which is difficult to achieve using conventional methods. This paper makes two main contributions in this context. First, it discusses the simulations of turbulent Rayleigh-B\'enard convection up to Rayleigh numbers of $Ra=10^{12}$ computed with NekRS on GPUs. The largest simulation was run on 840 nodes with 3360 GPU on the JUWELS Booster supercomputer. Secondly, an in-situ workflow using ASCENT is presented, which was successfully used to visualize the high-frequency turbulent fluctuations.

physics.flu-dyn

DDES Study of Confined and Unconfined NACA Wing Sections Using Spectral Elements

We develop hybrid RANS-LES strategies within the spectral element code Nek5000 based on the $k-\tau$ class of turbulence models. We chose airfoil sections at small flight configurations as our target problem to comprehensively test the solver accuracy and performance. We present verification and validation results of an unconfined NACA0012 wing section in a pure RANS and in a hybrid RANS-LES setup for an angle of attack ranging from 0 to 90 degrees. The RANS results shows good corroboration with existing experimental and numerical datasets for low incoming flow angles. A small discrepancy appears at higher angle in comparison with the experiments, which is in line with our expectations from a RANS formulation. On the other hand, DDES captures both the attached and separated flow dynamics well when compared with available numerical datasets. We demonstrate that for the hybrid turbulence modeling approach a high-order spectral element discretization converges faster (i.e., with less resolution) and captures the flow dynamics more accurately than representative low-order finite-volume and finite-difference approaches. We also revise some of the guidelines on sample size requirements for statistics convergence. Furthermore, we analyze some of the observed discrepancies of our unconfined DDES at higher angles with the experiments by evaluating the side wall "blocking" effect. We carry out additional simulations in a confined 'numerical wind tunnel' and assess the observed differences as a function of Reynolds number.

cs.CE

Modeling Turbulence in the Atmospheric Boundary Layer with Spectral Element and Finite Volume Methods

We present large-eddy-simulation (LES) modeling approaches for the simulation of atmospheric boundary layer turbulence that are of direct relevance to wind energy production. In this paper, we study a GABLS benchmark problem using high-order spectral element code Nek5000/RS and a block-structured second-order finite-volume code AMR-Wind which are supported under the DOE's Exascale Computing Project (ECP) Center for Efficient Exascale Discretizations (CEED) and ExaWind projects, respectively, targeting application simulations on various acceleration-device based exascale computing platforms. As for Nek5000/RS we demonstrate our newly developed subgrid-scale (SGS) models based on mean-field eddy viscosity (MFEV), high-pass filter (HPF), and Smagorinsky (SMG) with traction boundary conditions. For the traction boundary conditions, a novel analytical approach is presented that solves for the surface friction velocity and surface kinematic temperature flux. For AMR-Wind, standard SMG is used and discussed in detail the traction boundary conditions for convergence. We provide low-order statistics, convergence and turbulent structure analysis. Verification and convergence studies were performed for both codes at various resolutions and it was found that Nek5000/RS demonstrate convergence with resolution for all ABL bulk parameters, including boundary layer and low level jet (LLJ) height. Extensive comparisons are presented with simulation data from the literature.

cs.CE

Spectral Element Simulation of Liquid Metal Magnetohydrodynamics

A spectral-element-based formulation of incompressible MHD is presented in the context of the open-source fluid-thermal code, Nek5000/RS. The formulation supports magnetic fields in a solid domain that surrounds the fluid domain. Several steady-state and time-transient model problems are presented as part of the code verification process. Nek5000/RS is designed for large-scale turbulence simulations, which will be the next step with this new MHD capability.

cs.CE

Exascale Simulations of Fusion and Fission Systems

We discuss pioneering heat and fluid flow simulations of fusion and fission energy systems with NekRS on exascale computing facilities, including Frontier and Aurora. The Argonne-based code, NekRS, is a highly-performant open-source code for the simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on rapidly convergent high-order spectral element discretizations that feature minimal numerical dissipation and dispersion. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Frontier, Nek5000/RS has achieved an unprecedented milestone in breaching over 1 trillion degrees of freedom with the spectral element methods for the simulation of the CHIMERA fusion technology testing platform. We also demonstrate for the first time the use of high-order overset grids at scale.

cs.CE

Scaling Computational Fluid Dynamics: In Situ Visualization of NekRS using SENSEI

In the realm of Computational Fluid Dynamics (CFD), the demand for memory and computation resources is extreme, necessitating the use of leadership-scale computing platforms for practical domain sizes. This intensive requirement renders traditional checkpointing methods ineffective due to the significant slowdown in simulations while saving state data to disk. As we progress towards exascale and GPU-driven High-Performance Computing (HPC) and confront larger problem sizes, the choice becomes increasingly stark: to compromise data fidelity or to reduce resolution. To navigate this challenge, this study advocates for the use of in situ analysis and visualization techniques. These allow more frequent data "snapshots" to be taken directly from memory, thus avoiding the need for disruptive checkpointing. We detail our approach of instrumenting NekRS, a GPU-focused thermal-fluid simulation code employing the spectral element method (SEM), and describe varied in situ and in transit strategies for data rendering. Additionally, we provide concrete scientific use-cases and report on runs performed on Polaris, Argonne Leadership Computing Facility's (ALCF) 44 Petaflop supercomputer and J\"ulich Wizard for European Leadership Science (JUWELS) Booster, J\"ulich Supercomputing Centre's (JSC) 71 Petaflop High Performance Computing (HPC) system, offering practical insight into the implications of our methodology.

cs.DC

Nek5000/RS Performance on Advanced GPU Architectures

We demonstrate NekRS performance results on various advanced GPU architectures. NekRS is a GPU-accelerated version of Nek5000 that targets high performance on exascale platforms. It is being developed in DOE's Center of Efficient Exascale Discretizations, which is one of the co-design centers under the Exascale Computing Project. In this paper, we consider Frontier, Crusher, Spock, Polaris, Perlmutter, ThetaGPU, and Summit. Simulations are performed with 17x17 rod-bundle geometries from small modular reactor applications. We discuss strong-scaling performance and analysis.

cs.DC

Towards Exascale for Wind Energy Simulations

We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer (ABL) flows that are of direct relevance to wind energy production. The first is NekRS, a high-order, unstructured-grid, spectral element code. The second, AMR-Wind, is a block-structured, second-order finite-volume code with adaptive-mesh-refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.

cs.CE

Highly Optimized Full-Core Reactor Simulations on Summit

Nek5000/RS is a highly-performant open-source spectral element code for simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on high-order discretizations that realize the same (or lower) cost per gridpoint as traditional low-order methods. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Summit, Nek5000/RS has recently achieved an milestone in the simulation of nuclear reactors: the first full-core computational fluid dynamics simulations of reactor cores, including pebble beds with > 350,000 pebbles and 98M elements advanced in less than 0.25 seconds per Navier-Stokes timestep. With carefully tuned algorithms, it is possible to simulate a single flow-through time for a full reactor core in less than six hours on all of Summit.

physics.comp-ph

GPU Algorithms for Efficient Exascale Discretizations

In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.

cs.DC

Efficient Exascale Discretizations: High-Order Finite Element Methods

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially-assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications.

cs.DC

All-Hex Meshing Strategies For Densely Packed Spheres

We develop an all-hex meshing strategy for the interstitial space in beds of densely packed spheres that is tailored to turbulent flow simulations based on the spectral element method (SEM). The SEM achieves resolution through elevated polynomial order N and requires two to three orders of magnitude fewer elements than standard finite element approaches do. These reduced element counts place stringent requirements on mesh quality and conformity. Our meshing algorithm is based on a Voronoi decomposition of the sphere centers. Facets of the Voronoi cells are tessellated into quads that are swept to the sphere surface to generate a high-quality base mesh. Refinements to the algorithm include edge collapse to remove slivers, node insertion to balance resolution, localized refinement in the radial direction about each sphere, and mesh optimization. We demonstrate geometries with 10^2-10^5 spheres using approximately 300 elements per sphere (for three radial layers), along with mesh quality metrics, timings, flow simulations, and solver performance.

cs.CE

NekRS, a GPU-Accelerated Spectral Element Navier-Stokes Solver

The development of NekRS, a GPU-oriented thermal-fluids simulation code based on the spectral element method (SEM) is described. For performance portability, the code is based on the open concurrent compute abstraction and leverages scalable developments in the SEM code Nek5000 and in libParanumal, which is a library of high-performance kernels for high-order discretizations and PDE-based miniapps. Critical performance sections of the Navier-Stokes time advancement are addressed. Performance results on several platforms are presented, including scaling to 27,648 V100s on OLCF Summit, for calculations of up to 60B gridpoints.

cs.PF

Scalability of High-Performance PDE Solvers

Performance tests and analyses are critical to effective HPC software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this paper, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for PDEs that govern a wide range of physical applications. We consider a sequence of PDE- motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.

cs.PF

Quantum Sensing for High Energy Physics

Report of the first workshop to identify approaches and techniques in the domain of quantum sensing that can be utilized by future High Energy Physics applications to further the scientific goals of High Energy Physics.

hep-ex

Temporal control of graphene plasmons

Electrostatic gating and optical pumping schemes enable efficient time modulation of graphene's free carrier density, or Drude weight. We develop a theory for plasmon propagation in graphene under temporal modulation. When the modulation is on the timescale of the plasmonic period, we show that it is possible to create a backwards-propagating or standing plasmon wave and to amplify plasmons. The theoretical models show very good agreement with direct Maxwell simulations.

cond-mat.mes-hall