arXiv subjects

Paul Fischer

Publications and source records attributed to Paul Fischer.

At least 37 records · Page 2Linked to original sources

arXiv2024

Subgroup-Specific Risk-Controlled Dose Estimation in Radiotherapy

Cancer remains a leading cause of death, highlighting the importance of effective radiotherapy (RT). Magnetic resonance-guided linear accelerators (MR-Linacs) enable imaging during RT, allowing for inter-fraction, and perhaps even intra-fraction, adjustments of treatment plans. However, achieving this requires fast and accurate dose calculations. While Monte Carlo simulations offer accuracy, they are computationally intensive. Deep learning frameworks show promise, yet lack uncertainty quantification crucial for high-risk applications like RT. Risk-controlling prediction sets (RCPS) offer model-agnostic uncertainty quantification with mathematical guarantees. However, we show that naive application of RCPS may lead to only certain subgroups such as the image background being risk-controlled. In this work, we extend RCPS to provide prediction intervals with coverage guarantees for multiple subgroups with unknown subgroup membership at test time. We evaluate our algorithm on real clinical planing volumes from five different anatomical regions and show that our novel subgroup RCPS (SG-RCPS) algorithm leads to prediction intervals that jointly control the risk for multiple subgroups. In particular, our method controls the risk of the crucial voxels along the radiation beam significantly better than conventional RCPS.

cs.LG↗

arXiv2023

A Time-Relaxation Reduced Order Model for the Turbulent Channel Flow

Reg-ROMs are stabilization strategies that leverage spatial filtering to alleviate the spurious numerical oscillations generally displayed by the classical G-ROM in under-resolved numerical simulations of turbulent flows. In this paper, we propose a new Reg-ROM, the time-relaxation ROM (TR-ROM), which filters the marginally resolved scales. We compare the new TR-ROM with the two other Reg-ROMs in current use, i.e., the L-ROM and the EFR-ROM, in the numerical simulation of the turbulent channel flow at $Re_τ = 180$ and $Re_τ = 395$ in both the reproduction and the predictive regimes. For each Reg-ROM, we investigate two different filters: (i) the differential filter (DF), and (ii) a new higher-order algebraic filter (HOAF). In our numerical investigation, we monitor the Reg-ROM performance for the ROM dimension, $N$, and the filter order. We also perform sensitivity studies of the three Reg-ROMs for the time interval, relaxation parameter, and filter radius. The numerical results yield the following conclusions: (i) All three Reg-ROMs are significantly more accurate than the G-ROM and (ii) more accurate than the ROM projection, representing the best theoretical approximation of the training data in the given ROM space. (iii) With the optimal parameter values, the TR-ROM is more accurate than the other two Reg-ROMs in all tests. (iv) For most $N$ values, DF yields the most accurate results for all three Reg-ROMs. (v) The optimal parameters trained in the reproduction regime are also optimal for the predictive regime for most $N$ values. (vi) All three Reg-ROMs are sensitive to the filter radius and the filter order, and the EFR-ROM and the TR-ROM are sensitive to the relaxation parameter. (vii) The optimal range for the filter radius and the effect of relaxation parameter are similar for the two $\rm Re_τ$ values.

math.NA↗

arXiv2023

Scaling Computational Fluid Dynamics: In Situ Visualization of NekRS using SENSEI

In the realm of Computational Fluid Dynamics (CFD), the demand for memory and computation resources is extreme, necessitating the use of leadership-scale computing platforms for practical domain sizes. This intensive requirement renders traditional checkpointing methods ineffective due to the significant slowdown in simulations while saving state data to disk. As we progress towards exascale and GPU-driven High-Performance Computing (HPC) and confront larger problem sizes, the choice becomes increasingly stark: to compromise data fidelity or to reduce resolution. To navigate this challenge, this study advocates for the use of in situ analysis and visualization techniques. These allow more frequent data "snapshots" to be taken directly from memory, thus avoiding the need for disruptive checkpointing. We detail our approach of instrumenting NekRS, a GPU-focused thermal-fluid simulation code employing the spectral element method (SEM), and describe varied in situ and in transit strategies for data rendering. Additionally, we provide concrete scientific use-cases and report on runs performed on Polaris, Argonne Leadership Computing Facility's (ALCF) 44 Petaflop supercomputer and Jülich Wizard for European Leadership Science (JUWELS) Booster, Jülich Supercomputing Centre's (JSC) 71 Petaflop High Performance Computing (HPC) system, offering practical insight into the implications of our methodology.

cs.DC↗

arXiv2023

First Steps Towards a Runtime Analysis of Neuroevolution

We consider a simple setting in neuroevolution where an evolutionary algorithm optimizes the weights and activation functions of a simple artificial neural network. We then define simple example functions to be learned by the network and conduct rigorous runtime analyses for networks with a single neuron and for a more advanced structure with several neurons and two layers. Our results show that the proposed algorithm is generally efficient on two example problems designed for one neuron and efficient with at least constant probability on the example problem for a two-layer network. In particular, the so-called harmonic mutation operator choosing steps of size $j$ with probability proportional to $1/j$ turns out as a good choice for the underlying search space. However, for the case of one neuron, we also identify situations with hard-to-overcome local optima. Experimental investigations of our neuroevolutionary algorithm and a state-of-the-art CMA-ES support the theoretical findings.

cs.NE↗

arXiv2023

Nek5000/RS Performance on Advanced GPU Architectures

We demonstrate NekRS performance results on various advanced GPU architectures. NekRS is a GPU-accelerated version of Nek5000 that targets high performance on exascale platforms. It is being developed in DOE's Center of Efficient Exascale Discretizations, which is one of the co-design centers under the Exascale Computing Project. In this paper, we consider Frontier, Crusher, Spock, Polaris, Perlmutter, ThetaGPU, and Summit. Simulations are performed with 17x17 rod-bundle geometries from small modular reactor applications. We discuss strong-scaling performance and analysis.

cs.DC↗

arXiv2023

Uncertainty Estimation and Propagation in Accelerated MRI Reconstruction

MRI reconstruction techniques based on deep learning have led to unprecedented reconstruction quality especially in highly accelerated settings. However, deep learning techniques are also known to fail unexpectedly and hallucinate structures. This is particularly problematic if reconstructions are directly used for downstream tasks such as real-time treatment guidance or automated extraction of clinical paramters (e.g. via segmentation). Well-calibrated uncertainty quantification will be a key ingredient for safe use of this technology in clinical practice. In this paper we propose a novel probabilistic reconstruction technique (PHiRec) building on the idea of conditional hierarchical variational autoencoders. We demonstrate that our proposed method produces high-quality reconstructions as well as uncertainty quantification that is substantially better calibrated than several strong baselines. We furthermore demonstrate how uncertainties arising in the MR econstruction can be propagated to a downstream segmentation task, and show that PHiRec also allows well-calibrated estimation of segmentation uncertainties that originated in the MR reconstruction process.

eess.IV↗

arXiv2023

Optimal Chebyshev Smoothers and One-sided V-cycles

The solution to the Poisson equation arising from the spectral element discretization of the incompressible Navier-Stokes equation requires robust preconditioning strategies. One such strategy is multigrid. To realize the potential of multigrid methods, effective smoothing strategies are needed. Chebyshev polynomial smoothing proves to be an effective smoother. However, there are several improvements to be made, especially at the cost of symmetry. For the same cost per iteration, a full V-cycle with $k$ order Chebyshev polynomial smoothing may be substituted with a half V-cycle with order $2k$ Chebyshev polynomial smoothing, wherein the smoother is omitted on the up-leg of the V-cycle. The choice of omitting the post-smoother in favor of higher order Chebyshev pre-smoothing is shown to be advantageous in cases where the multigrid approximation property constant, $C$, is large. Results utilizing Lottes's fourth-kind Chebyshev polynomial smoother are shown. These methods demonstrate substantial improvement over the standard Chebyshev polynomial smoother. The authors demonstrate the effectiveness of this scheme in $p$-geometric multigrid, as well as a 2D model problem with finite differences.

math.NA↗

arXiv2022

Towards Exascale for Wind Energy Simulations

We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer (ABL) flows that are of direct relevance to wind energy production. The first is NekRS, a high-order, unstructured-grid, spectral element code. The second, AMR-Wind, is a block-structured, second-order finite-volume code with adaptive-mesh-refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.

cs.CE↗

arXiv2021

Tuning Spectral Element Preconditioners for Parallel Scalability on GPUs

The Poisson pressure solve resulting from the spectral element discretization of the incompressible Navier-Stokes equation requires fast, robust, and scalable preconditioning. In the current work, a parallel scaling study of Chebyshev-accelerated Schwarz and Jacobi preconditioning schemes is presented, with special focus on GPU architectures, such as OLCF's Summit. Convergence properties of the Chebyshev-accelerated schemes are compared with alternative methods, such as low-order preconditioners combined with algebraic multigrid. Performance and scalability results are presented for a variety of preconditioner and solver settings. The authors demonstrate that Chebyshev-accelerated-Schwarz methods provide a robust and effective smoothing strategy when using $p$-multigrid as a preconditioner in a Krylov-subspace projector. The variety of cases to be addressed, on a wide range of processor counts, suggests that performance can be enhanced by automated run-time selection of the preconditioner and associated parameters.

math.NA↗

arXiv2021

Highly Optimized Full-Core Reactor Simulations on Summit

Nek5000/RS is a highly-performant open-source spectral element code for simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on high-order discretizations that realize the same (or lower) cost per gridpoint as traditional low-order methods. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Summit, Nek5000/RS has recently achieved an milestone in the simulation of nuclear reactors: the first full-core computational fluid dynamics simulations of reactor cores, including pebble beds with > 350,000 pebbles and 98M elements advanced in less than 0.25 seconds per Navier-Stokes timestep. With carefully tuned algorithms, it is possible to simulate a single flow-through time for a full reactor core in less than six hours on all of Summit.

physics.comp-ph↗

arXiv2021

GPU Algorithms for Efficient Exascale Discretizations

In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.

cs.DC↗

arXiv2021

Efficient Exascale Discretizations: High-Order Finite Element Methods

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially-assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications.

cs.DC↗

arXiv2021

All-Hex Meshing Strategies For Densely Packed Spheres

We develop an all-hex meshing strategy for the interstitial space in beds of densely packed spheres that is tailored to turbulent flow simulations based on the spectral element method (SEM). The SEM achieves resolution through elevated polynomial order N and requires two to three orders of magnitude fewer elements than standard finite element approaches do. These reduced element counts place stringent requirements on mesh quality and conformity. Our meshing algorithm is based on a Voronoi decomposition of the sphere centers. Facets of the Voronoi cells are tessellated into quads that are swept to the sphere surface to generate a high-quality base mesh. Refinements to the algorithm include edge collapse to remove slivers, node insertion to balance resolution, localized refinement in the radial direction about each sphere, and mesh optimization. We demonstrate geometries with 10^2-10^5 spheres using approximately 300 elements per sphere (for three radial layers), along with mesh quality metrics, timings, flow simulations, and solver performance.

cs.CE↗

arXiv2021

Multirate Timestepping for the Incompressible Navier-Stokes Equations in Overlapping Grids

We develop a multirate timestepper for semi-implicit solutions of the unsteady incompressible Navier-Stokes equations (INSE) based on a recently-developed multidomain spectral element method (SEM). For {\em incompressible} flows, multirate timestepping (MTS) is particularly challenging because of the tight coupling implied by the incompressibility constraint, which manifests as an elliptic subproblem for the pressure at each timestep. The novelty of our approach stems from the development of a stable overlapping Schwarz method applied directly to the Navier-Stokes equations, rather than to the convective, viscous, and pressure substeps that are at the heart of most INSE solvers. Our MTS approach is based on a predictor-corrector (PC) strategy that preserves the temporal convergence of the underlying semi-implicit timestepper. We present numerical results demonstrating that this approach scales to an arbitrary number of overlapping grids, accurately models complex turbulent flow phenomenon, and improves computational efficiency in comparison to singlerate timestepping-based calculations.

physics.flu-dyn↗

arXiv2021

NekRS, a GPU-Accelerated Spectral Element Navier-Stokes Solver

The development of NekRS, a GPU-oriented thermal-fluids simulation code based on the spectral element method (SEM) is described. For performance portability, the code is based on the open concurrent compute abstraction and leverages scalable developments in the SEM code Nek5000 and in libParanumal, which is a library of high-performance kernels for high-order discretizations and PDE-based miniapps. Critical performance sections of the Navier-Stokes time advancement are addressed. Performance results on several platforms are presented, including scaling to 27,648 V100s on OLCF Summit, for calculations of up to 60B gridpoints.

cs.PF↗

arXiv2020

Stability analysis of a singlerate and multirate predictor-corrector scheme for overlapping grids

We use matrix stability analysis for a singlerate and multirate predictor-corrector scheme (PC) used to solve the incompressible Navier-Stokes equations (INSE) in overlapping grids. By simplifying the stability analysis with the unsteady heat equation in 1D, we demonstrate that, as expected, the stability of the PC scheme increases with increase in the resolution and overlap of subdomains. For singlerate timestepping, we also find that the high-order PC scheme is stable when the number of corrector iterations ($Q$) is odd. This difference in the stability of odd- and even-$Q$ is novel and has not been demonstrated in the literature for overlapping grid-based methods. We address the odd-even behavior in the stability of the PC scheme by modifying the last corrector iterate, which leads to a scheme whose stability increases monotonically with $Q$. For multirate timestepping, we observe that the stability of the PC scheme depends on the timestep ratio ($η$). For $η=2$, even-$Q$ is more stable than odd-$Q$. For $η\ge3$, even-$Q$ is more stable than odd-$Q$ for a small nondimensional timestep size and the odd-even behavior vanishes as the timestep size increases. The stability analysis presented in this work gives novel insight into a high-order temporal discretization for ODEs and PDEs, and has helped us develop an improved PC scheme for solving the incompressible Navier-Stokes equations.

math.NA↗

arXiv2020

Scalability of High-Performance PDE Solvers

Performance tests and analyses are critical to effective HPC software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this paper, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for PDEs that govern a wide range of physical applications. We consider a sequence of PDE- motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.

cs.PF↗

arXiv2020

Direct Numerical Simulation of Rotating Ellipsoidal Particles using Moving Nonconforming Schwarz-Spectral Element Method

We present application of a highly-scalable overlapping grid-based nonconforming Schwarz-spectral element method (Schwarz-SEM) to study the dynamics of rotating ellipsoidal particles. The current study is one of the first to explore the effect of rotation on ellipsoidal particles using fully resolved simulations (direct numerical simulation). The rotating ellipsoidal particles show substantial difference in the dynamics of the flow, when compared against non-rotating particles. The difference is primarily due to periodic attachment and separation of the flow to the surface of the particle for the rotating cases, which results in a higher drag on the particles when compared to the corresponding non-rotating cases. The dynamics is also different from a rotating spherical particle, where a steady shear layer develops near the surface of the sphere. For the rotating ellipsoidal particles, this mechanism results in a phase-difference between the position of observed maximum and minimum drag, and the position of expected maximum and minimum drag (i.e., maximum and minimum projected area). A similar phase-difference is also observed for the lift acting on the rotating ellipsoidal particles. The results presented here demonstrate the importance of explicitly modeling the shape and rotation of particles when we study the dynamics of non-spherical particles. Finally, the study also validates the use of non-conforming Schwarz-SEM for tackling problems in fully resolved particulate flow dynamics.

physics.flu-dyn↗