arXiv subjects

Hari Sundar

Publications and source records attributed to Hari Sundar.

At least 19 recordsLinked to original sources

arXiv2026

Field conserving adaptive mesh refinement (AMR) scheme on massively parallel adaptive octree meshes

Adaptive mesh refinement (AMR) is widely used to efficiently resolve localized features in time-dependent partial differential equations (PDEs) by selectively refining and coarsening the mesh. However, in long-horizon simulations, repeated intergrid interpolations can introduce systematic drift in conserved quantities, especially for variational discretizations with continuous basis functions. While interpolation from parent-to-child during refinement in continuous Galerkin (CG) discretizations is naturally conservative, the standard injection-based child-to-parent coarsening interpolation is generally not. We propose a simple, scalable field-conserving coarsening operator for parallel, octree-based AMR. The method enforces discrete global conservation during coarsening by first computing field conserving coarse-element values at quadrature points and then recovering coarse nodal degrees of freedom via an $L^2$ projection (mass-matrix solve), which simultaneously controls the $L_2$ error. We evaluate the approach on mass-conserving phase-field models, including the Cahn--Hilliard and Cahn--Hilliard--Navier--Stokes systems, and compare against injection in terms of conservation error, solution quality, and computational cost.

math.NA↗

arXiv2025

DafnyMPI: A Dafny Library for Verifying Message-Passing Concurrent Programs

The Message Passing Interface (MPI) is widely used in parallel, high-performance programming, yet writing bug-free software that uses MPI remains difficult. We introduce DafnyMPI, a novel, scalable approach to formally verifying MPI software. DafnyMPI allows proving deadlock freedom, termination, and functional equivalence with simpler sequential implementations. In contrast to existing specialized frameworks, DafnyMPI avoids custom concurrency logics and instead relies on Dafny, a verification-ready programming language used for sequential programs, extending it with concurrent reasoning abilities. DafnyMPI is implemented as a library that enables safe MPI programming by requiring users to specify the communication topology upfront and to verify that calls to communication primitives such as MPI_ISEND and MPI_WAIT meet their preconditions. We formalize DafnyMPI using a core calculus and prove that the preconditions suffice to guarantee deadlock freedom. Functional equivalence is proved via rely-guarantee reasoning over message payloads and a system that guarantees safe use of read and write buffers. Termination and the absence of runtime errors are proved using standard Dafny techniques. To further demonstrate the applicability of DafnyMPI, we verify numerical solutions to three canonical partial differential equations. We believe DafnyMPI demonstrates how to make formal verification viable for a broader class of programs and provides proof engineers with additional tools for software verification of parallel and concurrent systems.

cs.PL↗

arXiv2025

A geometrically informed algebraic multigrid preconditioned iterative approach for solving high-order finite element systems

Algebraic multigrid (AMG) is conventionally applied in a black-box fashion, agnostic to the underlying geometry. In this work, we propose that using geometric information -- when available -- to assist with setting up the AMG hierarchy is beneficial, especially for solving linear systems resulting from high-order finite element discretizations. High-order problems draw considerable interest to both the scientific and engineering communities, but lack efficient solvers, at least open-source codes, tailored for unstructured high-order discretizations targeting large-scale, real-world applications. For geometric multigrid, it is known that using p-coarsening before h-coarsening can provide better scalability, but setting up p-coarsening is non-trivial in AMG. We develop a geometrically informed algebraic multigrid (GIAMG) method, as well as an associated high-performance computing program, which is able to set up a grid hierarchy that includes p-coarsening at the top grids with minimal information of the geometry from the user. A major advantage of using p-coarsening with AMG -- beyond the benefits known in the context of geometric multigrid (GMG) -- is the increased sparsification of coarse grid operators. We extensively evaluate GIAMG by testing on the 3D Helmholtz and incompressible flow problems, and demonstrate mesh-independent convergence, and excellent parallel scalability. We also compare the performance of GIAMG with existing AMG packages, including Hypre and ML.

math.NA↗

arXiv2024

Localization landscape of optical waves in multifractal photonic membranes

In this paper, we investigate the localization properties of optical waves in disordered systems with multifractal scattering potentials. In particular, we apply the localization landscape theory to the classical Helmholtz operator and, without solving the associated eigenproblem, show accurate predictions of localized eigenmodes for one- and two-dimensional multifractal structures. Finally, we design and fabricate nanoperforated photonic membranes in silicon nitride (SiN) and image directly their multifractal modes using leaky-mode spectroscopy in the visible spectral range. The measured data demonstrate optical resonances with multiscale intensity fluctuations in good qualitative agreement with numerical simulations. The proposed approach provides a convenient strategy to design multifractal photonic membranes, enabling rapid exploration of extended scattering structures with tailored disorder for enhanced light-matter interactions.

physics.optics↗

arXiv2023

Waveform Modelling for the Laser Interferometer Space Antenna

LISA, the Laser Interferometer Space Antenna, will usher in a new era in gravitational-wave astronomy. As the first anticipated space-based gravitational-wave detector, it will expand our view to the millihertz gravitational-wave sky, where a spectacular variety of interesting new sources abound: from millions of ultra-compact binaries in our Galaxy, to mergers of massive black holes at cosmological distances; from the beginnings of inspirals that will venture into the ground-based detectors' view to the death spiral of compact objects into massive black holes, and many sources in between. Central to realising LISA's discovery potential are waveform models, the theoretical and phenomenological predictions of the pattern of gravitational waves that these sources emit. This white paper is presented on behalf of the Waveform Working Group for the LISA Consortium. It provides a review of the current state of waveform models for LISA sources, and describes the significant challenges that must yet be overcome.

gr-qc↗

arXiv2023

Generating Finite Element Codes combining Adaptive Octrees with Complex Geometries

We present a high-level domain-specific language (DSL) interface to drive an adaptive incomplete $k$-d tree-based framework for finite element (FEM) solutions to PDEs. This DSL provides three key advances: (a) it abstracts out the complexity of implementing non-trivial FEM formulations, (b) it simplifies deploying these formulations on arbitrarily complicated and adaptively refined meshes, and (c) it exhibits good parallel performance. Taken together, the DSL interface allows end-users to rapidly and efficiently prototype new mathematical approaches, and deploy them on large clusters for solving practical problems. We illustrate this DSL by implementing a workflow for solving PDEs using the recently developed shifted boundary method (SBM). The SBM requires approximating relatively complicated integrals over boundary surfaces. Using a high-level DSL greatly simplifies this process and allows rapid exploration of variations. We demonstrate these tools on a variety of 2-D and 3-D configurations. With fewer than 20 lines of input, we can produce a parallel code that scales well to thousands of processes. This generated code is made accessible and readable to be easily modified and hand-tuned, making this tool useful even to experts with the target software.

cs.CE↗

arXiv2023

Automating GPU Scalability for Complex Scientific Models: Phonon Boltzman Transport Equation

Heterogeneous computing environments combining CPU and GPU resources provide a great boost to large-scale scientific computing applications. Code generation utilities that partition the work into CPU and GPU tasks while considering data movement costs allow researchers to more quickly and easily develop high-performance solutions, and make these resources accessible to a larger user base. We present developments for a domain-specific language (DSL) and code generation framework for solving partial differential equations (PDEs). These enhancements facilitate GPU-accelerated solution of the Boltzmann transport equation (BTE) for phonons, which is the governing equation for simulating thermal transport in semiconductor materials at sub-micron scales. The solution of the BTE involves thousands of coupled PDEs as well as complicated boundary conditions and nonlinear processing at each time step. These developments enable the DSL to generate configurable hybrid GPU/CPU code that couples accelerated kernels with user-defined code. We observed performance improvements of around 18X compared to a CPU-only version produced by this same DSL with minimal additional programming effort.

cs.CE↗

arXiv2023

An autoencoder compression approach for accelerating large-scale inverse problems

PDE-constrained inverse problems are some of the most challenging and computationally demanding problems in computational science today. Fine meshes that are required to accurately compute the PDE solution introduce an enormous number of parameters and require large scale computing resources such as more processors and more memory to solve such systems in a reasonable time. For inverse problems constrained by time dependent PDEs, the adjoint method that is often employed to efficiently compute gradients and higher order derivatives requires solving a time-reversed, so-called adjoint PDE that depends on the forward PDE solution at each timestep. This necessitates the storage of a high dimensional forward solution vector at every timestep. Such a procedure quickly exhausts the available memory resources. Several approaches that trade additional computation for reduced memory footprint have been proposed to mitigate the memory bottleneck, including checkpointing and compression strategies. In this work, we propose a close-to-ideal scalable compression approach using autoencoders to eliminate the need for checkpointing and substantial memory storage, thereby reducing both the time-to-solution and memory requirements. We compare our approach with checkpointing and an off-the-shelf compression approach on an earth-scale ill-posed seismic inverse problem. The results verify the expected close-to-ideal speedup for both the gradient and Hessian-vector product using the proposed autoencoder compression approach. To highlight the usefulness of the proposed approach, we combine the autoencoder compression with the data-informed active subspace (DIAS) prior to show how the DIAS method can be affordably extended to large scale problems without the need of checkpointing and large memory.

math.NA↗

arXiv2022

A projection-based, semi-implicit time-stepping approach for the Cahn-Hilliard Navier-Stokes equations on adaptive octree meshes

The Cahn-Hilliard Navier-Stokes (CHNS) system provides a computationally tractable model that can be used to effectively capture interfacial dynamics in two-phase fluid flows. In this work, we present a semi-implicit, projection-based finite element framework for solving the CHNS system. We use a projection-based semi-implicit time discretization for the Navier-Stokes equation and a fully-implicit time discretization for the Cahn-Hilliard equation. We use a conforming continuous Galerkin (cG) finite element method in space equipped with a residual-based variational multiscale (RBVMS) formulation. Pressure is decoupled using a projection step, which results in two linear positive semi-definite systems for velocity and pressure, instead of the saddle point system of a pressure-stabilized method. All the linear systems are solved using an efficient and scalable algebraic multigrid (AMG) method. We deploy this approach on a massively parallel numerical implementation using parallel octree-based adaptive meshes. The overall approach allows the use of relatively large time steps with much faster time-to-solve than similar fully-implicit methods. We present comprehensive numerical experiments showing detailed comparisons with results from the literature for canonical cases, including the single bubble rise and Rayleigh-Taylor instability.

math.NA↗

arXiv2022

Massively parallel simulations of binary black holes with Dendro-GR

We present results from the new Dendro-GR code. These include simulations of binary black hole mergers for mass ratios up to q=16. Dendro-GR uses Wavelet Adaptive Multi-Resolution (WAMR) to generate an unstructured grid adapted to the spacetime geometry together with an octree based data structure. We demonstrate good scaling, improved convergence properties and efficient use of computational resources. We validate the code with comparisons to LazEv.

gr-qc↗

arXiv2022

Breakup dynamics in primary jet atomization using mesh- and interface- refined Cahn-Hilliard Navier-Stokes

We present a technique to perform interface-resolved simulations of complex breakup dynamics in two-phase flows using the Cahn-Hilliard Navier-Stokes equations. The method dynamically decreases the interface thickness parameter in relevant regions and simultaneously increases local mesh resolution, preventing numerical artifacts. We perform a detailed numerical simulation of pulsed jet atomization that shows a complex cascade of break-up mechanisms involving sheet rupture and filament formation. To understand the effect of refinement on the breakup, we analyze the droplet size distribution. The proposed approach opens up resolved simulations for various multiphase flow phenomena.

physics.flu-dyn↗

arXiv2022

Scalable adaptive algorithms for next-generation multiphase flow simulations

High-fidelity flow simulations are indispensable when analyzing systems exhibiting multiphase flow phenomena. The accuracy of multiphase flow simulations is strongly contingent upon the finest mesh resolution used to represent the fluid-fluid interfaces. However, the increased resolution comes at a higher computational cost. In this work, we propose algorithmic advances that aim to reduce the computational cost without compromising on the physics by selectively detecting key regions of interest (droplets/filaments) that require significantly higher resolution. The framework uses an adaptive octree-based meshing framework that is integrated with PETSc's linear algebra solvers. We demonstrate scaling of the framework up to 114,688 processes on TACC's Frontera. Finally, we deploy the framework to simulate one of the most resolved simulations of primary jet atomization. This simulation -- equivalent to 35 trillion grid points on a uniform grid -- is 64 times larger than the current state-of-the-art simulations and provides unprecedented insights into an important flow physics problem with a diverse array of engineering applications.

physics.comp-ph↗

arXiv2022

A fully-coupled framework for solving Cahn-Hilliard Navier-Stokes equations: Second-order, energy-stable numerical methods on adaptive octree based meshes

We present a fully-coupled, implicit-in-time framework for solving a thermodynamically-consistent Cahn-Hilliard Navier-Stokes system that models two-phase flows. In this work, we extend the block iterative method presented in Khanwale et al. [\textit{Simulating two-phase flows with thermodynamically consistent energy stable Cahn-Hilliard Navier-Stokes equations on parallel adaptive octree based meshes}, J. Comput. Phys. (2020)], to a fully-coupled, provably second-order accurate scheme in time, while maintaining energy-stability. The new method requires fewer matrix assemblies in each Newton iteration resulting in faster solution time. The method is based on a fully-implicit Crank-Nicolson scheme in time and a pressure stabilization for an equal order Galerkin formulation. That is, we use a conforming continuous Galerkin (cG) finite element method in space equipped with a residual-based variational multiscale (RBVMS) procedure to stabilize the pressure. We deploy this approach on a massively parallel numerical implementation using parallel octree-based adaptive meshes. We present comprehensive numerical experiments showing detailed comparisons with results from the literature for canonical cases, including the single bubble rise, Rayleigh-Taylor instability, and lid-driven cavity flow problems. We analyze in detail the scaling of our numerical implementation.

math.NA↗

arXiv2021

Case study of SARS-CoV-2 transmission risk assessment in indoor environments using cloud computing resources

Complex flow simulations are conventionally performed on HPC clusters. However, the limited availability of HPC resources and steep learning curve of executing on traditional supercomputer infrastructure has drawn attention towards deploying flow simulation software on the cloud. We showcase how a complex computational framework -- that can evaluate COVID-19 transmission risk in various indoor classroom scenarios -- can be abstracted and deployed on cloud services. The availability of such cloud-based personalized planning tools can enable educational institutions, medical institutions, public sector workers (courthouses, police stations, airports, etc.), and other entities to comprehensively evaluate various in-person interaction scenarios for transmission risk. We deploy the simulation framework on the Azure cloud framework, utilizing the Dendro-kT mesh generation tool and PETSc solvers. The cloud abstraction is provided by RocketML cloud infrastructure. We compare the performance of the cloud machines with state-of-the-art HPC machine TACC Frontera. Our results suggest that cloud-based HPC resources are a viable strategy for a diverse array of end-users to rapidly and efficiently deploy simulation software.

cs.DC↗

arXiv2021

Scalable adaptive PDE solvers in arbitrary domains

Efficiently and accurately simulating partial differential equations (PDEs) in and around arbitrarily defined geometries, especially with high levels of adaptivity, has significant implications for different application domains. A key bottleneck in the above process is the fast construction of a `good' adaptively-refined mesh. In this work, we present an efficient novel octree-based adaptive discretization approach capable of carving out arbitrarily shaped void regions from the parent domain: an essential requirement for fluid simulations around complex objects. Carving out objects produces an $\textit{incomplete}$ octree. We develop efficient top-down and bottom-up traversal methods to perform finite element computations on $\textit{incomplete}$ octrees. We validate the framework by (a) showing appropriate convergence analysis and (b) computing the drag coefficient for flow past a sphere for a wide range of Reynolds numbers ($\mathcal{O}(1-10^6)$) encompassing the drag crisis regime. Finally, we deploy the framework on a realistic geometry on a current project to evaluate COVID-19 transmission risk in classrooms.

math.NA↗

arXiv2021

Industrial scale large eddy simulations (LES) with adaptive octree meshes using immersogeometric analysis

We present a variant of the immersed boundary method integrated with octree meshes for highly efficient and accurate Large-Eddy Simulations (LES) of flows around complex geometries. We demonstrate the scalability of the proposed method up to $\mathcal{O}(32K)$ processors. This is achieved by (a) rapid in-out tests; (b) adaptive quadrature for an accurate evaluation of forces; (c) tensorized evaluation during matrix assembly. We showcase this method on two non-trivial applications: accurately computing the drag coefficient of a sphere across Reynolds numbers $1-10^6$ encompassing the drag crisis regime; simulating flow features across a semi-truck for investigating the effect of platooning on efficiency.

math.NA↗

arXiv2020

Scalable Local Timestepping on Octree Grids

Numerical solutions of hyperbolic partial differential equations(PDEs) are ubiquitous in science and engineering. Method of lines is a popular approach to discretize PDEs defined in spacetime, where space and time are discretized independently. When using explicit timesteppers on adaptive grids, the use of a global timestep-size dictated by the finest grid-spacing leads to inefficiencies in the coarser regions. Even though adaptive space discretizations are widely used in computational sciences, temporal adaptivity is less common due to its sophisticated nature. In this paper, we present highly scalable algorithms to enable local timestepping (LTS) for explicit timestepping schemes on fully adaptive octrees. We demonstrate the accuracy of our methods as well as the scalability of our framework across 16K cores in TACC's Frontera. We also present a speed up estimation model for LTS, which predicts the speedup compared to global timestepping (GTS) with an average of 0.1 relative error.

cs.MS↗

arXiv2020

Simulating two-phase flows with thermodynamically consistent energy stable Cahn-Hilliard Navier-Stokes equations on parallel adaptive octree based meshes

We report on simulations of two-phase flows with deforming interfaces at various density contrasts by solving thermodynamically consistent Cahn-Hilliard Navier-Stokes equations. An (essentially) unconditionally energy-stable Crank-Nicolson-type time integration scheme is used. Detailed proofs of energy stability of the semi-discrete scheme and for the existence of solutions of the advective-diffusive Cahn-Hilliard operator are provided. In space we discretize with a conforming continuous Galerkin finite element method in conjunction with a residual-based variational multi-scale (VMS) approach in order to provide pressure stabilization. We deploy this approach on a massively parallel numerical implementation using fast octree-based adaptive meshes. A detailed scaling analysis of the solver is presented. Numerical experiments showing convergence and validation with experimental results from the literature are presented for a large range of density ratios.

math.NA↗