arXiv subjects

Paul Mullowney

Publications and source records attributed to Paul Mullowney.

6 recordsLinked to original sources

arXiv2023

ILU Smoothers for Low Mach Navier-Stokes Pressure Solvers

Incomplete LU (ILU) smoothers are effective in the algebraic multigrid (AMG) $V$-cycle for reducing high-frequency components of the error. However, the requisite direct triangular solves are comparatively slow on GPUs. Previous work has demonstrated the advantages of Jacobi iteration as an alternative to direct solution of these systems. Depending on the threshold and fill-level parameters chosen, the factors can be highly non-normal and Jacobi is unlikely to converge in a low number of iterations. We demonstrate that row scaling can reduce the departure from normality, allowing us to replace the inherently sequential solve with a rapidly converging Richardson iteration. There are several advantages beyond the lower compute time. Scaling is performed locally for a diagonal block of the global matrix because it is applied directly to the factor. Further, an ILUT Schur complement smoother maintains a constant GMRES iteration count as the number of MPI ranks increases, and thus parallel strong-scaling is improved. Our algorithms have been incorporated into hypre, and we demonstrate improved time to solution for linear systems arising in the Nalu-Wind and PeleLM pressure solvers. For large problem sizes, GMRES$+$AMG executes at least five times faster when using iterative triangular solves compared with direct solves on massively-parallel GPUs.

math.NA↗

arXiv2023

Experiences Readying Applications for Exascale

The advent of exascale computing invites an assessment of existing best practices for developing application readiness on the world's largest supercomputers. This work details observations from the last four years in preparing scientific applications to run on the Oak Ridge Leadership Computing Facility's (OLCF) Frontier system. This paper addresses a range of topics in software including programmability, tuning, and portability considerations that are key to moving applications from existing systems to future installations. A set of representative workloads provides case studies for general system and software testing. We evaluate the use of early access systems for development across several generations of hardware. Finally, we discuss how best practices were identified and disseminated to the community through a wide range of activities including user-guides and trainings. We conclude with recommendations for ensuring application readiness on future leadership computing systems.

cs.DC↗

arXiv2021

Neumann Series in GMRES and Algebraic Multigrid Smoothers

Neumann series underlie both Krylov methods and algebraic multigrid smoothers. A low-synch modified Gram-Schmidt (MGS)-GMRES algorithm is described that employs a Neumann series to accelerate the projection step. A corollary to the backward stability result of Paige et al. (2006) demonstrates that the truncated Neumann series approximation is sufficient for convergence of GMRES. The lower triangular solver associated with the correction matrix $T_m = (\: I + L_m \:)^{-1}$ may then be replaced by a matrix-vector product with $T_m = I - L_m$. Next, Neumann series are applied to accelerate the classical Rüge-Stuben algebraic multigrid preconditioner using both a polynomial Gauss-Seidel or incomplete ILU smoother. The sparse triangular solver employed in these smoothers is replaced by an inner iteration based upon matrix-vector products. Henrici's departure from normality of the associated iteration matrices leads to a better understanding of these series. Connections are made between the (non)normality of the $L$ and $U$ factors and nonlinear stability analysis, as well as the pseudospectra of the coefficient matrix. Furthermore, re-orderings that preserve structural symmetry also reduce the departure from normality of the upper triangular factor and improve the relative residual of the triangular solves. To demonstrate the effectiveness of this approach on many-core architectures, the proposed solver and preconditioner are applied to the pressure continuity equation for the incompressible Navier-Stokes equations of fluid motion. The pressure solve time is reduced considerably with no change in the convergence rate and the polynomial Gauss-Seidel smoother is compared with a Jacobi smoother. Numerical and timing results are presented for Nalu-Wind and the PeleLM combustion codes, where ILU with iterative triangular solvers is shown to be much more effective than polynomial Gauss-Seidel.

math.NA↗

arXiv2021

Two-Stage Gauss--Seidel Preconditioners and Smoothers for Krylov Solvers on a GPU cluster

Gauss-Seidel (GS) relaxation is often employed as a preconditioner for a Krylov solver or as a smoother for Algebraic Multigrid (AMG). However, the requisite sparse triangular solve is difficult to parallelize on many-core architectures such as graphics processing units (GPUs). In the present study, the performance of the traditional GS relaxation based on a triangular solve is compared with two-stage variants, replacing the direct triangular solve with a fixed number of inner Jacobi-Richardson (JR) iterations. When a small number of inner iterations is sufficient to maintain the Krylov convergence rate, the two-stage GS (GS2) often outperforms the traditional algorithm on many-core architectures. We also compare GS2 with JR. When they perform the same number of flops for SpMV (e.g. three JR sweeps compared to two GS sweeps with one inner JR sweep), the GS2 iterations, and the Krylov solver preconditioned with GS2, may converge faster than the JR iterations. Moreover, for some problems (e.g. elasticity), it was found that JR may diverge with a damping factor of one, whereas two-stage GS may improve the convergence with more inner iterations. Finally, to study the performance of the two-stage smoother and preconditioner for a practical problem, %(e.g. using tuned damping factors), these were applied to incompressible fluid flow simulations on GPUs.

math.NA↗

arXiv2013

A GEMM interface and implementation on NVIDIA GPUs for multiple small matrices

We present an interface and an implementation of the General Matrix Multiply (GEMM) routine for multiple small matrices processed simultaneously on NVIDIA graphics processing units (GPUs). We focus on matrix sizes under 16. The implementation can be easily extended to larger sizes. For single precision matrices, our implementation is 30% to 600% faster than the batched cuBLAS implementation distributed in the CUDA Toolkit 5.0 on NVIDIA Tesla K20c. For example, we obtain 104 GFlop/s and 216 GFlop/s when multiplying 100,000 independent matrix pairs of size 10 and 16, respectively. Similar improvement in performance is obtained for other sizes, in single and double precision for real and complex types, and when the number of matrices is smaller. Apart from our implementation, our different function interface also plays an important role in the improved performance. Applications of this software include Finite Element computation on GPUs.

cs.MS↗

arXiv2008

Chaotic Advection and the Emergence of Tori in the Küppers-Lortz State

Motivated by the roll-switching behavior observed in rotating Rayleigh-Bénard convection, we define a Küppers-Lortz (K-L) state as a volume-preserving flow with periodic roll switching. For an individual roll state, the Lagrangian particle trajectories are periodic. In a system with roll-switching, the particles can exhibit three-dimensional, chaotic motion. We study a simple phenomenological map that models the Lagrangian dynamics in a K-L state. When the roll axes differ by $120^{\circ}$ in the plane of rotation, we show that the phase space is dominated by invariant tori if the ratio of switching time to roll turnover time is small. When this parameter approaches zero these tori limit onto the classical hexagonal convection patterns, and, as it gets large, the dynamics becomes fully chaotic and well-mixed. For intermediate values, there are interlinked toroidal and poloidal structures separated by chaotic regions. We also compute the exit time distributions and show that the unbounded chaotic orbits are normally diffusive. Although the map presumes instantaneous switching between roll states, we show that the qualitative features of the flow persist when the model has smooth, overlapping time-dependence for the roll amplitudes (the Busse-Heikes model).

nlin.CD↗