arXiv subjects

Alexei Strelchenko

Publications and source records attributed to Alexei Strelchenko.

18 recordsLinked to original sources

arXiv2024

Exploring code portability solutions for HEP with a particle tracking test code

Traditionally, high energy physics (HEP) experiments have relied on x86 CPUs for the majority of their significant computing needs. As the field looks ahead to the next generation of experiments such as DUNE and the High-Luminosity LHC, the computing demands are expected to increase dramatically. To cope with this increase, it will be necessary to take advantage of all available computing resources, including GPUs from different vendors. A broad landscape of code portability tools -- including compiler pragma-based approaches, abstraction libraries, and other tools -- allow the same source code to run efficiently on multiple architectures. In this paper, we use a test code taken from a HEP tracking algorithm to compare the performance and experience of implementing different portability solutions.

hep-ex↗

arXiv2024

Application of performance portability solutions for GPUs and many-core CPUs to track reconstruction kernels

Next generation High-Energy Physics (HEP) experiments are presented with significant computational challenges, both in terms of data volume and processing power. Using compute accelerators, such as GPUs, is one of the promising ways to provide the necessary computational power to meet the challenge. The current programming models for compute accelerators often involve using architecture-specific programming languages promoted by the hardware vendors and hence limit the set of platforms that the code can run on. Developing software with platform restrictions is especially unfeasible for HEP communities as it takes significant effort to convert typical HEP algorithms into ones that are efficient for compute accelerators. Multiple performance portability solutions have recently emerged and provide an alternative path for using compute accelerators, which allow the code to be executed on hardware from different vendors. We apply several portability solutions, such as Kokkos, SYCL, C++17 std::execution::par and Alpaka, on two mini-apps extracted from the mkFit project: p2z and p2r. These apps include basic kernels for a Kalman filter track fit, such as propagation and update of track parameters, for detectors at a fixed z or fixed r position, respectively. The two mini-apps explore different memory layout formats. We report on the development experience with different portability solutions, as well as their performance on GPUs and many-core CPUs, measured as the throughput of the kernels from different GPU and CPU vendors such as NVIDIA, AMD and Intel.

physics.acc-ph↗

arXiv2023

Evaluating Portable Parallelization Strategies for Heterogeneous Architectures in High Energy Physics

High-energy physics (HEP) experiments have developed millions of lines of code over decades that are optimized to run on traditional x86 CPU systems. However, we are seeing a rapidly increasing fraction of floating point computing power in leadership-class computing facilities and traditional data centers coming from new accelerator architectures, such as GPUs. HEP experiments are now faced with the untenable prospect of rewriting millions of lines of x86 CPU code, for the increasingly dominant architectures found in these computational accelerators. This task is made more challenging by the architecture-specific languages and APIs promoted by manufacturers such as NVIDIA, Intel and AMD. Producing multiple, architecture-specific implementations is not a viable scenario, given the available person power and code maintenance issues. The Portable Parallelization Strategies team of the HEP Center for Computational Excellence is investigating the use of Kokkos, SYCL, OpenMP, std::execution::parallel and alpaka as potential portability solutions that promise to execute on multiple architectures from the same source code, using representative use cases from major HEP experiments, including the DUNE experiment of the Long Baseline Neutrino Facility, and the ATLAS and CMS experiments of the Large Hadron Collider. This cross-cutting evaluation of portability solutions using real applications will help inform and guide the HEP community when choosing their software and hardware suites for the next generation of experimental frameworks. We present the outcomes of our studies, including performance metrics, porting challenges, API evaluations, and build system integration.

hep-ex↗

arXiv2023

Two-link Staggered Quark Smearing in QUDA

Gauge covariant smearing based on the 3D lattice Laplacian can be used to create extended operators that have better overlap with hadronic ground states. For staggered quarks, we make use of two-link parallel transport to preserve taste properties. We have implemented the procedure in QUDA. We present the performance of this code on the NVIDIA A100 GPUs in Indiana University's Big Red 200 supercomputer and on the AMD MI250X GPUs in Oak Ridge Leadership Computer Facility's (OLCF's) Crusher and discuss its scalability. We also study the performance improvement from using NVSHMEM on OLCF's Summit. Reusing precomputed two-link products for all sources and sinks, it reduces the total smearing time for a baryon correlator measurement by a factor of 100-120 as compared with the original MILC code and reduces the overall time by 60-70%.

hep-lat↗

arXiv2022

Lattice QCD and Particle Physics

Contribution from the USQCD Collaboration to the Proceedings of the US Community Study on the Future of Particle Physics (Snowmass 2021).

hep-lat↗

arXiv2022

Portability: A Necessary Approach for Future Scientific Software

Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High Energy Physics Center for Computational Excellence (HEP/CCE) is investigating solutions for portability techniques that will allow the coding of an algorithm once, and the ability to execute it on a variety of hardware products from many vendors, especially including accelerators. We think without these solutions, the scientific success of our experiments and endeavors is in danger, as software development could be expert driven and costly to be able to run on available hardware infrastructure. We think the best solution for the community would be an extension to the C++ standard with a very low entry bar for users, supporting all hardware forms and vendors. We are very far from that ideal though. We argue that in the future, as a community, we need to request and work on portability solutions and strive to reach this ideal.

physics.comp-ph↗

arXiv2021

Porting CMS Heterogeneous Pixel Reconstruction to Kokkos

Programming for a diverse set of compute accelerators in addition to the CPU is a challenge. Maintaining separate source code for each architecture would require lots of effort, and development of new algorithms would be daunting if it had to be repeated many times. Fortunately there are several portability technologies on the market such as Alpaka, Kokkos, and SYCL. These technologies aim to improve the developer productivity by making it possible to use the same source code for many different architectures. In this paper we use heterogeneous pixel reconstruction code from the CMS experiment at the CERNL LHC as a realistic use case of a GPU-targeting HEP reconstruction software, and report experience from prototyping a portable version of it using Kokkos. The development was done in a standalone program that attempts to model many of the complexities of a HEP data processing framework such as CMSSW. We also compare the achieved event processing throughput to the original CUDA code and a CPU version of it.

physics.comp-ph↗

arXiv2020

Computing Nucleon Charges with Highly Improved Staggered Quarks

This work continues our program of lattice-QCD baryon physics using staggered fermions for both the sea and valence quarks. We present a proof-of-concept study that demonstrates, for the first time, how to calculate baryon matrix elements using staggered quarks for the valence sector. We show how to relate the representations of the continuum staggered flavor-taste group $\text{SU}(8)_{FT}$ to those of the discrete lattice symmetry group. The resulting calculations yield the normalization factors relating staggered baryon matrix elements to their physical counterparts. We verify this methodology by calculating the isovector vector and axial-vector charges $g_V$ and $g_A$. We use a single ensemble from the MILC Collaboration with 2+1+1 flavors of sea quark, lattice spacing $a\approx 0.12$ fm, and a pion mass $M_\pi\approx305$ MeV. On this ensemble, we find results consistent with expectations from current conservation and neutron beta decay. Thus, this work demonstrates how highly-improved staggered quarks can be used for precision calculations of baryon properties, and, in particular, the isovector nucleon charges.

hep-lat↗

arXiv2019

Nucleon Mass with Highly Improved Staggered Quarks

We present the first computation in a program of lattice-QCD baryon physics using staggered fermions for sea and valence quarks. For this initial study, we present a calculation of the nucleon mass, obtaining $964\pm16$ MeV with all sources of statistical and systematic errors controlled and accounted for. This result is the most precise determination to date of the nucleon mass from first principles. We use the highly-improved staggered quark action, which is computationally efficient. Three gluon ensembles are employed, which have approximate lattice spacings $a=0.09$ fm, $0.12$ fm, and $0.15$ fm, each with equal-mass $u$/$d$, $s$, and $c$ quarks in the sea. Further, all ensembles have the light valence and sea $u$/$d$ quarks tuned to reproduce the physical pion mass, avoiding complications from chiral extrapolations or nonunitarity. Our work opens a new avenue for precise calculations of baryon properties, which are both feasible and relevant to experiments in particle and nuclear physics.

hep-lat↗

arXiv2018

Multigrid for Staggered Lattice Fermions

Critical slowing down in Krylov methods for the Dirac operator presents a major obstacle to further advances in lattice field theory as it approaches the continuum solution. Here we formulate a multi-grid algorithm for the Kogut-Susskind (or staggered) fermion discretization which has proven difficult relative to Wilson multigrid due to its first-order anti-Hermitian structure. The solution is to introduce a novel spectral transformation by the K\"ahler-Dirac spin structure prior to the Galerkin projection. We present numerical results for the two-dimensional, two-flavor Schwinger model, however, the general formalism is agnostic to dimension and is directly applicable to four-dimensional lattice QCD.

hep-lat↗

arXiv2017

Pushing Memory Bandwidth Limitations Through Efficient Implementations of Block-Krylov Space Solvers on GPUs

Lattice quantum chromodynamics simulations in nuclear physics have benefited from a tremendous number of algorithmic advances such as multigrid and eigenvector deflation. These improve the time to solution but do not alleviate the intrinsic memory-bandwidth constraints of the matrix-vector operation dominating iterative solvers. Batching this operation for multiple vectors and exploiting cache and register blocking can yield a super-linear speed up. Block-Krylov solvers can naturally take advantage of such batched matrix-vector operations, further reducing the iterations to solution by sharing the Krylov space between solves. However, practical implementations typically suffer from the quadratic scaling in the number of vector-vector operations. Using the QUDA library, we present an implementation of a block-CG solver on NVIDIA GPUs which reduces the memory-bandwidth complexity of vector-vector operations from quadratic to linear. We present results for the HISQ discretization, showing a 5x speedup compared to highly-optimized independent Krylov solves on NVIDIA's SaturnV cluster.

hep-lat↗

arXiv2016

Accelerating Lattice QCD Multigrid on GPUs Using Fine-Grained Parallelization

The past decade has witnessed a dramatic acceleration of lattice quantum chromodynamics calculations in nuclear and particle physics. This has been due to both significant progress in accelerating the iterative linear solvers using multi-grid algorithms, and due to the throughput improvements brought by GPUs. Deploying hierarchical algorithms optimally on GPUs is non-trivial owing to the lack of parallelism on the coarse grids, and as such, these advances have not proved multiplicative. Using the QUDA library, we demonstrate that by exposing all sources of parallelism that the underlying stencil problem possesses, and through appropriate mapping of this parallelism to the GPU architecture, we can achieve high efficiency even for the coarsest of grids. Results are presented for the Wilson-Clover discretization, where we demonstrate up to 10x speedup over present state-of-the-art GPU-accelerated methods on Titan. Finally, we look to the future, and consider the software implications of our findings.

hep-lat↗

arXiv2015

Accelerating Twisted Mass LQCD with QPhiX

We present the implementation of twisted mass fermion operators for the QPhiX library. We analyze the performance on the Intel Xeon Phi (Knights Corner) coprocessor as well as on Intel Xeon Haswell CPUs. In particular, we demonstrate that on the Xeon Phi 7120P the Dslash kernel is able to reach 80\% of the theoretical peak bandwidth, while on a Xeon Haswell E5-2630 CPU our generated code for the Dslash operator with AVX2 instructions outperforms the corresponding implementation in the tmLQCD library by a factor of $\sim 5\times$ in single precision. We strong scale the code up to 6.8 (14.1) Tflops in single (half) precision on 64 Xeon Haswell CPUs.

hep-lat↗

arXiv2014

A QUDA-branch to compute disconnected diagrams in GPUs

Although QUDA allows for an efficient computation of many QCD quantities, it is surprinsingly lacking tools to evaluate disconnected diagrams, for which GPUs are specially well suited. We aim to fill this gap by creating our own branch of QUDA, which includes new kernels and functions required to calculate fermion loops using several methods and fermionic regularizations.

hep-lat↗

arXiv2013

Implementation of the twisted mass fermion operator in the QUDA library

We discuss an extension of the QUDA library for the Wilson twisted mass operator. A performance analysis is presented for both degenerate and non-degenerate flavor doublets. The degenerate twisted mass fermion operator runs at up to 190, 487 and 856 Gflops, for double, single and half precisions respectively on recent NVIDIA Kepler GPUs, while our implementation for the non-degenerate flavor doublet allows to reach 163, 516 and 879 GFlops, respectively. The code is currently in production for the hadron structure study.

hep-lat↗

arXiv2009

Monte Carlo simulations on Graphics Processing Units

Implementation of basic local Monte-Carlo algorithms on ATI Graphics Processing Units (GPU) is investigated. The Ising model and pure SU(2) gluodynamics simulations are realized with the Compute Abstraction Layer (CAL) of ATI Stream environment using the Metropolis and the heat-bath algorithms, respectively. We present an analysis of both CAL programming model and the efficiency of the corresponding simulation algorithms on GPU. In particular, the significant performance speed-up of these algorithms in comparison with serial execution is observed.

hep-lat↗

arXiv2008

On space-time noncommutative U(1) model at high temperature

We extend the results of Ref. [arXiv:0705.4294] to noncommutative gauge theories at finite temperature. In particular, by making use of the background field method, we analyze renormalization issues and the high-temperature asymptotics of the one-loop Euclidean free energy of the noncommutative U(1) gauge model within imaginary time formalism. As a by-product, the heat trace of the non-minimal photon kinetic operator on noncommutative $S^1 \times R^3$ manifold taken in an arbitrary background gauge is investigated. All possible types of noncommutativity on $S^1 \times R^3$ are considered. It is demonstrated that the non-planar sector of the model does not contribute to the free energy of the system at high temperature. The obtained results are discussed.

hep-th↗

arXiv2007

Heat kernel of non-minimal gauge field kinetic operators on Moyal plane

We generalize the Endo formula originally developed for the computation of the heat kernel asymptotic expansion for non-minimal operators in commutative gauge theories to the noncommutative case. In this way, the first three non-zero heat trace coefficients of the non-minimal U(N) gauge field kinetic operator on the Moyal plane taken in an arbitrary background are calculated. We show that the non-planar part of the heat trace asymptotics is determined by U(1) sector of the gauge model. The non-planar or mixed heat kernel coefficients are shown to be gauge-fixing dependent in any dimension of space-time. In the case of the degenerate deformation parameter the lowest mixed coefficients in the heat expansion produce non-local gauge-fixing dependent singularities of the one-loop effective action that destroy the renormalizability of the U(N) model at one-loop level. The twisted-gauge transformation approach is discussed.

hep-th↗