Searcharxiv⌕ Search

arXiv · 2611.22183

A Compilation Model for Divergent Parallel Workloads

Abstract

Efficient execution on modern parallel machines requires more than abundant parallelism; it demands instruction coherence, memory coherence, and load balance to maximize SIMD utilization and memory bandwidth. Efficiently mapping \textit{divergent} parallel workloads to modern machines, where control flow, memory accesses, and unbounded iteration depend on per-thread runtime values, often requires global program restructuring that trades off increased coherence with reduced locality, parallelism, or extra work. We observe that the key optimizations used to accelerate divergent parallel programs can be expressed as compositions of well-known program transformations from the functional programming and loop optimization communities. We operationalize this insight through \textit{parallel dispatch graphs}: a compilation model that represents divergent programs by representing control flow, data flow between logical stages, and parallel loops within a single abstraction. To facilitate rapid exploration of points in the optimization space, we provide a scheduling language that includes composable transformations of data-dependent control flow and data flow, together with standard loop transformations and hardware bindings. This separation of concerns allows the core algorithm to remain portable, while the execution schedule can be specialized to the specifics of target hardware and input data values. We evaluate our system using popular applications from the domain of spatial computing, and show that the optimization strategies of hand-optimized parallel tree traversals, path tracers, and Monte Carlo PDE solvers can be reproduced as schedules in our system. Our generated code matches or outperforms the hand-optimized implementations (geomeans $1.02$--$2.95\times$ faster). Additionally, we demonstrate empirically that the best schedule varies across programs, machines, and inputs.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Alexander J Root, Christophe Gyurgyik, Devanshu Ladsaria, Rohan Sawhney, Jonathan Ragan-Kelley, Andrew Adams, Matt Pharr, Kayvon Fatahalian, Fredrik Kjolstad. 2026-11-17. A Compilation Model for Divergent Parallel Workloads. https://searcharxiv.com/abs/2611.22183

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

The Best of Times, the Worst of Times: Moment-Based Analysis of Probabilistic Cost Structures

Many practical costs---such as the completion time of a quantum repeater, the rounds of an RFID collision-resolution protocol, or the response time of a fork--join computation---are described by hierarchical cost expressions: a tree of independent subcomputations with nonnegative integer costs, combined by addition, max, and min. We compute their moments with sound error bounds. Moment-based analyses for additive costs do not apply, because the moments of $\max(X,Y)$ are not determined by those of $X$ and $Y$; propagating full distributions becomes expensive as trees grow; and scalable approximations are hand-derived for specific structures and lack error bounds. Our analysis proceeds bottom-up, summarizing each subtree by a surrogate distribution: an exact short-horizon prefix plus a compact parametric tail fitted to preserve the queried moment. Each prefix is computed exactly from the children's prefixes---for a retry, whose cost sums an unbounded number of attempts, by solving a local recurrence equation---so all approximation is confined to the tails. Abstraction introduces shape error but no moment error; nonlinear operators, such as max and min, convert shape error into moment error. The analysis tracks two bounds, one on shape error and one on moment error. Tracking both lets it report a sound interval for the mean, the second moment, and the variance. Lengthening the prefix, or solving subtrees exactly, narrows the interval at the cost of additional computation. For quantum-repeater chains of up to 4096 links with high swap-success probability, our Dickens tool bounds the mean completion time within 1%, whereas the exact Markov-chain solution exhausts memory with 16 links, and the best hand-derived approximation we evaluated has 47% error with 64 links. Dickens also bounds second moments, variances, and deadline-miss probabilities, and handles RFID and fork--join costs.

cs.PL↗

Datatypes In, Zippers Out!

A zipper represents a focused position in an algebraic data structure together with the context needed to reconstruct the whole. Datatype derivatives determine one-hole context types, but do not by themselves choose a zipper's concrete representation or specify navigation costs and demands. We present Kamot, a tool that converts regular algebraic datatype declarations in a restricted Haskell fragment into zipper implementations with operational contracts. Navigation, replacement, and reconstruction are proved correct against a reference traversal. Kamot supports mutual recursion and regular containers inside containers. A compositional analysis derives sound bounds on transitions and constructor demands for certified traversal directions. These bounds are affine in trail depth (the number of saved frames) and guarantee termination of each move on productive, potentially infinite inputs. For inhabited families the analysis is complete: when no move certificate is emitted, no bound depending only on trail depth exists, and some productive input makes a move diverge. We prove that the generated code preserves results and the reference order of constructor matches, including finite prefixes of divergent searches. Lean 4 verifies simulation, bounds, and completeness for finite values of single self-recursive datatypes, in both directions. On 19 datatype models and 81 generated families, every generated module passes Haskell type checking, and independent checkers validate certificates, traces, and bounds. In a prefetching workload, starting only searches whose bounds fit a budget answers every local query and avoids discarded source evaluations on long returns, but conservative bounds can lose answers that fuelled search delivers.

cs.PL↗

Python Code Generation for Isabelle/HOL

The Isabelle theorem prover provides an environment for formalising mathematics and reasoning about programs. To make verified implementations practically usable, its code generator translates functions expressed in the executable fragment of its logic into conventional programming languages. Its targets have traditionally been functional languages (Standard ML, OCaml, Haskell, and Scala), recently joined by Go. Python, which is becoming the dominant language of scientific computing, data science, and education, has remained out of reach. In this work, we close this gap with a Python backend for Isabelle's code generator. This generator enables verified algorithms to be used in Python applications, or to serve as verified references against which established Python libraries can be differentially tested, without leaving the Python ecosystem. We validate the backend on regression suites covering large parts of the HOL library and by testing six verified data structures and algorithms against established Python libraries. The generated code is slower than hand-written libraries, but well suited as a verified reference.

cs.PL↗