arXiv subjects

Paul Hovland

Publications and source records attributed to Paul Hovland.

At least 19 recordsLinked to original sources

arXiv2026

ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload

Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare, driving the need for resilient training systems. Yet existing frameworks either focus on specific parallelism schemes or risk drifting away from a failure-free training trajectory. We propose ReCoVer, a resilient LLM pre-training system that upholds a single invariant: each iteration keeps the number of microbatches constant, ensuring per-iteration gradients remain stochastically equivalent to a failure-free run. The framework is organized as three decoupled protocol layers: (1) Fault-tolerant collectives that isolate faults from propagating across replicas; (2) in-step fine-grained recovery that preserves intra-iteration progress and prevents gradient corruption; (3) versatile-workload policy that dynamically redistributes microbatch quotas across the survivors. The design is parallelism-agnostic, integrating directly with both 3D parallelism and Hybrid Sharded Data Parallel (HSDP) as a drop-in substrate. We evaluate our implementation on end-to-end pre-training tasks for up to 512 GPUs, ReCoVer successfully preserves the training trajectory from a failure-free reference despite of 256 GPUs lost spread across the run. For comparison with checkpoint-and-restart baselines, ReCoVer demonstrates $2.23\times$ higher effective throughput after successive failures. This advantage results in ReCoVer processing 74.9% more tokens at 234 GPU-hours, with the gap widening as the training prolongs.

cs.DC↗

arXiv2026

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models

The scale of transformer model pre-training is constrained by the increasing computation and communication cost. Low-rank bottleneck architectures offer a promising solution to significantly reduce the training time and memory footprint with minimum impact on accuracy. Despite algorithmic efficiency, bottleneck architectures scale poorly under standard tensor parallelism. Simply applying 3D parallelism designed for full-rank methods leads to excessive communication and poor GPU utilization. To address this limitation, we propose BOOST, an efficient training framework tailored for large-scale low-rank bottleneck architectures. BOOST introduces a novel Bottleneck-aware Tensor Parallelism, and combines optimizations such as online-RMSNorm, linear layer grouping, and low-rank activation checkpointing to achieve end-to-end training speedup. Evaluations on different low-rank bottleneck architectures demonstrate that BOOST achieves 1.46-1.91$\times$ speedup over full-rank model baselines and 1.87-2.27$\times$ speedup over low-rank model with naively integrated 3D parallelism, with improved GPU utilization and reduced communication overhead.

cs.LG↗

arXiv2026

MonteQ: A Monte Carlo Tree Search Based Quantum Circuit Synthesis Framework

Hamiltonian simulation is one of the most promising paths toward quantum advantage. Most prior approaches to Hamiltonian simulation circuit synthesis focus on local rewrite rules and low-level optimizations, and give limited attention to high-level scheduling of Pauli terms under varying constraints. In practice, different simulation algorithms require different orderings of the Pauli terms, yet many prior IR-based methods assume a fixed commutation structure, which limits their flexibility. We present MonteQ, a novel quantum circuit synthesis framework for Hamiltonian simulation. MonteQ leverages a two-level design that combines low-level synthesis heuristics with an upper-level tree structure to explore sequences of Pauli rotations. To avoid enumerating this factorially large tree, the Monte Carlo Tree Search algorithm serves as workhorse for judiciously exploring promising paths to leaf nodes. With this two-level design, MonteQ supports both logical-level and hardware-aware synthesis by selecting different low-level heuristics. It also supports different ordering constraints on the Pauli rotations by adjusting the high-level tree structure. For example, MonteQ can preserve the target unitary by using a directed acyclic graph that records the commutation relations among the Pauli rotations, or it can relax unitary preservation constraint to uncover additional optimization options. Our experimental results show that MonteQ can achieve an improvement, as measured in CNOT gate counts, of up to 53% (30% on average) against state-of-the-art compilers like Rustiq on a set of representative synthesis tasks.

quant-ph↗

arXiv2025

HOPPS: Hardware-Aware Optimal Phase Polynomial Synthesis with Blockwise Optimization for Quantum Circuits

Blocks composed of {CNOT, Rz} are ubiquitous in modern quantum applications, notably in circuits such as QAOA ansatzes and quantum adders. After compilation, many of them exhibit large CNOT counts or depths, which lowers fidelity. Therefore, we introduce HOPPS: a SAT-based hardware-aware optimal phase polynomial synthesis algorithm that could generate {CNOT, Rz} blocks with CNOT count or depth optimality. Sometime {CNOT, Rz} blocks are large, such as in QAOA ansatzes, HOPPS's pursuit of optimality limits its scalability. To address this issue, we introduce an iterative blockwise optimization strategy: large circuits are partitioned into smaller blocks, each block is optimally refined, and the process is repeated for several iterations. Empirical results show that HOPPS is more efficient comparing with existing near optimal synthesis tools. Used as a peephole optimizer, HOPPS reduces the CNOT count by up to 50.0% and the CNOT depth by up to 57.1% under OLSQ. For large QAOA circuit, after mapping by Qiskit, circuit can be reduced CNOT count and depth by up to 44.4% and 42.4% by our iterative blockwise optimization. Index Terms-Phase Polynomial, Quantum Circuit Synthesis, Quantum Circuit Optimization.

quant-ph↗

arXiv2025

Non-Clifford Fusion: T-Gate Optimization for Quantum Simulation

Hamiltonian simulation is a key quantum algorithm for modeling complex systems. To implement a Hamiltonian simulation, it is typically decomposed into a list of Pauli strings, each corresponds to an RZ rotation gate with many Clifford gates. These RZ gates are generally synthesized into a sequence of Clifford and T gates in fault-tolerant quantum computers, where the T-gate count and T-gate depth are critical metrics for such systems. In this paper, we propose NCF, a compilation framework that reduces both the T-gate count and T-gate depth for Hamiltonian simulation. NCF partitions Pauli strings into groups, where each group can be conjugated (i.e., transformed) into a list of Pauli strings that apply quantum gates on a restricted subset of qubits, allowing for simultaneous synthesis of the whole group and reducing both T-gate count and depth. Experimental results demonstrate that NCF achieves an average reduction of 57.4%, 49.1%, and 49.0% in T-gate count, T-gate depth, and Clifford count, respectively, compared to the state-of-the-art method.

quant-ph↗

arXiv2025

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation

The full-size MLPs and the projection layers in attention introduce tremendous model sizes of large language models (LLMs), consuming extensive computational resources in pre-training. We empirically observe that the activations of pre-trained LLMs exhibit low-rank property. Motivated by such observations, we propose CoLA and its memory-efficient implementation, CoLA-M, to replace these full-size layers with compute-efficient auto-encoders that naturally enforce low-rank activations throughout training. This fundamental architectural change eliminates the activation redundancy and significantly boosts model capacity and training efficiency. Experiments on LLaMA models with 60 million to 7 billion parameters show that CoLA reduces the computing cost by $\bf 2\pmb{\times}$ and improves training throughput by $\bf 1.86\pmb{\times}$ while maintaining full-rank level performance. CoLA-M further squeezes memory cost without sacrificing throughput, offering a pre-training approach with collectively superior parameter, computing, and memory efficiency. The LLMs produced are also $\bf 2\pmb{\times}$ smaller, enabling faster inference with lower memory cost on resource-constrained platforms.

cs.LG↗

arXiv2025

Is Circuit Depth Accurate for Comparing Quantum Circuit Runtimes?

Although quantum circuit depth is commonly used to approximate circuit runtimes, it overlooks a prevailing trait of current hardware implementation: different gates have different execution times. Recognizing the potential for discrepancies, we investigate depth's accuracy for comparing runtimes between compiled versions of the same circuit. In particular, we assess the accuracy of traditional and multi-qubit depth for (1) predicting relative differences in runtime and (2) identifying compiled circuit version(s) with the shortest runtime. Finding that circuit depth is not accurate for either task, we introduce a new metric, gate-aware depth, that weights gates' contributions to runtime using an architecture's average gate execution times. Using average gate times allows gate-aware depth to capture variations by gate type without requiring exact knowledge of all gate times, increasing accuracy while maintaining portability across devices of the same architecture. Compared to traditional and multi-qubit depth, gate-aware depth reduces the average relative error of predictions in task (1) by 68 and 18 times and increases the average number of correct identifications in task (2) by 20 and 43 percentage points, respectively. Finally, we provide gate-aware depth weight configurations for current IBM Eagle and Heron architectures.

quant-ph↗

arXiv2025

State Dependent Optimization with Quantum Circuit Cutting

Quantum circuits can be reduced through optimization to better fit the constraints of quantum hardware. One such method, initial-state dependent optimization (ISDO), reduces gate count by leveraging knowledge of the input quantum states. Surprisingly, we found that ISDO is broadly applicable to the downstream circuits produced by circuit cutting. Circuit cutting also requires measuring upstream qubits and has some flexibility of selection observables to do reconstruction. Therefore, we propose a state-dependent optimization (SDO) framework that incorporates ISDO, our newly proposed measure-state dependent optimization (MSDO), and a biased observable selection strategy. Building on the strengths of the SDO framework and recognizing the scalability challenges of circuit cutting, we propose non-separate circuit cutting-a more flexible approach that enables optimizing gates without fully separating them. We validate our methods on noisy simulations of QAOA, QFT, and BV circuits. Results show that our approach consistently mitigates noise and improves overall circuit performance, demonstrating its promise for enhancing quantum algorithm execution on near-term hardware.

quant-ph↗

arXiv2025

QuCLEAR: Clifford Extraction and Absorption for Quantum Circuit Optimization

Quantum computing carries significant potential for addressing practical problems. However, currently available quantum devices suffer from noisy quantum gates, which degrade the fidelity of executed quantum circuits. Therefore, quantum circuit optimization is crucial for obtaining useful results. In this paper, we present QuCLEAR, a compilation framework designed to optimize quantum circuits. QuCLEAR significantly reduces both the two-qubit gate count and the circuit depth through two novel optimization steps. First, we introduce the concept of Clifford Extraction, which extracts Clifford subcircuits to the end of the circuit while optimizing the gates. Second, since Clifford circuits are classically simulatable, we propose Clifford Absorption, which efficiently processes the extracted Clifford subcircuits classically. We demonstrate our framework on quantum simulation circuits, which have wide-ranging applications in quantum chemistry simulation, many-body physics, and combinatorial optimization problems. Near-term algorithms such as VQE and QAOA also fall within this category. Experimental results across various benchmarks show that QuCLEAR achieves up to a $77.7\%$ reduction in CNOT gate count and up to an $84.1\%$ reduction in entangling depth compared to state-of-the-art methods.

quant-ph↗

arXiv2024

Integrating ytopt and libEnsemble to Autotune OpenMC

ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. The experimental results show that we achieve improvement up to 29.49\% in FoM and up to 30.44\% in EDP.

cs.PF↗

arXiv2024

MITgcm-AD v2: Open source tangent linear and adjoint modeling framework for the oceans and atmosphere enabled by the Automatic Differentiation tool Tapenade

The Massachusetts Institute of Technology General Circulation Model (MITgcm) is widely used by the climate science community to simulate planetary atmosphere and ocean circulations. A defining feature of the MITgcm is that it has been developed to be compatible with an algorithmic differentiation (AD) tool, TAF, enabling the generation of tangent-linear and adjoint models. These provide gradient information which enables dynamics-based sensitivity and attribution studies, state and parameter estimation, and rigorous uncertainty quantification. Importantly, gradient information is essential for computing comprehensive sensitivities and performing efficient large-scale data assimilation, ensuring that observations collected from satellites and in-situ measuring instruments can be effectively used to optimize a large uncertain control space. As a result, the MITgcm forms the dynamical core of a key data assimilation product employed by the physical oceanography research community: Estimating the Circulation and Climate of the Ocean (ECCO) state estimate. Although MITgcm and ECCO are used extensively within the research community, the AD tool TAF is proprietary and hence inaccessible to a large proportion of these users. The new version 2 (MITgcm-AD v2) framework introduced here is based on the source-to-source AD tool Tapenade, which has recently been open-sourced. Another feature of Tapenade is that it stores required variables by default (instead of recomputing them) which simplifies the implementation of efficient, AD-compatible code. The framework has been integrated with the MITgcm model main branch and is now freely available.

physics.ao-ph↗

arXiv2024

QuTracer: Mitigating Quantum Gate and Measurement Errors by Tracing Subsets of Qubits

Quantum error mitigation plays a crucial role in the current noisy-intermediate-scale-quantum (NISQ) era. As we advance towards achieving a practical quantum advantage in the near term, error mitigation emerges as an indispensable component. One notable prior work, Jigsaw, demonstrates that measurement crosstalk errors can be effectively mitigated by measuring subsets of qubits. Jigsaw operates by running multiple copies of the original circuit, each time measuring only a subset of qubits. The localized distributions yielded from measurement subsetting suffer from less crosstalk and are then used to update the global distribution, thereby achieving improved output fidelity. Inspired by the idea of measurement subsetting, we propose QuTracer, a framework designed to mitigate both gate and measurement errors in subsets of qubits by tracing the states of qubit subsets throughout the computational process. In order to achieve this goal, we introduce a technique, qubit subsetting Pauli checks (QSPC), which utilizes circuit cutting and Pauli Check Sandwiching (PCS) to trace the qubit subsets distribution to mitigate errors. The QuTracer framework can be applied to various algorithms including, but not limited to, VQE, QAOA, quantum arithmetic circuits, QPE, and Hamiltonian simulations. In our experiments, we perform both noisy simulations and real device experiments to demonstrate that QuTracer is scalable and significantly outperforms the state-of-the-art approaches.

quant-ph↗

arXiv2024

Challenges with Differentiable Quantum Dynamics

Differentiable quantum dynamics require automatic differentiation of a complex-valued initial value problem, which numerically integrates a system of ordinary differential equations from a specified initial condition, as well as the eigendecomposition of a matrix. We explored several automatic differentiation frameworks for these tasks, finding that no framework natively supports our application requirements. We therefore demonstrate a need for broader support of complex-valued, differentiable numerical integration in scientific computing libraries.

quant-ph↗

arXiv2024

Differentiating Through Linear Solvers

Computer programs containing calls to linear solvers are a known challenge for automatic differentiation. Previous publications advise against differentiating through the low-level solver implementation, and instead advocate for high-level approaches that express the derivative in terms of a modified linear system that can be solved with a separate solver call. Despite this ubiquitous advice, we are not aware of prior work comparing the accuracy of both approaches. With this article we thus empirically study a simple question: What happens if we ignore common wisdom, and differentiate through linear solvers?

cs.MS↗

arXiv2024

Transfer-Learning-Based Autotuning Using Gaussian Copula

As diverse high-performance computing (HPC) systems are built, many opportunities arise for applications to solve larger problems than ever before. Given the significantly increased complexity of these HPC systems and application tuning, empirical performance tuning, such as autotuning, has emerged as a promising approach in recent years. Despite its effectiveness, autotuning is often a computationally expensive approach. Transfer learning (TL)-based autotuning seeks to address this issue by leveraging the data from prior tuning. Current TL methods for autotuning spend significant time modeling the relationship between parameter configurations and performance, which is ineffective for few-shot (that is, few empirical evaluations) tuning on new tasks. We introduce the first generative TL-based autotuning approach based on the Gaussian copula (GC) to model the high-performing regions of the search space from prior data and then generate high-performing configurations for new tasks. This allows a sampling-based approach that maximizes few-shot performance and provides the first probabilistic estimation of the few-shot budget for effective TL-based autotuning. We compare our generative TL approach with state-of-the-art autotuning techniques on several benchmarks. We find that the GC is capable of achieving 64.37% of peak few-shot performance in its first evaluation. Furthermore, the GC model can determine a few-shot transfer budget that yields up to 33.39$\times$ speedup, a dramatic improvement over the 20.58$\times$ speedup using prior techniques.

cs.LG↗

arXiv2023

Efficient precision simulation of processes with many-jet final states at the LHC

We present a scalable technique for the simulation of collider events with multi-jet final states, based on an improved parton-level event file format. The method is implemented for both leading- and next-to-leading order QCD calculations. We perform a comprehensive analysis of the I/O performance and validate our new framework using Higgs-boson plus multi-jet production with up to seven jets. We make the resulting code base available for public use.

hep-ph↗

arXiv2023

Enhancing Virtual Distillation with Circuit Cutting for Quantum Error Mitigation

Virtual distillation is a technique that aims to mitigate errors in noisy quantum computers. It works by preparing multiple copies of a noisy quantum state, bridging them through a circuit, and conducting measurements. As the number of copies increases, this process allows for the estimation of the expectation value with respect to a state that approaches the ideal pure state rapidly. However, virtual distillation faces a challenge in realistic scenarios: preparing multiple copies of a quantum state and bridging them through a circuit in a noisy quantum computer will significantly increase the circuit size and introduce excessive noise, which will degrade the performance of virtual distillation. To overcome this challenge, we propose an error mitigation strategy that uses circuit-cutting technology to cut the entire circuit into fragments. With this approach, the fragments responsible for generating the noisy quantum state can be executed on a noisy quantum device, while the remaining fragments are efficiently simulated on a noiseless classical simulator. By running each fragment circuit separately on quantum and classical devices and recombining their results, we can reduce the noise accumulation and enhance the effectiveness of the virtual distillation technique. Our strategy has good scalability in terms of both runtime and computational resources. We demonstrate our strategy's effectiveness through noisy simulation and experiments on a real quantum device.

quant-ph↗

arXiv2023

Understanding Automatic Differentiation Pitfalls

Automatic differentiation, also known as backpropagation, AD, autodiff, or algorithmic differentiation, is a popular technique for computing derivatives of computer programs accurately and efficiently. Sometimes, however, the derivatives computed by AD could be interpreted as incorrect. These pitfalls occur systematically across tools and approaches. In this paper we broadly categorize problematic usages of AD and illustrate each category with examples such as chaos, time-averaged oscillations, discretizations, fixed-point loops, lookup tables, and linear solvers. We also review debugging techniques and their effectiveness in these situations. With this article we hope to help readers avoid unexpected behavior, detect problems more easily when they occur, and have more realistic expectations from AD tools.

math.NA↗