Searcharxiv⌕ Search

arXiv subjects

Michael Sullivan

Publications and source records attributed to Michael Sullivan.

At least 19 recordsLinked to original sources
arXiv2026

Accelerating Constrained Decoding with Token Space Compression

To guarantee that an LLM's outputs conform to a specified structure, context-free grammar (CFG) decoding engines force the selection of next tokens that produce strings that conform to a given CFG. While current CFG-constrained decoding engines are highly optimized, the inherent costs arising from the massive per-step search space -- i.e. the entire token vocabulary -- result in intractably high overhead for more complex CFGs: precisely the situation where CFG engines are most useful. In this paper, we introduce CFGzip, an offline technique for compressing the token search space, which massively reduces CFG engine overhead. In experiments, we report latency reduction of up to two orders of magnitude when CFGzip is used with a SoTA grammar engine, yielding an up to 7.5x speedup in total constrained generation time: with CFGzip, constrained decoding is now feasible at scale for complex CFGs.

cs.AI↗
arXiv2026

HyperLiDAR: Adaptive Post-Deployment LiDAR Segmentation via Hyperdimensional Computing

LiDAR semantic segmentation plays a pivotal role in 3D scene understanding for edge applications such as autonomous driving. However, significant challenges remain for real-world deployments, particularly for on-device post-deployment adaptation. Real-world environments can shift as the system navigates through different locations, leading to substantial performance degradation without effective and timely model adaptation. Furthermore, edge systems operate under strict computational and energy constraints, making it infeasible to adapt conventional segmentation models (based on large neural networks) directly on-device. To address the above challenges, we introduce HyperLiDAR, the first lightweight, post-deployment LiDAR segmentation framework based on Hyperdimensional Computing (HDC). The design of HyperLiDAR fully leverages the fast learning and high efficiency of HDC, inspired by how the human brain processes information. To further improve the adaptation efficiency, we identify the high data volume per scan as a key bottleneck and introduce a buffer selection strategy that focuses learning on the most informative points. We conduct extensive evaluations on two state-of-the-art LiDAR segmentation benchmarks and two representative devices. Our results show that HyperLiDAR outperforms or achieves comparable adaptation performance to state-of-the-art segmentation methods, while achieving up to a 13.8x speedup in retraining.

cs.CV↗
arXiv2026

Legendrian and Lagrangian higher torsion

Let $M$ be a closed manifold. We introduce a family of Legendrian isotopy invariants for Legendrians in $J^1M$, which we collectively call Legendrian higher torsion. Given a choice of a class $\mathcal{F}$ of fibre bundles over $M$, equipped with suitable unitary local systems, the Legendrian higher torsion of a Legendrian $\Lambda \subset J^1M$ is the subset of $H^*(M;\mathbf{R})$ consisting of higher Reidemeister torsion cohomology classes of fibre bundles $W$ over $M$ in the class $\mathcal{F}$ such that $\Lambda$ admits a generating function on a stabilization of $W$. For the class of tube bundles in the sense of Waldhausen we call the invariant tube torsion. We show that the tube torsion of a nearby Lagrangian $L \subset T^*M$ is well-defined when the stable Gauss map $L \to U/O$ is trivial (for example when $L$ is a nearby Lagrangian homotopy sphere) and it consists of a union of cosets of a normalized version of the Pontryagin character. We also identify a distinguished coset, invariant under Hamiltonian isotopy of $L$, which we call nearby Lagrangian torsion. We do not know whether nearby Lagrangians must have trivial tube torsion, as would follow from the nearby Lagrangian conjecture. However, we show that there exist Legendrians $\Lambda \subset J^1M$ with nontrivial tube torsion whose projection $\Lambda \to M$ is homotopic to a diffeomorphism.

math.SG↗
arXiv2026

AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing

The task of authorship style transfer involves rewriting text in the style of a target author while preserving the meaning of the original text. Existing style transfer methods train a single model on large corpora to model all target styles at once: this high-cost approach offers limited flexibility for target-specific adaptation, and often sacrifices meaning preservation for style transfer. In this paper, we propose AuthorMix: a lightweight, modular, and interpretable style transfer framework. We train individual, style-specific LoRA adapters on a small set of high-resource authors, allowing the rapid training of specialized adaptation models for each new target via learned, layer-wise adapter mixing, using only a handful of target-style training examples. AuthorMix outperforms existing, SoTA style-transfer baselines-as well as GPT-5.1-for low-resource targets, achieving the highest overall score and substantially improving meaning preservation in both automatic and human evaluations.

cs.CL↗
arXiv2025

Serial Polar Automorphism Ensemble Decoders for Physical Unclonable Functions

Physical unclonable functions (PUFs) involve challenging practical applications of error-correcting codes (ECCs), requiring extremely low failure rates on the order of $10^{-6}$ and below despite raw input bit error rates as high as 22%. These requirements call for an efficient ultra-low rate code design. In this work, we propose a novel coding scheme tailored for PUFs based on Polar codes and a low-complexity version of automorphism ensemble decoding (AED). Notably, our serial AED scheme reuses a single successive cancellation (SC) decoder across multiple decoding attempts. By introducing cascaded and recursive interleavers, we efficiently scale the number of AED candidates without requiring expensive large multiplexers. An aggressive quantization strategy of only 3 bits per message further reduces the area requirements of the underlying SC decoder. The resulting coding scheme achieves the same block error rate of $10^{-6}$ as our baseline based on Bose-Ray-Chaudhuri-Hocquenghem (BCH) codes while requiring 1.75x fewer codeword bits to encode the same K = 312 payload bits. This reduction translates directly into 1.75x less helper data storage and, consequently, a smaller overall chip area.

cs.IT↗
arXiv2025

Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions

In this paper, we propose a novel, automatic tree-based evaluation metric for LLM-generated step-by-step assembly instructions, that more accurately reflects spatiotemporal aspects of construction than traditional metrics such as BLEU and BERT similarity scores. We apply our proposed metric to the domain of sewing instructions, and show that our metric better correlates with manually-annotated error counts as well as human quality ratings, demonstrating our metric's superiority for evaluating the spatiotemporal soundness of sewing instructions. Further experiments show that our metric is more robust than traditional approaches against artificially-constructed counterfactual examples that are specifically constructed to confound metrics that rely on textual similarity.

cs.CL↗
arXiv2025

GRPO is Secretly a Process Reward Model

Process reward models (PRMs) allow for fine-grained credit assignment in reinforcement learning (RL), and seemingly contrast with outcome reward models (ORMs), which assign a single reward to an entire trajectory. However, we provide theoretical proof in this work that the Group Relative Policy Optimization (GRPO) RL algorithm equipped with an ORM is in fact equivalent to a PRM-aware RL objective equipped with a non-trivial, Monte-Carlo-based PRM (given mild assumptions). Leveraging the framework of GRPO-as-a-PRM, we identify a flaw in the GRPO objective that interacts with imbalanced process steps and rewards to hinder both exploration and exploitation (under different conditions). We propose a simple modification to the algorithm to mitigate this defect ($\lambda$-GRPO), and show that LLMs tuned with $\lambda$-GRPO outperform LLMs tuned with standard GRPO on downstream reasoning tasks\textemdash and reach peak performance more rapidly. These results show that we can leverage the hidden, built-in PRM structure within the vanilla GRPO algorithm to boost model performance without employing an explicit PRM, and with a negligible impact on training time and cost.

cs.LG↗
arXiv2025

Procedural Environment Generation for Tool-Use Agents

Although the power of LLM tool-use agents has ignited a flurry of recent research in this area, the curation of tool-use training data remains an open problem$-$especially for online RL training. Existing approaches to synthetic tool-use data generation tend to be non-interactive, and/or non-compositional. We introduce RandomWorld, a pipeline for the procedural generation of interactive tools and compositional tool-use data. We show that models tuned via SFT and RL on synthetic RandomWorld data improve on a range of tool-use benchmarks, and set the new SoTA for two metrics on the NESTFUL dataset. Further experiments show that downstream performance scales with the amount of RandomWorld-generated training data, opening up the possibility of further improvement through the use of entirely synthetic data.

cs.LG↗
arXiv2025

Exploring Graph Representations of Logical Forms for Language Modeling

We make the case for language models over logical forms (LFLMs), arguing that such models are more data-efficient than their textual counterparts. To that end, we introduce the Graph-based Formal-Logical Distributional Semantics (GFoLDS) prototype, a pretrained LM over graph representations of logical forms, as a proof-of-concept of LFLMs. Using GFoLDS, we present strong experimental evidence that LFLMs can leverage the built-in, basic linguistic knowledge inherent in such models to immediately begin learning more complex patterns. On downstream tasks, we show that GFoLDS vastly outperforms textual, transformer LMs (BERT) pretrained on the same data, indicating that LFLMs can learn with substantially less data than models over plain text. Furthermore, we show that the performance of this model is likely to scale with additional parameters and pretraining data, suggesting the viability of LFLMs in real-world applications.

cs.CL↗
arXiv2025

Playpen: An Environment for Exploring Learning Through Conversational Interaction

Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appropriateness of a model's response. In this paper, we investigate whether Dialogue Games -- goal-directed and rule-governed activities driven predominantly by verbal actions -- can also serve as a source of feedback signals for learning. We introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning; direct alignment (DPO); and reinforcement learning with GRPO. We experiment with post-training a small LLM (Llama-3.1-8B-Instruct), evaluating performance on unseen instances of training games as well as unseen games, and on standard benchmarks. We find that imitation learning through SFT improves performance on unseen instances, but negatively impacts other skills, while interactive learning with GRPO shows balanced improvements without loss of skills. We release the framework and the baseline training setups to foster research in the promising new direction of learning in (synthetic) interaction.

cs.CL↗
arXiv2020

Estimating Silent Data Corruption Rates Using a Two-Level Model

High-performance and safety-critical system architects must accurately evaluate the application-level silent data corruption (SDC) rates of processors to soft errors. Such an evaluation requires error propagation all the way from particle strikes on low-level state up to the program output. Existing approaches that rely on low-level simulations with fault injection cannot evaluate full applications because of their slow speeds, while application-level accelerated fault testing in accelerated particle beams is often impractical. We present a new two-level methodology for application resilience evaluation that overcomes these challenges. The proposed approach decomposes application failure rate estimation into (1) identifying how particle strikes in low-level unprotected state manifest at the architecture-level, and (2) measuring how such architecture-level manifestations propagate to the program output. We demonstrate the effectiveness of this approach on GPU architectures. We also show that using just one of the two steps can overestimate SDC rates and produce different trends---the composition of the two is needed for accurate reliability modeling.

cs.DC↗
arXiv2019

Kayotee: A Fault Injection-based System to Assess the Safety and Reliability of Autonomous Vehicles to Faults and Errors

Fully autonomous vehicles (AVs), i.e., AVs with autonomy level 5, are expected to dominate road transportation in the near-future and contribute trillions of dollars to the global economy. The general public, government organizations, and manufacturers all have significant concern regarding resiliency and safety standards of the autonomous driving system (ADS) of AVs . In this work, we proposed and developed (a) `Kayotee' - a fault injection-based tool to systematically inject faults into software and hardware components of the ADS to assess the safety and reliability of AVs to faults and errors, and (b) an ontology model to characterize errors and safety violations impacting reliability and safety of AVs. Kayotee is capable of characterizing fault propagation and resiliency at different levels - (a) hardware, (b) software, (c) vehicle dynamics, and (d) traffic resilience. We used Kayotee to study a proprietary ADS technology built by Nvidia corporation and are currently applying Kayotee to other open-source ADS systems.

cs.SE↗
arXiv2019

Machine detector interface for the $e^+e^-$ future circular collider

The international Future Circular Collider (FCC) study aims at a design of $pp$, $e^+e^-$, $ep$ colliders to be built in a new 100 km tunnel in the Geneva region. The $e^+e^-$ collider (FCC-ee) has a centre of mass energy range between 90 (Z-pole) and 375 GeV (tt_bar). To reach such unprecedented energies and luminosities, the design of the interaction region is crucial. The crab-waist collision scheme has been chosen for the design and it will be compatible with all beam energies. In this paper we will describe the machine detector interface layout including the solenoid compensation scheme. We will describe how this layout fulfills all the requirements set by the parameters table and by the physical constraints. We will summarize the studies of the impact of the synchrotron radiation, the analysis of trapped modes and of the backgrounds induced by single beam and luminosity effects giving an estimate of the losses in the interaction region and in the detector.

physics.acc-ph↗
arXiv2019

Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs

GPUs offer orders-of-magnitude higher memory bandwidth than traditional CPU-only systems. However, GPU device memory tends to be relatively small and the memory capacity can not be increased by the user. This paper describes Buddy Compression, a scheme to increase both the effective GPU memory capacity and bandwidth while avoiding the downsides of conventional memory-expanding strategies. Buddy Compression compresses GPU memory, splitting each compressed memory entry between high-speed device memory and a slower-but-larger disaggregated memory pool (or system memory). Highly-compressible memory entries can thus be accessed completely from device memory, while incompressible entries source their data using both on and off-device accesses. Increasing the effective GPU memory capacity enables us to run larger-memory-footprint HPC workloads and larger batch-sizes or models for DL workloads than current memory capacities would allow. We show that our solution achieves an average compression ratio of 2.2x on HPC workloads and 1.5x on DL workloads, with a slowdown of just 1~2%.

cs.AR↗
arXiv2018

CRUM: Checkpoint-Restart Support for CUDA's Unified Memory

Unified Virtual Memory (UVM) was recently introduced on recent NVIDIA GPUs. Through software and hardware support, UVM provides a coherent shared memory across the entire heterogeneous node, migrating data as appropriate. The older CUDA programming style is akin to older large-memory UNIX applications which used to directly load and unload memory segments. Newer CUDA programs have started taking advantage of UVM for the same reasons of superior programmability that UNIX applications long ago switched to assuming the presence of virtual memory. Therefore, checkpointing of UVM will become increasingly important, especially as NVIDIA CUDA continues to gain wider popularity: 87 of the top 500 supercomputers in the latest listings are GPU-accelerated, with a current trend of ten additional GPU-based supercomputers each year. A new scalable checkpointing mechanism, CRUM (Checkpoint-Restart for Unified Memory), is demonstrated for hybrid CUDA/MPI computations across multiple computer nodes. CRUM supports a fast, forked checkpointing, which mostly overlaps the CUDA computation with storage of the checkpoint image in stable storage. The runtime overhead of using CRUM is 6% on average, and the time for forked checkpointing is seen to be a factor of up to 40 times less than traditional, synchronous checkpointing.

cs.DC↗
arXiv2012

Transverse string topology and the cord algebra

We define a coalgebra structure for open strings transverse to any framed codimension 2 submanifold. When the submanifold is a knot in R^3, we show this structure recovers a specialization of the Ng cord algebra, a non-trivial knot invariant which is not determined by a number of other knot invariants.

math.GT↗
arXiv2011

Knot contact homology

The conormal lift of a link $K$ in $\R^3$ is a Legendrian submanifold $\Lambda_K$ in the unit cotangent bundle $U^* \R^3$ of $\R^3$ with contact structure equal to the kernel of the Liouville form. Knot contact homology, a topological link invariant of $K$, is defined as the Legendrian homology of $\Lambda_K$, the homology of a differential graded algebra generated by Reeb chords whose differential counts holomorphic disks in the symplectization $\R \times U^*\R^3$ with Lagrangian boundary condition $\R \times \Lambda_K$. We perform an explicit and complete computation of the Legendrian homology of $\Lambda_K$ for arbitrary links $K$ in terms of a braid presentation of $K$, confirming a conjecture that this invariant agrees with a previously-defined combinatorial version of knot contact homology. The computation uses a double degeneration: the braid degenerates toward a multiple cover of the unknot which in turn degenerates to a point. Under the first degeneration, holomorphic disks converge to gradient flow trees with quantum corrections. The combined degenerations give rise to a new generalization of flow trees called multiscale flow trees. The theory of multiscale flow trees is the key tool in our computation and is already proving to be useful for other computations as well.

math.SG↗