Searcharxiv⌕ Search

arXiv subjects

Aurojit Panda

Publications and source records attributed to Aurojit Panda.

At least 19 recordsLinked to original sources
arXiv2026

Enforcing Application-Layer Policies in eBPF

Service meshes have recently emerged as the de-facto standard for deploying microservices. Conceptually, they provide a uniform abstraction for inter-process communication (IPC) between services by implementing common networking mechanisms---such as encryption, routing, and load balancing---and by allowing these mechanisms to be configured and composed through high-level policies. Supporting these policies, however, comes with a significant performance cost, since service meshes interpose proxies (``sidecars'') on the data path between every service. This paper presents Beeline, a fast path for service meshes which can enforce the vast majority of application-layer policies seen in the wild directly in kernel space. Given high-level policies, Beeline automatically synthesizes an eBPF-based data plane which enforces them in the kernel. Beeline accelerates existing microservices without any code modification, and transparently falls back to existing service proxies (the slow path) for the few unsupported policies. We fully implemented Beeline, with support for both TLS and HTTP/2. Compared to state-of-the-art service meshes, Beeline reduces the median request latency of realistic applications by up to $6\times$ while sustaining $3\times$ more throughput.

cs.NI↗
arXiv2026

Probabilistic Fair Ordering of Events

A growing class of applications depends on fair ordering, where events that occur earlier should be processed before later ones. Providing such guarantees is difficult in practice because clock synchronization is inherently imperfect: events generated at different clients within a short time window may carry timestamps that cannot be reliably ordered. Rather than attempting to eliminate synchronization error, we embrace it and establish a probabilistically fair sequencing process. Tommy is a sequencer that uses a statistical model of per-clock synchronization error to compare noisy timestamps probabilistically. Although this enables ordering of two events, the probabilistic comparator is intransitive, making global ordering non-trivial. We address this challenge by mapping the sequencing problem to a classical ranking problem from social choice theory, which offers principled mechanisms for reasoning with intransitive comparisons. Using this formulation, Tommy produces a partial order of events, achieving significantly better fairness than a Spanner TrueTime-based baseline approach.

cs.NI↗
arXiv2026

Practical One-Round-Trip BFT Replication

As Byzantine Fault Tolerant (BFT) protocols are increasingly adopted for user-facing applications such as payments and smart contracts, it is crucial that they provide low latency. To reduce latency, some BFT consensus protocols use a leaderless, speculative, fast path where clients broadcast requests directly to replicas, enabling end-to-end commit latency of two message delays ($2\Delta$). However, such a fast path is extremely fragile: concurrent requests can cause replicas to diverge when they receive requests in different orders, triggering costly recovery procedures. This paper presents Aspen, a leaderless speculative BFT protocol that handles concurrent requests while achieving near-optimal latency of $2\Delta + \epsilon$. The $\epsilon$ term is a short waiting delay introduced by Aspen's best effort ordering layer, which uses loosely synchronized clocks and network delay estimates to provide a tentative order. To make its fast path even more robust to intermittent divergence, Aspen adds extra replicas ($n = 3f + 2p + 1$) as well as novel recovery mechanisms that allow the system to tolerate divergence while preserving safety and performance. In experiments with geo-distributed replicas, Aspen reduces the median latency of requests by $1.1\times$--$3.8\times$ compared to state-of-the-art BFT protocols, while sustaining up to $0.75\times$ the peak throughput of throughput-optimized designs.

cs.DC↗
arXiv2025

Scaling Point-based Differentiable Rendering for Large-scale Reconstruction

Point-based Differentiable Rendering (PBDR) enables high-fidelity 3D scene reconstruction, but scaling PBDR to high-resolution and large scenes requires efficient distributed training systems. Existing systems are tightly coupled to a specific PBDR method. And they suffer from severe communication overhead due to poor data locality. In this paper, we present Gaian, a general distributed training system for PBDR. Gaian provides a unified API expressive enough to support existing PBDR methods, while exposing rich data-access information, which Gaian leverages to optimize locality and reduce communication. We evaluated Gaian by implementing 4 PBDR algorithms. Our implementations achieve high performance and resource efficiency: across six datasets and up to 128 GPUs, it reduces communication by up to 91% and improves training throughput by 1.50x-3.71x.

cs.DC↗
arXiv2025

CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) is an increasingly popular novel view synthesis approach due to its fast rendering time, and high-quality output. However, scaling 3DGS to large (or intricate) scenes is challenging due to its large memory requirement, which exceed most GPU's memory capacity. In this paper, we describe CLM, a system that allows 3DGS to render large scenes using a single consumer-grade GPU, e.g., RTX4090. It does so by offloading Gaussians to CPU memory, and loading them into GPU memory only when necessary. To reduce performance and communication overheads, CLM uses a novel offloading strategy that exploits observations about 3DGS's memory access pattern for pipelining, and thus overlap GPU-to-CPU communication, GPU computation and CPU computation. Furthermore, we also exploit observation about the access pattern to reduce communication volume. Our evaluation shows that the resulting implementation can render a large scene that requires 100 million Gaussians on a single RTX4090 and achieve state-of-the-art reconstruction quality.

cs.CV↗
arXiv2025

Beyond Lamport, Towards Probabilistic Fair Ordering

A growing class of applications demands \emph{fair ordering} of events, which ensures that events generated earlier are processed before later events. However, achieving such sequencing is challenging due to the inherent errors in clock synchronization: two events at two clients generated close together may have timestamps that cannot be compared confidently. We advocate for an approach that embraces, rather than eliminates, clock synchronization errors. Instead of attempting to remove the error from a timestamp, \systemname{}, our proposed system, leverages a statistical model to compare two noisy timestamps probabilistically by learning per-clock synchronization error distributions. Our preliminary statistical model computes the probability that one event precedes another by only relying on local clocks of clients. This serves as a foundation for a new relation: \emph{likely-happened-before} denoted by $\xrightarrow{p}$ where $p$ represents the probability that an event happened before another. The $\xrightarrow{p}$ relation provides a basis for ordering multiple events which are otherwise considered \emph{concurrent} by Lamport's \emph{happened-before} ($\rightarrow$) relation. We highlight various related challenges including the intransitivity of the $\xrightarrow{p}$ relation as opposed to the transitive $\rightarrow$ relation. We outline several research directions: online fair sequencing, stochastically fair total ordering, and handling byzantine clients.

cs.NI↗
arXiv2025

Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference

Distributed machine learning training and inference is common today because today's large models require more memory and compute than can be provided by a single GPU. Distributed models are generally produced by programmers who take a sequential model specification and apply several distribution strategies to distribute state and computation across GPUs. Unfortunately, bugs can be introduced in the process, and a distributed model implementation's outputs might differ from the sequential model's outputs. In this paper, we describe an approach to statically identify such bugs by checking model refinement, that is, can the sequential model's outputs be reconstructed from the distributed model's outputs? Our approach, implemented in GraphGuard, uses iterative rewriting to prove model refinement. Our approach can scale to today's large models and deployments: we evaluate it using GPT and Llama-3. Further, it provides actionable output that aids in bug localization.

cs.DC↗
arXiv2025

Understanding Stragglers in Large Model Training Using What-if Analysis

Large language model (LLM) training is one of the most demanding distributed computations today, often requiring thousands of GPUs with frequent synchronization across machines. Such a workload pattern makes it susceptible to stragglers, where the training can be stalled by few slow workers. At ByteDance we find stragglers are not trivially always caused by hardware failures, but can arise from multiple complex factors. This work aims to present a comprehensive study on the straggler issues in LLM training, using a five-month trace collected from our ByteDance LLM training cluster. The core methodology is what-if analysis that simulates the scenario without any stragglers and contrasts with the actual case. We use this method to study the following questions: (1) how often do stragglers affect training jobs, and what effect do they have on job performance; (2) do stragglers exhibit temporal or spatial patterns; and (3) what are the potential root causes for stragglers?

cs.DC↗
arXiv2025

The Dawn of Disaggregation and the Coherence Conundrum: A Call for Federated Coherence

Disaggregated memory is an upcoming data center technology that will allow nodes (servers) to share data efficiently. Sharing data creates a debate on the level of cache coherence the system should provide. While current proposals aim to provide coherence for all or parts of the disaggregated memory, we argue that this approach is problematic, because of scalability limitations and hardware complexity. Instead, we propose and formally define federated coherence, a model that provides coherence only within nodes, not across nodes. Federated coherence can use current intra-node coherence provided by processors without requiring expensive mechanisms for inter-node coherence. Developers can use federated coherence with a few simple programming paradigms and a synchronization library. We sketch some potential applications.

cs.DC↗
arXiv2025

Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification

Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from overthinking, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24\% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency.

cs.AI↗
arXiv2024

Extracting Database Access-Control Policies From Web Applications

To safeguard sensitive user data, web developers typically rely on implicit access-control policies, which they implement using access checks and query filters. This ad hoc approach is error-prone as these scattered checks and filters are easy to misplace or misspecify, and the lack of an explicit policy precludes external access-control enforcement. More critically, it is difficult for humans to discern what policy is embedded in application code (i.e., what data the application may access) -- an issue that worsens as development teams evolve. This paper tackles policy extraction: the task of extracting the access-control policy embedded in an application by summarizing its data queries. An extracted policy, once vetted for errors, can stand alone as a specification for the application's data access, and can be enforced to ensure compliance as code changes over time. We introduce Ote, a policy extractor for Ruby on Rails web applications. Ote uses concolic execution to explore execution paths through the application, generating traces of SQL queries and conditions that trigger them. It then merges and simplifies these traces into a final policy that aligns with the observed behaviors. We applied Ote to three real-world applications and compared extracted policies to handwritten ones, revealing several errors in the latter.

cs.SE↗
arXiv2024

On Scaling Up 3D Gaussian Splatting Training

3D Gaussian Splatting (3DGS) is increasingly popular for 3D reconstruction due to its superior visual quality and rendering speed. However, 3DGS training currently occurs on a single GPU, limiting its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. We introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. As each Gaussian affects a small, dynamic subset of rendered pixels, Grendel employs sparse all-to-all communication to transfer the necessary Gaussians to pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image at a time, Grendel supports batched training with multiple views. We explore various optimization hyperparameter scaling strategies and find that a simple sqrt(batch size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the Rubble dataset, we achieve a test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPUs, compared to a PSNR of 26.28 using 11.2 million Gaussians on a single GPU. Grendel is an open-source project available at: https://github.com/nyu-systems/Grendel-GS

cs.CV↗
arXiv2023

Application-Defined Receive Side Dispatching on the NIC

Application layer (L7) processing is increasingly implemented in proxies (e.g., Envoy) to simplify administration and management. However, prior work has observed that this reduces application performance and increases resource requirements. The reason is that moving logic out of the application required duplicating some computation and additional inter-process communication. This paper describes QingNiao, a system that moves L7 dispatch (a function implemented by all L7 proxies and affects all messages received by an application) to a NIC that is on the application's communication path. Unfortunately, the data formats and protocols used by modern applications pose a challenge when moving L7 dispatch to NICs. Consequently, when designing QingNiao we had to rethink not just the NIC hardware, but also how applications encode data sent over the network. We prototyped QingNiao using a 100GbE FPGA NIC, and show that for real-world applications QingNiao can achieve 6.6x to 7.15x higher throughput compared to software proxies.

cs.NI↗
arXiv2023

Bringing Reconfigurability to the Network Stack

Reconfiguring the network stack allows applications to specialize the implementations of communication libraries depending on where they run, the requests they serve, and the performance they need to provide. Specializing applications in this way is challenging because developers need to choose the libraries they use when writing a program and cannot easily change them at runtime. This paper introduces Bertha, which allows these choices to be changed at runtime without limiting developer flexibility in the choice of network and communication functions. Bertha allows applications to safely use optimized communication primitives (including ones with deployment limitations) without limiting deployability. Our evaluation shows cases where this results in 16x higher throughput and 63% lower latency than current portable approaches while imposing minimal overheads when compared to a hand-optimized versions that use deployment-specific communication primitives.

cs.NI↗
arXiv2022

NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers

Deep-learning (DL) compilers such as TVM and TensorRT are increasingly being used to optimize deep neural network (DNN) models to meet performance, resource utilization and other requirements. Bugs in these compilers can result in models whose semantics differ from the original ones, producing incorrect results that corrupt the correctness of downstream applications. However, finding bugs in these compilers is challenging due to their complexity. In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers. Our core approach consists of (i) generating diverse yet valid DNN test models that can exercise a large part of the compiler's transformation logic using light-weight operator specifications; (ii) performing gradient-based search to find model inputs that avoid any floating-point exceptional values during model execution, reducing the chance of missed bugs or false alarms; and (iii) using differential testing to identify bugs. We implemented this approach in NNSmith which has found 72 new bugs for TVM, TensorRT, ONNXRuntime, and PyTorch to date. Of these 58 have been confirmed and 51 have been fixed by their respective project maintainers.

cs.LG↗
arXiv2022

This is not the End: Rethinking Serverless Function Termination

Elastic scaling is one of the central benefits provided by serverless platforms, and requires that they scale resource up and down in response to changing workloads. Serverless platforms scale-down resources by terminating previously launched instances (which are containers or processes). The serverless programming model ensures that terminating instances is safe assuming all application code running on the instance has either completed or timed out. Safety thus depends on the serverless platform's correctly determining that application processing is complete. In this paper, we start with the observation that current serverless platforms do not account for pending asynchronous I/O operations when determining whether application processing is complete. These platforms are thus unsafe when executing programs that use asynchronous I/O, and incorrectly deciding that application processing has terminated can result in data inconsistency when these platforms are used. We show that the reason for this problem is that current serverless semantics couple termination and response generation in serverless applications. We address this problem by proposing an extension to current semantics that decouples response generation and termination, and demonstrate the efficacy and benefits of our proposal by extending OpenWhisk, an open source serverless platform.

cs.DC↗
arXiv2022

3PO: Programmed Far-Memory Prefetching for Oblivious Applications

Using memory located on remote machines, or far memory, as a swap space is a promising approach to meet the increasing memory demands of modern datacenter applications. Operating systems have long relied on prefetchers to mask the increased latency of fetching pages from swap space to main memory. Unfortunately, with traditional prefetching heuristics, performance still degrades when applications use far memory. In this paper we propose a new prefetching technique for far-memory applications. We focus our efforts on memory-intensive, oblivious applications whose memory access patterns are independent of their inputs, such as matrix multiplication. For this class of applications we observe that we can perfectly prefetch pages without relying on heuristics. However, prefetching perfectly without requiring significant application modifications is challenging. In this paper we describe the design and implementation of 3PO, a system that provides pre-planned prefetching for general oblivious applications. We demonstrate that 3PO can accelerate applications, e.g., running them 30-150% faster than with Linux's prefetcher with 20% local memory. We also use 3PO to understand the fundamental software overheads of prefetching in a paging-based system, and the minimum performance penalty that they impose when we run applications under constrained local memory.

cs.OS↗