arXiv subjects

Tuhin Khare

Publications and source records attributed to Tuhin Khare.

4 recordsLinked to original sources

arXiv2026

QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture

The field of computer architecture, which bridges high-level software abstractions and low-level hardware implementations, remains absent from current large language model (LLM) evaluations. To this end, we present QuArch (pronounced 'quark'), the first benchmark designed to facilitate the development and evaluation of LLM knowledge and reasoning capabilities specifically in computer architecture. QuArch v1.0 provides a comprehensive collection of 2,671 expert-validated question-answer (QA) pairs covering various aspects of computer architecture, including processor design, memory systems, and interconnection networks. Our evaluation reveals that while frontier models possess domain-specific knowledge, they struggle with skills that require higher-order thinking in computer architecture. Frontier model accuracies vary widely (from 34% to 73%) on these advanced questions, highlighting persistent gaps in architectural reasoning across analysis, design, and implementation QAs. Furthermore, via fine-tuning we find that QuArch can translate to improved performance on a realistic memory hierarchy design task, resulting in up to 1.99x more area-efficient solutions and up to 40% more viable solutions overall. By holistically assessing fundamental skills, QuArch provides a foundation for building and measuring LLM capabilities that can accelerate innovation in computing systems. The QuArch benchmark and leaderboard are publicly available at: https://quarch.ai/.

cs.AR↗

arXiv2026

How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving

Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-prefill aggregation, to prefill-decode (P/D) disaggregation, and most recently to operator-level Attention-FFN Disaggregation (AFD). This trend is especially important for mixture-of-experts (MoE) models, where memory-bound attention, compute-intensive expert FFNs, and MoE dispatch/combine communication create distinct resource demands. AFD further exposes this heterogeneity by placing attention and MoE-FFN execution on separate GPU groups. Each level of disaggregation deepens the scheduling design space across workload characteristics, resource allocation, and interconnect topology, raising the central question: when does each level actually pay off? We systematically characterize this trade-off for MoE inference across realistic workloads spanning input/output sequence lengths, prefix-KV reuse, and per-user latency constraints. Using chunked-prefill and P/D disaggregation as baselines, we study the benefits and limits of AFD at scale through a framework that fuses on-device kernel measurements with high-fidelity network simulation. Under strict TTFT/TPOT SLOs, AFD sustains around 4k tokens/s of system throughput on DeepSeek-V3.2 across chat, coding, and agentic-coding workloads, where non-AFD deployments are infeasible. We distill concrete takeaways for jointly optimizing throughput and interactivity, including how to partition attention and FFN across GPUs as a function of workload and model architecture, providing design principles for current rack- and cluster-scale deployments as well as future disaggregated AI infrastructure.

cs.LG↗

arXiv2026

Characterizing FaaS Workflows on Public Clouds: The Good, the Bad and the Ugly

Function-as-a-service (FaaS) is a popular serverless computing paradigm for developing event-driven functions that elastically scale on public clouds. FaaS workflows, such as AWS Step Functions and Azure Durable Functions, are composed from FaaS functions, like AWS Lambda and Azure Functions, to build practical applications. But, the complex interactions between functions in the workflow and the limited visibility into the internals of proprietary FaaS platforms are major impediments to gaining a deeper understanding of FaaS workflow platforms. While several works characterize FaaS platforms to derive such insights, there is a lack of a principled and rigorous study for FaaS workflow platforms, which have unique scaling, performance and costing behavior influenced by the platform design, dataflow and workloads. In this article, we perform extensive evaluations of three popular FaaS workflow platforms from AWS and Azure, running 25 micro-benchmark and application workflows over 132k invocations. Our detailed analysis confirms some conventional wisdom but also uncovers unique insights on the function execution, workflow orchestration, inter-function interactions, cold-start scaling and monetary costs. Our observations help developers better configure and program these platforms, set performance and scalability expectations, and identify research gaps on enhancing the platforms.

cs.DC↗

arXiv2023

Parallelizing Quantum-Classical Workloads: Profiling the Impact of Splitting Techniques

Quantum computers are the next evolution of computing hardware. Quantum devices are being exposed through the same familiar cloud platforms used for classical computers, and enabling seamless execution of hybrid applications that combine quantum and classical components. Quantum devices vary in features, e.g., number of qubits, quantum volume, CLOPS, noise profile, queuing delays and resource cost. So, it may be useful to split hybrid workloads with either large quantum circuits or large number of quantum circuits, into smaller units. In this paper, we profile two workload splitting techniques on IBM's Quantum Cloud: (1) Circuit parallelization, to split one large circuit into multiple smaller ones, and (2) Data parallelization to split a large number of circuits run on one hardware to smaller batches of circuits run on different hardware. These can improve the utilization of heterogenous quantum hardware, but involve trade-offs. We evaluate these techniques on two key algorithmic classes: Variational Quantum Eigensolver (VQE) and Quantum Support Vector Machine (QSVM), and measure the impact on circuit execution times, pre- and post-processing overhead, and quality of the result relative to a baseline without parallelization. Results are obtained on real hardware and complemented by simulations. We see that (1) VQE with circuit cutting is ~39\% better in ground state estimation than the uncut version, and (2) QSVM that combines data parallelization with reduced feature set yields upto 3x improvement in quantum workload execution time and reduces quantum resource use by 3x, while providing comparable accuracy. Error mitigation can improve the accuracy by ~7\% and resource foot-print by ~4\% compared to the best case among the considered scenarios.

quant-ph↗