arXiv subjects

Joyjit Kundu

Publications and source records attributed to Joyjit Kundu.

At least 19 recordsLinked to original sources

arXiv2026

KernelArc: A Multi-Agent Framework for GPU Kernel Optimization

We present KernelArc, a multi-agent framework for autonomous GPU kernel optimization across heterogeneous workloads. Strategy-specialized agents run in parallel and coordinate through conclusions-only shared memory, a deterministic benchmark guard, and read-only cross-agent state with plateau-triggered drafting. We evaluate KernelArc on NVIDIA H100 and B200 GPUs using category-representative SOL-ExecBench workloads. The resulting implementations span custom BF16 GEMM, static cuBLASLt Expert-API configuration tables, fused mixture-of-experts backward, shape-gated decoder-layer fusion, native NVFP4 grouped-query attention, and paged prefill attention. In the public SOL-ExecBench leaderboard snapshot recorded on August~20, 2026, KernelArc ranked first on every representative L1, L2, Quantization, and FlashInfer task evaluated. The trajectories support the paper's central motivation: shared multi-agent search can broaden exploration and reach stronger incumbents within a fixed candidate budget, while the value of individual coordination features depends on the kernel and optimization stage.

cs.AI↗

arXiv2026

Technology solutions targeting the performance of gen-AI inference in resource constrained platforms

The rise of generative AI workloads, particularly language model inference, is intensifying on/off-chip memory pressure. Multimodal inputs such as video streams or images and downstream applications like Question Answering (QA) and analysis over large documents incur long context lengths, requiring caching of massive Key and Value states of the previous tokens. Even a low degree of concurrent inference serving on resource-constrained devices, like mobiles, can further add to memory capacity pressure and runtime memory management complexity. In this paper, we evaluate the performance implications of two emerging technology solutions to alleviate the memory pressure in terms of both capacity and bandwidth using a hierarchical roofline-based analytical performance model. For large models (e.g., 13B parameters) and context lengths, we investigate the performance implications of High Bandwidth Storage (HBS) and outline bandwidth/latency requirements to achieve an acceptable throughput for interactivity. For small models (e.g., 1B parameters), we evaluate the merit of a bonded global buffer memory chiplet and propose how to best utilize it.

cs.AR↗

arXiv2026

Characterizing Machine Learning Force Fields as Emerging Molecular Dynamics Workloads on Graphics Processing Units

Molecular dynamics (MD) simulates the time evolution of atomic systems governed by interatomic forces, and the fidelity of these simulations depends critically on the underlying force model. Classical force fields (CFFs) rely on fixed functional forms fitted to experimental or theoretical data, offering computational efficiency and broad applicability but limited accuracy in chemically diverse or reactive environments. In contrast, machine learning force fields (MLFFs) deliver near quantum chemical accuracy at molecular-mechanics cost by learning interatomic interactions directly from high level electronic structure data. While MLFFs offer improved accuracy at a fraction of the cost of quantum methods, they introduce significant computational overhead, particularly in descriptor evaluation and neural network inference. These operations pose challenges for parallel hardware due to irregular memory access, minimum data reuse and inefficient kernel execution. This work investigates the hardware performance of such models using poly alanine chains, a novel benchmark molecule system(s) with controllable input size, which used as performance evaluation test cases highlighting the computational bottlenecks of the graphical processor units when scaling out MLFF simulations. The analysis identifies key bottlenecks in descriptor and force computation, memory handling, highlighting the opportunities for improvements in the emerging area of MLFF based MD in drug discovery, that has received limited attention from a computer architecture perspective.

cs.PF↗

arXiv2025

System-performance and cost modeling of Large Language Model training and inference

Large language models (LLMs), based on transformer architectures, have revolutionized numerous domains within artificial intelligence, science, and engineering due to their exceptional scalability and adaptability. However, the exponential growth in LLM size and complexity has outpaced advancements in compute capacity, memory bandwidth, network performance, and cost efficiency, posing significant challenges to their scalability on distributed systems. To address these limitations, alternative model architectures, optimization strategies, communication-aware network topologies, and novel system design approaches have been proposed in literature. This paper introduces a performance-cost modeling methodology for LLM training and inference that integrates state-of-the-art compute techniques with memory optimizations, and latest communication techniques. Building on an analytical performance model, our approach incorporates recent innovations such as the flash attention technique and mixture of experts models to address the memory bandwidth and compute bottlenecks. It also considers the impact of different network topologies and topology-specific communication algorithms with 5D parallellism. The framework also integrates a chiplet cost model. The proposed modeling methodology provides valuable insights to guide future compute system design and facilitates hardware-software co-development, in particular due to its ability to analyze performance-cost trade-offs for various system architectural configurations.

cs.AR↗

arXiv2024

A System Level Performance Evaluation for Superconducting Digital Systems

Superconducting Digital (SCD) technology offers significant potential for enhancing the performance of next generation large scale compute workloads. By leveraging advanced lithography and a 300 mm platform, SCD devices can reduce energy consumption and boost computational power. This paper presents a cross-layer modeling approach to evaluate the system-level performance benefits of SCD architectures for Large Language Model (LLM) training and inference. Our findings, based on experimental data and Pulse Conserving Logic (PCL) design principles, demonstrate substantial performance gain in both training and inference. We are, thus, able to convincingly show that the SCD technology can address memory and interconnect limitations of present day solutions for next-generation compute systems.

cs.AR↗

arXiv2024

Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inference

Aligning future system design with the ever-increasing compute needs of large language models (LLMs) is undoubtedly an important problem in today's world. Here, we propose a general performance modeling methodology and workload analysis of distributed LLM training and inference through an analytical framework that accurately considers compute, memory sub-system, network, and various parallelization strategies (model parallel, data parallel, pipeline parallel, and sequence parallel). We validate our performance predictions with published data from literature and relevant industry vendors (e.g., NVIDIA). For distributed training, we investigate the memory footprint of LLMs for different activation re-computation methods, dissect the key factors behind the massive performance gain from A100 to B200 ($\sim$ 35x speed-up closely following NVIDIA's scaling trend), and further run a design space exploration at different technology nodes (12 nm to 1 nm) to study the impact of logic, memory, and network scaling on the performance. For inference, we analyze the compute versus memory boundedness of different operations at a matrix-multiply level for different GPU systems and further explore the impact of DRAM memory technology scaling on inference latency. Utilizing our modeling framework, we reveal the evolution of performance bottlenecks for both LLM training and inference with technology scaling, thus, providing insights to design future systems for LLM training and inference.

cs.AR↗

arXiv2021

The dimensional evolution of structure and dynamics in hard sphere liquids

The formulation of the mean-field, infinite-dimensional solution of hard sphere glasses is a significant milestone for theoretical physics. How relevant this description might be for understanding low-dimensional glass-forming liquids, however, remains unclear. These liquids indeed exhibit a complex interplay between structure and dynamics, and the importance of this interplay might only slowly diminish as dimension $d$ increases. A careful numerical assessment of the matter has long been hindered by the exponential increase of computational costs with $d$. By revisiting a once common simulation technique involving the use of periodic boundary conditions modeled on $D_d$ lattices, we here partly sidestep this difficulty, thus allowing the study of hard sphere liquids up to $d=13$. Parallel efforts by Mangeat and Zamponi [Phys. Rev. E 93, 012609 (2016)] have expanded the mean-field description of glasses to finite $d$ by leveraging standard liquid-state theory, and thus help bridge the gap from the other direction. The relatively smooth evolution of both structure and dynamics across the $d$ gap allows us to relate the two approaches, and to identify some of the missing features that a finite-$d$ theory of glasses might hope to include to achieve near quantitative agreement.

cond-mat.stat-mech↗

arXiv2021

Machine Learned Phase Transitions in a System of Anisotropic Particles on a Square Lattice

The area of Machine learning (ML) has seen exceptional growth in recent years. Successful implementation of ML methods in various branches of physics has led to new insights. These methods have been shown to classify phases in condensed matter systems. Here we study the classification problem of phases in a system of hard rigid rods on a square lattice around a continuous and a discontinuous phase transition. On comparing a number of methods we find that convolutional neural network (CNN) classifies the phases with the highest accuracy when only snapshots are given as inputs. We study how the system size affects the model performance. We further compare the performance of CNN in classifying the phases around a continuous and a discontinuous phase transition. Further, we show that one can even beat the accuracy of CNN with simpler models by using physics-guided features. Lastly, we show that the critical point in this system can be learned without any prior estimate by using only the information of the ordered phase (as training set). Our study reveals the ML techniques that have been successful in studying spin systems can be easily adapted to more complex systems.

cond-mat.stat-mech↗

arXiv2020

Hysteresis curves reveal the microscopic origin of cooperative CO$_2$ adsorption in diamine-appended metal-organic frameworks

Diamine-appended metal{organic frameworks (MOFs) of the form Mg2(dobpdc)(diamine)2 adsorb CO2 in a cooperative fashion, exhibiting an abrupt change in CO2 occupancy with pressure or temperature. This change is accompanied by hysteresis. While hysteresis is suggestive of a firstorder phase transition, we show that hysteretic temperature-occupancy curves associated with this material are qualitatively unlike the curves seen in the presence of a phase transition; they are instead consistent with CO2 chain polymerization, within one-dimensional channels in the MOF, in the absence of a phase transition. Our simulations of a microscopic model reproduce this dynamics, and point the way toward rational control, in and out of equilibrium, of cooperative adsorption in this industrially important class of materials.

cond-mat.mtrl-sci↗

arXiv2019

Finite-dimensional vestige of spinodal criticality above the dynamical glass transition

Finite-dimensional signatures of spinodal criticality are notoriously difficult to come by. The dynamical transition of glass-forming liquids, first described by mode-coupling theory, is a spinodal instability preempted by thermally activated processes that also limit how close the instability can be approached. We combine numerical tools to directly observe vestiges of the spinodal criticality in finite-dimensional glass formers. We use the swap Monte Carlo algorithm to efficiently thermalise configurations beyond the mode-coupling crossover, and analyze their dynamics using a scheme to screen out activated processes, in spatial dimensions ranging from $d=3$ to $d=9$. We observe a strong softening of the mean-field square-root singularity in $d=3$ that is progressively restored as $d$ increases above $d=8$, in surprisingly good agreement with perturbation theory.

cond-mat.stat-mech↗

arXiv2019

Postponing the dynamical transition density using competing interactions

Systems of dense spheres interacting through very short-ranged attraction are known from theory, simulations and colloidal experiments to exhibit dynamical reentrance. The liquid state can thus be fluidized to higher densities than otherwise possible with interactions that are purely repulsive or long-ranged attractive. A recent mean-field, infinite-dimensional calculation predicts that the dynamical arrest of the fluid can be further delayed by adding a longer-ranged repulsive contribution to the short-ranged attraction. We examine this proposal by performing extensive numerical simulations in a three-dimensional system. We first find the short-ranged attraction parameters necessary to achieve the densest liquid state, and then explore the parameters space for an additional longer-ranged repulsion that could enhance the effect. In the family of systems studied, no significant (within numerical accuracy) delay of the dynamical arrest is observed beyond what is already achieved by the short-ranged attraction. Possible explanations are discussed.

cond-mat.stat-mech↗

arXiv2018

Bypassing sluggishness: SWAP algorithm and glassiness in high dimensions

The recent implementation of a swap Monte Carlo algorithm (SWAP) for polydisperse mixtures fully bypasses computational sluggishness and closes the gap between experimental and simulation timescales in physical dimensions $d=2$ and $3$. Here, we consider suitably optimized systems in $d=2, 3,\dots, 8$, to obtain insights into the performance and underlying physics of SWAP. We show that the speedup obtained decays rapidly with increasing the dimension. SWAP nonetheless delays systematically the onset of the activated dynamics by an amount that remains finite in the limit $d \to \infty$. This shows that the glassy dynamics in high dimensions $d>3$ is now computationally accessible using SWAP, thus opening the door for the systematic consideration of finite-dimensional deviations from the mean-field description.

cond-mat.stat-mech↗

arXiv2017

Cooperative gas adsorption without a phase transition in metal-organic frameworks

Cooperative adsorption of gases by porous frameworks permits more efficient uptake and removal than does the more usual non-cooperative (Langmuir-type) adsorption. Cooperativity, signaled by a step-like isotherm, is usually attributed to a phase transition of the framework. However, the class of metal-organic frameworks mmen-M$_2$(dobpdc) exhibit cooperative adsorption of CO2 but show no evidence of a phase transition. Here we show how cooperativity emerges in these frameworks in the absence of a phase transition. We use a combination of quantum and statistical mechanics to show that cooperativity results from a sharp but finite increase, with pressure, of the mean length of chains of CO2 molecules that polymerize within the framework. Our study provides microscopic understanding of the emergent features of cooperative binding, including the position, slope and height of the isotherm step, and indicates how to optimize gas storage and separation in these materials.

cond-mat.mtrl-sci↗

arXiv2016

Selective gas capture via kinetic trapping

Conventional approaches to the capture of CO_2 by metal-organic frameworks focus on equilibrium conditions, and frameworks that contain little CO_2 in equilibrium are often rejected as carbon-capture materials. Here we use a statistical mechanical model, parameterized by quantum mechanical data, to suggest that metal-organic frameworks can be used to separate CO_2 from a typical flue gas mixture when used under {\em nonequilibrium} conditions. The origin of this selectivity is an emergent gas-separation mechanism that results from the acquisition by different gas types of different mobilities within a crowded framework. The resulting distribution of gas types within the framework is in general spatially and dynamically heterogeneous. Our results suggest that relaxing the requirement of equilibrium can substantially increase the parameter space of conditions and materials for which selective gas capture can be effected.

physics.chem-ph↗

arXiv2015

The Ramachandran number: an order parameter for protein geometry

Three-dimensional protein structures usually contain regions of local order, called secondary structure, such as $α$-helices and $β$-sheets. Secondary structure is characterized by the local rotational state of the protein backbone, quantified by two dihedral angles called $ϕ$ and $ψ$. Particular types of secondary structure can generally be described by a single (diffuse) location on a two-dimensional plot drawn in the space of the angles $ϕ$ and $ψ$, called a Ramachandran plot. By contrast, a recently-discovered nanomaterial made from peptoids, structural isomers of peptides, displays a secondary-structure motif corresponding to two regions on the Ramachandran plot [Mannige et al., Nature 526, 415 (2015)]. In order to describe such `higher-order' secondary structure in a compact way we introduce here a means of describing regions on the Ramachandran plot in terms of a single Ramachandran number, ${\mathcal{R}}$, which is a structurally meaningful combination of $ϕ$ and $ψ$. We show that the potential applications of ${\mathcal{R}}$ are numerous: it can be used to describe the geometric content of protein structures, and can be used to draw diagrams that reveal, at a glance, the frequency of occurrence of regular secondary structures and disordered regions in large protein datasets. We propose that ${\mathcal{R}}$ might be used as an order parameter for protein geometry for a wide range of applications.

q-bio.BM↗

arXiv2015

Phase diagram of a bidispersed hard rod lattice gas in two dimensions

We obtain, using extensive Monte Carlo simulations, virial expansion and a high-density perturbation expansion about the fully packed monodispersed phase, the phase diagram of a system of bidispersed hard rods on a square lattice. We show numerically that when the length of the longer rods is $7$, two continuous transitions may exist as the density of the longer rods in increased, keeping the density of shorter rods fixed: first from a low-density isotropic phase to a nematic phase, and second from the nematic to a high-density isotropic phase. The difference between the critical densities of the two transitions decreases to zero at a critical density of the shorter rods such that the fully packed phase is disordered for any composition. When both the rod lengths are larger than $6$, we observe the existence of two transitions along the fully packed line as the composition is varied. Low-density virial expansion, truncated at second virial coefficient, reproduces features of the first transition. By developing a high-density perturbation expansion, we show that when one of the rods is long enough, there will be at least two isotropic-nematic transitions along the fully packed line as the composition is varied.

cond-mat.stat-mech↗

arXiv2015

Phase transitions in systems of hard rectangles with non-integer aspect ratio

We investigate, using Monte Carlo simulations, the phase diagram of a system of hard rectangles of size $m\times mk$ on a square lattice when the aspect ratio $k$ is a non-integer. The existence of a disordered isotropic phase, a nematic with only orientational order, a columnar phase with orientational and partial translational order, and a high density phase with no orientational order is shown. The high density phase is a solid-like sublattice phase only if the length and width of the rectangles are not mutually prime, else, it is an isotropic phase. The minimum value of $k$ beyond which the nematic and columnar phases exist are determined for $m=2$ and $3$. The nature of the transitions between different phases is determined, and the critical exponents are numerically obtained for the continuous transitions.

cond-mat.stat-mech↗

arXiv2014

High-Activity Expansion for the Columnar Phase of the Hard Rectangle Gas

We study a system of monodispersed hard rectangles of size $m \times d$, where $d\geq m$ on a two dimensional square lattice. For large enough aspect ratio, the system is known to undergo three entropy driven phase transitions with increasing activity $z$: first from disordered to nematic, second from nematic to columnar and third from columnar to sublattice phases. We study the nematic-columnar transition by developing a high-activity expansion in integer powers of $z^{-1/d}$ for the columnar phase in a model where the rectangles are allowed to orient only in one direction. By deriving the exact expression for the first $d+2$ terms in the expansion, we obtain lower bounds for the critical density and activity. For $m$, $k\gg 1$, these bounds decrease with increasing $k$ and decreasing $m$.

cond-mat.stat-mech↗