arXiv subjects

Amit Singh

Publications and source records attributed to Amit Singh.

At least 19 recordsLinked to original sources

arXiv2026

Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems

Context adaptation automates prompt engineering in LLM-based systems by iteratively revising tunable prompts from task feedback, without modifying model weights. Extending this paradigm to multi-LLM agentic systems is crucial: existing methods suffer from inaccurate credit assignment and lack convergence guarantees. We propose \textbf{G}raph-based \textbf{T}arget \textbf{B}ack-\textbf{P}ropagation (GTBP), a context adaptation framework for agentic workflows modeled as directed acyclic graphs. GTBP propagates local target outputs backward through the workflow graph and uses target--output discrepancies to guide a stage-wise prompt update mechanism. Theoretically, we show that GTBP's stage-wise prompt updates become stable over iterations, and that a sufficiently capable LLM optimizer can decrease the overall objective. Empirically, GTBP consistently outperforms strong baselines across three benchmarks while maintaining comparable computational cost.

cs.LG↗

arXiv2026

Fine-Tuning and Serving Gemma 4 31B on Google Cloud TPU: A Technical Comparison with GPU Baselines

We present the first end-to-end demonstration of fine-tuning and serving Google's Gemma 4 31B model on TPU hardware, providing an empirical comparison of TPU and GPU platforms for large language model adaptation. Using LoRA on a Google TPU v5p-8 for training and TPU v6e-8 (Trillium) for inference, we document the full set of code-level adaptations required to port a GPU-native training recipe - built on PyTorch, HuggingFace TRL, and FSDP - to the JAX + Tunix/Qwix stack. These adaptations span mesh configuration, LoRA module naming conventions, sharding annotation corrections, gradient checkpoint, data pipeline restructuring, and a custom Orbax-to-safetensor checkpoint merging procedure. For inference, we detail the vLLM-TPU Docker setup necessary to serve Gemma 4 on v6e-8 and characterize the resulting latency and throughput profile. Compared with a similar-costing 2xH100 GPU baseline under identical hyperparameters, TPU training completes 1.61x faster at 2.12x lower cost. For inference, we cover the vLLM-TPU Docker setup required to serve Gemma 4 on v6e-8 and explain the observed latency and throughput characteristics across a QPS sweep spanning 512 to 16k input tokens. Across both workloads we compare performance and cost against a 2xH100 GPU baseline running identical hyperparameters. The TPU completes training 1.61x faster at 2.12x lower cost. For inference, TPU v6e-8 matches GPU at short context (<=2048 tokens) and decisively outperforms at long context: 66% higher throughput and 23.6x faster TTFT at 4096-token inputs (61 ms vs 1,443 ms at QPS=4). Our work removes a critical gap in the open tooling ecosystem and provides practitioners with a recipe for Gemma 4 Dense 31B deployment on the TPU infrastructure.

cs.DC↗

arXiv2026

HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval

In the competitive landscape of sponsored search, balancing retrieval quality with production latency is a critical challenge. While large retrieval models based on Small Language Models (SLMs) such as Qwen3-Embedding-4B/8B set strong upper bounds on public benchmarks, their deployment in high-throughput, latency-sensitive environments remains impractical. In this paper, we present HARNESS-LM (HLM), a three-phase training framework for transferring the capabilities of large-scale retrievers into compact, cost-efficient models. The approach comprises: (1) training a high-performance reference ("teacher") retriever by fine-tuning a billion-parameter-scale SLM; (2) aligning query representations via an L2 objective to distill knowledge into a sub-600M parameter student encoder; and (3) applying a final contrastive refinement stage to optimize the student for retrieval performance. We also present a comprehensive empirical study of key design choices, including alignment objectives, embedding dimensionality, model scale, architecture, and optimization strategies, to identify configurations that are most effective in production settings. On a real-world Bing Ads evaluation benchmark, HLM recovers over 98% of the reference retriever's precision across multiple settings, while delivering up to 27x lower online query-encoder latency and 20x higher throughput on NVIDIA A100 GPUs. Online A/B testing on Bing Ads further shows a +1% Revenue, +0.6% Impression, and +0.4% Click uplift over the current ensemble of retrievers running in production with the deployed 190M parameter model, clearly highlighting the practical efficacy of the HLM recipe in a real-world sponsored search setting.

cs.IR↗

arXiv2026

H2LooP Spark Preview: Continual Pretraining of Large Language Models for Low-Level Embedded Systems Code

Large language models (LLMs) demonstrate strong code generation abilities in general-purpose programming languages but remain limited in specialized domains such as low-level embedded systems programming. This domain involves hardware register manipulation, vendor-specific SDKs, real-time operating system APIs, and hardware abstraction layers that are underrepresented in standard pretraining corpora. We introduce H2LooP Spark Preview, a continual pretraining (CPT) pipeline that adapts the OLMo-3-7B-a fully open language model to the embedded systems domain using BF16 LoRA with rank-stabilized scaling on 8 NVIDIA H100 GPUs. Our training corpus is constructed from repository-datasheet pairs covering 100B tokens of raw embedded systems data across 117 manufacturers, processed using the hierarchical datasheet-to-code mapping approach proposed in SpecMap (Nipane et al., 2026). The resulting curated dataset split contains 23.5B tokens across 13 embedded domains. Continual pretraining with high-rank LoRA (r=512) yields substantial gains, reducing in-domain perplexity by 70.4% and held-out repository perplexity by 66.1%. On generative code completion benchmarks spanning 13 embedded domains, our 7B model outperforms Claude Opus 4.6 and Qwen3-Coder-30B on 8 categories in token accuracy, showing that targeted continual pretraining enables smaller open-weight models to rival frontier systems on specialized technical tasks. We release the production training checkpoint on Huggingface as an open-source artifact.

cs.LG↗

arXiv2026

SpecMap: Hierarchical LLM Agent for Datasheet-to-Code Traceability Link Recovery in Systems Engineering

Establishing precise traceability between embedded systems datasheets and their corresponding code implementations remains a fundamental challenge in systems engineering, particularly for low-level software where manual mapping between specification documents and large code repositories is infeasible. Existing Traceability Link Recovery approaches primarily rely on lexical similarity and information retrieval techniques, which struggle to capture the semantic, structural, and symbol level relationships prevalent in embedded systems software. We present a hierarchical datasheet-to-code mapping methodology that employs large language models for semantic analysis while explicitly structuring the traceability process across multiple abstraction levels. Rather than performing direct specification-to-code matching, the proposed approach progressively narrows the search space through repository-level structure inference, file-level relevance estimation, and fine-grained symbollevel alignment. The method extends beyond function-centric mapping by explicitly covering macros, structs, constants, configuration parameters, and register definitions commonly found in systems-level C/C++ codebases. We evaluate the approach on multiple open-source embedded systems repositories using manually curated datasheet-to-code ground truth. Experimental results show substantial improvements over traditional information-retrieval-based baselines, achieving up to 73.3% file mapping accuracy. We significantly reduce computational overhead, lowering total LLM token consumption by 84% and end-to-end runtime by approximately 80%. This methodology supports automated analysis of large embedded software systems and enables downstream applications such as training data generation for systems-aware machine learning models, standards compliance verification, and large-scale specification coverage analysis.

cs.SE↗

arXiv2025

Stacking-induced ferroelectricity in tetralayer graphene

Recent studies have reported emergent ferroelectric behavior in twisted or moir\'e-engineered graphene-based van der Waals heterostructures, yet the microscopic origin of this effect remains under debate. Pristine mono- or few-layer graphene lacks a permanent dipole due to its centrosymmetric lattice, making the emergence of ferroelectricity unlikely. However, mixed-stacked graphene, such as the ABCB tetralayer configuration, breaks both inversion and mirror symmetry and has been theoretically predicted to support electrically switchable dipoles. ABCB graphene represents the simplest natural graphene polytype exhibiting intrinsic out-of-plane polarization, arising from asymmetric charge carrier distribution across its layers. Here, we report robust ferroelectric behavior in dual-gated, non-aligned ABCB tetralayer graphene encapsulated in hexagonal boron nitride. The device exhibits pronounced hysteresis in resistance under both top and bottom gate modulation, with the effect persisting up to room temperature. This hysteresis originates from reversible layer-polarized charge reordering, driven by gate-induced transitions between ABCB and BCBA stacking configurations -- without requiring moir\'e superlattices. Our findings establish stacking-order-induced symmetry breaking as a fundamental route to electronic ferroelectricity in graphene and open pathways for non-volatile memory applications based on naturally occurring mixed-stacked multilayer graphene.

cond-mat.mes-hall↗

arXiv2025

Decoupling the effects of ripples from tensile strain on the thermal conductivity of graphene and understanding the role of curvature on the thermal conductivity of graphene with grain boundaries

Ripples, curvature, and grain boundaries in graphene can significantly alter its thermal conductivity (TC), which is paramount in various applications including thermal management etc. In this study, we conducted extensive equilibrium MD simulations and used the Green-Kubo method to elucidate the impact of ripples on the TC of graphene and the impact of curvature and tilt angles characterizing a grain boundary (GB) on the TC of polycrystalline graphene. Although tensile and compressive strains have been known to control the amount of ripples, the effects of strain and ripple on the TC have not been decoupled. With the help of Green-Kubo simulations on larger graphene samples and simulations based on the spectral energy density method on smaller samples without ripples, we show that both samples show an approximately 30% decrease in TC between tensile strains 3% and 10% when ripples become negligible even in the larger sample. Between 0 and 3% strain, when both ripples and strains are present in the larger sample, we decouple the effect of ripples from strain on the TC and show that ripples alone reduce the TC of the larger unstrained sample by approximately 61%. We also present a unique technique to introduce curvatures in the graphene sheet containing GBs by performing MD simulations under NPT ensembles with a constant pressure of 200 bar along the x-axis. Our analysis suggests that the Green-Kubo TC linearly decreases with curvature; however, the rate of decrease depends on the tilt angles of the grain boundary. An analysis of the TC in graphene samples with GBs of varying tilt angles reveals that the TC strongly depends on the tilt angles. Thus, our research underscores the pivotal role of ripple, curvature, and GB in modulating the thermal conductivity of graphene.

cond-mat.mes-hall↗

arXiv2024

Lattice thermal conductivity and phonon properties of polycrystalline graphene

Using spectral energy density method, we predict the phonon scattering mean lifetimes of polycrystalline graphene (PC-G) having polycrystallinity only along $\rm{x}$-axis with seven different misorientation (tilt) angles at room temperature. Contrary to other studies on PC-G samples, our results indicate strong dependence of the thermal conductivity (TC) on the tilt angles. We also show that the square of the group velocity components along $\rm{x}$ and $\rm{y}$ axes and the phonon lifetimes are uncorrelated and the phonon density of states are almost the same for all samples with different tilt angles. Further, a distribution of the group velocity component along $\rm{x}$ or $\rm{y}$ axis as function of normal frequency is found to be exponentially decaying whereas that of phonon lifetime showed piecewise constant function behavior with respect to frequency. We provide parameters for these distribution functions and suggest another measure of the TC based on these distributions. Finally, we perform a size-dependent analysis for two tilt angles, $21.78^\circ$ and $32.20^\circ$, and find that bulk TC components decrease by around 34% to 62% in comparison to the bulk TC values of the pristine graphene. Our analysis reveals intriguing insights into the interplay between grain orientation, phonon scattering and thermal conductivity in graphene.

cond-mat.mtrl-sci↗

arXiv2024

Improving Retrieval in Sponsored Search by Leveraging Query Context Signals

Accurately retrieving relevant bid keywords for user queries is critical in Sponsored Search but remains challenging, particularly for short, ambiguous queries. Existing dense and generative retrieval models often fail to capture nuanced user intent in these cases. To address this, we propose an approach to enhance query understanding by augmenting queries with rich contextual signals derived from web search results and large language models, stored in an online cache. Specifically, we use web search titles and snippets to ground queries in real-world information and utilize GPT-4 to generate query rewrites and explanations that clarify user intent. These signals are efficiently integrated through a Fusion-in-Decoder based Unity architecture, enabling both dense and generative retrieval with serving costs on par with traditional context-free models. To address scenarios where context is unavailable in the cache, we introduce context glancing, a curriculum learning strategy that improves model robustness and performance even without contextual signals during inference. Extensive offline experiments demonstrate that our context-aware approach substantially outperforms context-free models. Furthermore, online A/B testing on a prominent search engine across 160+ countries shows significant improvements in user engagement and revenue.

cs.IR↗

arXiv2024

Dynamical quantum phase transition and thermal equilibrium in the lattice Thirring model

Using tensor network methods, we simulate the real-time evolution of the lattice Thirring model quenched out of equilibrium in both the critical and massive phases and study the appearance of dynamical quantum phase transitions, as nonanalyticities in the Loschmidt rate. Although the presence of a dynamical quantum phase transition in the model does not correspond to quenches across the critical line of the equilibrium phase diagram at zero temperature, we identify a threshold in the energy density of the initial state, necessary for a dynamical quantum phase transition to be present. Moreover, in the case of the gapped quench Hamiltonian, we unveil a connection of this threshold to a transition between different regions in the finite-temperature phase diagram.

hep-lat↗

arXiv2024

Orientation reconstruction of transformation $\alpha$ titanium alloys via polarized light microscopy: methodology and assessment

Emerging microstructural characterization methods have received increased attention owing to their promise of relatively inexpensive and rapid measurement of polycrystalline surface morphology and crystallographic orientations. Among these nascent methods, polarized light microscopy (PLM) is attractive for characterizing alloys comprised of hexagonal crystals, but is hindered by its inability to measure complete crystal orientations. In this study, we explore the potential to reconstruct quasi-deterministic orientations for titanium microstructures characterized via PLM by considering the Burgers orientation relationship between the room temperature $\alpha$ (HCP) phase fibers measured via PLM, and the $\beta$ (BCC) phase orientations of the parent grains present above the transus temperature. We describe this method -- which is capable of narrowing down the orientations to one of four possibilities -- and demonstrate its abilities on idealized computational samples in which the parent $\beta$ microstructure is fully, unambiguously known. We further utilize this method to inform the instantiation of samples for crystal plasticity simulations, and demonstrate the significant improvement in deformation field predictions when utilizing this reconstruction method compared to using results from traditional PLM.

cond-mat.mtrl-sci↗

arXiv2024

Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval

Generative Retrieval introduces a new approach to Information Retrieval by reframing it as a constrained generation task, leveraging recent advancements in Autoregressive (AR) language models. However, AR-based Generative Retrieval methods suffer from high inference latency and cost compared to traditional dense retrieval techniques, limiting their practical applicability. This paper investigates fully Non-autoregressive (NAR) language models as a more efficient alternative for generative retrieval. While standard NAR models alleviate latency and cost concerns, they exhibit a significant drop in retrieval performance (compared to AR models) due to their inability to capture dependencies between target tokens. To address this, we question the conventional choice of limiting the target token space to solely words or sub-words. We propose PIXAR, a novel approach that expands the target vocabulary of NAR models to include multi-word entities and common phrases (up to 5 million tokens), thereby reducing token dependencies. PIXAR employs inference optimization strategies to maintain low inference latency despite the significantly larger vocabulary. Our results demonstrate that PIXAR achieves a relative improvement of 31.0% in MRR@10 on MS MARCO and 23.2% in Hits@5 on Natural Questions compared to standard NAR models with similar latency and cost. Furthermore, online A/B experiments on a large commercial search engine show that PIXAR increases ad clicks by 5.08% and revenue by 4.02%.

cs.CL↗

arXiv2024

UniDEC : Unified Dual Encoder and Classifier Training for Extreme Multi-Label Classification

Extreme Multi-label Classification (XMC) involves predicting a subset of relevant labels from an extremely large label space, given an input query and labels with textual features. Models developed for this problem have conventionally made use of dual encoder (DE) to embed the queries and label texts and one-vs-all (OvA) classifiers to rerank the shortlisted labels by the DE. While such methods have shown empirical success, a major drawback is their computational cost, often requiring upto 16 GPUs to train on the largest public dataset. Such a high cost is a consequence of calculating the loss over the entire label space. While shortlisting strategies have been proposed for classifiers, we aim to study such methods for the DE framework. In this work, we develop UniDEC, a loss-independent, end-to-end trainable framework which trains the DE and classifier together in a unified manner with a multi-class loss, while reducing the computational cost by 4-16x. This is done via the proposed pick-some-label (PSL) reduction, which aims to compute the loss on only a subset of positive and negative labels. These labels are carefully chosen in-batch so as to maximise their supervisory signals. Not only does the proposed framework achieve state-of-the-art results on datasets with labels in the order of millions, it is also computationally and resource efficient in achieving this performance on a single GPU. Code is made available at https://github.com/the-catalyst/UniDEC.

cs.LG↗

arXiv2024

Negative Photo Conductivity Triggered with Visible Light in Wide Bandgap Oxide-Based Optoelectronic Crossbar Memristive Array for Photograph Sensing and Neuromorphic Computing Applications

Photoresponsivity studies of wide-bandgap oxide-based devices have emerged as a vibrant and popular research area. Researchers have explored various material systems in their quest to develop devices capable of responding to illumination. In this study, we engineered a mature wide bandgap oxide-based bilayer heterostructure synaptic memristor to emulate the human brain for applications in neuromorphic computing and photograph sensing. The device exhibits advanced electric and electro-photonic synaptic functions, such as long-term potentiation (LTP), long-term depression (LTD), and paired pulse facilitation (PPF), by applying successive electric and photonic pulses. Moreover, the device exhibits exceptional electrical SET and photonic RESET endurance, maintaining its stability for a minimum of 1200 cycles without any degradation. Density functional theory calculations of the band structures provide insights into the conduction mechanism of the device. Based on this memristor array, we developed an autoencoder and convolutional neural network for noise reduction and image recognition tasks, which achieves a peak signal-to-noise ratio of 562 and high accuracy of 84.23%, while consuming lower energy by four orders of magnitude compared with the Tesla P40 GPU. This groundbreaking research not only opens doors for the integration of our device into image processing but also represents a significant advancement in the realm of in-memory computing and photograph sensing features in a single cell.

physics.app-ph↗

arXiv2024

Theoretical investigation of the vertical dielectric screening dependence on defects for few-layered van der Waals materials

First-principle calculations were employed to analyze the effects induced by vacancies of molybdenum (Mo) and sulfur (S) on the dielectric properties of few-layered MoS2. We explored the combined effects of vacancies and dipole interactions on the dielectric properties of few-layered MoS2. In the presence of dielectric screening, we investigated uniformly distributed Mo and S vacancies, and then considered the case of concentrated vacancies. Our results show that the dielectric screening remarkably depends on the distribution of vacancies owing to the polarization induced by the vacancies and on the interlayer distances. This conclusion was validated for a wide range of wide-gap semiconductors with different positions and distributions of vacancies, providing an effective and reliable method for calculating and predicting electrostatic screening of dimensionally reduced materials. We further provided a method for engineering the dielectric constant by changing the interlayer distance, tuning the number of vacancies and the distribution of vacancies in few-layered van der Waals materials for their application in nanodevices and supercapacitors.

cond-mat.mtrl-sci↗

arXiv2024

Quantum Annealing Approach for the Optimal Real-time Traffic Control using QUBO

Traffic congestion is one of the major issues in urban areas, particularly when traffic loads exceed the roads capacity, resulting in higher petrol consumption and carbon emissions as well as delays and stress for road users. In Asia, the traffic situation can be further deteriorated by road sharing of scooters. How to control the traffic flow to mitigate the congestion has been one of the central issues in transportation research. In this study, we employ a quantum annealing approach to optimize the traffic signals control at a real-life intersection with mixed traffic flows of vehicles and scooters. Considering traffic flow is a continuous and emerging phenomenon, we used quadratic unconstrained binary optimization (QUBO) formalism for traffic optimization, which has a natural equivalence to the Ising model and can be solved efficiently on the quantum annealers, quantum computers or digital annealers. In this article, we first applied the QUBO traffic optimization to artificially generated traffic for a simple intersection, and then we used real-time traffic data to simulate a real Dongda-Keyuan intersection with dedicated cars and scooter lanes, as well as mixed scooter and car lanes. We introduced two types of traffic light control systems for traffic optimization C-QUBO and QUBO. Our rigorous QUBO optimizations show that C-QUBO and QUBO outperform the commonly used fixed cycle method, with QUBO outperforming C-QUBO in some instances. It has been found that QUBO optimization significantly relieves traffic congestion for the unbalanced traffic volume. Furthermore, we found that dynamic changes in traffic light signal duration greatly reduce traffic congestion.

quant-ph↗

arXiv2024

Graph Regularized Encoder Training for Extreme Classification

Deep extreme classification (XC) aims to train an encoder architecture and an accompanying classifier architecture to tag a data point with the most relevant subset of labels from a very large universe of labels. XC applications in ranking, recommendation and tagging routinely encounter tail labels for which the amount of training data is exceedingly small. Graph convolutional networks (GCN) present a convenient but computationally expensive way to leverage task metadata and enhance model accuracies in these settings. This paper formally establishes that in several use cases, the steep computational cost of GCNs is entirely avoidable by replacing GCNs with non-GCN architectures. The paper notices that in these settings, it is much more effective to use graph data to regularize encoder training than to implement a GCN. Based on these insights, an alternative paradigm RAMEN is presented to utilize graph metadata in XC settings that offers significant performance boosts with zero increase in inference computational costs. RAMEN scales to datasets with up to 1M labels and offers prediction accuracy up to 15% higher on benchmark datasets than state of the art methods, including those that use graph metadata to train GCNs. RAMEN also offers 10% higher accuracy over the best baseline on a proprietary recommendation dataset sourced from click logs of a popular search engine. Code for RAMEN will be released publicly.

cs.LG↗

arXiv2023

Forms of Understanding for XAI-Explanations

Explainability has become an important topic in computer science and artificial intelligence, leading to a subfield called Explainable Artificial Intelligence (XAI). The goal of providing or seeking explanations is to achieve (better) 'understanding' on the part of the explainee. However, what it means to 'understand' is still not clearly defined, and the concept itself is rarely the subject of scientific investigation. This conceptual article aims to present a model of forms of understanding for XAI-explanations and beyond. From an interdisciplinary perspective bringing together computer science, linguistics, sociology, philosophy and psychology, a definition of understanding and its forms, assessment, and dynamics during the process of giving everyday explanations are explored. Two types of understanding are considered as possible outcomes of explanations, namely enabledness, 'knowing how' to do or decide something, and comprehension, 'knowing that' -- both in different degrees (from shallow to deep). Explanations regularly start with shallow understanding in a specific domain and can lead to deep comprehension and enabledness of the explanandum, which we see as a prerequisite for human users to gain agency. In this process, the increase of comprehension and enabledness are highly interdependent. Against the background of this systematization, special challenges of understanding in XAI are discussed.

cs.AI↗