arXiv subjects

Deepak Gupta

Publications and source records attributed to Deepak Gupta.

At least 19 recordsLinked to original sources

arXiv2026

Jacobian-guided Noise Injection for Quantization Robustness in Large Language Models

Quantization of Large Language Models (LLMs) is often hindered by the sensitivity of the self-attention mechanism to discretization errors. We identify the softmax operator as a bottleneck for quantization stability due to its sensitivity to outliers and state-dependent Jacobian. We theoretically establish that suppressing the norm of this Jacobian helps in bounding quantization-induced performance degradation. Based on this, we propose Jacobian-Guided Noise Injection, a training strategy that injects zero-mean Gaussian noise into pre-attention logits, with variance derived directly from the Jacobian Frobenius norm. Unlike prior approaches that rely on heuristic or penalise jacobian directly, our method provides a way to identify the optimal noise variance based on the local attention sensitivity. We evaluate the method on SOTA LLM architectures, where it demonstrates improved robustness over popular PTQ methods. Empirical analysis reveals that the proposed method gives up to +37% relative gains on Top-1 accuracy on ImageNet-1K for SigLIP and improves relative perplexity by upto 40% on WikiText for language models in low bit quantisation settings, proving the efficacy of the approach.

cs.LG↗

arXiv2026

Quantum Physics-Informed Neural Networks for Solving Integro and Fractional PDEs

Quantum neural networks have emerged as powerful models for approximating nonlinear functions. Yet their use in solving integro-differential equations (IDEs) and fractional integro-partial differential equations (FIPDEs), which involve inherently nonlocal operators, remains unexplored. This work introduces a quantum physics-informed neural network (QPINN) framework that combines a quantum neural network with the governing equations of general nonlinear IDEs and FIPDEs. The proposed quantum network uses an affine feature map and variational quantum circuits to produce trial solutions with explicit trigonometric structure. We prove a quantitative $L^{2}(\mu)$ universal approximation theorem for this architecture, achieving a convergence rate of $\mathcal{O}(n^{-1/2})$. This extends classical Fourier approximation theory to quantum circuits for physics-informed learning. We propose two QPINN variants: the numerical-quadrature QPINN (N-QPINN), which handles nonlocal integrals and fractional operators via high-order numerical quadrature while computing local derivatives through automatic differentiation of quantum trial solutions; and the auxiliary-function QPINN (A-QPINN), which eliminates numerical quadrature by introducing auxiliary variables that reformulate each integro-differential equation as an equivalent coupled system of partial differential equations, enabling a multi-output quantum neural network to simultaneously represent the solution and its associated variables. A series of numerical experiments demonstrates that the proposed QPINN framework accurately captures the behavior of nonlinear IDEs and FIPDEs and outperforms classical physics-informed neural networks.

math.AP↗

arXiv2026

DOT-MoE: Differentiable Optimal Transport for MoEfication

The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency. While Mixture of Experts (MoEs) architectures address this by decoupling model size from inference cost, training MoEs from scratch is often unstable and compute intensive. Conversion of pre-trained dense models into sparse MoEs has emerged as an alternative solution; however, existing methods typically rely on heuristic neuron clustering or random splitting to partition the Feed-Forward Network (FFN) into experts. In this work, we propose DOT-MoE, a novel framework that formulates the decomposition of dense layers as a Differentiable Optimal Transport (DOT) problem. Instead of static heuristics, we model neuron assignment as a balanced transport problem, utilizing differentiable Sinkhorn-Knopp iterations to enforce strict expert capacity constraints. Furthermore, we utilize Straight-Through Estimators (STE) to jointly learn the discrete neuron-to-expert assignment and the token-to-expert routing policy end-to-end. Extensive experiments across multiple architectures and benchmarks demonstrate that DOT-MoE significantly outperforms structured pruning, heuristic clustering, and random-split baselines, retaining 90% of the original dense model's performance while reducing active parameters by 50%.

cs.LG↗

arXiv2026

Entropy Production Rate in Stochastically Time-evolving Asymmetric Networks

Networks that characterise the interactions between units composing complex systems are typically treated as fixed. Yet, such networks often stochastically evolve over time, shaping the collective behavior of complex systems. To date, we lack a general non-equilibrium thermodynamic treatment of such time-dependent networks. In this Letter, to address this problem, we model fluctuating interactions between units of nonlinear network systems as uncorrelated colored noise (i.e., annealed disorder) with a correlation time. This approach enables us to quantify how the entropy production rate (EPR) depends on both the time-scale and the strength of the disorder. Using {\it dynamical mean field theory}, we derive an exact expression for EPR at {\it any} transient time that is validated by simulations of the full dynamics and establish a relation between EPR and autocorrelation at stationarity. We find that annealed disorder shifts the transition from the fixed-point to chaos toward a higher variance of the interactions, thereby suppressing chaos but at the cost of greater dissipation.

cond-mat.stat-mech↗

arXiv2026

Overview of TREC 2025 Biomedical Generative Retrieval (BioGen) Track

Recent advances in large language models (LLMs) have made significant progress across multiple biomedical tasks, including biomedical question answering, lay-language summarization of the biomedical literature, and clinical note summarization. These models have demonstrated strong capabilities in processing and synthesizing complex biomedical information and in generating fluent, human-like responses. Despite these advancements, hallucinations or confabulations remain key challenges when using LLMs in biomedical and other high-stakes domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs' abilities to ground generated statements in verifiable sources have shown that models perform significantly

cs.IR↗

arXiv2026

S2D: Selective Spectral Decay for Quantization-Friendly Conditioning of Neural Activations

Activation outliers in large-scale transformer models pose a fundamental challenge to model quantization, creating excessively large ranges that cause severe accuracy drops during quantization. We empirically observe that outlier severity intensifies with pre-training scale (e.g., progressing from CLIP to the more extensively trained SigLIP and SigLIP2). Through theoretical analysis as well as empirical correlation studies, we establish the direct link between these activation outliers and dominant singular values of the weights. Building on this insight, we propose Selective Spectral Decay ($S^2D$), a geometrically-principled conditioning method that surgically regularizes only the weight components corresponding to the largest singular values during fine-tuning. Through extensive experiments, we demonstrate that $S^2D$ significantly reduces activation outliers and produces well-conditioned representations that are inherently quantization-friendly. Models trained with $S^2D$ achieve up to 7% improved PTQ accuracy on ImageNet under W4A4 quantization and 4% gains when combined with QAT. These improvements also generalize across downstream tasks and vision-language models, enabling the scaling of increasingly large and rigorously trained models without sacrificing deployment efficiency.

cs.LG↗

arXiv2026

BioACE: An Automated Framework for Biomedical Answer and Citation Evaluations

With the increasing use of large language models (LLMs) for generating answers to biomedical questions, it is crucial to evaluate the quality of the generated answers and the references provided to support the facts in the generated answers. Evaluation of text generated by LLMs remains a challenge for question answering, retrieval-augmented generation (RAG), summarization, and many other natural language processing tasks in the biomedical domain, due to the requirements of expert assessment to verify consistency with the scientific literature and complex medical terminology. In this work, we propose BioACE, an automated framework for evaluating biomedical answers and citations against the facts stated in the answers. The proposed BioACE framework considers multiple aspects, including completeness, correctness, precision, and recall, in relation to the ground-truth nuggets for answer evaluation. We developed automated approaches to evaluate each of the aforementioned aspects and performed extensive experiments to assess and analyze their correlation with human evaluations. In addition, we considered multiple existing approaches, such as natural language inference (NLI) and pre-trained language models and LLMs, to evaluate the quality of evidence provided to support the generated answers in the form of citations into biomedical literature. With the detailed experiments and analysis, we provide the best approaches for biomedical answer and citation evaluation as a part of BioACE (https://github.com/deepaknlp/BioACE) evaluation package.

cs.CL↗

arXiv2025

A Dataset and Benchmark for Consumer Healthcare Question Summarization

The quest for seeking health information has swamped the web with consumers health-related questions. Generally, consumers use overly descriptive and peripheral information to express their medical condition or other healthcare needs, contributing to the challenges of natural language understanding. One way to address this challenge is to summarize the questions and distill the key information of the original question. Recently, large-scale datasets have significantly propelled the development of several summarization tasks, such as multi-document summarization and dialogue summarization. However, a lack of a domain-expert annotated dataset for the consumer healthcare questions summarization task inhibits the development of an efficient summarization system. To address this issue, we introduce a new dataset, CHQ-Sum,m that contains 1507 domain-expert annotated consumer health questions and corresponding summaries. The dataset is derived from the community question answering forum and therefore provides a valuable resource for understanding consumer health-related posts on social media. We benchmark the dataset on multiple state-of-the-art summarization models to show the effectiveness of the dataset

cs.CL↗

arXiv2025

Wavelet-Accelerated Physics-Informed Quantum Neural Network for Multiscale Partial Differential Equations

This work proposes a wavelet-based physics-informed quantum neural network framework to efficiently address multiscale partial differential equations that involve sharp gradients, stiffness, rapid local variations, and highly oscillatory behavior. Traditional physics-informed neural networks (PINNs) have demonstrated substantial potential in solving differential equations, and their quantum counterparts, quantum-PINNs, exhibit enhanced representational capacity with fewer trainable parameters. However, both approaches face notable challenges in accurately solving multiscale features. Furthermore, their reliance on automatic differentiation for constructing loss functions introduces considerable computational overhead, resulting in longer training times. To overcome these challenges, we developed a wavelet-accelerated physics-informed quantum neural network that eliminates the need for automatic differentiation, significantly reducing computational complexity. The proposed framework incorporates the multiresolution property of wavelets within the quantum neural network architecture, thereby enhancing the network's ability to effectively capture both local and global features of multiscale problems. Numerical experiments demonstrate that our proposed method achieves superior accuracy while requiring less than five percent of the trainable parameters compared to classical wavelet-based PINNs, resulting in faster convergence. Moreover, it offers a speedup of three to five times compared to existing quantum PINNs, highlighting the potential of the proposed approach for efficiently solving challenging multiscale and oscillatory problems.

cs.LG↗

arXiv2025

Nonequilibrium fluctuations in a harmonic trap with annealed stochastic stiffness

We provide a comprehensive analysis of the positional dynamics and average thermodynamics of an overdamped Brownian particle subject to both, harmonic confinement and annealed disorder due to a temporarily fluctuating trap stiffness. We model this stiffness via a stationary Ornstein-Uhlenbeck (OU) process whose correlation time can be tuned from white noise to a quenched limit. We analytically calculate the positional distribution in these limits and provide exact expressions for the $n$th positional moments at finite correlation times, revealing important insights regarding stationarity. Further, we analyze the average work performed on the particle and the heat dissipated into the environment at all times, illustrating the nonequilibrium character of the system and its relaxation into a steady state. Our analytical results are validated by numerical Langevin simulations.

cond-mat.stat-mech↗

arXiv2025

Inference of a time delay in stochastic systems

Time delay is ubiquitous in many experimental and real-world situations. It is often unclear whether time delay plays a significant role in observed phenomena, and if it does, how long the time lag really is. This would be invaluable knowledge when analyzing and modeling such systems. Hitherto, no universal method is available by which the time delay can be inferred. To address this problem, we propose and demonstrate two different methods to infer time delay in overdamped Langevin systems with delayed feedback. In the first part, we focus on the power spectral density based on the positional data and use a characteristic signature of the time delay to infer the delay time. In limiting cases, we establish a direct relation of the observations made for nonlinear time-delayed feedback forces to analytical results obtained for the linear system. In other situations despite the absence of this direct relation, the characteristic signature remains and can be exploited by a semiautomatic method to infer the delay time. Furthermore, it may not always desirable or possible to observe a system for a long time to infer dependencies and parameters. Thus, in the second part, we propose for the first time a probing method combined with a neural network to infer the delay time, which requires only short observation time series. These proposed methods for inferring time delays in stochastic systems may prove to be valuable tools for gaining deeper insight into the role of delay across a wide range of applications -- from the behavior of individual colloidal particles under feedback control to emergent collective phenomena such as flocking and swarming.

physics.data-an↗

arXiv2025

NOVO: Unlearning-Compliant Vision Transformers

Machine unlearning (MUL) refers to the problem of making a pre-trained model selectively forget some training instances or class(es) while retaining performance on the remaining dataset. Existing MUL research involves fine-tuning using a forget and/or retain set, making it expensive and/or impractical, and often causing performance degradation in the unlearned model. We introduce {\pname}, an unlearning-aware vision transformer-based architecture that can directly perform unlearning for future unlearning requests without any fine-tuning over the requested set. The proposed model is trained by simulating unlearning during the training process itself. It involves randomly separating class(es)/sub-class(es) present in each mini-batch into two disjoint sets: a proxy forget-set and a retain-set, and the model is optimized so that it is unable to predict the forget-set. Forgetting is achieved by withdrawing keys, making unlearning on-the-fly and avoiding performance degradation. The model is trained jointly with learnable keys and original weights, ensuring withholding a key irreversibly erases information, validated by membership inference attack scores. Extensive experiments on various datasets, architectures, and resolutions confirm {\pname}'s superiority over both fine-tuning-free and fine-tuning-based methods.

cs.CV↗

arXiv2025

Efficiently driving F$_1$ molecular motor in experiment by suppressing nonequilibrium variation

F$_1$-ATPase (F$_1$) is central to cellular energy transduction. Forcibly rotated by another motor F$_\mathrm{o}$, F$_1$ catalyzes ATP synthesis by converting mechanical work into chemical free energy stored in the molecule ATP. The details of how F$_\mathrm{o}$ drives F$_1$ are not fully understood; however, evaluating efficient ways to rotate F$_1$ could provide fruitful insights into this driving since there is a selective pressure to improve efficiency. Here, we show that rotating F$_1$ with an angle clamp is significantly more efficient than a constant torque. Our experiments, combined with theory and simulation, indicate that the angle clamp significantly suppresses the nonequilibrium variation that contributes to the futile dissipation of input work.

physics.bio-ph↗

arXiv2025

MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering

Question Answering (QA) and Visual Question Answering (VQA) are well-studied problems in the language and vision domain. One challenging scenario involves multiple sources of information, each of a different modality, where the answer to the question may exist in one or more sources. This scenario contains richer information but is highly complex to handle. In this work, we formulate a novel question-answer generation (QAG) framework in an environment containing multi-source, multimodal information. The answer may belong to any or all sources; therefore, selecting the most prominent answer source or an optimal combination of all sources for a given question is challenging. To address this issue, we propose a question-guided attention mechanism that learns attention across multiple sources and decodes this information for robust and unbiased answer generation. To learn attention within each source, we introduce an explicit alignment between questions and various information sources, which facilitates identifying the most pertinent parts of the source information relative to the question. Scalability in handling diverse questions poses a challenge. We address this by extending our model to a sparse mixture-of-experts (sparse-MoE) framework, enabling it to handle thousands of question types. Experiments on T5 and Flan-T5 using three datasets demonstrate the model's efficacy, supported by ablation studies.

cs.CL↗

arXiv2025

Emergence of ecological structure and species rarity from fluctuating metabolic strategies

Ecosystems frequently display the coexistence of diverse species under resource competition, typically resulting in skewed distributions of rarity and abundance. A potential driver of such coexistence is environmental fluctuations that favor different species over time. How to include and treat such temporal variability in existing consumer-resource models is still an open problem. In this work, we study correlated temporal fluctuations in species' resource uptake rates -- i.e. metabolic strategies -- within a stochastic consumer-resource framework. In a biologically relevant regime, we are able to find analytically the species abundance distributions through the path integral formalism. Our results reveal that stochastic dynamic metabolic strategies induce community structures that align more closely with empirical ecological observations. Within this framework, ecological communities show a higher diversity than expected under static competitive scenarios. We find that all species become extinct when the ratio of the number of species to the number of resources exceeds a critical threshold. Conversely, diversity peaks at intermediate values of the same ratio. Furthermore, when metabolic strategies of different species are different on average, maximal biodiversity is achieved for intermediate values of the amplitude of fluctuations. This work establishes a robust theoretical framework for exploring how temporal dynamics and stochasticity drive biodiversity and community structure.

q-bio.PE↗

arXiv2025

Thermodynamic Cost of Recurrent Erasure

Recent experiments have implemented resetting by means of a time-varying external harmonic trap whereby the trap stiffness is changed from an initial to a final value in finite-time and then the system is reset when it relaxes to an equilibrium distribution in the final trap. Such setups are very similar to those studied in the context of the finite-time Landauer erasure principle. We analyze the thermodynamic costs of such a setup by deriving a moment generating function for the work cost of recurrently changing the trap stiffness in finite-time, thereby maintaining a non-equilibrium steady state. We analyze the mean and variance of the work required for a specific experimentally viable protocol and also obtain an optimal protocol which minimizes the mean cost. For both these procedures, our analysis captures both the large-time and short-time corrections. For the optimal protocol, we obtain a closed form expression for the mean cost for all protocol durations, thereby making contact with earlier work on geometric measures of dissipation-minimizing optimal protocols that implement information erasure.

cond-mat.stat-mech↗

arXiv2024

Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion

Seismic full waveform inversion (FWI) has seen promising advancements through deep learning. Existing approaches typically focus on task-specific models trained and evaluated in isolation that lead to limited generalization across different geological scenarios. In this work we introduce a task-agnostic foundational model for FWI that captures general features across tasks. We first demonstrate that full fine-tuning of this foundational model outperforms task-specific models built from scratch by delivering superior performance across multiple benchmarks. Building upon this we employ parameter-efficient fine-tuning (PEFT) to further reduce computational overhead. By fine-tuning only a small fraction of the model parameters PEFT achieves comparable results to full fine-tuning while significantly lowering memory and computational requirements. Additionally, PEFT excels in out-of-distribution tasks where it outperforms both full fine-tuning and task-specific models. These findings establish the value of foundational modeling for FWI and highlight PEFT as an effective strategy for efficient and scalable adaptation across diverse tasks.

cs.CE↗

arXiv2024

Overview of TREC 2024 Medical Video Question Answering (MedVidQA) Track

One of the key goals of artificial intelligence (AI) is the development of a multimodal system that facilitates communication with the visual world (image and video) using a natural language query. Earlier works on medical question answering primarily focused on textual and visual (image) modalities, which may be inefficient in answering questions requiring demonstration. In recent years, significant progress has been achieved due to the introduction of large-scale language-vision datasets and the development of efficient deep neural techniques that bridge the gap between language and visual understanding. Improvements have been made in numerous vision-and-language tasks, such as visual captioning visual question answering, and natural language video localization. Most of the existing work on language vision focused on creating datasets and developing solutions for open-domain applications. We believe medical videos may provide the best possible answers to many first aid, medical emergency, and medical education questions. With increasing interest in AI to support clinical decision-making and improve patient engagement, there is a need to explore such challenges and develop efficient algorithms for medical language-video understanding and generation. Toward this, we introduced new tasks to foster research toward designing systems that can understand medical videos to provide visual answers to natural language questions, and are equipped with multimodal capability to generate instruction steps from the medical video. These tasks have the potential to support the development of sophisticated downstream applications that can benefit the public and medical professionals.

cs.CV↗