arXiv subjects

Nilava Metya

Publications and source records attributed to Nilava Metya.

3 recordsLinked to original sources

arXiv2026

Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics

We study minimal attention-only transformers under all-token corruption and show they admit a two-stage empirical Bayes interpretation. A single attention step computes a kernel-weighted posterior mean with respect to the empirical distribution defined by the context. Depth refines this distribution through particle dynamics (Stage 1), while a long-range skip-connection carries the noisy input as a query for posterior inference (Stage 2), revealing distinct statistical roles for depth and attention residuals. The framework isolates a minimal setting in which the context itself induces a depth-dependent energy landscape governing in-context inference. We show that effective denoising can emerge without an explicit noise schedule: a fixed kernel bandwidth and finite integration horizon suffice, yielding a principled depth-noise relationship. We further establish a posterior-mean recovery guarantee for a class of well-behaved priors, where the empirical estimator converges to the Bayes-optimal predictor under asymptotic conditions. Connecting these dynamics to reverse-diffusion limits, our results provide a statistical interpretation of attention as in-context inference via sample-based posterior estimation, without explicit density modeling.

cs.LG↗

arXiv2025

Temporal Robustness in Discrete Time Linear Dynamical Systems

Discrete time linear dynamical systems, including Markov chains, have found many applications including in security settings such as in cybersecurity operations center (CSOC) management and in managing health risks. However, in these two scenarios, there is uncertainty about the time horizon for which the system runs. This creates uncertainty about the cost (or reward) incurred based on the state distribution when the system stops. Given past data samples of how long a system ran, we theoretically analyze the cost incurred at the stop of the system as a distributional robust cost estimation task in a Wasserstein ambiguity set. Towards this, we show an equivalence between a discrete time Markov Chain on a probability simplex and a global asymptotic stable (GAS) discrete time linear dynamical system, allowing us to base our study on a GAS system only. Then, we provide various polynomial time algorithms and hardness results for different cases in our theoretical study, including a novel proof of a fundamental result about Wassertein distance based polytope. We experiment with real world data in CSOC domain and prior data in health domain to reveal the benefits of our model and approach.

math.OC↗

arXiv2024

Degrees of the Wasserstein Distance to Small Toric Models

The study of the closest point(s) on a statistical model from a given distribution in the probability simplex with respect to a fixed Wasserstein metric gives rise to a polyhedral norm distance optimization problem. There are two components to the complexity of determining the Wasserstein distance from a data point to a model. One is the combinatorial complexity that is governed by the combinatorics of the Lipschitz polytope of the finite metric to be used. Another is the algebraic complexity, which is governed by the polar degrees of the Zariski closure of the model. We find formulas for the polar degrees of rational normal scrolls and graphical models whose underlying graphs are star trees. Also, the polar degrees of the graphical models with four binary random variables where the graphs are a path on four vertices and the four-cycle, as well as for small, no-three-way interaction models, were computed. We investigate the algebraic degree of computing the Wasserstein distance to a small subset of these models. It was observed that this algebraic degree is typically smaller than the corresponding polar degree.

math.AG↗