SearcharxivSearch

arXiv subjects

Nihal Mehta

Publications and source records attributed to Nihal Mehta.

2 recordsLinked to original sources

Minimax regret treatment rules with finite samples when a quantile is the object of interest

Consider a setup in which a decision maker is informed about the population by a finite sample and based on that sample has to decide whether or not to apply a certain treatment. We work out finite sample minimax regret treatment rules under various sampling schemes when outcomes are restricted onto the unit interval. In contrast to Stoye (2009) where the focus is on maximization of expected utility the focus here is instead on a particular quantile of the outcome distribution. We find that in the case where the sample consists of a fixed number of untreated and a fixed number of treated units, any treatment rule is minimax regret optimal. The same is true in the case of random treatment assignment in the sample with any assignment probability and in the case of testing an innovation when the known quantile of the untreated population equals 1/2. However if the known quantile exceeds 1/2 then never treating is the unique optimal rule and if it is smaller than 1/2 always treating is optimal. We also consider the case where a covariate is included.

econ.EM

Self-Attention as Distributional Projection: A Unified Interpretation of Transformer Architecture

This paper presents a mathematical interpretation of self-attention by connecting it to distributional semantics principles. We show that self-attention emerges from projecting corpus-level co-occurrence statistics into sequence context. Starting from the co-occurrence matrix underlying GloVe embeddings, we demonstrate how the projection naturally captures contextual influence, with the query-key-value mechanism arising as the natural asymmetric extension for modeling directional relationships. Positional encodings and multi-head attention then follow as structured refinements of this same projection principle. Our analysis demonstrates that the Transformer architecture's particular algebraic form follows from these projection principles rather than being an arbitrary design choice.

cs.LG