SearcharxivSearch

arXiv subjects

Karun Adusumilli

Publications and source records attributed to Karun Adusumilli.

10 recordsLinked to original sources

Designing Persuasive Experiments

Incentives in experimental design are often misaligned: experimenters design and finance experiments to seek regulatory approval, while regulators seek to maximize social-welfare. We propose a framework to resolve this conflict, wherein regulators set a minimum welfare threshold, and experimenters optimize designs subject to this constraint. It requires no knowledge of experimenters' private preferences or costs and mitigates strategic Bayesian persuasion. Under normal priors, Neyman-allocation is always the optimal-sampling strategy, regardless of specific objectives. We also characterize the optimal stopping-rule. A numerical study calibrated to clinical-trial data shows sample-size reductions of over 48% relative to classical designs attaining the same social-welfare.

econ.EM

You've Got to be Efficient: Ambiguity, Misspecification and Variational Preferences

This article introduces a framework for evaluating statistical decisions under both prior ambiguity and likelihood misspecification. We begin with an ambiguity set - a frequentist model that pairs a possibly misspecified likelihood with every possible prior - and uniformly expand it by a Kullback-Leibler radius to accommodate likelihood misspecification. We show that optimal decisions under this framework are equivalent to minimax decisions with an exponentially tilted loss function. Misspecification manifests as an exponential tilting of the loss, while ambiguity corresponds to a search for the least favorable prior. This separation between ambiguity and misspecification enables local asymptotic analysis under global misspecification, achieved by localizing the priors alone. Remarkably, for both estimation and treatment assignment, we show that optimal decisions coincide with those under correct specification, regardless of the degree of misspecification. These results extend to semi-parametric models. As a practical consequence, our findings imply that practitioners should prefer maximum likelihood over the simulated method of moments, and efficient GMM estimators - such as two-step GMM - over diagonally weighted alternatives.

econ.EM

From Cross-Validation to SURE: Asymptotic Risk of Tuned Regularized Estimators

We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by $n$-fold cross-validation (CV). The out-of-sample prediction loss of such estimators converges in distribution to the squared-error loss (risk function) of shrinkage estimators in the normal means model, tuned by Stein's unbiased risk estimate (SURE). This risk function provides a more fine-grained picture of predictive performance than uniform bounds on worst-case regret, which are common in learning theory: it quantifies how risk varies with the true parameter. As key intermediate steps, we show that (i) $n$-fold CV converges uniformly to SURE, and (ii) while SURE typically has multiple local minima, its global minimum is generically well separated. Well-separation ensures that uniform convergence of CV to SURE translates into convergence of the tuning parameter chosen by CV to that chosen by SURE.

math.ST

Continuous time asymptotic representations for adaptive experiments

This article develops a continuous-time asymptotic framework for analyzing adaptive experiments -- settings in which data collection and treatment assignment evolve dynamically in response to incoming information. A key challenge in analyzing fully adaptive experiments, where the assignment policy is updated after each observation, is that the sequence of policy rules often lack a well-defined asymptotic limit. To address this, we focus instead on the empirical allocation process, which captures the fraction of observations assigned to each treatment over time. We show that, under general conditions, any adaptive experiment and its associated empirical allocation process can be approximated by a limit experiment defined by Gaussian diffusions with unknown drifts and a corresponding continuous-time allocation process. This limit representation facilitates the analysis of optimal decision rules by reducing the dimensionality of the state-space and leveraging the tractability of Gaussian diffusions. We apply the framework to derive optimal estimators, analyze in-sample regret for adaptive experiments, and construct e-processes for anytime-valid inference. Notably, we introduce the first definition of any-time and any-experiment valid inference for multi-treatment settings.

econ.EM

Dynamically Optimal Treatment Allocation

Dynamic decisions are pivotal to economic policy making. We show how existing evidence from randomized control trials can be utilized to guide personalized decisions in challenging dynamic environments with budget and capacity constraints. Recent advances in reinforcement learning now enable the solution of many complex, real-world problems for the first time. We allow for restricted classes of policy functions and prove that their regret decays at rate n^(-0.5), the same as in the static case. Applying our methods to job training, we find that by exploiting the problem's dynamic structure, we achieve significantly higher welfare compared to static approaches.

econ.EM

Optimal tests following sequential experiments

Recent years have seen tremendous advances in the theory and application of sequential experiments. While these experiments are not always designed with hypothesis testing in mind, researchers may still be interested in performing tests after the experiment is completed. The purpose of this paper is to aid in the development of optimal tests for sequential experiments by analyzing their asymptotic properties. Our key finding is that the asymptotic power function of any test can be matched by a test in a limit experiment where a Gaussian process is observed for each treatment, and inference is made for the drifts of these processes. This result has important implications, including a powerful sufficiency result: any candidate test only needs to rely on a fixed set of statistics, regardless of the type of sequential experiment. These statistics are the number of times each treatment has been sampled by the end of the experiment, along with final value of the score (for parametric models) or efficient influence function (for non-parametric models) process for each treatment. We then characterize asymptotically optimal tests under various restrictions such as unbiasedness, α-spending constraints etc. Finally, we apply our our results to three key classes of sequential experiments: costly sampling, group sequential trials, and bandit experiments, and show how optimal inference can be conducted in these scenarios.

econ.EM

Temporal-Difference estimation of dynamic discrete choice models

We study the use of Temporal-Difference learning for estimating the structural parameters in dynamic discrete choice models. Our algorithms are based on the conditional choice probability approach but use functional approximations to estimate various terms in the pseudo-likelihood function. We suggest two approaches: The first - linear semi-gradient - provides approximations to the recursive terms using basis functions. The second - Approximate Value Iteration - builds a sequence of approximations to the recursive terms by solving non-parametric estimation problems. Our approaches are fast and naturally allow for continuous and/or high-dimensional state spaces. Furthermore, they do not require specification of transition densities. In dynamic games, they avoid integrating over other players' actions, further heightening the computational advantage. Our proposals can be paired with popular existing methods such as pseudo-maximum-likelihood, and we propose locally robust corrections for the latter to achieve parametric rates of convergence. Monte Carlo simulations confirm the properties of our algorithms in practice.

econ.EM

How to sample and when to stop sampling: The generalized Wald problem and minimax policies

We study sequential experiments where sampling is costly and a decision-maker aims to determine the best treatment for full scale implementation by (1) adaptively allocating units between two possible treatments, and (2) stopping the experiment when the expected welfare (inclusive of sampling costs) from implementing the chosen treatment is maximized. Working under a continuous time limit, we characterize the optimal policies under the minimax regret criterion. We show that the same policies also remain optimal under both parametric and non-parametric outcome distributions in an asymptotic regime where sampling costs approach zero. The minimax optimal sampling rule is just the Neyman allocation: it is independent of sampling costs and does not adapt to observed outcomes. The decision-maker halts sampling when the product of the average treatment difference and the number of observations surpasses a specific threshold. The results derived also apply to the so-called best-arm identification problem, where the number of observations is exogenously specified.

econ.EM

Neyman allocation is minimax optimal for best arm identification with two arms

This note describes the optimal policy rule, according to the local asymptotic minimax regret criterion, for best arm identification when there are only two treatments. It is shown that the optimal sampling rule is the Neyman allocation, which allocates a constant fraction of units to each treatment in a manner that is proportional to the standard deviation of the treatment outcomes. When the variances are equal, the optimal ratio is one-half. This policy is independent of the data, so there is no adaptation to previous outcomes. At the end of the experiment, the policy maker adopts the treatment with higher average outcomes.

econ.EM

Risk and optimal policies in bandit experiments

We provide a decision theoretic analysis of bandit experiments under local asymptotics. Working within the framework of diffusion processes, we define suitable notions of asymptotic Bayes and minimax risk for these experiments. For normally distributed rewards, the minimal Bayes risk can be characterized as the solution to a second-order partial differential equation (PDE). Using a limit of experiments approach, we show that this PDE characterization also holds asymptotically under both parametric and non-parametric distributions of the rewards. The approach further describes the state variables it is asymptotically sufficient to restrict attention to, and thereby suggests a practical strategy for dimension reduction. The PDEs characterizing minimal Bayes risk can be solved efficiently using sparse matrix routines or Monte-Carlo methods. We derive the optimal Bayes and minimax policies from their numerical solutions. These optimal policies substantially dominate existing methods such as Thompson sampling; the risk of the latter is often twice as high.

econ.EM