arXiv subjects

Sukjin Han

Publications and source records attributed to Sukjin Han.

18 recordsLinked to original sources

arXiv2026

How Well Do LLMs Predict Human Behavior? A Measure of their Pretrained Knowledge

Large language models (LLMs) are increasingly used to predict human behavior. We propose a measure for evaluating how much knowledge a pretrained LLM brings to such a prediction: its equivalent sample size, defined as the amount of task-specific data needed to match the predictive accuracy of the LLM. We estimate this measure by comparing the prediction error of a fixed LLM in a given domain to that of flexible machine learning models trained on increasing samples of domain-specific data. We further provide a statistical inference procedure by developing a new asymptotic theory for cross-validated prediction error. Finally, we apply this method to the Panel Study of Income Dynamics. We find that LLMs encode considerable predictive information for some economic variables but much less for others, suggesting that their value as substitutes for domain-specific data differs markedly across settings.

econ.EM↗

arXiv2025

On Quantile Treatment Effects, Rank Similarity,and Variation of Instrumental Variables

This paper develops a nonparametric framework to identify and estimate distributional treatment effects under nonseparable endogeneity. We begin by revisiting the widely adopted \emph{rank similarity} (RS) assumption and characterizing it by the relationship it imposes between observed and counterfactual potential outcome distributions. The characterization highlights the restrictiveness of RS, motivating a weaker identifying condition. Under this alternative, we construct identifying bounds on the distributional treatment effects of interest through a linear semi-infinite programming (SILP) formulation. Our identification strategy also clarifies how richer exogenous instrument variation, such as multi-valued or multiple instruments, can further tighten these bounds. Finally, exploiting the SILP's saddle-point structure and Karush-Kuhn-Tucker (KKT) conditions, we establish large-sample properties for the empirical SILP: consistency and asymptotic distribution results for the estimated bounds and associated solutions.

econ.EM↗

arXiv2025

Copyright and Competition: Estimating Supply and Demand with Unstructured Data

We study the competitive and welfare effects of copyright in creative industries in the face of cost-reducing technologies such as generative artificial intelligence. Creative products often feature unstructured attributes (e.g., images and text) that are complex and high-dimensional. To address this challenge, we study a stylized design product -- fonts -- using data from the world's largest font marketplace. We construct neural network embeddings to quantify unstructured attributes and measure visual similarity in a manner consistent with human perception. Spatial regression and event-study analyses demonstrate that competition is local in the visual characteristics space. Building on this evidence, we develop a structural model of supply and demand that incorporates embeddings and captures product positioning under copyright-based similarity constraints. Our estimates reveal consumers' heterogeneous design preferences and producers' cost-effective mimicry advantages. Counterfactual analyses show that copyright protection can raise consumer welfare by encouraging product relocation, and that the optimal policy depends on the interaction between copyright and cost-reducing technologies.

econ.EM↗

arXiv2024

Mining Causality: AI-Assisted Search for Instrumental Variables

The instrumental variables (IVs) method is a leading empirical strategy for causal inference. Finding IVs is a heuristic and creative process, and justifying its validity -- especially exclusion restrictions -- is largely rhetorical. We propose using large language models (LLMs) to search for new IVs through narratives and counterfactual reasoning, similar to how a human researcher would. The stark difference, however, is that LLMs can dramatically accelerate this process and explore an extremely large search space. We demonstrate how to construct prompts to search for potentially valid IVs. We contend that multi-step and role-playing prompting strategies are effective for simulating the endogenous decision-making processes of economic agents and for navigating language models through the realm of real-world scenarios, rather than anchoring them within the narrow realm of academic discourses on IVs. We apply our method to three well-known examples in economics: returns to schooling, supply and demand, and peer effects. We then extend our strategy to finding (i) control variables in regression and difference-in-differences and (ii) running variables in regression discontinuity designs.

econ.EM↗

arXiv2024

Estimating Causal Effects of Discrete and Continuous Treatments with Binary Instruments

We propose an instrumental variable framework for identifying and estimating causal effects of discrete and continuous treatments with binary instruments. The basis of our approach is a local copula representation of the joint distribution of the potential outcomes and unobservables determining treatment assignment. This representation allows us to introduce an identifying assumption, so-called constant local dependence, that restricts the local dependence of the copula with respect to the treatment propensity. We show that constant local dependence identifies treatment effects for the entire population and other subpopulations such as the treated. The identification results are constructive and lead to practical estimation and inference procedures based on distribution regression. An application to estimating the effect of sleep on well-being uncovers interesting patterns of heterogeneity.

econ.EM↗

arXiv2024

Set-Valued Control Functions

The control function approach allows the researcher to identify various causal effects of interest. While powerful, it requires a strong invertibility assumption in the selection process, which limits its applicability. This paper expands the scope of the nonparametric control function approach by allowing the control function to be set-valued and derive sharp bounds on structural parameters. The proposed generalization accommodates a wide range of selection processes involving discrete endogenous variables, random coefficients, treatment selections with interference, and dynamic treatment selections. The framework also applies to partially observed or identified controls that are directly motivated from economic models.

econ.EM↗

arXiv2024

Inference for Interval-Identified Parameters Selected from an Estimated Set

Interval identification of parameters such as average treatment effects, average partial effects and welfare is particularly common when using observational data and experimental data with imperfect compliance due to the endogeneity of individuals' treatment uptake. In this setting, the researcher is typically interested in a treatment or policy that is either selected from the estimated set of best-performers or arises from a data-dependent selection rule. In this paper, we develop new inference tools for interval-identified parameters chosen via these forms of selection. We develop three types of confidence intervals for data-dependent and interval-identified parameters, discuss how they apply to several examples of interest and prove their uniform asymptotic validity under weak assumptions.

econ.EM↗

arXiv2024

Testing Information Ordering for Strategic Agents

Specifying the information structure in strategic environments is difficult for empirical researchers. We develop a test of information ordering that examines whether the true information structure is at least as informative as a proposed baseline. Using Bayes Correlated Equilibrium (BCE), we translate the ordering of information structures into testable moment inequalities and establish uniform asymptotic validity for our testing procedure. In an application to U.S. airline markets, we test whether hub airlines have informational advantages beyond cost and demand benefits. We reject the privileged information hypothesis, with rejections concentrated in large, competitive markets.

econ.EM↗

arXiv2023

On Quantile Treatment Effects, Rank Similarity, and Variation of Instrumental Variables

This paper investigates how certain relationship between observed and counterfactual distributions serves as an identifying condition for treatment effects when the treatment is endogenous, and shows that this condition holds in a range of nonparametric models for treatment effects. To this end, we first provide a novel characterization of the prevalent assumption restricting treatment heterogeneity in the literature, namely rank similarity. Our characterization demonstrates the stringency of this assumption and allows us to relax it in an economically meaningful way, resulting in our identifying condition. It also justifies the quest of richer exogenous variations in the data (e.g., multi-valued or multiple instrumental variables) in exchange for weaker identifying conditions. The primary goal of this investigation is to provide empirical researchers with tools that are robust and easy to implement but still yield tight policy evaluations.

econ.EM↗

arXiv2023

Policy Learning with Distributional Welfare

In this paper, we explore optimal treatment allocation policies that target distributional welfare. Most literature on treatment choice has considered utilitarian welfare based on the conditional average treatment effect (ATE). While average welfare is intuitive, it may yield undesirable allocations especially when individuals are heterogeneous (e.g., with outliers) - the very reason individualized treatments were introduced in the first place. This observation motivates us to propose an optimal policy that allocates the treatment based on the conditional quantile of individual treatment effects (QoTE). Depending on the choice of the quantile probability, this criterion can accommodate a policymaker who is either prudent or negligent. The challenge of identifying the QoTE lies in its requirement for knowledge of the joint distribution of the counterfactual outcomes, which is not generally point-identified. We introduce minimax policies that are robust to this model uncertainty. A range of identifying assumptions can be used to yield more informative policies. For both stochastic and deterministic policies, we establish the asymptotic bound on the regret of implementing the proposed policies. The framework can be generalized to any setting where welfare is defined as a functional of the joint distribution of the potential outcomes.

stat.ME↗

arXiv2022

Synthetic Blips: Generalizing Synthetic Controls for Dynamic Treatment Effects

We propose a generalization of the synthetic control and interventions methods to the setting with dynamic treatment effects. We consider the estimation of unit-specific treatment effects from panel data collected under a general treatment sequence. Here, each unit receives multiple treatments sequentially, according to an adaptive policy that depends on a latent, endogenously time-varying confounding state. Under a low-rank latent factor model assumption, we develop an identification strategy for any unit-specific mean outcome under any sequence of interventions. The latent factor model we propose admits linear time-varying and time-invariant dynamical systems as special cases. Our approach can be viewed as an identification strategy for structural nested mean models -- a widely used framework for dynamic treatment effects -- under a low-rank latent factor assumption on the blip effects. Unlike these models, however, it is more permissive in observational settings, thereby broadening its applicability. Our method, which we term synthetic blip effects, is a backwards induction process in which the blip effect of a treatment at each period and for a target unit is recursively expressed as a linear combination of the blip effects of a group of other units that received the designated treatment. This strategy avoids the combinatorial explosion in the number of units that would otherwise be required by a naive application of prior synthetic control and intervention methods in dynamic treatment settings. We provide estimation algorithms that are easy to implement in practice and yield estimators with desirable properties. Using unique Korean firm-level panel data, we demonstrate how the proposed framework can be used to estimate individualized dynamic treatment effects and to derive optimal treatment allocation rules in the context of financial support for exporting firms.

econ.EM↗

arXiv2021

Optimal Dynamic Treatment Regimes and Partial Welfare Ordering

Dynamic treatment regimes are treatment allocations tailored to heterogeneous individuals. The optimal dynamic treatment regime is a regime that maximizes counterfactual welfare. We introduce a framework in which we can partially learn the optimal dynamic regime from observational data, relaxing the sequential randomization assumption commonly employed in the literature but instead using (binary) instrumental variables. We propose the notion of sharp partial ordering of counterfactual welfares with respect to dynamic regimes and establish mapping from data to partial ordering via a set of linear programs. We then characterize the identified set of the optimal regime as the set of maximal elements associated with the partial ordering. We relate the notion of partial ordering with a more conventional notion of partial identification using topological sorts. Practically, topological sorts can be served as a policy benchmark for a policymaker. We apply our method to understand returns to schooling and post-school training as a sequence of treatments by combining data from multiple sources. The framework of this paper can be used beyond the current context, e.g., in establishing rankings of multiple treatments or policies across different counterfactual scenarios.

econ.EM↗

arXiv2021

Shapes as Product Differentiation: Neural Network Embedding in the Analysis of Markets for Fonts

Many differentiated products have key attributes that are unstructured and thus high-dimensional (e.g., design, text). Instead of treating unstructured attributes as unobservables in economic models, quantifying them can be important to answer interesting economic questions. To propose an analytical framework for these types of products, this paper considers one of the simplest design products-fonts-and investigates merger and product differentiation using an original dataset from the world's largest online marketplace for fonts. We quantify font shapes by constructing embeddings from a deep convolutional neural network. Each embedding maps a font's shape onto a low-dimensional vector. In the resulting product space, designers are assumed to engage in Hotelling-type spatial competition. From the image embeddings, we construct two alternative measures that capture the degree of design differentiation. We then study the causal effects of a merger on the merging firm's creative decisions using the constructed measures in a synthetic control method. We find that the merger causes the merging firm to increase the visual variety of font design. Notably, such effects are not captured when using traditional measures for product offerings (e.g., specifications and the number of products) constructed from structured data.

econ.EM↗

arXiv2020

A Computational Approach to Identification of Treatment Effects for Policy Evaluation

For counterfactual policy evaluation, it is important to ensure that treatment parameters are relevant to policies in question. This is especially challenging under unobserved heterogeneity, as is well featured in the definition of the local average treatment effect (LATE). Being intrinsically local, the LATE is known to lack external validity in counterfactual environments. This paper investigates the possibility of extrapolating local treatment effects to different counterfactual settings when instrumental variables are only binary. We propose a novel framework to systematically calculate sharp nonparametric bounds on various policy-relevant treatment parameters that are defined as weighted averages of the marginal treatment effect (MTE). Our framework is flexible enough to fully incorporate statistical independence (rather than mean independence) of instruments and a large menu of identifying assumptions beyond the shape restrictions on the MTE that have been considered in prior studies. We apply our method to understand the effects of medical insurance policies on the use of medical services.

econ.EM↗

arXiv2019

Censored Quantile Instrumental Variable Estimation with Stata

Many applications involve a censored dependent variable and an endogenous independent variable. Chernozhukov et al. (2015) introduced a censored quantile instrumental variable estimator (CQIV) for use in those applications, which has been applied by Kowalski (2016), among others. In this article, we introduce a Stata command, cqiv, that simplifes application of the CQIV estimator in Stata. We summarize the CQIV estimator and algorithm, we describe the use of the cqiv command, and we provide empirical examples.

econ.EM↗

arXiv2019

Multiple Treatments with Strategic Interaction

We develop an empirical framework to identify and estimate the effects of treatments on outcomes of interest when the treatments are the result of strategic interaction (e.g., bargaining, oligopolistic entry, peer effects). We consider a model where agents play a discrete game with complete information whose equilibrium actions (i.e., binary treatments) determine a post-game outcome in a nonseparable model with endogeneity. Due to the simultaneity in the first stage, the model as a whole is incomplete and the selection process fails to exhibit the conventional monotonicity. Without imposing parametric restrictions or large support assumptions, this poses challenges in recovering treatment parameters. To address these challenges, we first establish a monotonic pattern of the equilibria in the first-stage game in terms of the number of treatments selected. Based on this finding, we derive bounds on the average treatment effects (ATEs) under nonparametric shape restrictions and the existence of excluded exogenous variables. We show that instrument variation that compensates strategic substitution helps solve the multiple equilibria problem. We apply our method to data on airlines and air pollution in cities in the U.S. We find that (i) the causal effect of each airline on pollution is positive, and (ii) the effect is increasing in the number of firms but at a decreasing rate.

econ.EM↗

arXiv2019

Estimation in a Generalization of Bivariate Probit Models with Dummy Endogenous Regressors

The purpose of this paper is to provide guidelines for empirical researchers who use a class of bivariate threshold crossing models with dummy endogenous variables. A common practice employed by the researchers is the specification of the joint distribution of the unobservables as a bivariate normal distribution, which results in a bivariate probit model. To address the problem of misspecification in this practice, we propose an easy-to-implement semiparametric estimation framework with parametric copula and nonparametric marginal distributions. We establish asymptotic theory, including root-n normality, for the sieve maximum likelihood estimators that can be used to conduct inference on the individual structural parameters and the average treatment effect (ATE). In order to show the practical relevance of the proposed framework, we conduct a sensitivity analysis via extensive Monte Carlo simulation exercises. The results suggest that the estimates of the parameters, especially the ATE, are sensitive to parametric specification, while semiparametric estimation exhibits robustness to underlying data generating processes. We then provide an empirical illustration where we estimate the effect of health insurance on doctor visits. In this paper, we also show that the absence of excluded instruments may result in identification failure, in contrast to what some practitioners believe.

econ.EM↗

arXiv2019

Identification in Nonparametric Models for Dynamic Treatment Effects

This paper develops a nonparametric model that represents how sequences of outcomes and treatment choices influence one another in a dynamic manner. In this setting, we are interested in identifying the average outcome for individuals in each period, had a particular treatment sequence been assigned. The identification of this quantity allows us to identify the average treatment effects (ATE's) and the ATE's on transitions, as well as the optimal treatment regimes, namely, the regimes that maximize the (weighted) sum of the average potential outcomes, possibly less the cost of the treatments. The main contribution of this paper is to relax the sequential randomization assumption widely used in the biostatistics literature by introducing a flexible choice-theoretic framework for a sequence of endogenous treatments. We show that the parameters of interest are identified under each period's two-way exclusion restriction, i.e., with instruments excluded from the outcome-determining process and other exogenous variables excluded from the treatment-selection process. We also consider partial identification in the case where the latter variables are not available. Lastly, we extend our results to a setting where treatments do not appear in every period.

econ.EM↗