arXiv subjects

Sokbae Lee

Publications and source records attributed to Sokbae Lee.

51 records · Page 3Linked to original sources

arXiv2018

High Dimensional Classification through $\ell_0$-Penalized Empirical Risk Minimization

We consider a high dimensional binary classification problem and construct a classification procedure by minimizing the empirical misclassification risk with a penalty on the number of selected features. We derive non-asymptotic probability bounds on the estimated sparsity as well as on the excess misclassification risk. In particular, we show that our method yields a sparse solution whose l0-norm can be arbitrarily close to true sparsity with high probability and obtain the rates of convergence for the excess misclassification risk. The proposed procedure is implemented via the method of mixed integer linear programming. Its numerical performance is illustrated in Monte Carlo experiments.

stat.ME↗

arXiv2018

Best Subset Binary Prediction

We consider a variable selection problem for the prediction of binary outcomes. We study the best subset selection procedure by which the covariates are chosen by maximizing Manski (1975, 1985)'s maximum score objective function subject to a constraint on the maximal number of selected variables. We show that this procedure can be equivalently reformulated as solving a mixed integer optimization problem, which enables computation of the exact or an approximate solution with a definite approximation error bound. In terms of theoretical results, we obtain non-asymptotic upper and lower risk bounds when the dimension of potential covariates is possibly much larger than the sample size. Our upper and lower risk bounds are minimax rate-optimal when the maximal number of selected variables is fixed and does not increase with the sample size. We illustrate usefulness of the best subset binary prediction approach via Monte Carlo simulations and an empirical application of the work-trip transportation mode choice.

stat.ME↗

arXiv2018

Identifying Effects of Multivalued Treatments

Multivalued treatment models have typically been studied under restrictive assumptions: ordered choice, and more recently unordered monotonicity. We show how treatment effects can be identified in a more general class of models that allows for multidimensional unobserved heterogeneity. Our results rely on two main assumptions: treatment assignment must be a measurable function of threshold-crossing rules, and enough continuous instruments must be available. We illustrate our approach for several classes of models.

econ.EM↗

arXiv2017

Exact computation of GMM estimators for instrumental variable quantile regression models

We show that the generalized method of moments (GMM) estimation problem in instrumental variable quantile regression (IVQR) models can be equivalently formulated as a mixed integer quadratic programming problem. This enables exact computation of the GMM estimators for the IVQR models. We illustrate the usefulness of our algorithm via Monte Carlo experiments and an application to demand for fish.

stat.CO↗

arXiv2016

Oracle Estimation of a Change Point in High Dimensional Quantile Regression

In this paper, we consider a high-dimensional quantile regression model where the sparsity structure may differ between two sub-populations. We develop $\ell_1$-penalized estimators of both regression coefficients and the threshold parameter. Our penalized estimators not only select covariates but also discriminate between a model with homogeneous sparsity and a model with a change point. As a result, it is not necessary to know or pretest whether the change point is present, or where it occurs. Our estimator of the change point achieves an oracle property in the sense that its asymptotic distribution is the same as if the unknown active sets of regression coefficients were known. Importantly, we establish this oracle property without a perfect covariate selection, thereby avoiding the need for the minimum level condition on the signals of active covariates. Dealing with high-dimensional quantile regression with an unknown change point calls for a new proof technique since the quantile loss function is non-smooth and furthermore the corresponding objective function is non-convex with respect to the change point. The technique developed in this paper is applicable to a general M-estimation framework with a change point, which may be of independent interest. The proposed methods are then illustrated via Monte Carlo experiments and an application to tipping in the dynamics of racial segregation.

stat.ME↗

arXiv2016

Doubly Robust Uniform Confidence Band for the Conditional Average Treatment Effect Function

In this paper, we propose a doubly robust method to present the heterogeneity of the average treatment effect with respect to observed covariates of interest. We consider a situation where a large number of covariates are needed for identifying the average treatment effect but the covariates of interest for analyzing heterogeneity are of much lower dimension. Our proposed estimator is doubly robust and avoids the curse of dimensionality. We propose a uniform confidence band that is easy to compute, and we illustrate its usefulness via Monte Carlo experiments and an application to the effects of smoking on birth weights.

stat.ME↗

arXiv2016

Optimal Data Collection for Randomized Control Trials

In a randomized control trial, the precision of an average treatment effect estimator can be improved either by collecting data on additional individuals, or by collecting additional covariates that predict the outcome variable. We propose the use of pre-experimental data such as a census, or a household survey, to inform the choice of both the sample size and the covariates to be collected. Our procedure seeks to minimize the resulting average treatment effect estimator's mean squared error, subject to the researcher's budget constraint. We rely on a modification of an orthogonal greedy algorithm that is conceptually simple and easy to implement in the presence of a large number of potential covariates, and does not require any tuning parameters. In two empirical applications, we show that our procedure can lead to substantial gains of up to 58%, measured either in terms of reductions in data collection costs or in terms of improvements in the precision of the treatment effect estimator.

stat.ME↗

arXiv2015

Uniform Asymptotics for Nonparametric Quantile Regression with an Application to Testing Monotonicity

In this paper, we establish a uniform error rate of a Bahadur representation for local polynomial estimators of quantile regression functions. The error rate is uniform over a range of quantiles, a range of evaluation points in the regressors, and over a wide class of probabilities for observed random variables. Most of the existing results on Bahadur representations for local polynomial quantile regression estimators apply to the fixed data generating process. In the context of testing monotonicity where the null hypothesis is of a complex composite hypothesis, it is particularly relevant to establish Bahadur expansions that hold uniformly over a large class of data generating processes. In addition, we establish the same error rate for bootstrap local polynomial estimators which can be useful for various bootstrap inference. As an illustration, we apply to testing monotonicity of quantile regression and present Monte Carlo experiments based on this example.

math.ST↗

arXiv2015

Testing for a General Class of Functional Inequalities

In this paper, we propose a general method for testing inequality restrictions on nonparametric functions. Our framework includes many nonparametric testing problems in a unified framework, with a number of possible applications in auction models, game theoretic models, wage inequality, and revealed preferences. Our test involves a one-sided version of $L_{p}$ functionals of kernel-type estimators $(1\leq p <\infty )$ and is easy to implement in general, mainly due to its recourse to the bootstrap method. The bootstrap procedure is based on nonparametric bootstrap applied to kernel-based test statistics, with estimated "contact sets." We provide regularity conditions under which the bootstrap test is asymptotically valid uniformly over a large class of distributions, including the cases that the limiting distribution of the test statistic is degenerate. Our bootstrap test is shown to exhibit good power properties in Monte Carlo experiments, and we provide a general form of the local power function. As an illustration, we consider testing implications from auction theory, provide primitive conditions for our test, and demonstrate its usefulness by applying our test to real data. We supplement this example with the second empirical illustration in the context of wage inequality.

math.ST↗

arXiv2014

Structural Change in Sparsity

In the high-dimensional sparse modeling literature, it has been crucially assumed that the sparsity structure of the model is homogeneous over the entire population. That is, the identities of important regressors are invariant across the population and across the individuals in the collected sample. In practice, however, the sparsity structure may not always be invariant in the population, due to heterogeneity across different sub-populations. We consider a general, possibly non-smooth M-estimation framework, allowing a possible structural change regarding the identities of important regressors in the population. Our penalized M-estimator not only selects covariates but also discriminates between a model with homogeneous sparsity and a model with a structural change in sparsity. As a result, it is not necessary to know or pretest whether the structural change is present, or where it occurs. We derive asymptotic bounds on the estimation loss of the penalized M-estimators, and achieve the oracle properties. We also show that when there is a structural change, the estimator of the threshold parameter is super-consistent. If the signal is relatively strong, the rates of convergence can be further improved and asymptotic distributional properties of the estimators including the threshold estimator can be established using an adaptive penalization. The proposed methods are then applied to quantile regression and logistic regression models and are illustrated via Monte Carlo experiments.

stat.ME↗

arXiv2014

The Lasso for High-Dimensional Regression with a Possible Change-Point

We consider a high-dimensional regression model with a possible change-point due to a covariate threshold and develop the Lasso estimator of regression coefficients as well as the threshold parameter. Our Lasso estimator not only selects covariates but also selects a model between linear and threshold regression models. Under a sparsity assumption, we derive non-asymptotic oracle inequalities for both the prediction risk and the $\ell_1$ estimation loss for regression coefficients. Since the Lasso estimator selects variables simultaneously, we show that oracle inequalities can be established without pretesting the existence of the threshold effect. Furthermore, we establish conditions under which the estimation error of the unknown threshold parameter can be bounded by a nearly $n^{-1}$ factor even when the number of regressors can be much larger than the sample size ($n$). We illustrate the usefulness of our proposed estimation method via Monte Carlo simulations and an application to real data.

math.ST↗

arXiv2013

Maximum Score Estimation of Preference Parameters for a Binary Choice Model under Uncertainty

This paper develops maximum score estimation of preference parameters in the binary choice model under uncertainty in which the decision rule is affected by conditional expectations. The preference parameters are estimated in two stages: we estimate conditional expectations nonparametrically in the first stage and then the preference parameters in the second stage based on Manski (1975, 1985)'s maximum score estimator using the choice data and first stage estimates. The paper establishes consistency and derives rate of convergence of the two-stage maximum score estimator. Moreover, the paper also provides sufficient conditions under which the two-stage estimator is asymptotically equivalent in distribution to the corresponding single-stage estimator that assumes the first stage input is known. These results are of independent interest for maximum score estimation with nonparametrically generated regressors. The paper also presents some Monte Carlo simulation results for finite-sample behavior of the two-stage estimator.

stat.ME↗

arXiv2013

Local Identification of Nonparametric and Semiparametric Models

In parametric, nonlinear structural models a classical sufficient condition for local identification, like Fisher (1966) and Rothenberg (1971), is that the vector of moment conditions is differentiable at the true parameter with full rank derivative matrix. We derive an analogous result for the nonparametric, nonlinear structural models, establishing conditions under which an infinite-dimensional analog of the full rank condition is sufficient for local identification. Importantly, we show that additional conditions are often needed in nonlinear, nonparametric models to avoid nonlinearities overwhelming linear effects. We give restrictions on a neighborhood of the true value that are sufficient for local identification. We apply these results to obtain new, primitive identification conditions in several important models, including nonseparable quantile instrumental variable (IV) models, single-index IV models, and semiparametric consumption-based asset pricing models.

math.ST↗

arXiv2013

Intersection Bounds: Estimation and Inference

We develop a practical and novel method for inference on intersection bounds, namely bounds defined by either the infimum or supremum of a parametric or nonparametric function, or equivalently, the value of a linear programming problem with a potentially infinite constraint set. We show that many bounds characterizations in econometrics, for instance bounds on parameters under conditional moment inequalities, can be formulated as intersection bounds. Our approach is especially convenient for models comprised of a continuum of inequalities that are separable in parameters, and also applies to models with inequalities that are non-separable in parameters. Since analog estimators for intersection bounds can be severely biased in finite samples, routinely underestimating the size of the identified set, we also offer a median-bias-corrected estimator of such bounds as a by-product of our inferential procedures. We develop theory for large sample inference based on the strong approximation of a sequence of series or kernel-based empirical processes by a sequence of "penultimate" Gaussian processes. These penultimate processes are generally not weakly convergent, and thus non-Donsker. Our theoretical results establish that we can nonetheless perform asymptotically valid inference based on these processes. Our construction also provides new adaptive inequality/moment selection methods. We provide conditions for the use of nonparametric kernel and series estimators, including a novel result that establishes strong approximation for any general series estimator admitting linearization, which may be of independent interest.

math.ST↗

arXiv2012

Testing Functional Inequalities

This paper develops tests for inequality constraints of nonparametric regression functions. The test statistics involve a one-sided version of $L_p$-type functionals of kernel estimators $(1 \leq p < \infty)$. Drawing on the approach of Poissonization, this paper establishes that the tests are asymptotically distribution free, admitting asymptotic normal approximation. In particular, the tests using the standard normal critical values have asymptotically correct size and are consistent against general fixed alternatives. Furthermore, we establish conditions under which the tests have nontrivial local power against Pitman local alternatives. Some results from Monte Carlo simulations are presented.

math.ST↗