arXiv subjects

Ya'acov Ritov

Publications and source records attributed to Ya'acov Ritov.

At least 37 records · Page 2Linked to original sources

arXiv2021

On robust learning in the canonical change point problem under heavy tailed errors in finite and growing dimensions

This paper presents a number of new findings about the canonical change point estimation problem. The first part studies the estimation of a change point on the real line in a simple stump model using the robust Huber estimating function which interpolates between the $\ell_1$ (absolute deviation) and $\ell_2$ (least squares) based criteria. While the $\ell_2$ criterion has been studied extensively, its robust counterparts and in particular, the $\ell_1$ minimization problem have not. We derive the limit distribution of the estimated change point under the Huber estimating function and compare it to that under the $\ell_2$ criterion. Theoretical and empirical studies indicate that it is more profitable to use the Huber estimating function (and in particular, the $\ell_1$ criterion) under heavy tailed errors as it leads to smaller asymptotic confidence intervals at the usual levels compared to the $\ell_2$ criterion. We also compare the $\ell_1$ and $\ell_2$ approaches in a parallel setting, where one has $m$ independent single change point problems and the goal is to control the maximal deviation of the estimated change points from the true values, and establish rigorously that the $\ell_1$ estimation criterion provides a superior rate of convergence to the $\ell_2$, and that this relative advantage is driven by the heaviness of the tail of the error distribution. Finally, we derive minimax optimal rates for the change plane estimation problem in growing dimensions and demonstrate that Huber estimation attains the optimal rate while the $\ell_2$ scheme produces a rate sub-optimal estimator for heavy tailed errors. In the process of deriving our results, we establish a number of properties about the minimizers of compound Binomial and compound Poisson processes which are of independent interest.

math.ST↗

arXiv2021

Inference In High-dimensional Single-Index Models Under Symmetric Designs

The problem of statistical inference for regression coefficients in a high-dimensional single-index model is considered. Under elliptical symmetry, the single index model can be reformulated as a proxy linear model whose regression parameter is identifiable. We construct estimates of the regression coefficients of interest that are similar to the debiased lasso estimates in the standard linear model and exhibit similar properties: root-n-consistency and asymptotic normality. The procedure completely bypasses the estimation of the unknown link function, which can be extremely challenging depending on the underlying structure of the problem. Furthermore, under Gaussianity, we propose more efficient estimates of the coefficients by expanding the link function in the Hermite polynomial basis. Finally, we illustrate our approach via carefully designed simulation experiments.

math.ST↗

arXiv2020

Scalable Monte Carlo Inference and Rescaled Local Asymptotic Normality

In this paper, we generalize the property of local asymptotic normality (LAN) to an enlarged neighborhood, under the name of rescaled local asymptotic normality (RLAN). We obtain sufficient conditions for a regular parametric model to satisfy RLAN. We show that RLAN supports the construction of a statistically efficient estimator which maximizes a cubic approximation to the log-likelihood on this enlarged neighborhood. In the context of Monte Carlo inference, we find that this maximum cubic likelihood estimator can maintain its statistical efficiency in the presence of asymptotically increasing Monte Carlo error in likelihood evaluation.

math.ST↗

arXiv2020

The Hot Hand and Its Effect on the NBA

This paper aims to revisit and expand upon previous work on the "hot hand" phenomenon in basketball, specifically in the NBA. Using larger, modern data sets, we test streakiness of shooting patterns and the presence of hot hand behavior in free throw shooting, while going further by examining league-wide hot hand trends and the changes in individual player behavior. Additionally, we perform simulations in order to assess their power. While we find no evidence of the hot hand in game-play and only weak evidence in free throw trials, we find that some NBA players exhibit behavioral changes based on the outcome of their previous shot.

stat.AP↗

arXiv2020

Design of $c$-Optimal Experiments for High dimensional Linear Models

We study random designs that minimize the asymptotic variance of a de-biased lasso estimator when a large pool of unlabeled data is available but measuring the corresponding responses is costly. The optimal sampling distribution arises as the solution of a semidefinite program. The improvements in efficiency that result from these optimal designs are demonstrated via simulation experiments.

math.ST↗

arXiv2020

Optimal Linear Discriminators For The Discrete Choice Model In Growing Dimensions

Manski's celebrated maximum score estimator for the discrete choice model, which is an optimal linear discriminator, has been the focus of much investigation in both the econometrics and statistics literatures, but its behavior under growing dimension scenarios largely remains unknown. This paper addresses that gap. Two different cases are considered: $p$ grows with $n$ but at a slow rate, i.e. $p/n \rightarrow 0$; and $p \gg n$ (fast growth). In the binary response model, we recast Manski's score estimation as empirical risk minimization for a classification problem, and derive the $\ell_2$ rate of convergence of the score estimator under a \emph{transition condition} in terms of our margin parameter that calibrates the level of difficulty of the estimation problem. We also establish upper and lower bounds for the minimax $\ell_2$ error in the binary choice model that differ by a logarithmic factor, and construct a minimax-optimal estimator in the slow growth regime. Some extensions to the general case -- the multinomial response model -- are also considered. Last but not least, we use a variety of learning algorithms to compute the maximum score estimator in growing dimensions.

math.ST↗

arXiv2020

Markovian And Non-Markovian Processes with Active Decision Making Strategies For Addressing The COVID-19 Pandemic

We study and predict the evolution of Covid-19 in six US states from the period May 1 through August 31 using a discrete compartment-based model and prescribe active intervention policies, like lockdowns, on the basis of minimizing a loss function, within the broad framework of partially observed Markov decision processes. For each state, Covid-19 data for 40 days (starting from May 1 for two northern states and June 1 for four southern states) are analyzed to estimate the transition probabilities between compartments and other parameters associated with the evolution of the epidemic. These quantities are then used to predict the course of the epidemic in the given state for the next 50 days (test period) under various policy allocations, leading to different values of the loss function over the training horizon. The optimal policy allocation is the one corresponding to the smallest loss. Our analysis shows that none of the six states need lockdowns over the test period, though the no lockdown prescription is to be interpreted with caution: responsible mask use and social distancing of course need to be continued. The caveats involved in modeling epidemic propagation of this sort are discussed at length. A sketch of a non-Markovian formulation of Covid-19 propagation (and more general epidemic propagation) is presented as an attractive avenue for future research in this area.

stat.AP↗

arXiv2020

Inference Without Compatibility

We consider hypotheses testing problems for three parameters in high-dimensional linear models with minimal sparsity assumptions of their type but without any compatibility conditions. Under this framework, we construct the first $\sqrt{n}$-consistent estimators for low-dimensional coefficients, the signal strength, and the noise level. We support our results using numerical simulations and provide comparisons with other estimators.

math.ST↗

arXiv2019

Generalized Maximum Likelihood Estimators and their applications to stratified sampling and post-stratification with many unobserved strata

Consider the problem of estimating a weighted average of the means of $n$ strata, based on a random sample with realized $K_i$ observations from stratum $i, \; i=1,...,n$. This task is non-trivial in cases where for a significant portion of the strata the corresponding $K_i=0$. Such a situation may happen in post-stratification, when it is desired to have a very fine sftratification. A fine stratification could be desired in order that assumptions, or, approximations, like Missing At Random conditional on strata, will be appealing. A fine stratification could also be desired in observational studies, when it is desired to estimate average treatment effect, by averaging the effects in small and homogenous strata. Our approach is based on applying Generalized Maximum Likelihood Estimators (GMLE), and ideas that are related to Non-Parametric Empirical Bayes, in order to estimate the means of strata $i$ with corresponding $K_i=0$. There are no assumptions about a relation between the means of the unobserved strata (i.e., with $K_i=0$) and those of the observed strata. The performance of our approach is demonstrated both in simulations and on a real data set. Some consistency and asymptotic results are also presented. In addition, related basic results about GMLE estimation of the mean of mixtures of exponential families are provided.

math.ST↗

arXiv2019

Estimating the Random Effect in Big Data Mixed Models

We consider three problems in high-dimensional Gaussian linear mixed models. Without any assumptions on the design for the fixed effects, we construct an asymptotic $F$-statistic for testing whether a collection of random effects is zero, derive an asymptotic confidence interval for a single random effect at the parametric rate $\sqrt{n}$, and propose an empirical Bayes estimator for a part of the mean vector in ANOVA type models that performs asymptotically as well as the oracle Bayes estimator. We support our results with numerical simulations and provide comparisons with oracle estimators. The procedures developed are applied to the Trends in International Mathematics and Sciences Study (TIMSS) data.

math.ST↗

arXiv2018

Self-reporting and screening: Data with current-status and censored observations

We consider survival data that combine three types of observations: uncensored, right-censored, and left-censored. Such data arises from screening a medical condition, in situations where self-detection arises naturally. Our goal is to estimate the failure-time distribution, based on these three observation types. We propose a novel methodology for distribution estimation using both parametric and nonparametric techniques. We then evaluate the performance of these estimators via simulated data. Finally, as a case study, we estimate the patience of patients who arrive at an emergency department and wait for treatment. Three categories of patients are observed: those who leave the system and announce it, and thus their patience time is observed; those who get service and thus their patience time is right-censored by the waiting time; and those who leave the system without announcing it. For the third category, the patients' absence is revealed only when they are called to service, which is after they have already left; formally, their patience time is left-censored. Other applications of our proposed methodology are discussed.

math.ST↗

arXiv2017

Robust estimation of mixing measures in finite mixture models

In finite mixture models, apart from underlying mixing measure, true kernel density function of each subpopulation in the data is, in many scenarios, unknown. Perhaps the most popular approach is to choose some kernel functions that we empirically believe our data are generated from and use these kernels to fit our models. Nevertheless, as long as the chosen kernel and the true kernel are different, statistical inference of mixing measure under this setting will be highly unstable. To overcome this challenge, we propose flexible and efficient robust estimators of the mixing measure in these models, which are inspired by the idea of minimum Hellinger distance estimator, model selection criteria, and superefficiency phenomenon. We demonstrate that our estimators consistently recover the true number of components and achieve the optimal convergence rates of parameter estimation under both the well- and mis-specified kernel settings for any fixed bandwidth. These desirable asymptotic properties are illustrated via careful simulation studies with both synthetic and real data.

math.ST↗

arXiv2017

On conditional parity as a notion of non-discrimination in machine learning

We identify conditional parity as a general notion of non-discrimination in machine learning. In fact, several recently proposed notions of non-discrimination, including a few counterfactual notions, are instances of conditional parity. We show that conditional parity is amenable to statistical analysis by studying randomization as a general mechanism for achieving conditional parity and a kernel-based test of conditional parity.

stat.ML↗

arXiv2017

The utility of a Bayesian analysis of complex models and the study of archeological ${}^{14}$C data

The paper presents a critical introduction to the complex statistical models used in ${}^{14}$C dating. The emphasis is on the estimation of the transit time between a sequence of archeological layers. Although a frequentist estimation of the parameters is relatively simple, confidence intervals constructions are not standard as the models are not regular. I argue that that the Bayesian paradigm is a natural approach to these models. It is simple, and gives immediate solutions to credible sets, with natural interpretation and simple construction. Indeed it is the standard tool of ${}^{14}$C analysis. However and necessarily, the Bayesian approach is based on technical assumptions that may dominate the scientific conclusion in a hard to predict way. I exemplify the discussion in two ways. Firstly, I simulate toy models. Secondly, I analyze a particular data set from the Iron Age period in Tel Rehov. These data are important to the debate on the absolute time of the Iron Age I/IIA transition in the Levant, and in particular to the feasibility of the Bible story about the United Monarchy of David and Solomon. Our conclusion is that the data in question cannot resolve this debate.

stat.AP↗

arXiv2016

On Bayesian robust regression with diverging number of predictors

This paper concerns the robust regression model when the number of predictors and the number of observations grow in a similar rate. Theory for M-estimators in this regime has been recently developed by several authors [El Karoui et al., 2013, Bean et al., 2013, Donoho and Montanari, 2013]. Motivated by the inability of M-estimators to successfully estimate the Euclidean norm of the coefficient vector, we consider a Bayesian framework for this model. We suggest a two-component mixture of normals prior for the coefficients and develop a Gibbs sampler procedure for sampling from relevant posterior distributions, while utilizing a scale mixture of normal representation for the error distribution . Unlike M-estimators, the proposed Bayes estimator is consistent in the Euclidean norm sense. Simulation results demonstrate the superiority of the Bayes estimator over traditional estimation methods.

math.ST↗

arXiv2015

Identifying a minimal class of models for high-dimensional data

Model selection consistency in the high-dimensional regression setting can be achieved only if strong assumptions are fulfilled. We therefore suggest to pursue a different goal, which we call a minimal class of models. The minimal class of models includes models that are similar in their prediction accuracy but not necessarily in their elements. We suggest a random search algorithm to reveal candidate models. The algorithm implements simulated annealing while using a score for each predictor that we suggest to derive using a combination of the Lasso and the Elastic Net. The utility of using a minimal class of models is demonstrated in the analysis of two datasets.

stat.ME↗

arXiv2014

On asymptotically optimal confidence regions and tests for high-dimensional models

We propose a general method for constructing confidence intervals and statistical tests for single or low-dimensional components of a large parameter vector in a high-dimensional model. It can be easily adjusted for multiplicity taking dependence among tests into account. For linear models, our method is essentially the same as in Zhang and Zhang [J. R. Stat. Soc. Ser. B Stat. Methodol. 76 (2014) 217-242]: we analyze its asymptotic properties and establish its asymptotic optimality in terms of semiparametric efficiency. Our method naturally extends to generalized linear models with convex loss functions. We develop the corresponding theory which includes a careful analysis for Gaussian, sub-Gaussian and bounded correlated designs.

math.ST↗

arXiv2012

Around the goal: Examining the effect of the first goal on the second goal in soccer using survival analysis methods

In this paper we apply survival techniques to soccer data, treating a goal scoring as the event of interest. It specifically concerns the relationship between the time of the first goal in the game and the time of the second goal. In order to do so, the relevant survival analysis concepts are readjusted to fit the problem and a Cox model is developed for the hazard function. Attributes such as time dependent covariates and a frailty term are also being considered. We also use a reliable propensity score to summarize the pre-game covariates. The conclusions derived from the results are that a first goal occurrence could either expedite or impede the next goal scoring, depending on the time it was scored. Moreover, once a goal is scored, another goal scoring become more and more likely as the game progresses. Furthermore, the first goal effect is the same whether the goal was scored or conceded.

stat.AP↗