Searcharxiv⌕ Search

arXiv subjects

Zhiyi Chi

Publications and source records attributed to Zhiyi Chi.

32 records · Page 2Linked to original sources
arXiv2010

Stochastic Lipschitz continuity for high dimensional Lasso with multiple linear covariate structures or hidden linear covariates

Two extensions of generalized linear models are considered. In the first one, response variables depend on multiple linear combinations of covariates. In the second one, only response variables are observed while the linear covariates are missing. We derive stochastic Lipschitz continuity results for the loss functions involved in the regression problems and apply them to get bounds on estimation error for Lasso. Multivariate comparison results on Rademacher complexity are obtained as tools to establish the stochastic Lipschitz continuity results.

math.ST↗
arXiv2010

A local stochastic Lipschitz condition with application to Lasso for high dimensional generalized linear models

For regularized estimation, the upper tail behavior of the random Lipschitz coefficient associated with empirical loss functions is known to play an important role in the error bound of Lasso for high dimensional generalized linear models. The upper tail behavior is known for linear models but much less so for nonlinear models. We establish exponential type inequalities for the upper tail of the coefficient and illustrate an application of the results to Lasso likelihood estimation for high dimensional generalized linear models.

math.ST↗
arXiv2009

$L_0$ regularized estimation for nonlinear models that have sparse underlying linear structures

We study the estimation of $β$ for the nonlinear model $y = f(X\sp{\top}β) + ε$ when $f$ is a nonlinear transformation that is known, $β$ has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters ($n\ll p$). We show that in order to bound the $L_2$ error of the $L_0$ regularized estimator $\hatβ$, i.e., $\|\hatβ- β\|_2$, it is sufficient to establish two conditions. Based on this, we obtain bounds of the $L_2$ error for (1) $L_0$ regularized maximum likelihood estimation (MLE) for exponential linear models and (2) $L_0$ regularized least square (LS) regression for the more general case where $f$ is analytic. For the analytic case, we rely on power series expansion of $f$, which requires taking into account the singularities of $f$.

math.ST↗
arXiv2009

Data volume and power of multiple tests with small sample size per null

In multiple hypothesis testing, the volume of data, defined as the number of replications per null times the total number of nulls, usually defines the amount of resource required. On the other hand, power is an important measure of performance for multiple testing. Due to practical constraints, the number of replications per null may not be large enough in terms of the difference between false and true nulls. For the case where the population fraction of false nulls is constant, we show that, as the difference between false and true nulls becomes increasingly subtle while the number of replications per null cannot increase fast enough, (1) in order to have enough chance that the data to be collected will yield some trustworthy findings, as measured by a conditional version of the positive false discovery rate (pFDR), the volume of data has to grow at a rate much faster than in the case where the number of replications per null can be large enough, and (2) in order to control the pFDR asymptotically, power has to decay to 0 in a rate highly sensitive to rejection criterion and there is no asymptotically most powerful procedures among those that control the pFDR asymptotically at the same level.

math.ST↗
arXiv2008

FDR control for multiple hypothesis testing on composite nulls

Multiple hypothesis testing often involves composite nulls, i.e., nulls that are associated with two or more distributions. In many cases, it is reasonable to assume that there is a prior distribution on the distributions despite it is unknown. When the number of distributions under true nulls is finite, we show that under the above assumption, the false discover rate (FDR) can be controlled using $p$-values computed under constraints imposed by the empirical distribution of the observations. Comparing to FDR control using $p$-values defined as maximum significance level over all null distributions, the proposed FDR control can have substantially more power.

math.ST↗
arXiv2008

False discovery rate control with multivariate $p$-values

Multivariate statistics are often available as well as necessary in hypothesis tests. We study how to use such statistics to control not only false discovery rate (FDR) but also positive FDR (pFDR) with good power. We show that FDR can be controlled through nested regions of multivariate $p$-values of test statistics. If the distributions of the test statistics are known, then the regions can be constructed explicitly to achieve FDR control with maximum power among procedures satisfying certain conditions. On the other hand, our focus is where the distributions are only partially known. Under certain conditions, a type of nested regions are proposed and shown to attain (p)FDR control with asymptotically maximum power as the pFDR control level approaches its attainable limit. The procedure based on the nested regions is compared with those based on other nested regions that are easier to construct as well as those based on more straightforward combinations of the test statistics.

math.ST↗
arXiv2008

On the asymptotic of likelihood ratios for self-normalized large deviations

Motivated by multiple statistical hypothesis testing, we obtain the limit of likelihood ratio of large deviations for self-normalized random variables, specifically, the ratio of $P(\sqrt{n}(\bar X +d/n) \ge x_n V)$ to $P(\sqrt{n}\bar X \ge x_n V)$, as $n\toi$, where $\bar X$ and $V$ are the sample mean and standard deviation of iid $X_1, ..., X_n$, respectively, $d>0$ is a constant and $x_n \toi$. We show that the limit can have a simple form $e^{d/z_0}$, where $z_0$ is the unique maximizer of $z f(x)$ with $f$ the density of $X_i$. The result is applied to derive the minimum sample size per test in order to control the error rate of multiple testing at a target level, when real signals are different from noise signals only by a small shift.

math.ST↗
arXiv2007

On the performance of FDR control: Constraints and a partial solution

The False Discovery Rate (FDR) paradigm aims to attain certain control on Type I errors with relatively high power for multiple hypothesis testing. The Benjamini--Hochberg (BH) procedure is a well-known FDR controlling procedure. Under a random effects model, we show that, in general, unlike the FDR, the positive FDR (pFDR) of the BH procedure cannot be controlled at an arbitrarily low level due to the limited evidence provided by the observations to separate false and true nulls. This results in a criticality phenomenon, which is characterized by a transition of the procedure's power from being positive to asymptotically 0 without any reduction in the pFDR, once the target FDR control level is below a positive critical value. To address the constraints on the power and pFDR control imposed by the criticality phenomenon, we propose a procedure which applies BH-type procedures at multiple locations in the domain of $p$-values. Both analysis and simulations show that the proposed procedure can attain substantially improved power and pFDR control.

math.ST↗
arXiv2007

Uniform convergence of exact large deviations for renewal reward processes

Let (X_n,Y_n) be i.i.d. random vectors. Let W(x) be the partial sum of Y_n just before that of X_n exceeds x>0. Motivated by stochastic models for neural activity, uniform convergence of the form $\sup_{c\in I}|a(c,x)\operatorname {Pr}\{W(x)\gecx\}-1|=o(1)$, $x\to\infty$, is established for probabilities of large deviations, with a(c,x) a deterministic function and I an open interval. To obtain this uniform exact large deviations principle (LDP), we first establish the exponentially fast uniform convergence of a family of renewal measures and then apply it to appropriately tilted distributions of X_n and the moment generating function of W(x). The uniform exact LDP is obtained for cases where X_n has a subcomponent with a smooth density and Y_n is not a linear transform of X_n. An extension is also made to the partial sum at the first exceedance time.

math.PR↗
arXiv2007

Sample size and positive false discovery rate control for multiple testing

The positive false discovery rate (pFDR) is a useful overall measure of errors for multiple hypothesis testing, especially when the underlying goal is to attain one or more discoveries. Control of pFDR critically depends on how much evidence is available from data to distinguish between false and true nulls. Oftentimes, as many aspects of the data distributions are unknown, one may not be able to obtain strong enough evidence from the data for pFDR control. This raises the question as to how much data are needed to attain a target pFDR level. We study the asymptotics of the minimum number of observations per null for the pFDR control associated with multiple Studentized tests and $F$ tests, especially when the differences between false nulls and true nulls are small. For Studentized tests, we consider tests on shifts or other parameters associated with normal and general distributions. For $F$ tests, we also take into account the effect of the number of covariates in linear regression. The results show that in determining the minimum sample size per null for pFDR control, higher order statistical properties of data are important, and the number of covariates is important in tests to detect regression effects.

math.ST↗
arXiv2005

Limit laws of estimators for critical multi-type Galton-Watson processes

We consider the asymptotics of various estimators based on a large sample of branching trees from a critical multi-type Galton-Watson process, as the sample size increases to infinity. The asymptotics of additive functions of trees, such as sizes of trees and frequencies of types within trees, a higher-order asymptotic of the ``relative frequency'' estimator of the left eigenvector of the mean matrix, a higher-order joint asymptotic of the maximum likelihood estimators of the offspring probabilities and the consistency of an estimator of the right eigenvector of the mean matrix, are established.

math.PR↗
arXiv2005

Large deviations for template matching between point processes

We study the asymptotics related to the following matching criteria for two independent realizations of point processes X\sim X and Y\sim Y. Given l>0, X\cap [0,l) serves as a template. For each t>0, the matching score between the template and Y\cap [t,t+l) is a weighted sum of the Euclidean distances from y-t to the template over all y\in Y\cap [t,t+l). The template matching criteria are used in neuroscience to detect neural activity with certain patterns. We first consider W_l(θ), the waiting time until the matching score is above a given threshold θ. We show that whether the score is scalar- or vector-valued, (1/l)\log W_l(θ) converges almost surely to a constant whose explicit form is available, when X is a stationary ergodic process and Y is a homogeneous Poisson point process. Second, as l\to\infty, a strong approximation for -\log [\Pr{W_l(θ)=0}] by its rate function is established, and in the case where X is sufficiently mixing, the rates, after being centered and normalized by \sqrtl, satisfy a central limit theorem and almost sure invariance principle. The explicit form of the variance of the normal distribution is given for the case where X is a homogeneous Poisson process as well.

math.PR↗
arXiv2000

Estimators for Stochastic ``Unification-Based'' Grammars

Log-linear models provide a statistically sound framework for Stochastic ``Unification-Based'' Grammars (SUBGs) and stochastic versions of other kinds of grammars. We describe two computationally-tractable ways of estimating the parameters of such grammars from a training corpus of syntactic analyses, and apply these to estimate a stochastic version of Lexical-Functional Grammar.

cs.CL↗