arXiv subjects

Nikita Puchkin

Publications and source records attributed to Nikita Puchkin.

23 records · Page 2Linked to original sources

arXiv2022

Inference via Randomized Test Statistics

We show that external randomization may enforce the convergence of test statistics to their limiting distributions in particular cases. This results in a sharper inference. Our approach is based on a central limit theorem for weighted sums. We apply our method to a family of rank-based test statistics and a family of phi-divergence test statistics and prove that, with overwhelming probability with respect to the external randomization, the randomized statistics converge at the rate $O(1/n)$ (up to some logarithmic factors) to the limiting chi-square distribution in Kolmogorov metric.

math.ST↗

arXiv2022

Exponential Savings in Agnostic Active Learning through Abstention

We show that in pool-based active classification without assumptions on the underlying distribution, if the learner is given the power to abstain from some predictions by paying the price marginally smaller than the average loss $1/2$ of a random guess, exponential savings in the number of label requests are possible whenever they are possible in the corresponding realizable problem. We extend this result to provide a necessary and sufficient condition for exponential savings in pool-based active classification under the model misspecification.

cs.LG↗

arXiv2022

Structure-adaptive manifold estimation

We consider a problem of manifold estimation from noisy observations. Many manifold learning procedures locally approximate a manifold by a weighted average over a small neighborhood. However, in the presence of large noise, the assigned weights become so corrupted that the averaged estimate shows very poor performance. We suggest a structure-adaptive procedure, which simultaneously reconstructs a smooth manifold and estimates projections of the point cloud onto this manifold. The proposed approach iteratively refines the weights on each step, using the structural information obtained at previous steps. After several iterations, we obtain nearly "oracle" weights, so that the final estimates are nearly efficient even in the presence of relatively large noise. In our theoretical study, we establish tight lower and upper bounds proving asymptotic optimality of the method for manifold estimation under the Hausdorff loss, provided that the noise degrades to zero fast enough.

math.ST↗

arXiv2020

Manifold-based time series forecasting

Prediction for high dimensional time series is a challenging task due to the curse of dimensionality problem. Classical parametric models like ARIMA or VAR require strong modeling assumptions and time stationarity and are often overparametrized. This paper offers a new flexible approach using recent ideas of manifold learning. The considered model includes linear models such as the central subspace model and ARIMA as particular cases. The proposed procedure combines manifold denoising techniques with a simple nonparametric prediction by local averaging. The resulting procedure demonstrates a very reasonable performance for real-life econometric time series. We also provide a theoretical justification of the manifold estimation procedure.

math.ST↗

arXiv2019

An adaptive multiclass nearest neighbor classifier

We consider a problem of multiclass classification, where the training sample $S_n = \{(X_i, Y_i)\}_{i=1}^n$ is generated from the model $\mathbb P(Y = m | X = x) = η_m(x)$, $1 \leq m \leq M$, and $η_1(x), \dots, η_M(x)$ are unknown $α$-Holder continuous functions.Given a test point $X$, our goal is to predict its label. A widely used $\mathsf k$-nearest-neighbors classifier constructs estimates of $η_1(X), \dots, η_M(X)$ and uses a plug-in rule for the prediction. However, it requires a proper choice of the smoothing parameter $\mathsf k$, which may become tricky in some situations. In our solution, we fix several integers $n_1, \dots, n_K$, compute corresponding $n_k$-nearest-neighbor estimates for each $m$ and each $n_k$ and apply an aggregation procedure. We study an algorithm, which constructs a convex combination of these estimates such that the aggregated estimate behaves approximately as well as an oracle choice. We also provide a non-asymptotic analysis of the procedure, prove its adaptation to the unknown smoothness parameter $α$ and to the margin and establish rates of convergence under mild assumptions.

stat.ML↗