arXiv subjects

Keegan Kang

Publications and source records attributed to Keegan Kang.

4 recordsLinked to original sources

arXiv2026

Improving TensorSketch Using Complex Random Variables

\texttt{TensorSketch} by~\cite{pham2013fast,kar2012random} provides efficient sketching algorithms for high-dimensional polynomial kernels $\vec{x}^{\otimes p} \in \R^{d^p}$. \cite{kar2012random} uses dense Johnson-Lindenstrauss (JL)-type projections with computational cost $O(pDd)$, where $D$ denotes the sketch dimension, whereas~\cite{pham2013fast} extends the sparse \texttt{CountSketch}~\citep{count_sketch} algorithm, yielding a faster algorithm for high-dimensional sparse inputs with running time $O\big(p(\nnz{\vec{x}} + D \log D)\big)$. However, the variance of both estimators grows exponentially with the polynomial degree $p$, scaling as $3^{p}/D$. Recent work by~\cite{pmlr-v206-wacker23a} showed that using complex-valued distribution reduces this dependence to $2^{p}/D$ for the approach of~\cite{kar2012random}. However, their method relies on dense JL-type projections with computational cost $O(pDd)$ and does not extend to the algorithm of~\cite{pham2013fast}. In this work, we introduce a simple variant of \texttt{TensorSketch}~\citep{pham2013fast} that achieves the same variance bound as~\cite{pmlr-v206-wacker23a}, while retaining its advantage of the input-sparsity running time. We validate our results with supporting experiments on synthetic and real-world datasets.

cs.DS↗

arXiv2026

It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms

Maximum likelihood estimators (MLE) and control variate estimators (CVE) have been used in conjunction with known information across sketching algorithms and applications in machine learning. We prove that under certain conditions in an exponential family, an optimal CVE will achieve the same asymptotic variance as the MLE, giving a fixed point algorithm for the MLE. Experiments show the fixed point algorithm is faster and numerically stable compared to other root finding algorithms for the MLE for the bivariate Normal distribution, and we expect this to hold across distributions satisfying these conditions. We show how this algorithm leads to reproducibility for algorithms using MLE / CVE, and demonstrate how the algorithm leads to finding the MLE when the CV weights are known.

stat.ML↗

arXiv2025

Stochastic Trace and Diagonal Estimator for Tensors

We consider the problem of estimating the trace and diagonal entries of an N-order tensor (where $N \geq 2$) under the framework where the tensor can only be accessed through tensor-vector multiplication. The aim is to estimate the tensor's diagonal entries and trace by minimizing the number of tensor-vector queries. The seminal work of Hutchinson and its extended version due to Bekas et al. give unbiased estimates of the trace and diagonal elements of a given matrix, respectively, using matrix-vector queries. However, to the best of our knowledge, no analogous results are known for estimating the trace and diagonal entries of higher-order tensors using tensor-vector queries. This paper addresses this gap and presents unbiased estimators for the trace and diagonal entries of a tensor under this model. Our proposed methods can be seen as generalizations of Hutchinson's and Bekas et al.'s estimators and reduce to their estimators when N = 2. We provide a rigorous theoretical analysis of our proposals and complement it with supporting simulations.

math.NA↗

arXiv2015

Feature Representation in Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are powerful models that achieve impressive results for image classification. In addition, pre-trained CNNs are also useful for other computer vision tasks as generic feature extractors. This paper aims to gain insight into the feature aspect of CNN and demonstrate other uses of CNN features. Our results show that CNN feature maps can be used with Random Forests and SVM to yield classification results that outperforms the original CNN. A CNN that is less than optimal (e.g. not fully trained or overfitting) can also extract features for Random Forest/SVM that yield competitive classification accuracy. In contrast to the literature which uses the top-layer activations as feature representation of images for other tasks, using lower-layer features can yield better results for classification.

cs.CV↗