Searcharxiv⌕ Search

arXiv subjects

Soroor Ghandali

Publications and source records attributed to Soroor Ghandali.

2 recordsLinked to original sources

Sensitivity-Gated Accumulator Caching for On-Device Learning of Low-Bit Hand Networks

Small hand-tracking networks are deployed as 4- and 8-bit codes, and they should keep adapting to a user's camera after deployment, on the device and within roughly the memory of the model itself. Quantization-aware training cannot do this cheaply, because it keeps a 32-bit master copy of every weight together with its optimizer state, about 96 bits per weight. Training the codes directly with stochastic rounding needs no copy but injects rounding noise at every step, and the methods that remove this noise keep a residual buffer for every weight. We introduce sensitivity-gated accumulator caching (SGAC), which keeps residual state for only a small and changing set of weight groups. A set-associative cache of 8-bit accumulators collects the sub-step updates of the groups with the largest Fisher-weighted squared code step and writes them into the codes as whole steps; an evicted line is flushed by stochastic rounding, and a group without a line does not move until it is admitted. We prove that the write-back is unbiased and show, under a local quadratic model, that rounding a sub-step update costs loss of the same order as the update's own gain, so that rounding pays only where the expected gradient is large relative to the curvature. On a 383k-parameter hand-keypoint network, SGAC trains with 8.1 bits of state per weight. Scored on the 4/8-bit network that the device runs after adaptation, it has the highest mean accuracy of nine deployed low-bit networks on two simulated cameras. On twelve ImageNet-C corruptions it has the highest average of corrupted and clean accuracy of seven deployed methods, ahead of the PEQA-style, ECO-style and direct low-bit baselines with confidence intervals that exclude zero, and it matches an 8-bit accumulator kept for every weight with 56% of that state.

cs.CV↗

Slow Beats Fast at the Kesten-Stigum Threshold: Minimax, Fisher-Information and Belief-Propagation Characterizations of the Information-Computation Gap in Sparse Stochastic Block Models

We study community recovery in the sparse symmetric stochastic block model with $q$ communities, average degree $d$ and signal strength $λ$ through statistical decision theory and Fisher information, and obtain three characterizations of the Kesten-Stigum threshold $dλ^2=1$ and of the information-computation gap below it. First, on each community-size profile the minimax risk of any class of rules closed under averaging and vertex relabeling equals its Bayes risk under the uniform prior; the posterior mean is the unique Bayes rule and is admissible, and the Bayes risk of degree-$D$ polynomial rules is the trivial risk times $1-\mathrm{Corr}_D^2$. Combined with known low-degree and information-theoretic results, this gives the gap as a worst-case statement: for $q\ge 5$ there is a window below the threshold in which no low-degree rule beats the trivial risk asymptotically, while an exponential-time rule does on a set of labelings of probability $1-o(1)$. Second, the Fisher information about $λ$ carried by cycle counts is a series with terms of order $k(dλ^2)^k$, convergent exactly when $dλ^2<1$; below the threshold the relative error of every unbiased cycle-based estimator of $λ^k$ stays above an explicit constant, and every cycle-count test has success probability bounded below one. Third, the derivative of belief propagation at its uninformative fixed point multiplies a random perturbation by $|λ|\sqrt{d}$ per iteration, and one EM step taken there leaves $λ$ unchanged. A signal-to-noise computation recovers the condition $dλ^{1/χ}>1$ of Chin et al. for $q=n^χ$ communities and identifies personalized PageRank as a walk count with suboptimal weights. Experiments on networks with up to $3\times 10^5$ vertices confirm the threshold for $q=2$, the hard window for $q=5$, and the many-community scaling.

cs.IT↗