Atreya Vedantam | Searcharxiv

arXiv2026

Characterization of Blind Code Rate Recovery in Linear Block Codes

Forward Error Correction (FEC) is used ubiquitously in the communication pipeline. We explore noncooperative decoding where we aim to recover the code rate of a linear block code. We present a metric to characterize the quality of the code rate recovery which uses any rank based estimation technique. We derive a closed form expression for this metric in terms of the algorithmic and the environmental parameters and assert that it should be low for good recovery. We use this metric to derive an expression for a better code rate estimate in high noise conditions and compare it with existing estimates. Finally we validate the derived expression for the metric and the improved performance in the code rate estimate by simulating the recovery of a Low Density Parity Check (LDPC) code. This also enables us to derive the optimal algorithmic parameters for recovery.

cs.IT↗

arXiv2026

Grokking in Linear Models for Logistic Regression

Grokking, the phenomenon of delayed generalization, is often attributed to the depth and compositional structure of deep neural networks. We study grokking in one of the simplest possible settings: the learning of a linear model with logistic loss for binary classification on data that are linearly (and max margin) separable about the origin. We investigate three testing regimes: (1) test data drawn from the same distribution as the training data, in which case grokking is not observed; (2) test data concentrated around the margin, in which case grokking is observed; and (3) adversarial test data generated via projected gradient descent (PGD) attacks, in which case grokking is also observed. We theoretically show that the implicit bias of gradient descent induces a three-phase learning process-population-dominated, support-vector-dominated unlearning, and support-vector-dominated generalization-during which delayed generalization can arise. Our analysis further relates the emergence of grokking to asymmetries in the data, both in the number of examples per class and in the distribution of support vectors across classes, and yields a characterization of the grokking time. We experimentally validate our theory by planting different distributions of population points and support vectors, and by analyzing accuracy curves and hyperplane dynamics. Overall, our results demonstrate that grokking does not require depth or representation learning, and can emerge even in linear models through the dynamics of the bias term.

cs.LG↗