arXiv subjects

Vishwajith Ramesh

Publications and source records attributed to Vishwajith Ramesh.

4 recordsLinked to original sources

arXiv2026

Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate

When a user asks an assistant to forget a record, the test is whether the memory now matches the state it would hold if the record had never been stored. Independently encoded rows can be removed directly; a recurrent memory folds records into an evolving state. One hope is a receipt: save the difference the record made when it arrived, carry it forward through later updates, and subtract it, so that deletion costs one fixed-size edit no matter how long the conversation runs. We show that a transported receipt reaches exact omission if and only if the changes the record induces in later updates cancel out on net, and we measure whether they do on the released 48B Kimi Linear hybrid. They do not: after 4,096 further tokens the record still leaves an imprint of about 4.5% of the state norm that none of the tested receipt classes removes, recomputing half the suffix closes less than half the gap, and the per-token log a receipt needs costs more than a full checkpoint after 88 tokens. The same write-rule classification held on Mamba-2, Falcon-H1, and RWKV-7 with predictions recorded before the runs. Restoring a checkpoint from before the record and replaying the surviving suffix matches the never-stored state exactly on every array we check. In the hybrid suffix sweep, masking the record's attention rows brings sampled recovery close to the never-stored floor even though the recurrent imprint remains, and an auditor who rebuilds the reference can still detect it. Among the evaluated methods, checkpoint replay achieves exact omission, with work proportional to the replayed suffix.

cs.LG↗

arXiv2026

Can an AI Assistant Really Forget? Auditable Deletion from Addressable Memory

Certifying that a deletion did what it declared does not certify that the record left no trace: a small distance to the implementation's own reference does not imply a small distance to the state that never stored the record. This paper installs a deletion interface into a pretrained language model and measures both distances. We retrofit a support-vector memory gate into the global attention layers of a frozen Gemma 3 without changing a weight. Each stored record owns a set of rows, and deleting it removes those rows and re-solves only the storage problems they touched. At 4B the retrofit admits exactly the records the base model recalls, at a paired perplexity cost under $2\%$; the same recipe fails at 1B and 12B, which we report. Every executed deletion agreed with an independently reconstructed reference on every registered probe, and under sampling, targeted elicitation, related-data relearning, and membership inference an edited record was about as hard to extract as one never stored, while a prompt instruction to ignore the same record left it fully extractable. On $96$ long conversational histories with decoded answers, the edited assistant disclosed the deleted record in $15$ histories against $13$ for a rebuild that never stored it and $54$ for the instruction, and a blinded review of the outputs the matcher had cleared found that its misses were aliases or normalization failures of the answer, with no paraphrase among them. The edit also suppressed the deleted answer below the never-stored level, a signature an auditor can read. The result is a retrofit that makes a frozen model's memory addressable, a certificate for what the retrofit does, and a measurement of the distance that remains to the stronger guarantee; which of the two a system can offer is decided when the memory is written.

cs.LG↗

arXiv2026

What a Deletion Certificate Covers, and Where It Expires: Auditable Removal from a Support-Vector Memory

A dense key--value cache gives an operator no way to verify a deletion: it does not say which stored entries currently contribute nothing to the output, and it offers no reference state that the edited memory should match. We build a context memory whose entries carry explicit weights and ask what a deletion certificate over it can cover and where it expires. A one-class support-vector boundary fit around the keys of a context at test time supplies the coefficients used in the readout, so that each key is active (positive coefficient) or reserve (zero coefficient). Removing a reserve key leaves the readout unchanged without a re-solve, and deleting an active key with a decremental solver reaches the same state as re-solving on the remaining keys at the same coefficient cap. A three-key construction shows the limit of both properties: an inert key can acquire positive weight after one more token is admitted, so reserve status certifies the present solve and does not license permanent pruning. In $1,200$ double-precision trials on Gaussian, near-duplicate, clinical (MIMIC-IV vitals), and learned keys, every trial reached the reference state, all but one through the maintained update, with median gate-score disagreement below $10^{-6}$ in every regime and a worst case of $1.1\times10^{-2}$ associated with numerically near-tied solutions and partition disagreement; maintained deletion ran $24$--$223$ times faster than re-solving. Declaring a minimum-norm tie-break as part of the reference, at a tolerance above the two solvers' disagreement in objective value, brings the worst readout disagreement below $2\%$ of the readout range in every regime and leaves a gate-score disagreement of up to $4.2\times10^{-3}$. A deletion certificate should name its reference state and validity horizon. We show what a system must retain, or refuse to admit, to extend that horizon.

cs.LG↗

arXiv2017

Deep supervised learning using local errors

Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile backpropagation with the learning mechanisms observed in biological neural networks as it requires the neurons to maintain a memory of the input long enough until the higher-layer errors arrive. In this paper, we propose an alternative learning mechanism where errors are generated locally in each layer using fixed, random auxiliary classifiers. Lower layers could thus be trained independently of higher layers and training could either proceed layer by layer, or simultaneously in all layers using local error information. We address biological plausibility concerns such as weight symmetry requirements and show that the proposed learning mechanism based on fixed, broad, and random tuning of each neuron to the classification categories outperforms the biologically-motivated feedback alignment learning technique on the MNIST, CIFAR10, and SVHN datasets, approaching the performance of standard backpropagation. Our approach highlights a potential biological mechanism for the supervised, or task-dependent, learning of feature hierarchies. In addition, we show that it is well suited for learning deep networks in custom hardware where it can drastically reduce memory traffic and data communication overheads.

cs.NE↗