arXiv subjects

Qingkai Dong

Publications and source records attributed to Qingkai Dong.

2 recordsLinked to original sources

arXiv2026

Heirloom: Carrying a Phone's Personalization Across Base-Model Generations

On-device language models can learn a user's preferences through adapters trained on private data. However, a base-model upgrade may change the layers or vocabulary, leaving the personal adapter incompatible. Replacing it with a non-personal adapter loses much of the personalization, while retraining requires retaining the user's labeled history. We present Heirloom, a system that transfers a personal adapter to a new base model on the phone without retaining labels. Heirloom uses the old personal model as a teacher, collecting its predictions on stored unlabeled inputs, vendor-provided prompts, or prompts generated by the new base. It then unloads the old base and trains an adapter for the new model to match those predictions, keeping only one base model in memory at a time. We evaluate Heirloom across three generation upgrades from two model families, an architecture-preserving refresh, and three personalization tasks. On movie tagging, migration from stored inputs maintains pre-upgrade accuracy and differs from labeled retraining by at most 1.7 percentage points across the three upgrades. On a phone's neural processing unit (NPU), a fixed-length migration for the 4B upgrade takes 38 minutes and consumes 20% of the battery end to end, making the stored-input setting feasible during overnight charging.

cs.LG↗

arXiv2026

Rare-Feature-Adaptive Subsampling for Big Data Regression

Rare binary features cause instability and slow convergence in regression. Subsampling massive datasets may discard observations carrying rare-feature information. We show that rare-feature coefficient convergence rates depend on feature prevalence and the inclusion probabilities assigned to rare observations. This finding motivates subsampling designs that adapt sampling probabilities to feature rarity. We first study optimal rare-census designs, which extend the negative subsampling approach by retaining every rare observation. However, they may face practical constraints. We then propose a computationally efficient rarity-adaptive L-optimal design that incorporates inverse prevalence scaling. Because implementing optimal designs requires pilot estimates whose quality depends on adequate representation of rare features, we propose a balanced design, a one-step rarity-adaptive design based only on binary features. It can be used alone, as a pilot, or combined with designs targeting other objectives. For implementation in two stages, we develop a union-sample estimator that reuses pilot observations while accounting for sample overlap and establish its asymptotic normality. Simulations and applications show improved stability and estimation efficiency over existing methods. One application evaluates designs that jointly protect rare outcomes and rare features.

stat.ME↗