arXiv · 2611.10881
Robust and Efficient Continual Model Merging via Global Singular Subspace Separation and Restoration
Abstract
Continual model merging incrementally integrates task-specific models without revisiting past models or data. However, repeated merging entangles past task components and distorts their singular geometry, making balanced task-wise allocation increasingly difficult. We propose Global Singular Subspace Separation and Restoration (GS3R), which augments the merged update with a compact restoration state. At each merge, GS3R restores the retained singular geometry, separates task-wise subspaces, and globally reallocates the available rank across tasks fairly. To reduce merging-time memory overhead, we further introduce Pre-merge Restoration and Flushing (PRF). Experiments on vision, NLP, and multimodal benchmarks show that GS3R provides a strong accuracy-efficiency trade-off.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Juncheol Shin, Byeongho Yu, Semin Kim, Eunhyeok Park. 2026-11-07. Robust and Efficient Continual Model Merging via Global Singular Subspace Separation and Restoration. https://searcharxiv.com/abs/2611.10881
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.