arXiv subjects

Mariana Costa

Publications and source records attributed to Mariana Costa.

3 recordsLinked to original sources

arXiv2026

SpaCE: Rethinking Spatial Capacity and Generalization in Multi-Frame Multimodal Large Language Models

Multi-modal large language models (MLLMs) have achieved remarkable empirical progress in spatial understanding through large-scale training on spatial visual question answering datasets. However, the theoretical foundations of multi-frame spatial reasoning remain entirely unexplored. We present SpaCE, a rigorous theoretical framework that characterizes the spatial reasoning capacity, sample complexity, and generalization guarantees of MLLMs operating on multi-frame inputs. We establish four main results. First, we prove an information-theoretic upper bound on spatial reasoning accuracy in terms of the mutual information between multi-frame observations and spatial targets. Second, we derive a sample complexity bound of order $Θ(d_{\mathrm{eff}} \cdot K_{\max} / (\varepsilon^2 \cdot δ))$, where $d_{\mathrm{eff}}$ is the effective spatial dimension and $K_{\max}$ bounds the KL divergence of the learned posterior. Third, we provide a PAC-Bayes generalization bound for multi-frame spatial reasoning under distribution shift. Fourth, we formally characterize the bias-variance trade-off between explicit 3D representations and implicit reasoning approaches, identifying the crossover conditions under which each paradigm is provably preferable. We validate our theoretical predictions on the MultiSPA, CA-VQA, and SpatialRGPT benchmarks, demonstrating that our bounds are empirically tight and that frame complementarity is the key driver of multi-frame spatial capacity. Our framework provides the first principled theoretical foundation for understanding when, why, and how multi-frame spatial reasoning in MLLs succeeds.

eess.IV↗

arXiv2026

Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guarantees

Knowledge distillation has emerged as a powerful technique for compressing large language models (LLMs) into efficient, deployable architectures while preserving their advanced capabilities. Recent advances in low-rank knowledge distillation, particularly methods like Low-Rank Clone (LRC), have demonstrated remarkable empirical success, achieving comparable performance to full-parameter distillation with significantly reduced training data and computational overhead. However, the theoretical foundations underlying these methods remain poorly understood. In this paper, we establish a rigorous theoretical framework for low-rank knowledge distillation in language models. We prove that under mild assumptions, low-rank projection preserves the optimization dynamics, yielding explicit convergence rates of $O(1/\sqrt{T})$. We derive generalization bounds that characterize the fundamental trade-off between model compression and generalization capability, showing that the generalization error scales with the rank parameter as $O(r(m+n)/\sqrt{n})$. Furthermore, we provide an information-theoretic analysis of the activation cloning mechanism, revealing its role in maximizing the mutual information between the teacher's and student's intermediate representations. Our theoretical results offer principled guidelines for rank selection, mathematically suggesting an optimal rank $r^* = O(\sqrt{n})$ where $n$ is the sample size. Experimental validation on standard language modeling benchmarks confirms our theoretical predictions, demonstrating that the empirical convergence, rank scaling, and generalization behaviors align closely with our bounds.

stat.ML↗

arXiv2026

Enhancing Self-Correction in Large Language Models through Multi-Perspective Reflection

While Chain-of-Thought (CoT) prompting advances LLM reasoning, challenges persist in consistency, accuracy, and self-correction, especially for complex or ethically sensitive tasks. Existing single-dimensional reflection methods offer insufficient improvements. We propose MyGO Poly-Reflective Chain-of-Thought (PR-CoT), a novel methodology employing structured multi-perspective reflection. After initial CoT, PR-CoT guides the LLM to self-assess its reasoning across multiple predefined angles: logical consistency, information completeness, biases/ethics, and alternative solutions. Implemented purely via prompt engineering, this process refines the initial CoT into a more robust and accurate final answer without model retraining. Experiments across arithmetic, commonsense, ethical decision-making, and logical puzzles, using GPT-three point five and GPT-four models, demonstrate PR-CoT's superior performance. It significantly outperforms traditional CoT and existing reflection methods in logical consistency and error correction, with notable gains in nuanced domains like ethical decision-making. Ablation studies, human evaluations, and qualitative analyses further validate the contribution of each reflection perspective and the overall efficacy of our poly-reflective paradigm in fostering more reliable LLM reasoning.

cs.CL↗