arXiv subjects

Anherutowa Calvo

Publications and source records attributed to Anherutowa Calvo.

3 recordsLinked to original sources

arXiv2026

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

The curvature exponent $α$ in $h_k \propto σ_k^α$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($α\approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections). Why? We prove the Spectral Alignment Decomposition: $α= 2 + d\logΦ_k / d\logσ_k$, where $Φ_k$ measures alignment between Kronecker factor eigenbases and gradient singular directions. This reduces "why does $α$ vary?" to a geometric question we answer for LayerNorm, residual connections, and softmax heads. The decomposition implies a spectral transfer identity $s = αγ$ linking curvature exponent, effective gradient rank-decay $γ$, and Hessian decay exponent $s$. The identity is algebraic; its empirical content is that $α$ and $γ$, fit on independent data (HVPs vs. SVD), recover $s$ to ~2% median error across 93 layers, five architectures, and three datasets -- with no free parameters. A zeta-function bound on participation ratio shows curvature concentrates onto effectively one direction per layer. As a proof of concept, we derive the architecture-adaptive preconditioner $T(σ;α)$ and show that Spectral Newton -- implementing $T$ in the gradient singular basis -- outperforms AdamW on vision benchmarks where $α\approx 2$.

cs.LG↗

arXiv2026

Predictable Gradient Manifolds in Deep Learning: Temporal Path-Length and Intrinsic Rank as a Complexity Regime

Deep learning optimization exhibits structure that is not captured by worst-case gradient bounds. Empirically, gradients along training trajectories are often temporally predictable and evolve within a low-dimensional subspace. In this work we formalize this observation through a measurable framework for predictable gradient manifolds. We introduce two computable quantities: a prediction-based path length that measures how well gradients can be forecast from past information, and a predictable rank that quantifies the intrinsic temporal dimension of gradient increments. We show how classical online and nonconvex optimization guarantees can be restated so that convergence and regret depend explicitly on these quantities, rather than on worst-case variation. Across convolutional networks, vision transformers, language models, and synthetic control tasks, we find that gradient trajectories are locally predictable and exhibit strong low-rank structure over time. These properties are stable across architectures and optimizers, and can be diagnosed directly from logged gradients using lightweight random projections. Our results provide a unifying lens for understanding optimization dynamics in modern deep learning, reframing standard training as operating in a low-complexity temporal regime. This perspective suggests new directions for adaptive optimizers, rank-aware tracking, and prediction-based algorithm design grounded in measurable properties of real training runs.

cs.LG↗

arXiv2026

Chronology as a Consistency Invariant in Composable Information Systems

We formalize a minimal setting in which a chronology (a strict partial order on events) is forced by consistency of distributed information under local composability. The system maintains distributed records interpreted as constraints over a global possibility space (Omega, Sigma), optionally with a measure mu. Events act locally by monotonically tightening records, and independent events commute (diamond/trace semantics), yielding schedule invariance. We define operational influence without assuming primitive time: e influences f if executing e can change what constraint f writes on a shared site. Influence cycles alone need not imply inconsistency, so we distinguish weak influence (dependence) from strong influence (exclusive branching on an observable predicate). Assuming global satisfiability of all reachable record states, the diamond property, monotone information writing, and a mild branch-determinacy axiom for witnessed exclusivity, we prove that strong influence is acyclic and therefore induces an intrinsic chronology. We also show trace invariance and minimality of the derived order, introduce a monotone information clock based on -log mu(feasible set), and give an escape taxonomy: any model that admits strong-influence cycles without inconsistency must violate global consistency, local composability, monotone writing, or branch determinacy.

cs.LO↗