Searcharxiv⌕ Search

arXiv subjects

Constantin Kogler

Publications and source records attributed to Constantin Kogler.

8 recordsLinked to original sources
arXiv2026

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks

Effective initialization in deep networks requires an understanding of random neural networks. In this work, a rigorous probabilistic analysis of deep bias-free random Leaky ReLU networks is provided. We prove a Law of Large Numbers and a Central Limit Theorem for the logarithm of the norm of network activations, establishing that, as the number of layers increases, their growth is governed by a parameter called the Lyapunov exponent. This parameter characterizes a sharp phase transition between vanishing and exploding activations, and we calculate the Lyapunov exponent explicitly for Gaussian or orthogonal weight matrices. Our results reveal that standard methods, such as He initialization or orthogonal initialization, do not guarantee activation stability for deep networks of low width. Based on these theoretical insights, we propose a novel initialization method, referred to as Lyapunov initialization, which sets the Lyapunov exponent to zero and thereby ensures that the neural network is as stable as possible, leading empirically to improved learning.

stat.ML↗
arXiv2025

Permutation-Invariant Spectral Learning via Dyson Diffusion

Diffusion models are central to generative modeling and have been adapted to graphs by diffusing adjacency matrix representations. The challenge of having up to $n!$ such representations for graphs with $n$ nodes is only partially mitigated by using permutation-equivariant learning architectures. Despite their computational efficiency, existing graph diffusion models struggle to distinguish certain graph families and their spectra, unless graph data are augmented with ad hoc features. This shortcoming stems from enforcing the inductive bias within the learning architecture. In this work, we leverage random matrix theory to analytically extract the spectral properties of the diffusion process, allowing us to push most of the inductive bias from the architecture into the dynamics. Building on this, we introduce the Dyson Diffusion Model, which employs Dyson's Brownian motion to capture the spectral dynamics of an Ornstein-Uhlenbeck process on the adjacency matrix. Furthermore, conditioned on the spectral dynamics, we formulate a Lie group diffusion, appropriately modeling the remaining degrees of freedom. Strikingly, the resulting learning problem becomes permutation invariant at the Lie algebra level. We demonstrate that the Dyson Diffusion Model learns graph spectra accurately and outperforms existing graph diffusion models.

stat.ML↗
arXiv2025

Dimension of contracting on average self-similar measures

We generalise Hochman's theorem on the dimension of self-similar measures to contracting on average measures and show that a weaker condition than exponential separation on all scales is sufficient. Our proof uses a technique we call the variance summation method, avoiding the use of inverse theorems for entropy.

math.DS↗
arXiv2025

Entropy Theory for Random Walks on Lie Groups

We develop entropy and variance results for the product of independent identically distributed random variables on Lie groups. Our results apply to the study of stationary measures in various contexts.

math.PR↗
arXiv2024

On absolute continuity of inhomogeneous and contracting on average self-similar measures

We give a condition for absolute continuity of self-similar measures in arbitrary dimensions. This allows us to construct the first explicit absolutely continuous examples of inhomogeneous self-similar measures in dimension one and two. In fact, for $d\geq 1$ and any given rotations in $O(d)$ acting irreducibly on $\mathbb{R}^d$ as well as any distinct translations, all having algebraic coefficients, we construct absolutely continuous self-similar measures with the given rotations and translations. We furthermore strengthen Varj\'u's result for Bernoulli convolutions, treat complex Bernoulli convolutions and in dimension $\geq 3$ improve the condition on absolute continuity by Lindenstrauss-Varj\'u. Moreover, self-similar measures of contracting on average measures are studied, which may include expanding similarities in their support.

math.DS↗
arXiv2024

Effective density of non-degenerate random walks on homogeneous spaces

We prove effective density of random walks on homogeneous spaces, assuming that the underlying measure is supported on matrices generating a dense subgroup and having algebraic entries. The main novelty is an argument passing from high dimension to effective equidistribution in the setting of random walks on homogeneous spaces, exploiting spectral gap of the associated convolution operator.

math.PR↗
arXiv2022

Local limit theorem for random walks on symmetric spaces

We reduce the local limit theorem for a non-compact semisimple Lie group acting on its symmetric space to establishing that a natural operator associated to the measure is quasicompact. Under strong Diophantine assumptions on the underlying measure, we deduce the necessary spectral results for the operator in question. We thereby give the first examples of finitely supported measures satisfying such a local limit theorem. Moreover, quantitative error rates for the local limit theorem are proved under additional assumptions.

math.PR↗