SearcharxivSearch

arXiv subjects

Aleksandr Berdnikov

Publications and source records attributed to Aleksandr Berdnikov.

9 recordsLinked to original sources

Sky sphere representation in language models

We analyze whether language models of size ~100B have a representation of the night sky map that is decodable from their residual stream. We find that most of the considered open-source models do have such a representation, and it often even surfaces to the top principal components on prompts that ask questions like ``what is close to this object in the night sky''. In all but one model this representation showed significant scores in LOO testing, containing up to 65-85% of variance ($R^2$-score) and having median angular error down to $12^\circ-21^\circ$. We verify that our representation is not a simple leak from a correlated flat representation. To our knowledge, this representation is the first example of a curved high-dimensional irreducible feature manifold. Codes used in the paper are published at https://github.com/l3erdnik/Decodable-sky

cs.LG

Urysohn Width and Surgeries

We analyze the behavior of Urysohn width of manifolds under a connected sum operation, specifically, bounding widths of summands in terms of widths of the sum and vice versa. Our methods also apply to the universal covers of these spaces, and to more general type of surgeries. Lastly, we provide examples that show the optimality of constants in our estimates.

math.MG

Parsimonious cones

We construct embeddings of simplicial complexes into a (surface of a) simplicial ball whose triangulation has bounded degrees and low volume. This construction can be used either to efficiently "simplify a complicated space" by realizing it as a part of a ball/sphere, or to "complexify" a sphere - to give it a specific metric that inherits desired properties from an embedded complex.

math.GT

Degrees of maps and multiscale geometry

We study the degree of an $L$-Lipschitz map between Riemannian manifolds, proving new upper bounds and constructing new examples. For instance, if $X_k$ is the connected sum of $k$ copies of $\mathbb CP^2$ for $k \ge 4$, then we prove that the maximum degree of an $L$-Lipschitz self-map of $X_k$ is between $C_1 L^4 (\log L)^{-4}$ and $C_2 L^4 (\log L)^{-1/2}$. More generally, we divide simply connected manifolds into three topological types with three different behaviors. Each type is defined by purely topological criteria. For scalable simply connected $n$-manifolds, the maximal degree is $\sim L^n$. For formal but non-scalable simply connected $n$-manifolds, the maximal degree grows roughly like $L^n (\log L)^{\theta(1)}$. And for non-formal simply connected $n$-manifolds, the maximal degree is bounded by $L^\alpha$ for some $\alpha < n$.

math.MG

Scalable spaces

\emph{Scalable spaces} are simply connected compact manifolds or finite complexes whose real cohomology algebra embeds in their algebra of (flat) differential forms. This is a rational homotopy invariant property and all scalable spaces are formal; indeed, scalability can be thought of as a metric version of formality. They are also characterized by particularly nice behavior from the point of view of quantitative homotopy theory. Among other results, we show that spaces which are formal but not scalable provide counterexamples to Gromov's long-standing conjecture on distortion in higher homotopy groups.

math.GT

Lipschitz null-homotopy of mappings $S^3 \rightarrow S^2$

This work focuses on important step in quantitative topology: given homotopic mappings from $S^m$ to $S^n$ of Lipschitz constant $L$, build the (asymptotically) simplest homotopy between them (meaning having the least Lipschitz constant). The present paper resolves this problem for the first case where Hopf invariant plays a role: $m = 3$, $n = 2$, constructing a homotopy with Lipschitz constant $O(L)$

math.GT

Waist of maps measured via Urysohn width

We discuss various questions of the following kind: for a continuous map $X \to Y$ from a compact metric space to a simplicial complex, can one guarantee the existence of a fiber large in the sense of Urysohn width? The $d$-width measures how well a space can be approximated by a $d$-dimensional complex. The results of this paper include the following. 1) Any piecewise linear map $f: [0,1]^{m+2} \to Y^m$ from the unit euclidean $(m+2)$-cube to an $m$-polyhedron must have a fiber of $1$-width at least $\frac{1}{2βm +m^2 + m + 1}$, where $β= \sup_y \text{ rk } H_1(f^{-1}(y))$ measures the topological complexity of the map. 2) There exists a piecewise smooth map $X^{3m+1} \to \mathbb{R}^m$, with $X$ a riemannian $(3m+1)$-manifold of large $3m$-width, and with all fibers being topological $(2m+1)$-balls of arbitrarily small $(m+1)$-width.

math.MG

Local-to-global Urysohn width estimates

The notion of the Urysohn $d$-width measures to what extent a metric space can be approximated by a $d$-dimensional simplicial complex. We investigate how local Urysohn width bounds on a riemannian manifold affect its global width. We bound the $1$-width of a Riemannian manifold in terms of its first homology and the supremal width of its unit balls. Answering a question of Larry Guth, we give examples of $n$-manifolds of considerable $(n-1)$-width in which all unit balls have arbitrarily small $1$-width. We also give examples of topologically simple manifolds that are locally nearly low-dimensional.

math.MG