Searcharxiv⌕ Search

arXiv subjects

Klaus Schmidt

Publications and source records attributed to Klaus Schmidt.

At least 19 recordsLinked to original sources
arXiv2025

Video Event Reasoning and Prediction by Fusing World Knowledge from LLMs with Vision Foundation Models

Current video understanding models excel at recognizing "what" is happening but fall short in high-level cognitive tasks like causal reasoning and future prediction, a limitation rooted in their lack of commonsense world knowledge. To bridge this cognitive gap, we propose a novel framework that synergistically fuses a powerful Vision Foundation Model (VFM) for deep visual perception with a Large Language Model (LLM) serving as a knowledge-driven reasoning core. Our key technical innovation is a sophisticated fusion module, inspired by the Q-Former architecture, which distills complex spatiotemporal and object-centric visual features into a concise, language-aligned representation. This enables the LLM to effectively ground its inferential processes in direct visual evidence. The model is trained via a two-stage strategy, beginning with large-scale alignment pre-training on video-text data, followed by targeted instruction fine-tuning on a curated dataset designed to elicit advanced reasoning and prediction skills. Extensive experiments demonstrate that our model achieves state-of-the-art performance on multiple challenging benchmarks. Notably, it exhibits remarkable zero-shot generalization to unseen reasoning tasks, and our in-depth ablation studies validate the critical contribution of each architectural component. This work pushes the boundary of machine perception from simple recognition towards genuine cognitive understanding, paving the way for more intelligent and capable AI systems in robotics, human-computer interaction, and beyond.

cs.CV↗
arXiv2025

Restricted Permutations and Permanents of Infinite Amenable Groups

Let $\Gamma $ be an infinite discrete group and $\mathsf{A}\subset \Gamma $ a nonempty finite subset. The set of permutations $\sigma $ of $\Gamma $ such that $s^{-1}\sigma (s)\in \mathsf{A}$ for every $s\in \Gamma $ can be identified with a shift of finite type $X_\mathsf{A}\subset \mathsf{A}^{\Gamma}$ over $\Gamma $. In this paper we study dynamical properties of such shift spaces, like invariant probability measures, topological entropy, and topological pressure, under the hypothesis that $\Gamma $ is amenable. In this case the topological entropy $\textrm{h}_{\textrm{top}}(X_\mathsf{A})$ can be expressed as logarithmic growth rate of permanents of certain finite (0,1)-matrices associated with right F{\o}lner sequences in $\Gamma $. Motivated by the difficulty of computing such permanents we introduce the notion of the permanent $\textrm{per}(f)$ for nonnegative elements $f$ in the real group ring $\mathbb{R}\Gamma $ of $\Gamma $ whose support is the alphabet $\mathsf{A}$ of the shift space $X_\mathsf{A}$, and compare, for arbitrary $f \in \mathbb{R}\Gamma $, the Fuglede-Kadison determinant $\textrm{det} _\textrm{FK}(f)$ with the permanent $\textrm{per}(|f|)$ of the absolute value $|f|$ of $f$. Although this approach is effective in only few examples, discussed below, it is interesting from a conceptual point of view that the permanent $\textrm{per}(f)$ of a nonnegative element $f\in \mathbb{R}\Gamma $ can be viewed as topological pressure of the restricted-permutation shift space $X_\mathsf{A}$ associated with the function $\log f$ on the alphabet $\mathsf{A}=\textrm{supp}(f)$ of $X_\mathsf{A}$.

math.DS↗
arXiv2024

Divisibility of Integer Laurent Polynomials, Homoclinic Points, and Lacunary Independence

Let $f$, $p$, and $q$ be Laurent polynomials with integer coefficients in one or several variables, and suppose that $f$ divides $p+q$. We establish sufficient conditions to guarantee that $f$ individually divides $p$ and $q$. These conditions involve a bound on coefficients, a separation between the supports of $p$ and $q$, and, surprisingly, a requirement on the complex variety of $f$ called atorality satisfied by many but not all polynomials. Our proof involves a related dynamical system and the fundamental dynamical notion of homoclinic point. Without the atorality assumption our methods fail, and it is unknown whether our results hold without this assumption.

math.DS↗
arXiv2021

Decimation limits of principal algebraic $\mathbb{Z}^d$-actions

Let $f$ be a Laurent polynomial in $d$ commuting variables with integer coefficients. Associated to $f$ is the principal algebraic $\mathbb{Z}^d$-action $\alpha_f$ on a compact subgroup $X_f$ of $\mathbb{T}^{\mathbb{Z}^d}$ determined by $f$. Let $N\ge1$ and restrict points in $X_f$ to coordinates in $N\mathbb{Z}^d$. The resulting algebraic $N\mathbb{Z}^d$-action is again principal, and is associated to a polynomial $g_N$ whose support grows with $N$ and whose coefficients grow exponentially with $N$. We prove that by suitably renormalizing these decimations we can identify a limiting behavior given by a continuous concave function on the Newton polytope of $f$, and show that this decimation limit is the negative of the Legendre dual of the Ronkin function of $f$. In certain cases with two variables, the decimation limit coincides with the surface tension of random surfaces related to dimer models, but the statistical physics methods used to prove this are quite different and depend on special properties of the polynomial.

math.DS↗
arXiv2021

Bohr chaoticity of principal algebraic actions and Riesz product measures

For a continuous $\mathbb{N}^d$ or $\mathbb{Z}^d$ action on a compact space, we introduce the notion of Bohr chaoticity, which is an invariant of topological conjugacy and which is proved stronger than having positive entropy. We prove that all principal algebraic $\mathbb{Z}$ actions of positive entropy are Bohr-chaotic. The same is proved for principal algebraic $\mathbb{Z}^d$ ($d\ge 2$) actions of positive entropy under the condition of existence of summable homoclinic points.

math.DS↗
arXiv2020

New Examples of Bernoulli Algebraic Actions

We give examples of principal algebraic actions of the noncommutative free group $F$ of rank two, as well as other groups, by automorphisms of a connected compact abelian group for which there is an explicit measurable isomorphism to a Bernoulli action of the group. The isomorphism is defined using homoclinic points, a method that has been used earlier to construct symbolic covers of algebraic actions. To our knowledge, these are the first nontrivial examples of the Bernoullicity of an algebraic action of $F$.

math.DS↗
arXiv2018

Mahler's Work and Algebraic Dynamical Systems

After Furstenberg had provided a first glimpse of remarkable rigidity phenomena associated with the joint action of several commuting automorphisms (or endomorphisms) of a compact abelian group, further key examples motivated the development of an extensive theory of such actions. Two of Mahler's achievements, the recognition of the significance of Mahler measure of multivariate polynomials in relating the lengths and heights of products of polynomials in terms of the corresponding quantities for the constituent factors, and his work on additive relations in fields, have unexpectedly played important roles in the study of entropy and higher order mixing for these actions. This article briefly surveys these connections between Mahler's work and dynamics. It also sketches some of the dynamical outgrowths of his work that are very active today, including the investigation of the Fuglede-Kadison determinant of a convolution operator in a group von Neumann algebra as a noncommutative generalization of Mahler measure, as well as diophantine questions related to the growth rates of periodic points and their relation to entropy.

math.DS↗
arXiv2017

Permutations of $\mathbb{Z}^d$ with restricted movement

We investigate dynamical properties of the set of permutations of $\mathbb{Z}^d$ with restricted movement, i.e., permutations $π$ of $\mathbb{Z}^d$ such that $π(\mathbf{n})-\mathbf{n}$ lies, for every $\mathbf{n}\in \mathbb{Z}^d$, in a prescribed finite set $A\subset \mathbb{Z}^d$. For $d=1$, such permutations occur, for example, in restricted orbit equivalence, or in the calculation of determinants of certain bi-infinite multi-diagonal matrices. For $d\ge2$ these sets of permutations provide natural classes of examples of multidimensional shifts of finite type.

math.DS↗
arXiv2016

A Wiener Lemma for the discrete Heisenberg group: Invertibility criteria and applications to algebraic dynamics

This article contains a Wiener Lemma for the convolution algebra $\ell^1(\mathbb H,\mathbb C)$ and group $C^\ast$-algebra $C^\ast(\mathbb H)$ of the discrete Heisenberg group $\mathbb H$. At first, a short review of Wiener's Lemma in its classical form and general results about invertibility in group algebras of nilpotent groups will be presented. The known literature on this topic suggests that invertibility investigations in the group algebras of $\mathbb H$ rely on the complete knowledge of $\widehat{\mathbb H}$ -- the dual of $\mathbb H$, i.e., the space of unitary equivalence classes of irreducible unitary representations. We will describe the dual of ${\mathbb H}$ explicitly and discuss its structure. Wiener's Lemma provides a convenient condition to verify invertibility in $\ell^1(\mathbb H,\mathbb C)$ and $C^\ast(\mathbb H)$ which bypasses $\widehat{\mathbb H}$. The proof of Wiener's Lemma for $\mathbb H$ relies on local principles and can be generalised to countable nilpotent groups. As our analysis shows, the main representation theoretical objects to study invertibility in group algebras of nilpotent groups are the corresponding primitive ideal spaces. Wiener's Lemma for $\mathbb H$ has interesting applications in algebraic dynamics and Time-Frequency Analysis which will be presented in this article as well.

math.DS↗
arXiv2015

Representations of toral automorphisms

This survey gives an account of an algebraic construction of symbolic covers and representations of ergodic automorphisms of compact abelian groups, initiated by A.M. Vershik around 1992 for hyperbolic automorphisms of finite-dimensional tori. The key ingredient in this approach, which was subsequently extended to arbitrary expansive automorphisms of compact abelian groups, is the use of homoclinic points of the automorphism. Although existence and abundance of homoclinic points is intimately connected to expansiveness of the automorphism, it is nevertheless possible to extend certain aspects of this construction to nonexpansive irreducible automorphisms of compact abelian groups (like irreducible toral automorphisms whose dominant eigenvalue is a Salem number). The later sections of this survey discuss the phenomena and problems arising in this extension.

math.DS↗
arXiv2015

A Survey of Algebraic Actions of the Discrete Heisenberg Group

The study of actions of countable groups by automorphisms of compact abelian groups has recently undergone intensive development, revealing deep connections with operator algebras and other areas. The discrete Heisenberg group is the simplest noncommutative example, where dynamical phenomena related to its noncommutativity already illustrate many of these connections. The explicit structure of this group means that these phenomena have concrete descriptions, which are not only instances of the general theory but are also testing grounds for further work. We survey here what is known about such actions of the discrete Heisenberg group, providing numerous examples and emphasizing many of the open problems that remain.

math.DS↗
arXiv2014

Algebraic actions of the discrete Heisenberg group: Expansiveness and homoclinic points

We survey some of the known criteria for expansiveness of principal algebraic actions of countably infinite discrete groups. In the special case of the discrete Heisenberg group we propose a new approach to this problem based on Allan's local principle. Furthermore, we present a first example of an absolutely summable homoclinic point for a nonexpansive action of the discrete Heisenberg group and use it to construct an equal-entropy symbolic cover of the system.

math.DS↗
arXiv2013

Ergodicity of principal algebraic group actions

An \textit{algebraic} action of a discrete group $Γ$ is a homomorphism from $Γ$ to the group of continuous automorphisms of a compact abelian group $X$. By duality, such an action of $Γ$ is determined by a module $M=\widehat{X}$ over the integer group ring $\mathbb{Z}Γ$ of $Γ$. The simplest examples of such modules are of the form $M=\mathbb{Z}Γ/\mathbb{Z}Γf$ with $f\in \mathbb{Z}Γ$; the corresponding algebraic action is the \textit{principal algebraic $Γ$-action} $α_f$ defined by $f$. In this note we prove the following extensions of results by Hayes \cite{Hayes} on ergodicity of principal algebraic actions: If $Γ$ is a countably infinite discrete group which is not virtually cyclic, and if $f\in\mathbb{Z}Γ$ satisfies that right multiplication by $f$ on $\ell ^2(Γ,\mathbb{R})$ is injective, then the principal $Γ$-action $α_f$ is ergodic (Theorem \ref{t:ergodic2}). If $Γ$ contains a finitely generated subgroup with a single end (e.g. a finitely generated amenable subgroup which is not virtually cyclic), or an infinite nonamenable subgroup with vanishing first $\ell ^2$-Betti number (e.g., an infinite property $T$ subgroup), the injectivity condition on $f$ can be replaced by the weaker hypothesis that $f$ is not a right zero-divisor in $\mathbb{Z}Γ$ (Theorem \ref{t:ergodic1}). Finally, if $Γ$ is torsion-free, not virtually cyclic, and satisfies Linnell's \textit{analytic zero-divisor conjecture}, then $α_f$ is ergodic for every $f\in \mathbb{Z}Γ$ (Remark \ref{r:analytic zero divisor}).

math.DS↗
arXiv2012

Homoclinic points, atoral polynomials, and periodic points of algebraic Z^d-actions

Cyclic algebraic Z^d-actions are defined by ideals of Laurent polynomials in d commuting variables. Such an action is expansive precisely when the complex variety of the ideal is disjoint from the multiplicative d-torus. For such expansive actions it is known that the limit for the growth rate of periodic points exists and is equal to the entropy of the action. In an earlier paper the authors extended this result to ideals whose variety intersects the d-torus in a finite set. Here we further extend it to the case when the dimension of intersection of the variety with the d-torus is at most d-2. The main tool is the construction of homoclinic points which decay rapidly enough to be summable.

math.DS↗
arXiv2009

Entropy and growth rate of periodic points of algebraic Z^d-actions

Expansive algebraic Z^d-actions corresponding to ideals are characterized by the property that the complex variety of the ideal is disjoint from the multiplicative unit torus. For such actions it is known that the limit for the growth rate of periodic points exists and equals the entropy of the action. We extend this result to actions for which the complex variety intersects the multiplicative torus in a finite set. The main technical tool is the use of homoclinic points which decay rapidly enough to be summable.

math.DS↗
arXiv2009

Abelian Sandpiles and the Harmonic Model

We present a construction of an entropy-preserving equivariant surjective map from the $d$-dimensional critical sandpile model to a certain closed, shift-invariant subgroup of $\mathbb{T}^{\mathbb{Z}^d}$ (the `harmonic model'). A similar map is constructed for the dissipative abelian sandpile model and is used to prove uniqueness and the Bernoulli property of the measure of maximal entropy for that model.

math.DS↗
arXiv2007

Algebraic Polymorphisms

In this paper we consider a special class of polymorphisms with invariant measure, - (cf.[1])- the algebraic polymorphisms of compact groups. A general polymorphism is -- by definition -- a many-valued map with invariant measure, and the conjugate operator of a polymorphism is a Markov operator (i.e., a positive operator on $L^2$ of norm 1 which preserves the constants). In the algebraic case a polymorphism is a correspondence in the sense of algebraic geometry, but here we investigate it from a dynamical point of view. The most important examples are the algebraic polymorphisms of torus, where we introduce a parametrization of the semigroup of toral polymorphisms in terms of rational matrices and describe the spectra of the corresponding Markov operators.

math.DS↗