arXiv subjects

Jan Peszek

Publications and source records attributed to Jan Peszek.

At least 19 recordsLinked to original sources

arXiv2026

On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers

We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles that interact dynamically through successive linear self-attention layers. We show that in embedding dimension two, for any key, query, and value matrices, the dynamics can be reformulated as a generalized Kuramoto-type model with pure second-harmonic coupling. This formulation is amenable to Watanabe--Strogatz theory which reveals the dynamics are intrinsically low-dimensional regardless of the parameter matrices. For a class of token initializations associated with the Ott--Antonsen (OA) manifold, we show that the parameter matrices induce a diverse variety of long-time behaviors in linear transformers, including clustering, oscillations, and bifurcations. The oscillations and bifurcations are characterized by uncovering a hidden Hamiltonian structure in the dynamics. By establishing a structural stability result, we further show that dynamics initialized near the OA manifold exhibit the same long-time behavior as those initialized exactly on the manifold. Motivated by our theory in dimension two, we conduct numerical experiments for analogous parameter regimes in higher-dimensional transformers. Our numerical experiments suggest that the long-time behaviors characterized in our theoretical results persist in higher dimensions.

cs.LG↗

arXiv2026

Topological and Purely Topological Alignment Dynamics

We study the Euler Alignment system of collective behavior, equipped with `topological' interaction protocols, which were introduced to the mathematical literature by Shvydkoy and Tadmor. Interactions subject to these protocols may depend on both the Euclidean distance between agents and on the mass distribution between them -- the `topological' component. When the interaction protocol is regular, we prove sufficient conditions for the existence of global-in-time classical solutions, related to the initial nonnegativity of a conserved quantity of the system. The remainder of our results explore the case where the interactions are `purely' topological and the interactions do not depend on the Euclidean distance. We show that in this case, the system decouples into an autonomous velocity equation in mass coordinates together with a scalar conservation law with time-dependent flux determined by the velocity. We analyze the long-time behavior for the dynamics associated to both regular and singular protocols.

math.AP↗

arXiv2024

Alignment with nonlinear velocity couplings: collision-avoidance and micro-to-macro mean-field limits

We investigate the pressureless fractional Euler-alignment system with nonlinear velocity couplings, referred to as the $p$-Euler-alignment system. This model features a nonlinear velocity alignment force, interpreted as a density-weighted fractional $p$-Laplacian when the singularity parameter $\alpha$ exceeds the spatial dimension $d$. Our primary goal is to establish the existence of solutions for strongly singular interactions ($\alpha \ge d$) and compactly supported initial conditions. We construct solutions as mean-field limits of empirical measures from a kinetic variant of the $p$-Euler-alignment system. Specifically, we show that a sequence of empirical measures converges to a finite Radon measure, whose local density and velocity satisfy the $p$-Euler-alignment system. Our results are the first to prove the existence of solutions to this system in multi-dimensional settings without significant initial data restrictions, covering both nonlinear ($p>2$) and linear ($p=2$) cases. Additionally, we establish global existence, uniqueness, and collision avoidance for the corresponding particle ODE system under non-collisional initial conditions, extending previous results for $1 \le p \le \alpha + 2$. This analysis supports our mean-field limit argument and contributes to understanding alignment models with singular communication.

math.AP↗

arXiv2024

Non-local dissipative Aw-Rascle model and its relation with Matrix-valued communication in Euler alignment

We compare the multi-dimensional generalisation of the Aw-Rascle model with the pressureless Euler-alignment system, in which the communication weight is matrix-valued. Our generalisation includes the velocity offset in the form of a gradient of a non-local density function, given by the convolution with a kernel $K$. We investigate connections between these models at the macroscopic, mesoscopic and macroscopic (hydrodynamic) level, and overview the results on the mean-field limit for various assumptions on $K$.

math.AP↗

arXiv2024

Mean-field limit of 2D stationary particle systems with signed Coulombian interactions

We study the mean-field limits of critical points of interaction energies with Coulombian singularity. An important feature of our setting is that we allow interaction between particles of opposite signs. Particles of opposite signs attract each other whereas particles of the same signs repel each other. In 2D, we prove that the associated empirical measures converge to a limiting measure $\mu$ that satisfies a two-fold criticality condition: in velocity form or in vorticity form. Our setting includes the stationary attraction-repulsion problem with Coulombian singularity and the stationary system of point-vortices in fluid mechanics. In this last context, in the case where the limiting measure is in $H^{-1}_{\text{loc}}({\mathbb R}^2)$, we recover the classical criticality condition stating that $\nabla^\perp g \ast \mu$, with $g(x)=-\log |x|$, is a stationary solution of the incompressible Euler equation. This result, is, to the best of our knowledge, new in the case of particles with different signs (for particles of the positive sign it was obtained by Schochet in 1996). In order to derive the limiting criticality condition in the velocity form, we follow an approach devised by Sandier-Serfaty in the context of Ginzburg-Landau vortices. This consists of passing to the limit in the stress-energy tensor associated with the velocity field. On the other hand, the criticality condition in the vorticity form is obtained by arguments closer to the ones of Schochet.

math.AP↗

arXiv2022

Inevitable monokineticity of strongly singular alignment

We prove that certain types of measure-valued mappings are monokinetic i.e. the distribution of velocity is concentrated in a Dirac mass. These include weak measure-valued solutions to the strongly singular Cucker-Smale model with singularity of order $\alpha$ greater or equal to the dimension of the ambient space. Consequently, we are able to answer a couple of open questions related to the singular Cucker-Smale model. First, we prove that weak measure-valued solutions to the strongly singular Cucker-Smale kinetic equation are monokinetic, under very mild assumptions that they are uniformly compactly supported and weakly continuous in time. This can be interpreted as a rigorous derivation of the macroscopic fractional Euler-alignment system from kinetic Cucker-Smale equation without the need to perform any hydrodynamical limit. This suggests superior suitability of the macroscopic framework to describe large-crowd limits of strongly singular Cucker-Smale dynamics. Second, we perform a direct micro- to macroscopic mean-field limit from the Cucker-Smale particle system to the fractional Euler-alignment model. This leads to the final result -- existence of weak solutions to the fractional Euler-alignment system with almost arbitrary initial data in $\mathbb{R}^1$, including the possibility of vacuum. Existence can be extended to $\mathbb{R}^2$ under the a priori assumption that the density of the mean-field limit has no atoms.

math.AP↗

arXiv2022

A fuzzy $q$-closest alignment model

The paper examines the problems related to the well-posedness of the Cucker-Smale model with communication restricted to the $q$-closest neighbors, known also as the Cucker-Dong model. With agents oscillating on the boundary of different clusters, the system becomes difficult to precisely define, which leads to further problems with kinetic limits as the number of agents tends to infinity. We introduce the fuzzy $q$-closest system, which circumvents the issues with well-posedness. For such a system we prove a stability estimate for measure-valued solutions and perform the kinetic mean-field limit.

math.AP↗

arXiv2022

Measure solutions to a kinetic Cucker-Smale model with singular and matrix-valued communication

We introduce a multi-dimensional variant of the kinetic Cucker-Smale model with singular and matrix-valued communication weight, which reduces to the singular kinetic Cucker-Smale equation in the one-dimensional case. We propose an appropriate notion of weak measure-valued solution to this second-order system and a suitable first-order reduction, which persist beyond the blow-up time in classical norms. The core of the paper is to show that both formulations are equivalent in this singular regime, thus extending the previous results from regular to weakly singular communication weights. As a consequence, we obtain: (i) global-in-time well-posedness of weak measure-valued solutions, (ii) quantitative convergence rates to equilibrium, and (iii) uniform-in-time mean-field limits thanks to the above-mentioned equivalence and our recent work on fibered gradient flows.

math.AP↗

arXiv2022

Data Clustering as an Emergent Consensus of Autonomous Agents

We present a data segmentation method based on a first-order density-induced consensus protocol. We provide a mathematically rigorous analysis of the consensus model leading to the stopping criteria of the data segmentation algorithm. To illustrate our method, the algorithm is applied to two-dimensional shape datasets and selected images from Berkeley Segmentation Dataset. The method can be seen as an augmentation of classical clustering techniques for multimodal feature space, such as DBSCAN. It showcases a curious connection between data clustering and collective behavior.

cs.CV↗

arXiv2022

Heterogeneous gradient flows in the topology of fibered optimal transport

We introduce an optimal transport topology on the space of probability measures over a fiber bundle, which penalizes the transport cost from one fiber to another. For simplicity, we illustrate our construction in the Euclidean case $\mathbb{R}^d\times \mathbb{R}^d$, where we penalize the quadratic cost in the second component. Optimal transport becomes then constrained to happen along fixed fibers. Despite the degeneracy of the infinitely-valued and discontinuous cost, we prove that the space of probability measures $(\mathcal{P}_{2,\nu}(\mathbb{R}^{2d}),W_{2,\nu})$ with fixed marginal $\nu\in \mathcal{P}(\mathbb{R}^d)$ in the second component becomes a Polish space under the fibered transport distance, which enjoys a weak Riemannian structure reminiscent of the one proposed by F. Otto for the classical quadratic Wasserstein space. Three fundamental issues are addressed: 1) We develop an abstract theory of gradient flows with respect to the new topology; 2) We show applications that identify a novel fibered gradient flow structure on a large class of evolution PDEs with heterogeneities; 3) We exploit our method to derive long-time behavior and global-in-time mean-field limits in a multidimensional Cucker-Smale-type alignment model with weakly singular coupling.

math.AP↗

arXiv2020

Density-induced Consensus Protocol

The paper introduces a model of collective behavior where agents receive information only from sufficiently dense crowds in their immediate vicinity. The system is an asymmetric, density-induced version of the Cucker-Smale model with short-range interactions. We prove the basic mathematical properties of the system and concentrate on the presentation of interesting behaviors of the solutions. The results are illustrated by numerical simulations.

math.AP↗

arXiv2018

Flocking with short-range interactions

We study the large-time behavior of continuum alignment dynamics based on Cucker-Smale (CS)-type interactions which involve short-range kernels, that is, communication kernels with support much smaller than the diameter of the crowd. We show that if the amplitude of the interactions is larger than a finite threshold, then unconditional hydrodynamic flocking follows. Since we do not impose any regularity nor do we require the kernels to be bounded, the result covers both regular and singular interaction kernels. Moreover, we treat initial densities in the general class of compactly supported measures which are required to have positive mass on average (over balls at small enough scale), but otherwise vacuum is allowed at smaller scales. Consequently, our arguments of hydrodynamic flocking apply, mutatis mutandis, to the agent-based CS model with finitely many Dirac masses. In particular, discrete flocking threshold is shown to depend on the number of dense clusters of communication but otherwise does not grow with the number of agents.

math.AP↗

arXiv2018

Singular Cucker-Smale Dynamics

The existing state of the art for singular models of flocking is overviewed, starting from microscopic model of Cucker and Smale with singular communication weight, through its mesoscopic mean-filed limit, up to the corresponding macroscopic regime. For the microscopic Cucker-Smale (CS) model, the collision-avoidance phenomenon is discussed, also in the presence of bonding forces and the decentralized control. For the kinetic mean-field model, the existence of global-in-time measure-valued solutions, with a special emphasis on a weak atomic uniqueness of solutions is sketched. Ultimately, for the macroscopic singular model, the summary of the existence results for the Euler-type alignment system is provided, including existence of strong solutions on one-dimensional torus, and the extension of this result to higher dimensions upon restriction on the smallness of initial data. Additionally, the pressureless Navier-Stokes-type system corresponding to particular choice of alignment kernel is presented, and compared - analytically and numerically - to the porous medium equation.

math.AP↗

arXiv2018

A collisionless singular Cucker-Smale model with decentralized formation control

We address the design of decentralized feedback control laws inducing consensus and prescribed spatial patterns over a singular interacting particle system of Cucker-Smale type. The control design consists of a feedback term regulating the distance between each agent and pre-assigned subset of neighbours. Such a design represents a multidimensional extension of existing control laws for 1d platoon formation control. For the proposed controller we study consensus emergence, collision-avoidance and formation control features in terms of energy estimates for the closed-loop system. Numerical experiments in 1, 2 and 3 dimensions assess the different features of the proposed design.

math.OC↗

arXiv2018

Cucker-Smale model with a bonding force and a singular interaction kernel

We prove the lack of asymptotic collisions between particles following the Cucker-Smale flocking model with a bonding force and its simplification. Moreover, we prove that in the case of the CSB model with a singular communication weight, finite-in-time collisions are impossible. Consequently, we establish existence of the global-in-time minimal distance between the particles. Furthermore, we show that asymptotic distribution of particles is confined within a ball of a given radius.

math.DS↗

arXiv2018

Regular solutions to the fractional Euler alignment system in the Besov spaces framework

We here construct (large) local and small global-in-time regular unique solutions to the fractional Euler alignment system in the whole space ${\mathbb R}^d$, in the case where the deviation of the initial density from a constant is sufficiently small. Our analysis strongly relies on the use of Besov spaces of the type $L^1(0,T;\dot B^s_{p,1})$, which allow to get time independent estimates for the density even though it satisfies a transport equation with no damping. Our choice of a functional setting is not optimal but aims at providing a transparent and accessible argumentation.

math.AP↗

arXiv2016

Flocking particles in a non-Newtonian shear thickening fluid

We prove existence and uniqueness of strong solutions to the Cucker--Smale flocking model coupled with an incompressible viscous non-Newtonian fluid, with the stress tensor of a power--law structure for $p\geq\frac{11}{5}$. The coupling is performed through a drag force on a periodic spatial domain $T^3$.

math.AP↗

arXiv2016

Sharp conditions to avoid collisions in singular Cucker-Smale interactions

We consider the Cucker-Smale flocking model with a singular communication weight $\psi(s) = s^{-\alpha}$ with $\alpha > 0$. We provide a critical value of the exponent $\alpha$ in the communication weight leading to global regularity of solutions or finite-time collision between particles. For $\alpha \geq 1$, we show that there is no collision between particles in finite time if they are placed in different positions initially. For $\alpha\geq 2$ we investigate a version of the Cucker-Smale model with expanded singularity i.e. with weight $\psi_\delta(s) = (s-\delta)^{-\alpha}$, $\delta\geq 0$. For such model we provide a uniform with respect to the number of particles estimate that controls the $\delta$-distance between particles. In case of $\delta = 0$ it reduces to the estimate of non-collisioness.

math.DS↗