arXiv subjects

Leonid Berlyand

Publications and source records attributed to Leonid Berlyand.

At least 19 recordsLinked to original sources

arXiv2026

Randomly initialized autoencoders: fixed points and edge-of-chaos

In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.

cs.LG↗

arXiv2026

Change of bifurcation type in 2D free boundary model of a moving cell with nonlinear diffusion

We introduce a 2D free boundary problem with nonlinear diffusion that models a living cell moving on a substrate. We prove that this nonlinearity results in a qualitative of solution behavior compared to the linear diffusion case (Rybalko et al. TAMS 2023), namely the switch between direct and inverse pitchfork bifurcation. Our objectives are twofold: (i) develop a rigorous framework to prove existence of bifurcation and determine its type (subcritical vs. superctitical) and (ii) the derivation of explicit analytical formulas that control the change of bifurcation type in terms of physical parameters and explain the underlying biophysical mechanisms. While the standard way of applying the Crandall-Rabinowitz theorem via the solution operator seems difficult in our quasilinear PDE system, we apply the theorem directly, by developing a multidimensional, vectorial framework. To determine the bifurcation type, we extract the curvature of the bifurcating curve from the expansion of the solutions around the steady state. The formula for the curvature is obtained via a solvability condition where instead of the Fredholm alternative, we propose a test function trick, suited for free boundary problems. Our rigorous analytical results are in agreement with numerical observations from the physical literature in 1D (Drozdowski et al. Comm. Phys. 2023) and provide the first extension of this phenomenon to a 2D free boundary model.

math.AP↗

arXiv2026

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution

We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets. The main practical contribution is accuracy retention under short calibration and fine-tuning schedules, rather than a long post-pruning reoptimization pipeline. The theory gives deterministic data-path certificates: if the removed component $R$ has small propagated logit effect $L_s \| R ψ_1(s) \|_\infty$, pruning decreases an elastic-net objective and preserves samples whose dense margin exceeds twice the perturbation. The zero-budget case gives perfect pruning; a prune--restore extension models weight restoration inside a fixed sparse-execution pattern; and an additive $L_2$-regularized model shows admissible random-like components vanish at the training limit, with persistent spikes stabilizing as the MP bulk collapses. Under iid-Gaussian sufficient conditions, the fitted MP edge $σ_+$ gives a high-probability layerwise budget signal. On ImageNet-1k, after only three distillation epochs, ViT-B/16 $2{:}4{+}$ToMe reaches $83.41\%$ top-1 ($-1.70$ pp from dense) at $59.81\%$ sparse-execution MAC reduction, with $1.388\times$ best-observed A40 native-$2{:}4$ backend speedup for the same checkpoint and ToMe graph; a separate no-ToMe A100 endpoint gives $2.705\times$. At structured sparsity, ViT-B/16 $6{:}12$ reaches $83.74\%$, ViT-L/16 $8{:}16$ dense+permutation reaches $85.33\%$ ($-0.51$ pp), and ConvNeXtV2-Base $12{:}16$ reaches $86.35\%$ ($-0.37$ pp). For CNNs, ResNet50 $8{:}16$ dense+permutation reaches $75.87\%$ ($-0.26$ pp), and ResNet152d CAST-conv+permutation reaches $81.33\%$ ($-1.53$ pp) at ${\sim}50\%$ MAC accounting with a $1.62\times$ A40 im2col$+2{:}4$ sparse-GEMM audit.

cs.LG↗

arXiv2026

Asymptotic behavior of eigenvalues of large rank perturbations of large random matrices

The paper is concerned with deformed Wigner random matrices. These matrices are closely related to Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly correlated. The spectrum of such matrices plays a key role in rigorous underpinning of the novel pruning technique based on Random Matrix Theory. In practice, the spectrum of the matrix $S$ can be rather complicated. In this paper, we develop an asymptotic analysis for the case of full rank $S$ with increasing number of outlier eigenvalues.

math-ph↗

arXiv2025

Multiscale Analysis of a Kinetic Model of Confined Suspensions of Self-Propelled Rods

The behavior of active matter under confinement poses significant challenges due to the intricate coupling between dynamics near boundaries and those in the bulk. A defining feature of active matter systems is that a substantial portion of their dynamics takes place near confining boundaries. In our previous work, we developed a kinetic framework that enables direct computation of the probability distribution functions for both the position and orientation of active rods. A distinguishing aspect of this approach is its explicit treatment of wall accumulation through the use of two coupled probability distribution functions: one describing the bulk population and the other representing rods accumulated at the boundary. Another novel feature is the structure of the governing equation, which is degenerate: it is second-order in one non-temporal variable and first-order in another. The main focus of this paper is to rigorously justify this model via multi-scale analysis. We first establish well-posedness of the system and then employ two distinct multi-scale derivations to obtain the model as a singular limit of a more classical kinetic system in the regime of vanishing translational diffusion. For analytical clarity, we consider the case in which active rods, once accumulated at the wall, remain permanently confined there. This work provides a rigorous mathematical foundation for reduced kinetic models of confined active matter, bridging microscopic dynamics and macroscopic accumulation phenomena.

math.AP↗

arXiv2025

Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization

Deep neural networks (DNNs) have brought significant advancements in various applications in recent years, such as image recognition, speech recognition, and natural language processing. In particular, Vision Transformers (ViTs) have emerged as a powerful class of models in the field of deep learning for image classification. In this work, we propose a novel Random Matrix Theory (RMT)-based method for pruning pre-trained DNNs, based on the sparsification of weights and singular vectors, and apply it to ViTs. RMT provides a robust framework to analyze the statistical properties of large matrices, which has been shown to be crucial for understanding and optimizing the performance of DNNs. We demonstrate that our RMT-based pruning can be used to reduce the number of parameters of ViT models (trained on ImageNet) by 30-50\% with less than 1\% loss in accuracy. To our knowledge, this represents the state-of-the-art in pruning for these ViT models. Furthermore, we provide a rigorous mathematical underpinning of the above numerical studies, namely we proved a theorem for fully connected DNNs, and other more general DNN structures, describing how the randomness in the weight matrices of a DNN decreases as the weights approach a local or global minimum (during training). We verify this theorem through numerical experiments on fully connected DNNs, providing empirical support for our theoretical findings. Moreover, we prove a theorem that describes how DNN loss decreases as we remove randomness in the weight layers, and show a monotone dependence of the decrease in loss with the amount of randomness that we remove. Our results also provide significant RMT-based insights into the role of regularization during training and pruning.

cs.LG↗

arXiv2024

Nonlinear stability in a free boundary model of active locomotion

Contraction-driven self-propulsion of a large class of living cells can be modeled by a Keller-Segel system with free boundaries. The ensuing "active" system, exhibiting both dissipation and anti-dissipation, features stationary and traveling wave solutions. While the former represent static cells, the latter describe propagating pulses (solitary waves) mimicking the autonomous locomotion of the same cells. In this paper we provide the first proof of the asymptotic nonlinear stability of both of such solutions, static and dynamic. In the case of stationary solutions, the linear stability is established using the spectral theorem for compact, self-adjoint operators, and thus linear stability is determined classically, solely by eigenvalues. For traveling waves the picture is more complex because the linearized problem is non-self-adjoint, opening the possibility of a "dark" area in the phase space which is not "visible" in the purely eigenvalue/eigenvector approach. To establish linear stability in this case we employ spectral methods together with the Gearhart-Pruss-Greiner (GPG) theorem, which controls the entire spectrum via bounds on the resolvent operator. For both stationary and traveling wave solutions, nonlinear stability is then proved by showing how the nonlinear part of the problem may be dominated by the linear part and then employing a Gronwall inequality argument. The developed novel methodology can prove useful also in other problems involving non-self-adjoint (non-Hermitian or non-reciprocal) operators which are ubiquitous in the modeling of "active" matter.

math.AP↗

arXiv2024

Enhancing Accuracy in Deep Learning Using Random Matrix Theory

We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs), focusing on layer pruning that is reducing the number of DNN parameters (weights). Our numerical results show that this pruning leads to a drastic reduction of parameters while not reducing the accuracy of DNNs and CNNs. Moreover, pruning the fully connected DNNs actually increases the accuracy and decreases the variance for random initializations. Our numerics indicate that this enhancement in accuracy is due to the simplification of the loss landscape. We next provide rigorous mathematical underpinning of these numerical results by proving the RMT-based Pruning Theorem. Our results offer valuable insights into the practical application of RMT for the creation of more efficient and accurate deep-learning models.

cs.LG↗

arXiv2023

Bifurcation of finger-like structures in traveling waves of epithelial tissues spreading

We consider a continuum active polar fluid model for the spreading of epithelial monolayers introduced by R. Alert, C. Blanch-Mercader, and J. Casademunt, 2019. The corresponding free boundary problem possesses flat front traveling wave solutions. Linear stability of these solutions under periodic perturbations is considered. It is shown that the solutions are stable for short-wave perturbations while exhibiting long-wave instability under certain conditions on the model parameters (if the traction force is sufficiently strong). Then, considering the prescribed period as the bifurcation parameter, we establish the emergence of nontrivial traveling wave solutions with a finger-like periodic structure (pattern). We also construct asymptotic expansions of the solutions in the vicinity of the bifurcation point and study their stability. We show that, depending on the value of the contractility coefficient, the bifurcation can be a subcritical or a supercritical pitchfork.

math.AP↗

arXiv2022

Dynamics and steady state of squirmer motion in liquid crystal

We analyze a nonlinear PDE system describing the motion of a microswimmer in a nematic liquid crystal environment. For the microswimmer's motility, the squirmer model is used in which self-propulsion enters the model through the slip velocity on the microswimmer's surface. The liquid crystal is described using the well-established Beris-Edwards formulation. In previous computational studies, it was shown that the squirmer, regardless of its initial configuration, eventually orients itself either parallel or perpendicular to the preferred orientation dictated by the liquid crystal. Furthermore, the corresponding solution of the coupled nonlinear system converges to a steady state. In this work, we rigorously establish the existence of steady state and also the finite-time existence for the time-dependent problem. Finally, using a two-scale asymptotic expansion we derive a reduced model for the collective swimming of squirmers as they reach their steady state orientation and speed.

math.AP↗

arXiv2021

A novel multi-scale loss function for classification problems in machine learning

We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.

math.NA↗

arXiv2021

Emergence of traveling waves and their stability in a free boundary model of cell motility

We introduce a two-dimensional Hele-Shaw type free boundary model for motility of eukaryotic cells on substrates. The key ingredients of this model are the Darcy law for overdamped motion of the cytoskeleton gel (active gel) coupled with advection-diffusion equation for myosin density leading to elliptic-parabolic Keller-Segel system. This system is supplemented with Hele-Shaw type boundary conditions: Young-Laplace equation for pressure and continuity of velocities. We first show that radially symmetric stationary solutions become unstable and bifurcate to traveling wave solutions at a critical value of the total myosin mass. Next we perform linear stability analysis of these traveling wave solutions and identify the type of bifurcation (sub- or supercritical). Our study sheds light on the mathematics underlying instability/stability transitions in this model. Specifically, we show that these transitions occur via generalized eigenvectors of the linearized operator.

math.AP↗

arXiv2021

Stability of Contraction-Driven Cell Motion

We consider motility of keratocyte cells driven by myosin contraction and introduce a 2D free boundary model for such motion. This model generalizes a 1D model from [12] by combining a 2D Keller-Segel model and a Hele-Shaw type boundary condition with the Young-Laplace law resulting in a boundary curvature term which provides a regularizing effect. We show that this model has a family of traveling solutions with constant shape and velocity which bifurcates from a family of radially symmetric stationary states. Our goal is to establish observable steady motion of the cell with constant velocity. Mathematically, this amounts to establishing stability of the traveling solutions. Our key result is an explicit asymptotic formula for the stability-determining eigenvalue of the linearized problem. This formula greatly simplifies the task of numerically computing the sign of this eigenvalue and reveals the physical mechanisms of stability. The derivation of this formula is based on a special ansatz for the corresponding eigenvector which exhibits an interesting singular behavior such that it asymptotically (in the small-velocity limit) becomes parallel to another eigenvector. This reflects the non-self-adjoint nature of the linearized problem, a signature of living systems. Finally, our results describe the onset of motion via a transition from unstable radial stationary solutions to stable asymmetric traveling solutions.

physics.bio-ph↗

arXiv2020

Stability for the Training of Deep Neural Networks and Other Classifiers

We examine the stability of loss-minimizing training processes that are used for deep neural networks (DNN) and other classifiers. While a classifier is optimized during training through a so-called loss function, the performance of classifiers is usually evaluated by some measure of accuracy, such as the overall accuracy which quantifies the proportion of objects that are well classified. This leads to the guiding question of stability: does decreasing loss through training always result in increased accuracy? We formalize the notion of stability, and provide examples of instability. Our main result consists of two novel conditions on the classifier which, if either is satisfied, ensure stability of training, that is we derive tight bounds on accuracy as loss decreases. We also derive a sufficient condition for stability on the training set alone, identifying flat portions of the data manifold as potential sources of instability. The latter condition is explicitly verifiable on the training dataset. Our results do not depend on the algorithm used for training, as long as loss decreases with training.

math.AP↗

arXiv2019

Stability of steady states and bifurcation to traveling waves in a free boundary model of cell motility

We introduce a two-dimensional Keller-Segel type free boundary model for motility of eukaryotic cells on substrates. The key ingredients of this model are the Darcy law for overdamped motion of the cytoskeleton (active) gel and Hele-Shaw type boundary conditions (Young-Laplace equation for pressure and continuity of velocities). We first show that radially symmetric steady state solutions become unstable and bifurcate to traveling wave solutions. Next we establish linear and nonlinear stability of the steady states. We show that linear stability analysis is inconclusive for both steady states and traveling waves. Therefore we use invariance properties to prove nonlinear stability of steady states.

math.AP↗

arXiv2019

A kinetic approach to active rod dynamics in confined domains

The study of active matter consisting of many self-propelled (active) swimmers in an imposed flow is important for many applications. Self-propelled swimmers may represent both living and artificial ones such as bacteria and chemically driven bi-metallic nano-particles. In this work we focus on a kinetic description of active matter represented by self-propelled rods swimming in a viscous fluid confined by a wall. It is well-known that walls may significantly affect the trajectories of active rods in contrast to unbounded or periodic containers. Among such effects are accumulation at walls and upstream motion (also known as negative rheotaxis). Our first main result is the rigorous derivation of boundary conditions for the active rods' probability distribution function in the limit of vanishing inertia. Finding such a limit is important due to (i) the fact that in many examples of active matter inertia is negligible, since swimming occurs in a low Reynolds number regime, and (ii) this limit allows us to reduce the dimension - and so computational complexity - of the kinetic description. For the resulting model, we derive the system in the limit of vanishing translational diffusion which is also typically negligible for active particles. This system allows for tracking separately active particles accumulated at walls and active particles swimming in the bulk of the fluid.

cond-mat.soft↗

arXiv2019

Existence of superconducting solutions for a reduced Ginzburg-Landau model in the presence of strong electric currents

In this work we consider a reduced Ginzburg-Landau model in which the magnetic field is neglected and the magnitude of the current density is significantly stronger than that considered in a recent work by the same authors. We prove the existence of a solution which can be obtained by solving a non-convex minimization problem away from the boundary of the domain. Near the boundary, we show that this solution is essentially one-dimensional. We also establish some linear stability results for a simplified, one-dimensional version of the original problem.

math-ph↗

arXiv2018

Continuum approximations to systems of correlated interacting particles

We consider a system of interacting particles with random initial conditions. Continuum approximations of the system, based on truncations of the BBGKY hierarchy, are described and simulated for various initial distributions and types of interaction. Specifically, we compare the Mean Field Approximation (MFA), the Kirkwood Superposition Approximation (KSA), and a recently developed truncation of the BBGKY hierarchy (the Truncation Approximation - TA). We show that KSA and TA perform more accurately than MFA in capturing approximate distributions (histograms) obtained from Monte Carlo simulations. Furthermore, TA is more numerically stable and less computationally expensive than KSA.

cond-mat.stat-mech↗