arXiv subjects

Peio Ibarrondo

Publications and source records attributed to Peio Ibarrondo.

3 recordsLinked to original sources

arXiv2026

Is Stochastic Gradient Descent Effective? A PDE Perspective on Machine Learning processes

In this paper we analyze the behaviour of the stochastic gradient descent (SGD), a widely used method in supervised learning for optimizing neural network weights via a minimization of non-convex loss functions. Since the pioneering work of E, Li and Tai (2017), the underlying structure of such processes can be understood via parabolic PDEs of Fokker-Planck type, which are at the core of our analysis. Even if Fokker-Planck equations have a long history and a extensive literature, almost nothing is known when the potential is non-convex or when the diffusion matrix is degenerate, and this is the main difficulty that we face in our analysis. We identify two different regimes: in the initial phase of SGD, the loss function drives the weights to concentrate around the nearest local minimum. We refer to this phase as the drift regime and we provide quantitative estimates on this concentration phenomenon. Next, we introduce the diffusion regime, where stochastic fluctuations help the learning process to escape suboptimal local minima. We analyze the Mean Exit Time (MET) and prove upper and lower bounds of the MET. Finally, we address the asymptotic convergence of SGD, for a non-convex cost function and a degenerate diffusion matrix, that do not allow to use the standard approaches, and require new techniques. For this purpose, we exploit two different methods: duality and entropy methods. We provide new results about the dynamics and effectiveness of SGD, offering a deep connection between stochastic optimization and PDE theory, and some answers and insights to basic questions in the Machine Learning processes: How long does SGD take to escape from a bad minimum? Do neural network parameters converge using SGD? How do parameters evolve in the first stage of training with SGD?

cs.LG↗

arXiv2024

Time-Fractional Porous Medium Type Equations. Sharp Time Decay and Regularization

We consider a class of porous medium type of equations with Caputo time derivative. The prototype problem reads as $\Dc u=-\A u^m$ and is posed on a bounded Euclidean domain $Ω\subset\mathbb{R}^N$ with zero Dirichlet boundary conditions. The operator $\A$ falls within a wide class of either local or nonlocal operators, and the nonlinearity is allowed to be of degenerate or singular type, namely, $0 1$. This equation is the most general form of a variety of models used to describe anomalous diffusion processes with memory effects, and finds application in various fields, including visco-elastic materials, signal processing, biological systems and geophysical science. We show existence of unique solution and new $L^p-L^\infty$ smoothing effects. The comparison principle, which we provide in the most general setting, serves as a crucial tool in the proof and provides a novel monotonicity formula. Consequently, we establish that the regularizing effects from the diffusion are stronger than the memory effects introduced by the fractional time derivative. Moreover, the solution attains the boundary conditions pointwise. Finally, we prove that the solution does not vanish in finite time if $0 0$. Our findings indicate that memory effects weaken the spatial diffusion and mitigate the difference between slow and fast diffusion.

math.AP↗

arXiv2022

The Cauchy-Dirichlet Problem for Singular Nonlocal Diffusions on Bounded Domains

We study the homogeneous Cauchy-Dirichlet Problem (CDP) for a nonlinear and nonlocal diffusion equation of singular type of the form $\partial_t u =-\mathcal{L} u^m$ posed on a bounded Euclidean domain $Ω\subset\mathbb{R}^N$ with smooth boundary and $N\ge 1$. The linear diffusion operator $\mathcal{L}$ is a sub-Markovian operator, allowed to be of nonlocal type, while the nonlinearity is of singular type, namely $u^m=|u|^{m-1}u$ with $0<m<1$. The prototype equation is the Fractional Fast Diffusion Equation (FFDE), when $\mathcal{L}$ is one of the three possible Dirichlet Fractional Laplacians on $Ω$. Our main results shall provide a complete basic theory for solutions to (CDP): existence and uniqueness in the biggest class of data known so far, both for nonnegative and signed solutions; sharp smoothing estimates: besides the classical $L^p-L^\infty$ smoothing effects, we provide new weighted estimates, which represent a novelty also in well studied local case, i.e. for solutions to the FDE $u_t=Δu^m$. We compare two strategies to prove smoothing effects: Moser iteration VS Green function method. Due to the singular nonlinearity and to presence of nonlocal diffusion operators, the question of how solutions satisfy the lateral boundary conditions is delicate. We answer with quantitative upper boundary estimates that show how boundary data are taken. Once solutions exists and are bounded we show that they extinguish in finite time and we provide upper and lower estimates for the extinction time, together with explicit sharp extinction rates in different norms. The methods of this paper are constructive, in the sense that all the relevant constants involved in the estimates are computable.

math.AP↗