arXiv subjects

Alexey Voronin

Publications and source records attributed to Alexey Voronin.

5 recordsLinked to original sources

arXiv2026

On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime

Spectral bias, the tendency of neural networks to learn low frequencies first, can be both a blessing and a curse. While it enhances the generalization capabilities by suppressing high-frequency noise, it can be a limitation in scientific tasks that require capturing fine-scale structures. The delayed generalization phenomenon known as grokking is another barrier to rapid training of neural networks. Grokking has been hypothesized to arise as learning transitions from the NTK to the feature-rich regime. This paper explores the impact of preconditioned gradient descent (PGD), such as Gauss-Newton, on spectral bias and grokking phenomena. We demonstrate through theoretical and empirical results how PGD can mitigate issues associated with spectral bias. Additionally, building on the rich learning regime grokking hypothesis, we study how PGD can be used to reduce delays associated with grokking. Our conjecture is that PGD, without the impediment of spectral bias, enables uniform exploration of the parameter space in the NTK regime. Our experimental results confirm this prediction, providing strong evidence that grokking represents a transitional behavior between the lazy regime characterized by the NTK and the rich regime. These findings deepen our understanding of the interplay between optimization dynamics, spectral bias, and the phases of neural network learning.

cs.LG↗

arXiv2025

Generalized Optimal AMG Convergence Theory for Stokes Equations Using Smooth Aggregation and Vanka Relaxation Strategies

This paper discusses our recent generalized optimal algebraic multigrid (AMG) convergence theory applied to the steady-state Stokes equations discretized using Taylor-Hood elements ($\pmb{ \mathbb{P}}_2/\mathbb{P}_{1}$). The generalized theory is founded on matrix-induced orthogonality of the left and right eigenvectors of a generalized eigenvalue problem involving the system matrix and relaxation operator. This framework establishes a rigorous lower bound on the spectral radius of the two-grid error-propagation operator, enabling precise predictions of the convergence rate for symmetric indefinite problems, such as those arising from saddle-point systems. We apply this theory to the recently developed monolithic smooth aggregation AMG (SA-AMG) solver for Stokes, constructed using evolution-based strength of connection, standard aggregation, and smoothed prolongation. The performance of these solvers is evaluated using additive and multiplicative Vanka relaxation strategies. Additive Vanka relaxation constructs patches algebraically on each level, resulting in a nonsymmetric relaxation operator due to the partition of unity being applied on one side of the block-diagonal matrix. Although symmetry can be restored by eliminating the partition of unity, this compromises convergence. Alternatively, multiplicative Vanka relaxation updates velocity and pressure sequentially within each patch, propagating updates multiplicatively across the domain and effectively addressing velocity-pressure coupling, ensuring a symmetric relaxation. We demonstrate that the generalized optimal AMG theory consistently provides accurate lower bounds on the convergence rate for SA-AMG applied to Stokes equations. These findings suggest potential avenues for further enhancement in AMG solver design for saddle-point systems.

math.NA↗

arXiv2024

Monolithic Algebraic Multigrid Preconditioners for the Stokes Equations

We investigate a novel monolithic algebraic multigrid (AMG) preconditioner for the Taylor-Hood ($\pmb{\mathbb{P}}_2/\mathbb{P}_1$) and Scott-Vogelius ($\pmb{\mathbb{P}}_2/\mathbb{P}_1^{disc}$) discretizations of the Stokes equations. The algorithm is based on the use of the lower-order $\pmb{\mathbb{P}}_1\text{iso}\kern1pt\pmb{\mathbb{P}}_2/\mathbb{P}_1$ operator within a defect-correction setting, in combination with AMG construction of interpolation operators for velocities and pressures. The preconditioning framework is primarily algebraic, though the $\pmb{\mathbb{P}}_1\text{iso}\kern1pt\pmb{\mathbb{P}}_2/\mathbb{P}_1$ operator must be provided. We investigate two relaxation strategies in this setting. Specifically, a novel block factorization approach is devised for Vanka patch systems, which significantly reduces storage requirements and computational overhead, and a Chebyshev adaptation of the LSC-DGS relaxation is developed to improve parallelism. The preconditioner demonstrates robust performance across a variety of 2D and 3D Stokes problems, often matching or exceeding the effectiveness of an inexact block-triangular (or Uzawa) preconditioner, especially in challenging scenarios such as elongated-domain problems.

math.NA↗

arXiv2024

Monolithic Multigrid Preconditioners for High-Order Discretizations of Stokes Equations

This work introduces and assesses the efficiency of a monolithic $ph$MG multigrid framework designed for high-order discretizations of stationary Stokes systems using Taylor-Hood and Scott-Vogelius elements. The proposed approach integrates coarsening in both approximation order ($p$) and mesh resolution ($h$), to address the computational and memory efficiency challenges that are often encountered in conventional high-order numerical simulations. Our numerical results reveal that $ph$MG offers significant improvements over traditional spatial-coarsening-only multigrid ($h$MG) techniques for problems discretized with Taylor-Hood elements across a variety of problem sizes and discretization orders. In particular, the $ph$MG method exhibits superior performance in reducing setup and solve times, particularly when dealing with higher discretization orders and unstructured problem domains. For Scott-Vogelius discretizations, while monolithic $ph$MG delivers low iteration counts and competitive solve phase timings, it exhibits a discernibly slower setup phase when compared to a multilevel (non-monolithic) full-block-factorization (FBF) preconditioner where $ph$MG is employed only for the velocity unknowns. This is primarily due to the setup costs of the larger mixed-field relaxation patches with monolithic $ph$MG versus the patch setup costs with a single unknown type for FBF.

math.NA↗

arXiv2021

Low-order preconditioning of the Stokes equations

A well-known strategy for building effective preconditioners for higher-order discretizations of some PDEs, such as Poisson's equation, is to leverage effective preconditioners for their low-order analogs. In this work, we show that high-quality preconditioners can also be derived for the Taylor-Hood discretization of the Stokes equations in much the same manner. In particular, we investigate the use of geometric multigrid based on the $\boldsymbol{ \mathbb{Q}}_1iso\boldsymbol{ \mathbb{Q}}_2/ \mathbb{Q}_1$ discretization of the Stokes operator as a preconditioner for the $\boldsymbol{ \mathbb{Q}}_2/\mathbb{Q}_1$ discretization of the Stokes system. We utilize local Fourier analysis to optimize the damping parameters for Vanka and Braess-Sarazin relaxation schemes and to achieve robust convergence. These results are then verified and compared against the measured multigrid performance. While geometric multigrid can be applied directly to the $\boldsymbol{ \mathbb{Q}}_2/\mathbb{Q}_1$ system, our ultimate motivation is to apply algebraic multigrid within solvers for $\boldsymbol{ \mathbb{Q}}_2/\mathbb{Q}_1$ systems via the $\boldsymbol{ \mathbb{Q}}_1iso\boldsymbol{ \mathbb{Q}}_2/ \mathbb{Q}_1$ discretization, which will be considered in a companion paper.

math.NA↗