arXiv subjects

Dongwook Shin

Publications and source records attributed to Dongwook Shin.

3 recordsLinked to original sources

arXiv2026

Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations

During SGD training, the gradients often align strongly with the dominant subspace spanned by the top-$k$ eigenvectors of the Hessian of the loss. While this seems to naturally imply that loss reduction mainly occurs within this space, prior work has shown that updates within this dominant subspace make no meaningful progress in reducing the loss. In this work, we argue that the dominant subspace is better understood not as the main space for loss reduction, but as a key subspace for explaining the sharpness dynamics of mini-batch SGD. To explain the role of the dominant subspace in reducing top-$k$ sharpness, we show how the averaged gradient over fluctuations in the dominant directions produces a sharpness correction term, and derive a sharpness correction term induced by mini-batch noise in the dominant directions. Experimental results show that adding the derived correction term to GD brings the sharpness evolution of GD closer to that of SGD.

cs.LG↗

arXiv2026

A residual-based finite element surrogate solver for elliptic partial differential equations

We propose a residual-based finite element surrogate solver for elliptic partial differential equations. The method combines convolutional neural networks with classical finite element discretization in a data-free setting, where the loss function is defined directly from the finite element residual. This enables the approximation of the solution operator without requiring paired input-output data. A key feature of the proposed approach is that it can be analyzed using standard finite element theory under mesh refinement. We establish a relationship between the training loss and the error in the $H^1$-seminorm, and derive training criteria that ensure optimal convergence rates. To improve efficiency, we introduce a decomposition strategy that separates the contributions of different input components. This allows the model to learn simpler sub-operators. Numerical experiments demonstrate that the proposed method achieves stable convergence under grid refinement, remains robust on complex geometries and oscillatory solutions, and extends naturally to nonlinear equations.

math.NA↗

arXiv2026

Sparse FEONet: A Low-Cost, Memory-Efficient Operator Network via Finite-Element Local Sparsity for Parametric PDEs

In this paper, we study the finite element operator network (FEONet), an operator-learning method for parametric problems, originally introduced in J. Y. Lee, S. Ko, and Y. Hong, Finite Element Operator Network for Solving Elliptic-Type Parametric PDEs, SIAM J. Sci. Comput., 47(2), C501-C528, 2025. FEONet realizes the parameter-to-solution map on a finite element space and admits a training procedure that does not require training data, while exhibiting high accuracy and robustness across a broad class of problems. However, its computational cost increases and accuracy may deteriorate as the number of elements grows, posing notable challenges for large-scale problems. In this paper, we propose a new sparse network architecture motivated by the structure of the finite elements to address this issue. Throughout extensive numerical experiments, we show that the proposed sparse network achieves substantial improvements in computational cost and efficiency while maintaining comparable accuracy. We also establish theoretical results demonstrating that the sparse architecture can approximate the target operator effectively and provide a stability analysis ensuring reliable training and prediction.

math.NA↗