Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs

Abstract

Solving high-frequency partial differential equations (PDEs) with neural networks is notoriously difficult due to the spectral bias of conventional architectures. We propose the Multi-Scale Separable Fourier Neural Network (MS-SFNN), a framework designed to overcome this limitation by explicitly encoding multi-scale Fourier features within a separable representation. The network factorizes the solution into $d$ single-coordinate subnetworks with fixed, randomly initialized weights; these subnetworks are combined via element-wise products to form a rich set of basis functions. This separable construction scales linearly with the problem dimension, thus inherently alleviating the curse of dimensionality. Crucially, each subnetwork is equipped with trainable scaling factors coupled with cosine activations, providing an adaptive mechanism for multi-scale frequency selection that endows the model with strong spectral approximation capability. The PDE solution is expressed as a linear combination of these learned basis functions, and the combination coefficients are determined by solving a large-scale least-squares system. To resolve the memory bottleneck in high-frequency or three-dimensional settings, we replace automatic differentiation (AD) with analytical derivatives of the basis functions and use a memory-efficient batched QR decomposition for solving the least-squares systems efficiently. Extensive numerical experiments demonstrate that MS-SFNN achieves superior accuracy and substantially outperforms state-of-the-art methods, including Physics-Informed Neural Network (PINN) and the Separated-Variable Spectral Neural Network (SV-SNN).