Making Gaussian Kolmogorov-Arnold Networks Reliable and Accurate

Abstract

Kolmogorov-Arnold Networks (KANs) replace fixed activations with learnable univariate edge functions whose behavior depends strongly on the chosen basis. Gaussian radial basis functions provide a simple and efficient alternative to splines, but their accuracy and stability are highly sensitive to the scale parameter \(\epsilon\), which has not been studied systematically. We analyze this dependence through the geometry and conditioning of the first-layer feature matrix. Because the first layer is defined directly on the input domain, any loss of feature distinguishability introduced there propagates through the entire network. Based on this analysis, we identify the practical operating interval \[ \epsilon \in \left[\frac{1}{G-1},\frac{2}{G-1}\right], \] where \(G\) is the number of Gaussian centers. This interval is proposed as a stable design rule rather than a universal optimum. Extensive experiments on function approximation and physics-informed problems confirm its reliability across different collocation densities, grid resolutions, architectures, and input dimensions. We also show how the same principle supports fixed-scale selection, variable-scale models, constrained optimization of \(\epsilon\), and efficient scale search using early-stage training error. These results establish scale selection as a central design principle for reliable and accurate Gaussian KANs.