Optimal Linear Baseline Models for Scientific Machine Learning

Abstract

Across scientific domains, a fundamental challenge is to characterize and compute the mappings from underlying physical processes to observed signals and measurements. While nonlinear neural networks have achieved considerable success, they remain theoretically opaque, which hinders adoption in contexts where interpretability is paramount. In contrast, linear neural networks serve as a simple yet effective foundation for gaining insight into these complex relationships. In this work, we develop a unified theoretical framework for analyzing linear encoder-decoder architectures through the lens of Bayes risk minimization for solving data-driven scientific machine learning problems. We derive closed-form, rank-constrained linear and affine linear optimal mappings for forward modeling and inverse recovery tasks. Our results generalize existing formulations by accommodating rank-deficiencies in data, forward operators, and measurement processes. We validate our theoretical results by conducting numerical experiments on datasets from simple biomedical imaging, financial factor analysis, and simulations involving nonlinear fluid dynamics via the shallow water equations. This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.