Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction

Abstract

Mean-squared error is the default objective for training autoencoders, yet compressed reconstructions often depend not only on pointwise accuracy but also on preserving local spatial order. We study whether structural auxiliary losses can improve, rather than trade off against, MSE in finite-capacity autoencoders. We introduce finite-difference sign error (FDSE), a local-order auxiliary objective that penalizes disagreements between the signs of neighboring finite differences in the target and reconstruction. FDSE is simple, architecture-agnostic, and differentiable through smooth sign surrogates. Across four tensor reconstruction tasks, we find that moderate mixtures of MSE and FDSE can substantially reduce validation MSE relative to pure MSE training. In coefficient sweeps, FDSE mixtures reduce validation MSE by 2.3$\times$--7.0$\times$ over pure MSE on these tasks, while comparisons with other auxiliary objectives show FDSE to be among the strongest structural objectives tested. The effect is not universal: pure FDSE performs poorly, and gains are largest for coherent spatial fields where local order carries information about the underlying signal. These results suggest that, in compressed-latent reconstruction, appropriately weighted local-structure supervision can guide optimization toward solutions with better pointwise accuracy, rather than merely improving perceptual or structural metrics at MSE's expense.