arXiv subjects

Przemysław Spyra

Publications and source records attributed to Przemysław Spyra.

3 recordsLinked to original sources

arXiv2026

Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Hardware-Validated Benchmarking Study of Forward-Only Algorithms

Backpropagation (BP) powers modern deep learning, but it must hold every activation in memory until a global backward sweep consumes it, and that sweep locks each layer until the one above has finished. Forward-only algorithms, from Hinton's Forward-Forward (FF) through Cascaded Forward (CaFo) to Mono-Forward (MF), replace the global gradient with local learning signals and are promoted as more efficient on the basis of FLOP counts rather than measurements. We test that claim on hardware. Across almost 1000 runs on four vision benchmarks (MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100), we read energy, power and memory from an NVIDIA A100 GPU and compare each algorithm against a tuned BP baseline. Removing the backward pass does not by itself save energy: FF costs 4.1x to 6.3x BP's energy and CaFo 3.0x to 7.1x. Mono-Forward is the exception. Given the same number of data passes as BP, it uses 12.5% to 19.9% less energy and 24.6% to 27.5% less memory on the 3x2000 multilayer perceptrons, the largest configurations tested, at lower power and near-equal wall-clock time; on CIFAR-100 its test accuracy is statistically equivalent to BP's. A pre-registered ablation identifies gradient detachment, not MF's auxiliary losses, as the dominant factor tested. Trained to convergence, MF exceeds BP's test accuracy on CIFAR-10 and CIFAR-100 by 3.90 and 5.04 percentage points, but needs 3.5x to 4.5x as many passes and 3.2x to 4.0x the energy. Faster convergence is therefore the remaining target for an end-to-end gain.

cs.LG↗

arXiv2026

Algorithmic Stability in Infinite Dimensions: Characterizing Unconditional Convergence in Banach Spaces

The distinction between conditional, unconditional, and absolute convergence in infinite-dimensional spaces has fundamental implications for computational algorithms. While these concepts coincide in finite dimensions, the Dvoretzky-Rogers theorem establishes their strict separation in general Banach spaces. We present a comprehensive characterization theorem unifying seven equivalent conditions for unconditional convergence: permutation invariance, net convergence, subseries tests, sign stability, bounded multiplier properties, and weak uniform convergence. These theoretical results directly inform algorithmic stability analysis, governing permutation invariance in gradient accumulation for Stochastic Gradient Descent and justifying coefficient thresholding in frame-based signal processing. Our work bridges classical functional analysis with contemporary computational practice, providing rigorous foundations for order-independent and numerically robust summation processes.

cs.CL↗

arXiv2025

Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training

The rising computational and energy demands of deep neural networks (DNNs), driven largely by backpropagation (BP), challenge sustainable AI development. This paper rigorously investigates three BP-free training methods: the Forward-Forward (FF), Cascaded-Forward (CaFo), and Mono-Forward (MF) algorithms, tracing their progression from foundational concepts to a demonstrably superior solution. A robust comparative framework was established: each algorithm was implemented on its native architecture (MLPs for FF and MF, a CNN for CaFo) and benchmarked against an equivalent BP-trained model. Hyperparameters were optimized with Optuna, and consistent early stopping criteria were applied based on validation performance, ensuring all models were optimally tuned before comparison. Results show that MF not only competes with but consistently surpasses BP in classification accuracy on its native MLPs. Its superior generalization stems from converging to a more favorable minimum in the validation loss landscape, challenging the assumption that global optimization is required for state-of-the-art results. Measured at the hardware level using the NVIDIA Management Library (NVML) API, MF reduces energy consumption by up to 41% and shortens training time by up to 34%, translating to a measurably smaller carbon footprint as estimated by CodeCarbon. Beyond this primary result, we present a hardware-level analysis that explains the efficiency gains: exposing FF's architectural inefficiencies, validating MF's computationally lean design, and challenging the assumption that all BP-free methods are inherently more memory-efficient. By documenting the evolution from FF's conceptual groundwork to MF's synthesis of accuracy and sustainability, this work offers a clear, data-driven roadmap for future energy-efficient deep learning.

cs.LG↗