arXiv subjects

George Dimitriadis

Publications and source records attributed to George Dimitriadis.

2 recordsLinked to original sources

arXiv2026

Rethinking GA2Ms: performance and out-of-distribution generalisation

Generalised Additive Models (GAMs) are a cornerstone of interpretable machine learning, offering transparent predictions through structured additive decompositions. Despite advances from classical GAMs to modern neural and spline-based variants, two notable gaps remain. First, comprehensive benchmarking of spline-based GAMs on large-scale tabular learning tasks, the dominant modality in applied settings, is lacking. Second, their robustness under distribution shift, a setting where structured inductive biases should confer advantages, remains unexplored. We propose a multi-resolution, higher-order Composite Generalised Additive Model (CoGA2M) and evaluate it on the TALENT benchmark suite. CoGA2M ranks in the top third across all baselines (29 for regression, 32 for classification), establishing competitive performance with state-of-the-art tabular methods. We further extend TALENT with out-of-distribution (OOD) splits exhibiting train-test distributional shift and benchmark CoGA2M against LightGBM, a consistently top-performing model on TALENT. Under distribution shift, CoGA2M outperforms LightGBM, demonstrating a clear structural advantage. Our results position spline-based GAMs as strong contenders for tabular prediction and reveal a previously unrecognised benefit of additive structure in the OOD regime.

cs.LG↗

arXiv2026

Disassociating performance from compositional feature learning

Out-of-distribution (OOD) generalisation through composition requires a system to discover invariant properties from input-output associations and transfer them to novel inputs and unseen tasks. We argue that confirming compositional learning requires more than OOD evaluation alone: one must also verify that the learned features are genuinely compositional and that the system encodes their compositional rules. We demonstrate this through two tasks with clearly defined OOD metrics, generated via composable high-level abstractions, on which three standard architectures (MLP, CNN, Transformer) and an object-centric, slot-based architecture fail to generalise OOD. We pair these tasks with two novel attention-based architectures featuring an interpretable final hidden layer designed to expose whether compositional representations emerge. One architecture carries an engineered inductive bias that enables near-perfect OOD performance on one task. Our results show that even with appropriate biases and near-perfect OOD accuracy, a model can fail to learn the compositional feature structures necessary for systematic generalisation. The interpretable layer reveals that successful OOD performance is driven by task-specific biases rather than the discovery of reusable compositional primitives. These findings indicate that OOD benchmarks alone are insufficient for evaluating compositionality in neural networks.

cs.LG↗