CFM-Bench: A Unified Multi-Domain, Multi-Task Benchmark for Channel Foundation Models

Abstract

Channel foundation models (CFMs) are commonly evaluated in model-specific pipelines that differ in data, radio configurations, partitions, adaptation procedures, task definitions, and metrics, preventing reproducible comparison across CFMs and against task-specific networks. We release CFM-Bench, a unified multi-domain, multi-task benchmark comprising 157,900 official single-frame examples from six domains spanning 3GPP statistical simulation, two ray-tracing pipelines, terrestrial and aerial measurements, and synchronized vehicular multimodal simulation. Source-specific interfaces preserve complex channel state information (CSI) and the physical metadata available in each domain while allowing documented model-specific preprocessing. To reduce spatio-temporal leakage, official partitions isolate complete trajectories, measurement sessions, flights, vehicle links, simulation realizations, or buffered spatial regions. CFM-Bench excludes all benchmark splits from foundation-model pretraining, reserves the official training split for downstream fine-tuning, and reports the data used during model development. Six task groups across PHY, RAN, and ISAC applications cover CSI feedback, frequency and temporal channel extrapolation, propagation-state classification, current- and future-beam prediction, and single-frame and temporal localization. Representative experiments on CSI feedback, channel extrapolation, current-beam prediction, and wireless positioning provide reproducible reference results for pretrained channel-prediction models and task-specific neural networks. These results show that relative model performance can vary across data domains, highlighting the importance of using common data partitions, task definitions, and evaluation metrics. CFM-Bench provides a common substrate for evaluating the transferability of channel representations across models, domains, and tasks.