Turning Tabular Foundation Models into Graph Foundation Models

Abstract

While foundation models have revolutionized fields such as natural language processing and computer vision, their potential in graph machine learning remains largely unexplored. One of the key challenges in designing graph foundation models (GFMs) is handling diverse node features that can vary across different graph datasets. While many works on GFMs have focused exclusively on text-attributed graphs, the problem of handling arbitrary features of other types in GFMs has not been fully addressed. However, this problem is not unique to the graph domain, as it also arises in the field of machine learning for tabular data. In this work, motivated by the recent success of tabular foundation models (TFMs) like TabPFNv2 and LimiX, we propose G2T-FM, a simple framework that allows tabular foundation models to be applied to graph node-level tasks. Specifically, G2T-FM augments the original node features with neighborhood feature aggregation, adds structural embeddings, and then applies a TFM to the constructed node representations. Even in the in-context learning setting, our model achieves strong results when combined with a strong TFM, outperforming both prior GFMs and well-tuned GNNs trained from scratch. Moreover, after finetuning, G2T-FM consistently surpasses well-tuned GNN baselines, often by a significant margin. In summary, our paper reveals the potential of a previously overlooked direction: utilizing tabular foundation models for graph machine learning tasks.