GS-Net: Heterogeneous Vehicle Data Reuse via Generalizable Plug-and-Play 3DGS Module

Abstract

End-to-end autonomous driving is increasingly data-driven, yet data reuse across vehicles remains limited. Each new vehicle often requires additional data collection and retraining because camera translation, orientation, and field of view differ across sensor layouts. Cross-sensor view synthesis offers a promising route for cross-platform data reuse by synthesizing images under novel sensor configurations from existing sensor data. To realize this goal, we propose GS-Net, a lightweight plug-and-play module that aggregates local geometric context from sparse Structure-from-Motion (SfM) point clouds and expands each point into multiple dense Gaussian primitives in a single forward pass, learning a cross-scene generalizable initialization for standard 3DGS that improves rendering quality for both interpolated views along the original sensor trajectories and extrapolated views at new sensor positions. In such settings, target camera viewpoints often lie beyond the convex hull of training cameras, whereas existing methods and benchmarks predominantly evaluate in-hull interpolation, leaving the extrapolation regime underexplored. To enable quantitative evaluation under this setting, we introduce CARLA-NVS, the first benchmark explicitly designed for cross-sensor view synthesis. Unlike existing autonomous driving datasets that typically employ no more than eight cameras in fixed configurations, CARLA-NVS features 12 cameras uniformly distributed at 30-degree azimuth intervals, enabling controlled evaluation of both interpolated and extrapolated viewpoints. Experiments on CARLA-NVS show that GS-Net improves rendering quality by 2.08 dB PSNR on interpolated views and 1.86 dB on extrapolated views over standard 3DGS, while achieving a 50x faster initialization compared to MVS-based densification. These results offer a practical step toward scalable cross-vehicle data reuse in autonomous driving.