arXiv subjects

Nick Schneider

Publications and source records attributed to Nick Schneider.

11 recordsLinked to original sources

arXiv2026

Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent years, neural simulation has become predominant, with methods such as NuRec playing a central role; however, these methods remain relatively slow and typically require per-scene tuning. In this work, we present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a fully simulatable 3D Gaussian Splatting (3DGS) world in a single forward pass. The model accepts multi-view input from a calibrated camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections, while providing native support for non-pinhole camera models via 3DGUT. It reconstructs a 10-20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR on the Waymo Open Dataset that is 2.01 dB above the strongest evaluated baseline. Instant NuRec is deeply integrated into NuRec and is compatible with AlpaSim for closed-loop simulation.

cs.GR↗

arXiv2026

Approximation classes for the anisotropic space-time finite element method. An almost characterization

We study the approximation of $L_p$-functions, $p\in (0,\infty]$, on cylindrical space-time domains $Ω_T:=[0,T]\times Ω$, $0<T<\infty$, $Ω\subset \R^d$ Lipschitz, $d\in \mathbb{N}$, with respect to continuous anisotropic space-time finite elements on prismatic meshes. In particular, we propose a suitable refinement technique which creates (locally refined) prismatic meshes with sufficient smoothness and the desired anisotropy, and prove complexity estimates. Furthermore, we define a (quasi-)interpolation operator on this type of meshes and use it to characterize the corresponding approximation classes by showing direct and inverse estimates in terms of anisotropic Besov norms.

math.NA↗

arXiv2025

Anisotropic approximation on space-time domains

We investigate anisotropic (piecewise) polynomial approximation of functions in Lebesgue spaces as well as anisotropic Besov spaces. For this purpose we study temporal and spacial moduli of smoothness and their properties. In particular, we prove Jackson- and Whitney-type inequalities on Lipschitz cylinders, i.e., space-time domains $I\times D$ with a finite interval $I$ and a bounded Lipschitz domain $D\subset \R^d$, $d\in \N$. As an application, we prove a direct estimate result for adaptive space-time finite element approximation in the discontinuous setting.

math.NA↗

arXiv2023

The case for studying other planetary magnetospheres and atmospheres in Heliophysics

Heliophysics is the field that "studies the nature of the Sun, and how it influences the very nature of space - and, in turn, the atmospheres of planetary bodies and the technology that exists there." However, NASA's Heliophysics Division tends to limit study of planetary magnetospheres and atmospheres to only those of Earth. This leaves exploration and understanding of space plasma physics at other worlds to the purview of the Planetary Science and Astrophysics Divisions. This is detrimental to the study of space plasma physics in general since, although some cross-divisional funding opportunities do exist, vital elements of space plasma physics can be best addressed by extending the expertise of Heliophysics scientists to other stellar and planetary magnetospheres. However, the diverse worlds within the solar system provide crucial environmental conditions that are not replicated at Earth but can provide deep insight into fundamental space plasma physics processes. Studying planetary systems with Heliophysics objectives, comprehensive instrumentation, and new grant opportunities for analysis and modeling would enable a novel understanding of fundamental and universal processes of space plasma physics. As such, the Heliophysics community should be prepared to consider, prioritize, and fund dedicated Heliophysics efforts to planetary targets to specifically study space physics and aeronomy objectives.

astro-ph.IM↗

arXiv2023

Io's Optical Aurorae in Jupiter's Shadow

Decline and recovery timescales surrounding eclipse are indicative of the controlling physical processes in Io's atmosphere. Recent studies have established that the majority of Io's molecular atmosphere, SO2 and SO, condenses during its passage through Jupiter's shadow. The eclipse response of Io's atomic atmosphere is less certain, having been characterized solely by ultraviolet aurorae. Here we explore the response of optical aurorae for the first time. We find oxygen to be indifferent to the changing illumination, with [O I] brightness merely tracking the plasma density at Io's position in the torus. In shadow, line ratios confirm sparse SO2 coverage relative to O, since their collisions would otherwise quench the emission. Io's sodium aurora mostly disappears in eclipse and e-folding timescales, for decline and recovery differ sharply: ~10 minutes at ingress and nearly 2 hr at egress. Only ion chemistry can produce such a disparity; Io's molecular ionosphere is weaker at egress due to rapid recombination. Interruption of a NaCl+ photochemical pathway best explains Na behavior surrounding eclipse, implying that the role of electron impact ionization is minor relative to photons. Auroral emission is also evident from potassium, confirming K as the major source of far red emissions seen with spacecraft imaging at Jupiter. In all cases, direct electron impact on atomic gas is sufficient to explain the brightness without invoking significant dissociative excitation of molecules. Surprisingly, the nonresponse of O and rapid depletion of Na is opposite the temporal behavior of their SO2 and NaCl parent molecules during Io's eclipse phase.

astro-ph.EP↗

arXiv2021

Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation

In order to handle the challenges of autonomous driving, deep learning has proven to be crucial in tackling increasingly complex tasks, such as 3D detection or instance segmentation. State-of-the-art approaches for image-based detection tasks tackle this complexity by operating in a cascaded fashion: they first extract a 2D bounding box based on which additional attributes, e.g. instance masks, are inferred. While these methods perform well, a key challenge remains the lack of accurate and cheap annotations for the growing variety of tasks. Synthetic data presents a promising solution but, despite the effort in domain adaptation research, the gap between synthetic and real data remains an open problem. In this work, we propose a weakly supervised domain adaptation setting which exploits the structure of cascaded detection tasks. In particular, we learn to infer the attributes solely from the source domain while leveraging 2D bounding boxes as weak labels in both domains to explain the domain shift. We further encourage domain-invariant features through class-wise feature alignment using ground-truth class information, which is not available in the unsupervised setting. As our experiments demonstrate, the approach is competitive with fully supervised settings while outperforming unsupervised adaptation approaches by a large margin.

cs.CV↗

arXiv2019

Sodium and Potassium Signatures of Volcanic Satellites Orbiting Close-in Gas Giant Exoplanets

Extrasolar satellites are generally too small to be detected by nominal searches. By analogy to the most active body in the Solar System, Io, we describe how sodium (Na I) and potassium (K I) $\textit{gas}$ could be a signature of the geological activity venting from an otherwise hidden exo-Io. Analyzing $\sim$ a dozen close-in gas giants hosting robust alkaline detections, we show that an Io-sized satellite can be stable against orbital decay below a planetary tidal $\mathcal{Q}_p \lesssim 10^{11}$. This tidal energy is focused into the satellite driving a $\sim 10^{5 \pm 2}$ higher mass loss rate than Io's supply to Jupiter's Na exosphere, based on simple atmospheric loss estimates. The remarkable consequence is that several exo-Io column densities are on average $\textit{more than sufficient}$ to provide the $\sim$ 10$^{10 \pm 1}$ Na cm$^{-2}$ required by the equivalent width of exoplanet transmission spectra. Furthermore, the benchmark observations of both Jupiter's extended ($\sim 1000$ R$_J$) Na exosphere and Jupiter's atmosphere in transmission spectroscopy yield similar Na column densities that are purely exogenic in nature. As a proof of concept, we fit the "high-altitude" Na at WASP 49-b with an ionization-limited cloud similar to the observed Na profile about Io. Moving forward, we strongly encourage time-dependent ingress and egress monitoring along with spectroscopic searches for other volcanic volatiles.

astro-ph.EP↗

arXiv2018

Boosting LiDAR-based Semantic Labeling by Cross-Modal Training Data Generation

Mobile robots and autonomous vehicles rely on multi-modal sensor setups to perceive and understand their surroundings. Aside from cameras, LiDAR sensors represent a central component of state-of-the-art perception systems. In addition to accurate spatial perception, a comprehensive semantic understanding of the environment is essential for efficient and safe operation. In this paper we present a novel deep neural network architecture called LiLaNet for point-wise, multi-class semantic labeling of semi-dense LiDAR data. The network utilizes virtual image projections of the 3D point clouds for efficient inference. Further, we propose an automated process for large-scale cross-modal training data generation called Autolabeling, in order to boost semantic labeling performance while keeping the manual annotation effort low. The effectiveness of the proposed network architecture as well as the automated data generation process is demonstrated on a manually annotated ground truth dataset. LiLaNet is shown to significantly outperform current state-of-the-art CNN architectures for LiDAR data. Applying our automatically generated large-scale training data yields a boost of up to 14 percentage points compared to networks trained on manually annotated data only.

cs.CV↗

arXiv2017

Sparsity Invariant CNNs

In this paper, we consider convolutional neural networks operating on sparse inputs with an application to depth upsampling from sparse laser scan data. First, we show that traditional convolutional networks perform poorly when applied to sparse data even when the location of missing data is provided to the network. To overcome this problem, we propose a simple yet effective sparse convolution layer which explicitly considers the location of missing data during the convolution operation. We demonstrate the benefits of the proposed network architecture in synthetic and real experiments with respect to various baseline approaches. Compared to dense baselines, the proposed sparse convolution network generalizes well to novel datasets and is invariant to the level of sparsity in the data. For our evaluation, we derive a novel dataset from the KITTI benchmark, comprising 93k depth annotated RGB images. Our dataset allows for training and evaluating depth upsampling and depth prediction techniques in challenging real-world settings and will be made available upon publication.

cs.CV↗

arXiv2017

RegNet: Multimodal Sensor Registration Using Deep Neural Networks

In this paper, we present RegNet, the first deep convolutional neural network (CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between multimodal sensors, exemplified using a scanning LiDAR and a monocular camera. Compared to existing approaches, RegNet casts all three conventional calibration steps (feature extraction, feature matching and global regression) into a single real-time capable CNN. Our method does not require any human interaction and bridges the gap between classical offline and target-less online calibration approaches as it provides both a stable initial estimation as well as a continuous online correction of the extrinsic parameters. During training we randomly decalibrate our system in order to train RegNet to infer the correspondence between projected depth measurements and RGB image and finally regress the extrinsic calibration. Additionally, with an iterative execution of multiple CNNs, that are trained on different magnitudes of decalibration, our approach compares favorably to state-of-the-art methods in terms of a mean calibration error of 0.28 degrees for the rotational and 6 cm for the translation components even for large decalibrations up to 1.5 m and 20 degrees.

cs.CV↗

arXiv2016

Semantically Guided Depth Upsampling

We present a novel method for accurate and efficient up- sampling of sparse depth data, guided by high-resolution imagery. Our approach goes beyond the use of intensity cues only and additionally exploits object boundary cues through structured edge detection and semantic scene labeling for guidance. Both cues are combined within a geodesic distance measure that allows for boundary-preserving depth in- terpolation while utilizing local context. We model the observed scene structure by locally planar elements and formulate the upsampling task as a global energy minimization problem. Our method determines glob- ally consistent solutions and preserves fine details and sharp depth bound- aries. In our experiments on several public datasets at different levels of application, we demonstrate superior performance of our approach over the state-of-the-art, even for very sparse measurements.

cs.CV↗