arXiv subjects

Anthony Rowe

Publications and source records attributed to Anthony Rowe.

17 recordsLinked to original sources

arXiv2026

RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields

Radars are an ideal complement to cameras: both are inexpensive, solid-state sensors, with cameras offering fine angular resolution, while radars provide metric depth and robustness under adverse weather. However, radar data is more difficult to interpret than camera images and varies significantly between sensors, necessitating increased reliance on simulation for prototyping sensors and processing pipelines. Recent work treating radar reconstruction as a novel view synthesis problem has shown great promise in reconstructing radar-relevant geometry and simulating low-level radar data. However, such methods are constrained by the low spatial resolution of the underlying radar. To address this, we propose a unified differentiable renderer, RadarSim, which leverages the high angular resolution of RGB cameras to generate Doppler radar range images from a camera-initialized neural field. Using a novel data set of calibrated radar camera recordings from a custom hand-held rig, we demonstrate that RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.

cs.CV↗

arXiv2026

Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models

3D scene understanding spans reasoning about free space, object grounding, hypothetical object insertions, complex geometric relationships, and integrating all of these with external tools and data sources. Existing 3D understanding methods typically rely on large-scale 3D-language training or focus on object grounding and simple spatial relationships. We argue that the broad generalization that motivates 3D-language training can be achieved at inference time, without 3D-specific training. We propose Flame3D, a training-free framework that represents scenes as editable visual-textual 3D memories and exposes them to an off-the-shelf MLLM through composable spatial tools. Flame3D also lets the agent synthesize custom spatial programs at inference time, enabling open-ended reasoning over layouts, empty space, and objects not yet present in the scene. External data and corrections can be added to the memory without retraining. In addition to showing competitive performance to finetuned 3D-LMM methods on ScanQA, we study multi-hop 3D reasoning capabilities of Flame3D by evaluating it on a curated compositional spatial-reasoning benchmark, Compose3D. We find that fixed tools fall short and that the agent's ability to synthesize spatial operations at inference time is essential. These results invite the question: should future progress in 3D scene understanding focus on richer scene memories and expressive compositional abstractions?

cs.CV↗

arXiv2026

UNRIO: Uncertainty-Aware Velocity Learning for Radar-Inertial Odometry

mmWave radars are robust to darkness and occlusions such as dust and smoke, and can directly constrain ego-velocity from a single frame via Doppler measurements, making them attractive sensors for odometry in visually denied conditions. However, almost all existing radar-inertial odometry systems rely on lossy radar point clouds that are highly sparse, generally concentrated in a narrow angular band, and aliased at high speeds. We propose to instead estimate ego-velocity directly from unfiltered mmWave I/Q signals. Taking advantage of a foundation model for 4D radar spectrum, we develop a system that integrates uncertainty-aware velocity predictions with IMU measurements using an uncertainty-weighted sliding-window pose graph to accurately compute odometry even when provided radar data with aliasing or an unfavorable field of view. Evaluated on public benchmark datasets, our system, UNRIO, attains the lowest relative pose error on the majority of sequences across held-out environments, despite differing chirp configurations, motion patterns, and platforms with its strongest gains coming where point clouds fail most: lateral motion in IQ1M and heavily aliased Doppler in ColoRadar.

cs.RO↗

arXiv2025

Millimeter Wave Inverse Pinhole Imaging

Millimeter wave (mmWave) radars are popular for perception in vision-denied contexts due to their compact size. This paper explores emerging use-cases that involve static mount or momentarily-static compact radars, for example, a hovering drone. The key challenge with static compact radars is that their limited form-factor also limits their angular resolution. This paper presents Umbra, a mmWave high resolution imaging system, that introduces the concept of rotating mmWave "inverse pinholes" for angular resolution enhancement. We present the imaging system model, design, and evaluation of mmWave inverse pinholes. The inverse pinhole is attractive for its lightweight nature, which enables low-power rotation, upgrading static-mount radars. We also show how propellers in aerial vehicles act as natural inverse pinholes and can enjoy the benefits of high-resolution imaging even while they are momentarily static, e.g., hovering. Our evaluation shows Umbra resolving up to 2.5$^{\circ}$ with just a single antenna, a 5$\times$ improvement compared to 14$^{\circ}$ from a compact mmWave radar baseline.

eess.IV↗

arXiv2025

OpenFLAME: Federated Visual Positioning System to Enable Large-Scale Augmented Reality Applications

World-scale augmented reality (AR) applications need a ubiquitous 6DoF localization backend to anchor content to the real world consistently across devices. Large organizations such as Google and Niantic are 3D scanning outdoor public spaces in order to build their own Visual Positioning Systems (VPS). These centralized VPS solutions fail to meet the needs of many future AR applications -- they do not cover private indoor spaces because of privacy concerns, regulations, and the labor bottleneck of updating and maintaining 3D scans. In this paper, we present OpenFLAME, a federated VPS backend that allows independent organizations to 3D scan and maintain a separate VPS service for their own spaces. This enables access control of indoor 3D scans, distributed maintenance of the VPS backend, and encourages larger coverage. Sharding of VPS services introduces several unique challenges -- coherency of localization results across spaces, quality control of VPS services, selection of the right VPS service for a location, and many others. We introduce the concept of federated image-based localization and provide reference solutions for managing and merging data across maps without sharing private data.

cs.CV↗

arXiv2025

Towards Foundational Models for Single-Chip Radar

mmWave radars are compact, inexpensive, and durable sensors that are robust to occlusions and work regardless of environmental conditions, such as weather and darkness. However, this comes at the cost of poor angular resolution, especially for inexpensive single-chip radars, which are typically used in automotive and indoor sensing applications. Although many have proposed learning-based methods to mitigate this weakness, no standardized foundational models or large datasets for the mmWave radar have emerged, and practitioners have largely trained task-specific models from scratch using relatively small datasets. In this paper, we collect (to our knowledge) the largest available raw radar dataset with 1M samples (29 hours) and train a foundational model for 4D single-chip radar, which can predict 3D occupancy and semantic segmentation with quality that is typically only possible with much higher resolution sensors. We demonstrate that our Generalizable Radar Transformer (GRT) generalizes across diverse settings, can be fine-tuned for different tasks, and shows logarithmic data scaling of 20\% per $10\times$ data. We also run extensive ablations on common design decisions, and find that using raw radar data significantly outperforms widely-used lossy representations, equivalent to a $10\times$ increase in training data. Finally, we roughly estimate that $\approx$100M samples (3000 hours) of data are required to fully exploit the potential of GRT.

cs.CV↗

arXiv2025

Uniting the World by Dividing it: Federated Maps to Enable Spatial Applications

The emergence of the Spatial Web -- the Web where content is tied to real-world locations has the potential to improve and enable many applications such as augmented reality, navigation, robotics, and more. The Spatial Web is missing a key ingredient that is impeding its growth -- a spatial naming system to resolve real-world locations to names. Today's spatial naming systems are digital maps such as Google and Apple maps. These maps and the location-based services provided on top of these maps are primarily controlled by a few large corporations and mostly cover outdoor public spaces. Emerging classes of applications, such as persistent world-scale augmented reality, require detailed maps of both outdoor and indoor spaces. Existing centralized mapping infrastructures are proving insufficient for such applications because of the scale of cartography efforts required and the privacy of indoor map data. In this paper, we present a case for a federated spatial naming system, or in other words, a federated mapping infrastructure. This enables disparate parties to manage and serve their own maps of physical regions and unlocks scalability of map management, isolation and privacy of maps. Map-related services such as address-to-location mapping, location-based search, and routing needs re-architecting to work on federated maps. We discuss some essential services and practicalities of enabling these services.

cs.DC↗

arXiv2025

Interference-Aware Edge Runtime Prediction with Conformal Matrix Completion

Accurately estimating workload runtime is a longstanding goal in computer systems, and plays a key role in efficient resource provisioning, latency minimization, and various other system management tasks. Runtime prediction is particularly important for managing increasingly complex distributed systems in which more sophisticated processing is pushed to the edge in search of better latency. Previous approaches for runtime prediction in edge systems suffer from poor data efficiency or require intensive instrumentation; these challenges are compounded in heterogeneous edge computing environments, where historical runtime data may be sparsely available and instrumentation is often challenging. Moreover, edge computing environments often feature multi-tenancy due to limited resources at the network edge, potentially leading to interference between workloads and further complicating the runtime prediction problem. Drawing from insights across machine learning and computer systems, we design a matrix factorization-inspired method that generates accurate interference-aware predictions with tight provably-guaranteed uncertainty bounds. We validate our method on a novel WebAssembly runtime dataset collected from 24 unique devices, achieving a prediction error of 5.2% -- 2x better than a naive application of existing methods.

cs.LG↗

arXiv2024

OpenFLAME: A Federated Spatial Naming Infrastructure

Spatial applications, i.e., applications that tie digital information with the physical world, have improved many of our daily activities, such as navigation and ride-sharing. This class of applications also holds significant promise of enabling new industries such as augmented reality and robotics. The development of these applications is enabled by a system that can resolve real-world locations to names, or a spatial naming system. Today, mapping platforms provided by organizations like Google and Apple serve as spatial naming systems. These maps are centralized and primarily cover outdoor spaces. We envision that future spatial applications, such as persistent world-scale augmented reality, would require detailed and precise spatial data across indoor and outdoor spaces. The scale of cartography efforts required to survey indoor spaces and their privacy needs inhibit existing centralized maps from incorporating such spaces into their platform. In this paper, we present the design and implementation of OpenFLAME stands for Open Federated Localization and Mapping Engine, a federated spatial naming system, or in other words, a federated mapping infrastructure. It enables independent parties to manage and serve their own maps of physical regions. This unlocks scalability of map management, isolation, and privacy of maps. The discovery system that identifies maps hosted at a given location is a primary component of our system. We implement OpenFLAME on top of the existing Domain Name System (DNS), which enables us to leverage its existing infrastructure. We implement map services such as address-to-location mapping, routing, and localization on top of our federated mapping infrastructure.

cs.DC↗

arXiv2024

DART: Implicit Doppler Tomography for Radar Novel View Synthesis

Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.

cs.CV↗

arXiv2023

Empowering WebAssembly with Thin Kernel Interfaces

Wasm is gaining popularity outside the Web as a well-specified low-level binary format with ISA portability, low memory footprint and polyglot targetability, enabling efficient in-process sandboxing of untrusted code. Despite these advantages, Wasm adoption for new domains is often hindered by the lack of many standard system interfaces which precludes reusability of existing software and slows ecosystem growth. This paper proposes thin kernel interfaces for Wasm, which directly expose OS userspace syscalls without breaking intra-process sandboxing, enabling a new class of virtualization with Wasm as a universal binary format. By virtualizing the bottom layer of userspace, kernel interfaces enable effortless application ISA portability, compiler backend reusability, and armor programs with Wasm's built-in control flow integrity and arbitrary code execution protection. Furthermore, existing capability-based APIs for Wasm, such as WASI, can be implemented as a Wasm module over kernel interfaces, improving reuse, robustness, and portability through better layering. We present an implementation of this concept for two kernels -- Linux and Zephyr -- by extending a modern Wasm engine and evaluate our system's performance on a number of sophisticated applications which can run for the first time on Wasm.

cs.OS↗

arXiv2022

High Resolution Point Clouds from mmWave Radar

This paper explores a machine learning approach for generating high resolution point clouds from a single-chip mmWave radar. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.

cs.RO↗

arXiv2022

Quantum Extremal Learning

We propose a quantum algorithm for `extremal learning', which is the process of finding the input to a hidden function that extremizes the function output, without having direct access to the hidden function, given only partial input-output (training) data. The algorithm, called quantum extremal learning (QEL), consists of a parametric quantum circuit that is variationally trained to model data input-output relationships and where a trainable quantum feature map, that encodes the input data, is analytically differentiated in order to find the coordinate that extremizes the model. This enables the combination of established quantum machine learning modelling with established quantum optimization, on a single circuit/quantum computer. We have tested our algorithm on a range of classical datasets based on either discrete or continuous input variables, both of which are compatible with the algorithm. In case of discrete variables, we test our algorithm on synthetic problems formulated based on Max-Cut problem generators and also considering higher order correlations in the input-output relationships. In case of the continuous variables, we test our algorithm on synthetic datasets in 1D and simple ordinary differential functions. We find that the algorithm is able to successfully find the extremal value of such problems, even when the training dataset is sparse or a small fraction of the input configuration space. We additionally show how the algorithm can be used for much more general cases of higher dimensionality, complex differential equations, and with full flexibility in the choice of both modeling and optimization ansatz. We envision that due to its general framework and simple construction, the QEL algorithm will be able to solve a wide variety of applications in different fields, opening up areas of further research.

quant-ph↗

arXiv2021

Multi-User Augmented Reality with Infrastructure-free Collaborative Localization

Multi-user augmented reality (AR) could someday empower first responders with the ability to see team members around corners and through walls. For this vision of people tracking in dynamic environments to be practical, we need a relative localization system that is nearly instantly available across wide-areas without any existing infrastructure or manual setup. In this paper, we present LocAR, an infrastructure-free 6-degrees-of-freedom (6DoF) localization system for AR applications that uses motion estimates and range measurements between users to establish an accurate relative coordinate system. We show that not only is it possible to perform collaborative localization without infrastructure or global coordinates, but that our approach provides nearly the same level of accuracy as fixed infrastructure approaches for AR teaming applications. LocAR uses visual-inertial odometry (VIO) in conjunction with ultra-wideband (UWB) ranging radios to estimate the relative position of each device in an ad-hoc manner. The system leverages a collaborative 6DoF particle filtering formulation that operates on sporadic messages exchanged between nearby users. Unlike map or landmark sharing approaches, this allows for collaborative AR sessions even if users do not overlap the same spaces. LocAR consists of an open-source UWB firmware and reference mobile phone application that can display the location of team members in real-time using mobile AR. We evaluate LocAR across multiple buildings under a wide-variety of conditions including a contiguous 30,000 square foot region spanning multiple floors and find that it achieves median geometric error in 3D of less than 1 meter between five users freely walking across 3 floors.

eess.SY↗

arXiv2021

A Hybrid mmWave and Camera System for Long-Range Depth Imaging

mmWave radars offer excellent depth resolution even at very long ranges owing to their high bandwidth. But their angular resolution is at least an order-of-magnitude worse than camera and lidar systems. Hence, mmWave radar is not a capable 3-D imaging solution in isolation. We propose Metamoran, a system that combines the complimentary strengths of radar and camera to obtain accurate, high resolution depth images over long ranges even in high clutter environments, all from a single fixed vantage point. Metamoran enables rich long-range depth imaging with applications in security and surveillance, roadside safety infrastructure and wide-area mapping. Our approach leverages the high angular resolution from cameras using computer vision techniques, including image segmentation and monocular depth estimation, to obtain object shape. Our core contribution is a method to convert this object shape into an RF I/Q equivalent, which we use in a novel radar processing pipeline to help declutter the scene and capture extremely weak reflections from objects at long distances. We perform a detailed evaluation of Metamoran's depth imaging capabilities in 400 diverse scenes. Our evaluation shows that Metamoran estimates the depth of static objects up to 90 m and moving objects up to 305 m and with a median error of 28 cm, an improvement of 13$\times$ compared to a naive radar+camera baseline and 23$\times$ compared to monocular depth estimation.

cs.CV↗

arXiv2020

Data-driven Thermal Model Inference with ARMAX, in Smart Environments, based on Normalized Mutual Information

Understanding the models that characterize the thermal dynamics in a smart building is important for the comfort of its occupants and for its energy optimization. A significant amount of research has attempted to utilize thermodynamics (physical) models for smart building control, but these approaches remain challenging due to the stochastic nature of the intermittent environmental disturbances. This paper presents a novel data-driven approach for indoor thermal model inference, which combines an Autoregressive Moving Average with eXogenous inputs model (ARMAX) with a Normalized Mutual Information scheme (NMI). Based on this information-theoretic method, NMI, causal dependencies between the indoor temperature and exogenous inputs are explicitly obtained as a guideline for the ARMAX model to find the dominating inputs. For validation, we use three datasets based on building energy systems-against which we compare our method to an autoregressive model with exogenous inputs (ARX), a regularized ARMAX model, and state-space models.

eess.SY↗

arXiv2015

Tracking Motion and Proxemics using Thermal-sensor Array

Indoor tracking has all-pervasive applications beyond mere surveillance, for example in education, health monitoring, marketing, energy management and so on. Image and video based tracking systems are intrusive. Thermal array sensors on the other hand can provide coarse-grained tracking while preserving privacy of the subjects. The goal of the project is to facilitate motion detection and group proxemics modeling using an 8 x 8 infrared sensor array. Each of the 8 x 8 pixels is a temperature reading in Fahrenheit. We refer to each 8 x 8 matrix as a scene. We collected approximately 902 scenes with different configurations of human groups and different walking directions. We infer direction of motion of a subject across a set of scenes as left-to-right, right-to-left, up-to-down and down-to-up using cross-correlation analysis. We used features from connected component analysis of each background subtracted scene and performed Support Vector Machine classification to estimate number of instances of human subjects in the scene.

cs.CV↗