arXiv subjects

Ashkan Ganj

Publications and source records attributed to Ashkan Ganj.

6 recordsLinked to original sources

arXiv2025

AR as an Evaluation Playground: Bridging Metrics and Visual Perception of Computer Vision Models

Quantitative metrics are central to evaluating computer vision (CV) models, but they often fail to capture real-world performance due to protocol inconsistencies and ground-truth noise. While visual perception studies can complement these metrics, they often require end-to-end systems that are time-consuming to implement and setups that are difficult to reproduce. We systematically summarize key challenges in evaluating CV models and present the design of ARCADE, an evaluation platform that leverages augmented reality (AR) to enable easy, reproducible, and human-centered CV evaluation. ARCADE uses a modular architecture that provides cross-platform data collection, pluggable model inference, and interactive AR tasks, supporting both metric and visual perception evaluation. We demonstrate ARCADE through a user study with 15 participants and case studies on two representative CV tasks, depth and lighting estimation, showing that ARCADE can reveal perceptual flaws in model quality that are often missed by traditional metrics. We also evaluate ARCADE's usability and performance, showing its flexibility as a reliable real-time platform.

cs.CV↗

arXiv2024

HybridDepth: Robust Metric Depth Fusion by Leveraging Depth from Focus and Single-Image Priors

We propose HYBRIDDEPTH, a robust depth estimation pipeline that addresses key challenges in depth estimation,including scale ambiguity, hardware heterogeneity, and generalizability. HYBRIDDEPTH leverages focal stack, data conveniently accessible in common mobile devices, to produce accurate metric depth maps. By incorporating depth priors afforded by recent advances in singleimage depth estimation, our model achieves a higher level of structural detail compared to existing methods. We test our pipeline as an end-to-end system, with a newly developed mobile client to capture focal stacks, which are then sent to a GPU-powered server for depth estimation. Comprehensive quantitative and qualitative analyses demonstrate that HYBRIDDEPTH outperforms state-of-the-art(SOTA) models on common datasets such as DDFF12 and NYU Depth V2. HYBRIDDEPTH also shows strong zero-shot generalization. When trained on NYU Depth V2, HYBRIDDEPTH surpasses SOTA models in zero-shot performance on ARKitScenes and delivers more structurally accurate depth maps on Mobile Depth. The code is available at https://github.com/cake-lab/HybridDepth/.

cs.CV↗

arXiv2023

Mobile AR Depth Estimation: Challenges & Prospects -- Extended Version

Metric depth estimation plays an important role in mobile augmented reality (AR). With accurate metric depth, we can achieve more realistic user interactions such as object placement and occlusion detection. While specialized hardware like LiDAR demonstrates its promise, its restricted availability, i.e., only on selected high-end mobile devices, and performance limitations such as range and sensitivity to the environment, make it less ideal. Monocular depth estimation, on the other hand, relies solely on mobile cameras, which are ubiquitous, making it a promising alternative for mobile AR. In this paper, we investigate the challenges and opportunities of achieving accurate metric depth estimation in mobile AR. We tested four different state-of-the-art monocular depth estimation models on a newly introduced dataset (ARKitScenes) and identified three types of challenges: hard-ware, data, and model related challenges. Furthermore, our research provides promising future directions to explore and solve those challenges. These directions include (i) using more hardware-related information from the mobile device's camera and other available sensors, (ii) capturing high-quality data to reflect real-world AR scenarios, and (iii) designing a model architecture to utilize the new information.

cs.CV↗

arXiv2023

Get-A-Sense: Designing Spatial Context Awareness for Mobile AR Environment Understanding

Physical environment understanding is vital in delivering immersive and interactive mobile augmented reality (AR) user experiences. Recently, we have witnessed a transition in the design of environment understanding systems, from visual data focused to centering on the concept of spatial context, including user, device, and environment information. Even though spatial context can benefit the environment understanding tasks, e.g., we demonstrate in a case study that lighting estimation performance can be improved by as much as 59%, not all environment understanding systems support spatial context. Furthermore, even for the environment understanding systems that support spatial context, not all useful spatial context has been leveraged; and the design and implementation can differ vastly. In this paper, we advocate for the design of a spatial context-aware and shared environment understanding system that can effectively support multiple tasks simultaneously. Fundamentally, there are many practical challenges in designing such a unified environment understanding system. We discuss the challenges in the context of three open questions, (i) how to fully leverage user mobility, (ii) how to design a unified data model, and (iii) finally how to build a shared system for multiple tasks.

cs.DC↗

arXiv2023

Toward Scalable and Controllable AR Experimentation

To understand how well a proposed augmented reality (AR) solution works, existing papers often conducted tailored and isolated evaluations for specific AR tasks, e.g., depth or lighting estimation, and compared them to easy-to-setup baselines, either using datasets or resorting to time-consuming data capturing. Conceptually simple, it can be extremely difficult to evaluate an AR system fairly and in scale to understand its real-world performance. The difficulties arise for three key reasons: lack of control of the physical environment, the time-consuming data capturing, and the difficulties to reproduce baseline results. This paper presents our design of an AR experimentation platform, ExpAR, aiming to provide scalable and controllable AR experimentation. ExpAR is envisioned to operate as a standalone deployment or a federated platform; in the latter case, AR researchers can contribute physical resources, including scene setup and capturing devices, and allow others to time share these resources. Our design centers around the generic sensing-understanding-rendering pipeline and is driven by the evaluation limitations observed in recent AR systems papers. We demonstrate the feasibility of this vision with a preliminary prototype and our preliminary evaluations suggest the importance of further investigating different device capabilities to stream in 30 FPS. The ExpAR project site can be found at https://cake.wpi.edu/expar.

cs.DC↗

arXiv2023

LR-Net: A Block-based Convolutional Neural Network for Low-Resolution Image Classification

The success of CNN-based architecture on image classification in learning and extracting features made them so popular these days, but the task of image classification becomes more challenging when we apply state of art models to classify noisy and low-quality images. It is still difficult for models to extract meaningful features from this type of image due to its low-resolution and the lack of meaningful global features. Moreover, high-resolution images need more layers to train which means they take more time and computational power to train. Our method also addresses the problem of vanishing gradients as the layers become deeper in deep neural networks that we mentioned earlier. In order to address all these issues, we developed a novel image classification architecture, composed of blocks that are designed to learn both low level and global features from blurred and noisy low-resolution images. Our design of the blocks was heavily influenced by Residual Connections and Inception modules in order to increase performance and reduce parameter sizes. We also assess our work using the MNIST family datasets, with a particular emphasis on the Oracle-MNIST dataset, which is the most difficult to classify due to its low-quality and noisy images. We have performed in-depth tests that demonstrate the presented architecture is faster and more accurate than existing cutting-edge convolutional neural networks. Furthermore, due to the unique properties of our model, it can produce a better result with fewer parameters.

cs.CV↗