arXiv subjects

Jens Grubert

Publications and source records attributed to Jens Grubert.

52 records · Page 3Linked to original sources

arXiv2019

ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality

Physical keyboards are common peripherals for personal computers and are efficient standard text entry devices. Recent research has investigated how physical keyboards can be used in immersive head-mounted display-based Virtual Reality (VR). So far, the physical layout of keyboards has typically been transplanted into VR for replicating typing experiences in a standard desktop environment. In this paper, we explore how to fully leverage the immersiveness of VR to change the input and output characteristics of physical keyboard interaction within a VR environment. This allows individual physical keys to be reconfigured to the same or different actions and visual output to be distributed in various ways across the VR representation of the keyboard. We explore a set of input and output mappings for reconfiguring the virtual presentation of physical keyboards and probe the resulting design space by specifically designing, implementing and evaluating nine VR-relevant applications: emojis, languages and special characters, application shortcuts, virtual text processing macros, a window manager, a photo browser, a whack-a-mole game, secure password entry and a virtual touch bar. We investigate the feasibility of the applications in a user study with 20 participants and find that, among other things, they are usable in VR. We discuss the limitations and possibilities of remapping the input and output characteristics of physical keyboards in VR based on empirical findings and analysis and suggest future research directions in this area.

cs.HC↗

arXiv2018

The Office of the Future: Virtual, Portable and Global

Virtual Reality has the potential to change the way we work. We envision the future office worker to be able to work productively everywhere solely using portable standard input devices and immersive head-mounted displays. Virtual Reality has the potential to enable this, by allowing users to create working environments of their choice and by relieving them from physical world limitations such as constrained space or noisy environments. In this article, we investigate opportunities and challenges for realizing this vision and discuss implications from recent findings of text entry in virtual reality as a core office task.

cs.HC↗

arXiv2018

Efficient Pose Tracking from Natural Features in Standard Web Browsers

Computer Vision-based natural feature tracking is at the core of modern Augmented Reality applications. Still, Web-based Augmented Reality typically relies on location-based sensing (using GPS and orientation sensors) or marker-based approaches to solve the pose estimation problem. We present an implementation and evaluation of an efficient natural feature tracking pipeline for standard Web browsers using HTML5 and WebAssembly. Our system can track image targets at real-time frame rates tablet PCs (up to 60 Hz) and smartphones (up to 25 Hz).

cs.CV↗

arXiv2018

Mobiles as Portals for Interacting with Virtual Data Visualizations

We propose a set of techniques leveraging mobile devices as lenses to explore, interact and annotate n-dimensional data visualizations. The democratization of mobile devices, with their arrays of integrated sensors, opens up opportunities to create experiences for anyone to explore and interact with large information spaces anywhere. In this paper, we propose to revisit ideas behind the Chameleon prototype of Fitzmaurice et al. initially envisioned in the 90s for navigation, before spatially-aware devices became mainstream. We also take advantage of other input modalities such as pen and touch to not only navigate the space using the mobile as a lens, but interact and annotate it by adding toolglasses.

cs.HC↗

arXiv2018

Effects of Hand Representations for Typing in Virtual Reality

Alphanumeric text entry is a challenge for Virtual Reality (VR) applications. VR enables new capabilities, impossible in the real world, such as an unobstructed view of the keyboard, without occlusion by the user's physical hands. Several hand representations have been proposed for typing in VR on standard physical keyboards. However, to date, these hand representations have not been compared regarding their performance and effects on presence for VR text entry. Our work addresses this gap by comparing existing hand representations with minimalistic fingertip visualization. We study the effects of four hand representations (no hand representation, inverse kinematic model, fingertip visualization using spheres and video inlay) on typing in VR using a standard physical keyboard with 24 participants. We found that the fingertip visualization and video inlay both resulted in statistically significant lower text entry error rates compared to no hand or inverse kinematic model representations. We found no statistical differences in text entry speed.

cs.HC↗

arXiv2018

Text Entry in Immersive Head-Mounted Display-based Virtual Reality using Standard Keyboards

We study the performance and user experience of two popular mainstream text entry devices, desktop keyboards and touchscreen keyboards, for use in Virtual Reality (VR) applications. We discuss the limitations arising from limited visual feedback, and examine the efficiency of different strategies of use. We analyze a total of 24 hours of typing data in VR from 24 participants and find that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40-45\% of their typing speed on a touchscreen keyboard. We also find no significant learning effects, indicating that users can transfer their typing skills fast into VR. Besides investigating baseline performances, we study the position in which keyboards and hands are rendered in space. We find that this does not adversely affect performance for desktop keyboard typing and results in a performance trade-off for touchscreen keyboard typing.

cs.HC↗

arXiv2017

BodyDigitizer: An Open Source Photogrammetry-based 3D Body Scanner

With the rising popularity of Augmented and Virtual Reality, there is a need for representing humans as virtual avatars in various application domains ranging from remote telepresence, games to medical applications. Besides explicitly modelling 3D avatars, sensing approaches that create person-specific avatars are becoming popular. However, affordable solutions typically suffer from a low visual quality and professional solution are often too expensive to be deployed in nonprofit projects. We present an open-source project, BodyDigitizer, which aims at providing both build instructions and configuration software for a high-resolution photogrammetry-based 3D body scanner. Our system encompasses up to 96 Rasperry PI cameras, active LED lighting, a sturdy frame construction and open-source configuration software. %We demonstrate the applicability of the body scanner in a nonprofit Mixed Reality health project. The detailed build instruction and software are available at http://www.bodydigitizer.org.

cs.CV↗

arXiv2017

A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays

Optical see-through head-mounted displays (OST HMDs) are a major output medium for Augmented Reality, which have seen significant growth in popularity and usage among the general public due to the growing release of consumer-oriented models, such as the Microsoft Hololens. Unlike Virtual Reality headsets, OST HMDs inherently support the addition of computer-generated graphics directly into the light path between a user's eyes and their view of the physical world. As with most Augmented and Virtual Reality systems, the physical position of an OST HMD is typically determined by an external or embedded 6-Degree-of-Freedom tracking system. However, in order to properly render virtual objects, which are perceived as spatially aligned with the physical environment, it is also necessary to accurately measure the position of the user's eyes within the tracking system's coordinate frame. For over 20 years, researchers have proposed various calibration methods to determine this needed eye position. However, to date, there has not been a comprehensive overview of these procedures and their requirements. Hence, this paper surveys the field of calibration methods for OST HMDs. Specifically, it provides insights into the fundamentals of calibration techniques, and presents an overview of both manual and automatic approaches, as well as evaluation methods and metrics. Finally, it also identifies opportunities for future research. % relative to the tracking coordinate system, and, hence, its position in 3D space.

cs.HC↗

arXiv2017

Authoring and Living Next-Generation Location-Based Experiences

Authoring location-based experiences involving multiple participants, collaborating or competing in both indoor and outdoor mixed realities, is extremely complex and bound to serious technical challenges. In this work, we present the first results of the MAGELLAN European project and how these greatly simplify this creative process using novel authoring, augmented reality (AR) and indoor geolocalisation techniques.

cs.HC↗

arXiv2017

Feasibility of Corneal Imaging for Handheld Augmented Reality

Smartphones are a popular device class for mobile Augmented Reality but suffer from a limited input space. Around-device interaction techniques aim at extending this input space using various sensing modalities. In this paper we present our work towards extending the input area of mobile devices using front-facing device-centered cameras that capture reflections in the cornea. As current generation mobile devices lack high resolution front-facing cameras, we study the feasibility of around-device interaction using corneal reflective imaging based on a high resolution camera. We present a workflow, a technical prototype and a feasibility evaluation.

cs.HC↗

arXiv2017

Die Zukunft sehen: Die Chancen und Herausforderungen der Erweiterten und Virtuellen Realität für industrielle Anwendungen

Digitalization offers chances as well as risks for industrial companies. This article describes how the area of Mixed Reality, with its manifestations Augmented and Virtual Reality, can support industrial applications in the age of digitalization. Starting from a historical perspective on Augmented and Virtual Reality, this article surveys recent developments in the domain of Mixed Reality, relevant for industrial use cases. --- Die Digitalisierung bietet für Industrieunternehmen neue Chancen, stellt diese jedoch auch vor Herausforderungen. Dieser Artikel beleuchtet wie das Gebiet der vermischten Realität mit seinen Ausprägungen der erweiterten Realität und der virtuellen Realität für industriellen Anwendungen im Zeitalter der Digitalisierung Vorteile schaffen kann. Ausgehend von einer historischen Betrachtung, werden aktuelle Entwicklungen auf dem Gebiet der erweiterten und virtuellen Realität diskutiert.

cs.HC↗

arXiv2017

Towards Around-Device Interaction using Corneal Imaging

Around-device interaction techniques aim at extending the input space using various sensing modalities on mobile and wearable devices. In this paper, we present our work towards extending the input area of mobile devices using front-facing device-centered cameras that capture reflections in the human eye. As current generation mobile devices lack high resolution front-facing cameras we study the feasibility of around-device interaction using corneal reflective imaging based on a high resolution camera. We present a workflow, a technical prototype and an evaluation, including a migration path from high resolution to low resolution imagers. Our study indicates, that under optimal conditions a spatial sensing resolution of 5 cm in the vicinity of a mobile phone is possible.

cs.HC↗

arXiv2017

Adaptive User Perspective Rendering for Handheld Augmented Reality

Handheld Augmented Reality commonly implements some variant of magic lens rendering, which turns only a fraction of the user's real environment into AR while the rest of the environment remains unaffected. Since handheld AR devices are commonly equipped with video see-through capabilities, AR magic lens applications often suffer from spatial distortions, because the AR environment is presented from the perspective of the camera of the mobile device. Recent approaches counteract this distortion based on estimations of the user's head position, rendering the scene from the user's perspective. To this end, approaches usually apply face-tracking algorithms on the front camera of the mobile device. However, this demands high computational resources and therefore commonly affects the performance of the application beyond the already high computational load of AR applications. In this paper, we present a method to reduce the computational demands for user perspective rendering by applying lightweight optical flow tracking and an estimation of the user's motion before head tracking is started. We demonstrate the suitability of our approach for computationally limited mobile devices and we compare it to device perspective rendering, to head tracked user perspective rendering, as well as to fixed point of view user perspective rendering.

cs.HC↗

arXiv2017

Towards Interaction Around Unmodified Camera-equipped Mobile Devices

Around-device interaction promises to extend the input space of mobile and wearable devices beyond the common but restricted touchscreen. So far, most around-device interaction approaches rely on instrumenting the device or the environment with additional sensors. We believe, that the full potential of ordinary cameras, specifically user-facing cameras, which are integrated in most mobile devices today, are not used to their full potential, yet. We To this end, we present a novel approach for extending the input space around unmodified mobile devices using built-in front-facing cameras of unmodified handheld devices. Our approach estimates hand poses and gestures through reflections in sunglasses, ski goggles or visors. Thereby, GlassHands creates an enlarged input space, rivaling input reach on large touch displays. We discuss the idea, its limitations and future work.

cs.HC↗

arXiv2016

3D Character Customization for Non-Professional Users in Handheld Augmented Reality

In gaming, customizing individual characters, can create personal bonds between players and their characters. Hence, character customization is a standard component in many games. While mobile Augmented Reality (AR) games become popular, to date, no 3D character editor for AR games exists. We investigate the feasibility of 3D character customization for smartphone-based AR in an iterative design process. Specifically, we present findings from creating AR prototypes in a handheld AR setting. In a first user study, we found that a tangible AR prototype resulted in higher hedonistic measures than a camera-based approach. In a follow up study, we compared the tangible AR prototype with a non-AR touchscreen version for selection, scaling, translation and rotation tasks in a 3D character customization setting. The tangible AR version resulted in significantly better results for stimulation and novelty measures than the non-AR version. At the same time, it maintained a proficient level in pragmatic measures such as accuracy and efficiency.

cs.HC↗

arXiv2016

Challenges in Mobile Multi-Device Ecosystems

Coordinated multi-display environments from the desktop, second-screen to gigapixel display walls are increasingly common. Personal and intimate mobile and wearable devices such as head-mounted displays, smartwatches, smartphones and tablets are rarely part of such multi-device ecosystems. With this paper, we contribute to a better understanding about factors that impede the creation and use of such mobile multi-device ecosystems. We base our findings on literature research and an expert survey. Specifically, we present grounded challenges relevant for the design, development and use of mobile multi-device environments.

cs.HC↗