Emerging Natural User Interfaces in Mobile Computing: A Bottoms-Up Survey

Abstract

Mobile and wearable interfaces and interaction paradigms are highly constrained by the available screen real estate, and the computational and power resources. Although there exist many ways of displaying information to mobile users, inputting data to a mobile device is, usually, limited to a conventional touch based interaction, that distracts users from their ongoing activities. Furthermore, emerging applications, like augmented, mixed and virtual reality (AR/MR/VR), require new types of input methods in order to interact with complex virtual worlds, challenging the traditional techniques of Human-Computer Interaction (HCI). Leveraging of Natural User Interfaces (NUIs), as a paradigm of using natural intuitive actions to interact with computing systems, is one of many ways to meet these challenges in mobile computing and its modern applications. Brain-Machine Interfaces that enable thought-only hands-free interaction, Myoelectric input methods that track body gestures and gaze-tracking input interfaces - are the examples of NUIs applicable to mobile and wearable interactions. The wide adoption of wearable devices and the penetration of mobile technologies, alongside with the growing market of AR/MR/VR, motivates the exploration and implementation of new interaction paradigms. The concurrent development of bio-signal acquisition techniques and accompanying ecosystems offers a useful toolbox to address open challenges. In this survey, we present state-of-the-art bio-signal acquisition methods, summarize and evaluate recent developments in the area of NUIs and outline potential application in mobile scenarios. The survey will provide a bottoms-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) portable NUI hardware solutions, (iii) NUI-enabled applications, as well as (iv) research challenges and open problems.