arXiv subjects

Mohamed Khamis

Publications and source records attributed to Mohamed Khamis.

At least 19 recordsLinked to original sources

arXiv2026

The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts

As eye-tracking becomes increasingly common in modern mobile devices, the potential for hands-free, gaze-based interaction grows, but current gesture sets are largely expert-designed and often misaligned with how users naturally move their eyes. To address this gap, we introduce a two-phase methodology for developing intuitive gaze gestures. First, four co-design workshops with 20 non-expert participants generated 102 initial concepts. Next, four gaze interaction experts reviewed and refined these into a set of 32 gestures. We found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar; i.e., activation (dwell) + action (gaze gesture or blink), to ensure intentionality and mitigate the classic Midas Touch problem. Experts prioritized gestures that are ergonomically sound, aligned with natural saccades, and reliably distinguishable. The resulting user-grounded, expert-validated gesture set, along with actionable design principles, provides a foundation for developing intuitive, hands-free interfaces for gaze-enabled devices.

cs.HC↗

arXiv2026

GazeBlend: Exploring Paired Gaze-Based Input Techniques for Navigation and Selection Tasks on Mobile Devices

The potential of gaze for hands-free mobile interaction is increasingly evident. While each gaze input technique presents distinct advantages and limitations, a combination can amplify strengths and mitigate challenges. We report on the results of a user study (N=24), in which we compared the usability and performance of pairing three popular gaze input techniques: Dwell Time, Pursuits, and Gaze Gestures, for navigation and selection tasks while sitting and walking. Results show that pairing gestures for navigation with either Dwell time or Pursuits for selection improves task completion time and rate compared to using either individually. We discuss the implications of pairing gaze input techniques, such as how Pursuits may negatively impact other techniques, likely due to the visual clutter it adds, how integrating gestures for navigation reduces the chances of unintentional selections, and the impact of motor activity on performance. Our findings provide insights for effective gaze-enabled interfaces.

cs.HC↗

arXiv2025

Mind the Gaze: Improving the Usability of Dwell Input by Adapting Gaze Targets Based on Viewing Distance

Dwell input shows promise for handheld mobile contexts, but its performance is impacted by target size and viewing distance. While fixed target sizes suffice in static setups, in mobile settings, frequent posture changes alter viewing distances, which in turn distort perceived size and hinder dwell performance. We address this through GAUI, a Gaze-based Adaptive User Interface that dynamically resizes targets to maximise performance at the given viewing distance. In a two-phased study (N=24), GAUI leveraged the strengths of its distance-responsive design, outperforming the large UI static baseline in task time, and being less error-prone than the small UI static baseline. It was rated the most preferred interface overall. Participants reflected on using GAUI in six different postures. We discuss how their experience is impacted by posture, and propose guidelines for designing context-aware adaptive UIs for dwell interfaces on handheld mobile devices that maximise performance.

cs.HC↗

arXiv2025

Investigating the Effect of Encumbrance on Gaze- and Touch-based Target Acquisition on Handheld Mobile Devices

The potential of using gaze as an input modality in the mobile context is growing. While users often encumber themselves by carrying objects and using mobile devices while walking, the impact of encumbrance on gaze input performance remains unexplored. To investigate this, we conducted a user study (N=24) to evaluate the effect of encumbrance on the performance of 1) Gaze using Dwell time (with/without visual feedback), 2) GazeTouch (with/without visual feedback), and 3) One- or two-hand touch input. While Touch generally performed better, Gaze, especially with feedback, showed a consistent performance regardless of whether participants were encumbered or unencumbered. Participants' preferences for input modalities varied with encumbrance: they preferred Gaze when encumbered, and touch when unencumbered. Our findings enhance understanding of the effect of encumbrance on gaze input and contribute towards selecting appropriate input modalities in future mobile user interfaces to account for situational impairments.

cs.HC↗

arXiv2025

GazeCopilot: Evaluating Novel Gaze-Informed Prompting for AI-Supported Code Comprehension and Readability

AI-powered coding assistants, like GitHub Copilot, are increasingly used to boost developers' productivity. However, their output quality hinges on the contextual richness of the prompts. Meanwhile, gaze behaviour carries rich cognitive information, providing insights into how developers process code. We leverage this in Real-time GazeCopilot, a novel approach that refines prompts using real-time gaze data to improve code comprehension and readability by integrating gaze metrics, like fixation patterns and pupil dilation, into prompts to adapt suggestions to developers' cognitive states. In a controlled lab study with 25 developers, we evaluated Real-time GazeCopilot against two baselines: Standard Copilot, which relies on text prompts provided by developers, and Pre-set GazeCopilot, which uses a hard-coded prompt that assumes developers' gaze metrics indicate they are struggling with all aspects of the code, allowing us to assess the impact of leveraging the developer's personal real-time gaze data. Our results show that prompts dynamically generated using developers' real-time gaze data significantly improve code comprehension accuracy, reduce comprehension time, and improve perceived readability compared to Standard Copilot. Our Real-time GazeCopilot approach selectively refactors only code aspects where gaze data indicate difficulty, outperforming the overgeneralized refactoring done by Pre-set GazeCopilot by avoiding revising code the developer already understands.

cs.HC↗

arXiv2025

Understanding Dynamic Human-Robot Proxemics in the Case of Four-Legged Canine-Inspired Robots

The integration of humanoid and animal-shaped robots into specialized domains, such as healthcare, multi-terrain operations, and psychotherapy, necessitates a deep understanding of proxemics--the study of spatial behavior that governs effective human-robot interactions. Unlike traditional robots in manufacturing or logistics, these robots must navigate complex human environments where maintaining appropriate physical and psychological distances is crucial for seamless interaction. This study explores the application of proxemics in human-robot interactions, focusing specifically on quadruped robots, which present unique challenges and opportunities due to their lifelike movement and form. Utilizing a motion capture system, we examine how different interaction postures of a canine robot influence human participants' proxemic behavior in dynamic scenarios. By capturing and analyzing position and orientation data, this research aims to identify key factors that affect proxemic distances and inform the design of socially acceptable robots. The findings underscore the importance of adhering to human psychological and physical distancing norms in robot design, ensuring that autonomous systems can coexist harmoniously with humans.

cs.HC↗

arXiv2025

MAC-Gaze: Motion-Aware Continual Calibration for Mobile Gaze Tracking

Mobile gaze tracking faces a fundamental challenge: maintaining accuracy as users naturally change their postures and device orientations. Traditional calibration approaches, like one-off, fail to adapt to these dynamic conditions, leading to degraded performance over time. We present MAC-Gaze, a Motion-Aware continual Calibration approach that leverages smartphone Inertial measurement unit (IMU) sensors and continual learning techniques to automatically detect changes in user motion states and update the gaze tracking model accordingly. Our system integrates a pre-trained visual gaze estimator and an IMU-based activity recognition model with a clustering-based hybrid decision-making mechanism that triggers recalibration when motion patterns deviate significantly from previously encountered states. To enable accumulative learning of new motion conditions while mitigating catastrophic forgetting, we employ replay-based continual learning, allowing the model to maintain performance across previously encountered motion conditions. We evaluate our system through extensive experiments on the publicly available RGBDGaze dataset and our own 10-hour multimodal MotionGaze dataset (481K+ images, 800K+ IMU readings), encompassing a wide range of postures under various motion conditions including sitting, standing, lying, and walking. Results demonstrate that our method reduces gaze estimation error by 19.9% on RGBDGaze (from 1.73 cm to 1.41 cm) and by 31.7% on MotionGaze (from 2.81 cm to 1.92 cm) compared to traditional calibration approaches. Our framework provides a robust solution for maintaining gaze estimation accuracy in mobile scenarios.

cs.HC↗

arXiv2025

Quantifying the Impact of Motion on 2D Gaze Estimation in Real-World Mobile Interactions

Mobile gaze tracking involves inferring a user's gaze point or direction on a mobile device's screen from facial images captured by the device's front camera. While this technology inspires an increasing number of gaze-interaction applications, achieving consistent accuracy remains challenging due to dynamic user-device spatial relationships and varied motion conditions inherent in mobile contexts. This paper provides empirical evidence on how user mobility and behaviour affect mobile gaze tracking accuracy. We conduct two user studies collecting behaviour and gaze data under various motion conditions - from lying to maze navigation - and during different interaction tasks. Quantitative analysis has revealed behavioural regularities among daily tasks and identified head distance, head pose, and device orientation as key factors affecting accuracy, with errors increasing by up to 48.91% in dynamic conditions compared to static ones. These findings highlight the need for more robust, adaptive eye-tracking systems that account for head movements and device deflection to maintain accuracy across diverse mobile contexts.

cs.HC↗

arXiv2025

Do We Need Responsible XR? Drawing on Responsible AI to Inform Ethical Research and Practice into XRAI / the Metaverse

This position paper for the CHI 2025 workshop "Everyday AR through AI-in-the-Loop" reflects on whether as a field HCI needs to define Responsible XR as a parallel to, and in conjunction with, Responsible AI, addressing the unique vulnerabilities posed by mass adoption of wearable AI-enabled AR glasses and XR devices that could enact AI-driven human perceptual augmentation.

cs.HC↗

arXiv2025

Exploring the Perspectives of Social VR-Aware Non-Parent Adults and Parents on Children's Use of Social Virtual Reality

Social Virtual Reality (VR), where people meet in virtual spaces via 3D avatars, is used by children and adults alike. Children experience new forms of harassment in social VR where it is often inaccessible to parental oversight. To date, there is limited understanding of how parents and non-parent adults within the child social VR ecosystem perceive the appropriateness of social VR for different age groups and the measures in place to safeguard children. We present results of a mixed-methods questionnaire (N=149 adults, including 79 parents) focusing on encounters with children in social VR and perspectives towards children's use of social VR. We draw novel insights on the frequency of social VR use by children under 13 and current use of, and future aspirations for, child protection interventions. Compared to non-parent adults, parents familiar with social VR propose lower minimum ages and are more likely to allow social VR without supervision. Adult users experience immaturity from children in social VR, while children face abuse, encounter age-inappropriate behaviours and self-disclose to adults. We present directions to enhance the safety of social VR through pre-planned controls, real-time oversight, post-event insight and the need for evidence-based guidelines to support parents and platforms around age-appropriate interventions.

cs.HC↗

arXiv2024

"What you think is private is no longer" -- Investigating the Aftermath of Shoulder Surfing on Smartphones in Everyday Life through the Eyes of the Victims

Shoulder surfing has been studied extensively, however, it remains unexplored whether and how it impacts users. Understanding this is important as it determines whether shoulder surfing poses a significant concern and, if so, how best to address it. By surveying smartphone users in the UK, we explore how shoulder surfing impacts a) the privacy perceptions of victim users and b) their interaction with smartphones. We found that the impact of being shoulder surfed is highly individual. It is perceived as unavoidable and frequently occurring, leading to increased time for task completion. Individuals are concerned for their own and other peoples privacy, seeing shoulder surfing as a gateway to more serious threats like identity or device theft. Participants expressed a willingness to alter their behaviour and use software based protective measures to prevent shoulder surfing, yet, this comes with a set of user defined criteria, such as effectiveness, affordability, reliability, and availability. We discuss future work directions for user-centred shoulder surfing mitigation.

cs.HC↗

arXiv2024

SoK: Privacy Personalised -- Mapping Personal Attributes \& Preferences of Privacy Mechanisms for Shoulder Surfing

Shoulder surfing is a byproduct of smartphone use that enables bystanders to access personal information (such as text and photos) by making screen observations without consent. To mitigate this, several protection mechanisms have been proposed to protect user privacy. However, the mechanisms that users prefer remain unexplored. This paper explores correlations between personal attributes and properties of shoulder surfing protection mechanisms. For this, we first conducted a structured literature review and identified ten protection mechanism categories against content-based shoulder surfing. We then surveyed N=192 users and explored correlations between personal attributes and properties of shoulder surfing protection mechanisms. Our results show that users agreed that the presented mechanisms assisted in protecting their privacy, but they preferred non-digital alternatives. Among the mechanisms, participants mainly preferred an icon overlay mechanism followed by a tangible mechanism. We also found that users who prioritized out-of-device privacy and a high tendency to interact with technology favoured the personalisation of protection mechanisms. On the contrary, age and smartphone OS did not impact users' preference for perceived usefulness and personalisation of mechanisms. Based on the results, we present key takeaways to support the design of future protection mechanisms.

cs.HC↗

arXiv2024

IFTT-PIN: A Self-Calibrating PIN-Entry Method

Personalising an interface to the needs and preferences of a user often incurs additional interaction steps. In this paper, we demonstrate a novel method that enables the personalising of an interface without the need for explicit calibration procedures, via a process we call self-calibration. A second-order effect of self-calibration is that an outside observer cannot easily infer what a user is trying to achieve because they cannot interpret the user's actions. To explore this security angle, we developed IFTT-PIN (If This Then PIN) as the first self-calibrating PIN-entry method. When using IFTT-PIN, users are free to choose any button for any meaning without ever explicitly communicating their choice to the machine. IFTT-PIN infers both the user's PIN and their preferred button mapping at the same time. This paper presents the concept, implementation, and interactive demonstrations of IFTT-PIN, as well as an evaluation against shoulder surfing attacks. Our study (N=24) shows that by adding self-calibration to an existing PIN entry method, IFTT-PIN statistically significantly decreased PIN attack decoding rate by ca. 8.5 times (p=1.1e-9), while only decreasing the PIN entry encoding rate by ca. 1.4 times (p=0.02), leading to a positive security-usability trade-off. IFTT-PIN's entry rate significantly improved 21 days after first exposure (p=3.6e-6) to the method, suggesting self-calibrating interfaces are memorable despite using an initially undefined user interface. Self-calibration methods might lead to novel opportunities for interaction that are more inclusive and versatile, a potentially interesting challenge for the community. A short introductory video is available at https://youtu.be/pP5sfniNRns.

cs.HC↗

arXiv2023

An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.

cs.HC↗

arXiv2023

DynamicRead: Exploring Robust Gaze Interaction Methods for Reading on Handheld Mobile Devices under Dynamic Conditions

Enabling gaze interaction in real-time on handheld mobile devices has attracted significant attention in recent years. An increasing number of research projects have focused on sophisticated appearance-based deep learning models to enhance the precision of gaze estimation on smartphones. This inspires important research questions, including how the gaze can be used in a real-time application, and what type of gaze interaction methods are preferable under dynamic conditions in terms of both user acceptance and delivering reliable performance. To address these questions, we design four types of gaze scrolling techniques: three explicit technique based on Gaze Gesture, Dwell time, and Pursuit; and one implicit technique based on reading speed to support touch-free, page-scrolling on a reading application. We conduct a 20-participant user study under both sitting and walking settings and our results reveal that Gaze Gesture and Dwell time-based interfaces are more robust while walking and Gaze Gesture has achieved consistently good scores on usability while not causing high cognitive workload.

cs.HC↗

arXiv2022

State-of-the-Art in Smart Contact Lenses for Human Machine Interaction

Contact lenses have traditionally been used for vision correction applications. Recent advances in microelectronics and nanofabrication on flexible substrates have now enabled sensors, circuits and other essential components to be integrated on a small contact lens platform. This has opened up the possibility of using contact lenses for a range of human-machine interaction applications including vision assistance, eye tracking, displays and health care. In this article, we systematically review the range of smart contact lens materials, device architectures and components that facilitate this interaction for different applications. In fact, evidence from our systematic review demonstrates that these lenses can be used to display information, detect eye movements, restore vision and detect certain biomarkers in tear fluid. Consequently, whereas previous state-of the-art reviews in contact lenses focused exclusively on biosensing, our systematic review covers a wider range of smart contact lens applications in HMI. Moreover, we present a new method of classifying the literature on smart contact lenses according to their six constituent building blocks, which are the sensing, energy management, driver electronics, communications, substrate and the interfacing modules. Based on recent developments in each of these categories, we speculate the challenges and opportunities of smart contact lenses for human-machine interaction. Moreover, we propose a novel self-powered smart contact lens concept with integrated energy harvesters, sensors and communication modules to enable autonomous operation. Our review is therefore a critical evaluation of current data and is presented with the aim of guiding researchers to new research directions in smart contact lenses.

cs.HC↗

arXiv2022

The Dark Side of Perceptual Manipulations in Virtual Reality

"Virtual-Physical Perceptual Manipulations" (VPPMs) such as redirected walking and haptics expand the user's capacity to interact with Virtual Reality (VR) beyond what would ordinarily physically be possible. VPPMs leverage knowledge of the limits of human perception to effect changes in the user's physical movements, becoming able to (perceptibly and imperceptibly) nudge their physical actions to enhance interactivity in VR. We explore the risks posed by the malicious use of VPPMs. First, we define, conceptualize and demonstrate the existence of VPPMs. Next, using speculative design workshops, we explore and characterize the threats/risks posed, proposing mitigations and preventative recommendations against the malicious use of VPPMs. Finally, we implement two sample applications to demonstrate how existing VPPMs could be trivially subverted to create the potential for physical harm. This paper aims to raise awareness that the current way we apply and publish VPPMs can lead to malicious exploits of our perceptual vulnerabilities.

cs.HC↗

arXiv2019

"Please enter your PIN" -- On the Risk of Bypass Attacks on Biometric Authentication on Mobile Devices

Nowadays, most mobile devices support biometric authentication schemes like fingerprint or face unlock. However, these probabilistic mechanisms can only be activated in combination with a second alternative factor, usually knowledge-based authentication. In this paper, we show that this aspect can be exploited in a bypass attack. In this bypass attack, the attacker forces the user to "bypass" the biometric authentication by, for example, resetting the phone. This forces the user to enter an easy-to-observe passcode instead. We present the threat model and provide preliminary results of an online survey. Based on our results, we discuss potential countermeasures. We conclude that better feedback design and security-optimized fallback mechanisms can help further improve the overall security of mobile unlock mechanisms while preserving usability.

cs.HC↗