Morteza Khaleghimeybodi

arXiv2026

Towards Localizing Conversation Partners using Head Motion

Many individuals struggle to understand conversation partners in noisy settings, particularly amid background speakers or due to hearing impairments. Emerging wearables like smartglasses offer a transformative opportunity to enhance speech from conversation partners. Crucial to this is identifying the direction in which the user wants to listen, which we refer to as the user's acoustic zones of interest. While current spatial audio-based methods can resolve the direction of vocal input, they are agnostic to listening preferences and have limited functionality in noisy settings with interfering speakers. To address this, behavioral cues are needed to actively infer a user's acoustic zones of interest. We explore the effectiveness of head-orienting behavior, captured by Inertial Measurement Units (IMUs) on smartglasses, as a modality for localizing these zones in seated conversations. We introduce HALo, a head-orientation-based acoustic zone localization network that leverages smartglasses' IMUs to non-invasively infer auditory zones of interest corresponding to conversation partner locations. By integrating an a priori estimate of the number of conversation partners, our approach yields a 21% performance improvement over existing methods. We complement this with CoCo, which classifies the number of conversation partners using only IMU data, achieving 0.74 accuracy and a 35% gain over rule-based and generic time-series baselines. We discuss practical considerations for feature extraction and inference and provide qualitative analyses over extended sessions. We also demonstrate a minimal end-to-end speech enhancement system, showing that head-orientation-based localization offers clear advantages in extremely noisy settings with multiple conversation partners.

cs.HC↗

arXiv2024

Non-verbal Hands-free Control for Smart Glasses using Teeth Clicks

Smart glasses are emerging as a popular wearable computing platform potentially revolutionizing the next generation of human-computer interaction. The widespread adoption of smart glasses has created a pressing need for discreet and hands-free control methods. Traditional input techniques, such as voice commands or tactile gestures, can be intrusive and non-discreet. Additionally, voice-based control may not function well in noisy acoustic conditions. We propose a novel, discreet, non-verbal, and non-tactile approach to controlling smart glasses through subtle vibrations on the skin induced by teeth clicking. We demonstrate that these vibrations can be sensed by accelerometers embedded in the glasses with a low-footprint predictive model. Our proposed method, called STEALTHsense, utilizes a temporal broadcasting-based neural network architecture with just 88K trainable parameters and 7.14M Multiply and Accumulate (MMAC) per inference unit. We benchmark our proposed STEALTHsense against state-of-the-art deep learning approaches and traditional low-footprint machine learning approaches. We conducted a study across 21 participants to collect representative samples for two distinct teeth-clicking patterns and many non-patterns for robust training of STEALTHsense, achieving an average cross-person accuracy of 0.93. Field testing confirmed its effectiveness, even in noisy conditions, underscoring STEALTHsense's potential for real-world applications, offering a promising solution for smart glasses interaction.

cs.HC↗