arXiv subjects

Steve Mann

Publications and source records attributed to Steve Mann.

7 recordsLinked to original sources

arXiv2026

Toward a Stable and Deployable Adaptive Chirplet Transform: Residual Projection, Hybrid GPU Acceleration, and Multi-Channel Scalability

The Adaptive Chirplet Transform is a flexible framework that can decompose non-stationary signals into sparse chirplets; it has been applied to signals such as electroencephalography, electromyography and radar. However, the practical deployment of this transform has been hindered by two challenges: algorithmic instability in prior implementations, which can lead to divergent decompositions, and the computational cost of searching over a high-dimensional parameter space. This paper addresses both by a sequence of contributions. Firstly, unit normalization and residual-based projection are introduced to align the decomposition with Matching Pursuit Theory, thereby eliminating divergence and substantially reducing residual error across all signal domains, as demonstrated on three representative signal types. A hybrid CPU-GPU architecture offloads chirplet family generation to the CPU while parallelizing the search on the GPU, removing bottlenecks in CPU-only search and GPU-only generation, achieving speedups of 6.6-7.38x on desktop hardware, with consistent gains observed across laptop and embedded platforms. Multichannel batching enabled simultaneous multi-signal processing, amplifying the speedup, which scaled from 3.94x for a single channel to 8.22x at 10 channels. Finally, a hierarchical coarse-to-fine search, inspired by Logon Expectation Maximization, is introduced. This reduced peak memory usage below 1 GB while maintaining similar reconstruction quality, at the cost of longer runtime. Together, these contributions establish a correct, stable and practically deployable foundation for chirplet-based signal decomposition. Index Terms: Chirplet Transform, GPU Computing, Matching Pursuit, Signal Decomposition, Sparse Representation, Time-Frequency Analysis

eess.SP↗

arXiv2024

Advancing Technology for Humanity and Earth (+Water+Air)

As technology advances, the integration of physical, virtual, and social worlds has led to a complex landscape of ``Realities'' such as Virtual Reality (VR), Augmented Reality (AR), metaverse, spatial computing, and other emerging paradigms. This paper builds upon and refines the concept of eXtended Reality (XR) as the unifying framework that not only interpolates across these diverse realities but also extrapolates (extends) to create entirely new possibilities. XR is the ``physical spatial metaverse,'' bridging the physical world, the virtual world of artificial intelligence, and the social world of human interaction. These three worlds define the Socio-Cyber-Physical Taxonomy of XR that allows us to identify underexplored research areas such as Diminished Reality (DR), and chart future directions to {\bf advance technology for people and planet}. We highlight the six core properties of XR for applications in sustainability, healthcare, frontline work, and daily life. Central to this vision is the development of AI-driven wearable technologies, such as the smart eyeglass, that sustainably extend human capabilities.

cs.HC↗

arXiv2022

Beyond the Metaverse: XV (eXtended meta/uni/Verse)

We propose the term and concept XV (eXtended meta/omni/uni/Verse) as an alternative to, and generalization of, the shared/social virtual reality widely known as ``metaverse''. XV is shared/social XR. We, and many others, use XR (eXtended Reality) as a broad umbrella term and concept to encompass all the other realities, where X is an ``anything'' variable, like in mathematics, to denote any reality, X $\in$ \{physical, virtual, augmented, \ldots \} reality. Therefore XV inherits this generality from XR. We begin with a very simple organized taxonomy of all these realities in terms of two simple building blocks: (1) physical reality (PR) as made of ``atoms'', and (2) virtual reality (VR) as made of ``bits''. Next we introduce XV as combining all these realities with extended society as a three-dimensional space and taxonomy of (1) ``atoms'' (physical reality), (2) ``bits'' (virtuality), and (3) ``genes'' (sociality). Thus those working in the liminal space between Virtual Reality (VR), Augmented Reality (AR), metaverse, and their various extensions, can describe their work and research as existing in the new field of XV. XV includes the metaverse along with extensions of reality itself like shared seeing in the infrared, ultraviolet, and shared seeing of electromagnetic radio waves, sound waves, and electric currents in motors. For example, workers in a mechanical room can look at a pump and see a superimposed time-varying waveform of the actual rotating magnetic field inside its motor, in real time, while sharing this vision across multiple sites. Presented at IEEE Standards Association, Behind and Beyond the Metaverse: XV (eXtended meta/uni/Verse), Thurs. Dec. 8, 2022, 2:15-3:30pm, EST.

eess.SY↗

arXiv2021

"Vironment": An Art of Wearable Social Distancing

"Vironment" is a series of art pieces, social commentary, technology, etc., based on wearable health technologies of social-distancing, culminating in a social-distancing device that takes the familiar world of security and surveillance technologies that surround us and re-situates it on the body of the wearer (technologies that become part of us). This piece also introduces a conceptual framework for (1) the sensing of the self together with (2) sensing of others and (3) sensing of the environment around us.

cs.CY↗

arXiv2021

Wearable Affective Memory Augmentation

Human memory prioritizes the storage and recall of information that is emotionally-arousing and/or important in a process known as value-directed memory. When experiencing a stream of information (e.g. conversation, book, lecture, etc.), the individual makes conscious and subconscious value assessments of the incoming information and uses this as a metric to determine what to remember. In order to improve automatic recall of memory, previous memory augmentation systems have sensed users' physiological state to determine which sensory media should be prioritized. Here, we propose to prioritize memories using the affective state of individuals that the user is interacting with. Thereby, the proposed wearable Affective Memory Augmentation system uses affective information from the user's social companions in order to facilitate value-directed memory.

cs.HC↗

arXiv2021

Deep Learning for Enhanced Scratch Input

The vibrations generated from scratching and tapping on surfaces can be highly expressive and recognizable, and have therefore been proposed as a method of natural user interface (NUI). Previous systems require custom sensor hardware such as contact microphones and have struggled with gesture classification accuracy. We propose a deep learning approach to scratch input. Using smartphones and tablets laid on tabletops or other similar surfaces, our system achieved a gesture classification accuracy of 95.8\%, substantially reducing gesture misclassification from previous works. Further, our system achieved this performance when tested on a wide variety of surfaces, mobile devices, and in high noise environments. The results indicate high potential for the application of deep learning techniques to natural user interface (NUI) systems that can readily convert large unpowered surfaces into a user interface using just a smartphone with no special-purpose sensors or hardware.

cs.HC↗

arXiv2018

All Reality: Virtual, Augmented, Mixed (X), Mediated (X,Y), and Multimediated Reality

The contributions of this paper are: (1) a taxonomy of the "Realities" (Virtual, Augmented, Mixed, Mediated, etc.), and (2) some new kinds of "reality" that come from nature itself, i.e. that expand our notion beyond synthetic realities to include also phenomenological realities. VR (Virtual Reality) replaces the real world with a simulated experience (virtual world). AR (Augmented Reality) allows a virtual world to be experienced while also experiencing the real world at the same time. Mixed Reality provides blends that interpolate between real and virtual worlds in various proportions, along a "Virtuality" axis, and extrapolate to an "X-axis". Mediated Reality goes a step further by mixing/blending and also modifying reality. This modifying of reality introduces a second axis. Mediated Reality is useful as a seeing aid (e.g. modifying reality to make it easier to understand), and for psychology experiments like Stratton's 1896 upside-down eyeglasses experiment. We propose Multimediated Reality as a multidimensional multisensory mediated reality that includes not just interactive multimedia-based reality for our five senses, but also includes additional senses (like sensory sonar, sensory radar, etc.), as well as our human actions/actuators. These extra senses are mapped to our human senses using synthetic synesthesia. This allows us to directly experience real (but otherwise invisible) phenomena, such as wave propagation and wave interference patterns, so that we can see radio waves and sound waves and how they interact with objects and each other. Multimediated reality is multidimensional, multimodal, multisensory, and multiscale. It is also multidisciplinary, in that we must consider not just the user, but also how the technology affects others, e.g. how its physical appearance affects social situations.

cs.HC↗