arXiv subjects

Jakob Struye

Publications and source records attributed to Jakob Struye.

17 recordsLinked to original sources

arXiv2026

Millimeter-Wave Gesture Recognition in ISAC: Does Reducing Sensing Airtime Hamper Accuracy?

Most Integrated Sensing and Communications (ISAC) systems require dividing airtime across their two modes. However, the specific impact of this decision on sensing performance remains unclear and underexplored. In this paper, we therefore investigate the impact on a gesture recognition system using a Millimeter-Wave (mmWave) ISAC system. With our dataset of power per beam pair gathered with two mmWave devices performing constant beam sweeps while test subjects performed distinct gestures, we train a gesture classifier using Convolutional Neural Networks. We then subsample these measurements, emulating reduced sensing airtime, showing that a sensing airtime of 25 % only reduces classification accuracy by 0.15 percentage points from full-time sensing. Alongside this high-quality sensing at low airtime, mmWave systems are known to provide extremely high data throughputs, making mmWave ISAC a prime enabler for applications such as truly wireless Extended Reality.

eess.SP↗

arXiv2025

Beyond Sub-6 GHz: Leveraging mmWave Wi-Fi for Gait-Based Person Identification

Person identification plays a vital role in enabling intelligent, personalized, and secure human-computer interaction. Recent research has demonstrated the feasibility of leveraging Wi-Fi signals for passive person identification using a person's unique gait pattern. Although most existing work focuses on sub-6 GHz frequencies, the emergence of mmWave offers new opportunities through its finer spatial resolution, though its comparative advantages for person identification remain unexplored. This work presents the first comparative study between sub-6 GHz and mmWave Wi-Fi signals for person identification with commercial off-the-shelf (COTS) Wi-Fi, using a novel dataset of synchronized measurements from the two frequency bands in an indoor environment. To ensure a fair comparison, we apply identical training pipelines and model configurations across both frequency bands. Leveraging end-to-end deep learning, we show that even at low sampling rates (10 Hz), mmWave Wi-Fi signals can achieve high identification accuracy (91.2% on 20 individuals) when combined with effective background subtraction.

cs.LG↗

arXiv2025

mmHSense: Multi-Modal and Distributed mmWave ISAC Datasets for Human Sensing

This article presents mmHSense, a set of open labeled mmWave datasets to support human sensing research within Integrated Sensing and Communication (ISAC) systems. The datasets can be used to explore mmWave ISAC for various end applications such as gesture recognition, person identification, pose estimation, and localization. Moreover, the datasets can be used to develop and advance signal processing and deep learning research on mmWave ISAC. This article describes the testbed, experimental settings, and signal features for each dataset. Furthermore, the utility of the datasets is demonstrated through validation on a specific downstream task. In addition, we demonstrate the use of parameter-efficient fine-tuning to adapt ISAC models to different tasks, significantly reducing computational complexity while maintaining performance on prior tasks.

cs.CV↗

arXiv2025

Can Millimeter-Wave Support Interactive Extended Reality Under Rapid Rotational Motion?

Using Millimeter-Wave (mmWave) wireless communications is often named as the prime enabler for mobile interactive Extended Reality (XR), as it offers multi-gigabit data rates at millisecond-range latency. To achieve this, mmWave nodes must focus their energy towards each other, which is especially challenging in XR scenarios, where the transceiver on the user's XR device may rotate rapidly. To evaluate the feasibility of mmWave XR, we present the first throughput and latency evaluation of state-of-the-art mmWave hardware under rapid rotational motion, for different PHY and MAC-layer parameter configurations. We show that this parameter configuration has a significant impact on performance, and that specialized beamforming approaches for rapid rotational motion may be necessary to enable uninterrupted, high-quality mobile interactive XR experiences.

cs.NI↗

arXiv2025

Opportunities and Challenges for Virtual Reality Streaming over Millimeter-Wave: An Experimental Analysis

Achieving extremely high-quality and truly immersive interactive Virtual Reality (VR) is expected to require a wireless link to the cloud, providing multi-gigabit throughput and extremely low latency. A prime candidate for fulfilling these requirements is millimeter-wave (mmWave) communications, operating in the 30 to 300 GHz bands, rather than the traditional sub-6 GHz. Evaluations with first-generation mmWave Wi-Fi hardware, based on the IEEE 802.11ad standard, have so far largely remained limited to lower-layer metrics. In this work, we present the first experimental analysis of the capabilities of mmWave for streaming VR content, using a novel testbed capable of repeatably creating blockage through mobility. Using this testbed, we show that (a) motion may briefly interrupt transmission, (b) a broken line of sight may degrade throughput unpredictably, and (c) TCP-based streaming frameworks need careful tuning to behave well over mmWave.

cs.NI↗

arXiv2025

Generating Realistic Synthetic Head Rotation Data for Extended Reality using Deep Learning

Extended Reality is a revolutionary method of delivering multimedia content to users. A large contributor to its popularity is the sense of immersion and interactivity enabled by having real-world motion reflected in the virtual experience accurately and immediately. This user motion, mainly caused by head rotations, induces several technical challenges. For instance, which content is generated and transmitted depends heavily on where the user is looking. Seamless systems, taking user motion into account proactively, will therefore require accurate predictions of upcoming rotations. Training and evaluating such predictors requires vast amounts of orientational input data, which is expensive to gather, as it requires human test subjects. A more feasible approach is to gather a modest dataset through test subjects, and then extend it to a more sizeable set using synthetic data generation methods. In this work, we present a head rotation time series generator based on TimeGAN, an extension of the well-known Generative Adversarial Network, designed specifically for generating time series. This approach is able to extend a dataset of head rotations with new samples closely matching the distribution of the measured time series.

cs.CV↗

arXiv2025

Toward Interactive Multi-User Extended Reality Using Millimeter-Wave Networking

Extended Reality (XR) enables a plethora of novel interactive shared experiences. Ideally, users are allowed to roam around freely, while audiovisual content is delivered wirelessly to their Head-Mounted Displays (HMDs). Therefore, truly immersive experiences will require massive amounts of data, in the range of tens of gigabits per second, to be delivered reliably at extremely low latencies. We identify Millimeter-Wave (mmWave) communications, at frequencies between 24 and 300 GHz, as a key enabler for such experiences. In this article, we show how the mmWave state of the art does not yet achieve sufficient performance, and identify several key active research directions expected to eventually pave the way for extremely-high-quality mmWave-enabled interactive multi-user XR.

cs.NI↗

arXiv2025

Multi-Gigabit Interactive Extended Reality over Millimeter-Wave: An End-to-End System Approach

Achieving high-quality wireless interactive Extended Reality (XR) will require multi-gigabit throughput at extremely low latency. The Millimeter-Wave (mmWave) frequency bands, between 24 and 300GHz, can achieve such extreme performance. However, maintaining a consistently high Quality of Experience with highly mobile users is challenging, as mmWave communications are inherently directional. In this work, we present and evaluate an end-to-end approach to such a mmWave-based mobile XR system. We perform a highly realistic simulation of the system, incorporating accurate XR data traffic, detailed mmWave propagation models and actual user motion. We evaluate the impact of the beamforming strategy and frequency on the overall performance. In addition, we provide the first system-level evaluation of the CoVRage algorithm, a proactive and spatially aware user-side beamforming approach designed specifically for highly mobile XR environments.

cs.NI↗

arXiv2025

CoVRage: Millimeter-Wave Beamforming for Mobile Interactive Virtual Reality

Contemporary Virtual Reality (VR) setups often include an external source delivering content to a Head-Mounted Display (HMD). "Cutting the wire" in such setups and going truly wireless will require a wireless network capable of delivering enormous amounts of video data at an extremely low latency. The massive bandwidth of higher frequencies, such as the millimeter-wave (mmWave) band, can meet these requirements. Due to high attenuation and path loss in the mmWave frequencies, beamforming is essential. In wireless VR, where the antenna is integrated into the HMD, any head rotation also changes the antenna's orientation. As such, beamforming must adapt, in real-time, to the user's head rotations. An HMD's built-in sensors providing accurate orientation estimates may facilitate such rapid beamforming. In this work, we present coVRage, a receive-side beamforming solution tailored for VR HMDs. Using built-in orientation prediction present on modern HMDs, the algorithm estimates how the Angle of Arrival (AoA) at the HMD will change in the near future, and covers this AoA trajectory with a dynamically shaped oblong beam, synthesized using sub-arrays. We show that this solution can cover these trajectories with consistently high gain, even in light of temporally or spatially inaccurate orientational data.

cs.NI↗

arXiv2024

Toward Standardized Performance Evaluation of Flow-guided Nanoscale Localization

Nanoscale devices with Terahertz (THz) communication capabilities are envisioned to be deployed within human bloodstreams. Such devices will enable fine-grained sensing-based applications for detecting early indications (i.e., biomarkers) of various health conditions, as well as actuation-based ones such as targeted drug delivery. Associating the locations of such events with the events themselves would provide an additional utility for precision diagnostics and treatment. This vision yielded a new class of in-body localization coined under the term "flow-guided nanoscale localization". Such localization can be piggybacked on THz communication for detecting body regions in which biological events were observed based on the duration of one circulation of a nanodevice in the bloodstream. From a decades-long research on objective benchmarking of "traditional" indoor localization, as well as its eventual standardization (e.g., ISO/IEC 18305:2016), we know that in early stages the reported performance results were often incomplete (e.g., targeting a subset of relevant performance metrics), carrying out benchmarking experiments in different evaluation environments and scenarios, and utilizing inconsistent performance indicators. To avoid such a "lock-in" in flow-guided localization, in this paper we propose a workflow for standardized performance evaluation of such localization. The workflow is implemented in the form of an open-source simulation framework that is able to jointly account for the mobility of the nanodevices, in-body THz communication between with on-body anchors, and energy-related and other technological constraints (e.g., pulse-based modulation) at the nanodevice level. Accounting for these constraints, the framework is able to generate the raw data that can be streamlined into different flow-guided localization solutions for generating standardized performance benchmarks.

cs.NI↗

arXiv2024

Insights from the Design Space Exploration of Flow-Guided Nanoscale Localization

Nanodevices with Terahertz (THz)-based wireless communication capabilities are providing a primer for flow-guided localization within the human bloodstreams. Such localization is allowing for assigning the locations of sensed events with the events themselves, providing benefits along the lines of early and precise diagnostics, and reduced costs and invasiveness. Flow-guided localization is still in a rudimentary phase, with only a handful of works targeting the problem. Nonetheless, the performance assessments of the proposed solutions are already carried out in a non-standardized way, usually along a single performance metric, and ignoring various aspects that are relevant at such a scale (e.g., nanodevices' limited energy) and for such a challenging environment (e.g., extreme attenuation of in-body THz propagation). As such, these assessments feature low levels of realism and cannot be compared in an objective way. Toward addressing this issue, we account for the environmental and scale-related peculiarities of the scenario and assess the performance of two state-of-the-art flow-guided localization approaches along a set of heterogeneous performance metrics such as the accuracy and reliability of localization.

cs.NI↗

arXiv2024

Graph Neural Networks as an Enabler of Terahertz-based Flow-guided Nanoscale Localization over Highly Erroneous Raw Data

Contemporary research advances in nanotechnology and material science are rooted in the emergence of nanodevices as a versatile tool that harmonizes sensing, computing, wireless communication, data storage, and energy harvesting. These devices offer novel pathways for disease diagnostics, treatment, and monitoring within the bloodstreams. Ensuring precise localization of events of diagnostic interest, which underpins the concept of flow-guided in-body nanoscale localization, would provide an added diagnostic value to the detected events. Raw data generated by the nanodevices is pivotal for this localization and consist of an event detection indicator and the time elapsed since the last passage of a nanodevice through the heart. The energy constraints of the nanodevices lead to intermittent operation and unreliable communication, intrinsically affecting this data. This posits a need for comprehensively modelling the features of this data. These imperfections also have profound implications for the viability of existing flow-guided localization approaches, which are ill-prepared to address the intricacies of the environment. Our first contribution lies in an analytical model of raw data for flow-guided localization, dissecting how communication and energy capabilities influence the nanodevices' data output. This model acts as a vital bridge, reconciling idealized assumptions with practical challenges of flow-guided localization. Toward addressing these practical challenges, we also present an integration of Graph Neural Networks (GNNs) into the flow-guided localization paradigm. GNNs excel in capturing complex dynamic interactions inherent to the localization of events sensed by the nanodevices. Our results highlight the potential of GNNs not only to enhance localization accuracy but also extend coverage to encompass the entire bloodstream.

cs.LG↗

arXiv2023

Predictive Context-Awareness for Full-Immersive Multiuser Virtual Reality with Redirected Walking

The advancement of Virtual Reality (VR) technology is focused on improving its immersiveness, supporting multiuser Virtual Experiences (VEs), and enabling users to move freely within their VEs while remaining confined to specialized VR setups through Redirected Walking (RDW). To meet their extreme data-rate and latency requirements, future VR systems will require supporting wireless networking infrastructures operating in millimeter Wave (mmWave) frequencies that leverage highly directional communication in both transmission and reception through beamforming and beamsteering. We propose the use of predictive context-awareness to optimize transmitter and receiver-side beamforming and beamsteering. By predicting users' short-term lateral movements in multiuser VR setups with Redirected Walking (RDW), transmitter-side beamforming and beamsteering can be optimized through Line-of-Sight (LoS) "tracking" in the users' directions. At the same time, predictions of short-term orientational movements can be utilized for receiver-side beamforming for coverage flexibility enhancements. We target two open problems in predicting these two context information instances: i) predicting lateral movements in multiuser VR settings with RDW, and ii) generating synthetic head rotation datasets for training orientational movements predictors. Our experimental results demonstrate that Long Short-Term Memory (LSTM) networks feature promising accuracy in predicting lateral movements, and context-awareness stemming from VEs further enhances this accuracy. Additionally, we show that a TimeGAN-based approach for orientational data generation can create synthetic samples that closely match experimentally obtained ones.

cs.NI↗

arXiv2022

Short-Term Trajectory Prediction for Full-Immersive Multiuser Virtual Reality with Redirected Walking

Full-immersive multiuser Virtual Reality (VR) envisions supporting unconstrained mobility of the users in the virtual worlds, while at the same time constraining their physical movements inside VR setups through redirected walking. For enabling delivery of high data rate video content in real-time, the supporting wireless networks will leverage highly directional communication links that will "track" the users for maintaining the Line-of-Sight (LoS) connectivity. Recurrent Neural Networks (RNNs) and in particular Long Short-Term Memory (LSTM) networks have historically presented themselves as a suitable candidate for near-term movement trajectory prediction for natural human mobility, and have also recently been shown as applicable in predicting VR users' mobility under the constraints of redirected walking. In this work, we extend these initial findings by showing that Gated Recurrent Unit (GRU) networks, another candidate from the RNN family, generally outperform the traditionally utilized LSTMs. Second, we show that context from a virtual world can enhance the accuracy of the prediction if used as an additional input feature in comparison to the more traditional utilization of solely the historical physical movements of the VR users. Finally, we show that the prediction system trained on a static number of coexisting VR users be scaled to a multi-user system without significant accuracy degradation.

cs.NI↗

arXiv2021

Millimeter-Wave Beamforming with Continuous Coverage for Mobile Interactive Virtual Reality

Contemporary Virtual Reality (VR) setups commonly consist of a Head-Mounted Display (HMD) tethered to a content-generating server. "Cutting the wire" in such setups and going truly wireless will require a wireless network capable of delivering enormous amounts of video data at an extremely low latency. Higher frequencies, such as the millimeter-wave (mmWave) band, can support these requirements. Due to high attenuation and path loss in the mmWave frequencies, beamforming is essential. For VR setups, beamforming must adapt in real-time to the user's head rotations, but can rely on the HMD's built-in sensors providing accurate orientation estimates. In this work, we present coVRage, a beamforming solution tailored for VR HMDs. Based on past and current head orientations, the HMD predicts how the Angle of Arrival (AoA) from the access point will change in the near future, and covers this AoA trajectory with a dynamically shaped beam, synthesized using sub-arrays. We show that this solution can cover such trajectories with consistently high gain, unlike regular single-beam solutions.

cs.NI↗

arXiv2021

Towards Ultra-Low-Latency mmWave Wi-Fi for Multi-User Interactive Virtual Reality

The need for cables with high-fidelity Virtual Reality (VR) headsets remains a stumbling block on the path towards interactive multi-user VR. Due to strict latency constraints, designing fully wireless headsets is challenging, with the few commercially available solutions being expensive. These solutions use proprietary millimeter wave (mmWave) communications technologies, as extremely high frequencies are needed to meet the throughput and latency requirements of VR applications. In this work, we investigate whether such a system could be built using specification-compliant IEEE 802.11ad hardware, which would significantly reduce the cost of wireless mmWave VR solutions. We present a theoretical framework to calculate attainable live VR video bitrates for different IEEE 802.11ad channel access methods, using 1 or more head-mounted displays connected to a single Access Point (AP). Using the ns-3 simulator, we validate our theoretical framework, and demonstrate that a properly configured IEEE 802.11ad AP can support at least 8 headsets receiving a 4K video stream for each eye, with transmission latency under 1 millisecond.

cs.NI↗

arXiv2020

HTMRL: Biologically Plausible Reinforcement Learning with Hierarchical Temporal Memory

Building Reinforcement Learning (RL) algorithms which are able to adapt to continuously evolving tasks is an open research challenge. One technology that is known to inherently handle such non-stationary input patterns well is Hierarchical Temporal Memory (HTM), a general and biologically plausible computational model for the human neocortex. As the RL paradigm is inspired by human learning, HTM is a natural framework for an RL algorithm supporting non-stationary environments. In this paper, we present HTMRL, the first strictly HTM-based RL algorithm. We empirically and statistically show that HTMRL scales to many states and actions, and demonstrate that HTM's ability for adapting to changing patterns extends to RL. Specifically, HTMRL performs well on a 10-armed bandit after 750 steps, but only needs a third of that to adapt to the bandit suddenly shuffling its arms. HTMRL is the first iteration of a novel RL approach, with the potential of extending to a capable algorithm for Meta-RL.

cs.LG↗