arXiv subjects

Lee Friedman

Publications and source records attributed to Lee Friedman.

At least 19 recordsLinked to original sources

arXiv2025

Gaze Prediction as Time-Series Forecasting for Virtual Reality Applications: Quantifying Performance Variability and Extreme-Case Errors

Gaze prediction is essential for addressing motion-to-photon latency and ensuring seamless foveated rendering in Virtual Reality. The reliability of gaze forecasting is highly sensitive to individual differences and the eye movements being predicted. We evaluate recurrent, transformer-based, and classification-guided architectures to assess their generalization capabilities across oculomotor events. Using the GazeBase VR and Meta Quest Pro datasets, we analyzed the relationship between the median (P50) and high-percentile (P95) error profiles across subjects. The analysis reveals significant performance variability, showing that subjects with low P50 errors do not always exhibit the lowest extreme-case errors. Consequently, low median errors do not guarantee the robustness of the utilized solution. We discuss inference performance and address the class imbalance problem in short-term gaze prediction. These results identify a gap in standardized evaluation methods, necessitating a shift toward P95-focused, subject-specific metrics to develop reliable and perceptually stable gaze-contingent systems.

cs.HC↗

arXiv2024

Gaze Prediction as a Function of Eye Movement Type and Individual Differences

Eye movement prediction is a promising area of research with the potential to improve performance and the user experience of systems based on eye-tracking technology. In this study, we analyze individual differences in gaze prediction performance. We use three fundamentally different models within the analysis: the lightweight Long Short-Term Memory network (LSTM), the transformer-based network for multivariate time series representation learning (TST), and the Oculomotor Plant Mathematical Model wrapped in the Kalman Filter framework (OPKF). Each solution was assessed on different eye-movement types. We show important subject-to-subject variation for all models and eye-movement types. We found that fixation noise is associated with poorer gaze prediction in fixation. For saccades, higher velocities are associated with poorer gaze prediction performance. We think these individual differences are important and propose that future research should report statistics related to inter-subject variation. We also propose that future models should be designed to reduce subject-to-subject variation.

cs.HC↗

arXiv2024

Evidence for five types of fixation during a random saccade eye tracking task: Implications for the study of oculomotor fatigue

Our interest was to evaluate changes in fixation duration as a function of time-on-task (TOT) during a random saccade task. We employed a large, publicly available dataset. The frequency histogram of fixation durations was multimodal and modelled as a Gaussian mixture. We found five fixation types. The ``ideal'' response would be a single accurate saccade after each target movement, with a typical saccade latency of 200-250 msec, followed by a long fixation (> 800 msec) until the next target jump. We found fixations like this, but they comprised only 10% of all fixations and were the first fixation after target movement only 23.4% of the time. More frequently (57.4% of the time), the first fixation after target movement was short (117.7 msec mean) and was commonly followed by a corrective saccade. Across the entire 100 sec of the task, median total fixation duration decreased. This decrease was approximated with a power law fit with R^2=0.94. A detailed examination of the frequency of each of our five fixation types over time on task (TOT) revealed that the three shortest duration fixation types became more and more frequent with TOT whereas the two longest fixations became less and less frequent. In all cases, the changes over TOT followed power law relationships, with R^2 values between 0.73 and 0.93. We concluded that, over the 100 second duration of our task, long fixations are common in the first 15 to 22 seconds but become less common after that. Short fixations are relatively uncommon in the first 15 to 22 seconds but become more and more common as the task progressed. Apparently. the ability to produce an ideal response, although somewhat likely in the first 22 seconds, rapidly declines. This might be related to a noted decline in saccade accuracy over time.

q-bio.NC↗

arXiv2024

Evaluation of Eye Tracking Signal Quality for Virtual Reality Applications: A Case Study in the Meta Quest Pro

We present an extensive, in-depth analysis of the eye tracking capabilities of the Meta Quest Pro virtual reality headset using a dataset of eye movement recordings collected from 78 participants. In addition to presenting classical signal quality metrics--spatial accuracy, spatial precision and linearity--in ideal settings, we also study the impact of background luminance and headset slippage on device performance. We additionally present a user-centered analysis of eye tracking signal quality, where we highlight the potential differences in user experience as a function of device performance. This work contributes to a growing understanding of eye tracking signal quality in virtual reality headsets, where the performance of applications such as gaze-based interaction, foveated rendering, and social gaze are directly dependent on the quality of eye tracking signal.

cs.HC↗

arXiv2024

Temporal Persistence and Intercorrelation of Embeddings Learned by an End-to-End Deep Learning Eye Movement-driven Biometrics Pipeline

What qualities make a feature useful for biometric performance? In prior research, pre-dating the advent of deep learning (DL) approaches to biometric analysis, a strong relationship between temporal persistence, as indexed by the intraclass correlation coefficient (ICC), and biometric performance (Equal Error Rate, EER) was noted. More generally, the claim was made that good biometric performance resulted from a relatively large set of weakly intercorrelated features with high ICC. The present study aimed to determine whether the same relationships are found in a state-of-the-art DL-based eye movement biometric system (``Eye-Know-You-Too''), as applied to two publicly available eye movement datasets. To this end, we manipulate various aspects of eye-tracking signal quality, which produces variation in biometric performance, and relate that performance to the temporal persistence and intercorrelation of the resulting embeddings. Data quality indices were related to EER with either linear or logarithmic fits, and the resulting model R^2 was noted. As a general matter, we found that temporal persistence was an important predictor of DL-based biometric performance, and also that DL-learned embeddings were generally weakly intercorrelated.

cs.CV↗

arXiv2023

Signal vs Noise in Eye-tracking Data: Biometric Implications and Identity Information Across Frequencies

Prior research states that frequencies below 75 Hz in eye-tracking data represent the primary eye movement termed ``signal'' while those above 75 Hz are deemed ``noise''. This study examines the biometric significance of this signal-noise distinction and its privacy implications. There are important individual differences in a person's eye movement, which lead to reliable biometric performance in the ``signal'' part. Despite minimal eye-movement information in the ``noise'' recordings, there might be significant individual differences. Our results confirm the ``signal'' predominantly contains identity-specific information, yet the ``noise'' also possesses unexpected identity-specific data. This consistency holds for both short-(approx. 20 min) and long-term (approx. 1 year) biometric evaluations. Understanding the location of identity data within the eye movement spectrum is essential for privacy preservation.

cs.HC↗

arXiv2023

Filtering Eye-Tracking Data From an EyeLink 1000: Comparing Heuristic, Savitzky-Golay, IIR and FIR Digital Filters

In a previous report (Raju et al.,2023) we concluded that, if the goal was to preserve events such as saccades, microsaccades, and smooth pursuit in eye-tracking recordings, data with sine wave frequencies less than 100 Hz (-3db) were the signal and data above 100 Hz were noise. We compare 5 filters in their ability to preserve signal and remove noise. Specifically, we compared the proprietary STD and EXTRA heuristic filters provided by our EyeLink 1000 (SR-Research, Ottawa, Canada), a Savitzky-Golay (SG) filter, an infinite impulse response (IIR) filter (low-pass Butterworth), and a finite impulse filter (FIR). For each of the non-heuristic filters, we systematically searched for optimal parameters. Both the IIR and the FIR filters were zero-phase filters. Mean frequency response profiles and amplitude spectra for all 5 filters are provided. In addition, we examined the effect of our filters on a noisy recording. Our FIR filter had the sharpest roll-off of any filter. Therefore, it maintained the signal and removed noise more effectively than any other filter. On this basis, we recommend the use of our FIR filter. Several reports have shown that filtering increased the temporal autocorrelation of a signal. To address this, the present filters were also evaluated in terms of autocorrelation (specifically the first 3 lags). Of all our filters, the STD filter introduced the least amount of autocorrelation.

cs.HC↗

arXiv2023

Checking the Statistical Assumptions Underlying the Application of the Standard Deviation and RMS Error to Eye-Movement Time Series: A Comparison between Human and Artificial Eyes

Spatial precision is often measured using the standard deviation (SD) of the eye position signal or the RMS of the sample-to-sample differences (StoS) signal during fixation. As both measures emerge from statistical theory applied to time-series, there are certain statistical assumptions that accompany their use. It is intuitively obvious that the SD is most useful when applied to unimodal distributions. Both measures assume stationarity, which means that the statistical properties of the signals are stable over time. Both metrics assume the samples of the signals are independent. The presence of autocorrelation indicates that the samples in the time series are not independent. We tested these assumptions with multiple fixations from two studies, a publicly available dataset that included both human and artificial eyes ("HA Dataset", N=224 fixations), and data from our laboratory of 4 subjects ("TXstate", N=37 fixations). Many position signal distributions were multimodal (HA: median=32%, TXstate: median=100%). No fixation position signals were stationary. All position signals were statistically significantly autocorrelated (p < 0:01). Thus, the statistical assumptions of the SD were not met for any fixation. All StoS signals were unimodal. Some StoS signals were stationary (HA: 34%, TXstate: 24%). Almost all StoS signals were statistically significantly autocorrelated (p < 0:01). For TXstate, 3 of 37 fixations met all assumptions. Thus, the statistical assumptions of the RMS were generally not met. The general failure of these assumptions calls into question the appropriateness of the SD or the RMS-StoS as metrics of precision for eye-trackers.

q-bio.NC↗

arXiv2023

Determining Which Sine Wave Frequencies Correspond to Signal and Which Correspond to Noise in Eye-Tracking Time-Series

The Fourier theorem states that any time-series can be decomposed into a set of sinusoidal frequencies, each with its own phase and amplitude. The literature suggests that some frequencies are important to reproduce key qualities of eye-movements ("signal") and some of frequencies are not important ("noise"). To investigate what is signal and what is noise, we analyzed our dataset in three ways: (1) visual inspection of plots of saccade, microsaccade and smooth pursuit exemplars; (2) analysis of the percentage of variance accounted for (PVAF) in 1,033 unfiltered saccade trajectories by each frequency band; (3) analyzing the main sequence relationship between saccade peak velocity and amplitude, based on a power law fit. Visual inspection suggested that frequencies up to 75 Hz are required to represent microsaccades. Our PVAF analysis indicated that signals in the 0-25 Hz band account for nearly 100% of the variance in saccade trajectories. Power law coefficients (a, b) return to unfiltered levels for signals low-pass filtered at 75 Hz or higher. We conclude that to maintain eye movement signal and reduce noise, a cutoff frequency of 75 Hz is appropriate. We explain why, given this finding, a minimum sampling rate of 750 Hz is suggested.

cs.HC↗

arXiv2022

GazeBaseVR, a large-scale, longitudinal, binocular eye-tracking dataset collected in virtual reality

We present GazeBaseVR, a large-scale, longitudinal, binocular eye-tracking (ET) dataset collected at 250 Hz with an ET-enabled virtual-reality (VR) headset. GazeBaseVR comprises 5,020 binocular recordings from a diverse population of 407 college-aged participants. Participants were recorded up to six times each over a 26-month period, each time performing a series of five different ET tasks: (1) a vergence task, (2) a horizontal smooth pursuit task, (3) a video-viewing task, (4) a self-paced reading task, and (5) a random oblique saccade task. Many of these participants have also been recorded for two previously published datasets with different ET devices, and some participants were recorded before and after COVID-19 infection and recovery. GazeBaseVR is suitable for a wide range of research on ET data in VR devices, especially eye movement biometrics due to its large population and longitudinal nature. In addition to ET data, additional participant details are provided to enable further research on topics such as fairness.

cs.HC↗

arXiv2022

Analysis of Heuristic and Digital Filters as Applied to Video-oculography Signals

In 1993, Stampe [1993] suggested two "heurisitic" filters that were designed for video-oculography data. Several manufacturers (e.g., SR-Research, Tobii T60 XL and SMI) have employed these filters as an option for recording eye-movements. For the EyeLink family of eye-trackers, these two filters are referred to as standard (STD) or EXTRA. We have implemented these filters as software functions. For those who use their eye-trackers for data-collection only, this will allow users to collect unfiltered data and simultaneously have access to unfiltered, STD filtered and EXTRA filtered data for the exact same recording. Based on the literature, which has employed various eye-tracking technologies, and our analysis of our EyeLink-1000 data, we conclude that the highest signal frequency content needed for most eye-tracking studies (i.e., saccades, microsaccades and smooth pursuit) is around 100 Hz, excluding fixation microtremor. For those who collect their data at 1000 Hz or higher, we test two zero-phase low-pass digital filters, one with a cutoff of 50 Hz and one with a cutoff of 100 Hz. We perform a Fourier (FFT) analysis to examine the frequency content for unfiltered data, STD data, EXTRA filtered data, and data filtered by low-pass digital filters. We also examine the frequency response of these filters. The digital filter with the 100 Hz cutoff dramatically outperforms both heuristic filters because the heuristic filters leave noise above 100 Hz. In the paper we provide additional conclusions and suggest the use of digital filters in scenarios where offline data processing is an option.

cs.HC↗

arXiv2020

A Re-Examination of the Evidence used by Hooge et al (2018) "Is human classification by experienced untrained observers a gold standard in fixation detection?"

Hooge et al. asked the question: "Is human classification by experienced untrained observers a gold standard in fixation detection?" They conclude the answer is no. If they had entitled their paper: "Is human classification by experienced untrained observers a gold standard in fixation detection when data quality is very poor, data are error-filled, data presentation was not optimal, and the analysis was seriously flawed?", I would have no case to make. In the present report, I will present evidence to support my view that this latter title is justified. The low quality data assessment is based on using a relatively imprecise eye-tracker, the absence of head restraint for any subjects, and the use of infants as the majority of subjects (60 of 70 subjects). Allowing subjects with more than 50% missing data (as much as 95%) is also evidence of low quality data. The error-filled assessment is based on evidence that a number of the "fixations" classified by "experts" have obvious saccades within them, and that, apparently, a number of fixations were classified on the basis of no signal at all. The evidence for non-optimal data presentation stems from the fact that, in a number of cases, perfectly good data was not presented to the coders. The flaws in the analysis are evidenced by the fact that entire stretches of missing data were considered classified, and that the measurement of saccade amplitude was based on many cases in which there was no saccade at all. Without general evidence to the contrary, it is correct to assume that some human classifiers under some conditions may meet the criteria for a gold standard, and classifiers under other conditions may not. This conditionality is not recognized by Hooge et al. A fair assessment would conclude that whether or not humans can be considered a gold standard is still very much an open question.

q-bio.QM↗

arXiv2020

Why Temporal Persistence of Biometric Features is so Valuable for Classification Performance

It is generally accepted that relatively more permanent (i.e., more temporally persistent) traits are more valuable for biometric performance than less permanent traits. Although this finding is intuitive, there is no current work identifying exactly where in the biometric analysis temporal persistence makes a difference. In this paper, we answer this question. In a recent report, we introduced the intraclass correlation coefficient (ICC) as an index of temporal persistence for such features. In that report, we also showed that choosing only the most temporally persistent features yielded superior performance in 12 of 14 datasets. Motivated by those empirical results, we present a novel approach using synthetic features to study which aspects of a biometric identification study are influenced by the temporal persistence of features. What we show is that using more temporally persistent features produces effects on the similarity score distributions that explain why this quality is so key to biometric performance. The results identified with the synthetic data are largely reinforced by an analysis of two datasets, one based on eye-movements and one based on gait. There was one difference between the synthetic and real data: In real data, features are intercorrelated, with the level of intercorrelation increasing with increasing ICC. This increasedhttps://www.overleaf.com/project/5e2b14694c5dc600017292e6 intercorrelation in real data was associated with an increase in the spread of the impostor similarity score distributions. Removing these intercorrelations for real datasets with a decorrelation step produced results which were very similar to that obtained with synthetic features.

cs.CR↗

arXiv2020

Biometric Performance as a Function of Gallery Size

Many developers of biometric systems start with modest samples before general deployment. They are interested in how their systems will work with much larger samples. We evaluated the effect of gallery size on biometric performance. Identification rates describe the performance of biometric identification, whereas ROC-based measures describe the performance of biometric authentication (verification). Therefore, we examined how increases in gallery size affected identification rates (i.e., Rank-1 Identification Rate, or Rank-1 IR) and ROC-based measures such as equal error rate (EER). We studied these phenomena with synthetic data as well as real data from a face recognition study. It is well known that the Rank-1 IR declines with increasing gallery size. We have provided further insight into this decline. We have shown that this relationship is linear in log(Gallery Size). We have also shown that this decline can be counteracted with the inclusion of additional information (features) for larger gallery sizes. We have also described the curves which can be used to predict how much additional information is required to stabilize the Rank-1 IR as a function of gallery size. These equations are also linear in log(gallery size). We have also shown that the entire ROC curve is not systematically affected by gallery size, and so ROC-based scalar performance metrics such as EER are also stable across gallery size.

cs.CR↗

arXiv2019

Three errors and two problems in a recent paper: gazenet: End-to-end eye-movement event detection with deep neural networks (Zemblys, Niehorster, and Holmqvist, 2019)

Zemblys et al. \cite{gazeNet} reported on a method for the classification of eye-movements ("gazeNet"). I have found 3 errors and two problems with that paper that are explained herein. \underline{\textit{\textbf{Error 1}}} The gazeNet classification method was built assuming that a hand-scored dataset from Lund University was all collected at 500 Hz, but in fact, six of the 34 recording files were actually collected at 200Hz. Of the six datasets that were used as the training set for the gazeNet algorithm, 2 were actually collected at 200Hz. \underline{\textit{\textbf{Problem 1}}} has to do with the fact that even among the 500Hz data, the inter-timestamp intervals varied widely. \underline{\textit{\textbf{Problem 2}}} is that there are many unusual discontinuities in the saccade trajectories from the Lund University dataset that make it a very poor choice for the construction of an automatic classification method. \underline{\textit{\textbf{Error 2}}} The gazeNet algorithm was trained on the Lund dataset, and then compared to other methods, not trained on this dataset, in terms of performance on this dataset. This is an inherently unfair comparison, and yet no where in the gazeNet paper is this unfairness mentioned. \underline{\textit{\textbf{Error 3}}} arises out of the novel event-related agreement analysis employed by the gazeNet authors. Although the authors intended to classify unmatched events as either false positives or false negatives, many are actually being classified as true negatives. True negatives are not errors, and any unmatched event misclassified as a true negative is actually driving kappa higher, whereas unmatched events should be driving kappa lower.

q-bio.QM↗

arXiv2019

Evaluating the Data Quality of Eye Tracking Signals from a Virtual Reality System: Case Study using SMI's Eye-Tracking HTC Vive

We evaluated the data quality of SMI's tethered eye-tracking head-mounted display based on the HTC Vive (ET-HMD) during a random saccade task. We measured spatial accuracy, spatial precision, temporal precision, linearity, and crosstalk. We proposed the use of a non-parametric spatial precision measure based on the median absolute deviation (MAD). Our linearity analysis considered both the slope and adjusted R-squared of a best-fitting line. We were the first to test for a quadratic component to crosstalk. We prepended a calibration task to the random saccade task and evaluated 2 methods to employ this user-supplied calibration. For this, we used a unique binning approach to choose samples to be included in the recalibration analyses. We compared our quality measures between the ET-HMD and our EyeLink 1000 (SR-Research, Ottawa, Ontario, CA). We found that the ET-HMD had significantly better spatial accuracy and linearity fit than our EyeLink, but both devices had similar spatial precision and linearity slope. We also found that, while the EyeLink had no significant crosstalk, the ET-HMD generally exhibited quadratic crosstalk. Fourier analysis revealed that the binocular signal was a low-pass filtered version of the monocular signal. Such filtering resulted in the binocular signal being useless for the study of high-frequency components such as saccade dynamics.

cs.HC↗

arXiv2019

The Linear Relationship between Temporal Persistence, Number of Independent Features and Target EER

If you have a target level of biometric performance (e.g. EER = 5% or 0.1%), how many units of unique information (uncorrelated features) are needed to achieve that target? We show, for normally distributed features, that the answer to that question depends on the temporal persistence of the feature set. We address these questions with synthetic features introduced in a prior report. We measure temporal persistence with an intraclass correlation coefficient (ICC). For 5 separate EER targets (5.0%, 2.0%, 1.0%, 0.5% and 0.1%) we provide linear relationships between the temporal persistence of the feature set and the log10(number of features). These linear relationships will help those in the planning stage, prior to setting up a new biometric system, determine the required temporal persistence and number of independent features needed to achieve certain EER targets.

cs.CR↗

arXiv2019

Custom Video-Oculography Device and Its Application to Fourth Purkinje Image Detection during Saccades

We built a custom video-based eye-tracker that saves every video frame as a full resolution image (MJPEG). Images can be processed offline for the detection of ocular features, including the pupil and corneal reflection (First Purkinje Image, P1) position. A comparison of multiple algorithms for detection of pupil and corneal reflection can be performed. The system provides for highly flexible stimulus creation, with mixing of graphic, image, and video stimuli. We can change cameras and infrared illuminators depending on the image qualities and frame rate desired. Using this system, we have detected the position of the Fourth Purkinje image (P4) in the frames. We show that when we estimate gaze by calculating P1-P4, signal compares well with gaze estimated with a DPI eye-tracker, which natively detects and tracks the P1 and P4.

cs.CV↗