arXiv subjects

Stephen D. Voran

Publications and source records attributed to Stephen D. Voran.

7 recordsLinked to original sources

arXiv2026

A Mixed-Behavior Vote Model for Multimedia Subjective Quality Votes, Means, and Variances

The relationship between subjective test vote variance and vote mean (or MOS) is well-studied, and the mathematically admissible vote variance region has been previously defined. We propose a reduced admissible variance region called the Unimodal Variance Region (UVR) that better describes real subjective rating behavior of multimedia. Further, subjective vote variance is often modeled as parabolic. We explain that, in practice, the parabolic model often violates the admissible region in the variance vs. MOS plane and we propose alternatives that respect the admissible region. We also present a parametrized random process to model votes that mixes voting processes and produces a realistic range of vote variances within the UVR at any desired MOS. This process was inspired by and comports with voting behavior that is observed in many subjective tests. By modeling vote variance from a subjective experiment, this vote model offers additional interpretable insights into voting behavior observed in a given experiment. We present example results from 16 datasets spanning speech, image, and video subjective quality experiments.

cs.MM↗

arXiv2026

Bounds on Agreement between Subjective and Objective Measurements

Objective estimators of multimedia quality are often judged by comparing estimates with subjective "truth data," most often via Pearson correlation coefficient (PCC) or mean-squared error (MSE). But subjective test results contain noise, so striving for a PCC of 1.0 or an MSE of 0.0 is neither realistic nor repeatable. Numerous efforts have been made to acknowledge and appropriately accommodate subjective test noise in objective-subjective comparisons, typically resulting in new analysis frameworks and figures-of-merit. We take a different approach. By making only basic assumptions, we derive bounds on PCC and MSE that can be expected for a subjective test. Consistent with intuition, these bounds are functions of subjective vote variance. When a subjective test includes vote variance information, the calculation of the bounds is easy, and in this case we say the resulting bounds are "fully data-driven." We provide two options for calculating bounds in cases where vote variance information is not available. One option is to use vote variance information from other subjective tests that do provide such information, and the second option is to use a model for subjective votes. Thus we introduce a binomial-based model for subjective votes (BinoVotes) that naturally leads to a mean opinion score (MOS) model, named BinoMOS, with multiple unique desirable properties. BinoMOS reproduces the discrete nature of MOS values and its dependence on the number of votes per file. This modeling provides vote variance information required by the PCC and MSE bounds and we compare this modeling with data from 18 subjective tests. The modeling yields PCC and MSE bounds that agree very well with those found from the data directly. These results allow one to set expectations for the PCC and MSE that might be achieved for any subjective test, even those where vote variance information is not available.

eess.AS↗

arXiv2026

Unseen but not Unknown: Using Dataset Concealment to Robustly Evaluate Speech Quality Estimation Models

We introduce Dataset Concealment (DSC), a rigorous new procedure for evaluating and interpreting objective speech quality estimation models. DSC quantifies and decomposes the performance gap between research results and real-world application requirements, while offering context and additional insights into model behavior and dataset characteristics. We also show the benefits of addressing the corpus effect by using the dataset Aligner from AlignNet when training models with multiple datasets. We demonstrate DSC and the improvements from the Aligner using nine training datasets and nine unseen datasets with three well-studied models: MOSNet, NISQA, and a Wav2Vec2.0-based model. DSC provides interpretable views of the generalization capabilities and limitations of models, while allowing all available data to be used at training. An additional result is that adding the 1000 parameter dataset Aligner to the 94 million parameter Wav2Vec model during training does significantly improve the resulting model's ability to estimate speech quality for unseen data.

eess.AS↗

arXiv2024

AlignNet: Learning dataset score alignment functions to enable better training of speech quality estimators

We develop two complementary advances for training no-reference (NR) speech quality estimators with independent datasets. Multi-dataset finetuning (MDF) pretrains an NR estimator on a single dataset and then finetunes it on multiple datasets at once, including the dataset used for pretraining. AlignNet uses an AudioNet to generate intermediate score estimates before using the Aligner to map intermediate estimates to the appropriate score range. AlignNet is agnostic to the choice of AudioNet so any successful NR speech quality estimator can benefit from its Aligner. The methods can be used in tandem, and we use two studies to show that they improve on current solutions: one study uses nine smaller datasets and the other uses four larger datasets. AlignNet with MDF improves on other solutions because it efficiently and effectively removes misalignments that impair the learning process, and thus enables successful training with larger amounts of more diverse data.

eess.AS↗

arXiv2024

Why some audio signal short-time Fourier transform coefficients have nonuniform phase distributions

The short-time Fourier transform (STFT) represents a window of audio samples as a set of complex coefficients. These are advantageously viewed as magnitudes and phases and the overall distribution of phases is very often assumed to be uniform. We show that when audio signal STFT phase distributions are analyzed per-frequency or per-magnitude range, they can be far from uniform. That is, the uniform phase distribution assumption obscures significant important details. We explain the significance of the nonuniform phase distributions and how they might be exploited, derive their source, and explain why the choice of the STFT window shape influences the nonuniformity of the resulting phase distributions.

eess.AS↗

arXiv2019

WEnets: A Convolutional Framework for Evaluating Audio Waveforms

We describe a new convolutional framework for waveform evaluation, WEnets, and build a Narrowband Audio Waveform Evaluation Network, or NAWEnet, using this framework. NAWEnet is single-ended (or no-reference) and was trained three separate times in order to emulate PESQ, POLQA, or STOI with testing correlations 0.95, 0.92, and 0.95, respectively when training on only 50% of available data and testing on 40%. Stacks of 1-D convolutional layers and non-linear downsampling learn which features are important for quality or intelligibility estimation. This straightforward architecture simplifies the interpretation of its inner workings and paves the way for future investigations into higher sample rates and accurate no-reference subjective speech quality predictions.

eess.AS↗

arXiv2019

A Multiple-Band Rydberg-Atom Based Receiver/Antenna: AM/FM Stereo Reception

With the re-definition of the International System of Units (SI) that occurred in October of 2018, there has recently been a great deal of attention on the development of atom-based sensors for metrology applications. In particular, great progress has been made in using Rydberg-atom based techniques for electric (E) field metrology. These Rydberg-atom based E-field sensors have made it possible to develop atom-based receivers and antennas, which potentially have many benefits over conventional technologies in detecting and receiving modulated signals. In this paper, we demonstrate the ``first'' multi-channel atom-based reception of both amplitude (AM) and frequency (FM) modulation signals. We demonstrate this by using two different atomic species in order to detect and receive AM and FM modulated signals in stereo. Also, in this paper we investigate the effect of Gaussian noise on the ability to receive AM/FM signals. These results illustrate the multi-band (or multi-channel) receiving capability of a atom-based receiver/antenna to produce high fidelity stereo reception from both AM and FM signals. This paper shows an interesting way of applying the relatively newer (and something esoteric) field of quantum-optics and atomic-physics to the century old topic of radio reception.

physics.atom-ph↗