arXiv subjects

Willy Wong

Publications and source records attributed to Willy Wong.

5 recordsLinked to original sources

arXiv2026

A Thermodynamic Structure of Asymptotic Inference

A thermodynamic framework for asymptotic inference is developed in which sample size and parameter variance define a state space. Within this description, Shannon information plays the role of entropy, and an integrating factor organizes its variation into a first-law-type balance equation. The framework supports a cyclic inequality analogous to a reversed second law, derived for the estimation of the mean. A non-trivial third-law-type result emerges as a lower bound on entropy set by representation noise. Optimal inference paths, global bounds on information gain, and a natural Carnot-like information efficiency follow from this structure, with efficiency fundamentally limited by a noise floor. Finally, de Bruijn's identity and the I-MMSE relation in the Gaussian-limit case appear as coordinate projections of the same underlying thermodynamic structure. This framework suggests that ensemble physics and inferential physics constitute shadow processes evolving in opposite directions within a unified thermodynamic description.

cs.IT↗

arXiv2025

A universal theorem of sensory information

A universal theorem of sensory information, analogous to the second law of thermodynamics, is derived. Beginning from a minimal description of a sensory neuron, a state-space representation of firing rate emerges naturally from Shannon's measure of information. A special case of this formulation predicts a previously unknown inequality governing sensory adaptation, which was confirmed across different modalities, species, and experimental conditions. Further analysis shows that the firing rate behaves like a state function in thermodynamics, leading to an entropy production equation from which a general law follows: any closed cycle of stimulation yields a non-negative net gain of sensory information.

q-bio.NC↗

arXiv2023

A Fundamental Inequality Governing the Rate Coding Response of Sensory Neurons

A fundamental inequality governing the spike activity of peripheral neurons is derived and tested against auditory data. This inequality states that the steady-state firing rate must lie between the arithmetic and geometric means of the spontaneous and peak activities during adaptation. Implications towards the development of auditory mechanistic models are explored.

q-bio.NC↗

arXiv2020

On the Rate Coding Response of Peripheral Sensory Neurons

The rate coding response of a single peripheral sensory neuron in the asymptotic, near-equilibrium limit can be derived using information theory, asymptotic Bayesian statistics and a theory of complex systems. Almost no biological knowledge is required. The theoretical expression shows good agreement with spike-frequency adaptation data across different sensory modalities and animal species. The approach permits the discovery of a new neurophysiological equation and shares similarities with statistical physics.

q-bio.NC↗

arXiv2019

Speech Separation Using Gain-Adapted Factorial Hidden Markov Models

We present a new probabilistic graphical model which generalizes factorial hidden Markov models (FHMM) for the problem of single-channel speech separation (SCSS) in which we wish to separate the two speech signals $X(t)$ and $V(t)$ from a single recording of their mixture $Y(t)=X(t)+V(t)$ using the trained models of the speakers' speech signals. Current techniques assume the data used in the training and test phases of the separation model have the same loudness. In this paper, we introduce GFHMM, gain adapted FHMM, to extend SCSS to the general case in which $Y(t)=g_xX(t)+g_vV(t)$, where $g_x$ and $g_v$ are unknown gain factors. GFHMM consists of two independent-state HMMs and a hidden node which model spectral patterns and gain difference, respectively. A novel inference method is presented using the Viterbi algorithm and quadratic optimization with minimal computational overhead. Experimental results, conducted on 180 mixtures with gain differences from 0 to 15~dB, show that the proposed technique significantly outperforms FHMM and its memoryless counterpart, i.e., vector quantization (VQ)-based SCSS.

cs.SD↗