Subjective Information Measure and Rate Fidelity Theory

Chenguang Lu

Home/Archive/arXiv

arXiv · 0705.3644

Subjective Information Measure and Rate Fidelity Theory

Chenguang Lu

View original source ↗Read source document ↗Export metadata ↓

Abstract

Using fish-covering model, this paper intuitively explains how to extend Hartley's information formula to the generalized information formula step by step for measuring subjective information: metrical information (such as conveyed by thermometers), sensory information (such as conveyed by color vision), and semantic information (such as conveyed by weather forecasts). The pivotal step is to differentiate condition probability and logical condition probability of a message. The paper illustrates the rationality of the formula, discusses the coherence of the generalized information formula and Popper's knowledge evolution theory. For optimizing data compression, the paper discusses rate-of-limiting-errors and its similarity to complexity-distortion based on Kolmogorov's complexity theory, and improves the rate-distortion theory into the rate-fidelity theory by replacing Shannon's distortion with subjective mutual information. It is proved that both the rate-distortion function and the rate-fidelity function are equivalent to a rate-of-limiting-errors function with a group of fuzzy sets as limiting condition, and can be expressed by a formula of generalized mutual information for lossy coding, or by a formula of generalized entropy for lossless coding. By analyzing the rate-fidelity function related to visual discrimination and digitized bits of pixels of images, the paper concludes that subjective information is less than or equal to objective (Shannon's) information; there is an optimal matching point at which two kinds of information are equal; the matching information increases with visual discrimination (defined by confusing probability) rising; for given visual discrimination, too high resolution of images or too much objective information is wasteful.

Explore related subjects

cs.IT cs.HC math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Chenguang Lu. 2007-05-24. Subjective Information Measure and Rate Fidelity Theory. https://arxiv.org/abs/0705.3644

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Direction-Shift Keying for High-Mobility mmWave Communications: Geometry-Based Resilience to Short Channel Coherence Times and Phase Noise

Communication reliability in high-mobility millimeter-wave (mmWave) systems is fundamentally challenged by rapid channel variations, manifested in short channel coherence times, and phase noise, which necessitate frequent channel estimation and phase tracking. To overcome these limitations, we introduce Direction-Shift Keying (DSK), a geometry-based modulation scheme that exploits the relative stability of propagation directions rather than the rapidly varying amplitude and phase of the wireless channel. DSK conveys information through distinguishable directions of arrival (DoAs), enabled by a distributed antenna system (DAS), thereby shifting the communication principle from channel-dependent to geometry-dependent signaling. We first derive the structure of the optimal DSK detector for a mobile device equipped with an arbitrary number of antennas. We then establish the fundamental law governing the temporal stability of propagation directions, termed Direction Coherence Time (DCT), and show that DCT scales as $d/v$, where $d$ is the transmitter--receiver distance and $v$ is the relative velocity. In contrast, conventional Channel Coherence Time (CCT) scales as $λ/v$, where $λ$ denotes the wavelength, revealing a coherence-time advantage on the order of $d/λ$, potentially exceeding four orders of magnitude in mmWave systems. Furthermore, we establish the inherent invariance of DSK to common phase rotations, providing resilience to common oscillator phase noise without explicit phase tracking. Numerical results corroborate the theoretical findings and demonstrate the potential of geometry-based signaling for reliable high-mobility mmWave communications.

cs.IT↗

arXiv2026

Conditioning Strengthens Classical Entropy and Matrix Inequalities: A Submodular Approach

Dembo, Cover, and Thomas (1991) developed an elegant information-theoretic framework for proving determinantal inequalities for positive definite matrices, which relies on the structural inequalities of differential entropy. Submodular functions, which subsume entropy, inherently satisfy these structural inequalities because they obey generalized forms of the fundamental properties of entropy -- a chain rule and the property that conditioning reduces the function's value (under an appropriate definition of conditioning). Applying subadditivity, Han's inequality (1978), and partition subadditivity (i.e., subadditivity over a partition) yields Hadamard's, Szász's, and Fischer's inequalities, respectively. Furthermore, this framework recovers Ky Fan's inequality (1955), a strengthening of Hadamard's inequality. We highlight a previously unnoticed information-theoretic insight behind this improvement: conditioning yields a tighter upper bound on the joint entropy than the unconditional one. In this paper, we establish conditional strengthenings of Han's inequality and partition subadditivity in the general setting of submodular functions. We derive equality conditions for these strengthened bounds and characterize when they strictly improve their unconditional counterparts. We specialize these results to differential entropy and apply them to establish strengthened versions of Szász's and Fischer's inequalities. The strengthening of Szász's inequality recovers Ky Fan's inequality as a special case, and is strictly stronger than the classical Szász's inequality for any non-diagonal positive definite matrix. We also derive an inequality concerning eigenvalues, which generalizes and strictly strengthens a corresponding eigenvalue inequality of Ky Fan. We provide numerical examples to explicitly illustrate the tightness of our proposed matrix determinantal bounds.

cs.IT↗

arXiv2026

On the Power Distance Representation of Bregman Divergences

We express Bregman divergences between parameters as power distances between corresponding weighted points, and investigate several applications of this representation. First, we show that the Bregman closest pair and diameter problems can be reformulated as bichromatic weighted point problems under the power distance. This reformulation enables Bregman nearest and farthest neighbor queries under power distances to be handled in practice through corresponding Euclidean neighbor queries in one additional dimension, after offsetting the weights. Next, we revisit the problem of computing minimum enclosing Bregman balls (Bregman MEBs) and establish their equivalence to double-sided weight power MEBs. We further show that Bregman MEBs can be approximated by relaxing the double-sided weight power MEB problem to its single-sided weight counterpart. We then present two approaches to solving general power MEB problems: An exact algorithm based on the LP-type framework and an arbitrarily accurate approximation scheme based on iterative Frank-Wolfe optimization. Finally, we show that the potential-function lifting transformations used to construct Bregman Voronoi diagrams can be reinterpreted as classical paraboloid lifting transformations applied to the corresponding weighted point sets. This establishes a direct connection between Bregman Voronoi diagrams and power diagrams, further illustrating how Bregman divergences can be represented as power distances.

cs.IT↗