arXiv subjects

Josiah Couch

Publications and source records attributed to Josiah Couch.

11 recordsLinked to original sources

arXiv2025

Which Similarity-Sensitive Entropy (Sentropy)?

Shannon entropy is not the only entropy that is relevant to machine-learning datasets, nor possibly even the most important one. Traditional entropies such as Shannon entropy capture information represented by elements' frequencies but not the richer information encoded by their similarities and differences. Capturing the latter requires similarity-sensitive entropy (``sentropy''). Sentropy can be measured using either the recently developed Leinster-Cobbold-Reeve framework (LCR) or the newer Vendi score (VS). This raises the practical question of which one to use: LCR or VS. Here we address this question theoretically and numerically, using 53 large and well-known imaging and tabular datasets. We find that LCR and VS values can differ by orders of magnitude and are complementary, except in limiting cases. We show that both LCR and VS results depend on how similarities are scaled, and introduce the notion of ``half-distance'' to parameterize this dependence. We prove the VS provides an upper bound on LCR for all non-negative values of the R\'enyi-Hill order parameter, as well as for negative values in the special case that the similarity matrix is full rank. We conclude that VS is preferable only when a dataset's elements can be usefully interpreted as linear combinations of a more fundamental set of ``ur-elements'' or when the system that the dataset describes has a quantum-mechanical character. In the broader case where one simply wishes to capture the rich information encoded by elements' similarities and differences as well as their frequencies, we propose that LCR should be favored; nevertheless, for certain half-distances the two methods can complement each other.

cs.IT↗

arXiv2025

X-Factor: Quality Is a Dataset-Intrinsic Property

In the universal quest to optimize machine-learning classifiers, three factors -- model architecture, dataset size, and class balance -- have been shown to influence test-time performance but do not fully account for it. Previously, evidence was presented for an additional factor that can be referred to as dataset quality, but it was unclear whether this was actually a joint property of the dataset and the model architecture, or an intrinsic property of the dataset itself. If quality is truly dataset-intrinsic and independent of model architecture, dataset size, and class balance, then the same datasets should perform better (or worse) regardless of these other factors. To test this hypothesis, here we create thousands of datasets, each controlled for size and class balance, and use them to train classifiers with a wide range of architectures, from random forests and support-vector machines to deep networks. We find that classifier performance correlates strongly by subset across architectures ($R^2=0.79$), supporting quality as an intrinsic property of datasets independent of dataset size and class balance and of model architecture. Digging deeper, we find that dataset quality appears to be an emergent property of something more fundamental: the quality of datasets' constituent classes. Thus, quality joins size, class balance, and model architecture as an independent correlate of performance and a separate target for optimizing machine-learning-based classification.

cs.LG↗

arXiv2024

Beyond Size and Class Balance: Alpha as a New Dataset Quality Metric for Deep Learning

In deep learning, achieving high performance on image classification tasks requires diverse training sets. However, the current best practice$\unicode{x2013}$maximizing dataset size and class balance$\unicode{x2013}$does not guarantee dataset diversity. We hypothesized that, for a given model architecture, model performance can be improved by maximizing diversity more directly. To test this hypothesis, we introduce a comprehensive framework of diversity measures from ecology that generalizes familiar quantities like Shannon entropy by accounting for similarities among images. (Size and class balance emerge as special cases.) Analyzing thousands of subsets from seven medical datasets showed that the best correlates of performance were not size or class balance but $A$$\unicode{x2013}$"big alpha"$\unicode{x2013}$a set of generalized entropy measures interpreted as the effective number of image-class pairs in the dataset, after accounting for image similarities. One of these, $A_0$, explained 67% of the variance in balanced accuracy, vs. 54% for class balance and just 39% for size. The best pair of measures was size-plus-$A_1$ (79%), which outperformed size-plus-class-balance (74%). Subsets with the largest $A_0$ performed up to 16% better than those with the largest size (median improvement, 8%). We propose maximizing $A$ as a way to improve deep learning performance in medical imaging.

cs.CV↗

arXiv2024

One-half reflected entropy is not a lower bound for entanglement of purification

In recent work, Akers et al. proved that the entanglement of purification $E_p(A:B)$ is bounded below by half of the $q$-Rényi reflected entropy $S_R^{(q)}(A:B)$ for all $q\geq2$, showing that $E_p(A:B) = \frac{1}{2} S_R^{(q)}(A:B)$ for a class of random tensor network states. Naturally, the authors raise the question of whether a similar bound holds at $q = 1$. Our work answers that question in the negative by finding explicit counter-examples, which we arrive at through numerical optimization. Nevertheless, this result does not preclude the possibility that restricted sets of states, such as CFT states with semi-classical gravity duals, could obey the bound in question.

quant-ph↗

arXiv2023

Scaling Monte-Carlo-Based Inference on Antibody and TCR Repertoires

Previously, it has been shown that maximum-entropy models of immune-repertoire sequence can be used to determine a person's vaccination status. However, this approach has the drawback of requiring a computationally intensive method to compute each model's partition function ($Z$), the normalization constant required for calculating the probability that the model will generate a given sequence. Specifically, the method required generating approximately $10^{10}$ sequences via Monte-Carlo simulations for each model. This is impractical for large numbers of models. Here we propose an alternative method that requires estimating $Z$ this way for only a few models: it then uses these expensive estimates to estimate $Z$ more efficiently for the remaining models. We demonstrate that this new method enables the generation of accurate estimates for 27 models using only three expensive estimates, thereby reducing the computational cost by an order of magnitude. Importantly, this gain in efficiency is achieved with only minimal impact on classification accuracy. Thus, this new method enables larger-scale investigations in computational immunology and represents a useful contribution to energy-based modeling more generally.

q-bio.QM↗

arXiv2022

Circuit Complexity in Topological Quantum Field Theory

Quantum circuit complexity has played a central role in recent advances in holography and many-body physics. Within quantum field theory, it has typically been studied in a Lorentzian (real-time) framework. In a departure from standard treatments, we aim to quantify the complexity of the Euclidean path integral. In this setting, there is no clear separation between space and time, and the notion of unitary evolution on a fixed Hilbert space no longer applies. As a proof of concept, we argue that the pants decomposition provides a natural notion of circuit complexity within the category of 2-dimensional bordisms and use it to formulate the circuit complexity of states and operators in 2-dimensional topological quantum field theory. We comment on analogies between our formalism and others in quantum mechanics, such as tensor networks and second quantization.

hep-th↗

arXiv2020

The Speed of Quantum Information Spreading in Chaotic Systems

We present a general theory of quantum information propagation in chaotic quantum many-body systems. The generic expectation in such systems is that quantum information does not propagate in localized form; instead, it tends to spread out and scramble into a form that is inaccessible to local measurements. To characterize this spreading, we define an information speed via a quench-type experiment and derive a general formula for it as a function of the entanglement density of the initial state. As the entanglement density varies from zero to one, the information speed varies from the entanglement speed to the butterfly speed. We verify that the formula holds both for a quantum chaotic spin chain and in field theories with an AdS/CFT gravity dual. For the second case, we study in detail the dynamics of entanglement in two-sided Vaidya-AdS-Reissner-Nordstrom black branes. We also show that, with an appropriate decoding process, quantum information can be construed as moving at the information speed, and, in the case of AdS/CFT, we show that a locally detectable signal propagates at the information speed in a spatially local variant of the traversable wormhole setup.

cond-mat.stat-mech↗

arXiv2019

Holographic Purification Complexity

We study holographic subregion complexity, and its possible connection to purification complexity suggested recently by Agón et al. In particular, we study the conjecture that subregion complexity is the purification complexity by considering holographic purifications of a holographic mixed state. We argue that these include states with any amount of coarse-graining consistent with being a purification of the mixed state in question, corresponding holographically to different choices of the cutoff surface. We find that within the complexity = volume and complexity = spacetime volume conjectures, the subregion complexity is equal to the holographic purification complexity. For complexity = action, the subregion complexity seems to provide an upper bound on the holographic purification complexity, though we show cases where this bound is not saturated. One such example is provided by black holes with a large genus behind the horizon, which were studied by Fu et al. As such, one must conclude that these offending geometries are not holographic, that CA must be modified, or else that holographic subregion complexity in CA is not dual to the purification complexity of the corresponding reduced state.

hep-th↗

arXiv2018

Holographic Complexity and Volume

The previously proposed "Complexity=Volume" or CV-duality is probed and developed in several directions. We show that the apparent lack of universality for large and small black holes is removed if the volume is measured in units of the maximal time from the horizon to the "final slice" (times Planck area). This also works for spinning black holes. We make use of the conserved "volume current", associated with a foliation of spacetime by maximal volume surfaces, whose flux measures their volume. This flux picture suggests that there is a transfer of the complexity from the UV to the IR in holographic CFTs, which is reminiscent of thermalization behavior deduced using holography. It also naturally gives a second law for the complexity when applied at a black hole horizon. We further establish a result supporting the conjecture that a boundary foliation determines a bulk maximal foliation without gaps, establish a global inequality on maximal volumes that can be used to deduce the monotonicity of the complexification rate on a boost-invariant background, and probe CV duality in the settings of multiple quenches, spinning black holes, and Rindler-AdS.

hep-th↗

arXiv2018

Holographic complexity and non-commutative gauge theory

We study the holographic complexity of noncommutative field theories. The four-dimensional $\mathcal{N}=4$ noncommutative super Yang-Mills theory with Moyal algebra along two of the spatial directions has a well known holographic dual as a type IIB supergravity theory with a stack of D3 branes and non-trivial NS-NS B fields. We start from this example and find that the late time holographic complexity growth rate, based on the "complexity equals action" conjecture, experiences an enhancement when the non-commutativity is turned on. This enhancement saturates a new limit which is exactly 1/4 larger than the commutative value. We then attempt to give a quantum mechanics explanation of the enhancement. Finite time behavior of the complexity growth rate is also studied. Inspired by the non-trivial result, we move on to more general setup in string theory where we have a stack of D$p$ branes and also turn on the B field. Multiple noncommutative directions are considered in higher $p$ cases.

hep-th↗

arXiv2017

Noether charge, black hole volume, and complexity

In this paper, we study the physical significance of the thermodynamic volumes of AdS black holes using the Noether charge formalism of Iyer and Wald. After applying this formalism to study the extended thermodynamics of a few examples, we discuss how the extended thermodynamics interacts with the recent complexity = action proposal of Brown et al. (CA-duality). We, in particular, discover that their proposal for the late time rate of change of complexity has a nice decomposition in terms of thermodynamic quantities reminiscent of the Smarr relation. This decomposition strongly suggests a geometric, and via CA-duality holographic, interpretation for the thermodynamic volume of an AdS black hole. We go on to discuss the role of thermodynamics in complexity = action for a number of black hole solutions, and then point out the possibility of an alternate proposal, which we dub "complexity = volume 2.0". In this alternate proposal, the complexity would be thought of as the spacetime volume of the Wheeler-DeWitt patch. Finally, we provide evidence that, in certain cases, our proposal for complexity is consistent with the Lloyd bound whereas CA-duality is not.

hep-th↗