arXiv subjects

Aohua Cheng

Publications and source records attributed to Aohua Cheng.

3 recordsLinked to original sources

arXiv2024

Simplex path integral and simplex renormalization group for high-order interactions

Modern theories of phase transitions and scale-invariance are rooted in path integral formulation and renormalization group (RG). Despite the applicability of these approaches on simple systems with only pairwise interactions, they are less effective on complex systems with un-decomposable high-order interactions (i.e., interactions among arbitrary sets of units). To precisely characterize the universality of high-order interacting systems, we propose simplex path integral and simplex renormalization group (SRG) as the generalizations of classic approaches to arbitrary high-order and heterogeneous interactions. We first formalize the trajectories of units governed by high-order interactions to define path integrals on corresponding simplices based on a high-order propagator. Then we develop a method to integrate out short-range high-order interactions in the momentum space, accompanied by a coarse graining procedure functioning on the simplex structure generated by high-order interactions. The proposed SRG, equipped with a divide-and-conquer framework, can deal with the absence of ergodicity arised from the sparse distribution of high-order interactions and renormalize a system with intertwined high-order interactions on the $p$-order according to its properties on the $q$-order ($p\leq q$). The associated scaling relation and its corollaries support to differentiate among scale-invariant, weakly scale-invariant, and scale-dependent systems across different orders. We have validated our theory in multi-order scale-invariance verification, topological invariance discovery, organizational structure identification, and information bottleneck analysis. These experiments demonstrate the capacity of our theory for identifying intrinsic statistical and topological properties of high-order interacting systems during system reduction.

cond-mat.stat-mech↗

arXiv2023

Theoretical foundations of studying criticality in the brain

Criticality is hypothesized as a physical mechanism underlying efficient transitions between cortical states and remarkable information processing capacities in the brain. While considerable evidence generally supports this hypothesis, non-negligible controversies persist regarding the ubiquity of criticality in neural dynamics and its role in information processing. Validity issues frequently arise during identifying potential brain criticality from empirical data. Moreover, the functional benefits implied by brain criticality are frequently misconceived or unduly generalized. These problems stem from the non-triviality and immaturity of the physical theories that analytically derive brain criticality and the statistic techniques that estimate brain criticality from empirical data. To help solve these problems, we present a systematic review and reformulate the foundations of studying brain criticality, i.e., ordinary criticality (OC), quasi-criticality (qC), self-organized criticality (SOC), and self-organized quasi-criticality (SOqC), using the terminology of neuroscience. We offer accessible explanations of the physical theories and statistic techniques of brain criticality, providing step-by-step derivations to characterize neural dynamics as a physical system with avalanches. We summarize error-prone details and existing limitations in brain criticality analysis and suggest possible solutions. Moreover, we present a forward-looking perspective on how optimizing the foundations of studying brain criticality can deepen our understanding of various neuroscience questions.

q-bio.NC↗

arXiv2022

Statistical Physics of Deep Neural Networks: Initialization toward Optimal Channels

In deep learning, neural networks serve as noisy channels between input data and its representation. This perspective naturally relates deep learning with the pursuit of constructing channels with optimal performance in information transmission and representation. While considerable efforts are concentrated on realizing optimal channel properties during network optimization, we study a frequently overlooked possibility that neural networks can be initialized toward optimal channels. Our theory, consistent with experimental validation, identifies primary mechanics underlying this unknown possibility and suggests intrinsic connections between statistical physics and deep learning. Unlike the conventional theories that characterize neural networks applying the classic mean-filed approximation, we offer analytic proof that this extensively applied simplification scheme is not valid in studying neural networks as information channels. To fill this gap, we develop a corrected mean-field framework applicable for characterizing the limiting behaviors of information propagation in neural networks without strong assumptions on inputs. Based on it, we propose an analytic theory to prove that mutual information maximization is realized between inputs and propagated signals when neural networks are initialized at dynamic isometry, a case where information transmits via norm-preserving mappings. These theoretical predictions are validated by experiments on real neural networks, suggesting the robustness of our theory against finite-size effects. Finally, we analyze our findings with information bottleneck theory to confirm the precise relations among dynamic isometry, mutual information maximization, and optimal channel properties in deep learning.

cs.LG↗