arXiv subjects

Fuming Yang

Publications and source records attributed to Fuming Yang.

5 recordsLinked to original sources

arXiv2026

Feature Lottery? A Bifurcation Theory of Concept Emergence

Neural networks acquire structured representations at specific moments during training, yet identifying these transitions typically relies on retrospective, label-dependent metrics. We introduce a bifurcation theory of representation dynamics to detect these moments in real time. Analyzing a passive GMM probe attached to the evolving encoder, we show the onset of structure corresponds to a supercritical pitchfork bifurcation driven by the loss Hessian. The system exhibits a theoretically predictable zero-crossing ($\beta_c$) that, compared to the network's current state ($\beta$), yields a dynamic ratio $\beta(t)/\beta_c(t)$: a universal, label-free phase coordinate for representation dynamics, computable entirely from hidden states. We empirically validate four distinct transition regimes predicted by this coordinate across diverse settings: SAEs on language models (Pythia), SSL (CIFAR), and grokking (modular arithmetic). Crucially, under finite dissipation, macroscopic symmetry-breaking can lag the initial zero-crossing by orders of magnitude, which providing a rigorous dynamical account of the delayed escape observed in grokking. Microscopically, the bifurcation creates a shared unstable subspace, forcing collective symmetry breaking. We term this the "feature lottery" in SAE training: a feature's terminal interpretability becomes predictable remarkably early. By only 5% of training, early atom purity robustly predicts final convergence purity, with top-decile early atoms achieving over 12x the baseline purity at convergence. Beyond explaining concept emergence, $\beta/\beta_c$ provides a practical early-warning indicator for training health, detecting the onset of usable structure, the crystallization of feature identity, and representational collapse epochs before downstream metrics react.

cs.LG↗

arXiv2025

Detecting Conflicts in Evidence Synthesis Models Using Score Discrepancies

Evidence synthesis models combine multiple data sources to estimate latent quantities of interest, enabling reliable inference on parameters that are difficult to measure directly. However, shared parameters across data sources can induce conflicts both among the data and with the assumed model structure. Detecting and quantifying such conflicts remains a challenge in model criticism. Here we propose a general framework for conflict detection in evidence synthesis models based on score discrepancies, extending prior-data conflict diagnostics to more general conflict checks in the latent space of hierarchical models. Simulation studies in an exchangeable model demonstrate that the proposed approach effectively detects between-data inconsistencies. Application to an influenza severity model illustrates its use, complementary to traditional deviance-based diagnostics, in complex real-world hierarchical settings. The proposed framework thus provides a flexible and broadly applicable tool for consistency assessment in Bayesian evidence synthesis.

stat.ME↗

arXiv2025

Towards 1000-fold Electron Microscopy Image Compression for Connectomics via VQ-VAE with Transformer Prior

Petascale electron microscopy (EM) datasets push storage, transfer, and downstream analysis toward their current limits. We present a vector-quantized variational autoencoder-based (VQ-VAE) compression framework for EM that spans 16x to 1024x and enables pay-as-you-decode usage: top-only decoding for extreme compression, with an optional Transformer prior that predicts bottom tokens (without changing the compression ratio) to restore texture via feature-wise linear modulation (FiLM) and concatenation; we further introduce an ROI-driven workflow that performs selective high-resolution reconstruction from 1024x-compressed latents only where needed.

cs.CV↗

arXiv2025

Sparse Graph Reconstruction and Seriation for Large-Scale Image Stacks

We study recovering a 1D order from a noisy, locally sampled pairwise comparison matrix under a tight query budget. We recast the task as reconstructing a sparse, noisy line graph and present, to our knowledge, the first method that provably builds a sparse graph containing all edges needed for exact seriation using only O(N(log N + K)) oracle queries, which is near-linear in N for fixed window K. The approach is parallelizable and supports both binary and bounded-noise distance oracles. Our five-stage pipeline consists of: (i) a random-hook Boruvka step to connect components via short-range edges in O(N log N) queries; (ii) iterative condensation to bound graph diameter; (iii) a double-sweep BFS to obtain a provisional global order; (iv) fixed-window densification around that order; and (v) a greedy SuperChain that assembles the final permutation. Under a simple top-1 margin and bounded relative noise we prove exact recovery; empirically, SuperChain still succeeds when only about 2N/3 of true adjacencies are present. On wafer-scale serial-section EM, our method outperforms spectral, MST, and TSP baselines with far fewer comparisons, and is applicable to other locally structured sequencing tasks such as temporal snapshot ordering, archaeological seriation, and playlist/tour construction.

cs.DS↗

arXiv2025

Budgeted Broadcast: An Activity-Dependent Pruning Rule for Neural Network Efficiency

Most pruning methods remove parameters ranked by impact on loss (e.g., magnitude or gradient). We propose Budgeted Broadcast (BB), which gives each unit a local traffic budget (the product of its long-term on-rate $a_i$ and fan-out $k_i$). A constrained-entropy analysis shows that maximizing coding entropy under a global traffic budget yields a selectivity-audience balance, $\log\frac{1-a_i}{a_i}=\beta k_i$. BB enforces this balance with simple local actuators that prune either fan-in (to lower activity) or fan-out (to reduce broadcast). In practice, BB increases coding entropy and decorrelation and improves accuracy at matched sparsity across Transformers for ASR, ResNets for face identification, and 3D U-Nets for synapse prediction, sometimes exceeding dense baselines. On electron microscopy images, it attains state-of-the-art F1 and PR-AUC under our evaluation protocol. BB is easy to integrate and suggests a path toward learning more diverse and efficient representations.

cs.LG↗