arXiv subjects

Isaac Chuang

Publications and source records attributed to Isaac Chuang.

At least 19 recordsLinked to original sources

arXiv2026

Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly. Meanwhile, training losses instead follow smooth power laws. Variants of both behaviors occur in architectures with very different microscopic structures, which is the signature of a few relevant collective variables. We show that a symmetry fixes what those variables are: a network layer is a sum over interchangeable units, so relabeling the units leaves it unchanged; given smoothness and the condition that a unit's gradient vanish at the origin, symmetry then enforces a universal leading form for the expansion about the near-zero weights present at the start of training, the quadratic $\Tr[WW^{\top}A(x)]$, in which every architectural detail is confined to a single ``structure matrix" $A(x)$ that we compute for each architecture. Perceptrons, attention layers, mixtures of experts, and convolutions become one model at different $A$. Its training dynamics then close on the ``order parameter" $M=WW^{\top}$ and, whenever the data matrices share an eigenbasis, reduce to a Lotka--Volterra equation whose modes switch on one after another. The smaller the initial weights, the further apart the switch-on times, and the plateaus appear as a singular limit of a smooth flow; when many modes are unresolved the same events merge into a power law in training time whose exponent the theory predicts. We confirm both numerically across training methods and architectures.

cs.LG↗

arXiv2026

Deep Reinforcement Learning for Individual Atomic Control and Cooling

Real-time feedback control of quantum systems is often limited by partial observations, nonlinear dynamics and measurement noise, which make accurate model-based controllers difficult to design. Here we show that deep reinforcement learning can cool the motion of a single neutral atom coupled to a high-finesse optical cavity using only the continuously monitored cavity transmission. We first train the controller in simulation and then transfer it to the experiment, where online fine-tuning adapts it to unmodeled experimental dynamics. The learned policy damps the atom's motion in real time and achieves a cooling time constant of 388 +/- 14 microseconds, corresponding to only two motional periods in the trap. It also outperforms a standard linear differentiator controller in cooling speed while maintaining comparable atom retention over a broad range of operating conditions. These results establish reinforcement learning as a practical strategy for feedback control in quantum-limited experiments where compact analytical models are incomplete.

quant-ph↗

arXiv2026

Generalized Bicycle Codes as Cyclic Submodules and their Automorphism Structure

Automorphisms of quantum codes, when they exist, offer a pathway toward fault-tolerant gate implementation via qubit relabeling. Although useful, the conditions under which automorphisms appear in a given code remain poorly understood. In this paper, we develop an algebraic framework for systematically analyzing and engineering automorphisms in Generalized Bicycle (GB) codes. Central to our approach is the derivation of a three-space dependency between the polynomial ring space, the parity check matrix space, and the $\mathbb{F}_2^{2\ell}$ qubit space, similar to the structure found in the study of classical cyclic codes. By expressing GB codes as a pair of cyclic submodules of $R_\ell^2$, where $R_\ell \cong \mathbb{F}_2[x]/\langle x^\ell-1\rangle$, we reduce the search for code automorphisms to a deterministic algebraic problem, deriving necessary and sufficient conditions for the existence of block-separable automorphisms built from cyclic shifts, ring automorphisms and block-swaps. We connect these conditions to the fold-transversal gate framework, providing explicit criteria for the existence of $H$-, $S$-, and $CX$-type fold-transversal gates. We further discuss structured bases for logical operators in order to determine the logical action of a given automorphism. Finally, we introduce the Maximal Cube Root (MCR) code family, a family of GB codes constructed around the principle of maximizing automorphism flexibility and fold-CX gates. We demonstrate a collection of $k=2$ MCR codes up to $d=13$ generating the 2-qubit Clifford group via automorphism and fold-transversal gates, with stabilizer weight ranging from 8 to 16, and $k>2$ MCR codes with a minimum of 20 distinct logical gates achievable from automorphisms. This serves as a first demonstration of inverse design: using these methods to build codes around a rich automorphism structure from the ground up.

quant-ph↗

arXiv2026

Thermodynamic Irreversibility of Training Algorithms

The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towards understanding the learning dynamics of modern AI systems. In this work, we establish a general framework for defining and analyzing the irreversibility of training algorithms. We show that four different ways to characterize the irreversibility of dynamical processes are equivalent to leading order in the step size $\eta$: numerical backward error $\phi_{\rm DE}$, time-renormalized correction $\phi_{\rm TR}$, microscopic time reversal asymmetry $\phi_{\rm TA}$, and the (regularized) stochastic-thermodynamic entropy production $\phi_{\rm ST}$. The irreversibility gives rise to a time-reversal-symmetry-breaking emergent force that generically breaks non-isometric continuous reparametrization symmetries, preserves orthogonal symmetries, and leads to a universal preference for those learning trajectories that minimize the entropy production rate.

cond-mat.stat-mech↗

arXiv2026

An Exponential Sample-Complexity Advantage for Coherent Quantum Inference

Standard quantum inference converts quantum data into classical outputs. We study an alternative inference setting in which the desired output is quantum, preserving coherence. Such settings include quantum purity amplification (QPA), mixed-state approximate purification or cloning, and density matrix exponentiation. We show that such protocols can achieve exponentially lower sample complexity than incoherent, measurement-mediated protocols. For QPA with principal eigenstate targets and $d$-dimensional inputs, coherent processing achieves error $\varepsilon$ using $O(1/\varepsilon)$ copies, versus the $\Omega(d/\varepsilon)$ copies required by any incoherent protocol. Together, these sharp coherent-incoherent separations seed a theory of coherent quantum inference, with an entanglement-breaking limit identifying the optimal incoherent counterpart of each coherent protocol.

quant-ph↗

arXiv2026

Quantum Purity Amplification for Arbitrary Eigenstates and Multiple Outputs

Quantum purity amplification (QPA) is the task of coherently transforming $n$ copies of a mixed state into high-fidelity copies of a chosen eigenstate. We solve QPA in the general setting of $n$ input copies, $m$ output copies, arbitrary target eigenstates, arbitrary local dimension $d$, and generic input spectra. We characterize the optimal channel and derive its all-site and one-site performance laws across output regimes. For the asymptotic analysis, we use a path-graph parametrization to show that, when the target eigenvalue has a constant spectral gap $D_{k,\mathrm{min}}$, achieving all-site error $\varepsilon$ requires a number of input copies independent of $d$ and scaling as $O(m/(\varepsilon D_{k,\mathrm{min}}^2))$. When $m/n$ approaches a constant, the performance exhibits phase-like regimes, which we characterize explicitly. For the nonasymptotic analysis, we develop a theory of generalized Young diagrams that yields tight sample complexity bounds and provides the first dimension-uniform guarantee for optimal QPA. We also provide asymptotically efficient implementations of the optimal protocol. Together, these results establish QPA as a rigorous example of coherent quantum information processing with dimension-uniform sample complexity, supplying the technical foundation for the coherent-incoherent separation developed in the companion work.

quant-ph↗

arXiv2026

Forced Gap Post-Selection for Quantum LDPC Codes and their Operations

We develop a simple and general post-selection strategy for high-rate quantum codes that is transferrable across decoders. After an initial baseline run, the decoder is re-run once per logical observable, and forced in these latter runs to provide a solution where the given observable has the complementary outcome. Shots are rejected that find logically complementary solutions with similar likelihoods compared to the baseline. Using the Relay-BP decoder, we benchmark the strategy on the $72$-qubit and $144$-qubit bivariate bicycle codes, as well as surgery gadgets for the latter. In comparison to previous post-selection strategies, our results offer an improved logical error rate by over a factor of $4$ on the same circuit and physical error rate, and at the same rate of post-selection. Our strategies are also lightweight, relying only on FPGA-friendly belief propagation, whereas the previous best used repeated rounds of a high-latency BP-OSD decoder.

quant-ph↗

arXiv2026

Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model

A large body of theory and empirical work hypothesizes a connection between the flatness of a neural network's loss landscape during training and its performance. However, there have been conceptually opposite pieces of evidence regarding when SGD prefers flatter or sharper solutions during training. In this work, we partially but causally clarify the flatness-seeking behavior of SGD by identifying and exactly solving an analytically solvable model that exhibits both flattening and sharpening behavior during training. In this model, the SGD training has no \textit{a priori} preference for flatness, but only a preference for minimal gradient fluctuations. This leads to the insight that, at least within this model, it is data distribution that uniquely determines the sharpness at convergence, and that a flat minimum is preferred if and only if the noise in the labels is isotropic across all output dimensions. When the noise in the labels is anisotropic, the model instead prefers sharpness and can converge to an arbitrarily sharp solution, depending on the imbalance in the noise in the labels spectrum. We reproduce this key insight in controlled settings with different model architectures such as MLP, RNN, and transformers.

cs.LG↗

arXiv2025

An efficient probabilistic hardware architecture for diffusion-like models

The proliferation of probabilistic AI has prompted proposals for specialized stochastic computers. Despite promising efficiency gains, these proposals have failed to gain traction because they rely on fundamentally limited modeling techniques and exotic, unscalable hardware. In this work, we address these shortcomings by proposing an all-transistor probabilistic computer that implements powerful denoising models at the hardware level. A system-level analysis indicates that devices based on our architecture could achieve performance parity with GPUs on a simple image benchmark using approximately 10,000 times less energy.

cs.LG↗

arXiv2025

Topological Invariance and Breakdown in Learning

We prove that for a broad class of permutation-equivariant learning rules (including SGD, Adam, and others), the training process induces a bi-Lipschitz mapping between neurons and strongly constrains the topology of the neuron distribution during training. This result reveals a qualitative difference between small and large learning rates $\eta$. With a learning rate below a topological critical point $\eta^*$, the training is constrained to preserve all topological structure of the neurons. In contrast, above $\eta^*$, the learning process allows for topological simplification, making the neuron manifold progressively coarser and thereby reducing the model's expressivity. Viewed in combination with the recent discovery of the edge of stability phenomenon, the learning dynamics of neuron networks under gradient descent can be divided into two phases: first they undergo smooth optimization under topological constraints, and then enter a second phase where they learn through drastic topological simplifications. A key feature of our theory is that it is independent of specific architectures or loss functions, enabling the universal application of topological methods to the study of deep learning.

cs.LG↗

arXiv2025

Proof of a perfect platonic representation hypothesis

In this note, we elaborate on and explain in detail the proof given by Ziyin et al. (2025) of the ``perfect" Platonic Representation Hypothesis (PRH) for the embedded deep linear network model (EDLN). We show that if trained with the stochastic gradient descent (SGD), two EDLNs with different widths and depths and trained on different data will become Perfectly Platonic, meaning that every possible pair of layers will learn the same representation up to a rotation. Because most of the global minima of the loss function are not Platonic, that SGD only finds the perfectly Platonic solution is rather extraordinary. The proof also suggests at least six ways the PRH can be broken. We also show that in the EDLN model, the emergence of the Platonic representations is due to the same reason as the emergence of progressive sharpening. This implies that these two seemingly unrelated phenomena in deep learning can, surprisingly, have a common cause. Overall, the theory and proof highlight the importance of understanding emergent "entropic forces" due to the irreversibility of SGD training and their role in representation learning. The goal of this note is to be instructive while avoiding jargon and lengthy technical details.

cs.LG↗

arXiv2025

Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning

With the rapid discovery of emergent phenomena in deep learning and large language models, understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD) and its variants. Building on the theory of parameter symmetries and an entropic loss landscape, we show that representation learning is crucially governed by emergent entropic forces arising from stochasticity and discrete-time updates. These forces systematically break continuous parameter symmetries and preserve discrete ones, leading to a series of gradient balance phenomena that resemble the equipartition property of thermal systems. These phenomena, in turn, (a) explain the universal alignment of neural representations between AI models and lead to a proof of the Platonic Representation Hypothesis, and (b) reconcile the seemingly contradictory observations of sharpness- and flatness-seeking behavior of deep learning optimization. Our theory and experiments demonstrate that a combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning.

cs.LG↗

arXiv2025

Heterosynaptic Circuits Are Universal Gradient Machines

We propose a design principle for the learning circuits of the biological brain. The principle states that almost any dendritic weights updated via heterosynaptic plasticity can implement a generalized and efficient class of gradient-based meta-learning. The theory suggests that a broad class of biologically plausible learning algorithms, together with the standard machine learning optimizers, can be grounded in heterosynaptic circuit motifs. This principle suggests that the phenomenology of (anti-) Hebbian (HBP) and heterosynaptic plasticity (HSP) may emerge from the same underlying dynamics, thus providing a unifying explanation. It also suggests an alternative perspective of neuroplasticity, where HSP is promoted to the primary learning and memory mechanism, and HBP is an emergent byproduct. We present simulations that show that (a) HSP can explain the metaplasticity of neurons, (b) HSP can explain the flexibility of the biology circuits, and (c) gradient learning can arise quickly from simple evolutionary dynamics that do not compute any explicit gradient. While our primary focus is on biology, the principle also implies a new approach to designing AI training algorithms and physically learnable AI hardware. Conceptually, our result demonstrates that contrary to the common belief, gradient computation may be extremely easy and common in nature.

q-bio.NC↗

arXiv2025

Parameter Symmetry Potentially Unifies Deep Learning Theory

The dynamics of learning in modern large AI systems is hierarchical, often characterized by abrupt, qualitative shifts akin to phase transitions observed in physical systems. While these phenomena hold promise for uncovering the mechanisms behind neural networks and language models, existing theories remain fragmented, addressing specific cases. In this position paper, we advocate for the crucial role of the research direction of parameter symmetries in unifying these fragmented theories. This position is founded on a centralizing hypothesis for this direction: parameter symmetry breaking and restoration are the unifying mechanisms underlying the hierarchical learning behavior of AI models. We synthesize prior observations and theories to argue that this direction of research could lead to a unified understanding of three distinct hierarchies in neural networks: learning dynamics, model complexity, and representation formation. By connecting these hierarchies, our position paper elevates symmetry -- a cornerstone of theoretical physics -- to become a potential fundamental principle in modern AI.

cs.LG↗

arXiv2024

Formation of Representations in Neural Networks

Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothesis (CRH), which posits a set of six alignment relations to universally govern the formation of representations in most hidden layers of a neural network. Under the CRH, the latent representations (R), weights (W), and neuron gradients (G) become mutually aligned during training. This alignment implies that neural networks naturally learn compact representations, where neurons and weights are invariant to task-irrelevant transformations. We then show that the breaking of CRH leads to the emergence of reciprocal power-law relations between R, W, and G, which we refer to as the Polynomial Alignment Hypothesis (PAH). We present a minimal-assumption theory proving that the balance between gradient noise and regularization is crucial for the emergence of the canonical representation. The CRH and PAH lead to an exciting possibility of unifying major key deep learning phenomena, including neural collapse and the neural feature ansatz, in a single framework.

cs.LG↗

arXiv2024

Optimal Quantum Purity Amplification

Quantum purity amplification (QPA) provides a novel approach to counteracting the pervasive noise that degrades quantum states. We present the optimal QPA protocol for general quantum systems and global noise, resolving a two-decade open problem. Under strong depolarization, our protocol achieves an exponential reduction in sample complexity over the best-known methods. We provide an efficient implementation of the protocol based on generalized quantum phase estimation. Additionally, we introduce SWAPNET, a sparse and shallow circuit that enables QPA for near-term experiments. Simulations in both digital and analog quantum settings, along with experiments on superconducting quantum processors, confirm the protocol's robustness and practical utility. Our findings suggest that QPA could improve the performance of quantum information processing tasks, particularly in the context of Noisy Intermediate-Scale Quantum (NISQ) devices, where reducing the effect of noise with limited resources is critical.

quant-ph↗

arXiv2024

Remove Symmetries to Control Model Expressivity and Improve Optimization

When symmetry is present in the loss function, the model is likely to be trapped in a low-capacity state that is sometimes known as a "collapse". Being trapped in these low-capacity states can be a major obstacle to training across many scenarios where deep learning technology is applied. We first prove two concrete mechanisms through which symmetries lead to reduced capacities and ignored features during training and inference. We then propose a simple and theoretically justified algorithm, syre, to remove almost all symmetry-induced low-capacity states in neural networks. When this type of entrapment is especially a concern, removing symmetries with the proposed method is shown to correlate well with improved optimization or performance. A remarkable merit of the proposed method is that it is model-agnostic and does not require any knowledge of the symmetry.

cs.LG↗

arXiv2022

Architectures for Multinode Superconducting Quantum Computers

Many proposals to scale quantum technology rely on modular or distributed designs where individual quantum processors, called nodes, are linked together to form one large multinode quantum computer (MNQC). One scalable method to construct an MNQC is using superconducting quantum systems with optical interconnects. However, a limiting factor of these machines will be internode gates, which may be two to three orders of magnitude noisier and slower than local operations. Surmounting the limitations of internode gates will require a range of techniques, including improvements in entanglement generation, the use of entanglement distillation, and optimized software and compilers, and it remains unclear how improvements to these components interact to affect overall system performance, what performance from each is required, or even how to quantify the performance of each. In this paper, we employ a `co-design' inspired approach to quantify overall MNQC performance in terms of hardware models of internode links, entanglement distillation, and local architecture. In the case of superconducting MNQCs with microwave-to-optical links, we uncover a tradeoff between entanglement generation and distillation that threatens to degrade performance. We show how to navigate this tradeoff, lay out how compilers should optimize between local and internode gates, and discuss when noisy quantum links have an advantage over purely classical links. Using these results, we introduce a roadmap for the realization of early MNQCs which illustrates potential improvements to the hardware and software of MNQCs and outlines criteria for evaluating the landscape, from progress in entanglement generation and quantum memory to dedicated algorithms such as distributed quantum phase estimation. While we focus on superconducting devices with optical interconnects, our approach is general across MNQC implementations.

quant-ph↗