arXiv subjects

Mufan Li

Publications and source records attributed to Mufan Li.

5 recordsLinked to original sources

arXiv2026

Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product

We study the squared singular value spectrum of a non-square product of independent real Gaussian matrices, equivalently the feature covariance spectrum of a deep linear neural network at initialization. Starting from the fixed-$m$ covariance diffusion previously obtained in the proportional depth-width limit, we record an equivalent matrix realization, describe its affine invariance, and derive the interacting diffusion satisfied by its eigenvalues. We then take a second limit, sending $m\to\infty$ on the accelerated spectral clock $\tau=mt$, which corresponds in this sequential construction to the relation $dm/n\to\bar\tau$. We establish convergence of the empirical spectral measure path to a deterministic mean-field limit and derive a closed Burgers equation for its $T$-transform. Together with the proportional depth-width limit, these results give a rigorous sequential route from the deep non-square Gaussian product to the free log-normal limit of its feature covariance spectrum; for more general initial laws, the transform yields a free multiplicative convolution form. We further analyze the support of the free log-normal law, give a fixed point iteration for numerical evaluation and a formal Marchenko--Pastur approximation at small time, and use the limiting spectrum to predict the risk in a toy random feature model.

math.PR↗

arXiv2025

Tracking four-dimensional atomic evolutions of single nanocatalysts throughout the life cycles

Structural changes induced by chemical reactions critically determine the catalytic performance and mechanism. However, precise tracking of the three-dimensional (3D) atomic structural evolution of individual bimetallic nanocatalysts remains challenging. Here we develop four-dimensional electrocatalytic atomic-resolution electron tomography, a method for directly tracking 3D atomic rearrangements in identical nanoparticles by electrocatalytic reactions. Using Pd-Pt bimetallic nanoparticles as a model system, we capture the atomic evolution of single nanocatalysts throughout electrocatalytic cycles. We observe two stages of evolutions: surface reconstruction and atom leaching, which are corroborated with the voltage-dependent behaviors probed by in situ electrochemical transmission electron microscopy. We identify chemical short-range order at atomic level and further reveal anisotropic chemical redistributions across different crystallographic orientations. These findings highlight the necessity of incorporating 3D spatiotemporal and chemical evolutions into the rational design of functional nanocatalysts in the future.

cond-mat.mtrl-sci↗

arXiv2025

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer

The transformer architecture is central to the success of modern Large Language Models (LLMs), in part due to its surprising ability to perform a wide range of tasks - including mathematical reasoning, memorization, and retrieval - using only gradient-based learning on next-token prediction. While the core component of a transformer is the self-attention mechanism, we question how much, and which aspects, of the performance gains can be attributed to it. To this end, we compare standard transformers to variants in which either the MLP layers or the attention weights are frozen at initialization. Surprisingly, we find that attention with frozen key and query weights is not only able to form induction heads, but can also perform competitively on language modeling. We formalize this by proving a new expressivity result for transformer models with frozen key and query weights. To further isolate the contribution of attention, we design MixiT, an architecture with entirely random attention scores, with provably stable signal propagation that overcomes prior depth-wise scaling challenges in random transformers. We use the successes and failures of MixiT to understand the role each transformer component plays, such as attention being largely responsible for in-context reasoning, and MLPs being responsible for, but collaborates with attention, on knowledge storage. Our results suggest that the transformer architecture has a built-in inductive bias towards forming specialized circuits, as it does even without learnable attention weights.

cs.LG↗

arXiv2025

Don't be lazy: CompleteP enables compute-efficient deep transformers

We study compute efficiency of LLM training when using different parameterizations, i.e., rules for adjusting model and optimizer hyperparameters (HPs) as model size changes. Some parameterizations fail to transfer optimal base HPs (such as learning rate) across changes in model depth, requiring practitioners to either re-tune these HPs as they scale up (expensive), or accept sub-optimal training when re-tuning is prohibitive. Even when they achieve HP transfer, we develop theory to show parameterizations may still exist in the lazy learning regime where layers learn only features close to their linearization, preventing effective use of depth and nonlinearity. Finally, we identify and adopt the parameterization we call CompleteP that achieves both depth-wise HP transfer and non-lazy learning in all layers. CompleteP enables a wider range of model width/depth ratios to remain compute-efficient, unlocking shapes better suited for different hardware settings and operational contexts. Moreover, CompleteP enables 12-34% compute efficiency improvements over the prior state-of-the-art. All experiments were run on Cerebras CS-3 systems. A minimal implementation is available at https://github.com/EleutherAI/nanoGPT-mup/tree/completep.

cs.LG↗

arXiv2025

Boron-assisted synthesis of compositionally complex amorphous oxides via short-range-order-constrained generative design

Engineering short-range atomic order in amorphous materials offers a promising yet still underexplored route to high-performance solids. Here, we establish a boron-assisted amorphization strategy through ApolloX, a theory-guided, short-range-order-constrained generative framework for identifying low-energy amorphous configurations in compositionally complex multimetal BOx systems. Using FeCoNiMoBOx as a representative model platform, ApolloX predicts an ensemble of candidate amorphous configurations across systematically varied boron contents. Ab initio molecular dynamics simulations based on these configurations show that increasing boron content suppresses atomic diffusion and disfavors crystallization, with the stabilization of BO3-centered local motifs emerging as a key structural feature associated with enhanced amorphization propensity. Guided by these predictions, we synthesize three representative FeCoNiMoBOx compositions with distinct boron contents and use synchrotron-based scattering and electron microscopy to verify compositional fidelity, structural homogeneity, and the targeted amorphous features, thereby experimentally validating the boron-regulated structural evolution predicted by theory. Beyond this representative system, the same strategy is extended to a broader library of multimetal BOx amorphous compositions spanning diverse metal combinations and boron loadings, demonstrating that the boron-assisted route is not limited to a single FeCoNiMoBOx family but is broadly transferable across compositionally complex amorphous oxides. Overall, our results establish boron incorporation as a practical design variable for tuning short-range order and amorphization in multicomponent oxides, and provide a general framework for the theory-guided discovery of compositionally complex amorphous materials with tunable properties.

cond-mat.mtrl-sci↗