arXiv subjects

Ruiqi Zhang

Publications and source records attributed to Ruiqi Zhang.

At least 73 records · Page 4Linked to original sources

arXiv2024

In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization

We study the \emph{in-context learning} (ICL) ability of a \emph{Linear Transformer Block} (LTB) that combines a linear attention component and a linear multi-layer perceptron (MLP) component. For ICL of linear regression with a Gaussian prior and a \emph{non-zero mean}, we show that LTB can achieve nearly Bayes optimal ICL risk. In contrast, using only linear attention must incur an irreducible additive approximation error. Furthermore, we establish a correspondence between LTB and one-step gradient descent estimators with learnable initialization ($\mathsf{GD}\text{-}\mathbfβ$), in the sense that every $\mathsf{GD}\text{-}\mathbfβ$ estimator can be implemented by an LTB estimator and every optimal LTB estimator that minimizes the in-class ICL risk is effectively a $\mathsf{GD}\text{-}\mathbfβ$ estimator. Finally, we show that $\mathsf{GD}\text{-}\mathbfβ$ estimators can be efficiently optimized with gradient flow, despite a non-convex training objective. Our results reveal that LTB achieves ICL by implementing $\mathsf{GD}\text{-}\mathbfβ$, and they highlight the role of MLP layers in reducing approximation error.

stat.ML↗

arXiv2024

AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition

Recent advancements in large language models (LLMs) have shown promise in multi-step reasoning tasks, yet their reliance on extensive manual labeling to provide procedural feedback remains a significant impediment. To address this challenge, in this paper, we propose a novel self-supervised framework AutoPRM that efficiently enhances the fine-tuning of LLMs for intricate reasoning challenges. Specifically, AutoPRM first decomposes complex problems into more manageable subquestions with a controllable granularity switch, then sequentially apply reinforcement learning to iteratively improve the subquestion solver. Additionally, we propose context-guided-decoding to avoid reward tampering and guide the subquestion solver towards the solution of the holistic problem. Extensive experiments show that AutoPRM significantly improves performance on mathematical and commonsense reasoning tasks over SOTA. More encouragingly, AutoPRM can be easily integrated with other orthogonal reasoning pipelines.

cs.CL↗

arXiv2023

Spreeze: High-Throughput Parallel Reinforcement Learning Framework

The promotion of large-scale applications of reinforcement learning (RL) requires efficient training computation. While existing parallel RL frameworks encompass a variety of RL algorithms and parallelization techniques, the excessively burdensome communication frameworks hinder the attainment of the hardware's limit for final throughput and training effects on a single desktop. In this paper, we propose Spreeze, a lightweight parallel framework for RL that efficiently utilizes a single desktop hardware resource to approach the throughput limit. We asynchronously parallelize the experience sampling, network update, performance evaluation, and visualization operations, and employ multiple efficient data transmission techniques to transfer various types of data between processes. The framework can automatically adjust the parallelization hyperparameters based on the computing ability of the hardware device in order to perform efficient large-batch updates. Based on the characteristics of the "Actor-Critic" RL algorithm, our framework uses dual GPUs to independently update the network of actors and critics in order to further improve throughput. Simulation results show that our framework can achieve up to 15,000Hz experience sampling and 370,000Hz network update frame rate using only a personal desktop computer, which is an order of magnitude higher than other mainstream parallel RL frameworks, resulting in a 73% reduction of training time. Our work on fully utilizing the hardware resources of a single desktop computer is fundamental to enabling efficient large-scale distributed RL training.

cs.LG↗

arXiv2023

Exposing nontrivial flat bands and superconducting pairing in infinite-layer nickelates

Flat bands coupled with magnetism and topological orders near or at the Fermi level are well known to drive exotic correlation physics and unconventional superconductivity. Here, based on first-principles modeling combined with an in-depth symmetry analysis, we reveal the presence of topological flat bands involving low-energy Ni-$3d_{z^2}$ states in the recently discovered superconductor LaNiO$_{2}$. Our analysis demonstrates that LaNiO$_2$ is an Axion insulator with $\mathbb{Z}_{4} = 2$ and that it supports topological crystalline insulating states protected by the glide mirror symmetries. The topological flat bands in LaNiO$_{2}$ are also shown to host odd-parity superconductivity. Our study indicates that the nickelates would provide an interesting materials platform for exploring the interplay of flat bands, topological states, and superconductivity.

cond-mat.supr-con↗

arXiv2023

Trained Transformers Learn Linear Models In-Context

Attention-based neural networks such as transformers have demonstrated a remarkable ability to exhibit in-context learning (ICL): Given a short prompt sequence of tokens from an unseen task, they can formulate relevant per-token and next-token predictions without any parameter updates. By embedding a sequence of labeled training data and unlabeled test data as a prompt, this allows for transformers to behave like supervised learning algorithms. Indeed, recent work has shown that when training transformer architectures over random instances of linear regression problems, these models' predictions mimic those of ordinary least squares. Towards understanding the mechanisms underlying this phenomenon, we investigate the dynamics of ICL in transformers with a single linear self-attention layer trained by gradient flow on linear regression tasks. We show that despite non-convexity, gradient flow with a suitable random initialization finds a global minimum of the objective function. At this global minimum, when given a test prompt of labeled examples from a new prediction task, the transformer achieves prediction error competitive with the best linear predictor over the test prompt distribution. We additionally characterize the robustness of the trained transformer to a variety of distribution shifts and show that although a number of shifts are tolerated, shifts in the covariate distribution of the prompts are not. Motivated by this, we consider a generalized ICL setting where the covariate distributions can vary across prompts. We show that although gradient flow succeeds at finding a global minimum in this setting, the trained transformer is still brittle under mild covariate shifts. We complement this finding with experiments on large, nonlinear transformer architectures which we show are more robust under covariate shifts.

stat.ML↗

arXiv2023

Assessing r2SCAN meta-GGA functional for structural parameters, cohesive energy, mechanical modulus and thermophysical properties of 3d, 4d and 5d transition metals

The recent development of the accurate and efficient semilocal density functionals on the third rung of Jacob's ladder of density functional theory such as the revised regularized strongly constrained and appropriately normed (r2SCAN) density functional could enable the rapid and highly reliable prediction of the elasticity and temperature dependence of thermophysical parameters of refractory elements and their intermetallic compounds using quasi-harmonic approximation (QHA). Here, we present a comparative evaluation of the equilibrium cell volumes, cohesive energy, mechanical moduli, and thermophysical properties (Debye temperature and thermal expansion coefficient) for 22 transition metals using semilocal density functionals, including local density approximation (LDA), the Perdew-Burke-Ernzerhof (PBE) and PBEsol generalized gradient approximations (GGA), and the r2SCAN meta-GGA. PBEsol and r2SCAN deliver the same level of accuracies for structural, mechanical and thermophysical properties. Otherwise, PBE and r2SCAN perform better than LDA and PBEsol for calculating cohesive energies of transition metals. Among the tested density functionals, r2SCAN provides an overall well-balanced performance for reliably computing the cell volumes, cohesive energies, mechanical properties, and thermophysical properties of various 3d, 4d, and 5d transition metals using QHA. Therefore, we recommend that r2SCAN could be employed as a workhorse method to evaluate the thermophysical properties of transition metal compounds and alloys in the high throughput workflows.

physics.comp-ph↗

arXiv2023

Explicifying Neural Implicit Fields for Efficient Dynamic Human Avatar Modeling via a Neural Explicit Surface

This paper proposes a technique for efficiently modeling dynamic humans by explicifying the implicit neural fields via a Neural Explicit Surface (NES). Implicit neural fields have advantages over traditional explicit representations in modeling dynamic 3D content from sparse observations and effectively representing complex geometries and appearances. Implicit neural fields defined in 3D space, however, are expensive to render due to the need for dense sampling during volumetric rendering. Moreover, their memory efficiency can be further optimized when modeling sparse 3D space. To overcome these issues, the paper proposes utilizing Neural Explicit Surface (NES) to explicitly represent implicit neural fields, facilitating memory and computational efficiency. To achieve this, the paper creates a fully differentiable conversion between the implicit neural fields and the explicit rendering interface of NES, leveraging the strengths of both implicit and explicit approaches. This conversion enables effective training of the hybrid representation using implicit methods and efficient rendering by integrating the explicit rendering interface with a newly proposed rasterization-based neural renderer that only incurs a texture color query once for the initial ray interaction with the explicit surface, resulting in improved inference efficiency. NES describes dynamic human geometries with pose-dependent neural implicit surface deformation fields and their dynamic neural textures both in 2D space, which is a more memory-efficient alternative to traditional 3D methods, reducing redundancy and computational load. The comprehensive experiments show that NES performs similarly to previous 3D approaches, with greatly improved rendering speed and reduced memory cost.

cs.CV↗

arXiv2023

Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data

In some applications of reinforcement learning, a dataset of pre-collected experience is already available but it is also possible to acquire some additional online data to help improve the quality of the policy. However, it may be preferable to gather additional data with a single, non-reactive exploration policy and avoid the engineering costs associated with switching policies. In this paper we propose an algorithm with provable guarantees that can leverage an offline dataset to design a single non-reactive policy for exploration. We theoretically analyze the algorithm and measure the quality of the final policy as a function of the local coverage of the original dataset and the amount of additional data collected.

cs.LG↗

arXiv2023

Peierls distortion driven multi-orbital origin of charge density waves in the undoped infinite-layer nickelate

Understanding similarities and differences between the cuprate and nickelate superconductors is drawing intense current interest. Competing charge orders have been observed recently in the $undoped$ infinite-layer nickelates in sharp contrast to the $undoped$ cuprates which exhibit robust antiferromagnetic insulating ground states. The microscopic mechanisms driving these differences remain unclear. Here, using in-depth first-principles and many-body theory based modeling, we show that the parent compound of the nickelate family, LaNiO$_2$, hosts a charge density wave (CDW) ground state with the predicted wavevectors in accord with the corresponding experimental findings. The CDW ground state is shown to be connected to a multi-orbital Peierls distortion. Our study points to the key role of electron-phonon coupling effects in the infinite-layer nickelates.

cond-mat.str-el↗

arXiv2023

Emergence of competing electronic states from non-integer nuclear charges

Understanding many-electron phenomena with competing near-degenerate electronic states is of fundamental importance to chemistry and condensed matter physics. One of the most significant challenges for exploring such many-electron phenomena is the necessity for large system sizes in order to realize competing states, far beyond those practical for first-principles methods. Here, we show how allowing non-integer nuclear charges expands the space of computationally tractable electron systems that host competing electronic states. The emergence of competing electronic states from non-integer nuclear charges is exemplified in the simple 2-electron H_{2} molecule and used to examine the microscopic structure of doped quasi-1D cuprate chains, showing how non-integer nuclear charges can open a window for first-principles calculations of difficult many-electron phenomena.

cond-mat.str-el↗

arXiv2022

High-throughput screening assisted discovery of a stable layered anti-ferromagnetic semiconductor: CdFeP2Se6

Recent advances in two-dimensional (2D) magnetism have heightened interest in layered magnetic materials due to their potential for spintronics. In particular, layered semiconducting antiferromagnets exhibit intriguing low-dimensional semiconducting behavior with both charge and spin as carrier controls. However, synthesis of these compounds is challenging and remains rare. Here, we conducted firstprinciples based high-throughput search to screen potentially stable mixed metal phosphorous trichalcogenides (MM'P2X6, where M and M' are transition metals and X is a chalcogenide) that have a wide range of tunable bandgaps and interesting magnetic properties. Among the potential candidates, we successfully synthesized a stable semiconducting layered magnetic material, CdFeP2Se6, that exhibits a short-range antiferromagnetic order at TN = 21 K with an indirect band gap of 2.23 eV. Our work suggests that highthroughput screening assisted synthesis be an effective method for layered magnetic materials discovery.

cond-mat.mtrl-sci↗

arXiv2022

Competing Incommensurate Spin Fluctuations and Magnetic Excitations in Infinite-Layer Nickelate Superconductors

The recently discovered infinite-layer nickelates show great promise in helping to disentangle the various cooperative mechanisms responsible for high-temperature superconductivity. However, lack of antiferromagnetic order in the pristine nickelates presents a challenge for connecting the physics of the cuprates and nickelates. Here, by using a quantum many-body Green's function-based approach to treat the electronic and magnetic structures, we unveil the presence of many two- and three-dimensional magnetic stripe instabilities that are shown to persist across the phase diagram of LaNiO$_2$. Our analysis indicates that the magnetic properties of the infinite-layer nickelates are closer to those of the doped cuprates which host inhomogeneous ground states rather than the undoped cuprates. The computed magnon spectrum in LaNiO$_2$ is found to contain an admixture of contributions from localized and itinerant carriers. The theoretically obtained magnon dispersion is in accord with the results of the corresponding RIXS experiments. Our study gives insight into the origin of inhomogeneity in the infinite-layer nickelates and their relationship with the cuprates.

cond-mat.str-el↗

arXiv2022

NDF: Neural Deformable Fields for Dynamic Human Modelling

We propose Neural Deformable Fields (NDF), a new representation for dynamic human digitization from a multi-view video. Recent works proposed to represent a dynamic human body with shared canonical neural radiance fields which links to the observation space with deformation fields estimations. However, the learned canonical representation is static and the current design of the deformation fields is not able to represent large movements or detailed geometry changes. In this paper, we propose to learn a neural deformable field wrapped around a fitted parametric body model to represent the dynamic human. The NDF is spatially aligned by the underlying reference surface. A neural network is then learned to map pose to the dynamics of NDF. The proposed NDF representation can synthesize the digitized performer with novel views and novel poses with a detailed and reasonable dynamic appearance. Experiments show that our method significantly outperforms recent human synthesis methods.

cs.CV↗

arXiv2022

Optimal Estimation of Off-Policy Policy Gradient via Double Fitted Iteration

Policy gradient (PG) estimation becomes a challenge when we are not allowed to sample with the target policy but only have access to a dataset generated by some unknown behavior policy. Conventional methods for off-policy PG estimation often suffer from either significant bias or exponentially large variance. In this paper, we propose the double Fitted PG estimation (FPG) algorithm. FPG can work with an arbitrary policy parameterization, assuming access to a Bellman-complete value function class. In the case of linear value function approximation, we provide a tight finite-sample upper bound on policy gradient estimation error, that is governed by the amount of distribution mismatch measured in feature space. We also establish the asymptotic normality of FPG estimation error with a precise covariance characterization, which is further shown to be statistically optimal with a matching Cramer-Rao lower bound. Empirically, we evaluate the performance of FPG on both policy gradient estimation and policy optimization, using either softmax tabular or ReLU policy networks. Under various metrics, our results show that FPG significantly outperforms existing off-policy PG estimation methods based on importance sampling and variance reduction techniques.

stat.ML↗

arXiv2022

Topological spiral magnetism in the Weyl semimetal SmAlSi

Weyl electrons are intensely studied due to novel charge transport phenomena such as chiral anomaly, Fermi arcs, and photogalvanic effect. Recent theoretical works suggest that Weyl electrons can also participate in magnetic interactions, and the Weyl-mediated indirect exchange coupling between local moments is proposed as a new mechanism of spiral magnetism that involves chiral electrons. Despite reports of incommensurate and non-collinear magnetic ordering in Weyl semimetals, an actual spiral order has remained hitherto undetected. Here, we present evidence of Weyl-mediated spiral magnetism in SmAlSi from neutron diffraction, transport, and thermodynamic data. We show that the spiral order in SmAlSi results from the nesting between topologically non-trivial Fermi pockets and weak magnetocrystalline anisotropy, unlike related materials (Ce,Pr,Nd)AlSi, where a strong anisotropy prevents the spins from freely rotating. We map the magnetic phase diagram of SmAlSi and reveal an A-phase where topological magnetic excitations may exist. This is corroborated by the observation of a topological Hall effect within the A-phase.

cond-mat.mtrl-sci↗

arXiv2022

Critical role of magnetic moments in heavy-fermion materials: revisiting mysteries of SmB$_{6}$

Heavy-fermion family exhibits fascinating and often puzzling properties due to the presence of open-shell $f$ ions and the complexity of the associated charge, orbital, and spin degrees of freedom. SmB$_6 $ is a prototypical heavy-fermion compound that is electrically insulating but yet it displays quantum oscillations, which are a telltale signature of the metallic state. Adding to the enigma is the possibility that SmB$_6$ is a topological Kondo insulator. Here, by treating the spin degree of freedom on an equal footing with other degrees of freedom using the parameter-free strongly-constrained and appropriately-normed (SCAN) density functional, we explore the ground-state electronic structure of SmB$_{6}$. A number of competing magnetic phases lying very closely in energy are found, indicating the key role of spin fluctuations in the material. The computed band structure, crystal-field splittings in the $f$-electron complex, the heavy effective electron mass at the Fermi energy, and the large specific heat are all in good agreement with the corresponding experimental results. In particular, our predicted FS explains the experimentally observed bulk quantum oscillations as well as the low electrical conductivity of SmB$_{6}$. The topological Kondo state of SmB$_6$ is shown to be robust regardless of its magnetic configuration. The excellent performance of SCAN in heavy-fermion systems is explained in terms of its ability to treat self-interaction errors and symmetry breaking within the framework of the density functional theory. Our study provides a new approach for modeling heavy-fermion materials.

cond-mat.str-el↗

arXiv2022

Off-Policy Fitted Q-Evaluation with Differentiable Function Approximators: Z-Estimation and Inference Theory

Off-Policy Evaluation (OPE) serves as one of the cornerstones in Reinforcement Learning (RL). Fitted Q Evaluation (FQE) with various function approximators, especially deep neural networks, has gained practical success. While statistical analysis has proved FQE to be minimax-optimal with tabular, linear and several nonparametric function families, its practical performance with more general function approximator is less theoretically understood. We focus on FQE with general differentiable function approximators, making our theory applicable to neural function approximations. We approach this problem using the Z-estimation theory and establish the following results: The FQE estimation error is asymptotically normal with explicit variance determined jointly by the tangent space of the function class at the ground truth, the reward structure, and the distribution shift due to off-policy learning; The finite-sample FQE error bound is dominated by the same variance term, and it can also be bounded by function class-dependent divergence, which measures how the off-policy distribution shift intertwines with the function approximator. In addition, we study bootstrapping FQE estimators for error distribution inference and estimating confidence intervals, accompanied by a Cramer-Rao lower bound that matches our upper bounds. The Z-estimation analysis provides a generalizable theoretical framework for studying off-policy estimation in RL and provides sharp statistical theory for FQE with differentiable function approximators.

stat.ML↗

arXiv2022

Sensitivity of the electronic and magnetic structures of cuprate superconductors to density functional approximations

We discuss the crystal, electronic, and magnetic structures of $\mathrm{La_{2-x}Sr_{x}CuO_{4}}$ (LSCO) for $x=0.0$ and $x=0.25$ employing 13 density functional approximations, representing the local, semi-local, and hybrid exchange-correlation approximations within the Perdew-Schmidt hierarchy. The meta-generalized gradient approximation (meta-GGA) class of functionals is found to perform well in capturing the key properties of LSCO, a prototypical high-temperature cuprate superconductor. In contrast, the local-spin-density approximation, GGA, and the hybrid density functional fail to capture the metal-insulator transition under doping.

cond-mat.supr-con↗