arXiv subjects

Pengxiang Xu

Publications and source records attributed to Pengxiang Xu.

16 recordsLinked to original sources

arXiv2026

Ascend to Science: Exploration of AI Chips for Scientific Computing

The rapid rise of AI-oriented accelerators has reshaped compute systems around low-precision tensor engines, raising a practical question for the HPC community: under what conditions can such hardware support scientific workloads that demand numerical robustness, irregular memory access, and scalability? Using the Ascend 910 NPU series as a representative tensor-centric platform, we characterize precision, execution, and memory-hierarchy bottlenecks that hinder the direct deployment of scientific codes. We then develop and evaluate workload-specific mappings across five application studies -- HPL-MxP, LRSVD, SGEMM-cube, PQSim, and SMC-X -- combining heterogeneous execution, mixed-precision numerical formulations, precision emulation, hierarchical memory orchestration, and communication--computation overlap. These studies show that AI-native NPUs can achieve numerical robustness, competitive performance, and satisfactory scalability when numerical formulation, execution placement, and data movement are addressed in a coordinated manner. Our results provide a state-of-the-practice case study of how scientific workloads can be adapted to tensor-centric architectures, while distinguishing transferable optimization principles from Ascend-specific implementation details.

cs.DC↗

arXiv2026

SMC-AI: Scaling Monte Carlo Simulation to Four Trillion Atoms with AI Accelerators

The rapid advancement of deep learning is reshaping the hardware design landscape toward AI tasks, posing fundamental challenges for HPC workloads such as atomistic simulation. Here we present SMC-AI, a general algorithmic framework that extends the SMC-X method for efficient canonical Monte Carlo simulation on AI accelerators, including GPUs and NPUs, while maintaining extreme scalability. The implementation of SMC-AI on an NPU cluster reaches unprecedented performance, achieving MC simulation of 4 trillion atoms on 4096 NPU dies. This represents the largest ML-accelerated atomistic simulation reported, delivering 32X system size and 1.3X throughput than previous records, with a relatively small computational budget. Excellent strong and weak scaling efficiency are reached for both the NPU and GPU implementation. By decoupling ML models from simulation, SMC-AI creates an abstraction that facilitates integration and porting of diverse ML models, laying a foundation for the future development of scalable scientific software.

physics.comp-ph↗

arXiv2026

Towards Computational Microscope of Chemical Order-Disorder via ML-Accelerated Monte Carlo Simulation

Tailoring the performance of next-generation high entropy materials requires a deep understanding of the competition between entropy-driven random solid solution and enthalpy-driven chemical ordering. Investigating such order and disorder complexity demands atomistic simulations that achieve high accuracy, efficiency, and generalizability across vast spatial, temporal, and especially chemical scales. While machine learning (ML) interatomic potentials have transformed molecular dynamics, they remain limited in capturing diffusion-driven chemical evolution over long timescales. The recently introduced SMC-X method brings exciting opportunities. Realizing its full potential requires a comprehensive study, which is the focus of this work. To assess model performance, we systematically benchmark invariant and equivariant architectures using a density functional theory dataset of more than 10,000 configurations spanning seven elements: Fe, Co, Ni, Al, Ti, Ta, and V. To understand the roles of pairwise and higher-order interactions, we decouple their contributions across chemical space using an explainable machine learning approach. We also examine the impact of lattice relaxation by comparing models trained on datasets with and without structural relaxation. Our results clarify how to choose ML surrogate models for Monte Carlo simulations, bridge the gap between theory and experiment, and lay a foundation for establishing ML-accelerated Monte Carlo as a computational microscope for chemical complexity.

cond-mat.mtrl-sci↗

arXiv2025

SMC-X: A Distributed Scalable Monte Carlo Simulation Method for Chemically Complex Alloys

To predict the complex chemical evolution in multicomponent alloys, it is highly desirable to have accurate atomistic simulation methods capable of reaching sufficiently large spatial and temporal scales. In this work, we advance the recently proposed SMC-X method through distributed computation on either GPUs or CPUs, pushing both spatial and temporal scales of atomistic simulation of chemically complex alloys to previously inaccessible scales. This includes a record-breaking 128-billion-atom HEA system extending to the micrometer regime in space, and a 1-billion-atom HEA evolved over more than three million Monte Carlo swap steps, approaching the minute regime in time. We show that such large-scale simulations are essential for bridging the gap between experimental observations and theoretical predictions of the nanoprecipitate sizes in HEAs, based on analysis using the Lifshitz-Slyozov-Wagner (LSW) theory for diffusion-controlled coarsening. This work demonstrates the great potential of SMC-X for simulation-driven exploration of the chemical complexity in high-entropy materials at large spatial and temporal scales.

cond-mat.mtrl-sci↗

arXiv2025

Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT

Parameter-Efficient Fine-Tuning (PEFT) methods are crucial for adapting large pre-trained models. Among these, LoRA is considered a foundational approach. Building on this, the influential DoRA method enhances performance by decomposing weight updates into magnitude and direction. However, its underlying mechanism remains unclear, and it introduces significant computational overhead. In this work, we first identify that DoRA's success stems from its capacity to increase the singular value entropy of the weight update matrix, which promotes a more uniform update distribution akin to full fine-tuning. We then reformulate DoRA into a mathematically equivalent and more efficient matrix form, revealing it as a learnable weight conditioning method. Based on this insight, we propose a unified framework for designing advanced PEFT methods by exploring two orthogonal dimensions: the architectural placement and the transformation type of the conditioning matrix. Within this framework, we introduce two novel methods: (1) \textbf{Pre-Diag}, which applies a diagonal conditioning matrix before the LoRA update to efficiently calibrate the pre-trained weights, thereby enhancing performance while reducing training time; and (2) \textbf{S}kewed \textbf{O}rthogonal \textbf{R}otation \textbf{A}daptation (\textbf{SORA}), which employs a parameter-efficient orthogonal rotation to perform a more powerful, norm-preserving transformation of the feature space. Extensive experiments on natural language understanding and generation tasks demonstrate that our proposed methods achieve superior performance and efficiency compared to both LoRA and DoRA. The code is available at https://github.com/MaeChd/SORA.

cs.LG↗

arXiv2025

Revealing Nanostructures in High-Entropy Alloys via Machine-Learning Accelerated Scalable Monte Carlo Simulation

The computational cost of traditional first-principles method quickly becomes prohibitively expensive as the number of atoms increases. This challenge is further amplified by the need to evaluate finite-temperature properties with Monte Carlo (MC) simulations, which is inherently challenging to parallelize due to sequential Markov chain updates. Here, we introduce Scalable Monte Carlo (SMC), an efficient MC simulation method that overcomes the parallelization bottlenecks in conventional MC simulation, reducing the computational complexity of a MC sweep from quadratic to linear. We present a GPU implementation of the SMC method, SMC-GPU, which simultaneously harnesses the thousands of processing cores on a GPU to accelerate the computation. By adopting a data-driven workflow that surrogates the computationally expensive density functional theory (DFT) with ML models, we demonstrate that SMC-GPU is capable of simulating systems of more than one-billion atoms, while maintaining the accuracy of first-principles methods. Using this unprecedented capability, we performed billion-atom MC simulations to investigate the nanostructure evolution of two important high-entropy alloys (HEAs), FeCoNiAlTi and MoNbTaW, in which the nanostructures are believed to be responsible for their superb mechanical properties. Our results reveal a rich diversity of nanostructures, including nanoparticles (NP), 3D-connected NP, and disorder protected nanophases. We quantitatively analyze the size, composition, and morphology of the nanostructures, as well as directly simulate the atom-probe-tomography (APT) needle. The results align well with available experimental observations. This work underscores the promising potential of leveraging large-scale MC simulation to explore the largely uncharted territory of nanostructure evolution in HEAs.

cond-mat.mtrl-sci↗

arXiv2025

SGEMM-cube: Precision-Recovery FP32 GEMM Approximation on Ascend NPUs with FP16 Matrix Engines

Modern AI accelerators provide high-throughput low-precision matrix engines, but often lack efficient support for FP32 GEMM. This paper presents SGEMM-cube, an FP32-accuracy GEMM approximation for Ascend NPUs built on FP16 Cube units. Following the fixed-length two-word splitting of Ootomo and Yokota, each FP32 operand is represented by an FP16 high component and a scaled FP16 residual. The product is reconstructed from three FP16 GEMMs while omitting the residual-residual term; the method therefore targets FP32-level accuracy rather than bit-exact emulation. Under an RN FP32-accumulation model, we provide a componentwise error analysis showing that the omitted term is no larger than the rounding error of a short FP32 inner product and that, for practically relevant inner-product lengths, the overall error is dominated by ordinary FP32 accumulation. We further analyze residual underflow and scaling under round-to-nearest conversion, compare two accumulation orders, and adapt L1-aware blocking and double buffering to Ascend's software-managed memory hierarchy. On Ascend 910A, SGEMM-cube is substantially more accurate than native FP16 GEMM, is comparable to the tested OpenBLAS FP32 SGEMM baseline for the evaluated input distributions and exponent range, and reaches 65.3\,TFLOP/s, or 77\% of the three-GEMM FP32-equivalent peak.

cs.DC↗

arXiv2024

Block Coordinate Descent Methods for Optimization under J-Orthogonality Constraints with Applications

The J-orthogonal matrix, also referred to as the hyperbolic orthogonal matrix, is a class of special orthogonal matrix in hyperbolic space, notable for its advantageous properties. These matrices are integral to optimization under J-orthogonal constraints, which have widespread applications in statistical learning and data science. However, addressing these problems is generally challenging due to their non-convex nature and the computational intensity of the constraints. Currently, algorithms for tackling these challenges are limited. This paper introduces JOBCD, a novel Block Coordinate Descent method designed to address optimizations with J-orthogonality constraints. We explore two specific variants of JOBCD: one based on a Gauss-Seidel strategy (GS-JOBCD), the other on a variance-reduced and Jacobi strategy (VR-J-JOBCD). Notably, leveraging the parallel framework of a Jacobi strategy, VR-J-JOBCD integrates variance reduction techniques to decrease oracle complexity in the minimization of finite-sum functions. For both GS-JOBCD and VR-J-JOBCD, we establish the oracle complexity under mild conditions and strong limit-point convergence results under the Kurdyka-Lojasiewicz inequality. To demonstrate the effectiveness of our method, we conduct experiments on hyperbolic eigenvalue problems, hyperbolic structural probe problems, and the ultrahyperbolic knowledge graph embedding problem. Extensive experiments using both real-world and synthetic data demonstrate that JOBCD consistently outperforms state-of-the-art solutions, by large margins.

cs.DS↗

arXiv2021

NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning

In this paper, a novel second-order method called NG+ is proposed. By following the rule ``the shape of the gradient equals the shape of the parameter", we define a generalized fisher information matrix (GFIM) using the products of gradients in the matrix form rather than the traditional vectorization. Then, our generalized natural gradient direction is simply the inverse of the GFIM multiplies the gradient in the matrix form. Moreover, the GFIM and its inverse keeps the same for multiple steps so that the computational cost can be controlled and is comparable with the first-order methods. A global convergence is established under some mild conditions and a regret bound is also given for the online learning setting. Numerical results on image classification with ResNet50, quantum chemistry modeling with Schnet, neural machine translation with Transformer and recommendation system with DLRM illustrate that GN+ is competitive with the state-of-the-art methods.

math.OC↗

arXiv2021

Simulating noisy variational quantum eigensolver with local noise models

Variational quantum eigensolver (VQE) is promising to show quantum advantage on near-term noisy-intermediate-scale quantum (NISQ) computers. One central problem of VQE is the effect of noise, especially the physical noise on realistic quantum computers. We study systematically the effect of noise for the VQE algorithm, by performing numerical simulations with various local noise models, including the amplitude damping, dephasing, and depolarizing noise. We show that the ground state energy will deviate from the exact value as the noise probability increase and normally noise will accumulate as the circuit depth increase. We build a noise model to capture the noise in a real quantum computer. Our numerical simulation is consistent with the quantum experiment results on IBM Quantum computers through Cloud. Our work sheds new light on the practical research of noisy VQE. The deep understanding of the noise effect of VQE may help to develop quantum error mitigation techniques on near team quantum computers.

quant-ph↗

arXiv2021

A variational quantum algorithm for Hamiltonian diagonalization

Hamiltonian diagonalization is at the heart of understanding physical properties and practical applications of quantum systems. It is highly desired to design quantum algorithms that can speedup Hamiltonian diagonalization, especially those can be implemented on near-term quantum devices. In this work, we propose a variational algorithm for Hamiltonians diagonalization (VQHD) of quantum systems, which explores the important physical properties, such as temperature, locality and correlation, of the system. The key idea is that the thermal states of the system encode the information of eigenvalues and eigenstates of the system Hamiltonian. To obtain the full spectrum of the Hamiltonian, we use a quantum imaginary time evolution algorithm with high temperature, which prepares a thermal state with a small correlation length. With Trotterization, this then allows us to implement each step of imaginary time evolution by a local unitary transformation on only a small number of sites. Diagonalizing these thermal states hence leads to a full knowledge of the Hamiltonian eigensystem. We apply our algorithm to diagonalize local Hamiltonians and return results with high precision. Our VQHD algorithm sheds new light on the applications of near-term quantum computers.

quant-ph↗

arXiv2021

Sketchy Empirical Natural Gradient Methods for Deep Learning

In this paper, we develop an efficient sketchy empirical natural gradient method (SENG) for large-scale deep learning problems. The empirical Fisher information matrix is usually low-rank since the sampling is only practical on a small amount of data at each iteration. Although the corresponding natural gradient direction lies in a small subspace, both the computational cost and memory requirement are still not tractable due to the high dimensionality. We design randomized techniques for different neural network structures to resolve these challenges. For layers with a reasonable dimension, sketching can be performed on a regularized least squares subproblem. Otherwise, since the gradient is a vectorization of the product between two matrices, we apply sketching on the low-rank approximations of these matrices to compute the most expensive parts. A distributed version of SENG is also developed for extremely large-scale applications. Global convergence to stationary points is established under some mild assumptions and a fast linear convergence is analyzed under the neural tangent kernel (NTK) case. Extensive experiments on convolutional neural networks show the competitiveness of SENG compared with the state-of-the-art methods. On the task ResNet50 with ImageNet-1k, SENG achieves 75.9\% Top-1 testing accuracy within 41 epochs. Experiments on the distributed large-batch training show that the scaling efficiency is quite reasonable.

math.OC↗

arXiv2021

DST: Data Selection and joint Training for Learning with Noisy Labels

Training a deep neural network heavily relies on a large amount of training data with accurate annotations. To alleviate this problem, various methods have been proposed to annotate the data automatically. However, automatically generating annotations will inevitably yields noisy labels. In this paper, we propose a Data Selection and joint Training (DST) method to automatically select training samples with accurate annotations. Specifically, DST fits a mixture model according to the original annotation as well as the predicted label for each training sample, and the mixture model is utilized to dynamically divide the training dataset into a correctly labeled dataset, a correctly predicted set and a wrong dataset. Then, DST is trained with these datasets in a supervised manner. Due to confirmation bias problem, we train the two networks alternately, and each network is tasked to establish the data division to teach another network. For each iteration, the correctly labeled and predicted labels are reweighted respectively by the probabilities from the mixture model, and a uniform distribution is used to generate the probabilities of the wrong samples. Experiments on CIFAR-10, CIFAR-100 and Clothing1M demonstrate that DST is the comparable or superior to the state-of-the-art methods.

cs.CV↗

arXiv2021

The impacts of optimization algorithm and basis size on the accuracy and efficiency of variational quantum eigensolver

Variational quantum eigensolver (VQE) is demonstrated to be the promising methodology for quantum chemistry based on near-term quantum devices. However, many problems are yet to be investigated for this methodology, such as the influences of optimization algorithm and basis size on the accuracy and efficiency for quantum computing. To address these issues, five molecules (H2, LiH, HF, N2 and F2) are studied in this work based on the VQE method using unitary coupled cluster (UCC) ansatz. The performance of the gradient optimization L-BFGS-B is compared with that of the direct search method COBYLA. The former converges more quickly, but the accuracy of energy surface is a little lower. The basis set shows a vital influence on the accuracy and efficiency. A large basis set generally provides an accurate energy surface, but induces a significant increase in computing time. The 631g basis is generally required from the energy surface of the simplest H2 molecule. For practical applications of VQE, complete active space (CAS) is suggested based on limited quantum resources. With the same number of qubits, more occupied orbitals included in CAS gives a better accuracy for the energy surface and a smaller evaluation number in the VQE optimization. Additionally, the electronic structure, such as filling fraction of orbitals, the bond strength of a molecule and the maximum nuclear charge also influences the performance of optimization, where half occupation of orbitals generally requires a large computation cost.

physics.chem-ph↗

arXiv2020

Simulating Noisy Quantum Circuits with Matrix Product Density Operators

Simulating quantum circuits with classical computers requires resources growing exponentially in terms of system size. Real quantum computer with noise, however, may be simulated polynomially with various methods considering different noise models. In this work, we simulate random quantum circuits in 1D with Matrix Product Density Operators (MPDO), for different noise models such as dephasing, depolarizing, and amplitude damping. We show that the method based on Matrix Product States (MPS) fails to approximate the noisy output quantum states for any of the noise models considered, while the MPDO method approximates them well. Compared with the method of Matrix Product Operators (MPO), the MPDO method reflects a clear physical picture of noise (with inner indices taking care of the noise simulation) and quantum entanglement (with bond indices taking care of two-qubit gate simulation). Consequently, in case of weak system noise, the resource cost of MPDO will be significantly less than that of the MPO due to a relatively small inner dimension needed for the simulation. In case of strong system noise, a relatively small bond dimension may be sufficient to simulate the noisy circuits, indicating a regime that the noise is large enough for an `easy' classical simulation. Moreover, we propose a more effective tensor updates scheme with optimal truncations for both the inner and the bond dimensions, performed after each layer of the circuit, which enjoys a canonical form of the MPDO for improving simulation accuracy. With truncated inner dimension to a maximum value $κ$ and bond dimension to a maximum value $χ$, the cost of our simulation scales as $\sim NDκ^3χ^3$, for an $N$-qubit circuit with depth $D$.

quant-ph↗

arXiv2019

Noncollinearity-modulated electronic properties of the monolayer CrI$_3$

Introducing noncollinear magnetization into a monolayer CrI$_3$ is proposed to be an effective approach to modulate the local electronic properties of the two-dimensional (2D) magnetic material. Using first-principles calculation, we illustrate that both the conduction and valence bands in the monolayer CrI$_3$ are lowered down by spin spiral states. The distinct electronic structure of the monolayer noncollinear CrI$_3$ can be applied in nanoscale functional devices. As a proof of concept, we show that a magnetic domain wall can form a one-dimensional conducting channel in the 2D semiconductor via proper gating. Other possible applications such as electron-hole separation and identical quantum dots are also discussed.

cond-mat.mtrl-sci↗