arXiv subjects

Huiyuan Li

Publications and source records attributed to Huiyuan Li.

At least 19 recordsLinked to original sources

arXiv2026

Shifting the Sweet Spot: High-Performance Matrix-Free Method for High-Order Elasticity

MFEM is a widely used finite-element library, but its native linear-elasticity Partial Assembly (PA) path still applies an $O((p+1)^6)$ contraction in the element operator, leaving the CPU operator-throughput sweet spot near $p\approx 2$ in our baseline measurements. This work closes this implementation gap for MFEM linear elasticity on affine tensor-product hexahedral meshes by integrating four well-established tensor-product PA optimizations (sum factorization, Voigt notation, macro-kernel fusion, and slice-wise loop reorganization) into MFEM's native linear-elasticity PA path. The resulting operator is evaluated in high-order GMG-PCG solves using MFEM's geometric multigrid (GMG) components. On AMD EPYC 7713, the optimized operator achieves $7\text{--}83\times$ kernel speedup and $3.6\text{--}16.8\times$ end-to-end speedup across $p\in\{1,2,4,8\}$. At fixed problem size, the kernel-time operator throughput peaks around $p=6$ and remains high at $p=8$, shifting the operator-throughput sweet spot to $p\ge 6$. The same trend is reproduced on Huawei~Kunpeng~920 (ARMv8.2). These results are accompanied by per-stage ablation and hardware-counter characterization; the implementation will be released on GitHub.

cs.DC↗

arXiv2026

Bridging the Gap between Sparse Matrix Reordering and Factorization: A Deep Learning Framework for Fill-in Reduction

Sparse matrix reordering can significantly reduce the fill-in during matrix factorization, thereby decreasing the computational and storage requirements in sparse matrix computations. Finding a minimal fill-in ordering is known to be an NP-hard problem. Moreover, there is a paradox: matrix reordering is applied before matrix factorization, but fill-ins that matrix reordering methods aim at are generated from matrix factorization. To bridge the gap between reordering and factorization, we propose a deep learning framework to minimize a fill-in surrogate function based on spectral embedding. First, we employ a multi-grid-like GNN architecture to learn to approximate the smallest eigenvectors of its graph Laplacian matrix, i.e. spectral embedding, and capture the global structural information of the matrix. Then, another multi-grid-like GNN architecture is used to minimize the potential space where fill-in can occur based on the rank distribution. Experimental results indicate that our approach achieves competitive performance compared with traditional graph-theoretic algorithms and deep learning methods.

cs.LG↗

arXiv2026

Learning Fill-in Reduction Ordering via Graph Policy Optimization for Sparse Matrices

Matrix reordering in large sparse solvers seeks a permutation that minimizes factorization fill-in to reduce memory and computation. Because the minimum fill-in ordering problem is NP-complete and fill-in is implicit in the sparsity pattern, graph-theoretic heuristics are used. Existing reinforcement learning methods either ignore sparsity patterns--missing the global fill-in--or lack local exact fill-in feedback. We propose a graph policy optimization method, modeling fill-ins from global and local views: both the policy and value networks use a multi-hop graph neural backbone to embed global fill-in; the policy further interacts with symbolic factorization over graphs to extract local, step-level fill-ins, and the resulting feedback is aligned with the value network via an adaptive saturation function to improve convergence. On the SuiteSparse Matrix Collection, our method achieves mean reductions of 29.3 in fill-ins and 31.3 in peak memory usage over state-of-the-art baselines.

cs.LG↗

arXiv2026

Self-Supervised Learning for Sparse Matrix Reordering

Rearranging the rows or columns of a sparse matrix using an appropriate ordering can significantly reduce fill-ins, i.e., new nonzeros introduced during matrix factorization, decreasing memory usage and runtime. However, finding an ordering that minimizes fill-ins is NP-complete. Existing approaches, including graph-theoretic and deep learning methods, rely on surrogate objectives without theoretical guarantees. The Fill-Path Theorem reveals a direct and intrinsic relationship between fill-in generation and the sparse structure of the matrix as path triplet inequalities. Here we first employ a multigrid graph network to capture structural information for each vertex. We then derive a triplet sampling strategy based on inequalities. Finally, we introduce an end-max chain loss function to reduce the number of triplets whose predicted scores satisfy these inequalities. Experimental evaluations on the publicly available SuiteSparse matrix collection demonstrate the superiority of the proposed method in terms of both fill-in reduction and speedup in LU factorization time.

cs.LG↗

arXiv2025

Factorization-in-Loop: Proximal Fill-in Minimization for Sparse Matrix Reordering

Fill-ins are new nonzero elements in the summation of the upper and lower triangular factors generated during LU factorization. For large sparse matrices, they will increase the memory usage and computational time, and be reduced through proper row or column arrangement, namely matrix reordering. Finding a row or column permutation with the minimal fill-ins is NP-hard, and surrogate objectives are designed to derive fill-in reduction permutations or learn a reordering function. However, there is no theoretical guarantee between the golden criterion and these surrogate objectives. Here we propose to learn a reordering network by minimizing $l_1$ norm of triangular factors of the reordered matrix to approximate the exact number of fill-ins. The reordering network utilizes a graph encoder to predict row or column node scores. For inference, it is easy and fast to derive the permutation from sorting algorithms for matrices. For gradient based optimization, there is a large gap between the predicted node scores and resultant triangular factors in the optimization objective. To bridge the gap, we first design two reparameterization techniques to obtain the permutation matrix from node scores. The matrix is reordered by multiplying the permutation matrix. Then we introduce the factorization process into the objective function to arrive at target triangular factors. The overall objective function is optimized with the alternating direction method of multipliers and proximal gradient descent. Experimental results on benchmark sparse matrix collection SuiteSparse show the fill-in number and LU factorization time reduction of our proposed method is 20% and 17.8% compared with state-of-the-art baselines.

cs.LG↗

arXiv2025

Towards a Higher Roofline for Matrix-Vector Multiplication in Matrix-Free HOSFEM

Modern GPGPUs provide massive arithmetic throughput, yet many scientific kernels remain limited by memory bandwidth. In particular, repeatedly loading precomputed auxiliary data wastes abundant compute resources while stressing the memory hierarchy. A promising strategy is to replace memory traffic with inexpensive recomputation, thereby alleviating bandwidth pressure and enabling applications to better exploit heterogeneous compute units. Guided by this strategy, we optimize the high-order/spectral finite element method (HOSFEM), a widely used approach for solving PDEs. Its performance is largely determined by AxLocal, a matrix-free kernel for element-local matrix-vector multiplications. In AxLocal, geometric factors dominate memory accesses while contributing minimally to computation, creating a bandwidth bottleneck that caps the performance roofline. To address this challenge, we propose the first practical, low-overhead on-the-fly recomputation of geometric factors for trilinear and parallelepiped elements. This reformulation reduces data movement and raises the achievable roofline, revealing untapped optimization potential for tensor contractions. With hardware-aware techniques including loop unrolling, Tensor Core acceleration, and constant memory utilization, the optimized kernels reach 85%-100% of the roofline efficiency. Compared with state-of-the-art implementations in the Nek series, they deliver speedups of 1.74x-4.10x on NVIDIA A100 and 1.99x-3.78x on Hygon K100, leading to a 1.12x-1.40x improvement in the full HOSFEM benchmark. These results demonstrate that combining algorithmic reformulation with hardware-specific tuning can remove long-standing bottlenecks and fully exploit the performance potential of large-scale high-order simulations.

cs.PF↗

arXiv2025

An Efficient Finite Element Method for Multi-dimensional Nonlocal Laplacian on Uniform Grids

Computing the stiffness matrix for the finite element discretization of the nonlocal Laplacian on unstructured meshes is difficult, because the operator is nonlocal and can even be singular. In this paper, we focus on the $C^0$-piecewise linear finite element method (FEM) for the nonlocal Laplacian on uniform grids within a $d$-dimensional rectangular domain. By leveraging the connection between FE bases and B-splines (having attractive convolution properties), we can reduce the involved $2d$-dimensional integrals for the stiffness matrix entries into integrations over $d$-dimensional balls with explicit integrands involving cubic B-splines and the kernel functions, which allows for explicit study of the singularities and accurate evaluations of such integrals in spherical coordinates. We show the nonlocal stiffness matrix has a block-Toeplitz structure, so the matrix-vector multiplication can be implemented using fast Fourier transform (FFT). In addition, when the interaction radius $δ\to 0^+,$ the nonlocal stiffness matrix automatically reduces to the local one. Although our semi-analytic approach on uniform grids cannot be extended to general domains with unstructured meshes, the resulting solver can seamlessly integrate with the grid-overlay (Go) technique for the nonlocal Laplacian on arbitrary bounded domains.

math.NA↗

arXiv2025

Efficient GPU-Centered Singular Value Decomposition Using the Divide-and-Conquer Method

Singular Value Decomposition (SVD) is a fundamental matrix factorization technique in linear algebra, widely applied in numerous matrix-related problems. However, traditional SVD approaches are hindered by slow panel factorization and frequent CPU-GPU data transfers in heterogeneous systems, despite advancements in GPU computational capabilities. In this paper, we introduce a GPU-centered SVD algorithm, incorporating a novel GPU-based bidiagonal divide-and-conquer (BDC) method. We reformulate the algorithm and data layout of different steps for SVD computation, performing all panel-level computations and trailing matrix updates entirely on GPU to eliminate CPU-GPU data transfers. Furthermore, we integrate related computations to optimize BLAS utilization, thereby increasing arithmetic intensity and fully leveraging the computational capabilities of GPUs. Additionally, we introduce a newly developed GPU-based BDC algorithm that restructures the workflow to eliminate matrix-level CPU-GPU data transfers and enable asynchronous execution between the CPU and GPU. Experimental results on AMD MI210 and NVIDIA V100 GPUs demonstrate that our proposed method achieves speedups of up to 1293.64x/7.47x and 14.10x/12.38x compared to rocSOLVER/cuSOLVER and MAGMA, respectively.

cs.DC↗

arXiv2025

A spectral Levenberg-Marquardt-Deflation method for multiple solutions of semilinear elliptic systems

Many nonlinear differential equations arising from practical problems may permit nontrivial multiple solutions relevant to applications, and these multiple solutions are helpful to deeply understand these practical problems and to improve some applications. Developing an efficient numerical method for finding multiple solutions is very necessary due to the nonlinearity and multiple solutions of these equations. Moreover, providing an efficient iteration plays an important role in successfully obtaining multiple solutions with fast and stable convergence. In the current paper, an efficient algorithm for finding multiple solutions of semilinear elliptic systems is proposed, where the trust region Levenberg-Marquardt method is firstly used to iterate the resulted nonlinear algebraic system. When the nonlinear term in these equations has only the first derivative, our algorithm can efficiently find multiple solutions as well. Several numerical experiments are tested to show the efficiency of our algorithm, and some solutions which have not been shown in the literature are also found and shown.

math.OC↗

arXiv2025

Fast Maxwell Solvers Based on Exact Discrete Eigen-Decompositions I. Two-Dimensional Case

In this paper, we propose fast solvers for Maxwell's equations in rectangular domains. We first discretize the simplified Maxwell's eigenvalue problems by employing the lowest-order rectangular Nédélec elements and derive the discrete eigen-solutions explicitly, providing a Hodge-Helmholtz decomposition framework at the discrete level. Based on exact eigen-decompositions, we further design fast solvers for various Maxwell's source problems, guaranteeing either the divergence-free constraint or the Gauss's law at the discrete level. With the help of fast sine/cosine transforms, the computational time grows asymptotically as $\mathcal{O}(n^2\log n)$ with $n$ being the number of grids in each direction. Our fast Maxwell solvers outperform other existing Maxwell solvers in the literature and fully rival fast scalar Poisson/Helmholtz solvers based on trigonometric transforms in either efficiency, robustness, or storage complexity. It is also utilized to perform an efficient pre-conditioning for solving Maxwell's source problems with variable coefficients. Finally, numerical experiments are carried out to illustrate the effectiveness and efficiency of the proposed fast solver.

math.NA↗

arXiv2025

Spectral Method for 1-D Neutron Transport Equation

In this paper, we present an efficient fully spectral approximation scheme for exploring the one-dimensional steady-state neutron transport equation. Our methodology integrates the spectral-(Petrov-)Galerkin scheme in the spatial dimension with the Legendre-Gauss collocation scheme in the directional dimension. The directional integral in the original problem is discretized with Legendre-Gauss quadrature. We furnish a rigorous proof of the solvability of this scheme and, to our best knowledge, conduct a comprehensive error analysis for the first time. Notably, the order of convergence is optimal in the directional dimension, while in the spatial dimension, it is suboptimal and, importantly, non-improvable. Finally, we verify the computational efficiency and error characteristics of the scheme through several numerical examples.

math.NA↗

arXiv2024

Optimizing Winograd Convolution on ARMv8 processors

As Convolutional Neural Networks (CNNs) gain prominence in deep learning, algorithms like Winograd Convolution have been introduced to enhance computational efficiency. However, existing implementations often face challenges such as high transformation overhead, suboptimal computation efficiency, and reduced parallel performance in some layers. We propose a fused Winograd Convolution algorithm optimized for ARMv8 CPUs, integrating input transformation, filter transformation, computation, and output transformation into a single pipeline. By maintaining consecutive memory access and using a custom z-shaped data layout, our approach fully utilizes an optimized GEMM micro-kernel with a ping-pong technique. Additionally, we introduce a multi-dimensional parallel strategy that adapts to convolutional layer scales. To maximize performance, we manually optimize each kernel in AArch64 assembly and carefully tune blocking parameters. Experimental results show speedups of up to 4.74x, 4.10x, 4.72x, and 10.57x over NCNN, NNPACK, FastConv, and ACL on the Kunpeng 920 platform using multiple threads, with respective gains of 3.85x, 2.81x, 4.20x, and 7.80x on the AWS Graviton2, and 3.32x, 3.68x, 8.00x, and 9.28x on the Phytium 2000+.

cs.PF↗

arXiv2024

An Adaptive Orthogonal Basis Method for Computing Multiple Solutions of Differential Equations with polynomial nonlinearities

This paper presents an innovative approach, the Adaptive Orthogonal Basis Method, tailored for computing multiple solutions to differential equations characterized by polynomial nonlinearities. Departing from conventional practices of predefining candidate basis pools, our novel method adaptively computes bases, considering the equation's nature and structural characteristics of the solution. It further leverages companion matrix techniques to generate initial guesses for subsequent computations. Thus this approach not only yields numerous initial guesses for solving such equations but also adapts orthogonal basis functions to effectively address discretized nonlinear systems. Through a series of numerical experiments, this paper demonstrates the method's effectiveness and robustness. By reducing computational costs in various applications, this novel approach opens new avenues for uncovering multiple solutions to differential equations with polynomial nonlinearities.

math.NA↗

arXiv2024

Highly efficient Gauss's law-preserving spectral algorithms for Maxwell's double-curl source and eigenvalue problems based on eigen-decomposition

In this paper, we present Gauss's law-preserving spectral methods and their efficient solution algorithms for curl-curl source and eigenvalue problems in two and three dimensions arising from Maxwell's equations. Arbitrary order $H(curl)$-conforming spectral basis functions in two and three dimensions are firstly proposed using compact combination of Legendre polynomials. A mixed formulation involving a Lagrange multiplier is then adopted to preserve the Gauss's law in the weak sense. To overcome the bottleneck of computational efficiency caused by the saddle-point nature of the mixed scheme, we present highly efficient solution algorithms based on reordering and decoupling of the resultant linear algebraic system and numerical eigen-decomposition of one dimensional mass matrix. The proposed solution algorithms are direct methods requiring only several matrix-matrix or matrix-tensor products of $N$-by-$N$ matrices, where $N$ is the highest polynomial order in each direction. Compared with other direct methods, the computational complexities are reduced from $O(N^6)$ and $O(N^9)$ to $O(N^3)$ and $O(N^4)$ with small and constant pre-factors for 2D and 3D cases, respectively, and can further be accelerated to $O(N^{2.807})$ and $O(N^{3.807})$, when boosted with the Strassen's matrix multiplication algorithm. Moreover, these algorithms strictly obey the Helmholtz-Hodge decomposition, thus totally eliminate the spurious eigen-modes of non-physical zero eigenvalues. Extensions of the proposed methods and algorithms to problems in complex geometries with variable coefficients and inhomogeneous boundary conditions are discussed to deal with more general situations. Ample numerical examples for solving Maxwell's source and eigenvalue problems are presented to demonstrate the accuracy and efficiency of the proposed methods.

math.NA↗

arXiv2023

An exact divergence-free spectral method for incompressible and resistive magneto-hydrodynamic equations in two and three dimensions

In this paper, we present exact divergence-free spectral method for solving the incompressible and resistive magneto-hydrodynamic (MHD) equations in two and three dimensions, as well as the efficient solution algorithm and unconditionally energy-stable fully-discretized numerical schemes. We introduce new ideas of constructing two families of exact divergence-free vectorial spectral basis functions on domains diffeomorphic to squares or cubes. These bases are obtained with the help of orthogonality and derivative relation of generalised Jacobi polynomials, several de Rham complexes, as well as the property of contravariant Piola transformation. They are well-suited for discretizing the velocity and magnetic fields, respectively, thereby ensuring point-wise preservation of the incompressibility condition and the magnetic Gauss's law. With the aid of these bases, we propose a family of exact divergence-free implicit-explicit $k$-step backward differentiation formula (DF-BDF-$k$) fully-discretized schemes for the MHD system. These schemes naturally decouple the pressure field from the velocity field. Consequently, the stability of the space-time fully-discretized numerical schemes based on these bases are significantly enhanced. These schemes exhibit unconditional stability for $k=1,2$, and demonstrate exceptional stability and accuracy for $k=3,4$, verified with extensive numerical results for long time simulations using large time step sizes. Furthermore, we present efficient solution algorithms for these two decoupled equations for the velocity and magnetic fields, respectively, by exploiting the sparsity and structure of the resultant linear algebraic systems. Ample numerical examples in two and three dimensions are provided to demonstrate the distinctive accuracy, efficiency and stability of our proposed method.

math.NA↗

arXiv2023

Müntz ball polynomials and Müntz spectral-Galerkin methods for singular eigenvalue problems

In this paper, we introduce a new family of orthogonal systems, termed as the Müntz ball polynomials (MBPs), which are orthogonal with respect to the weight function: $\|x\|^{2θ+2μ-2} (1-\|x\|^{2θ})^α$ with the parameters $α>-1, μ>- 1/2$ and $θ>0$ in the $d$-dimensional unit ball $x\in {\mathbb B}^d=\big\{x\in\mathbb{R}^d: r=\|x\|\leq1\big\}$. We then develop efficient and spectrally accurate MBP spectral-Galerkin methods for singular eigenvalue problems including degenerating elliptic problems with perturbed ellipticity and Schrödinger's operators with fractional potentials. We demonstrate that the use of such non-standard basis functions can not only tailor to the singularity of the solutions but also lead to sparse linear systems which can be solved efficiently.

math.NA↗

arXiv2023

An Efficient Spectral Trust-Region Deflation Method for Multiple Solutions

It is quite common that a nonlinear partial differential equation (PDE) admits multiple distinct solutions and each solution may carry a unique physical meaning. One typical approach for finding multiple solutions is to use the Newton method with different initial guesses that ideally fall into the basins of attraction confining the solutions. In this paper, we propose a fast and accurate numerical method for multiple solutions comprised of three ingredients: (i) a well-designed spectral-Galerkin discretization of the underlying PDE leading to a nonlinear algebraic system (NLAS) with multiple solutions; (ii) an effective deflation technique to eliminate a known (founded) solution from the other unknown solutions leading to deflated NLAS; and (iii) a viable nonlinear least-squares and trust-region (LSTR) method for solving the NLAS and the deflated NLAS to find the multiple solutions sequentially one by one. We demonstrate through ample examples of differential equations and comparison with relevant existing approaches that the spectral LSTR-Deflation method has the merits: (i) it is quite flexible in choosing initial values, even starting from the same initial guess for finding all multiple solutions; (ii) it guarantees high-order accuracy; and (iii) it is quite fast to locate multiple distinct solutions and explore new solutions which are not reported in literature.

math.NA↗

arXiv2021

Sparse Spectral-Galerkin Method on An Arbitrary Tetrahedron Using Generalized Koornwinder Polynomials

In this paper, we propose a sparse spectral-Galerkin approximation scheme for solving the second-order partial differential equations on an arbitrary tetrahedron. Generalized Koornwinder polynomials are introduced on the reference tetrahedron as basis functions with their various recurrence relations and differentiation properties being explored. The method leads to well-conditioned and sparse linear systems whose entries can either be calculated directly by the orthogonality of the generalized Koornwinder polynomials for differential equations with constant coefficients or be evaluated efficiently via our recurrence algorithm for problems with variable coefficients. Clenshaw algorithms for the evaluation of any polynomial in an expansion of the generalized Koornwinder basis are also designed to boost the efficiency of the method. Finally, numerical experiments are carried out to illustrate the effectiveness of the proposed Koornwinder spectral method.

math.NA↗