arXiv subjects

Karl Pierce

Publications and source records attributed to Karl Pierce.

7 recordsLinked to original sources

arXiv2026

Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Network

In this work we present a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN). This optimization procedure introduces contextual features into the first layer of a DNN. The parameters of DNN are optimized via standard gradient descent while keeping the input-feature basis fixed. After optimization of the DNN parameters, the feature layer is provided a chance to update and change before DNN optimization resumes. The feature layer has two types of functions: those that can be evaluated quickly in a matrix-free way on the domain (i.e. rank-1 features) and more complex features that must first be decomposed using tensor network (TN) decomposition strategies (tensor features). In particular, we study the effect of adding features which distill pretrained DNN into TNs using a discretize and decompose strategy. To efficiently decompose high-dimensional functions constructed from discretized DNN, we leverage a randomized tensor decomposition strategy. Using randomization, we are able to reduce the storage cost of decomposing high dimensional functions by at least 8 orders of magnitude. Using this approach, we are able to efficiently train models between 5 and 40 dimensions.

cs.LG↗

arXiv2026

Accelerating the Canonical Polyadic Alternating Least Squares Optimization via a Randomized Interpolative Decomposition

We present a novel leverage score-based sampling strategy for the randomized alternating least squares optimization (ALS) of the canonical polyadic decomposition (CPD-ALS). Unlike previous strategies, we determine row-wise samples for the CPD-ALS problem from the leverage scores of the target tensor which is being decomposed. We demonstrate that, when rows are sampled according to the leverage score distribution of the matricized target tensor, each least squares subproblem of the CPD-ALS problem achieves $(1+\epsilon)-$relative accuracy in the residual norm with probability at least $1-\delta$ using a sampling $s=\frac{R\gamma}{\beta} \max\left(\frac{4}{\delta \epsilon}, \frac{144\ln(2R/\delta)}{\epsilon_{0}^{2}}\right)$, where $\epsilon_{0}$ is a constant, $\beta$ is leverage score's approximation constant, $R$ is the target rank and $\gamma$ captures the coherence between the Khatri Rao product (KRP) of the CPD factor matrices and the exact KRP; $\gamma$ decreases as the ALS iterates converge. To efficiently approximate the leverage score distribution for each matricization of the target tensor without explicitly computing leverage scores we use a randomized strong rank-revealing QR (sRRQR) factorizations, SE-QRCS. By construction, this QR-based leverage score sampling method outperforms previously published schemes as it does not, in principle, require the resampling of the target tensor or recomputing the leverage scores of the KRP, minimizing the computational and storage overhead of the CPD-ALS procedure.

math.NA↗

arXiv2026

A CPD-enabled low-scaling environment solver in a coupled cluster based static quantum embedding theory

We incorporate a canonical polyadic decomposition (CPD) based low-level solver as a means to accelerate the environment-level solver for the recently developed MPCC embedding framework. Using CPD, we both factorize the three dominant order-three density-fitting two-electron integral (DF TEI) tensors and develop a novel formulation that reduces the storage complexity of the low-level solver from ${O}(N^3)$ to $O(NR)$, where $R$ is the CPD rank, and the computational scaling of the most time-consuming contractions from ${O}(N^4)$ to ${O}(NR^2)$. We provide benchmarks on representative chemical environments, namely water clusters $\mathrm{{(H_2O)_n}}$ with $n = 1$ to $6$ and linear alkane chains $\mathrm{{C_nH_{2n+2}}}$ with $n = 1$ to $6$. For both test sets, using the CPD-compressed DF TEI tensors reproduces the DF reference convergence behavior of the low-level solver, the subsequent high-level step, and the fully self-consistent MPCC iterations, while introducing only small, rank-controlled shifts in absolute energies. At a fixed tolerance in the absolute MPCC energy, the CP ranks required for these tensor approximations increase linearly with system size. Chemically relevant energy differences are likewise preserved, as demonstrated for water-cluster dissociation energies and in a proof-of-concept embedding calculation of methane in a four-water cluster.

physics.chem-ph↗

arXiv2025

Towards Using Matrix-Free Tensor Decompositions to Systematically Improve Approximate Tensor-Networks

We investigate a novel approach to approximate tensor-network contraction via the exact, matrix-free decomposition of full tensor-networks. We study this method as a means to eliminate the propagation of error in the approximation of tensor-networks. Importantly, this decomposition-based approach is generic, i.e. it does not depend on a specific tensor-network, the tensor index (physical) ordering, or the choice of tensor decomposition. Careful consideration should be made to determine the best decomposition strategy. Furthermore, this method does not rely on robust cancellation of errors (i.e. the Taylor expansion). As a means to study the effectiveness of the approach, we replace the exact contraction of the particle particle ladder (PPL) tensor diagram in the popular coupled-cluster with single and double excitation (CCSD) method with a low-rank tensor decomposition, namely the canonical polyadic decomposition (CPD). With this approach, we replace an $\mathcal{O}(N^6)$ tensor contractions with a potentially reduced-scaling $\mathcal{O}(N^4R)$ optimization problem, where $R$ is the CP rank, and we reduce the computational storage of the PPL tensor from $\mathcal{O}(N^4)$ to $\mathcal{O}(NR)$, although we do not take advantage of this compression in this study. To minimize the cost of the CPD optimization, we utilize the iterative structure of CCSD to efficiently initialize the CPD optimization. We show that accurate chemically-relevant energy values can be computed with an error of less than 1 kcal/mol using a relatively low CP rank.

physics.chem-ph↗

arXiv2025

Using Matrix-Free Tensor-Network Optimizations to Construct a Reduced-Scaling and Robust Second-Order M{\o}ller-Plesset Theory

We investigate the efficient combination of the canonical polyadic decomposition (CPD) and tensor hyper-contraction (THC) approaches. We first present a novel low-cost CPD solver which leverages a precomputed THC factorization of an order-$4$ tensor to efficiently optimize the order-$4$ CPD with $\mathcal{O}(NR^2)$ scaling. With the matrix-free THC-based optimization strategy in hand we can: efficiently generate CPD factorizations of the order-4 two-electron integral tensors; and develop novel electronic structure methods which take advantage of both the THC and CPD approximations. Next, we investigate the application of a combined CPD and THC approximation of the Laplace transform (LT) second-order M{\o}ller-Plesset (MP2) method. We exploit the ability to switch efficiently between the THC and CPD factorizations of the two electron integrals to reduce the computational complexity of the LT MP2 method while preserving the accuracy of the approach. Furthermore we take advantage of the robust fitting approximation to eliminate leading order error in the CPD approximated tensor networks. Finally, we show that modest values of THC and CPD rank preserve the accuracy of the LT MP2 method and that this CPD+THC LT MP2 strategy realizes a performance advantage over canonical LT MP2 in both computational wall-times and memory resource requirements.

physics.chem-ph↗

arXiv2022

Efficient construction of canonical polyadic approximations of tensor networks

We consider the problem of constructing a canonical polyadic (CP) decomposition for a tensor network, rather than a single tensor. We illustrate how it is possible to reduce the complexity of constructing an approximate CP representation of the network by leveraging its structure in the course of the CP factor optimization. The utility of this technique is demonstrated for the order-4 Coulomb interaction tensor approximated by 2 order-3 tensors via an approximate generalized square-root (SQ) factorization, such as density fitting or (pivoted) Cholesky. The complexity of constructing a 4-way CP decomposition is reduced from $\mathcal{O}(n^4 R_\text{CP})$ (for the non-approximated Coulomb tensor) to $\mathcal{O}(n^3 R_\text{CP})$ for the SQ-factorized tensor, where $n$ and $R_\text{CP}$ are the basis and CP ranks, respectively. This reduces the cost of constructing the CP approximation of 2-body interaction tensors of relevance to accurate many-body electronic structure by up to 2 orders of magnitude for systems with up to 36 atoms studied here. The full 4-way CP approximation of the Coulomb interaction tensor is shown to be more accurate than the known approaches utilizing CP-decomposed SQ factors (also obtained at the $\mathcal{O}(n^3 R_\text{CP})$ cost), such as the algebraic pseudospectral and tensor hypercontraction approaches. The CP decomposed SQ factors can also serve as a robust initial guess for the 4-way CP factors.

physics.chem-ph↗

arXiv2020

Robust approximation of tensor networks: application to grid-free tensor factorization of the Coulomb interaction

Approximation of a tensor network by approximating (e.g., factorizing) one or more of its constituent tensors can be improved by canceling the leading-order error due to the constituents' approximation. The utility of such robust approximation is demonstrated for robust canonical polyadic (CP) approximation of a (density-fitting) factorized 2-particle Coulomb interaction tensor. The resulting algebraic (grid-free) approximation for the Coulomb tensor, closely related to the factorization appearing in pseudospectral and tensor hypercontraction approaches, is efficient and accurate, with significantly reduced rank compared to the naive (non-robust) approximation. Application of the robust approximation to the particle-particle ladder term in the coupled-cluster singles and doubles reduces the size complexity from $\mathcal{O}(N^6)$ to $\mathcal{O}(N^5)$ with robustness ensuring negligible errors in chemically-relevant energy differences using CP ranks approximately equal to the size of the density-fitting basis.

physics.chem-ph↗