Searcharxiv⌕ Search

arXiv · 2611.21814

Newton's Method for CP Decomposition

Abstract

We present an algorithm for computing the CP tensor decomposition using Newton's method. The problem of fitting a CP model to a data tensor with least squares loss can be solved by various gradient-based optimization techniques, including Gauss-Newton methods that exploit the tensor structure of the problem and iteratively solve the linear system involving the approximate Hessian matrix to determine the step direction at each iteration. Gauss-Newton methods exhibit superlinear convergence when the residual is small but can otherwise struggle to converge when the approximation to the Hessian is not accurate. We show that the Gauss-Newton approach can be extended to Newton's method, using the true Hessian to obtain fast convergence even in the case of large residuals. In particular, we derive efficient means of applying the true Hessian to vectors for use within an iterative solver, we employ globalization and regularization techniques to make the Newton model robust to ill-conditioned and/or indefinite Hessians, and we propose hybrid use of Gauss-Newton and Newton steps to balance convergence behavior both far and near local minima. Our numerical experiments use synthetic examples to show in which scenarios Gauss-Newton and other methods struggle to converge, and we demonstrate the superiority of the hybrid approach on two application data sets.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

DuBose Tuller, Grey Ballard, Hussam Al Daas, Nicholas I. M. Gould. 2026-11-16. Newton's Method for CP Decomposition. https://searcharxiv.com/abs/2611.21814

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Discontinuity-Aware Solvers for Vapor-Compression Cycles

Physics-based models of vapor-compression cycles are often formulated as large, stiff DAE systems with derivative discontinuities introduced at the points of phase change. We use an implementation of a discontinuity-aware ODE and DAE solver scheme that uses domain-specific, user-declared information to compute appropriate time steps for more efficient integration. This scheme is made open-source within Julia's DifferentialEquations package. We demonstrate how discontinuity information can be specified symbolically in a declarative modeling language such as Dyad and passed to the solver to address the numerical challenges posed. We apply this discontinuity-aware technique to two applications: first, a scalar ODE with two discontinuities, and then a model of a full vapor-compression cycle. The discontinuity-aware solver yields significantly fewer step rejections and improved solver times, particularly with the Rodas5P solver at loose tolerances, as compared to standard error control, suggesting that exposing physical discontinuity information from equation-oriented models to adaptive solvers can improve practical simulation performance without requiring reassembly of the model.

cs.MS↗

Sparse Integer Fourier Transform and Its Application In Machine Learning

The Discrete Fourier transform, mainly implemented by Fast Fourier Transform (FFT), has been essential for numerous domains, such as digital signal processing and feature extraction in materials design. However, current FFT methods suffer from extensive non-trivial multiplications and twiddle-factor operations in the big data era. To improve the computational efficiency of FFT, this study proposes Sparse Integer Fourier Transform (SIFT), a method that leverages the fast computing of sparsity and small integers, and applies the Radix-n method to derive an FFT. The 2N-point DFT matrix in Radix-n FFT is decomposed into four sparse integer matrices. A diagonal matrix containing complex twiddle factors is efficiently approximated using an optimization strategy while the remaining matrices retain sparsity and integer entries. For radix-32 FFT, SIFT reduces the estimated arithmetic cost by 75%, with RMSE below $2.3765\times10^{-8}$ and MAPE below 0.01089%. We evaluate frequency extraction in a numerical test and an engineering case. In the numerical test, MAPE and RMSE are below 2.016% and $2.536\times10^{-6}$, respectively. For wind turbine fault diagnosis, SIFT extracts compact spectral features that retain fault information while reducing the classifier's input dimensionality relative to the full time series.

cs.MS↗

MRASLive: An online parameter estimation toolbox for time-dependent PDEs

We present MRASLive, a toolbox for online parameter identification in time-dependent partial differential equations based on a model reference adaptive system (MRAS). Building on the MRAS proposed by Kaltenbacher and Nguyen (Inverse Problems 37(5) 2021) and its first numerical realization by the authors (Applied Mathematics for Modern Challenges 8 2026), the toolbox is designed for ease of use, utilising the user's weak form specification to prepare the data, the discretized MRAS and its solvers, and to verify the assumptions of MRAS's convergence theory semi-automatically.

cs.MS↗