arXiv subjects

Salman Ahmadi-Asl

Publications and source records attributed to Salman Ahmadi-Asl.

At least 19 recordsLinked to original sources

arXiv2026

Efficient iterative techniques for solving tensor problems with the T-product

This paper develops two efficient iterative methods for solving tensor equations under the T-product framework. For T-symmetric positive definite tensor equations of the form $\mathcal{C} \star \mathcal{X} = \mathcal{D}$, we propose a conjugate-gradient-type algorithm that generates orthogonal residual and $\mathcal{C}$-orthogonal direction sequences, ensuring convergence within a finite number of steps. For general consistent tensor equations, we extend the method using a normal-equation transformation, and further adapt it to handle inconsistent systems by solving a least-squares minimization problem. Key advantages include direct tensor-based computations without explicit matrix expansion, rigorous finite-step convergence proofs, and the ability to obtain minimal Frobenius norm solutions. Numerical experiments on synthetic data, benchmark images, and video sequences demonstrate that the proposed algorithms achieve high precision with low computational time, confirming their practicality for large-scale multidimensional problems.

math.NA↗

arXiv2026

Adaptive Randomized Pivoting for Tensor Cross Approximation in the T-Product Framework

This paper studies extensions of adaptive randomized pivoting (ARP), recently introduced for matrix column subset selection, to tensors in the t-product framework. We propose two constructions. ARP-T-CUR applies matrix ARP-cross at the nonredundant Fourier frequencies and uses conjugate symmetry to preserve real-valued tensors. Assuming that the selected intersections are nonsingular almost surely, the matrix theory gives a direct expected-error bound for this Fourier-slicewise approximation. T-ARP instead selects common lateral and horizontal slices, with the same indices used at every Fourier frequency. This common-index constraint requires a new analysis. Under frequency-alignment and frequency-wise rank-growth assumptions, we prove an expected-error bound and recover the matrix ARP factor $r+1$ when the frequency-wise sampling distributions are aligned. We also derive bounds for tensor cross approximation and t-DEIM. The numerical experiments show that both proposed methods give accurate tensor approximations and perform favorably against the considered slice-selection baselines. A separate full-enumeration experiment verifies the assumptions and the expected-error bound for common-index T-ARP.

math.NA↗

arXiv2026

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is underexploited by the conventional matrix-centric view. Tensor decompositions and tensor networks provide a principled algebraic language for this structure, yet the literature often treats them as isolated compression mechanisms. This survey organizes tensor methods for LLMs through two complementary views: a seven-stage lifecycle taxonomy covering tokenization, embeddings, pre-training, adaptation, compression, inference, and interpretability, and a component view covering embeddings, attention, and feed-forward networks. We provide unified notation and theoretical foundations, analyze tensorization strategies for individual Transformer components, and compare methods at each lifecycle stage while making differences in evaluation protocols and model scales explicit. We further connect tensor methods to neighboring efficiency techniques and probabilistic tensor networks. Finally, we synthesize open challenges and introduce $ρ_{\rm gap}$, a metric for the compression-realization gap between theoretical memory reduction and measured system-level speedup. By treating tensorization as a common structural principle, the survey provides a structured entry point to tensorized language models and clarifies when parameter savings can plausibly translate into memory efficiency, computational efficiency, or interpretability. The GitHub page dedicated to this paper is accessible at \href{https://github.com/ma-tt-a/awesome-tensor-methods-for-llms}{this https URL}.

cs.LG↗

arXiv2026

Efficient Techniques for Low-Rank Tensor Approximation and Applications in Robust Object Detection

This paper introduces efficient randomized fixed-precision and single-pass algorithms for low-tubal-rank approximation of third-order tensors. The proposed fixed-precision algorithms are faster and more efficient than the existing algorithms for approximating the truncated tensor SVD (T-SVD). Besides, unlike existing single-pass methods, which directly extend early, unstable matrix algorithms, the proposed approach adapts enhanced and stabilized matrix techniques to the tensor setting. Through extensive numerical experiments, we identify a critical flaw in current single-pass algorithms: using sketching parameters of equal size often produces ill-conditioned tensor least-squares problems, leading to inaccurate approximations. The proposed algorithms are demonstrably robust to this issue, achieving superior performance under identical conditions. We also evaluate the robustness of existing single-pass methods on real-world data tensors, including images and videos, a topic that has not been thoroughly examined before. Numerical results confirm the effectiveness of the proposed methods. Three applications are presented: image compression, video super-resolution, and deep learning.

math.NA↗

arXiv2026

Iterative Methods for Computing the Moore--Penrose Inverse of Split-Quaternion Matrices with Applications

We study iterative methods for computing the Moore--Penrose inverse of split-quaternion matrices. We first establish a consistent framework based on a $2\times2$ real representation and the associated $i$-conjugate transpose. This representation gives a direct definition of the Moore--Penrose inverse and clarifies the treatment of nonzero zero divisors. We then analyze Newton--Schulz iterations for rectangular and rank-deficient matrices. Using a thin singular value decomposition of the real representative, we derive the convergence conditions, the evolution of the projector residuals, and an exact relation between the residual and the error. The resulting analysis applies both to an embedded real implementation and to a native split-quaternion iteration, which are shown to be equivalent under the representation. We also propose a low-degree polynomial initialization inspired by Souriau's inverse recursion. The polynomial is obtained by a least-squares approximation of the inverse Gram factor and is combined with a spectral acceptance test and a safe fallback initialization. Finally, we apply the proposed methods to cross and CUR approximations of split-quaternion matrices. We characterize the optimal middle factor for fixed sampled rows and columns and give conditions under which the cheaper cross factor gives an exact reconstruction. Numerical experiments illustrate the accuracy of the methods and the practical benefit of the polynomial initialization.

math.NA↗

arXiv2026

Fast randomized Kronecker tensor decomposition: algorithms and error analysis

This paper proposes fast randomized algorithms for computing the Kronecker Tensor Decomposition (KTD) by replacing the sequence of deterministic SVDs in the TTr1SVD framework with randomized SVDs incorporating oversampling and power iterations. The proposed algorithms can decompose a given tensor into the KTD format significantly faster than existing state-of-the-art deterministic methods. Our principal idea is to use randomization to reduce computational complexity while maintaining controlled accuracy. A detailed theoretical analysis is presented, including a recursive error bound that accounts for error propagation through the TTr1SVD tree structure. We prove that the expected Frobenius norm error is bounded by a sum of tail energies multiplied by factors that decay exponentially with the number of power iterations. Extensive simulations on synthetic and real-world datasets demonstrate several orders of magnitude acceleration compared to the deterministic approach, with applications to tensor completion, video/image compression, image denoising, and image super-resolution.

math.NA↗

arXiv2026

QuatIca: Advanced Numerical Linear Algebra and Optimization for Quaternionic Matrices in Python

Quaternion-valued representations provide a convenient way to model coupled multi-channel signals (e.g., RGB imagery, polarization data, vector fields, and multi-detector time series). Yet practical and numerically reliable software support remains far less mature than those based on the real/complex setting. Here, we present QuatIca, an open-source Python library for quaternion numerical linear algebra and optimization, designed for both research prototyping and reproducible experimentation. QuatIca provides core quaternion matrix operations and norms; dense decompositions and reductions (QR, LU, Q-SVD, eigendecomposition, Hessenberg/tridiagonal reduction, Cholesky decomposition, and Schur helpers); iterative solvers including quaternion GMRES (with preconditioning) and Newton-Schulz pseudoinverse schemes; and domain-focused routines for signal and image processing such as quaternion Tikhonov restoration. The library also includes OptiQ, which solves quaternion Hermitian semidefinite programs using log-det barrier Newton methods with $μ$-continuation. We highlight design choices that preserve quaternion structure, and we provide end-to-end demonstrations including quaternion image deblurring, Lorenz-attractor filtering, and quaternion image completion. QuatIca is distributed via PyPI and accompanied by open-source development on GitHub and continuously deployed documentation with runnable tutorials.

math.NA↗

arXiv2026

Randomized block Krylov method for approximation of truncated tensor SVD

This paper is devoted to studying the application of the block Krylov subspace method for approximation of the truncated tensor SVD (T-SVD). The theoretical results of the proposed randomized approach are presented. Several experimental experiments using synthetics and real-world data are conducted to verify the efficiency and feasibility of the proposed randomized approach, and the numerical results show that the proposed method provides promising results. Applications of the proposed approach to data completion and data compression are presented.

math.NA↗

arXiv2026

Pass-efficient Randomized Algorithms for Low-rank Approximation of Quaternion Matrices

Randomized algorithms for low-rank approximation of quaternion matrices have gained increasing attention in recent years. However, existing methods overlook pass efficiency, the ability to limit the number of passes over the input matrix-which is critical in modern computing environments dominated by communication costs. We address this gap by proposing a suite of pass-efficient randomized algorithms that let users directly trade pass budget for approximation accuracy. Our contributions include: (i) a family of arbitrary-pass randomized algorithms for low-rank approximation of quaternion matrices that operate under a user-specified number of matrix views, and (ii) a pass-efficient extension of block Krylov subspace methods that accelerates convergence for matrices with slowly decaying spectra. Furthermore, we establish spectral norm error bounds showing that the expected approximation error decays exponentially with the number of passes. Finally, we validate our framework through extensive numerical experiments and demonstrate its practical relevance across multiple applications, including quaternionic data compression, matrix completion, image super-resolution, and deep learning.

math.NA↗

arXiv2026

A New Tensor Network: Tubal Tensor Train and Its Applications

We introduce the tubal tensor train (TTT) decomposition, a tensor-network model that combines the t-product algebra of the tensor singular value decomposition (T-SVD) with the low-order core structure of the tensor train (TT) format. For an order-$(N+1)$ tensor with a distinguished tube mode, the proposed representation consists of two third-order boundary cores and $N-2$ fourth-order interior cores linked through the t-product. As a result, for bounded tubal ranks, the storage scales linearly with the number of modes, in contrast to direct high-order extensions of T-SVD. We present two computational strategies: a sequential fixed-rank construction, called TTT-SVD, and a Fourier-slice alternating scheme based on the alternating two-cores update (ATCU). We also state a TT-SVD-type error bound for TTT-SVD and illustrate the practical performance of the proposed model on image compression, video compression, tensor completion, and hyperspectral imaging.

math.NA↗

arXiv2026

A note on generalized tensor CUR approximation for tensor pairs and tensor triplets based on the tubal product

In this note, we briefly present a generalized tensor CUR (GTCUR) approximation for tensor pairs (X,Y) and tensor triplets (X,Y,Z) based on the tubal product (t-product). We use the tensor Discrete Empirical Interpolation Method (TDEIM) to do these extensions. We show how the TDEIM can be utilized to generalize the classical tensor CUR (TCUR) approximation, which acts only on a single tensor, to jointly compute the TCUR of two and three tensors. This approach can be used to sample relevant lateral/horizontal slices of one data tensor relative to one or two other data tensors. For some special cases, the Generalized TCUR (GTCUR) approximation is reduced to the classical TCUR for both tensor pairs and tensor triplets in a similar fashion as shown for the matrices.

math.NA↗

arXiv2025

Iterative Methods for Computing the Moore-Penrose Pseudoinverse of Quaternion Matrices, with Applications

We develop quaternion--native iterative methods for computing the Moore--Penrose (MP) pseudoinverse of quaternion matrices and analyze their convergence. Our starting point is a damped Newton--Schulz (NS) iteration tailored to noncommutativity: we enforce the appropriate left/right identities for rectangular inputs and prove convergence directly in $\mathbb{H}$ under a simple spectral scaling. We then derive higher--order (\emph{hyperpower}) NS schemes with exact residual recurrences that yield order-$p$ local convergence, together with factorizations that reduce the number of $s\times s$ quaternion products per iteration. Beyond NS, we introduce a randomized sketch--and--project method (RSP--Q), a hybrid RSP+NS scheme that interleaves inexpensive randomized projections with an exact hyperpower step, and a matrix--form conjugate gradient on the normal equations (CGNE--Q). All algorithms operate directly in $\mathbb{H}$ (no real or complex embeddings) and are matrix--free.

math.NA↗

arXiv2025

SAGE: Streaming Agreement-Driven Gradient Sketches for Representative Subset Selection

Training modern neural networks on large datasets is computationally and energy intensive. We present SAGE, a streaming data-subset selection method that maintains a compact Frequent Directions (FD) sketch of gradient geometry in $O(\ell D)$ memory and prioritizes examples whose sketched gradients align with a consensus direction. The approach eliminates $N \times N$ pairwise similarities and explicit $N \times \ell$ gradient stores, yielding a simple two-pass, GPU-friendly pipeline. Leveraging FD's deterministic approximation guarantees, we analyze how agreement scoring preserves gradient energy within the principal sketched subspace. Across multiple benchmarks, SAGE trains with small kept-rate budgets while retaining competitive accuracy relative to full-data training and recent subset-selection baselines, and reduces end-to-end compute and peak memory. Overall, SAGE offers a practical, constant-memory alternative that complements pruning and model compression for efficient training.

cs.LG↗

arXiv2025

Lightweight Attribute Localizing Models for Pedestrian Attribute Recognition

Pedestrian Attribute Recognition (PAR) focuses on identifying various attributes in pedestrian images, with key applications in person retrieval, suspect re-identification, and soft biometrics. However, Deep Neural Networks (DNNs) for PAR often suffer from over-parameterization and high computational complexity, making them unsuitable for resource-constrained devices. Traditional tensor-based compression methods typically factorize layers without adequately preserving the gradient direction during compression, leading to inefficient compression and a significant accuracy loss. In this work, we propose a novel approach for determining the optimal ranks of low-rank layers, ensuring that the gradient direction of the compressed model closely aligns with that of the original model. This means that the compressed model effectively preserves the update direction of the full model, enabling more efficient compression for PAR tasks. The proposed procedure optimizes the compression ranks for each layer within the ALM model, followed by compression using CPD-EPC or truncated SVD. This results in a reduction in model complexity while maintaining high performance.

cs.CV↗

arXiv2025

A Randomized Algorithm for Tensor Singular Value Decomposition using an Arbitrary Number of Passes

Efficient and fast computation of a tensor singular value decomposition (t-SVD) with a few passes over the underlying data tensor is crucial because of its many potential applications. The current/existing subspace randomized algorithms need (2q+2) passes over the data tensor to compute a t-SVD, where q is a non-negative integer number (power iteration parameter). In this paper, we propose an efficient and flexible randomized algorithm that can handle any number of passes q, which not necessary need be even. The flexibility of the proposed algorithm in using fewer passes naturally leads to lower computational and communication costs. This advantage makes it particularly appropriate when our task calls for several tensor decompositions or when the data tensors are huge. The proposed algorithm is a generalization of the methods developed for matrices to tensors. The expected/ average error bound of the proposed algorithm is derived. Extensive numerical experiments on random and real-world data sets are conducted, and the proposed algorithm is compared with some baseline algorithms. The extensive computer simulation experiments demonstrate that the proposed algorithm is practical, efficient, and in general outperforms the state of the arts algorithms. We also demonstrate how to use the proposed method to develop a fast algorithm for the tensor completion problem.

math.NA↗

arXiv2025

Nonnegative Tensor Decomposition Via Collaborative Neurodynamic Optimization

This paper introduces a novel collaborative neurodynamic model for computing nonnegative Canonical Polyadic Decomposition (CPD). The model relies on a system of recurrent neural networks to solve the underlying nonconvex optimization problem associated with nonnegative CPD. Additionally, a discrete-time version of the continuous neural network is developed. To enhance the chances of reaching a potential global minimum, the recurrent neural networks are allowed to communicate and exchange information through particle swarm optimization (PSO). Convergence and stability analyses of both the continuous and discrete neurodynamic models are thoroughly examined. Experimental evaluations are conducted on random and real-world datasets to demonstrate the effectiveness of the proposed approach.

math.NA↗

arXiv2024

Fast randomized algorithms for computing the generalized tensor SVD based on the tubal product

This work deals with developing two fast randomized algorithms for computing the generalized tensor singular value decomposition (GTSVD) based on the tubal product (t-product). The random projection method is utilized to compute the important actions of the underlying data tensors and use them to get small sketches of the original data tensors, which are easier to be handled. Due to the small size of the sketch tensors, deterministic approaches are applied to them to compute their GTSVDs. Then, from the GTSVD of the small sketch tensors, the GTSVD of the original large-scale data tensors is recovered. Some experiments are conducted to show the effectiveness of the proposed approach.

math.NA↗

arXiv2024

An Efficient Randomized Fixed-Precision Algorithm for Tensor Singular Value Decomposition

The existing randomized algorithms need an initial estimation of the tubal rank to compute a tensor singular value decomposition. This paper proposes a new randomized fixedprecision algorithm which for a given third-order tensor and a prescribed approximation error bound, automatically finds an optimal tubal rank and the corresponding low tubal rank approximation. The algorithm is based on the random projection technique and equipped with the power iteration method for achieving a better accuracy. We conduct simulations on synthetic and real-world datasets to show the efficiency and performance of the proposed algorithm.

math.NA↗