arXiv subjects

Khalide Jbilou

Publications and source records attributed to Khalide Jbilou.

At least 19 recordsLinked to original sources

arXiv2026

Low-Precision Rank Compensation for Matrices and Tensor Trains

Lower numerical precision reduces storage and memory traffic but raises the perturbation floor. We study rank compensation: reinvesting saved memory in a larger approximation rank. For matrices, the singular-value error identity yields a directly testable sufficient condition requiring the additional singular component to offset the perturbation from storing the rank-augmented approximation in lower precision. On ten SuiteSparse matrices, all 100 truncation-dominated configurations (50 FP32 and 50 FP16) are certified non-increases and strict accuracy wins, with mean error ratio $0.963$ and storage ratios $58.8\%$ and $29.4\%$ relative to the FP64 baseline. FP16 failures occur only in tail-rank stress tests near the perturbation floor. At the largest resident matrix-application batch, compensated FP32 and FP16 achieve geometric-mean A100 speedups of $1.28\times$ and $2.12\times$; neither accelerates the smallest batch. For Tensor-Train (TT) approximation, we give a conditional a posteriori extension based on the measured truncation gain and rounded-core perturbation. Across three-way and six-way synthetic tests, FP32 and FP16 achieve combined accuracy-memory wins in 10 of 20 and 14 of 20 trials. On public hyperspectral tensors and FROSTT top-active subtensors, the corresponding counts are 44 of 60 and 54 of 60; four FP16 Salinas-A tail-stress cases fail. No certified TT case exceeds the FP64 error beyond numerical tolerance. Reconstruction of six public tensors yields geometric-mean compensated speedups of $1.38\times$ (FP32) and $1.94\times$ (FP16). Timings cover resident downstream kernels, not factorization, transfers, or end-to-end acceleration.

math.NA↗

arXiv2026

Structured Multidimensional Representation Learning for Large Language Models

Transformer architectures achieve state-of-the-art performance across a wide range of pattern recognition and natural language processing tasks, but their scaling is accompanied by substantial parameter growth and redundancy in the embedding dimension. In this work, we introduce a structured spectral factorization of the embedding space based on the L-product for third-order tensors. By reshaping token representations into spectral tensor slices and performing attention and feed-forward operations in the transform domain, we obtain a Tensor Transformer architecture that decomposes the encoder into p independent spectral sub-transformers while preserving standard Transformer semantics. We prove that the proposed L-Transformer is spectrally equivalent to p parallel Transformers operating on reduceddimensional embeddings, which yields approximately 1/p reduction (up to lower-order terms such as biases and normalization parameters) in encoder parameters under fixed total embedding size. When instantiated with a real-valued Discrete Cosine Transform (DCT), the method remains fully differentiable and compatible with existing training pipelines. Beyond compression, the spectral decomposition introduces an inductive bias over embedding frequencies, enabling slice-dependent frequency scaling that improves generalization. Experiments on IMDB and AG~News show that the proposed model can substantially reduce encoder parameters (up to 75\% for p=4) while maintaining competitive accuracy. On IMDB, the tensorized encoder matches or improves upon the standard baseline under compression, whereas on AG~News at moderate width we observe a small accuracy decrease in exchange for a 4 times encoder reduction; at BERT-base width (d=768), performance returns to parity.

cs.CL↗

arXiv2026

Multidimensional Task Learning: A Unified Tensor Framework for Computer Vision Tasks

This paper introduces Multidimensional Task Learning (MTL), a unified mathematical framework based on Generalized Einstein MLPs (GE-MLPs) that operate directly on tensors via the Einstein product. We argue that current computer vision task formulations are inherently constrained by matrix-based thinking: standard architectures rely on matrix-valued weights and vectorvalued biases, requiring structural flattening that restricts the space of naturally expressible tasks. GE-MLPs lift this constraint by operating with tensor-valued parameters, enabling explicit control over which dimensions are preserved or contracted without information loss. Through rigorous mathematical derivations, we demonstrate that classification, segmentation, and detection are special cases of MTL, differing only in their dimensional configuration within a formally defined task space. We further prove that this task space is strictly larger than what matrix-based formulations can natively express, enabling principled task configurations such as spatiotemporal or cross modal predictions that require destructive flattening under conventional approaches. This work provides a mathematical foundation for understanding, comparing, and designing computer vision tasks through the lens of tensor algebra.

cs.CV↗

arXiv2026

A Computationally Efficient Multidimensional Vision Transformer

Vision Transformers have achieved state-of-the-art performance in a wide range of computer vision tasks, but their practical deployment is limited by high computational and memory costs. In this paper, we introduce a novel tensor-based framework for Vision Transformers built upon the Tensor Cosine Product (Cproduct). By exploiting multilinear structures inherent in image data and the orthogonality of cosine transforms, the proposed approach enables efficient attention mechanisms and structured feature representations. We develop the theoretical foundations of the tensor cosine product, analyze its algebraic properties, and integrate it into a new Cproduct-based Vision Transformer architecture (TCP-ViT). Numerical experiments on standard classification and segmentation benchmarks demonstrate that the proposed method achieves a uniform 1/C parameter reduction (where C is the number of channels) while maintaining competitive accuracy.

cs.LG↗

arXiv2026

Dimensionality Reduction on Riemannian Manifolds in Data Analysis

In this work, we investigate Riemannian geometry based dimensionality reduction methods that respect the underlying manifold structure of the data. In particular, we focus on Principal Geodesic Analysis (PGA) as a nonlinear generalization of PCA for manifold valued data, and extend discriminant analysis through Riemannian adaptations of other known dimensionality reduction methods. These approaches exploit geodesic distances, tangent space representations, and intrinsic statistical measures to achieve more faithful low dimensional embeddings. We also discuss related manifold learning techniques and highlight their theoretical foundations and practical advantages. Experimental results on representative datasets demonstrate that Riemannian methods provide improved representation quality and classification performance compared to their Euclidean counterparts, especially for data constrained to curved spaces such as hyperspheres and symmetric positive definite manifolds. This study underscores the importance of geometry aware dimensionality reduction in modern machine learning and data science applications.

cs.LG↗

arXiv2026

A survey of scalar and vector extrapolation

Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed sequences that converge more rapidly to the same limit without altering the underlying iterative process, thereby reducing computational cost and enhancing numerical accuracy. Historically, the origins of such techniques can be traced back to classical algebraic methods by AlKhwarizmi and early series acceleration techniques by Newton, while systematic approaches emerged in the 20th century with Aitken process and Richardson extrapolation. Later developments, including the Shanks transformation and Wynn epsilon algorithm, provided general frameworks capable of eliminating multiple dominant error components, with deep connections to Pade approximants and rational approximations of generating functions. This paper presents a comprehensive review of classical scalar extrapolation methods, including Richardson extrapolation, Aitken process, Shanks transformation, Wynn epsilon algorithm, and other algorithms. We examine their theoretical foundations, asymptotic error models, convergence properties, numerical stability, and practical implementation considerations. The second part of this work is dedicated to vector extrapolation methods: polynomial based ones and epsilon algorithm generalizations to vector sequences. Additionally, we highlight modern developments such as their applications to iterative solvers, Krylov subspace methods, and large-scale computational simulations. The aim of this review is to provide a unified perspective on scalar and vector extrapolation techniques, bridging historical origins, theoretical insights, and contemporary computational applications.

math.NA↗

arXiv2026

A Unified Trace-Optimization Framework for Multidimensionality Reduction

This paper presents a comprehensive overview of several multidimensional reduction methods focusing on Multidimensional Principal Component Analysis (MPCA), Multilinear Orthogonal Neighborhood Preserving Projection (MONPP), Multidimensional Locally Linear Embedding (MLLE), and Multidimensional Laplacian Eigenmaps (MLE). These techniques are formulated within a unified framework based on trace optimization, where the dimensionality reduction problem is expressed as maximization or minimization problems. In addition to the linear MPCA and MONPP approaches, kernel-based extensions of these methods also are presented. The latter methods make it possible to capture nonlinear relations between high-dimensional data. A comparative analysis highlights the theoretical foundations, assumptions, and computational efficiency of each method, as well as their practical applicability. The study provides insights and guidelines for selecting an appropriate dimensionality reduction technique suited to the application at hand.

math.NA↗

arXiv2025

Efficient Krylov-Regularization Solvers for Multiquadric RBF Discretizations of the 3D Helmholtz Equation

Meshless collocation with multiquadric radial basis functions (MQ-RBFs) delivers high accuracy for the three-dimensional Helmholtz equation but produces dense, severely ill-conditioned linear systems. We develop and evaluate three complementary methods that embed regularization in Krylov projections to overcome this instability at scale: (i) an inexpensive TSVD that replaces the full SVD by a short Golub-Kahan bidiagonalization and a small projected SVD, retaining the dominant spectral content at greatly reduced cost; (ii) classical Tikhonov regularization with principled parameter choice (GCV/L-curve), expressed in SVD form for transparent filtering; and (iii) a hybrid Krylov-Tikhonov (HKT) scheme that first projects with Golub-Kahan and then selects the regularization parameter on the reduced problem, yielding stable solutions in few iterations. Extensive tests on canonical domains (cube and sphere) and a realistic industrial pump-casing geometry demonstrate that HKT consistently matches or surpasses the accuracy of full TSVD/Tikhonov at a fraction of the runtime and memory, while inexpensive TSVD provides the fastest viable reconstructions when only the leading modes are needed. These results show that coupling Krylov projection with TSVD/Tikhonov regularization provides a robust, scalable pathway for MQ-RBF Helmholtz methods in complex three-dimensional settings.

math.NA↗

arXiv2025

Low-Rank Regularized Convex-Non-Convex Problems for Image Segmentation or Completion

This work proposes a novel convex-non-convex formulation of the image segmentation and the image completion problems. The proposed approach is based on the minimization of a functional involving two distinct regularization terms: one promotes low-rank structure in the solution, while the other one enforces smoothness. To solve the resulting optimization problem, we employ the alternating direction method of multipliers (ADMM). A detailed convergence analysis of the algorithm is provided, and the performance of the methods is demonstrated through a series of numerical experiments.

math.NA↗

arXiv2025

Parameter-Efficient Fine-Tuning of Multispectral Foundation Models for Hyperspectral Image Classification

Foundation models have achieved great success across diverse domains, including remote sensing (RS), thanks to their versatility and strong generalization abilities. However, most RS foundation models are designed for multispectral data, while hyperspectral imagery (HSI) - with its hundreds of spectral bands - remains less explored. Fine-tuning such models for downstream tasks is also challenging, often demanding considerable memory and storage. In this paper, we propose an efficient framework to fine-tune SpectralGPT, a multispectral foundation model, for hyperspectral image classification (HSIC). We explore several Parameter-Efficient Fine-Tuning (PEFT) methods, including Low-Rank Adaptation (LoRA), Kronecker-based adaptation (KronA), Low-Rank Kronecker (LoKr), and the recent LoRA+, which uses distinct learning rates for low-rank adapters scaled by a factor lambda. Inspired by LoRA+, we introduce KronA+, which applies a similar mechanism to the Kronecker matrices. We evaluate our approach on five datasets from different sensors, showing competitive performance with state-of-the-art HSI models. Our full fine-tuning (FFT) setup for SpectralGPT even outperforms a dedicated hyperspectral foundation model on some datasets while requiring only a quarter of the training epochs. Under the same number of epochs, KronA+ reaches similar performance with far fewer trainable parameters - just 0.056 percent - and adds only approximately 0.2 megabytes of storage, making it the most effective PEFT method tested.

cs.CV↗

arXiv2025

Online multidimensional dictionary learning

Dictionary learning is a widely used technique in signal processing and machine learning that aims to represent data as a linear combination of a few elements from an overcomplete dictionary. In this work, we propose a generalization of the dictionary learning technique using the t-product framework, enabling efficient handling of multidimensional tensor data. We address the dictionary learning problem through online methods suitable for tensor structures. To effectively address the sparsity problem, we utilize an accelerated Iterative Shrinkage-Thresholding Algorithm (ISTA) enhanced with an extrapolation technique known as Anderson acceleration. This approach significantly improves signal reconstruction results. Extensive experiments prove that our proposed method outperforms existing acceleration techniques, particularly in applications such as data completion. These results suggest that our approach can be highly beneficial for large-scale tensor data analysis in various domains.

math.NA↗

arXiv2025

Trace Ratio vs Ratio Trace Methods for Multidimensional Dimensionality Reduction

We propose a higher-order dimensionality reduction framework based on the Trace Ratio (TR) optimization problem. We establish conditions for existence and uniqueness of solutions and clarify the theoretical connection between the Trace Ratio and its surrogate, the Ratio Trace (RT) formulation. Building on these foundations, we design a Newton-type iterative algorithm that operates directly in the tensor domain via the Einstein product, avoiding data flattening and preserving multi-dimensional structure. This approach extends classical Linear Discriminant Analysis (LDA) to higher-order tensors, offering a natural generalization of trace-based dimensionality reduction from matrices to tensors. Numerical experiments on several benchmark datasets confirm the efficiency and robustness of the proposed methods, showing consistent improvements over existing matrix- and tensor-based techniques.

math.NA↗

arXiv2024

A rational Krylov methods for large scale linear multidimensional dynamical systems

In this paper, we investigate the use of multilinear algebra for reducing the order of multidimensional linear time-invariant (MLTI) systems. Our main tools are tensor rational Krylov subspace methods, which enable us to approximate the systems solution within a low-dimensional subspace. We introduce the tensor rational block Arnoldi and tensor rational block Lanczos algorithms. By utilizing these methods, we develop a model reduction approach based on projection techniques. Additionally, we demonstrate how these approaches can be applied to large-scale Lyapunov tensor equations, which are critical for the balanced truncation method, a well-known technique for order reduction. An adaptive method for choosing the interpolation points is also introduced. Finally, some numerical experiments are reported to show the effectiveness of the proposed adaptive approaches.

math.NA↗

arXiv2024

Quaternion tensor low rank Quaternion tensor low-rank approximation using a family of non-convex norms

In this paper, we propose a new approaches for low rank approximation of quaternion tensors \cite{chen2019low,zhang1997quaternions,hamilton1866elements}. The first method uses quasi-norms to approximate the tensor by a low-rank tensor using the QT-product \cite{miao2023quaternion}, which generalizes the known L-product to N-mode quaternions. The second method involves Non-Convex norms to approximate the Tucker and TT-rank for the completion problem. We demonstrate that the proposed methods can effectively approximate the tensor compared to the convexifying of the rank, such as the nuclear norm. We provide theoretical results and numerical experiments to show the efficiency of the proposed methods in the Inpainting and Denoising applications.

math.NA↗

arXiv2024

Non-negative Einstein tensor factorization for unmixing hyperspectral images

In this manuscript, we introduce a tensor-based approach to Non-Negative Tensor Factorization (NTF). The method entails tensor dimension reduction through the utilization of the Einstein product. To maintain the regularity and sparsity of the data, certain constraints are imposed. Additionally, we present an optimization algorithm in the form of a tensor multiplicative updates method, which relies on the Einstein product. To guarantee a minimum number of iterations for the convergence of the proposed algorithm, we employ the Reduced Rank Extrapolation (RRE) and the Topological Extrapolation Transformation Algorithm (TEA). The efficacy of the proposed model is demonstrated through tests conducted on Hyperspectral Images (HI) for denoising, as well as for Hyperspectral Image Linear Unmixing. Numerical experiments are provided to substantiate the effectiveness of the proposed model for both synthetic and real data.

math.NA↗

arXiv2024

Higher order multi-dimension reduction methods via Einstein product

This paper explores the extension of dimension reduction (DR) techniques to the multi-dimension case by using the Einstein product. Our focus lies on graph-based methods, encompassing both linear and nonlinear approaches, within both supervised and unsupervised learning paradigms. Additionally, we investigate variants such as repulsion graphs and kernel methods for linear approaches. Furthermore, we present two generalizations for each method, based on single or multiple weights. We demonstrate the straightforward nature of these generalizations and provide theoretical insights. Numerical experiments are conducted, and results are compared with original methods, highlighting the efficiency of our proposed methods, particularly in handling high-dimensional data such as color images.

math.NA↗

arXiv2024

Multidimensional extrapolated global proximal gradient and applications for image processing

The proximal gradient method is a generic technique introduced to tackle the non-smoothness in optimization problems, wherein the objective function is expressed as the sum of a differentiable convex part and a non-differentiable regularization term. Such problems with tensor format are of interest in many fields of applied mathematics such as image and video processing. Our goal in this paper is to address the solution of such problems with a more general form of the regularization term. An adapted iterative proximal gradient method is introduced for this purpose. Due to the slowness of the proposed algorithm, we use new tensor extrapolation methods to enhance its convergence. Numerical experiments on color image deblurring are conducted to illustrate the efficiency of our approach.

math.NA↗

arXiv2023

A low-rank non-convex norm method for multiview graph clustering

This study introduces a novel technique for multi-view clustering known as the "Consensus Graph-Based Multi-View Clustering Method Using Low-Rank Non-Convex Norm" (CGMVC-NC). Multi-view clustering is a challenging task in machine learning as it requires the integration of information from multiple data sources or views to cluster data points accurately. The suggested approach makes use of the structural characteristics of multi-view data tensors, introducing a non-convex tensor norm to identify correlations between these views. In contrast to conventional methods, this approach demonstrates superior clustering accuracy across several benchmark datasets. Despite the non-convex nature of the tensor norm used, the proposed method remains amenable to efficient optimization using existing algorithms. The approach provides a valuable tool for multi-view data analysis and has the potential to enhance our understanding of complex systems in various fields. Further research can explore the application of this method to other types of data and extend it to other machine-learning tasks.

cs.LG↗