arXiv subjects

K. Jbilou

Publications and source records attributed to K. Jbilou.

At least 19 recordsLinked to original sources

arXiv2026

Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language tasks but suffer from extremely large memory footprints and computational costs. In this paper, we introduce a tensor compression framework based on the cproduct for computing low rank approximation In the first part of our approach, we leverage the algebraic structure of the cproduct to represent weight tensors such as those in embedding layers, attention projections, and feed forward networks in a transform domain where frontal slices can be jointly approximated by low rank tensor factors. This enables computationally efficient compression that exploits multidimensional correlations beyond traditional SVD methods.

cs.LG↗

arXiv2025

Efficient Computation of Dominant Eigenvalues Using Adaptive Block Lanczos with Chebyshev Filtering

We present an efficient method for computing dominant eigenvalues of large, nonsymmetric, diagonalizable matrices based on an adaptive block Lanczos algorithm combined with Chebyshev polynomial filtering. The proposed approach improves numerical stability through two key components: (i) the Adaptive Block Lanczos (ABLE) method, which maintains biorthogonality using SVD based stabilization, and (ii) Chebyshev filtering, which enhances spectral separation via iterative polynomial filtering. Numerical experiments on dense and sparse test problems confirm the effectiveness of the ABLE Chebyshev algorithm.

math.NA↗

arXiv2024

Extended block Hessenberg process for the evaluation of matrix functions

In the present paper, we propose a block variant of the extended Hessenberg process for computing approximations of matrix functions and other problems producing large-scale matrices. Applications to the computation of a matrix function such as f(A)V, where A is an nxn large sparse matrix, V is an nxp block with p<<n, and f is a function are presented. Solving shifted linear systems with multiple right hand sides are also given. Computing approximations of these matrix problems appear in many scientific and engineering applications. Different numerical experiments are provided to show the effectiveness of the proposed method for these problems.

math.NA↗

arXiv2023

Einstien-Multidimensional Extrapolation methods

In this paper, we present a new framework for the recent multidimensional extrapolation methods: Tensor Global Minimal Polynomial (TG-MPE) and Tensor Global Reduced Rank Extrapolation (TG-RRE) methods. We develop a new approach to the one presented in \cite{17}. The proposed framework highlights, in addition their polynomial feature, the connection of TG-MPE and TG-RRE with nonlinear Krylov subspace methods. A unified algorithm is proposed for their implemention. Theoretical results are given and some numerical experiments on linear and nonlinear problems are considered to confirm the performance of the proposed algorithms.

math.NA↗

arXiv2023

A Rational Krylov Subspace Method for the Computation of the Matrix Exponential Operator

The computation of approximating e^tA B, where A is a large sparse matrix and B is a rectangular matrix, serves as a crucial element in numerous scientific and engineering calculations. A powerful way to consider this problem is to use Krylov subspace methods. The purpose of this work is to approximate the matrix exponential and some Cauchy-Stieltjes functions on a block vectors B of R^n*p using a rational block Lanczos algorithm. We also derive some error estimates and error bound for the convergence of the rational approximation and finally numerical results attest to the computational efficiency of the proposed method.

math.NA↗

arXiv2023

A model reduction method for large-scale linear multidimensional dynamical systems

In this work, we explore the application of multilinear algebra in reducing the order of multidimentional linear time-invariant (MLTI) systems. We use tensor Krylov subspace methods as key tools, which involve approximating the system solution within a low-dimensional subspace. We introduce the tensor extended block and global Krylov subspaces and the corresponding Arnoldi based processes. Using these methods, we develop a model reduction using projection techniques. We also show how these methods could be used to solve large-scale Lyapunov tensor equations that are needed in the balanced truncation method which is a technique for order reduction. We demonstrate how to extract approximate solutions via the Einstein product using the tensor extended block Arnoldi and the extended global Arnoldi processes.

math.NA↗

arXiv2022

Multilinear Discriminant Analysis using a new family of tensor-tensor products

Multilinear Discriminant Analysis (MDA) is a powerful dimension reduction method specifically formulated to deal with tensor data. Precisely, the goal of MDA is to find mode-specific projections that optimally separate tensor data from different classes. However, to solve this task, standard MDA methods use alternating optimization heuristics involving the computation of a succession of tensor-matrix products. Such approaches are most of the time difficult to solve and not natural, highligthing the difficulty to formulate this problem in fully tensor form. In this paper, we propose to solve multilinear discriminant analysis (MDA) by using the concept of transform domain (TD) recently proposed in \cite{Kilmer2011}. We show here that moving MDA to this specific transform domain make its resolution easier and more natural. More precisely, each frontal face of the transformed tensor is processed independently to build a separate optimization sub-problems easier to solve. Next, the obtained solutions are converted into projective tensors by inverse transform. By considering a large number of experiments, we show the effectiveness of our approach with respect to existing MDA methods.

math.NA↗

arXiv2021

Discrete cosine transform LSQR and GMRES methods for multidimensional ill-posed problems

In the present work, we propose new tensor Krylov subspace method for ill posed linear tensor problems such as in color or video image restoration. Those methods are based on the tensor-tensor discrete cosine transform that gives fast tensor-tensor product computations. In particular, we will focus on the tensor discrete cosine versions of GMRES, Golub-Kahan bidiagonalisation and LSQR methods. The presented numerical tests show that the methods are very fast and give good accuracies when solving some linear tensor ill-posed problems.

math.NA↗

arXiv2021

RBF approximation of three dimensional PDEs using Tensor Krylov subspace methods

In this paper, we propose different algorithms for the solution of a tensor linear discrete ill-posed problem arising in the application of the meshless method for solving PDEs in three-dimensional space using multiquadric radial basis functions. It is well known that the truncated singular value decomposition (TSVD) is the most common effective solver for ill-conditioned systems, but unfortunately the operation count for solving a linear system with the TSVD is computationally expensive for large-scale matrices. In the present work, we propose algorithms based on the use of the well known Einstein product for two tensors to define the tensor global Arnoldi and the tensor Gloub Kahan bidiagonalization algorithms. Using the so-called Tikhonov regularization technique, we will be able to provide computable approximate regularized solutions in a few iterations.

math.NA↗

arXiv2021

On some tensor tubal-Krylov subspace methods via the T-product

In the present paper, we introduce new tensor Krylov subspace methods for solving linear tensor equations. The proposed methods use the well known T-product for tensors and tensor subspaces related to tube fibers. We introduce some new tensor products and the related algebraic properties. These new products will enable us to develop third-order the tensor tubal GMRES and the tensor tubal Golub Kahan methods. We give some properties related to these methods and proopse some numerical experiments.

math.NA↗

arXiv2021

On the tensor nuclear norm and the total variation regularization for image and video completion

In the present paper we propose two new algorithms of tensor completion for three-order tensors. The proposed methods consist in minimizing the average rank of the underlying tensor using its approximate function namely the tensor nuclear norm and then the recovered data will be obtained by using the total variation regularisation technique. We will adopt the Alternating Direction Method of Multipliers (ADM), using the tensor T-product, to solve the main optimization problems associated to the two algorithms. In the last section, we present some numerical experiments and comparisons with the most known image completion methods.

math.NA↗

arXiv2020

Tensor Krylov subspace methods via the T-product for color image processing

The present paper is concerned with developing tensor iterative Krylov subspace methods to solve large multi-linear tensor equations. We use the well-known T-product for two tensors to define tensor global Arnoldi and tensor global Gloub-Kahan bidiagonalization algorithms. Furthermore, we illustrate how tensor-based global approaches can be exploited to solve ill-posed problems arising from recovering blurry multichannel (color) images and videos, using the so-called Tikhonov regularization technique, to provide computable approximate regularized solutions. We also review a generalized cross-validation and discrepancy principle type of criterion for the selection of the regularization parameter in the Tikhonov regularization. Applications to RGB image and video processing are given to demonstrate the efficiency of the algorithms.

math.NA↗

arXiv2020

Tensor GMRES and Golub-Kahan Bidiagonalization methods via the Einstein product with applications to image and video processing

In the present paper, we are interested in developing iterative Krylov subspace methods in tensor structure to solve a class of multilinear systems via Einstein product. In particular, we develop global variants of the GMRES and Gloub--Kahan bidiagonalization processes in tensor framework. We further consider the case that mentioned equation may be possibly corresponds to a discrete ill-posed problem. Applications arising from color image and video restoration are included.

math.NA↗

arXiv2020

Tensor extrapolation methods with applications

In this paper, we mainly develop the well-known vector and matrix polynomial extrapolation methods in tensor framework. To this end, some new products between tensors are defined and the concept of positive definitiveness is extended for tensors corresponding to T-product. Furthermore, we discuss on the solution of least-squares problem associated with a tensor equation using Tensor Singular Value Decomposition (TSVD). Motivated by the effectiveness of proposed vector extrapolation method in [Numer. Algorithms, 51 (2009), 195--208], we describe how an extrapolation technique can be also implemented on the sequence of tensors produced by truncated TSVD (TTSVD) for solving possibly ill-posed tensor equations.

math.NA↗

arXiv2020

The global extended-rational Arnoldi method for matrix function approximation

The numerical computation of matrix functions such as $f(A)V$, where $A$ is an $n\times n$ large and sparse square matrix, $V$ is an $n \times p$ block with $p\ll n$ and $f$ is a nonlinear matrix function, arises in various applications such as network analysis ($f(t)=exp(t)$ or $f(t)=t^3)$, machine learning $(f(t)=log(t))$, theory of quantum chromodynamics $(f(t)=t^{1/2})$, electronic structure computation, and others. In this work, we propose the use of global extended-rational Arnoldi method for computing approximations of such expressions. The derived method projects the initial problem onto an global extended-rational Krylov subspace $\mathcal{RK}^{e}_m(A,V)=\text{span}(\{\prod\limits_{i=1}^m(A-s_iI_n)^{-1}V,\ldots,(A-s_1I_n)^{-1}V,V$ $,AV, \ldots,A^{m-1}V\})$ of a low dimension. An adaptive procedure for the selection of shift parameters $\{s_1,\ldots,s_m\}$ is given. The proposed method is also applied to solve parameter dependent systems. Numerical examples are presented to show the performance of the global extended-rational Arnoldi for these problems.

math.NA↗

arXiv2018

Numerical methods for differential linear matrix equations via Krylov subspace methods

In the present paper, we present some numerical methods for computing approximate solutions to some large differential linear matrix equations. In the first part of this work, we deal with differential generalized Sylvester matrix equations with full rank right-hand sides using a global Galerkin and a norm-minimization approaches. In the second part, we consider large differential Lyapunov matrix equations with low rank right-hand sides and use the extended global Arnoldi process to produce low rank approximate solutions. We give some theoretical results and present some numerical experiments.

math.NA↗

arXiv2018

A generalized matrix Krylov subspace method for TV regularization

This paper presents an efficient algorithm to solve total variation (TV) regularizations of images contaminated by a both blur and noise. The unconstrained structure of the problem suggests that one can solve a constrained optimization problem by transforming the original unconstrained minimization problem to an equivalent constrained minimization one. An augmented Lagrangian method is developed to handle the constraints when the model is given with matrix variables, and an alternating direction method (ADM) is used to iteratively find solutions. The solutions of some sub-problems are belonging to subspaces generated by application of successive orthogonal projections onto a class of generalized matrix Krylov subspaces of increasing dimension.

math.NA↗

arXiv2018

Approximate solutions to large nonsymmetric differential Riccati problems with applications to transport theory

In the present paper, we consider large scale nonsymmetric differential matrix Riccati equations with low rank right hand sides. These matrix equations appear in many applications such as control theory, transport theory, applied probability and others. We show how to apply Krylov-type methods such as the extended block Arnoldi algorithm to get low rank approximate solutions. The initial problem is projected onto small subspaces to get low dimensional nonsymmetric differential equations that are solved using the exponential approximation or via other integration schemes such as Backward Differentiation Formula (BDF) or Rosenbrok method. We also show how these technique could be easily used to solve some problems from the well known transport equation. Some numerical experiments are given to illustrate the application of the proposed methods to large-scale problems

math.NA↗