Searcharxiv⌕ Search

arXiv subjects

Julien Langou

Publications and source records attributed to Julien Langou.

At least 37 records · Page 2Linked to original sources
arXiv2014

On matrix balancing and eigenvector computation

Balancing a matrix is a preprocessing step while solving the nonsymmetric eigenvalue problem. Balancing a matrix reduces the norm of the matrix and hopefully this will improve the accuracy of the computation. Experiments have shown that balancing can improve the accuracy of the computed eigenval- ues. However, there exists examples where balancing increases the eigenvalue condition number (potential loss in accuracy), deteriorates eigenvector accuracy, and deteriorates the backward error of the eigenvalue decomposition. In this paper we propose a change to the stopping criteria of the LAPACK balancing al- gorithm, GEBAL. The new stopping criteria is better at determining when a matrix is nearly balanced. Our experiments show that the new algorithm is able to maintain good backward error, while improving the eigenvalue accuracy when possible. We present stability analysis, numerical experiments, and a case study to demonstrate the benefit of the new stopping criteria.

math.NA↗
arXiv2014

Stability Analysis of QR factorization in an Oblique Inner Product

In this paper we consider the stability of the QR factorization in an oblique inner product. The oblique inner product is defined by a symmetric positive definite matrix A. We analyze two algorithm that are based a factorization of A and converting the problem to the Euclidean case. The two algorithms we consider use the Cholesky decomposition and the eigenvalue decomposition. We also analyze algorithms that are based on computing the Cholesky factor of the normal equa- tion. We present numerical experiments to show the error bounds are tight. Finally we present performance results for these algorithms as well as Gram-Schmidt methods on parallel architecture. The performance experiments demonstrate the benefit of the communication avoiding algorithms.

math.NA↗
arXiv2014

Designing LU-QR hybrid solvers for performance and stability

This paper introduces hybrid LU-QR al- gorithms for solving dense linear systems of the form Ax = b. Throughout a matrix factorization, these al- gorithms dynamically alternate LU with local pivoting and QR elimination steps, based upon some robustness criterion. LU elimination steps can be very efficiently parallelized, and are twice as cheap in terms of floating- point operations, as QR steps. However, LU steps are not necessarily stable, while QR steps are always stable. The hybrid algorithms execute a QR step when a robustness criterion detects some risk for instability, and they execute an LU step otherwise. Ideally, the choice between LU and QR steps must have a small computational overhead and must provide a satisfactory level of stability with as few QR steps as possible. In this paper, we introduce several robustness criteria and we establish upper bounds on the growth factor of the norm of the updated matrix incurred by each of these criteria. In addition, we describe the implementation of the hybrid algorithms through an exten- sion of the PaRSEC software to allow for dynamic choices during execution. Finally, we analyze both stability and performance results compared to state-of-the-art linear solvers on parallel distributed multicore platforms.

math.NA↗
arXiv2013

A Greedy Algorithm for Optimally Pipelining a Reduction

Collective communications are ubiquitous in parallel applications. We present two new algorithms for performing a reduction. The operation associated with our reduction needs to be associative and commutative. The two algorithms are developed under two different communication models (unidirectional and bidirectional). Both algorithms use a greedy scheduling scheme. For a unidirectional, fully connected network, we prove that our greedy algorithm is optimal when some realistic assumptions are respected. Previous algorithms fit the same assumptions and are only appropriate for some given configurations. Our algorithm is optimal for all configurations. We note that there are some configuration where our greedy algorithm significantly outperform any existing algorithms. This result represents a contribution to the state-of-the art. For a bidirectional, fully connected network, we present a different greedy algorithm. We verify by experimental simulations that our algorithm matches the time complexity of an optimal broadcast (with addition of the computation). Beside reversing an optimal broadcast algorithm, the greedy algorithm is the first known reduction algorithm to experimentally attain this time complexity. Simulations show that this greedy algorithm performs well in practice, outperforming any state-of-the-art reduction algorithms. Positive experiments on a parallel distributed machine are also presented.

cs.DC↗
arXiv2011

Hierarchical QR factorization algorithms for multi-core cluster systems

This paper describes a new QR factorization algorithm which is especially designed for massively parallel platforms combining parallel distributed multi-core nodes. These platforms make the present and the foreseeable future of high-performance computing. Our new QR factorization algorithm falls in the category of the tile algorithms which naturally enables good data locality for the sequential kernels executed by the cores (high sequential performance), low number of messages in a parallel distributed setting (small latency term), and fine granularity (high parallelism).

cs.DC↗
arXiv2011

Tiled QR factorization algorithms

This work revisits existing algorithms for the QR factorization of rectangular matrices composed of p-by-q tiles, where p >= q. Within this framework, we study the critical paths and performance of algorithms such as Sameh and Kuck, Modi and Clarke, Greedy, and those found within PLASMA. Although neither Modi and Clarke nor Greedy is optimal, both are shown to be asymptotically optimal for all matrices of size p = q^2 f(q), where f is any function such that \lim_{+\infty} f= 0. This novel and important complexity result applies to all matrices where p and q are proportional, p = λq, with λ>= 1, thereby encompassing many important situations in practice (least squares). We provide an extensive set of experiments that show the superiority of the new algorithms for tall matrices.

cs.DC↗
arXiv2010

A Critical Path Approach to Analyzing Parallelism of Algorithmic Variants. Application to Cholesky Inversion

Algorithms come with multiple variants which are obtained by changing the mathematical approach from which the algorithm is derived. These variants offer a wide spectrum of performance when implemented on a multicore platform and we seek to understand these differences in performances from a theoretical point of view. To that aim, we derive and present the critical path lengths of each algorithmic variant for our application problem which enables us to determine a lower bound on the time to solution. This metric provides an intuitive grasp of the performance of a variant and we present numerical experiments to validate the tightness of our lower bounds on practical applications. Our case study is the Cholesky inversion and its use in computing the inverse of a symmetric positive definite matrix.

cs.DC↗
arXiv2010

Computing the R of the QR factorization of tall and skinny matrices using MPI_Reduce

A QR factorization of a tall and skinny matrix with n columns can be represented as a reduction. The operation used along the reduction tree has in input two n-by-n upper triangular matrices and in output an n-by-n upper triangular matrix which is defined as the R factor of the two input matrices stacked the one on top of the other. This operation is binary, associative, and commutative. We can therefore leverage the MPI library capabilities by using user-defined MPI operations and MPI_Reduce to perform this reduction. The resulting code is compact and portable. In this context, the user relies on the MPI library to select a reduction tree appropriate for the underlying architecture.

math.NA↗
arXiv2010

Towards an Efficient Tile Matrix Inversion of Symmetric Positive Definite Matrices on Multicore Architectures

The algorithms in the current sequential numerical linear algebra libraries (e.g. LAPACK) do not parallelize well on multicore architectures. A new family of algorithms, the tile algorithms, has recently been introduced. Previous research has shown that it is possible to write efficient and scalable tile algorithms for performing a Cholesky factorization, a (pseudo) LU factorization, and a QR factorization. In this extended abstract, we attack the problem of the computation of the inverse of a symmetric positive definite matrix. We observe that, using a dynamic task scheduler, it is relatively painless to translate existing LAPACK code to obtain a ready-to-be-executed tile algorithm. However we demonstrate that non trivial compiler techniques (array renaming, loop reversal and pipelining) need then to be applied to further increase the parallelism of our application. We present preliminary experimental results.

cs.MS↗
arXiv2009

QR Factorization of Tall and Skinny Matrices in a Grid Computing Environment

Previous studies have reported that common dense linear algebra operations do not achieve speed up by using multiple geographical sites of a computational grid. Because such operations are the building blocks of most scientific applications, conventional supercomputers are still strongly predominant in high-performance computing and the use of grids for speeding up large-scale scientific problems is limited to applications exhibiting parallelism at a higher level. We have identified two performance bottlenecks in the distributed memory algorithms implemented in ScaLAPACK, a state-of-the-art dense linear algebra library. First, because ScaLAPACK assumes a homogeneous communication network, the implementations of ScaLAPACK algorithms lack locality in their communication pattern. Second, the number of messages sent in the ScaLAPACK algorithms is significantly greater than other algorithms that trade flops for communication. In this paper, we present a new approach for computing a QR factorization -- one of the main dense linear algebra kernels -- of tall and skinny matrices in a grid computing environment that overcomes these two bottlenecks. Our contribution is to articulate a recently proposed algorithm (Communication Avoiding QR) with a topology-aware middleware (QCG-OMPI) in order to confine intensive communications (ScaLAPACK calls) within the different geographical sites. An experimental study conducted on the Grid'5000 platform shows that the resulting performance increases linearly with the number of geographical sites on large-scale problems (and is in particular consistently higher than ScaLAPACK's).

cs.DC↗
arXiv2009

Translation and modern interpretation of Laplace's Théorie Analytique des Probabilités, pages 505-512, 516-520

The text of Laplace, \textit{Sur l'application du calcul des probabilités à la philosophie naturelle,} (Théorie Analytique des Probabilités. Troisième Édition. Premier Supplément), 1820, is quoted in the context of the Gram-Schmidt algorithm. We provide an English translation of Laplace's manuscript (originally in French) and interpret the algorithms of Laplace in a contemporary context. The two algorithms given by Laplace computes the mean and the variance of two components of the solution of a linear statistical model. The first algorithm can be interpreted as {\em reverse square-root-free modified Gram-Schmidt by row} algorithm on the regression matrix. The second algorithm can be interpreted as the {\em reverse square-root-free Cholesky} algorithm.

math.NA↗
arXiv2009

Any decreasing cycle-convergence curve is possible for restarted GMRES

Given a matrix order $n$, a restart parameter $m$ ($m < n$), a decreasing positive sequence $f(0) > f(1) > ... > f(q) \geq 0$, where $q < n/m$, it is shown that there exits an $n$-by-$n$ matrix $A$ and a vector $r_0$ with $\|r_0\|=f(0)$ such that $\|r_k\|=f(k)$, $k=1,...,q$, where $r_k$ is the residual at cycle $k$ of restarted GMRES with restart parameter $m$ applied to the linear system $Ax=b$, with initial residual $r_0=b-Ax_0$. Moreover, the matrix $A$ can be chosen to have any desired eigenvalues. We can also construct arbitrary cases of stagnation; namely, when $f(0) > f(1) > ... > f(i) = f(i+1) \geq 0 $ for any $i <q $. The restart parameter can be fixed or variable.

math.NA↗
arXiv2009

Rectangular Full Packed Format for Cholesky's Algorithm: Factorization, Solution and Inversion

We describe a new data format for storing triangular, symmetric, and Hermitian matrices called RFPF (Rectangular Full Packed Format). The standard two dimensional arrays of Fortran and C (also known as full format) that are used to represent triangular and symmetric matrices waste nearly half of the storage space but provide high performance via the use of Level 3 BLAS. Standard packed format arrays fully utilize storage (array space) but provide low performance as there is no Level 3 packed BLAS. We combine the good features of packed and full storage using RFPF to obtain high performance via using Level 3 BLAS as RFPF is a standard full format representation. Also, RFPF requires exactly the same minimal storage as packed format. Each LAPACK full and/or packed triangular, symmetric, and Hermitian routine becomes a single new RFPF routine based on eight possible data layouts of RFPF. This new RFPF routine usually consists of two calls to the corresponding LAPACK full format routine and two calls to Level 3 BLAS routines. This means {\it no} new software is required. As examples, we present LAPACK routines for Cholesky factorization, Cholesky solution and Cholesky inverse computation in RFPF to illustrate this new work and to describe its performance on several commonly used computer platforms. Performance of LAPACK full routines using RFPF versus LAPACK full routines using standard format for both serial and SMP parallel processing is about the same while using half the storage. Performance gains are roughly one to a factor of 43 for serial and one to a factor of 97 for SMP parallel times faster using vendor LAPACK full routines with RFPF than with using vendor and/or reference packed routines.

cs.MS↗
arXiv2008

Implementing Communication-Optimal Parallel and Sequential QR Factorizations

We present parallel and sequential dense QR factorization algorithms for tall and skinny matrices and general rectangular matrices that both minimize communication, and are as stable as Householder QR. The sequential and parallel algorithms for tall and skinny matrices lead to significant speedups in practice over some of the existing algorithms, including LAPACK and ScaLAPACK, for example up to 6.7x over ScaLAPACK. The parallel algorithm for general rectangular matrices is estimated to show significant speedups over ScaLAPACK, up to 22x over ScaLAPACK.

math.NA↗
arXiv2008

Communication-optimal parallel and sequential QR and LU factorizations: theory and practice

We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform, and just as stable as Householder QR. Our first algorithm, Tall Skinny QR (TSQR), factors m-by-n matrices in a one-dimensional (1-D) block cyclic row layout, and is optimized for m >> n. Our second algorithm, CAQR (Communication-Avoiding QR), factors general rectangular matrices distributed in a two-dimensional block cyclic layout. It invokes TSQR for each block column factorization.

math.NA↗
arXiv2008

Accelerating Scientific Computations with Mixed Precision Algorithms

On modern architectures, the performance of 32-bit operations is often at least twice as fast as the performance of 64-bit operations. By using a combination of 32-bit and 64-bit floating point arithmetic, the performance of many dense and sparse linear algebra algorithms can be significantly enhanced while maintaining the 64-bit accuracy of the resulting solution. The approach presented here can apply not only to conventional processors but also to other technologies such as Field Programmable Gate Arrays (FPGA), Graphical Processing Units (GPU), and the STI Cell BE processor. Results on modern processor architectures and the STI Cell BE are presented.

cs.MS↗
arXiv2008

Communication-optimal parallel and sequential QR and LU factorizations

We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform, and just as stable as Householder QR. We prove optimality by extending known lower bounds on communication bandwidth for sequential and parallel matrix multiplication to provide latency lower bounds, and show these bounds apply to the LU and QR decompositions. We not only show that our QR algorithms attain these lower bounds (up to polylogarithmic factors), but that existing LAPACK and ScaLAPACK algorithms perform asymptotically more communication. We also point out recent LU algorithms in the literature that attain at least some of these lower bounds.

math.NA↗