Searcharxiv⌕ Search

arXiv subjects

Ibtihel Ben Gharbia

Publications and source records attributed to Ibtihel Ben Gharbia.

5 recordsLinked to original sources

Acceleration of Newton's method for nonlinear algebraic systems by preflattening and elimination techniques

In this paper, we propose a new approach called {\em preflattening} to speed up the resolution of nonlinear algebraic systems by Newton-type methods. Preflattening is meant to be for nonlinear systems what preconditioning is for linear ones. The idea is to transform the system into an equivalent one that is more favorable to Newton's method. To this end, we introduce the notion of {\em flatness}, which is intended to play for nonlinear systems a role similar to that of the condition number for linear systems. For a physical model of interest in the geosciences, namely the chemical equilibrium problem, we combine this strategy with the nonlinear elimination technique and demonstrate the value of the resulting approach through numerical simulations.

math.NA↗

Hierarchical matrix approximability of inverse of convection dominated finite element matrices

Several researchers have developed a rich toolbox of matrix compression techniques that exploit structure and redundancy in large matrices. Classical methods such as the block low-rank format and the Fast Multipole Method make it possible to manipulate very large systems by representing them in a reduced form. Among the most sophisticated tools in this area are hierarchical matrices (H-matrices), which exploit local properties of the underlying kernel or operator to approximate matrix blocks by low-rank factors, organized in a recursive hierarchy. H-matrices offer a flexible and scalable framework, yielding nearly linear complexity in both storage and computation. Hierarchical matrix techniques, originally developed for boundary integral equations, have recently been applied to matrices stemming from the discretization of advection-dominated problems. However, their effectiveness is limited by the loss of coercivity induced by convection phenomena, where traditional methods fail. Initial work by Le Borne addressed this by modifying the admissibility criterion for structured grids with constant convection, but challenges remain for more general grids and advection fields. In this work, we propose a novel partitioning strategy based on "convection tubes", clusters aligned with the convection vector field. This method does not require a structured grid or constant convection, overcoming the limitations of previous approaches. We present both theoretical analyses and numerical experiments, that demonstrate the efficiency and robustness of our method for convection-dominated PDEs on unstructured grids. The approach builds on a Péclet-robust Caccioppoli inequality, crucial for handling convection-dominated problems.

math.NA↗

Analysis and Comparison of Two-Level KFAC Methods for Training Deep Neural Networks

As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), efficient approximations are necessary to make NGD scalable to Deep Neural Networks (DNNs). Many such approximations have been attempted. The most sophisticated of these is KFAC, which approximates the FIM as a block-diagonal matrix, where each block corresponds to a layer of the neural network. By doing so, KFAC ignores the interactions between different layers. In this work, we investigate the interest of restoring some low-frequency interactions between the layers by means of two-level methods. Inspired from domain decomposition, several two-level corrections to KFAC using different coarse spaces are proposed and assessed. The obtained results show that incorporating the layer interactions in this fashion does not really improve the performance of KFAC. This suggests that it is safe to discard the off-diagonal blocks of the FIM, since the block-diagonal approach is sufficiently robust, accurate and economical in computation time.

cs.LG↗

Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition

Several studies have shown the ability of natural gradient descent to minimize the objective function more efficiently than ordinary gradient descent based methods. However, the bottleneck of this approach for training deep neural networks lies in the prohibitive cost of solving a large dense linear system corresponding to the Fisher Information Matrix (FIM) at each iteration. This has motivated various approximations of either the exact FIM or the empirical one. The most sophisticated of these is KFAC, which involves a Kronecker-factored block diagonal approximation of the FIM. With only a slight additional cost, a few improvements of KFAC from the standpoint of accuracy are proposed. The common feature of the four novel methods is that they rely on a direct minimization problem, the solution of which can be computed via the Kronecker product singular value decomposition technique. Experimental results on the three standard deep auto-encoder benchmarks showed that they provide more accurate approximations to the FIM. Furthermore, they outperform KFAC and state-of-the-art first-order methods in terms of optimization speed.

cs.NE↗

Gas phase appearance and disappearance as a problem with complementarity constraints

The modeling of migration of hydrogen produced by the corrosion of the nuclear waste packages in an underground storage including the dissolution of hydrogen involves a set of nonlinear partial differential equations with nonlinear complementarity constraints. This article shows how to apply a modern and efficient solution strategy, the Newton-min method, to this geoscience problem and investigates its applicability and efficiency. In particular, numerical experiments show that the Newton-min method is quadratically convergent for this problem.

math.AP↗