arXiv subjects

Jonas Kusch

Publications and source records attributed to Jonas Kusch.

At least 37 records · Page 2Linked to original sources

arXiv2024

Geometry-aware training of factorized layers in tensor Tucker format

Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during training and inference phases. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format and parameterizes them as the product of two small rank matrices. However, this approach typically requires an initial full-model warm-up phase, prior knowledge of a feasible rank, and it is sensitive to parameter initialization. In this work, we introduce a novel approach to train the factors of a Tucker decomposition of the weight tensors. Our training proposal proves to be optimal in locally approximating the original unfactorized dynamics independently of the initialization. Furthermore, the rank of each mode is dynamically updated during training. We provide a theoretical analysis of the algorithm, showing convergence, approximation and local descent guarantees. The method's performance is further illustrated through a variety of experiments, showing remarkable training compression rates and comparable or even better performance than the full baseline and alternative layer factorization strategies.

cs.LG↗

arXiv2024

Macro-micro decomposition for consistent and conservative model order reduction of hyperbolic shallow water moment equations: A study using POD-Galerkin and dynamical low rank approximation

Geophysical flow simulations using hyperbolic shallow water moment equations require an efficient discretization of a potentially large system of PDEs, the so-called moment system. This calls for tailored model order reduction techniques that allow for efficient and accurate simulations while guaranteeing physical properties like mass conservation. In this paper, we develop the first model reduction for the hyperbolic shallow water moment equations and achieve mass conservation. This is accomplished using a macro-micro decomposition of the model into a macroscopic (conservative) part and a microscopic (non-conservative) part with subsequent model reduction using either POD-Galerkin or dynamical low-rank approximation only on the microscopic (non-conservative) part. Numerical experiments showcase the performance of the new model reduction methods including high accuracy and fast computation times together with guaranteed conservation and consistency properties.

math.NA↗

arXiv2024

Second-order robust parallel integrators for dynamical low-rank approximation

Due to its reduced memory and computational demands, dynamical low-rank approximation (DLRA) has sparked significant interest in multiple research communities. A central challenge in DLRA is the development of time integrators that are robust to the curvature of the manifold of low-rank matrices. Recently, a parallel robust time integrator that permits dynamic rank adaptation and enables a fully parallel update of all low-rank factors was introduced. Despite its favorable computational efficiency, the construction as a first-order approximation to the augmented basis-update & Galerkin integrator restricts the parallel integrator's accuracy to order one. In this work, an extension to higher order is proposed by a careful basis augmentation before solving the matrix differential equations of the factorized solution. A robust error bound with an improved dependence on normal components of the vector field together with a norm preservation property up to small terms is derived. These analytic results are complemented and demonstrated through a series of numerical experiments.

math.NA↗

arXiv2024

Asymptotic-preserving and energy stable dynamical low-rank approximation for thermal radiative transfer equations

The thermal radiative transfer equations model temperature evolution through a background medium as a result of radiation. When a large number of particles are absorbed in a short time scale, the dynamics tend to a non-linear diffusion-type equation called the Rosseland approximation. The main challenges for constructing numerical schemes that exhibit the correct limiting behavior are posed by the solution's high-dimensional phase space and multi-scale effects. In this work, we propose an asymptotic-preserving and rank-adaptive dynamical low-rank approximation scheme based on the macro-micro decomposition of the particle density and a modified augmented basis-update \& Galerkin integrator. We show that this scheme, for linear particle emission by the material, dissipates energy over time under a step size restriction that captures the hyperbolic and parabolic CFL conditions. We demonstrate the efficacy of the proposed method in a series of numerical experiments.

math.NA↗

arXiv2024

A robust second-order low-rank BUG integrator based on the midpoint rule

Dynamical low-rank approximation has become a valuable tool to perform an on-the-fly model order reduction for prohibitively large matrix differential equations. A core ingredient is the construction of integrators that are robust to the presence of small singular values and the resulting large time derivatives of the orthogonal factors in the low-rank matrix representation. Recently, the robust basis-update & Galerkin (BUG) class of integrators has been introduced. These methods require no steps that evolve the solution backward in time, often have favourable structure-preserving properties, and allow for parallel time-updates of the low-rank factors. The BUG framework is flexible enough to allow for adaptations to these and further requirements. However, the BUG methods presented so far have only first-order robust error bounds. This work proposes a second-order BUG integrator for dynamical low-rank approximation based on the midpoint rule. The integrator first performs a half-step with a first-order BUG integrator, followed by a Galerkin update with a suitably augmented basis. We prove a robust second-order error bound which in addition shows an improved dependence on the normal component of the vector field. These rigorous results are illustrated and complemented by a number of numerical experiments.

math.NA↗

arXiv2023

Energy stable and conservative dynamical low-rank approximation for the Su-Olson problem

Computational methods for thermal radiative transfer problems exhibit high computational costs and a prohibitive memory footprint when the spatial and directional domains are finely resolved. A strategy to reduce such computational costs is dynamical low-rank approximation (DLRA), which represents and evolves the solution on a low-rank manifold, thereby significantly decreasing computational and memory requirements. Efficient discretizations for the DLRA evolution equations need to be carefully constructed to guarantee stability while enabling mass conservation. In this work, we focus on the Su-Olson closure leading to a linearized internal energy model and derive a stable discretization through an implicit coupling of internal energy and particle density. Moreover, we propose a rank-adaptive strategy to preserve local mass conservation. Numerical results are presented which showcase the accuracy and efficiency of the proposed low-rank method compared to the solution of the full system.

math.NA↗

arXiv2023

A parallel rank-adaptive integrator for dynamical low-rank approximation

This work introduces a parallel and rank-adaptive matrix integrator for dynamical low-rank approximation. The method is related to the previously proposed rank-adaptive basis update & Galerkin (BUG) integrator but differs significantly in that all arising differential equations, both for the basis and the Galerkin coefficients, are solved in parallel. Moreover, this approach eliminates the need for a potentially costly coefficient update with augmented basis matrices. The integrator also incorporates a new step rejection strategy that enhances the robustness of both the parallel integrator and the BUG integrator. By construction, the parallel integrator inherits the robust error bound of the BUG and projector-splitting integrators. Comparisons of the parallel and BUG integrators are presented by a series of numerical experiments which demonstrate the efficiency of the proposed method, for problems from radiative transfer and radiation therapy.

math.NA↗

arXiv2022

Asymptotic--preserving and energy stable dynamical low-rank approximation

Radiation transport problems are posed in a high-dimensional phase space, limiting the use of finely resolved numerical simulations. An emerging tool to efficiently reduce computational costs and memory footprint in such settings is dynamical low-rank approximation (DLRA). Despite its efficiency, numerical methods for DLRA need to be carefully constructed to guarantee stability while preserving crucial properties of the original problem. Important physical effects that one likes to preserve with DLRA include capturing the diffusion limit in the high-scattering regimes as well as dissipating energy. In this work we propose and analyze a dynamical low-rank method based on the "unconventional" basis-update & Galerkin step integrator. We show that this method is asymptotic-preserving, i.e., it captures the diffusion limit, and energy stable under a CFL condition. The derived CFL condition captures the transition from the hyperbolic to the parabolic regime when approaching the diffusion limit.

math.NA↗

arXiv2022

Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations

Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In this work, we propose a novel algorithm to find efficient low-rank subnetworks. Remarkably, these subnetworks are determined and adapted already during the training phase and the overall time and memory resources required by both training and evaluating them are significantly reduced. The main idea is to restrict the weight matrices to a low-rank manifold and to update the low-rank factors rather than the full matrix during training. To derive training updates that are restricted to the prescribed manifold, we employ techniques from dynamic model order reduction for matrix differential equations. This allows us to provide approximation, stability, and descent guarantees. Moreover, our method automatically and dynamically adapts the ranks during training to achieve the desired approximation accuracy. The efficiency of the proposed method is demonstrated through a variety of numerical experiments on fully-connected and convolutional networks.

cs.LG↗

arXiv2022

KiT-RT: An extendable framework for radiative transfer and therapy

In this paper we present KiT-RT (Kinetic Transport Solver for Radiation Therapy), an open-source C++ based framework for solving kinetic equations in radiation therapy applications. The aim of this code framework is to provide a collection of classical deterministic solvers for unstructured meshes that allow for easy extendability. Therefore, KiT-RT is a convenient base to test new numerical methods in various applications and compare them against conventional solvers. The implementation includes spherical-harmonics, minimal entropy, neural minimal entropy and discrete ordinates methods. Solution characteristics and efficiency are presented through several test cases ranging from radiation transport to electron radiation therapy. Due to the variety of included numerical methods and easy extendability, the presented open source code is attractive for both developers, who want a basis to build their own numerical solvers and users or application engineers, who want to gain experimental insights without directly interfering with the codebase.

physics.med-ph↗

arXiv2022

A low-rank power iteration scheme for neutron transport criticality problems

Computing effective eigenvalues for neutron transport often requires a fine numerical resolution. The main challenge of such computations is the high memory effort of classical solvers, which limits the accuracy of chosen discretizations. In this work, we derive a method for the computation of effective eigenvalues when the underlying solution has a low-rank structure. This is accomplished by utilizing dynamical low-rank approximation (DLRA), which is an efficient strategy to derive time evolution equations for low-rank solution representations. The main idea is to interpret the iterates of the classical inverse power iteration as pseudo-time steps and apply the DLRA concepts in this framework. In our numerical experiment, we demonstrate that our method significantly reduces memory requirements while achieving the desired accuracy. Analytic investigations show that the proposed iteration scheme inherits the convergence speed of the inverse power iteration, at least for a simplified setting.

math.NA↗

arXiv2021

A flux reconstruction stochastic Galerkin scheme for hyperbolic conservation laws

The study of uncertainty propagation poses a great challenge to design numerical solvers with high fidelity. Based on the stochastic Galerkin formulation, this paper addresses the idea and implementation of the first flux reconstruction scheme for hyperbolic conservation laws with random inputs. Unlike the finite volume method, the treatments in physical and random space are consistent, e.g., the modal representation of solutions based on an orthogonal polynomial basis and the nodal representation based on solution collocation points. Therefore, the numerical behaviors of the scheme in the phase space can be designed and understood uniformly. A family of filters is extended to multi-dimensional cases to mitigate the well-known Gibbs phenomenon arising from discontinuities in both physical and random space. The filter function is switched on and off by the dynamic detection of discontinuous solutions, and a slope limiter is employed to preserve the positivity of physically realizable solutions. As a result, the proposed method is able to capture stochastic cross-scale flow evolution where resolved and unresolved regions coexist. Numerical experiments including wave propagation, Burgers' shock, one-dimensional Riemann problem, and two-dimensional shock-vortex interaction problem are presented to validate the scheme. The order of convergence of the current scheme is identified. The capability of the scheme for simulating smooth and discontinuous stochastic flow dynamics is demonstrated. The open-source codes to reproduce the numerical results are available under the MIT license.

physics.comp-ph↗

arXiv2021

A robust collision source method for rank adaptive dynamical low-rank approximation in radiation therapy

Deterministic models for radiation transport describe the density of radiation particles moving through a background material. In radiation therapy applications, the phase space of this density is composed of energy, spatial position and direction of flight. The resulting six-dimensional phase space prohibits fine numerical discretizations, which are essential for the construction of accurate and reliable treatment plans. In this work, we tackle the high dimensional phase space through a dynamical low-rank approximation of the particle density. Dynamical low-rank approximation (DLRA) evolves the solution on a low-rank manifold in time. Interpreting the energy variable as a pseudo-time lets us employ the DLRA framework to represent the solution of the radiation transport equation on a low-rank manifold for every energy. Stiff scattering terms are treated through an efficient implicit energy discretization and a rank adaptive integrator is chosen to dynamically adapt the rank in energy. To facilitate the use of boundary conditions and reduce the overall rank, the radiation transport equation is split into collided and uncollided particles through a collision source method. Uncollided particles are described by a directed quadrature set guaranteeing low computational costs, whereas collided particles are represented by a low-rank solution. It can be shown that the presented method is L$^2$-stable under a time step restriction which does not depend on stiff scattering terms. Moreover, the implicit treatment of scattering does not require numerical inversions of matrices. Numerical results for radiation therapy configurations as well as the line source benchmark underline the efficiency of the proposed method.

math.NA↗

arXiv2021

On the stability of robust dynamical low-rank approximations for hyperbolic problems

The dynamical low-rank approximation (DLRA) is used to treat high-dimensional problems that arise in such diverse fields as kinetic transport and uncertainty quantification. Even though it is well known that certain spatial and temporal discretizations when combined with the DLRA approach can result in numerical instability, this phenomenon is poorly understood. In this paper we perform a $L^2$ stability analysis for the corresponding nonlinear equations of motion. This reveals the source of the instability for the projector splitting integrator when first discretizing the equations and then applying the DLRA. Based on this we propose a projector splitting integrator, based on applying DLRA to the continuous system before performing the discretization, that recovers the classic CFL condition. We also show that the unconventional integrator has more favorable stability properties and explain why the projector splitting integrator performs better when approximating higher moments, while the unconventional integrator is generally superior for first order moments. Furthermore, an efficient and stable dynamical low-rank update for the scattering term in kinetic transport is proposed. Numerical experiments for kinetic transport and uncertainty quantification, which confirm the results of the stability analysis, are presented.

math.NA↗

arXiv2021

Oscillation Mitigation of Hyperbolicity-Preserving Intrusive Uncertainty Quantification Methods for Systems of Conservation Laws

In this article we study intrusive uncertainty quantification schemes for systems of conservation laws with uncertainty. Standard intrusive methods lead to oscillatory solutions which sometimes even cause the loss of hyperbolicity. We consider the stochastic Galerkin scheme, in which we filter the coefficients of the polynomial expansion in order to reduce oscillations. We further apply the multi-element approach and ensure the preservation of hyperbolic solutions through the hyperbolicity limiter. In addition to that, we study the intrusive polynomial moment method, which guarantees hyperbolicity at the cost of solving an optimization problem in every spatial cell and every time step. To reduce numerical costs, we apply the multi-element ansatz to IPM. This ansatz decouples the optimization problems of all multi elements. Thus, we are able to significantly decrease computational costs while improving parallelizability. We finally evaluate these oscillation mitigating approaches on various numerical examples such as a NACA airfoil and a nozzle test case for the two-dimensional Euler equations. In our numerical experiments, we observe the mitigation of spurious artifacts. Furthermore, using the multi-element ansatz for IPM significantly reduces computational costs.

math.NA↗

arXiv2021

A Realizable Filtered Intrusive Polynomial Moment Method

Intrusive uncertainty quantification methods for hyperbolic problems exhibit spurious oscillations at shocks, which leads to a significant reduction of the overall approximation quality. Furthermore, a challenging task is to preserve hyperbolicity of the gPC moment system. An intrusive method which guarantees hyperbolicity is the intrusive polynomial moment (IPM) method, which performs the gPC expansion on the entropy variables. The method, while still being subject to oscillations, requires solving a convex optimization problem in every spatial cell and every time step. The aim of this work is to mitigate oscillations in the IPM solution by applying filters. Filters reduce oscillations by damping high order gPC coefficients. Naive filtering, however, may lead to unrealizable moments, which means that the IPM optimization problem does not have a solution and the method breaks down. In this paper, we propose and analyze two separate strategies to guarantee the existence of a solution to the IPM problem. First, we propose a filter which maintains realizability by being constructed from an underlying Fokker-Planck equation. Second, we regularize the IPM optimization problem to be able to cope with non-realizable gPC coefficients. Consequently, standard filters can be applied to the regularized IPM method. We demonstrate numerical results for the two strategies by investigating the Euler equations with uncertain shock structures in one- and two-dimensional spatial settings. We are able to show a significant reduction of spurious oscillations by the proposed filters.

math.NA↗

arXiv2021

Dynamical low-rank approximation for Burgers' equation with uncertainty

Quantifying uncertainties in hyperbolic equations is a source of several challenges. First, the solution forms shocks leading to oscillatory behaviour in the numerical approximation of the solution. Second, the number of unknowns required for an effective discretization of the solution grows exponentially with the dimension of the uncertainties, yielding high computational costs and large memory requirements. An efficient representation of the solution via adequate basis functions permits to tackle these difficulties. The generalized polynomial chaos (gPC) polynomials allow such an efficient representation when the distribution of the uncertainties is known. These distributions are usually only available for input uncertainties such as initial conditions, therefore the efficiency of this ansatz can get lost during runtime. In this paper, we make use of the dynamical low-rank approximation (DLRA) to obtain a memory-wise efficient solution approximation on a lower dimensional manifold. We investigate the use of the matrix projector-splitting integrator and the unconventional integrator for dynamical low-rank approximation, deriving separate time evolution equations for the spatial and uncertain basis functions, respectively. This guarantees an efficient approximation of the solution even if the underlying probability distributions change over time. Furthermore, filters to mitigate the appearance of spurious oscillations are implemented, and a strategy to enforce boundary conditions is introduced. The proposed methodology is analyzed for Burgers' equation equipped with uncertain initial values represented by a two-dimensional random vector. The numerical results show a reduction of the memory requirements, and that the important characteristics of the original system are well captured.

math.NA↗

arXiv2021

A rank-adaptive robust integrator for dynamical low-rank approximation

A rank-adaptive integrator for the dynamical low-rank approximation of matrix and tensor differential equations is presented. The fixed-rank integrator recently proposed by two of the authors is extended to allow for an adaptive choice of the rank, using subspaces that are generated by the integrator itself. The integrator first updates the evolving bases and then does a Galerkin step in the subspace generated by both the new and old bases, which is followed by rank truncation to a given tolerance. It is shown that the adaptive low-rank integrator retains the exactness, robustness and symmetry-preserving properties of the previously proposed fixed-rank integrator. Beyond that, up to the truncation tolerance, the rank-adaptive integrator preserves the norm when the differential equation does, it preserves the energy for Schrödinger equations and Hamiltonian systems, and it preserves the monotonic decrease of the functional in gradient flows. Numerical experiments illustrate the behaviour of the rank-adaptive integrator.

math.NA↗