arXiv subjects

Karsten Kahl

Publications and source records attributed to Karsten Kahl.

At least 19 recordsLinked to original sources

arXiv2026

A Theory of Relaxation-Based Algebraic Multigrid

Algebraic multigrid (AMG) methods derive their optimal efficiency from the interplay between a relaxation process and a corresponding coarse grid correction. In many standard formulations, relaxation and coarse-graining are analyzed and treated as largely separate of one another. Here we propose an alternative theoretical approach centered entirely on the relaxation process, which exposes its fundamental role in the coarse-graining of the fine-scale problem. By treating the relaxation of the error as a dynamical system and applying a dimensional-reduction procedure analogous to the Mori-Zwanzig-Nakajima formalism, we derive exact expressions for the coarse-level equations and the interpolation operations, as well as a natural way of computing complementary transfer operators. We illustrate the unifying nature of this framework by recovering several well-known results for general non-symmetric systems, including ideal and optimal restriction and interpolation, as well as the limiting case of exact elimination. We further emphasize the pivotal importance of compatible-relaxation and identify dynamical corrections that naturally arise in our theory, which have the potential to enhance the convergence, robustness, and adaptivity of future algebraic multigrid methods.

math.NA↗

arXiv2026

Nodal Coarsening and Sparse Ideal Interpolation for H(curl) Problems in Algebraic Multigrid

We propose a sparse interpolation construction and a practical coarsening algorithm for the algebraic multigrid (AMG) method, tailored towards H(curl). Building on the generalized AMG framework, we introduce an interior/exterior splitting that yields both a refinement-based and a fully algebraic construction of the interpolation. The refinement-based approach follows geometric hierarchy, while the purely algebraic interpolation is constructed through a coarsening process that first coarsens a nodal dual problem and then builds coarse and fine variables using a matching algorithm. We establish the weak approximation property and the commuting relation under certain assumptions. Combined with matching block smoothers, the proposed interpolation yields an effective algebraic multilevel method. Numerical experiments show robustness under strong coefficient jumps, where the proposed methods substantially outperform standard geometric multigrid.

math.NA↗

arXiv2025

LFA applied to CNNs: Efficient Singular Value Decomposition of Convolutional Mappings by Local Fourier Analysis

The singular values of convolutional mappings encode interesting spectral properties, which can be used, e.g., to improve generalization and robustness of convolutional neural networks as well as to facilitate model compression. However, the computation of singular values is typically very resource-intensive. The naive approach involves unrolling the convolutional mapping along the input and channel dimensions into a large and sparse two-dimensional matrix, making the exact calculation of all singular values infeasible due to hardware limitations. In particular, this is true for matrices that represent convolutional mappings with large inputs and a high number of channels. Existing efficient methods leverage the Fast Fourier transformation (FFT) to transform convolutional mappings into the frequency domain, enabling the computation of singular values for matrices representing convolutions with larger input and channel dimensions. For a constant number of channels in a given convolution, an FFT can compute N singular values in O(N log N) complexity. In this work, we propose an approach of complexity O(N) based on local Fourier analysis, which additionally exploits the shift invariance of convolutional operators. We provide a theoretical analysis of our algorithm's runtime and validate its efficiency through numerical experiments. Our results demonstrate that our proposed method is scalable and offers a practical solution to calculate the entire set of singular values - along with the corresponding singular vectors if needed - for high-dimensional convolutional mappings.

cs.LG↗

arXiv2025

Optimal transfer operators in algebraic two-level methods for nonsymmetric and indefinite problems

Consider an algebraic two-level method applied to the $n$-dimensional linear system $A \mathbf{x} = \mathbf{b}$ using fine-space preconditioner (i.e., ``relaxation'' or ``smoother'') $M$, with $M \approx A$, restriction and interpolation $R$ and $P$, and algebraic coarse-space operator ${A_c := R^*AP}$. Then, what are the the best possible transfer operators $R$ and $P$ of a given dimension $n_c < n$? Brannick et al. (2018) showed that when $A$ and $M$ are Hermitian positive definite (HPD), the optimal interpolation is such that its range contains the $n_c$ smallest generalized eigenvectors of the matrix pencil $(A, M)$. Recently, in Ali et al. (2025) we generalized this framework to the non-HPD setting, by considering both right (interpolation) and left (restriction) generalized eigenvectors of $(A, M)$ and defining corresponding nonsymmetric transfer operators $\{R_\#,P_\#\}$. Tight convergence bounds for $\{R_\#,P_\#\}$ are derived in spectral radius, as well as a proof of pseudo-optimality. Note, $\{R_\#,P_\#\}$ are typically complex valued, which is not practical for real-valued problems. Here we build on Ali et al. (2025), first characterizing all inner products in which the coarse-space correction defined by $\{R_\#,P_\#\}$ is orthogonal. We then develop tight two-level convergence bounds in these norms, and prove that the underlying transfer operators $\{R_\#,P_\#\}$ are genuinely optimal. As a special case, our theory both recovers and extends the HPD results from Brannick et al. (2018). Finally, we show how to construct optimal, real-valued transfer operators in the case of that $A$ and $M$ are real valued, but are not HPD. Numerical examples arising from discretized advection and wave-equation problems are used to verify and illustrate the theory.

math.NA↗

arXiv2025

Poly-MgNet: Polynomial Building Blocks in Multigrid-Inspired ResNets

The structural analogies of ResNets and Multigrid (MG) methods such as common building blocks like convolutions and poolings where already pointed out by He et al.\ in 2016. Multigrid methods are used in the context of scientific computing for solving large sparse linear systems arising from partial differential equations. MG methods particularly rely on two main concepts: smoothing and residual restriction / coarsening. Exploiting these analogies, He and Xu developed the MgNet framework, which integrates MG schemes into the design of ResNets. In this work, we introduce a novel neural network building block inspired by polynomial smoothers from MG theory. Our polynomial block from an MG perspective naturally extends the MgNet framework to Poly-Mgnet and at the same time reduces the number of weights in MgNet. We present a comprehensive study of our polynomial block, analyzing the choice of initial coefficients, the polynomial degree, the placement of activation functions, as well as of batch normalizations. Our results demonstrate that constructing (quadratic) polynomial building blocks based on real and imaginary polynomial roots enhances Poly-MgNet's capacity in terms of accuracy. Furthermore, our approach achieves an improved trade-off of model accuracy and number of weights compared to ResNet as well as compared to specific configurations of MgNet.

cs.LG↗

arXiv2025

Nodal AMG Coarsening and Interpolation for PDE Systems

We present an approach to constructing a practical coarsening algorithm and interpolation operator for the algebraic multigrid (AMG) method, tailored towards systems of partial differential equations (PDEs) with large near-kernels, such as H(curl) and H(div). Our method builds on compatible relaxation (CR) and the ideal interpolation model within the generalized AMG (GAMG) framework but introduces several modifications to define an AMG method for PDE systems. We construct an interpolation operator through a coarsening process that first coarsens a nodal dual problem and then builds the coarse and fine variables using a matching algorithm. Our interpolation follows the ideal formulation; however, we enhance the sparsity of ideal interpolation by decoupling the fine and coarse variables completely. When the coarse variables align with the geometric refinement, our method reproduces re-discretization on unstructured meshes. Together with an automatic smoother construction scheme that identifies the local near kernels, our approach forms a complete two-grid method. Finally, we also show numerical results that demonstrate the effectiveness of this interpolation scheme by applying it to targeted problems and the Stokes system.

math.NA↗

arXiv2025

Generalized Optimal AMG Convergence Theory for Stokes Equations Using Smooth Aggregation and Vanka Relaxation Strategies

This paper discusses our recent generalized optimal algebraic multigrid (AMG) convergence theory applied to the steady-state Stokes equations discretized using Taylor-Hood elements ($\pmb{ \mathbb{P}}_2/\mathbb{P}_{1}$). The generalized theory is founded on matrix-induced orthogonality of the left and right eigenvectors of a generalized eigenvalue problem involving the system matrix and relaxation operator. This framework establishes a rigorous lower bound on the spectral radius of the two-grid error-propagation operator, enabling precise predictions of the convergence rate for symmetric indefinite problems, such as those arising from saddle-point systems. We apply this theory to the recently developed monolithic smooth aggregation AMG (SA-AMG) solver for Stokes, constructed using evolution-based strength of connection, standard aggregation, and smoothed prolongation. The performance of these solvers is evaluated using additive and multiplicative Vanka relaxation strategies. Additive Vanka relaxation constructs patches algebraically on each level, resulting in a nonsymmetric relaxation operator due to the partition of unity being applied on one side of the block-diagonal matrix. Although symmetry can be restored by eliminating the partition of unity, this compromises convergence. Alternatively, multiplicative Vanka relaxation updates velocity and pressure sequentially within each patch, propagating updates multiplicatively across the domain and effectively addressing velocity-pressure coupling, ensuring a symmetric relaxation. We demonstrate that the generalized optimal AMG theory consistently provides accurate lower bounds on the convergence rate for SA-AMG applied to Stokes equations. These findings suggest potential avenues for further enhancement in AMG solver design for saddle-point systems.

math.NA↗

arXiv2024

Reducing Texture Bias of Deep Neural Networks via Edge Enhancing Diffusion

Convolutional neural networks (CNNs) for image processing tend to focus on localized texture patterns, commonly referred to as texture bias. While most of the previous works in the literature focus on the task of image classification, we go beyond this and study the texture bias of CNNs in semantic segmentation. In this work, we propose to train CNNs on pre-processed images with less texture to reduce the texture bias. Therein, the challenge is to suppress image texture while preserving shape information. To this end, we utilize edge enhancing diffusion (EED), an anisotropic image diffusion method initially introduced for image compression, to create texture reduced duplicates of existing datasets. Extensive numerical studies are performed with both CNNs and vision transformer models trained on original data and EED-processed data from the Cityscapes dataset and the CARLA driving simulator. We observe strong texture-dependence of CNNs and moderate texture-dependence of transformers. Training CNNs on EED-processed images enables the models to become completely ignorant with respect to texture, demonstrating resilience with respect to texture re-introduction to any degree. Additionally we analyze the performance reduction in depth on a level of connected components in the semantic segmentation and study the influence of EED pre-processing on domain generalization as well as adversarial robustness.

cs.CV↗

arXiv2024

Constrained Local Approximate Ideal Restriction for Advection-Diffusion Problems

This paper focuses on developing a reduction-based algebraic multigrid method that is suitable for solving general (non)symmetric linear systems and is naturally robust from pure advection to pure diffusion. Initial motivation comes from a new reduction-based algebraic multigrid (AMG) approach, $\ell$AIR (local approximate ideal restriction), that was developed for solving advection-dominated problems. Though this new solver is very effective in the advection dominated regime, its performance degrades in cases where diffusion becomes dominant. This is consistent with the fact that in general, reduction-based AMG methods tend to suffer from growth in complexity and/or convergence rates as the problem size is increased, especially for diffusion dominated problems in two or three dimensions. Motivated by the success of $\ell$AIR in the advective regime, our aim in this paper is to generalize the AIR framework with the goal of improving the performance of the solver in diffusion dominated regimes. To do so, we propose a novel way to combine mode constraints as used commonly in energy minimization AMG methods with the local approximation of ideal operators used in $\ell$AIR. The resulting constrained $\ell$AIR (C$\ell$AIR) algorithm is able to achieve fast scalable convergence on advective and diffusive problems. In addition, it is able to achieve standard low complexity hierarchies in the diffusive regime through aggressive coarsening, something that has been previously difficult for reduction-based methods.

math.NA↗

arXiv2024

Generalized Optimal AMG Convergence Theory for Nonsymmetric and Indefinite Problems

Algebraic multigrid (AMG) is known to be an effective solver for many sparse symmetric positive definite (SPD) linear systems. For SPD systems, the convergence theory of AMG is well-understood in terms of the $A$-norm, but in a nonsymmetric setting, such an energy norm is non-existent. For this reason, convergence of AMG for nonsymmetric systems of equations remains an open area of research. A particular aspect missing from theory of nonsymmetric and indefinite AMG is the incorporation of general relaxation schemes. In the SPD setting, the classical form of optimal AMG interpolation provides a useful insight in determining the best possible two-grid convergence rate of a method based on an arbitrary symmetrized relaxation scheme. In this work, we discuss a generalization of the optimal AMG convergence theory targeting nonsymmetric problems, using a certain matrix-induced orthogonality of the left and right eigenvectors of a generalized eigenvalue problem relating the system matrix and relaxation operator. We show that using this generalization of the optimal convergence theory, one can obtain a measure of the spectral radius of the two grid error transfer operator that is mathematically equivalent to the derivation in the SPD setting for optimal interpolation, which instead uses norms. In addition, this generalization of the optimal AMG convergence theory can be further extended for symmetric indefinite problems, such as those arising from saddle point systems so that one can obtain a precise convergence rate of the resulting two-grid method based on optimal interpolation. We provide supporting numerical examples of the convergence theory for nonsymmetric advection-diffusion problems, two-dimensional Dirac equation motivated by $\gamma_5$-symmetry, and the mixed Darcy flow problem corresponding to a saddle point system.

math.NA↗

arXiv2023

Identifying Label Errors in Object Detection Datasets by Loss Inspection

Labeling datasets for supervised object detection is a dull and time-consuming task. Errors can be easily introduced during annotation and overlooked during review, yielding inaccurate benchmarks and performance degradation of deep neural networks trained on noisy labels. In this work, we for the first time introduce a benchmark for label error detection methods on object detection datasets as well as a label error detection method and a number of baselines. We simulate four different types of randomly introduced label errors on train and test sets of well-labeled object detection datasets. For our label error detection method we assume a two-stage object detector to be given and consider the sum of both stages' classification and regression losses. The losses are computed with respect to the predictions and the noisy labels including simulated label errors, aiming at detecting the latter. We compare our method to three baselines: a naive one without deep learning, the object detector's score and the entropy of the classification softmax distribution. We outperform all baselines and demonstrate that among the considered methods, ours is the only one that detects label errors of all four types efficiently. Furthermore, we detect real label errors a) on commonly used test datasets in object detection and b) on a proprietary dataset. In both cases we achieve low false positives rates, i.e., we detect label errors with a precision for a) of up to 71.5% and for b) with 97%.

cs.CV↗

arXiv2023

Krylov subspace restarting for matrix Laplace transforms

A common way to approximate $F(A)b$ -- the action of a matrix function on a vector -- is to use the Arnoldi approximation. Since a new vector needs to be generated and stored in every iteration, one is often forced to rely on restart algorithms which are either not efficient, not stable or only applicable to restricted classes of functions. We present a new representation of the error of the Arnoldi iterates if the function $F$ is given as a Laplace transform. Based on this representation we build an efficient and stable restart algorithm. In doing so we extend earlier work for the class of Stieltjes functions which are special Laplace transforms. We report several numerical experiments including comparisons with the restart method for Stieltjes functions.

math.NA↗

arXiv2023

Deep Active Learning with Noisy Oracle in Object Detection

Obtaining annotations for complex computer vision tasks such as object detection is an expensive and time-intense endeavor involving a large number of human workers or expert opinions. Reducing the amount of annotations required while maintaining algorithm performance is, therefore, desirable for machine learning practitioners and has been successfully achieved by active learning algorithms. However, it is not merely the amount of annotations which influences model performance but also the annotation quality. In practice, the oracles that are queried for new annotations frequently contain significant amounts of noise. Therefore, cleansing procedures are oftentimes necessary to review and correct given labels. This process is subject to the same budget as the initial annotation itself since it requires human workers or even domain experts. Here, we propose a composite active learning framework including a label review module for deep object detection. We show that utilizing part of the annotation budget to correct the noisy annotations partially in the active dataset leads to early improvements in model performance, especially when coupled with uncertainty-based query strategies. The precision of the label error proposals has a significant influence on the measured effect of the label review. In our experiments we achieve improvements of up to 4.5 mAP points of object detection performance by incorporating label reviews at equal annotation budget.

cs.CV↗

arXiv2023

LMD: Light-weight Prediction Quality Estimation for Object Detection in Lidar Point Clouds

Object detection on Lidar point cloud data is a promising technology for autonomous driving and robotics which has seen a significant rise in performance and accuracy during recent years. Particularly uncertainty estimation is a crucial component for down-stream tasks and deep neural networks remain error-prone even for predictions with high confidence. Previously proposed methods for quantifying prediction uncertainty tend to alter the training scheme of the detector or rely on prediction sampling which results in vastly increased inference time. In order to address these two issues, we propose LidarMetaDetect (LMD), a light-weight post-processing scheme for prediction quality estimation. Our method can easily be added to any pre-trained Lidar object detector without altering anything about the base model and is purely based on post-processing, therefore, only leading to a negligible computational overhead. Our experiments show a significant increase of statistical reliability in separating true from false predictions. We propose and evaluate an additional application of our method leading to the detection of annotation errors. Explicit samples and a conservative count of annotation error proposals indicates the viability of our method for large-scale datasets like KITTI and nuScenes. On the widely-used nuScenes test dataset, 43 out of the top 100 proposals of our method indicate, in fact, erroneous annotations.

cs.CV↗

arXiv2022

A flexible short recurrence Krylov subspace method for matrices arising in the time integration of port Hamiltonian systems and ODEs/DAEs with a dissipative Hamiltonian

For several classes of mathematical models that yield linear systems, the splitting of the matrix into its Hermitian and skew Hermitian parts is naturally related to properties of the underlying model. This is particularly so for discretizations of dissipative Hamiltonian ODEs, DAEs and port Hamiltonian systems where, in addition, the Hermitian part is positive definite or semi-definite. It is then possible to develop short recurrence optimal Krylov subspace methods in which the Hermitian part is used as a preconditioner. In this paper we develop new, right preconditioned variants of this approach which as their crucial new feature allow the systems with the Hermitian part to be solved only approximately in each iteration while keeping the short recurrences. This new class of methods is particularly efficient as it allows, for example, to use few steps of a multigrid solver or a (preconditioned) CG method for the Hermitian part in each iteration. We illustrate this with several numerical experiments for large scale systems.

math.NA↗

arXiv2022

Towards Rapid Prototyping and Comparability in Active Learning for Deep Object Detection

Active learning as a paradigm in deep learning is especially important in applications involving intricate perception tasks such as object detection where labels are difficult and expensive to acquire. Development of active learning methods in such fields is highly computationally expensive and time consuming which obstructs the progression of research and leads to a lack of comparability between methods. In this work, we propose and investigate a sandbox setup for rapid development and transparent evaluation of active learning in deep object detection. Our experiments with commonly used configurations of datasets and detection architectures found in the literature show that results obtained in our sandbox environment are representative of results on standard configurations. The total compute time to obtain results and assess the learning behavior can thereby be reduced by factors of up to 14 when comparing with Pascal VOC and up to 32 when comparing with BDD100k. This allows for testing and evaluating data acquisition and labeling strategies in under half a day and contributes to the transparency and development speed in the field of active learning for object detection.

cs.CV↗

arXiv2022

MGiaD: Multigrid in all dimensions. Efficiency and robustness by coarsening in resolution and channel dimensions

Current state-of-the-art deep neural networks for image classification are made up of 10 - 100 million learnable weights and are therefore inherently prone to overfitting. The complexity of the weight count can be seen as a function of the number of channels, the spatial extent of the input and the number of layers of the network. Due to the use of convolutional layers the scaling of weight complexity is usually linear with regards to the resolution dimensions, but remains quadratic with respect to the number of channels. Active research in recent years in terms of using multigrid inspired ideas in deep neural networks have shown that on one hand a significant number of weights can be saved by appropriate weight sharing and on the other that a hierarchical structure in the channel dimension can improve the weight complexity to linear. In this work, we combine these multigrid ideas to introduce a joint framework of multigrid inspired architectures, that exploit multigrid structures in all relevant dimensions to achieve linear weight complexity scaling and drastically reduced weight counts. Our experiments show that this structured reduction in weight count is able to reduce overfitting and thus shows improved performance over state-of-the-art ResNet architectures on typical image classification benchmarks at lower network complexity.

cs.CV↗

arXiv2021

Matrix functions via linear systems built from continued fractions

A widely used approach to compute the action $f(A)v$ of a matrix function $f(A)$ on a vector $v$ is to use a rational approximation $r$ for $f$ and compute $r(A)v$ instead. If $r$ is not computed adaptively as in rational Krylov methods, this is usually done using the partial fraction expansion of $r$ and solving linear systems with matrices $A- τI$ for the various poles $τ$ of $r$. Here we investigate an alternative approach for the case that a continued fraction representation for the rational function is known rather than a partial fraction expansion. This is typically the case, for example, for Padé approximations. From the continued fraction, we first construct a matrix pencil from which we then obtain what we call the CF-matrix (continued fraction matrix), a block tridiagonal matrix whose blocks consist of polynomials of $A$ with degree bounded by 1 for many continued fractions. We show that one can evaluate $r(A)v$ by solving a single linear system with the CF-matrix and present a number of first theoretical results as a basis for an analysis of future, specific solution methods for the large linear system. While the CF-matrix approach is of principal interest on its own as a new way to compute $f(A)v$, it can in particular be beneficial when a partial fraction expansion is not known beforehand and computing its parameters is ill-conditioned. We report some numerical experiments which show that with standard preconditioners we can achieve fast convergence in the iterative solution of the large linear system.

math.NA↗