arXiv subjects

Olaf Schenk

Publications and source records attributed to Olaf Schenk.

At least 19 recordsLinked to original sources

arXiv2026

Scalable Approximate Selected Inversion Based on Single-Level Incomplete $LDL^T$ Factorization and Spectral Corrections for Large Sparse Systems

This article introduces four parallel numerical techniques for computing entries of the inverse of large sparse symmetric systems, all grounded in incomplete $LDL^T$ (ILDL) factorizations: (1) the selected inversion method (SelInv), which applies the $LDL^T$ factorization to recover entries of the matrix inverse within the sparsity pattern of the computed factors; (2) an approximate inversion method based on a truncated Neumann series expansion applied to the inverse of the L factor (NInv), providing an alternative at the cost of reduced accuracy; (3) a Mix approximation that merges the best of both SelInv and NInv; and (4) Mix-SPAI, which applies sparse approximate inverse (SPAI) refinement on the output of the Mix method to improve entry-level accuracy. To further improve accuracy while maintaining a stable sparsity pattern, we additionally employ a low-rank correction based on eigenvector updates, providing an alternative to tightening the drop tolerance. We report the performance of the proposed numerical techniques on a comprehensive collection of sparse matrices from scientific and industrial applications.

math.NA↗

arXiv2026

The Regularization Parameter: Sparse Precision Matrix Estimation

Sparse precision matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the $\ell_1$-regularized Gaussian maximum-likelihood estimator. By prescribing the probability that each nonzero entry of the estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the estimator. On synthetic Gaussian and non-Gaussian datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.

stat.ML↗

arXiv2026

Nonlinear spectral clustering with C++ GraphBLAS

Nonlinear reformulations of the spectral clustering method have gained a lot of recent attention due to their increased numerical benefits and their solid mathematical background. However, the estimation of the multiple nonlinear eigenvectors is associated with an increased computational cost. We present an implementation of a direct multiway spectral clustering algorithm in the $p$-norm, for $p\in(1,2]$, using a novel C++ GraphBLAS API. The key operations are expressed in linear algebraic terms and are executed over the resulting sparse matrices and dense vectors, parameterized in the algebra pertinent to the computation. We demonstrate the effectiveness and accuracy of our shared-memory algorithm on several artificial test cases. Our numerical examples and comparative results against competitive methods indicate that the proposed implementation attains high quality clusters in terms of the balanced graph cut metric. The strong scaling capabilities of our algorithm are showcased on a range of datasets with up to $8$ million nodes and $48$ million edges.

cs.DC↗

arXiv2026

Parallel Quadratic Selected Inversion in Quantum Transport Simulation

Driven by Moore's Law, the dimensions of transistors have been pushed down to the nanometer scale. Advanced quantum transport (QT) solvers are required to accurately simulate such nano-devices. The non-equilibrium Green's function (NEGF) formalism lends itself optimally to these tasks, but it is computationally very intensive, involving the selected inversion (SI) of matrices and the selected solution of quadratic matrix (SQ) equations. Existing algorithms to tackle these numerical problems are ideally suited to GPU acceleration, e.g., the so-called recursive Green's function (RGF) technique, but they are typically sequential, require block-tridiagonal (BT) matrices as inputs, and their implementation has been so far restricted to shared memory parallelism, thus limiting the achievable device sizes. To address these shortcomings, we introduce distributed methods that build on RGF and enable parallel selected inversion and selected solution of the quadratic matrix equation. We further extend them to handle BT matrices with arrowhead, which allows for the investigation of multi-terminal transistor structures. We evaluate the performance of our approach on a real dataset from the QT simulation of a nano-ribbon transistor and compare it with the sparse direct package PARDISO. When scaling to 16 GPUs, our fused SI and SQ solver is 5.2x faster than the SI module of PARDISO applied to a device 16x shorter. These results highlight the potential of our method to accelerate NEGF-based nano-device simulations.

cs.DC↗

arXiv2025

Accelerated Spatio-Temporal Bayesian Modeling for Multivariate Gaussian Processes

Multivariate Gaussian processes (GPs) offer a powerful probabilistic framework to represent complex interdependent phenomena. They pose, however, significant computational challenges in high-dimensional settings, which frequently arise in spatial-temporal applications. We present DALIA, a highly scalable framework for performing Bayesian inference tasks on spatio-temporal multivariate GPs, based on the methodology of integrated nested Laplace approximations. Our approach relies on a sparse inverse covariance matrix formulation of the GP, puts forward a GPU-accelerated block-dense approach, and introduces a hierarchical, triple-layer, distributed memory parallel scheme. We showcase weak scaling performance surpassing the state-of-the-art by two orders of magnitude on a model whose parameter space is 8$\times$ larger and measure strong scaling speedups of three orders of magnitude when running on 496 GH200 superchips on the Alps supercomputer. Applying DALIA to air pollution data from northern Italy over 48 days, we showcase refined spatial resolutions over the aggregated pollutant measurements.

stat.CO↗

arXiv2025

Serinv: A Scalable Library for the Selected Inversion of Block-Tridiagonal with Arrowhead Matrices

The inversion of structured sparse matrices is a key but computationally and memory-intensive operation in many scientific applications. There are cases, however, where only particular entries of the full inverse are required. This has motivated the development of so-called selected-inversion algorithms, capable of computing only specific elements of the full inverse. Currently, most of them are either shared-memory codes or limited to CPU implementations. Here, we introduce Serinv, a scalable library providing distributed, GPU-based algorithms for the selected inversion and Cholesky decomposition of positive-definite, block-tridiagonal arrowhead matrices. This matrix class is highly relevant in statistical climate modeling and materials science applications. The performance of Serinv is demonstrated on synthetic and real datasets from statistical air temperature prediction models. In our numerical tests, Serinv achieves 32.3% strong and 47.2% weak scaling efficiency and up to two orders of magnitude speedup over the sparse direct solvers PARDISO and MUMPS on 16 GPUs.

cs.DC↗

arXiv2024

Building Interpretable Climate Emulators for Economics

We introduce a framework for developing efficient and interpretable climate emulators (CEs) for economic models of climate change. The paper makes two main contributions. First, we propose a general framework for constructing carbon-cycle emulators (CCEs) for macroeconomic models. The framework is implemented as a generalized linear multi-reservoir (box) model that conserves key physical quantities and can be customized for specific applications. We consider three versions of the CCE, which we evaluate within a simple representative agent economic model: (i) a three-box setting comparable to DICE-2016, (ii) a four-box extension, and (iii) a four-box version that explicitly captures land-use change. While the three-box model reproduces benchmark results well and the fourth reservoir adds little, incorporating the impact of land-use change on the carbon storage capacity of the terrestrial biosphere substantially alters atmospheric carbon stocks, temperature trajectories, and the optimal mitigation path. Second, we investigate pattern-scaling techniques that transform global-mean temperature projections from CEs into spatially heterogeneous warming fields. We show how regional baseline climates, non-uniform warming, and the associated uncertainties propagate into economic damages.

econ.EM↗

arXiv2023

Application of deep and reinforcement learning to boundary control problems

The boundary control problem is a non-convex optimization and control problem in many scientific domains, including fluid mechanics, structural engineering, and heat transfer optimization. The aim is to find the optimal values for the domain boundaries such that the enclosed domain adhering to the governing equations attains the desired state values. Traditionally, non-linear optimization methods, such as the Interior-Point method (IPM), are used to solve such problems. This project explores the possibilities of using deep learning and reinforcement learning to solve boundary control problems. We adhere to the framework of iterative optimization strategies, employing a spatial neural network to construct well-informed initial guesses, and a spatio-temporal neural network learns the iterative optimization algorithm using policy gradients. Synthetic data, generated from the problems formulated in the literature, is used for training, testing and validation. The numerical experiments indicate that the proposed method can rival the speed and accuracy of existing solvers. In our preliminary results, the network attains costs lower than IPOPT, a state-of-the-art non-linear IPM, in 51\% cases. The overall number of floating point operations in the proposed method is similar to that of IPOPT. Additionally, the informed initial guess method and the learned momentum-like behaviour in the optimizer method are incorporated to avoid convergence to local minima.

cs.LG↗

arXiv2023

AI Driven Near Real-time Locational Marginal Pricing Method: A Feasibility and Robustness Study

Accurate price predictions are essential for market participants in order to optimize their operational schedules and bidding strategies, especially in the current context where electricity prices become more volatile and less predictable using classical approaches. The Locational Marginal Pricing (LMP) pricing mechanism is used in many modern power markets, where the traditional approach utilizes optimal power flow (OPF) solvers. However, for large electricity grids this process becomes prohibitively time-consuming and computationally intensive. Machine learning (ML) based predictions could provide an efficient tool for LMP prediction, especially in energy markets with intermittent sources like renewable energy. This study evaluates the performance of popular machine learning and deep learning models in predicting LMP on multiple electricity grids. The accuracy and robustness of these models in predicting LMP is assessed considering multiple scenarios. The results show that ML models can predict LMP 4-5 orders of magnitude faster than traditional OPF solvers with 5-6\% error rate, highlighting the potential of ML models in LMP prediction for large-scale power models with the assistance of hardware infrastructure like multi-core CPUs and GPUs in modern HPC clusters.

cs.CE↗

arXiv2023

Sensitivity Analysis of High-Dimensional Models with Correlated Inputs

Sensitivity analysis is an important tool used in many domains of computational science to either gain insight into the mathematical model and interaction of its parameters or study the uncertainty propagation through the input-output interactions. In many applications, the inputs are stochastically dependent, which violates one of the essential assumptions in the state-of-the-art sensitivity analysis methods. Consequently, the results obtained ignoring the correlations provide values which do not reflect the true contributions of the input parameters. This study proposes an approach to address the parameter correlations using a polynomial chaos expansion method and Rosenblatt and Cholesky transformations to reflect the parameter dependencies. Treatment of the correlated variables is discussed in context of variance and derivative-based sensitivity analysis. We demonstrate that the sensitivity of the correlated parameters can not only differ in magnitude, but even the sign of the derivative-based index can be inverted, thus significantly altering the model behavior compared to the prediction of the analysis disregarding the correlations. Numerous experiments are conducted using workflow automation tools within the VECMA toolkit.

stat.ME↗

arXiv2023

Integrated Nested Laplace Approximations for Large-Scale Spatial-Temporal Bayesian Modeling

Bayesian inference tasks continue to pose a computational challenge. This especially holds for spatial-temporal modeling where high-dimensional latent parameter spaces are ubiquitous. The methodology of integrated nested Laplace approximations (INLA) provides a framework for performing Bayesian inference applicable to a large subclass of additive Bayesian hierarchical models. In combination with the stochastic partial differential equations (SPDE) approach it gives rise to an efficient method for spatial-temporal modeling. In this work we build on the INLA-SPDE approach, by putting forward a performant distributed memory variant, INLA-DIST, for large-scale applications. To perform the arising computational kernel operations, consisting of Cholesky factorizations, solving linear systems, and selected matrix inversions, we present two numerical solver options, a sparse CPU-based library and a novel blocked GPU-accelerated approach which we propose. We leverage the recurring nonzero block structure in the arising precision (inverse covariance) matrices, which allows us to employ dense subroutines within a sparse setting. Both versions of INLA-DIST are highly scalable, capable of performing inference on models with millions of latent parameters. We demonstrate their accuracy and performance on synthetic as well as real-world climate dataset applications.

stat.CO↗

arXiv2022

Level-based Blocking for Sparse Matrices: Sparse Matrix-Power-Vector Multiplication

The multiplication of a sparse matrix with a dense vector (SpMV) is a key component in many numerical schemes and its performance is known to be severely limited by main memory access. Several numerical schemes require the multiplication of a sparse matrix polynomial with a dense vector, which is typically implemented as a sequence of SpMVs. This results in low performance and ignores the potential to increase the arithmetic intensity by reusing the matrix data from cache. In this work we use the recursive algebraic coloring engine (RACE) to enable blocking of sparse matrix data across the polynomial computations. In the graph representing the sparse matrix we form levels using a breadth-first search. Locality relations of these levels are then used to improve spatial and temporal locality when accessing the matrix data and to implement an efficient multithreaded parallelization. Our approach is independent of the matrix structure and avoids shortcomings of existing "blocking" strategies in terms of hardware efficiency and parallelization overhead. We quantify the quality of our implementation using performance modelling and demonstrate speedups of up to 3$\times$ and 5$\times$ compared to an optimal SpMV-based baseline on a single multicore chip of recent Intel and AMD architectures. As a potential application, we demonstrate the benefit of our implementation for a Chebyshev time propagation scheme, representing the class of polynomial approximations to exponential integrators. Further numerical schemes which may benefit from our developments include $s$-step Krylov solvers and power clustering algorithms.

math.NA↗

arXiv2022

Parallelized integrated nested Laplace approximations for fast Bayesian inference

There is a growing demand for performing larger-scale Bayesian inference tasks, arising from greater data availability and higher-dimensional model parameter spaces. In this work we present parallelization strategies for the methodology of integrated nested Laplace approximations (INLA), a popular framework for performing approximate Bayesian inference on the class of Latent Gaussian models. Our approach makes use of nested OpenMP parallelism, a parallel line search procedure using robust regression in INLA's optimization phase and the state-of-the-art sparse linear solver PARDISO. We leverage mutually independent function evaluations in the algorithm as well as advanced sparse linear algebra techniques. This way we can flexibly utilize the power of today's multi-core architectures. We demonstrate the performance of our new parallelization scheme on a number of different real-world applications. The introduction of parallelism leads to speedups of a factor 10 and more for all larger models. Our work is already integrated in the current version of the open-source R-INLA package, making its improved performance conveniently available to all users.

stat.CO↗

arXiv2021

Multiway $p$-spectral graph cuts on Grassmann manifolds

Nonlinear reformulations of the spectral clustering method have gained a lot of recent attention due to their increased numerical benefits and their solid mathematical background. We present a novel direct multiway spectral clustering algorithm in the $p$-norm, for $p \in (1, 2]$. The problem of computing multiple eigenvectors of the graph $p$-Laplacian, a nonlinear generalization of the standard graph Laplacian, is recasted as an unconstrained minimization problem on a Grassmann manifold. The value of $p$ is reduced in a pseudocontinuous manner, promoting sparser solution vectors that correspond to optimal graph cuts as $p$ approaches one. Monitoring the monotonic decrease of the balanced graph cuts guarantees that we obtain the best available solution from the $p$-levels considered. We demonstrate the effectiveness and accuracy of our algorithm in various artificial test-cases. Our numerical examples and comparative results with various state-of-the-art clustering methods indicate that the proposed method obtains high quality clusters both in terms of balanced graph cut metrics and in terms of the accuracy of the labelling assignment. Furthermore, we conduct studies for the classification of facial images and handwritten characters to demonstrate the applicability in real-world datasets.

cs.LG↗

arXiv2020

Complete results for a numerical evaluation of interior point solvers for large-scale optimal power flow problems

Recent advances in open source interior-point optimization methods and power system related software have provided researchers and educators with the necessary platform for simulating and optimizing power networks with unprecedented convenience. Within the Matpower software platform a combination of several different interior point optimization methods are provided and four different optimal power flow (OPF) formulations are recently available: the Polar-Power, Polar-Current, Cartesian-Power, and Cartesian-Current. The robustness and reliability of interior-point methods for different OPF formulations for minimizing the generation cost starting from different initial guesses, for a wide range of networks provided in the Matpower library ranging from 1951 buses to 193000 buses, will be investigated. Performance profiles are presented for iteration counts, overall time, and memory consumption, revealing the most reliable optimization method for the particular metric.

math.OC↗

arXiv2020

Reduced-Space Interior Point Methods in Power Grid Problems

Due to critical environmental issues, the power systems have to accommodate a significant level of penetration of renewable generation which requires smart approaches to the power grid control. Associated optimal control problems are large-scale nonlinear optimization problems with up to hundreds of millions of variables and constraints. The interior point methods become computationally intractable, mainly due to the solution of large linear systems. This document addresses the computational bottlenecks of the interior point method during the solution of the security constrained optimal power flow problems by applying reduced space quasi-Newton IPM, which could utilize high-performance computers due to the inherent parallelism in the adjoint method. Reduced space IPM approach and the adjoint method is a novel approach when it comes to solving the (security constrained) optimal power flow problems. These were previously used in the PDE-constrained optimization. The presented methodology is suitable for high-performance architectures due to inherent parallelism in the adjoint method during the gradient evaluation, since the individual contingency scenarios are modeled by independent set of the constraints. Preliminary evaluation of the performance and convergence is performed to study the reduced space approach.

math.OC↗

arXiv2019

High Performance Block Incomplete LU Factorization

Many application problems that lead to solving linear systems make use of preconditioned Krylov subspace solvers to compute their solution. Among the most popular preconditioning approaches are incomplete factorization methods either as single-level approaches or within a multilevel framework. We will present a block incomplete factorization that is based on skillfully blocking the system initially and throughout the factorization. This approach allows for the use of cache-optimized dense matrix kernels such as level-3 BLAS or LAPACK. We will demonstrate how this block approach outperforms the scalar method often by orders of magnitude on modern architectures, paving the way for its prospective use inside various multilevel incomplete factorization approaches or other applications where the core part relies on an incomplete factorization.

math.NA↗

arXiv2019

New frontiers in Bayesian modeling using the INLA package in R

The INLA package provides a tool for computationally efficient Bayesian modeling and inference for various widely used models, more formally the class of latent Gaussian models. It is a non-sampling based framework which provides approximate results for Bayesian inference, using sparse matrices. The swift uptake of this framework for Bayesian modeling is rooted in the computational efficiency of the approach and catalyzed by the demand presented by the big data era. In this paper, we present new developments within the INLA package with the aim to provide a computationally efficient mechanism for the Bayesian inference of relevant challenging situations.

stat.ME↗