arXiv subjects

Roland Herzog

Publications and source records attributed to Roland Herzog.

At least 19 recordsLinked to original sources

arXiv2026

The Riemannian Convex Bundle Method

We introduce the convex bundle method to solve convex, non-smooth optimization problems on Riemannian manifolds of bounded sectional curvature. Each step of our method is based on a model that involves the convex hull of previously collected subgradients, parallelly transported into the current serious iterate. This approach generalizes the dual form of classical bundle subproblems in Euclidean space. We prove that, under mild conditions, the convex bundle method converges to a minimizer. Several numerical examples implemented using Manopt$.$jl illustrate the performance of the proposed method and compare it to the subgradient method, the cyclic proximal point algorithm, as well as the proximal bundle method.

math.OC↗

arXiv2026

A DC-Reformulation for Gradient-$L^0$-Constrained Problems

Cardinality constraints in optimization are commonly of $L^0$-type, and they lead to sparsely supported optimizers. An efficient way of dealing with these constraints algorithmically, when the objective functional is convex, is reformulating the constraint using the difference of suitable $L^1$- and largest-$K$-norms and subsequently solving a sequence of penalized subproblems in the difference-of-convex (DC) class. We extend this DC-reformulation approach to problems with $L^0$-type cardinality constraints on the support of the gradients, \ie, problems where sparsity of the gradient and thus piecewise constant solutions are the target. We apply the resulting algorithm to a classical energy minimization problem, an optimal control problem, and an image reconstruction problem.

math.OC↗

arXiv2026

Geometry Denoising with Preferred Normal Vectors

We introduce a new paradigm for geometry denoising using prior knowledge about the surface normal vector. This prior knowledge comes in the form of a set of preferred normal vectors, which we refer to as label vectors. A segmentation problem is naturally embedded in the denoising process. The segmentation is based on the similarity of the normal vector to the elements of the set of label vectors. Regularization is achieved by a total variation term. We formulate a split Bregman (ADMM) approach to solve the resulting optimization problem. The vertex update step is based on second-order shape calculus. We present various examples including the denoising of an eroded medieval gravestone inscription.

cs.CV↗

arXiv2026

Symbolic Regression for Shared Expressions: Introducing Partial Parameter Sharing

Symbolic regression aims to find symbolic expressions that describe datasets. Due to its inherent interpretability, symbolic regression (SR) is a powerful paradigm for scientific discovery. Recent advances have expanded SR to describe related phenomena using a single expression with varying sets of parameters, thereby introducing a single categorical variable. To illustrate, this enables the search for a single expression describing temperaturedependent viscosity across multiple fluids, while simultaneously identifying a distinct set of fluid-specific parameters. We expand upon prior efforts by considering multiple categorical variables and introducing intermediate levels of parameter sharing. Rather than parameters being either entirely universal or entirely unique, some parameters can also be shared across specific categories while remaining distinct for others. This allows for separating universal effects (shared parameters), category-specific trends (partially-shared parameters), and category interactions (non-shared parameters). We test the limits of this setup in terms of reducing data requirements and transfer learning using a synthetic, fitting-only example. Furthermore, we apply the method to an astrophysics dataset also used in a previous single-category study. In comparison, we achieve similar fit quality with significantly fewer parameters while extracting additional information about the problem.

cs.LG↗

arXiv2026

Subspace accelerated measure transport methods for fast and scalable sequential experimental design, with application to photoacoustic imaging

We propose a novel approach for sequential optimal experimental design (sOED) for Bayesian inverse problems involving expensive models with high-dimensional unknown parameters. This work focuses on designs that maximize the expected information gain (EIG) from prior to posterior, a task that is computationally very challenging in non-Gaussian settings. This challenge is amplified in sOED, as the incremental expected information gain (iEIG) must be repeatedly approximated across distinct stages, with both prior and posterior distributions being intractable. To address this, we derive a general-purpose, derivative-based upper bound for the iEIG, which not only guides design placement but also enables the construction of projectors onto likelihood-informed subspaces, facilitating parameter dimension reduction. By combining this approach with conditional measure transport maps for the sequence of posteriors, we develop a unified sOED and amortized inference framework scalable to high- and infinite-dimensional problems. Numerical experiments for two inverse problems governed by partial differential equations (PDEs) demonstrate the effectiveness of designs by maximizing the proposed bound.

math.OC↗

arXiv2026

Two Models for Surface Segmentation using the Total Variation of the Normal Vector

We consider the problem of surface segmentation, where the goal is to partition a surface represented by a triangular mesh. The segmentation is based on the similarity of the normal vector field to a given set of label vectors. We propose a variational approach and compare two different regularizers, both based on a total variation measure. The first regularizer penalizes the total variation of the assignment function directly, while the second regularizer penalizes the total variation in the label space. In order to solve the resulting optimization problems, we use variations of the split Bregman (ADMM) iteration adapted to the problem at hand. While computationally more expensive, the second regularizer yields better results in our experiments. In particular it removes noise more reliably in regions of constant curvature. In order to mitigate the computational cost, we present a manifold Newton scheme for the most expensive subproblem, which is related to the Riemannian center of mass on a sphere. This significantly improves the computational cost.

cs.CV↗

arXiv2026

Modeling and Simulation of Device Performance in Organic Photovoltaics

We present a pipeline to study the device performance of organic solar cells in silico. We introduce a mathematical model that includes the dynamics of excitons as well as their dissociation at bulk heterojunctions within the nanomorphology of the active layer. This is combined with realistic morphologies that we obtain from a detailed phase field model. To solve the coupled nonlinear system, we use a finite element discretization, robust linear solvers, and three numerical schemes, Newton, Gummel, and Semi--Newton--Gummel. This allows for an efficient simulation of the complete OPV device and results in current-voltage curves that can readily be compared to measured data.

math.NA↗

arXiv2025

Total Generalized Variation of the Normal Vector Field and Applications to Mesh Denoising

We propose a novel formulation for the second-order total generalized variation (TGV) of the normal vector on an oriented, triangular mesh embedded in $\R^3$. The normal vector is considered as a manifold-valued function, taking values on the unit sphere. Our formulation extends previous discrete TGV models for piecewise constant scalar data that utilize a Raviart-Thomas function space. To extend this formulation to the manifold setting, a tailor-made tangential Raviart-Thomas type finite element space is constructed in this work. The new regularizer is compared to existing methods in mesh denoising experiments.

cs.CV↗

arXiv2025

SensLI: Sensitivity-Based Layer Insertion for Neural Networks

The training of neural networks requires tedious and often manual tuning of the network architecture. We propose a systematic approach to inserting new layers during the training process. Our method eliminates the need to choose a fixed network size before training, is numerically inexpensive to execute and applicable to various architectures including fully connected feedforward networks, ResNets and CNNs. Our technique borrows ideas from constrained optimization and is based on first-order sensitivity information of the loss function with respect to the virtual parameters that additional layers, if inserted, would offer. In numerical experiments, our proposed sensitivity-based layer insertion technique (SensLI) exhibits improved performance on training loss and test error, compared to training on a fixed architecture, and reduced computational effort in comparison to training the extended architecture from the beginning. Our code is available on https://github.com/mathemml/SensLI.

cs.LG↗

arXiv2025

Preconditioning for a Cahn-Hilliard-Navier-Stokes model for morphology formation in organic solar cells

We present a model for the morphology evolution of printed organic solar cells which occurs during the drying of a mixture of polymer, the non-fullerene acceptor and the solvent. Our model uses a phase field approach coupled to a Navier-Stokes equation describing the macroscopic movement of the fluid. Additionally, we incorporate the evaporation process of the solvent using an Allen-Cahn equation. The model is discretized using a finite-element approach with a semi-implicit discretization in time. The resulting (non)linear systems are coupled and of large dimensionality. We present a preconditioned iterative scheme to solve them robustly with respect to changes in the discretization parameters. We illustrate that the preconditioned solver shows parameter-robust iteration numbers and that the model qualitatively captures the behavior of the film morphology during drying.

math.NA↗

arXiv2024

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent

This paper proposes a novel approach to adaptive step sizes in stochastic gradient descent (SGD) by utilizing quantities that we have identified as numerically traceable -- the Lipschitz constant for gradients and a concept of the local variance in search directions. Our findings yield a nearly hyperparameter-free algorithm for stochastic optimization, which has provable convergence properties and exhibits truly problem adaptive behavior on classical image classification tasks. Our framework is set in a general Hilbert space and thus enables the potential inclusion of a preconditioner through the choice of the inner product.

math.OC↗

arXiv2024

Nonlinear Fenchel Conjugates

The classical concept of Fenchel conjugation is tailored to extended real-valued functions defined on linear spaces. In this paper we generalize this concept to functions defined on arbitrary sets that do not necessarily bear any structure at all. This generalization is obtained by replacing linear test functions by general nonlinear ones. Thus, we refer to it as nonlinear Fenchel conjugation. We investigate elementary properties including the Fenchel-Moreau biconjugation theorem. Whenever the domain exhibits additional structure, the restriction to a suitable subset of test functions allows further results to be derived. For example, on smooth manifolds, the restriction to smooth test functions allows us to state the Fenchel-Young theorem for the viscosity Fréchet subdifferential. On Lie groups, the restriction to real-valued group homomorphisms relates nonlinear Fenchel conjugation to infimal convolution and yields a notion of convexity.

math.FA↗

arXiv2024

Tractable Optimal Experimental Design using Transport Maps

We present a flexible method for computing Bayesian optimal experimental designs (BOEDs) for inverse problems with intractable posteriors. The approach is applicable to a wide range of BOED problems and can accommodate various optimality criteria, prior distributions and noise models. The key to our approach is the construction of a transport-map-based surrogate to the joint probability law of the design, observational and inference random variables. This order-preserving transport map is constructed using tensor trains and can be used to efficiently sample from (and evaluate approximate densities of) conditional distributions that are required in the evaluation of many commonly-used optimality criteria. The algorithm is also extended to sequential data acquisition problems, where experiments can be performed in sequence to update the state of knowledge about the unknown parameters. The sequential BOED problem is made computationally feasible by preconditioning the approximation of the joint density at the current stage using transport maps constructed at previous stages. The flexibility of our approach in finding optimal designs is illustrated with some numerical examples inspired by disease modeling and the reconstruction of subsurface structures in aquifers.

stat.CO↗

arXiv2024

Shape Constraints in Symbolic Regression using Penalized Least Squares

We study the addition of shape constraints (SC) and their consideration during the parameter identification step of symbolic regression (SR). SC serve as a means to introduce prior knowledge about the shape of the otherwise unknown model function into SR. Unlike previous works that have explored SC in SR, we propose minimizing SC violations during parameter identification using gradient-based numerical optimization. We test three algorithm variants to evaluate their performance in identifying three symbolic expressions from synthetically generated data sets. This paper examines two benchmark scenarios: one with varying noise levels and another with reduced amounts of training data. The results indicate that incorporating SC into the expression search is particularly beneficial when data is scarce. Compared to using SC only in the selection process, our approach of minimizing violations during parameter identification shows a statistically significant benefit in some of our test cases, without being significantly worse in any instance.

cs.LG↗

arXiv2024

Unit-Aware Genetic Programming for the Development of Empirical Equations

When developing empirical equations, domain experts require these to be accurate and adhere to physical laws. Often, constants with unknown units need to be discovered alongside the equations. Traditional unit-aware genetic programming (GP) approaches cannot be used when unknown constants with undetermined units are included. This paper presents a method for dimensional analysis that propagates unknown units as ''jokers'' and returns the magnitude of unit violations. We propose three methods, namely evolutive culling, a repair mechanism, and a multi-objective approach, to integrate the dimensional analysis in the GP algorithm. Experiments on datasets with ground truth demonstrate comparable performance of evolutive culling and the multi-objective approach to a baseline without dimensional analysis. Extensive analysis of the results on datasets without ground truth reveals that the unit-aware algorithms make only low sacrifices in accuracy, while producing unit-adherent solutions. Overall, we presented a promising novel approach for developing unit-adherent empirical equations.

cs.LG↗

arXiv2024

Mesh Denoising and Inpainting using the Total Variation of the Normal and a Shape Newton Approach

We present a novel approach to denoising and inpainting problems for surface meshes. The purpose of these problems is to remove noise or fill in missing parts while preserving important features such as sharp edges. A discrete variant of the total variation of the unit normal vector field serves as a regularizing functional to achieve these goals. In order to solve the resulting problem, we use a version of the split Bregman (ADMM) iteration adapted to the problem. A new formulation of the total variation regularizer, as well as the use of an inexact Newton method for the shape optimization step, bring significant speed-up compared to earlier methods. Numerical examples are included, demonstrating the performance of our algorithm with some complex 3D geometries.

math.NA↗

arXiv2023

Time Regularization in Optimal Time Variable Learning

Recently, optimal time variable learning in deep neural networks (DNNs) was introduced in arXiv:2204.08528. In this manuscript we extend the concept by introducing a regularization term that directly relates to the time horizon in discrete dynamical systems. Furthermore, we propose an adaptive pruning approach for Residual Neural Networks (ResNets), which reduces network complexity without compromising expressiveness, while simultaneously decreasing training time. The results are illustrated by applying the proposed concepts to classification tasks on the well known MNIST and Fashion MNIST data sets. Our PyTorch code is available on https://github.com/frederikkoehne/time_variable_learning.

cs.LG↗

arXiv2023

Frobenius-Type Norms and Inner Products of Matrices and Linear Maps with Applications to Neural Network Training

The Frobenius norm is a frequent choice of norm for matrices. In particular, the underlying Frobenius inner product is typically used to evaluate the gradient of an objective with respect to matrix variable, such as those occuring in the training of neural networks. We provide a broader view on the Frobenius norm and inner product for linear maps or matrices, and establish their dependence on inner products in the domain and co-domain spaces. This shows that the classical Frobenius norm is merely one special element of a family of more general Frobenius-type norms. The significant extra freedom furnished by this realization can be used, among other things, to precondition neural network training.

cs.LG↗