arXiv subjects

Benno Kuckuck

Publications and source records attributed to Benno Kuckuck.

15 recordsLinked to original sources

arXiv2026

Finiteness Properties of Fibre Products over Virtually Nilpotent Quotients

The $n$-$(n+1)$-$(n+2)$ theorem, recently established by Cohen and Shusterman, says that if two groups of type $\mathrm{F}_{n+1}$ map onto a common quotient $Q$ of type $\mathrm{F}_{n+2}$ and one of the two kernels is of type $\mathrm{F}_n$, then the associated fibre product is of type $\mathrm{F}_{n+1}$. We prove a stronger, symmetric variant of this theorem when $Q$ is virtually nilpotent. More precisely, for $i\in\{1,2\}$, let $1\to N_i\to \Gamma_i\overset{\small\pi_i}{\to} Q\to 1$ be short exact sequences of groups and let $P=\{(\gamma_1,\gamma_2)\in\Gamma_1\times\Gamma_2\colon \pi_1(\gamma_1)=\pi_2(\gamma_2)\}$ be their fibre product. Let $k,l,m,n\in\mathbb N_0$, assume that $N_1$ and $N_2$ are of type $\mathrm{FP}_k$ and $\mathrm{FP}_l$, respectively, and that $\Gamma_1$ and $\Gamma_2$ are of type $\mathrm{F}_m$ and $\mathrm{F}_n$, respectively. We prove that $P$ is of type $\mathrm{F}_{\min\{k+l+1,m,n\}}$. The same formula holds with the homological finiteness properties $\mathrm{FP}_r$ in place of $\mathrm{F}_r$. This provides an alternative proof of the homotopical and homological Virtual Surjections Theorem, which gives a finiteness criterion for subgroups of direct products in terms of their embedding in the ambient product.

math.GR↗

arXiv2025

On the logical skills of large language models: evaluations using arbitrarily complex first-order logic problems

We present a method of generating first-order logic statements whose complexity can be controlled along multiple dimensions. We use this method to automatically create several datasets consisting of questions asking for the truth or falsity of first-order logic statements in Zermelo-Fraenkel set theory. While the resolution of these questions does not require any knowledge beyond basic notation of first-order logic and set theory, it does require a degree of planning and logical reasoning, which can be controlled up to arbitrarily high difficulty by the complexity of the generated statements. Furthermore, we do extensive evaluations of the performance of various large language models, including recent models such as DeepSeek-R1 and OpenAI's o3-mini, on these datasets. All of the datasets along with the code used for generating them, as well as all data from the evaluations is publicly available at https://github.com/bkuckuck/logical-skills-of-llms.

cs.LG↗

arXiv2024

An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning

The approximation of solutions of partial differential equations (PDEs) with numerical algorithms is a central topic in applied mathematics. For many decades, various types of methods for this purpose have been developed and extensively studied. One class of methods which has received a lot of attention in recent years are machine learning-based methods, which typically involve the training of artificial neural networks (ANNs) by means of stochastic gradient descent type optimization methods. While approximation methods for PDEs using ANNs have first been proposed in the 1990s they have only gained wide popularity in the last decade with the rise of deep learning. This article aims to provide an introduction to some of these methods and the mathematical theory on which they are based. We discuss methods such as physics-informed neural networks (PINNs) and deep BSDE methods and consider several operator learning approaches.

math.NA↗

arXiv2024

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations

It is a challenging topic in applied mathematics to solve high-dimensional nonlinear partial differential equations (PDEs). Standard approximation methods for nonlinear PDEs suffer under the curse of dimensionality (COD) in the sense that the number of computational operations of the approximation method grows at least exponentially in the PDE dimension and with such methods it is essentially impossible to approximately solve high-dimensional PDEs even when the fastest currently available computers are used. However, in the last years great progress has been made in this area of research through suitable deep learning (DL) based methods for PDEs in which deep neural networks (DNNs) are used to approximate solutions of PDEs. Despite the remarkable success of such DL methods in simulations, it remains a fundamental open problem of research to prove (or disprove) that such methods can overcome the COD in the approximation of PDEs. However, there are nowadays several partial error analysis results for DL methods for high-dimensional nonlinear PDEs in the literature which prove that DNNs can overcome the COD in the sense that the number of parameters of the approximating DNN grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $\varepsilon>0$ and the PDE dimension $d\in\mathbb{N}$. In the main result of this article we prove that for all $T,p\in(0,\infty)$ it holds that solutions $u_d\colon[0,T]\times\mathbb{R}^d\to\mathbb{R}$, $d\in\mathbb{N}$, of semilinear heat equations with Lipschitz continuous nonlinearities can be approximated in the $L^p$-sense on space-time regions without the COD by DNNs with the rectified linear unit (ReLU), the leaky ReLU, or the softplus activation function. In previous articles similar results have been established not for space-time regions but for the solutions $u_d(T,\cdot)$, $d\in\mathbb{N}$, at the terminal time $T$.

cs.LG↗

arXiv2023

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory

This book aims to provide an introduction to the topic of deep learning algorithms. We review essential components of deep learning algorithms in full mathematical detail including different artificial neural network (ANN) architectures (such as fully-connected feedforward ANNs, convolutional ANNs, recurrent ANNs, residual ANNs, and ANNs with batch normalization) and different optimization algorithms (such as the basic stochastic gradient descent (SGD) method, accelerated methods, and adaptive methods). We also cover several theoretical aspects of deep learning algorithms such as approximation capacities of ANNs (including a calculus for ANNs), optimization theory (including Kurdyka-{\L}ojasiewicz inequalities), and generalization errors. In the last part of the book some deep learning approximation methods for PDEs are reviewed including physics-informed neural networks (PINNs) and deep Galerkin methods. We hope that this book will be useful for students and scientists who do not yet have any background in deep learning at all and would like to gain a solid foundation as well as for practitioners who would like to obtain a firmer mathematical understanding of the objects and methods considered in deep learning.

cs.LG↗

arXiv2023

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for Kolmogorov partial differential equations with Lipschitz nonlinearities in the $L^p$-sense

Recently, several deep learning (DL) methods for approximating high-dimensional partial differential equations (PDEs) have been proposed. The interest that these methods have generated in the literature is in large part due to simulations which appear to demonstrate that such DL methods have the capacity to overcome the curse of dimensionality (COD) for PDEs in the sense that the number of computational operations they require to achieve a certain approximation accuracy $\varepsilon\in(0,\infty)$ grows at most polynomially in the PDE dimension $d\in\mathbb N$ and the reciprocal of $\varepsilon$. While there is thus far no mathematical result that proves that one of such methods is indeed capable of overcoming the COD, there are now a number of rigorous results in the literature that show that deep neural networks (DNNs) have the expressive power to approximate PDE solutions without the COD in the sense that the number of parameters used to describe the approximating DNN grows at most polynomially in both the PDE dimension $d\in\mathbb N$ and the reciprocal of the approximation accuracy $\varepsilon>0$. Roughly speaking, in the literature it is has been proved for every $T>0$ that solutions $u_d\colon [0,T]\times\mathbb R^d\to \mathbb R$, $d\in\mathbb N$, of semilinear heat PDEs with Lipschitz continuous nonlinearities can be approximated by DNNs with ReLU activation at the terminal time in the $L^2$-sense without the COD provided that the initial value functions $\mathbb R^d\ni x\mapsto u_d(0,x)\in\mathbb R$, $d\in\mathbb N$, can be approximated by ReLU DNNs without the COD. It is the key contribution of this work to generalize this result by establishing this statement in the $L^p$-sense with $p\in(0,\infty)$ and by allowing the activation function to be more general covering the ReLU, the leaky ReLU, and the softplus activation functions as special cases.

math.NA↗

arXiv2022

An overview on deep learning-based approximation methods for partial differential equations

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.

math.NA↗

arXiv2022

Deep learning approximations for non-local nonlinear PDEs with Neumann boundary conditions

Nonlinear partial differential equations (PDEs) are used to model dynamical processes in a large number of scientific fields, ranging from finance to biology. In many applications standard local models are not sufficient to accurately account for certain non-local phenomena such as, e.g., interactions at a distance. In order to properly capture these phenomena non-local nonlinear PDE models are frequently employed in the literature. In this article we propose two numerical methods based on machine learning and on Picard iterations, respectively, to approximately solve non-local nonlinear PDEs. The proposed machine learning-based method is an extended variant of a deep learning-based splitting-up type approximation method previously introduced in the literature and utilizes neural networks to provide approximate solutions on a subset of the spatial domain of the solution. The Picard iterations-based method is an extended variant of the so-called full history recursive multilevel Picard approximation scheme previously introduced in the literature and provides an approximate solution for a single point of the domain. Both methods are mesh-free and allow non-local nonlinear PDEs with Neumann boundary conditions to be solved in high dimensions. In the two methods, the numerical difficulties arising due to the dimensionality of the PDEs are avoided by (i) using the correspondence between the expected trajectory of reflected stochastic processes and the solution of PDEs (given by the Feynman-Kac formula) and by (ii) using a plain vanilla Monte Carlo integration to handle the non-local term. We evaluate the performance of the two methods on five different PDEs arising in physics and biology. In all cases, the methods yield good results in up to 10 dimensions with short run times. Our work extends recently developed methods to overcome the curse of dimensionality in solving PDEs.

math.NA↗

arXiv2021

Deep neural network approximation theory for high-dimensional functions

The purpose of this article is to develop a machinery to study the capacity of deep neural networks (DNNs) to approximate high-dimensional functions. In particular, we show that DNNs have the expressive power to overcome the curse of dimensionality in the approximation of a large class of functions. More precisely, we prove that these functions can be approximated by DNNs on compact sets such that the number of parameters necessary to represent the approximating DNNs grows at most polynomially in the reciprocal $1/\varepsilon$ of the prescribed approximation error $\varepsilon>0$ and in the input dimension $d\in\mathbb N$. To this end, we introduce certain approximation spaces, consisting of sequences of functions that can be efficiently approximated by DNNs. We then establish closure properties which we combine with known and new bounds on the number of parameters necessary to approximate locally Lipschitz continuous functions, maximum functions, and product functions by DNNs. The main result of this article demonstrates that DNNs have sufficient expressive power to approximate, without the curse of dimensionality, certain sequences of functions which can be constructed by means of a finite number of compositions using locally Lipschitz continuous functions, maxima, and products.

math.NA↗

arXiv2021

Strong $L^p$-error analysis of nonlinear Monte Carlo approximations for high-dimensional semilinear partial differential equations

Full-history recursive multilevel Picard (MLP) approximation schemes have been shown to overcome the curse of dimensionality in the numerical approximation of high-dimensional semilinear partial differential equations (PDEs) with general time horizons and Lipschitz continuous nonlinearities. However, each of the error analyses for MLP approximation schemes in the existing literature studies the $L^2$-root-mean-square distance between the exact solution of the PDE under consideration and the considered MLP approximation and none of the error analyses in the existing literature provides an upper bound for the more general $L^p$-distance between the exact solution of the PDE under consideration and the considered MLP approximation. It is the key contribution of this article to extend the $L^2$-error analysis for MLP approximation schemes in the literature to a more general $L^p$-error analysis with $p\in (0,\infty)$. In particular, the main result of this article proves that the proposed MLP approximation scheme indeed overcomes the curse of dimensionality in the numerical approximation of high-dimensional semilinear PDEs with the approximation error measured in the $L^p$-sense with $p \in (0,\infty)$.

math.NA↗

arXiv2020

Full error analysis for the training of deep neural networks

Deep learning algorithms have been applied very successfully in recent years to a range of problems out of reach for classical solution paradigms. Nevertheless, there is no completely rigorous mathematical error and convergence analysis which explains the success of deep learning algorithms. The error of a deep learning algorithm can in many situations be decomposed into three parts, the approximation error, the generalization error, and the optimization error. In this work we estimate for a certain deep learning algorithm each of these three errors and combine these three error estimates to obtain an overall error analysis for the deep learning algorithm under consideration. In particular, we thereby establish convergence with a suitable convergence speed for the overall error of the deep learning algorithm under consideration. Our convergence speed analysis is far from optimal and the convergence speed that we establish is rather slow, increases exponentially in the dimensions, and, in particular, suffers from the curse of dimensionality. The main contribution of this work is, instead, to provide a full error analysis (i) which covers each of the three different sources of errors usually emerging in deep learning algorithms and (ii) which merges these three sources of errors into one overall error estimate for the considered deep learning algorithm.

math.NA↗

arXiv2020

Counterexamples to local Lipschitz and local Hölder continuity with respect to the initial values for additive noise driven SDEs with smooth drift coefficient functions with at most polynomially growing derivatives

In the recent article [A. Jentzen, B. Kuckuck, T. Müller-Gronbach, and L. Yaroslavtseva, arXiv:1904.05963 (2019)] it has been proved that the solutions to every additive noise driven stochastic differential equation (SDE) which has a drift coefficient function with at most polynomially growing first order partial derivatives and which admits a Lyapunov-type condition (ensuring the the existence of a unique solution to the SDE) depend in a logarithmically Hölder continuous way on their initial values. One might then wonder whether this result can be sharpened and whether in fact, SDEs from this class necessarily have solutions which depend locally Lipschitz continuously on their initial value. The key contribution of this article is to establish that this is not the case. More precisely, we supply a family of examples of additive noise driven SDEs which have smooth drift coefficient functions with at most polynomially growing derivatives whose solutions do not depend on their initial value in a locally Lipschitz continuous, nor even in a locally Hölder continuous way.

math.PR↗

arXiv2019

On the strong regularity of degenerate additive noise driven stochastic differential equations with respect to their initial values

Recently in [M. Hairer, M. Hutzenthaler, and A. Jentzen, Ann. Probab. 43, 2 (2015), 468--527] and [A. Jentzen, T. Müller-Gronbach, and L. Yaroslavtseva, Commun. Math. Sci. 14, 6 (2016), 1477--1500] stochastic differential equations (SDEs) with smooth coefficient functions have been constructed which have an arbitrarily slowly converging modulus of continuity in the initial value. In these SDEs it is crucial that some of the first order partial derivatives of the drift coefficient functions grow at least exponentially and, in particular, quicker than any polynomial. However, in applications SDEs do typically have coefficient functions whose first order partial derivatives are polynomially bounded. In this article we study whether arbitrarily bad regularity phenomena in the initial value may also arise in the latter case and we partially answer this question in the negative. More precisely, we show that every additive noise driven SDE which admits a Lyapunov-type condition (which ensures the existence of a unique solution of the SDE) and which has a drift coefficient function whose first order partial derivatives grow at most polynomially is at least logarithmically Hölder continuous in the initial value.

math.PR↗

arXiv2018

Strong error analysis for stochastic gradient descent optimization algorithms

Stochastic gradient descent (SGD) optimization algorithms are key ingredients in a series of machine learning applications. In this article we perform a rigorous strong error analysis for SGD optimization algorithms. In particular, we prove for every arbitrarily small $\varepsilon \in (0,\infty)$ and every arbitrarily large $p\in (0,\infty)$ that the considered SGD optimization algorithm converges in the strong $L^p$-sense with order $\frac{1}{2}-\varepsilon$ to the global minimum of the objective function of the considered stochastic approximation problem under standard convexity-type assumptions on the objective function and relaxed assumptions on the moments of the stochastic errors appearing in the employed SGD optimization algorithm. The key ideas in our convergence proof are, first, to employ techniques from the theory of Lyapunov-type functions for dynamical systems to develop a general convergence machinery for SGD optimization algorithms based on such functions, then, to apply this general machinery to concrete Lyapunov-type functions with polynomial structures, and, thereafter, to perform an induction argument along the powers appearing in the Lyapunov-type functions in order to achieve for every arbitrarily large $ p \in (0,\infty) $ strong $ L^p $-convergence rates. This article also contains an extensive review of results on SGD optimization algorithms in the scientific literature.

math.NA↗

arXiv2013

Subdirect products of groups and the n-(n+1)-(n+2) Conjecture

We analyse the subgroup structure of direct products of groups. Earlier work on this topic has revealed that higher finiteness properties play a crucial role in determining which groups appear as subgroups of direct products of free groups or limit groups. Here, we seek to relate the finiteness properties of a subgroup to the way it is embedded in the ambient product. To this end we formulate a conjecture on finiteness properties of fibre products of groups. We present different approaches to this conjecture, proving a general result on finite generation of homology groups of fibre products and, for certain special cases, results on the stronger finiteness properties F_n and FP_n.

math.GR↗