arXiv subjects

Christoph Hertrich

Publications and source records attributed to Christoph Hertrich.

25 records · Page 2Linked to original sources

arXiv2024

Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete

We consider the problem of finding weights and biases for a two-layer fully connected neural network to fit a given set of data points as well as possible, also known as EmpiricalRiskMinimization. Our main result is that the associated decision problem is $\exists\mathbb{R}$-complete, that is, polynomial-time equivalent to determining whether a multivariate polynomial with integer coefficients has any real roots. Furthermore, we prove that algebraic numbers of arbitrarily large degree are required as weights to be able to train some instances to optimality, even if all data points are rational. Our result already applies to fully connected instances with two inputs, two outputs, and one hidden layer of ReLU neurons. Thereby, we strengthen a result by Abrahamsen, Kleist and Miltzow [NeurIPS 2021]. A consequence of this is that a combinatorial search algorithm like the one by Arora, Basu, Mianjy and Mukherjee [ICLR 2018] is impossible for networks with more than one output dimension, unless $\mathsf{NP}=\exists\mathbb{R}$.

cs.CC↗

arXiv2024

Training Neural Networks is NP-Hard in Fixed Dimension

We study the parameterized complexity of training two-layer neural networks with respect to the dimension of the input data and the number of hidden neurons, considering ReLU and linear threshold activation functions. Albeit the computational complexity of these problems has been studied numerous times in recent years, several questions are still open. We answer questions by Arora et al. [ICLR '18] and Khalife and Basu [IPCO '22] showing that both problems are NP-hard for two dimensions, which excludes any polynomial-time algorithm for constant dimension. We also answer a question by Froese et al. [JAIR '22] proving W[1]-hardness for four ReLUs (or two linear threshold neurons) with zero training error. Finally, in the ReLU case, we show fixed-parameter tractability for the combined parameter number of dimensions and number of ReLUs if the network is assumed to compute a convex map. Our results settle the complexity status regarding these parameters almost completely.

cs.CC↗

arXiv2023

Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes

We prove that the set of functions representable by ReLU neural networks with integer weights strictly increases with the network depth while allowing arbitrary width. More precisely, we show that $\lceil\log_2(n)\rceil$ hidden layers are indeed necessary to compute the maximum of $n$ numbers, matching known upper bounds. Our results are based on the known duality between neural networks and Newton polytopes via tropical geometry. The integrality assumption implies that these Newton polytopes are lattice polytopes. Then, our depth lower bounds follow from a parity argument on the normalized volume of faces of such polytopes.

cs.LG↗

arXiv2022

Coloring Drawings of Graphs

We consider cell colorings of drawings of graphs in the plane. Given a multi-graph $G$ together with a drawing $Γ(G)$ in the plane with only finitely many crossings, we define a cell $k$-coloring of $Γ(G)$ to be a coloring of the maximal connected regions of the drawing, the cells, with $k$ colors such that adjacent cells have different colors. By the $4$-color theorem, every drawing of a bridgeless graph has a cell $4$-coloring. A drawing of a graph is cell $2$-colorable if and only if the underlying graph is Eulerian. We show that every graph without degree 1 vertices admits a cell $3$-colorable drawing. This leads to the natural question which abstract graphs have the property that each of their drawings has a cell $3$-coloring. We say that such a graph is universally cell $3$-colorable. We show that every $4$-edge-connected graph and every graph admitting a nowhere-zero $3$-flow is universally cell $3$-colorable. We also discuss circumstances under which universal cell $3$-colorability guarantees the existence of a nowhere-zero $3$-flow. On the negative side, we present an infinite family of universally cell $3$-colorable graphs without a nowhere-zero $3$-flow. On the positive side, we formulate a conjecture which has a surprising relation to a famous open problem by Tutte known as the $3$-flow-conjecture. We prove our conjecture for subcubic and for $K_{3,3}$-minor-free graphs.

math.CO↗

arXiv2022

The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality

Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-layer ReLU networks with respect to various loss functions. After a brief discussion of other parameters, we focus on analyzing the influence of the dimension $d$ of the training data on the computational complexity. We provide running time lower bounds in terms of W[1]-hardness for parameter $d$ and prove that known brute-force strategies are essentially optimal (assuming the Exponential Time Hypothesis). In comparison with previous work, our results hold for a broad(er) range of loss functions, including $\ell^p$-loss for all $p\in[0,\infty]$. In particular, we extend a known polynomial-time algorithm for constant $d$ and convex loss functions to a more general class of loss functions, matching our running time lower bounds also in these cases.

cs.LG↗

arXiv2020

Scheduling a Proportionate Flow Shop of Batching Machines

In this paper we study a proportionate flow shop of batching machines with release dates and a fixed number $m \geq 2$ of machines. The scheduling problem has so far barely received any attention in the literature, but recently its importance has increased significantly, due to applications in the industrial scaling of modern bio-medicine production processes. We show that for any fixed number of machines, the makespan and the sum of completion times can be minimized in polynomial time. Furthermore, we show that the obtained algorithm can also be used to minimize the weighted total completion time, maximum lateness, total tardiness and (weighted) number of late jobs in polynomial time if all release dates are $0$. Previously, polynomial time algorithms have only been known for two machines.

cs.DS↗

arXiv2019

Sweep Algorithms for the Capacitated Vehicle Routing Problem with Structured Time Window

The capacitated Vehicle Routing Problem with structured Time Windows (cVRPsTW) is concerned with finding optimal tours for vehicles with given capacity constraints to deliver goods to customers within assigned time windows. In our problem variant these time windows have a special structure, namely they are non-overlapping and each time window holds several customers. This is a reasonable assumption for Attended Home Delivery services. Sweep algorithms are known as simple, yet effective heuristics for the classical capacitated Vehicle Routing Problem. We propose variants of the sweep algorithm that are not only able to deal with time windows, but also exploit the additional structure of the time windows in a cVRPsTW. Afterwards we suggest local improvement heuristics to decrease our objective function even further. A carefully constructed benchmark set that resembles real-world data is used to prove the efficacy of our algorithms in a computational study.

math.OC↗