arXiv subjects

Christoph Spiegel

Publications and source records attributed to Christoph Spiegel.

At least 19 recordsLinked to original sources

arXiv2026

Limits of combinatorial patchworking

It is shown that there are real plane algebraic curves of degree eight that cannot be realized as T-curves, i.e., via combinatorial patchworking. In fact, this holds for several real schemes (i.e., ambient isotopy types) with the maximal number of real components, called $M$-curves. On the other hand, each nonempty real scheme of lower degree, maximal or not, arises as a T-curve. By constructing one patchwork of the dilated triangle $d\cdotΔ_2$ for each nonempty real scheme of degree $d\leq 7$, we provide an explicit method for constructing polynomials realizing these real schemes. This resolves a question of Itenberg and Viro (1996).

math.AG↗

arXiv2026

An Unsure Note on an Un-Schur Problem

Graham, Rödl, and Ruciński originally posed the problem of determining the minimum number of monochromatic Schur triples that must appear in any 2-coloring of the first $n$ integers. This question was subsequently resolved independently by Datskovsky, Schoen, and Robertson and Zeilberger. Here we suggest studying a natural anti-Ramsey variant of this question and establish the first non-trivial bounds by proving that the maximum fraction of Schur triples that can be rainbow in a given $3$-coloring of the first $n$ integers is at least $0.4$ and at most $0.66364$. We conjecture the lower bound to be tight. This question is also motivated by a famous analogous problem in graph theory due to Erdős and Sós regarding the maximum number of rainbow triangles in any $3$-coloring of $K_n$, which was settled by Balogh et al.

math.CO↗

arXiv2026

Fast Isotopy Computation for T-Curves

A T-curve of degree $d$ is given by a regular unimodular triangulation of $d \cdot Δ_2$ together with a sign distribution on its lattice points. By Viro's Patchworking Theorem, this determines the ambient isotopy type (a.k.a. real scheme) of a smooth real plane projective algebraic curve of the same degree. We present a near-quadratic time algorithm for extracting that isotopy type from the triangulation and the signs. Through a GPU-accelerated implementation, this allows one to compute billions of real schemes per second, enabling exhaustive enumeration at scale. This algorithm was essential for our recent construction of all 121 real schemes of degree seven by T-curves.

math.AG↗

arXiv2025

PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs

Neural Networks can be effectively compressed through pruning, significantly reducing storage and compute demands while maintaining predictive performance. Simple yet effective methods like magnitude pruning remove less important parameters and typically require a costly retraining procedure to restore performance. However, with the rise of LLMs, full retraining has become infeasible due to memory and compute constraints. This study challenges the practice of retraining all parameters by showing that updating a small subset of highly expressive parameters can suffice to recover or even enhance performance after pruning. Surprisingly, retraining just 0.01%-0.05% of the parameters in GPT-architectures can match the performance of full retraining across various sparsity levels, significantly reducing compute and memory requirements, and enabling retraining of models with up to 30 billion parameters on a single GPU in minutes. To bridge the gap to full retraining in the high sparsity regime, we introduce two novel LoRA variants that, unlike standard LoRA, allow merging adapters back without compromising sparsity. Going a step further, we show that these methods can be applied for memory-efficient layer-wise reconstruction, significantly enhancing state-of-the-art retraining-free methods like Wanda (Sun et al., 2023) and SparseGPT (Frantar & Alistarh, 2023). Our findings present a promising alternative to avoiding retraining.

cs.LG↗

arXiv2025

Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers

Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condition for nonnegativity is the Sum of Squares (SOS) property, i.e., it can be written as a sum of squares of other polynomials. In practice, however, certifying the SOS criterion remains computationally expensive and often involves solving a Semidefinite Program (SDP), whose dimensionality grows quadratically in the size of the monomial basis of the SOS expression; hence, various methods to reduce the size of the monomial basis have been proposed. In this work, we introduce the first learning-augmented algorithm to certify the SOS criterion. To this end, we train a Transformer model that predicts an almost-minimal monomial basis for a given polynomial, thereby drastically reducing the size of the corresponding SDP. Our overall methodology comprises three key components: efficient training dataset generation of over 100 million SOS polynomials, design and training of the corresponding Transformer architecture, and a systematic fallback mechanism to ensure correct termination, which we analyze theoretically. We validate our approach on over 200 benchmark datasets, achieving speedups of over $100\times$ compared to state-of-the-art solvers and enabling the solution of instances where competing approaches fail. Our findings provide novel insights towards transforming the practical scalability of SOS programming.

cs.LG↗

arXiv2025

Infinitely many counterexamples to a conjecture of Lovász

Motivated by the well-known conjecture of Ryser which relates maximum matchings to minimum vertex covers in $r$-partite $r$-uniform hypergraphs, Lovász formulated a stronger conjecture. It states that one can always reduce the matching number by removing $r-1$ vertices. This conjecture was very recently disproven for $r=3$ by Clow, Haxell, and Mohar using the line graph of a $3$-regular graph of order $102$. Building on this, we describe a simple infinite family of counterexamples based on generalized Petersen graphs for the case $r=3$ and give specific counterexamples for $r=4$.

math.CO↗

arXiv2025

Neural Discovery in Mathematics: Do Machines Dream of Colored Planes?

We demonstrate how neural networks can drive mathematical discovery through a case study of the Hadwiger-Nelson problem, a long-standing open problem at the intersection of discrete geometry and extremal combinatorics that is concerned with coloring the plane while avoiding monochromatic unit-distance pairs. Using neural networks as approximators, we reformulate this mixed discrete-continuous geometric coloring problem with hard constraints as an optimization task with a probabilistic, differentiable loss function. This enables gradient-based exploration of admissible configurations that most significantly led to the discovery of two novel six-colorings, providing the first improvement in thirty years to the off-diagonal variant of the original problem. Here, we establish the underlying machine learning approach used to obtain these results and demonstrate its broader applicability through additional numerical insights.

cs.LG↗

arXiv2024

Forcing Graphs to be Forcing

Sidorenko's conjecture states that the number of copies of any given bipartite graph in another graph of given density is asymptotically minimized by a random graph. The forcing conjecture further strengthens this, claiming that any minimizer in fact needs to be quasi-random. Here we extend the family of bipartite graphs for which the forcing conjecture is known to hold to include balanced blow-ups of Sidorenko graphs and subdivisions of Sidorenko graphs by a forcing graph. This partially generalizes results by Conlon et al. (2018) and Conlon and Lee (2021). We also show that the box product of a Sidorenko graph with an edge is forcing, partially generalizing results of Kim, Lee, and Lee (2016) and, in particular, showing that cubes are forcing. We achieve these results through algebraic arguments building on Razborov's flag algebra framework (2007). This approach additionally allows us to construct Sidorenko hypergraphs from known 2-uniform Sidorenko graphs and to study forcing pairs.

math.CO↗

arXiv2024

New Ramsey Multiplicity Bounds and Search Heuristics

We study two related problems concerning the number of homogeneous subsets of given size in graphs that go back to questions of Erdős. Most notably, we improve the upper bounds on the Ramsey multiplicity of $K_4$ and $K_5$ and settle the minimum number of independent sets of size $4$ in graphs with clique number at most $4$. Motivated by the elusiveness of the symmetric Ramsey multiplicity problem, we also introduce an off-diagonal variant and obtain tight results when counting monochromatic $K_4$ or $K_5$ in only one of the colors and triangles in the other. The extremal constructions for each problem turn out to be blow-ups of a graph of constant size and were found through search heuristics. They are complemented by lower bounds established using flag algebras, resulting in a fully computer-assisted approach. For some of our theorems we can also derive that the extremal construction is stable in a very strong sense. More broadly, these problems lead us to the study of the region of possible pairs of clique and independent set densities that can be realized as the limit of some sequence of graphs.

math.CO↗

arXiv2024

Extending the Continuum of Six-Colorings

We present two novel six-colorings of the Euclidean plane that avoid monochromatic pairs of points at unit distance in five colors and monochromatic pairs at another specified distance $d$ in the sixth color. Such colorings have previously been known to exist for $0.41 < \sqrt{2} - 1 \le d \le 1 / \sqrt{5} < 0.45$. Our results significantly expand that range to $0.354 \le d \le 0.657$, the first improvement in 30 years. Notably, the constructions underlying this were derived by formalizing colorings suggested by a custom machine learning approach.

math.CO↗

arXiv2024

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

Neural networks can be significantly compressed by pruning, yielding sparse models with reduced storage and computational demands while preserving predictive performance. Model soups (Wortsman et al., 2022) enhance generalization and out-of-distribution (OOD) performance by averaging the parameters of multiple models into a single one, without increasing inference time. However, achieving both sparsity and parameter averaging is challenging as averaging arbitrary sparse models reduces the overall sparsity due to differing sparse connectivities. This work addresses these challenges by demonstrating that exploring a single retraining phase of Iterative Magnitude Pruning (IMP) with varied hyperparameter configurations such as batch ordering or weight decay yields models suitable for averaging, sharing identical sparse connectivity by design. Averaging these models significantly enhances generalization and OOD performance over their individual counterparts. Building on this, we introduce Sparse Model Soups (SMS), a novel method for merging sparse models by initiating each prune-retrain cycle with the averaged model from the previous phase. SMS preserves sparsity, exploits sparse network benefits, is modular and fully parallelizable, and substantially improves IMP's performance. We further demonstrate that SMS can be adapted to enhance state-of-the-art pruning-during-training approaches.

cs.LG↗

arXiv2024

Compression-aware Training of Neural Networks using Frank-Wolfe

Many existing Neural Network pruning approaches rely on either retraining or inducing a strong bias in order to converge to a sparse solution throughout training. A third paradigm, 'compression-aware' training, aims to obtain state-of-the-art dense models that are robust to a wide range of compression ratios using a single dense training run while also avoiding retraining. We propose a framework centered around a versatile family of norm constraints and the Stochastic Frank-Wolfe (SFW) algorithm that encourage convergence to well-performing solutions while inducing robustness towards convolutional filter pruning and low-rank matrix decomposition. Our method is able to outperform existing compression-aware approaches and, in the case of low-rank matrix decomposition, it also requires significantly less computational resources than approaches based on nuclear-norm regularization. Our findings indicate that dynamically adjusting the learning rate of SFW, as suggested by Pokutta et al. (2020), is crucial for convergence and robustness of SFW-trained models and we establish a theoretical foundation for that practice.

cs.LG↗

arXiv2023

The Four-Color Ramsey Multiplicity of Triangles

We study a generalization of a famous result of Goodman and establish that asymptotically at least a $1/256$ fraction of all triangles needs to be monochromatic in any four-coloring of the edges of a complete graph. We also show that any large enough extremal construction must be based on a blow-up of one of the two $R(3,3,3)$ Ramsey-colorings of $K_{16}$. This result is obtained through an efficient flag algebra formulation by exploiting problem-specific combinatorial symmetries that also allows us to study some related problems.

math.CO↗

arXiv2023

The Rado Multiplicity Problem in Vector Spaces over Finite Fields

We study an analogue of the Ramsey multiplicity problem for additive structures, in particular establishing the minimum number of monochromatic 3-APs in 3-colorings of $\mathbb{F}_3^n$ as well as obtaining the first non-trivial lower bound for the minimum number of monochromatic 4-APs in 2-colorings of $\mathbb{F}_5^n$. The former parallels results by Cumings et al (2013) in extremal graph theory and the latter improves upon results of Saad and Wolf (2017) The lower bounds are notably obtained by extending the flag algebra calculus of Razborov (2007) to additive structures in vector spaces over finite fields.

math.CO↗

arXiv2023

How I Learned to Stop Worrying and Love Retraining

Many Neural Network Pruning approaches consist of several iterative training and pruning steps, seemingly losing a significant amount of their performance after pruning and then recovering it in the subsequent retraining phase. Recent works of Renda et al. (2020) and Le & Hua (2021) demonstrate the significance of the learning rate schedule during the retraining phase and propose specific heuristics for choosing such a schedule for IMP (Han et al., 2015). We place these findings in the context of the results of Li et al. (2020) regarding the training of models within a fixed training budget and demonstrate that, consequently, the retraining phase can be massively shortened using a simple linear learning rate schedule. Improving on existing retraining approaches, we additionally propose a method to adaptively select the initial value of the linear schedule. Going a step further, we propose similarly imposing a budget on the initial dense training phase and show that the resulting simple and efficient method is capable of outperforming significantly more complex or heavily parameterized state-of-the-art approaches that attempt to sparsify the network during training. These findings not only advance our understanding of the retraining phase, but more broadly question the belief that one should aim to avoid the need for retraining and reduce the negative effects of 'hard' pruning by incorporating the sparsification process into the standard training.

cs.LG↗

arXiv2021

Projection-Free Adaptive Gradients for Large-Scale Optimization

The complexity in large-scale optimization can lie in both handling the objective function and handling the constraint set. In this respect, stochastic Frank-Wolfe algorithms occupy a unique position as they alleviate both computational burdens, by querying only approximate first-order information from the objective and by maintaining feasibility of the iterates without using projections. In this paper, we improve the quality of their first-order information by blending in adaptive gradients. We derive convergence rates and demonstrate the computational advantage of our method over the state-of-the-art stochastic Frank-Wolfe algorithms on both convex and nonconvex objectives. The experiments further show that our method can improve the performance of adaptive gradient algorithms for constrained optimization.

math.OC↗

arXiv2020

Deep Neural Network Training with Frank-Wolfe

This paper studies the empirical efficacy and benefits of using projection-free first-order methods in the form of Conditional Gradients, a.k.a. Frank-Wolfe methods, for training Neural Networks with constrained parameters. We draw comparisons both to current state-of-the-art stochastic Gradient Descent methods as well as across different variants of stochastic Conditional Gradients. In particular, we show the general feasibility of training Neural Networks whose parameters are constrained by a convex feasible region using Frank-Wolfe algorithms and compare different stochastic variants. We then show that, by choosing an appropriate region, one can achieve performance exceeding that of unconstrained stochastic Gradient Descent and matching state-of-the-art results relying on $L^2$-regularization. Lastly, we also demonstrate that, besides impacting performance, the particular choice of constraints can have a drastic impact on the learned representations.

cs.LG↗

arXiv2019

Independent Chains in Acyclic Posets

We consider the problem of determining the maximum order of an induced vertex-disjoint union of cliques in a graph. More specifically, given some family of graphs $\mathcal{G}$ of equal order, we are interested in the parameter $a(\mathcal{G}) = \min_{G \in \mathcal{G}} \max \{ |U| : U \subseteq V, G[U] \text{ is a vertex-disjoint union of cliques} \}$. We determine the value of this parameter precisely when $\mathcal{G}$ is the family of comparability graphs of $n$-element posets with acyclic cover graph. In particular, we show that $a(\mathcal{G}) = (n+o(n))/\log_2 (n)$ in this class.

math.CO↗