Searcharxiv⌕ Search

arXiv subjects

Hongyu Cheng

Publications and source records attributed to Hongyu Cheng.

12 recordsLinked to original sources
arXiv2026

Sample Complexity of Stochastic Optimization with Integer Variables

We establish sample complexity results for stochastic optimization over the integers, especially with a view to understand the complexity with respect to the corresponding continuous optimization problem. We show that integer optimization can sometimes require strictly more samples and sometimes strictly smaller number of samples, depending on the structure of the objective and constraints. 1. For Lipschitz objectives over subsets of the $\ell_\infty$ ball, the statistical complexity of general stochastic mixed-integer, nonlinear, nonconvex optimization is exactly the same as stochastic linear optimization with just bound constraints. 2. For Lipschitz objectives over subsets of the $\ell_2$ ball, we show that integer optimization can require strictly *smaller* sample size compared to the continuous setting in a certain regime. To get to this result, we also establish tight sample complexity results for nonconvex continuous stochastic optimization which, to the best of our knowledge, do not appear in prior work. 3. For strongly convex, smooth objectives, integer optimization has high statistical complexity compared to the continuous setting. In particular, we show that integer optimization requires $\Omega(1/\epsilon^2)$ samples to report an $\epsilon$-approximate solution, compared to the well-known $O(1/\epsilon)$ sample complexity from the continuous optimization literature.

cs.LG↗
arXiv2026

Linear Threshold for Oertel's Conjecture on the Mixed-Integer Volume

Gr\"unbaum's inequality guarantees that the centroid of a convex body has halfspace depth at least $1/e$: every halfspace containing the centroid captures at least a $1/e$ fraction of the body's volume. For mixed-integer convex sets $S=C\cap(\mathbb Z^n\times\mathbb{R}^d)$ where $C$ is a convex body, Oertel conjectured that there exists $y\in S$ such that every closed halfspace $H$ containing $y$ satisfies $H_d(S\cap H)\ge \frac{1}{2^ne} H_d(S)$, where $H_d(X)$ denotes the $d$-dimensional Hausdorff measure of $X$. This conjecture is closely connected to complexity bounds for cutting plane methods and information complexity in mixed-integer convex optimization. Basu and Oertel established this conjecture for sets of sufficiently large lattice width, with the required lower bound on lattice width depending exponentially on the dimension. More recently, Cristi and Salas reduced this threshold to a polynomial one by assuming that $\mathrm{proj}_{\mathbb{R}^n}(C)$ contains a Euclidean ball of radius at least $1178 d^2n^{3/2}$. We prove that when the projection contains an $\ell_\infty$ ball of radius $k\ge(3e/2)(n+d)$, which is linear in the dimensions $n$ and $d$, there exists $y^*\in S$ such that every closed halfspace $H$ containing $y^*$ satisfies $H_d(S\cap H)\ge\left(\frac{1}{e}-\frac{3(n+d)}{2k}\right)H_d(S)$. In particular, when $k\ge 3e(n+d)$ we obtain $H_d(S\cap H)\ge \frac{1}{2e}H_d(S)\ge \frac{1}{2^ne}H_d(S)$, thus verifying Oertel's conjecture for a significantly larger family of sets than previous results. We also show that this linear scaling is necessary: when the radius is sublinear in the total dimension, the maximum achievable halfspace depth can be arbitrarily small relative to the total mixed-integer volume, so no dimension-independent constant fraction lower bound is possible under such an assumption alone. However, the conjecture remains open in its full generality.

math.OC↗
arXiv2026

Theoretical Challenges in Learning for Branch-and-Cut

Machine learning is increasingly used to guide branch-and-cut (B&C) for mixed-integer linear programming by learning score-based policies for selecting branching variables and cutting planes. Many approaches train on local signals from lookahead heuristics such as strong branching, and linear programming (LP) bound improvement for cut selection. Training and evaluation of the learned models often focus on local score accuracy. We show that such local score-based methods can lead to search trees exponentially larger than optimal tree sizes, by identifying two sources of this gap. The first is that these widely used expert signals can be misaligned with overall tree size. LP bound improvement can select a root cut set that yields an exponentially larger strong branching tree than selecting cuts by a simple proxy score, and strong branching itself can be exponentially suboptimal (Dey et al., 2024). The second is that small discrepancies can be amplified by the branch-and-bound recursion. An arbitrarily small perturbation of the right-hand sides in a root cut set can change the minimum tree size from a single node to exponentially many. For branching, arbitrarily small score discrepancies, and differences only in tie-breaking, can produce trees of exponentially different sizes, and even a small number of decision differences along a trajectory can incur exponential growth. These results show that branch-and-cut policies trained and learned using local expert scores do not guarantee small trees, thus motivating the study of data-driven methods that produce policies better aligned with tree size rather than only accuracy on expert scores.

math.OC↗
arXiv2026

Dispersive estimate for quasi-periodic Klein-Gordon equation on 1-d lattices

The dispersive estimate plays a pivotal role in establishing the long-term behavior of solutions to the nonlinear equation, thereby being crucial for investigating the well-posedness of the equation.In this work we prove that the solutions to Klein-Gordon equation on 1-d lattices follow the dispersive estimate provided that potential is quasi-periodic with Diophantine frequencies and closed to positive constants.

math.DS↗
arXiv2025

Generalization Guarantees for Learning Branch-and-Cut Policies in Integer Programming

Mixed-integer programming (MIP) provides a powerful framework for optimization problems, with Branch-and-Cut (B&C) being the predominant algorithm in state-of-the-art solvers. The efficiency of B&C critically depends on heuristic policies for making sequential decisions, including node selection, cut selection, and branching variable selection. While traditional solvers often employ heuristics with manually tuned parameters, recent approaches increasingly leverage machine learning, especially neural networks, to learn these policies directly from data. A key challenge is to understand the theoretical underpinnings of these learned policies, particularly their generalization performance from finite data. This paper establishes rigorous sample complexity bounds for learning B&C policies where the scoring functions guiding each decision step (node, cut, branch) have a certain piecewise polynomial structure. This structure generalizes the linear models that form the most commonly deployed policies in practice and investigated recently in a foundational series of theoretical works by Balcan et al. Such piecewise polynomial policies also cover the neural network architectures (e.g., using ReLU activations) that have been the focal point of contemporary practical studies. Consequently, our theoretical framework closely reflects the models utilized by practitioners investigating machine learning within B&C, offering a unifying perspective relevant to both established theory and modern empirical research in this area. Furthermore, our theory applies to quite general sequential decision making problems beyond B&C.

cs.LG↗
arXiv2024

Learning Cut Generating Functions for Integer Programming

The branch-and-cut algorithm is the method of choice to solve large scale integer programming problems in practice. A key ingredient of branch-and-cut is the use of cutting planes which are derived constraints that reduce the search space for an optimal solution. Selecting effective cutting planes to produce small branch-and-cut trees is a critical challenge in the branch-and-cut algorithm. Recent advances have employed a data-driven approach to select optimal cutting planes from a parameterized family, aimed at reducing the branch-and-bound tree size (in expectation) for a given distribution of integer programming instances. We extend this idea to the selection of the best cut generating function (CGF), which is a tool in the integer programming literature for generating a wide variety of cutting planes that generalize the well-known Gomory Mixed-Integer (GMI) cutting planes. We provide rigorous sample complexity bounds for the selection of an effective CGF from certain parameterized families that provably performs well for any specified distribution on the problem instances. Our empirical results show that the selected CGF can outperform the GMI cuts for certain distributions. Additionally, we explore the sample complexity of using neural networks for instance-dependent CGF selection.

math.OC↗
arXiv2024

Sample Complexity of Algorithm Selection Using Neural Networks and Its Applications to Branch-and-Cut

Data-driven algorithm design is a paradigm that uses statistical and machine learning techniques to select from a class of algorithms for a computational problem an algorithm that has the best expected performance with respect to some (unknown) distribution on the instances of the problem. We build upon recent work in this line of research by considering the setup where, instead of selecting a single algorithm that has the best performance, we allow the possibility of selecting an algorithm based on the instance to be solved, using neural networks. In particular, given a representative sample of instances, we learn a neural network that maps an instance of the problem to the most appropriate algorithm for that instance. We formalize this idea and derive rigorous sample complexity bounds for this learning problem, in the spirit of recent work in data-driven algorithm design. We then apply this approach to the problem of making good decisions in the branch-and-cut framework for mixed-integer optimization (e.g., which cut to add?). In other words, the neural network will take as input a mixed-integer optimization instance and output a decision that will result in a small branch-and-cut tree for that instance. Our computational results provide evidence that our particular way of using neural networks for cut selection can make a significant impact in reducing branch-and-cut tree sizes, compared to previous data-driven approaches.

cs.LG↗
arXiv2022

Invariant tori for area-preserving maps with ultra-differentiable perturbation and Liouvillean frequency

We prove the existence of invariant tori to the area-preserving maps defined on $ \mathbb{R}^2\times\mathbb{T} $ \begin{equation*} \bar{x}=F(x,\theta), \qquad \bar{\theta}=\theta+\alpha\, \,(\alpha\in \mathbb{R}\setminus\mathbb{Q}), \end{equation*} where $ F $ is closed to a linear rotation, and the perturbation is ultra-differentiable in $ \theta\in \mathbb{T},$ which is very closed to $C^{\infty}$ regularity. Moreover, we assume that the frequency $\alpha$ is any irrational number without other arithmetic conditions and the smallness of the perturbation does not depend on $\alpha$. Thus, both the difficulties from the ultra-differentiability of the perturbation and Liouvillean frequency will appear in this work. The proof of the main result is based on the Kolmogorov-Arnold-Moser (KAM) scheme about the area-preserving maps with some new techniques.

math.DS↗
arXiv2021

Neural networks with linear threshold activations: structure and algorithms

In this article we present new results on neural networks with linear threshold activation functions. We precisely characterize the class of functions that are representable by such neural networks and show that 2 hidden layers are necessary and sufficient to represent any function representable in the class. This is a surprising result in the light of recent exact representability investigations for neural networks using other popular activation functions like rectified linear units (ReLU). We also give precise bounds on the sizes of the neural networks required to represent any function in the class. Finally, we design an algorithm to solve the empirical risk minimization (ERM) problem to global optimality for these neural networks with a fixed architecture. The algorithm's running time is polynomial in the size of the data sample, if the input dimension and the size of the network architecture are considered fixed constants. The algorithm is unique in the sense that it works for any architecture with any number of layers, whereas previous polynomial time globally optimal algorithms work only for very restricted classes of architectures. Using these insights, we propose a new class of neural networks that we call shortcut linear threshold networks. To the best of our knowledge, this way of designing neural networks has not been explored before in the literature. We show that these neural networks have several desirable theoretical properties.

cs.LG↗
arXiv2021

Global rigidity for ultra-differentiable quasiperiodic cocycles and its spectral applications

For quasiperiodic Schr\"odinger operators with one-frequency analytic potentials, from dynamical systems side, it has been proved that the corresponding quasiperiodic Schr\"odinger cocycle is either rotations reducible or has positive Lyapunov exponent for all irrational frequency and almost every energy. From spectral theory side, the "Schr\"odinger conjecture" and the "Last's intersection spectrum conjecture" have been verified. The proofs of above results crucially depend on the analyticity of the potentials. People are curious about if the analyticity is essential for those problems, see open problems by Fayad-Krikorian and Jitomirskaya-Mar. In this paper, we prove the above mentioned results for ultra-differentiable potentials.

math.DS↗
arXiv2019

Response solutions to the quasi-periodically forced systems with degenerate equilibrium: A simple proof of a result of W. Si and J. Si. and extensions

We give a simple proof of the existence of response solutions in some quasi-periodically forced systems with a degenerate fixed points. The same questions were answered in \cite{ss18} using two versions of KAM theory. Our method is based on reformulating the existence of response solutions as a fixed point problem in appropriate spaces of smooth functions. By algebraic manipulations, the fixed point problem is transformed into a contraction. Compared to the KAM method, the present method does not incur a loss of regularity. That is, the solutions we obtain have the same regularity as the forcing. Moreover, the method here applies when problems are only finitely differentiable. It also weakens slightly the non-degeneracy conditions. Since the method is based on the contraction mapping principle, we also obtain automatically smooth dependence on parameters and, when studying complex versions of the problem we discover the new phenomenon of monodromy. We also present results for higher dimensional systems, but for higher dimensional systems, the concept of degenerate fixed points is much more subtle than in one dimensional systems. To illustrate the power of the method, we also consider two problems not studied in \cite{ss18}: the forcing with zero average and second order oscillators. We show that in the zero average forcing case, the solutions are qualitatively different, but the second order oscillators is remarkably similar.

math.DS↗