arXiv subjects
Publications and source records attributed to Jon Lee.
We establish new properties of the NLP-Id upper bound for the maxi\-mum-entropy sampling problem (MESP). In particular, we give a detailed look at the concavity of its objective function as a function of the scaling parameter employed for NLP bounds for MESP. This leads to more relaxed choices for the scaling parameter for NLP-Id and even improved upper bounds for MESP.
We continue the study of the natural polytope $\mathcal{D}$ in $\mathbb{R}^{n+d}$ associated with the disjunction of a set of $n+1$ polytopes in $\mathbb{R}^d$, managed by $n$ binary variables. Already $\mathcal{D}$ had been characterized for arbitrary $n\geq 1$ and (i) $d\in\{1,2\}$, and (ii) for a broad generalization of hyper-rectangles. In both cases, the complete characterization employs full optimal big-M lifting. Here, we give a complete description of $\mathcal{D}$ for the case of $n=1$ and arbitrary $d$, when the (two) polytopes are arbitrary generalized cross polytopes. Furthermore, we characterize when our complete description employs only optimal big-M lifting. For $n>1$, we generalize the family of facet-describing inequalities used for $n=1$. Finally, we carry out some computational experiments demonstrating the value of our theoretical results.
Generalized inverses play a fundamental role in numerical linear algebra, particularly when matrices are rectangular, singular, or rank deficient. Even when the input matrix is sparse, generalized inverses such as the M-P pseudoinverse are typically dense, leading to high storage requirements, expensive matrix-vector multiplications, and reduced efficiency. We investigate computing sparse symmetric generalized inverses for sparse symmetric matrices, extending previous work on sparse generalized inverses for rectangular rank-deficient matrices. We consider minimizing the vector 1-norm over the set of symmetric generalized inverses, using vector 1-norm minimization as a surrogate for sparsity promotion. We give a new characterization of symmetric generalized inverses of symmetric matrices, which yields a compact affine reformulation of the problem. From this, we develop a Douglas-Rachford splitting (DRS) algorithm equipped with a closed-form projection onto the feasible affine space. Computational experiments compare the proposed DRS approach with exact linear-optimization formulations solved by a commercial optimizer, as well as with local-search heuristics. The results demonstrate that the proposed formulation and algorithm produce generalized inverses that are substantially sparser than the Moore-Penrose pseudoinverse while maintaining significantly smaller 1-norms than competing approaches. Moreover, the DRS algorithm exhibits superior scalability relative to exact linear-optimization formulations, successfully solving instances far beyond the reach of commercial solvers. As an application, we investigate the computation of least-squares solutions for problems involving many right-hand-side vectors and sparse rank-deficient design matrices.
The constrained generalized maximum-entropy sampling problem (CGMESP) is to select an order-s principal submatrix from an order-n covariance matrix, subject to some linear side constraints, so as to maximize the product of its t greatest eigenvalues, 0 < t <= s <n. GMESP refers to the version with no side constraints. Introduced more than 25 years ago, CGMESP is a natural generalization of two fundamental problems in statistical design theory: (i) constrained maximum-entropy sampling problem (CMESP); (ii) binary D-optimality (D-Opt). In the general case, it can be motivated by a selection problem in the context of principal component analysis (PCA). We present novel non-convex extended variable formulations for CGMESP. Using these formulations as points of departure, we present, first non-convex and then convex, continuous relaxations for CGMESP. We demonstrate many relations between different upper bounds for CGMESP, including upper bounds from the literature and our new upper bounds. We investigate the behavior of our relaxations related to the constraints linking the natural variables with the extended variables. We propose and investigate a generalized scaling technique for bound improvement. In the context of branch-and-bound, we determine the better of two natural branching techniques for fixing variables to zero. Finally, we present numerical experiments illustrating the value of our methods.
Motivated by previous efforts toward mathematically analyzing the treatment of monomials in spatial branch-and-bound, we study the convex hull of the graph of a simple monomial on a nonnegative box domain in arbitrary dimension, where at most one of the variable lower bounds is positive. We give: (i) a description via linear inequalities, and (ii) a formula for the volume.
The maximum-entropy remote sampling problem (MERSP) is to select a subset of $s$ random variables from a set of $n$ random variables, so as to maximize the information concerning a set of target random variables that are not directly observable. We assume that the set of all of these random variables follows a joint Gaussian distribution, and that we have the covariance matrix available. Finally, we measure information using Shannon's differential entropy. The main approach for exact solution of moderate-sized instances of MERSP has been branch-and-bound (B\&B), and so previous work concentrated on upper bounds. Prior to our work, there were two upper-bounding methods for MERSP: the so-called ``complementary NLP bound'' and the ``spectral bound'', both introduced 25 years ago. We are able now to establish domination results between these two upper bounds. Further, we propose a novel and effective ``hyper-scaled NLP bound'' (hNLP bound) based on a subtle convex relaxation. The ``complementary'' version of hNLP bound for MERSP generalizes the previous complementary NLP bound for MERSP. We provide theoretical guarantees, giving sufficient conditions under which the complementary hNLP bound strictly dominates the complementary NLP bound. In addition, the hNLP formulation allows us to derive upper bounds for rank-deficient covariance matrices when they satisfy a technical condition. This is in contrast to the previous NLP bound that worked with only positive definite covariance matrices (because it was wedded to a complementary formulation). Additionally, we describe procedures for calculating hyper-scaling parameters. Finally, for B\&B, we provide a variable-fixing methodology and results guiding the best way to construct subproblems. Numerical experiments on benchmark instances demonstrate the effectiveness of our approaches in advancing the algorithmic state-of-the-art for MERSP.
We introduce the dual-path fixing strategy to exploit dual algorithms for solving relaxations of mixed-integer nonlinear-optimization problems. Such dual algorithms are naturally applied in the context of branch-and-bound, and eventual impact on the success of branch-and-bound is our strong motivation. Our fixing strategy aims to be more powerful than the common strategy of fixing variables based on a single dual-feasible solution (e.g., standard reduced-cost fixing for mixed-integer linear optimization), but to be much faster than ``strong fixing'', essentially requiring no more time than that of the dual algorithm that we exploit. We have successfully tested our ideas on mixed-integer linear-optimization set-covering instances from the literature, in the context of the dual-simplex method applied to the continuous relaxations.
Circuits are fundamental objects in linear programming and oriented matroid theory, representing the elementary difference vectors of a polyhedron between points in its affine space. A recent concept introduced by Ekbatani, Natura, and V\'egh, the circuit imbalance, serves as a complexity measure relevant to iteration bounds for circuit-based augmentation and circuit diameters, as well as the general interpretability of circuits in terms of the underlying application. In this paper, we analyze linear programming formulations of relaxed combinatorial optimization problems to prove two contrasting types of results related to the circuit imbalance. On one hand, we identify simple and common constraint structures, in particular arising in graph-theoretic problems, that inherently lead to an exponential circuit imbalance. These constructions show that, in quite general situations, working with the entire set of circuits poses significant challenges for an application of circuit augmentation or the study of circuit diameters. On the other hand, through a case study of two classic graph-theoretic problems with exponential imbalance, the vertex graph coloring problem and the maximum weight forest problem, we exhibit the existence of sets and subsets of highly interpretable circuits of (best-case) imbalance 1. These sets correspond to the recoloring of vertices or to the addition or removal of edges, respectively, for example generalizing classic concepts of Kempe dynamics in coloring. Their interpretability in terms of the underlying application facilitates a study of circuit walks in the corresponding polytopes. We prove that a restriction of circuit walks to these sets suffices to not only guarantee reachability of the integral extreme-points of the skeleton, but leads to linear and constant circuit diameter bounds, respectively.
We establish strong connections between two fundamental nonlinear 0/1 optimization problems coming from the area of experimental design, namely maximum-entropy sampling and 0/1 D-optimality. The connections are based on maps between instances, and we analyze the behavior of these maps. Using these maps, we transport basic upper-bounding methods between these two problems. Further, we establish results relating how different branch-and-bound schemes based on these maps compare. Additionally, we observe some surprising numerical results, where bounding methods that did not seem promising in their direct application to real-data MESP instances, are now useful for MESP instances that come from 0/1 D-optimality.
We give sparsity results and present algorithms for calculating minimum (vector) 1-norm universal solvers connected to least-squares problems. In particular, besides universal least-squares solvers, we consider minimum-rank universal least-squares solvers, and simultaneous universal minimum-norm/least-squares solvers. For all of these, we present and compare several new alternative linear-programming formulations and very effective proximal-point algorithms. Overall, we found that our new Douglas-Rachford splitting algorithms for these problems performed best.
In 2022, we published the book Maximum-Entropy Sampling: Algorithms and Application (Springer). Since then, there have been several notable advancements on this topic. In this manuscript, we survey some recent highlights.
We propose integer-programming formulations for an optimal safety-landing site (SLS) location problem that arises in the design of urban air-transportation networks. We first develop a set-cover based approach for the case where the candidate location set is finite and composed of points, and we link the problems to solvable cases that have been studied. We then use a mixed-integer second-order cone program to model the situation where the locations of SLSs are restricted to convex sets only. Finally, we introduce strong fixing, which we found to be very effective in reducing the size of integer programs.
The 0/1 D-optimality problem and the Maximum-Entropy Sampling problem are two well-known NP-hard discrete maximization problems in experimental design. Algorithms for exact optimization (of moderate-sized instances) are based on branch-and-bound. The best upper-bounding methods are based on convex relaxation. We present ADMM (Alternating Direction Method of Multipliers) algorithms for solving these relaxations and experimentally demonstrate their practical value.
In optimal experimental design, the objective is to select a limited set of experiments that maximizes information about unknown model parameters based on factor levels. This work addresses the generalized D-optimal design problem, allowing for nonlinear relationships in factor levels. We develop scalable algorithms suitable for cases where the number of candidate experiments grows exponentially with the factor dimension, focusing on both first- and second-order models under design constraints. Particularly, our approach integrates convex relaxation with pricing-based local search techniques, which can provide upper bounds and performance guarantees. Unlike traditional local search methods, such as the ``Fedorov exchange" and its variants, our method effectively accommodates arbitrary side constraints in the design space. Furthermore, it yields both a feasible solution and an upper bound on the optimal value derived from the convex relaxation. Numerical results highlight the efficiency and scalability of our algorithms, demonstrating superior performance compared to the state-of-the-art commercial software, JMP
We study the natural extended-variable formulation for the disjunction of $n+1$ polytopes in $\mathbb{R}^d$. We demonstrate that the convex hull $D$ in the natural extended-variable space $\mathbb{R}^{d+n}$ is given by full optimal big-M lifting (i) when $d\leq 2$ (and that it is not generally true for $d\geq 3$), and also (ii) under some technical conditions, when the polytopes have a common facet-describing constraint matrix, for arbitrary $d\geq 1$ and $n\geq 1$. We give a broad family of examples with $d\geq 3$ and $n=1$, where the convex hull is not described after employing all full optimal big-M lifting inequalities, but it is described after one round of MIR inequalities. Additionally, we give some general results on the polyhedral structure of $D$, and we demonstrate that all facets of $D$ can be enumerated in polynomial time when $d$ is fixed.
MINLO (mixed-integer nonlinear optimization) formulations of the disjunction between the origin and a polytope via a binary indicator variable is broadly used in nonlinear combinatorial optimization for modeling a fixed cost associated with carrying out a group of activities and a convex cost function associated with the levels of the activities. The perspective relaxation of such models is often used to solve to global optimality in a branch-and-bound context, but it typically requires suitable conic solvers and is not compatible with general-purpose NLP software in the presence of other classes of constraints. This motivates the investigation of when simpler but weaker relaxations may be adequate. Comparing the volume (i.e., Lebesgue measure) of the relaxations as a measure of tightness, we lift some of the results related to the simplex case to the box case. In order to compare the volumes of different relaxations in the box case, it is necessary to find an appropriate concave upper bound that preserves the convexity and is minimal, which is more difficult than in the simplex case. To address the challenge beyond the simplex case, the triangulation approach is used.
The generalized maximum-entropy sampling problem (GMESP) is to select an order-$s$ principal submatrix from an order-$n$ covariance matrix, to maximize the product of its $t$ greatest eigenvalues, $0<t\leq s <n$. Introduced more than 25 years ago, GMESP is a natural generalization of two fundamental problems in statistical design theory: (i) maximum-entropy sampling problem (MESP); (ii) binary D-optimality (D-Opt). In the general case, it can be motivated by a selection problem in the context of principal component analysis (PCA). We introduce the first convex-optimization based relaxation for GMESP, study its behavior, compare it to an earlier spectral bound, and demonstrate its use in a branch-and-bound scheme. We find that such an approach is practical when $s-t$ is very small.
The circuits of a polyhedron are a superset of its edge directions. Circuit walks, a sequence of steps along circuits, generalize edge walks and are "short" if they have few steps or small total length. Both interpretations of short are relevant to the theory and application of linear programming. We study the hardness of several problems relating to the construction of short circuit walks. We establish that for a pair of vertices of a $0/1$-network-flow polytope, it is NP-complete to determine the length of a shortest circuit walk, even if we add the requirement that the walk must be sign-compatible. Our results also imply that determining the minimal number of circuits needed for a sign-compatible decomposition is NP-complete. Further, we show that it is NP-complete to determine the smallest total length (for $p$-norms $\lVert \cdot \rVert_p$, $1 < p \leq \infty$) of a circuit walk between a pair of vertices. One method to construct a short circuit walk is to pick up a correct facet at each step, which generalizes a non-revisiting walk. We prove that it is NP-complete to determine if there is a circuit direction that picks up a correct facet; in contrast, this problem can be solved in polynomial time for TU polyhedra.