arXiv subjects

Santanu S. Dey

Publications and source records attributed to Santanu S. Dey.

At least 55 records · Page 3Linked to original sources

arXiv2021

A K-Nearest Neighbor Heuristic for Real-Time DC Optimal Transmission Switching

While transmission switching is known to reduce power generation costs, the difficulty of solving even DC optimal transmission switching (DCOTS) has prevented optimal transmission switching from becoming commonplace in real-time power systems operation. In this paper, we present a k-nearest neighbors (KNN) heuristic for DCOTS which relies on the insight that, for routine operations on a fixed network, the DCOTS solutions for similar load profiles and generation cost profiles will likely turn off similar sets of lines. We take a data-driven approach and assume that we have DCOTS solutions for many historical instances, which is realistic given that the problem is solved every 5 minutes in practice. Given a new instance, we find a set of "close" instances from the past and return the best of their solutions for the new instance. We present a case study on 7 test networks ranging in size from 118 to 3,375 buses. We compare the proposed heuristic to DCOTS heuristics from the literature, commercial solver heuristics, and a simple greedy local search algorithm. In most cases, we find better quality solutions in less computational time. In addition, the computational time is within the limits imposed by real-time operations, even on larger networks. Last, we present an empirical study of our training data to understand why the heuristic works well.

math.OC↗

arXiv2021

Decomposable Formulation of Transmission Constraints for Decentralized Power Systems Optimization

One of the most complicating factors in decentralized optimization for power systems is the modeling of power flow equations. Existing formulations for DC power flows either have limited scalability or are very dense and unstructured, making them unsuitable for large-scale decentralized studies. In this work, we present a novel DC power flow formulation, based on sparsified injection shift factors, which has a decomposable block-diagonal structure, scales well for large systems, and can efficiently handle N-1 security requirements. Benchmarks on Multi-Zonal Security-Constrained Unit Commitment problems show that the proposed formulation can reliably and efficiently solve instances with up to 6,515 buses, with no convergence or numerical issues.

math.OC↗

arXiv2020

Complexity of Training ReLU Neural Network

In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.

cs.CC↗

arXiv2020

Sparse PSD approximation of the PSD cone

While semidefinite programming (SDP) problems are polynomially solvable in theory, it is often difficult to solve large SDP instances in practice. One technique to address this issue is to relax the global positive-semidefiniteness (PSD) constraint and only enforce PSD-ness on smaller $k\times k$ principal submatrices --- we call this the sparse SDP relaxation. Surprisingly, it has been observed empirically that in some cases this approach appears to produce bounds that are close to the optimal objective function value of the original SDP. In this paper, we formally attempt to compare the strength of the sparse SDP relaxation vis-à-vis the original SDP from a theoretical perspective. In order to simplify the question, we arrive at a data independent version of it, where we compare the sizes of SDP cone and the $k$-PSD closure, which is the cone of matrices where PSD-ness is enforced on all $k\times k$ principal submatrices. In particular, we investigate the question of how far a matrix of unit Frobenius norm in the $k$-PSD closure can be from the SDP cone. We provide two incomparable upper bounds on this farthest distance as a function of $k$ and $n$. We also provide matching lower bounds, which show that the upper bounds are tight within a constant in different regimes of $k$ and $n$. Other than linear algebra techniques, we extensively use probabilistic methods to arrive at these bounds. One of the lower bounds is obtained by observing a connection between matrices in the $k$-PSD closure and matrices satisfying the restricted isometry property (RIP).

math.OC↗

arXiv2020

Subset selection in sparse matrices

In subset selection we search for the best linear predictor that involves a small subset of variables. From a computational complexity viewpoint, subset selection is NP-hard and few classes are known to be solvable in polynomial time. Using mainly tools from discrete geometry, we show that some sparsity conditions on the original data matrix allow us to solve the problem in polynomial time.

math.OC↗

arXiv2019

Exact Augmented Lagrangian Duality for Mixed Integer Quadratic Programming

Mixed integer quadratic programming (MIQP) is the problem of minimizing a convex quadratic function over mixed integer points in a rational polyhedron. This paper focuses on the augmented Lagrangian dual (ALD) for MIQP. ALD augments the usual Lagrangian dual with a weighted nonlinear penalty on the dualized constraints. We first prove that ALD will reach a zero duality gap asymptotically as the weight on the penalty goes to infinity under some mild conditions on the penalty function. We next show that a finite penalty weight is enough for a zero gap when we use any norm as the penalty function. Finally, we prove a polynomially bound on the weight on the penalty term to obtain a zero gap.

math.OC↗

arXiv2019

On Subadditive Duality for Conic Mixed-Integer Programs

In this paper, we show that the subadditive dual of a feasible conic mixed-integer program (MIP) is a strong dual whenever it is feasible. Moreover, we show that this dual feasibility condition is equivalent to feasibility of the conic dual of the continuous relaxation of the conic MIP. In addition, we prove that all known conditions and other 'natural' conditions for strong duality, such as strict mixed-integer feasibility, boundedness of the feasible set or essentially strict feasibility imply that the subadditive dual is feasible. As an intermediate result, we extend the so-called 'finiteness property' from full-dimensional convex sets to intersections of full-dimensional convex sets and Dirichlet convex sets.

math.OC↗

arXiv2019

An Approximation Algorithm for training One-Node ReLU Neural Network

Training a one-node neural network with ReLU activation function (One-Node-ReLU) is a fundamental optimization problem in deep learning. In this paper, we begin with proving the NP-hardness of training One-Node-ReLU. We then present an approximation algorithm to solve One-Node-ReLU whose running time is $\mathcal{O}(n^k)$ where $n$ is the number of samples, $k$ is a predefined integral constant. Except $k$, this algorithm does not require pre-processing or tuning of parameters. We analyze the performance of this algorithm under various regimes. First, given any arbitrary set of training sample data set, we show that the algorithm guarantees a $\frac{n}{k}$-approximation for training One-Node-ReLU problem. As a consequence, in the realizable case (i.e. when the training error is zero), this approximation algorithm achieves the global optimal solution for the One-Node-ReLU problem. Second, we assume that the training sample data is obtained from an underlying one-node neural network with ReLU activation function, where the output is perturbed by a Gaussian noise. In this regime, we show that the same approximation algorithm guarantees a much better asymptotic approximation ratio which is independent of the number of samples $n$. Finally, we conduct extensive empirical studies and arrive at two conclusions. One, the approximation algorithm together with some heuristic performs better than gradient descent algorithm. Two, the solution of the approximation algorithm can be used as starting point for gradient descent -- a combination that works significantly better than gradient descent.

math.OC↗

arXiv2019

Non-unique lifting of integer variables in minimal inequalities

We explore the lifting question in the context of cut-generating functions. Most of the prior literature on this question focuses on cut-generating functions that have the unique lifting property. We develop a general theory for understanding the lifting question for cut-generating functions that do not necessarily have the unique lifting property.

math.OC↗

arXiv2018

The convex hull of a quadratic constraint over a polytope

A quadratically constrained quadratic program (QCQP) is an optimization problem in which the objective function is a quadratic function and the feasible region is defined by quadratic constraints. Solving non-convex QCQP to global optimality is a well-known NP-hard problem and a traditional approach is to use convex relaxations and branch-and-bound algorithms. This paper makes a contribution in this direction by showing that the exact convex hull of a general quadratic equation intersected with any bounded polyhedron is second-order cone representable. We present a simple constructive proof of this result.

math.OC↗

arXiv2018

Theoretical challenges towards cutting-plane selection

While many classes of cutting-planes are at the disposal of integer programming solvers, our scientific understanding is far from complete with regards to cutting-plane selection, i.e., the task of selecting a portfolio of cutting-planes to be added to the LP relaxation at a given node of the branch-and-bound tree. In this paper we review the different classes of cutting-planes available, known theoretical results about their relative strength, important issues pertaining to cut selection, and discuss some possible new directions to be pursued in order to accomplish cutting-plane selection in a more principled manner. Finally, we review some lines of work that we undertook to provide a preliminary theoretical underpinning for some of the issues related to cut selection.

math.OC↗

arXiv2018

Lower bounds on the lattice-free rank for packing and covering integer programs

In this paper, we present lower bounds on the rank of the split closure, the multi-branch closure and the lattice-free closure for packing sets as a function of the integrality gap. We also provide a similar lower bound on the split rank of covering polyhedra. These results indicate that whenever the integrality gap is high, these classes of cutting planes must necessarily be applied for many rounds in order to obtain the integer hull.

math.OC↗

arXiv2018

New SOCP relaxation and branching rule for bipartite bilinear programs

A bipartite bilinear program (BBP) is a quadratically constrained quadratic optimization problem where the variables can be partitioned into two sets such that fixing the variables in any one of the sets results in a linear program. We propose a new second order cone representable (SOCP) relaxation for BBP, which we show is stronger than the standard SDP relaxation intersected with the boolean quadratic polytope. We then propose a new branching rule inspired by the construction of the SOCP relaxation. We describe a new application of BBP called as the finite element model updating problem, which is a fundamental problem in structural engineering. Our computational experiments on this problem class show that the new branching rule together with an polyhedral outer approximation of the SOCP relaxation outperforms a state-of-the-art commercial global solver in obtaining dual bounds.

math.OC↗

arXiv2017

Sparse principal component analysis and its $l_1$-relaxation

Principal component analysis (PCA) is one of the most widely used dimensionality reduction methods in scientific data analysis. In many applications, for additional interpretability, it is desirable for the factor loadings to be sparse, that is, we solve PCA with an additional cardinality (l0) constraint. The resulting optimization problem is called the sparse principal component analysis (SPCA). One popular approach to achieve sparsity is to replace the l0 constraint by an l1 constraint. In this paper, we prove that, independent of the data, the optimal objective function value of the problem with l0 constraint is within a constant factor of the the optimal objective function value of the problem with l1 constraint. To the best of our knowledge, this is the first formal relationship established between the l0 and the l1 constraint version of the problem.

math.OC↗

arXiv2017

The Strength of Multi-row Aggregation Cuts for Sign-pattern Integer Programs

In this paper, we study the strength of aggregation cuts for sign-pattern integer programs (IPs). Sign-pattern IPs are a generalization of packing IPs and are of the form $\{x\in \mathbb{Z}^n_+\ | \ Ax\le b\}$ where for a given column $j$, $A_{ij}$ is either non-negative for all $i$ or non-positive for all $i$. Our first result is that the aggregation closure for such sign-pattern IPs can be 2-approximated by the original 1-row closure. This generalizes a result for packing IPs. On the other hand, unlike in the case of packing IPs, we show that the multi-row aggregation closure cannot be well approximated by the original multi-row closure. Therefore for these classes of integer programs general aggregated multi-row cutting planes can perform significantly better than just looking at cuts from multiple original constraints.

math.OC↗

arXiv2017

Strong mixed-integer formulations for the floor layout problem

The floor layout problem (FLP) tasks a designer with positioning a collection of rectangular boxes on a fixed floor in such a way that minimizes total communication costs between the components. While several mixed integer programming (MIP) formulations for this problem have been developed, it remains extremely challenging from a computational perspective. This work takes a systematic approach to constructing MIP formulations and valid inequalities for the FLP that unifies and recovers all known formulations for it. In addition, the approach yields new formulations that can provide a significant computational advantage and can solve previously unsolved instances. While the construction approach focuses on the FLP, it also exemplifies generic formulation techniques that should prove useful for broader classes of problems.

math.OC↗

arXiv2017

Beating the SDP bound for the floor layout problem: A simple combinatorial idea

For many mixed-integer programming (MIP) problems, high-quality dual bounds can be obtained either through advanced formulation techniques coupled with a state-of-the-art MIP solver, or through semidefinite programming (SDP) relaxation hierarchies. In this paper, we introduce an alternative bounding approach that exploits the "combinatorial implosion" effect by solving portions of the original problem and aggregating this information to obtain a global dual bound. We apply this technique to the one-dimensional and two-dimensional floor layout problems and compare it with the bounds generated by both state-of-the-art MIP solvers and by SDP relaxations. Specifically, we prove that the bounds obtained through the proposed technique are at least as good as those obtained through SDP relaxations, and present computational results that these bounds can be significantly stronger and easier to compute than these alternative strategies, particularly for very difficult problem instances.

math.OC↗

arXiv2017

Matrix Minor Reformulation and SOCP-based Spatial Branch-and-Cut Method for the AC Optimal Power Flow Problem

Alternating current optimal power flow (AC OPF) is one of the most fundamental optimization problems in electrical power systems. It can be formulated as a semidefinite program (SDP) with rank constraints. Solving AC OPF, that is, obtaining near optimal primal solutions as well as high quality dual bounds for this non-convex program, presents a major computational challenge to today's power industry for the real-time operation of large-scale power grids. In this paper, we propose a new technique for reformulation of the rank constraints using both principal and non-principal 2-by-2 minors of the involved Hermitian matrix variable and characterize all such minors into three types. We show the equivalence of these minor constraints to the physical constraints of voltage angle differences summing to zero over three- and four-cycles in the power network. We study second-order conic programming (SOCP) relaxations of this minor reformulation and propose strong cutting planes, convex envelopes, and bound tightening techniques to strengthen the resulting SOCP relaxations. We then propose an SOCP-based spatial branch-and-cut method to obtain the global optimum of AC OPF. Extensive computational experiments show that the proposed algorithm significantly outperforms the state-of-the-art SDP-based OPF solver and on a simple personal computer is able to obtain on average a 0.71% optimality gap in no more than 720 seconds for the most challenging power system instances in the literature.

math.OC↗