Adaptive Constraint Sampling for Stochastic Gradient Methods in Constrained Optimization
This paper proposes an adaptive constraint-sampling technique for stochastic gradient-based methods to efficiently solve constrained optimization problems with a large number of inequality constraints. The proposed constraint-sampling technique maintains a set of recently violated constraints and draws the constraint used in each iteration uniformly either from this set or from the set of all constraints. This keeps its per-iteration arithmetic overhead independent of the number of constraints. In simulations on synthetic convex optimization problems and in the synthesis of a control barrier function for the Dubins Car, the constraint-sampling technique is shown to significantly outperform uniform constraint sampling.