arXiv subjects

Pascal Van Hentenryck

Publications and source records attributed to Pascal Van Hentenryck.

At least 145 records · Page 8Linked to original sources

arXiv2021

Spatio-Temporal Point Processes with Attention for Traffic Congestion Event Modeling

We present a novel framework for modeling traffic congestion events over road networks. Using multi-modal data by combining count data from traffic sensors with police reports that report traffic incidents, we aim to capture two types of triggering effect for congestion events. Current traffic congestion at one location may cause future congestion over the road network, and traffic incidents may cause spread traffic congestion. To model the non-homogeneous temporal dependence of the event on the past, we use a novel attention-based mechanism based on neural networks embedding for point processes. To incorporate the directional spatial dependence induced by the road network, we adapt the "tail-up" model from the context of spatial statistics to the traffic network setting. We demonstrate our approach's superior performance compared to the state-of-the-art methods for both synthetic and real data.

cs.LG↗

arXiv2021

Differential Privacy of Hierarchical Census Data: An Optimization Approach

This paper is motivated by applications of a Census Bureau interested in releasing aggregate socio-economic data about a large population without revealing sensitive information about any individual. The released information can be the number of individuals living alone, the number of cars they own, or their salary brackets. Recent events have identified some of the privacy challenges faced by these organizations. To address them, this paper presents a novel differential-privacy mechanism for releasing hierarchical counts of individuals. The counts are reported at multiple granularities (e.g., the national, state, and county levels) and must be consistent across all levels. The core of the mechanism is an optimization model that redistributes the noise introduced to achieve differential privacy in order to meet the consistency constraints between the hierarchical levels. The key technical contribution of the paper shows that this optimization problem can be solved in polynomial time by exploiting the structure of its cost functions. Experimental results on very large, real datasets show that the proposed mechanism provides improvements of up to two orders of magnitude in terms of computational efficiency and accuracy with respect to other state-of-the-art techniques.

cs.DB↗

arXiv2021

Benders Subproblem Decomposition for Bilevel Problems with Convex Follower

Bilevel optimization formulates hierarchical decision-making processes that arise in many real-world applications such as in pricing, network design, and infrastructure defense planning. In this paper, we consider a class of bilevel optimization problems where the upper-level problem features some integer variables while the lower-level problem enjoys strong duality. We propose a dedicated Benders decomposition method for solving this class of bilevel problems, which decomposes the Benders subproblem into two more tractable, sequentially solvable problems that can be interpreted as the upper and the lower-level problems. We show that the Benders subproblem decomposition carries over to an interesting extension of bilevel problems, which connects the upper-level solution with the lower level dual solution, and discuss some special cases of bilevel problems that allow sequence-independent subproblem decomposition. Several novel schemes for generating numerically stable cuts, finding a good incumbent solution, and accelerating the search tree are discussed. A computational study demonstrates the computational benefits of the proposed method over a state-of-the-art bilevel-tailored branch-and-cut method, a commercial solver, and the standard Benders method on standard test cases and the motivating applications in sequential energy markets.

math.OC↗

arXiv2021

End-to-End Constrained Optimization Learning: A Survey

This paper surveys the recent attempts at leveraging machine learning to solve constrained optimization problems. It focuses on surveying the work on integrating combinatorial solvers and optimization methods with machine learning architectures. These approaches hold the promise to develop new hybrid machine learning and optimization methods to predict fast, approximate, solutions to combinatorial problems and to enable structural logical inference. This paper presents a conceptual review of the recent advancements in this emerging area.

cs.LG↗

arXiv2021

Market Segmentation in Online Platforms

This paper studies ranking policies in a stylized trial-offer marketplace model, in which a single firm offers products and has consumers with heterogeneous preferences. Consumer trials are influenced by past purchases and the ranking of each product. The platform owner needs to devise a ranking policy to display the products to maximize the number of purchases in the long run. The model proposed attempts to understand the impact of market segmentation in a trial-offer market with social influence. In our model, consumer choices are based on a very general choice model known as the mixed MNL. We analyze the long-term dynamics of this highly complex stochastic model and we quantify the expected benefits of market segmentation. When past purchases are displayed, consumer heterogeneity makes buyers try the sub-optimal products, reducing the overall sales rate. We show that consumer heterogeneity makes the ranking problem NP-hard. We then analyze the benefits of market segmentation. We find tight bounds to the expected benefits of offering a distinct ranking to each consumer segment. Finally, we show that the market segmentation strategy always benefits from social influence when the average quality ranking is used. One of the managerial implications is that the firm is better off using an aggregate ranking policy when the variety of consumer preference is limited, but it should perform a market segmentation policy when consumers are highly heterogeneous. We also show that this result is robust to relatively small consumer classification mistakes; when these are large, an aggregate ranking is preferred.

cs.SI↗

arXiv2021

Spatial Network Decomposition for Fast and Scalable AC-OPF Learning

This paper proposes a novel machine-learning approach for predicting AC-OPF solutions that features a fast and scalable training. It is motivated by the two critical considerations: (1) the fact that topology optimization and the stochasticity induced by renewable energy sources may lead to fundamentally different AC-OPF instances; and (2) the significant training time needed by existing machine-learning approaches for predicting AC-OPF. The proposed approach is a 2-stage methodology that exploits a spatial decomposition of the power network that is viewed as a set of regions. The first stage learns to predict the flows and voltages on the buses and lines coupling the regions, and the second stage trains, in parallel, the machine-learning models for each region. Experimental results on the French transmission system (up to 6,700 buses and 9,000 lines) demonstrate the potential of the approach. Within a short training time, the approach predicts AC-OPF solutions with very high fidelity and minor constraint violations, producing significant improvements over the state-of-the-art. The results also show that the predictions can seed a load flow optimization to return a feasible solution within 0.03% of the AC-OPF objective, while reducing running times significantly.

cs.LG↗

arXiv2021

Bilevel Optimization for Differentially Private Optimization in Energy Systems

This paper studies how to apply differential privacy to constrained optimization problems whose inputs are sensitive. This task raises significant challenges since random perturbations of the input data often render the constrained optimization problem infeasible or change significantly the nature of its optimal solutions. To address this difficulty, this paper proposes a bilevel optimization model that can be used as a post-processing step: It redistributes the noise introduced by a differentially private mechanism optimally while restoring feasibility and near-optimality. The paper shows that, under a natural assumption, this bilevel model can be solved efficiently for real-life large-scale nonlinear nonconvex optimization problems with sensitive customer data. The experimental results demonstrate the accuracy of the privacy-preserving mechanism and showcases significant benefits compared to standard approaches.

math.OC↗

arXiv2021

The Power Grid Library for Benchmarking AC Optimal Power Flow Algorithms

In recent years, the power systems research community has seen an explosion of novel methods for formulating the AC power flow equations. Consequently, benchmarking studies using the seminal AC Optimal Power Flow (AC-OPF) problem have emerged as the primary method for evaluating these emerging methods. However, it is often difficult to directly compare these studies due to subtle differences in the AC-OPF problem formulation as well as the network, generation, and loading data that are used for evaluation. To help address these challenges, this IEEE PES Task Force report proposes a standardized AC-OPF mathematical formulation and the PGLib-OPF networks for benchmarking AC-OPF algorithms. A motivating study demonstrates some limitations of the established network datasets in the context of benchmarking AC-OPF algorithms and a validation study demonstrates the efficacy of using the PGLib-OPF networks for this purpose. In the interest of scientific discourse and future additions, the PGLib-OPF benchmark library is open-access and all the of network data is provided under a creative commons license.

math.OC↗

arXiv2021

Commuting with Autonomous Vehicles: A Branch and Cut Algorithm with Redundant Modeling

This paper studies the benefits of autonomous vehicles in ride-sharing platforms dedicated to serving commuting needs. It considers the Commute Trip Sharing Problem with Autonomous Vehicles (CTSPAV), the optimization problem faced by a reservation-based platform that receives daily commute-trip requests and serves them with a fleet of autonomous vehicles. The CTSPAV can be viewed as a special case of the Dial- A-Ride Problem (DARP). However, this paper recognizes that commuting trips exhibit special spatial and temporal properties that can be exploited in a branch and cut algorithm that leverages a redundant modeling approach. In particular, the branch and cut algorithm relies on a MIP formulation that schedules mini routes representing inbound or outbound trips. This formulation is effective in finding high-quality solutions quickly but its relaxation is relatively weak. To remedy this limitation, the mini-route MIP is complemented by a DARP formulation which is not as effective in obtaining primal solutions but has a stronger relaxation. The benefits of the proposed approach are demonstrated by comparing it with another, more traditional, exact branch and cut procedure and a heuristic method based on mini routes. The methodological contribution is complemented by a comprehensive analysis of a CTSPAV platform for reducing vehicle counts, travel distances, and congestion. In particular, the case study for a medium-sized city reveals that a CTSPAV platform can reduce daily vehicle counts by a staggering 92% and decrease vehicles miles by 30%. The platform also significantly reduces congestion, measured as the number of vehicles on the road per unit time, by 60% during peak times. These benefits, however, come at the expense of introducing empty miles. Hence the paper also highlights the tradeoffs between future ride-sharing and car-pooling platforms.

math.OC↗

arXiv2020

The Benefits of Autonomous Vehicles for Community-Based Trip Sharing

This work reconsiders the concept of community-based trip sharing proposed by Hasan et al. (2018) that leverages the structure of commuting patterns and urban communities to optimize trip sharing. It aims at quantifying the benefits of autonomous vehicles for community-based trip sharing, compared to a car-pooling platform where vehicles are driven by their owners. In the considered problem, each rider specifies a desired arrival time for her inbound trip (commuting to work) and a departure time for her outbound trip (commuting back home). In addition, her commute time cannot deviate too much from the duration of a direct trip. Prior work motivated by reducing parking pressure and congestion in the city of Ann Arbor, Michigan, showed that a car-pooling platform for community-based trip sharing could reduce the number of vehicles by close to 60%. This paper studies the potential benefits of autonomous vehicles in further reducing the number of vehicles needed to serve all these commuting trips. It proposes a column-generation procedure that generates and assembles mini routes to serve inbound and outbound trips, using a lexicographic objective that first minimizes the required vehicle count and then the total travel distance. The optimization algorithm is evaluated on a large-scale, real-world dataset of commute trips from the city of Ann Arbor, Michigan. The results of the optimization show that it can leverage autonomous vehicles to reduce the daily vehicle usage by 92%, improving upon the results of the original Commute Trip Sharing Problem by 34%, while also reducing daily vehicle miles traveled by approximately 30%. These results demonstrate the significant potential of autonomous vehicles for the shared commuting of a community to a common work destination.

math.OC↗

arXiv2020

Bias and Variance of Post-processing in Differential Privacy

Post-processing immunity is a fundamental property of differential privacy: it enables the application of arbitrary data-independent transformations to the results of differentially private outputs without affecting their privacy guarantees. When query outputs must satisfy domain constraints, post-processing can be used to project the privacy-preserving outputs onto the feasible region. Moreover, when the feasible region is convex, a widely adopted class of post-processing steps is also guaranteed to improve accuracy. Post-processing has been applied successfully in many applications including census data-release, energy systems, and mobility. However, its effects on the noise distribution is poorly understood: It is often argued that post-processing may introduce bias and increase variance. This paper takes a first step towards understanding the properties of post-processing. It considers the release of census data and examines, both theoretically and empirically, the behavior of a widely adopted class of post-processing functions.

cs.LG↗

arXiv2020

Exact Mixed-integer Convex Programming Formulation for Optimal Water Network Design

In this paper, we consider the canonical water network design problem, which contains nonconvex potential loss functions and discrete resistance choices with varying costs. Traditionally, to resolve the nonconvexities of this problem, relaxations of the potential loss constraints have been applied to yield a more tractable mixed-integer convex program (MICP). However, design solutions to these relaxed problems may not be feasible with respect to the full nonconvex physics. In this paper, it is shown that, in fact, the original mixed-integer nonconvex program can be reformulated exactly as an MICP. Beginning with a convex program previously used for proving nonlinear network design feasibility, strong duality is invoked to construct a novel, convex primal-dual system embedding all physical constraints. This convex system is then augmented to form an exact MICP formulation of the original design problem. Using this novel MICP as a foundation, a global optimization algorithm is developed, leveraging heuristics, outer approximations, and feasibility cutting planes for infeasible designs. Finally, the algorithm is compared against the previous relaxation-based state of the art in water network design over a number of standard benchmark instances from the literature.

math.OC↗

arXiv2020

Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach

A critical concern in data-driven decision making is to build models whose outcomes do not discriminate against some demographic groups, including gender, ethnicity, or age. To ensure non-discrimination in learning tasks, knowledge of the sensitive attributes is essential, while, in practice, these attributes may not be available due to legal and ethical requirements. To address this challenge, this paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints while guaranteeing the privacy of sensitive attributes. The paper analyses the tension between accuracy, privacy, and fairness and the experimental evaluation illustrates the benefits of the proposed model on several prediction tasks.

cs.LG↗

arXiv2020

Differentially Private Optimal Power Flow for Distribution Grids

Although distribution grid customers are obliged to share their consumption data with distribution system operators (DSOs), a possible leakage of this data is often disregarded in operational routines of DSOs. This paper introduces a privacy-preserving optimal power flow (OPF) mechanism for distribution grids that secures customer privacy from unauthorised access to OPF solutions, e.g., current and voltage measurements. The mechanism is based on the framework of differential privacy that allows to control the participation risks of individuals in a dataset by applying a carefully calibrated noise to the output of a computation. Unlike existing private mechanisms, this mechanism does not apply the noise to the optimization parameters or its result. Instead, it optimizes OPF variables as affine functions of the random noise, which weakens the correlation between the grid loads and OPF variables. To ensure feasibility of the randomized OPF solution, the mechanism makes use of chance constraints enforced on the grid limits. The mechanism is further extended to control the optimality loss induced by the random noise, as well as the variance of OPF variables. The paper shows that the differentially private OPF solution does not leak customer loads up to specified parameters.

math.OC↗

arXiv2020

Combining Deep Learning and Optimization for Security-Constrained Optimal Power Flow

The security-constrained optimal power flow (SCOPF) is fundamental in power systems and connects the automatic primary response (APR) of synchronized generators with the short-term schedule. Every day, the SCOPF problem is repeatedly solved for various inputs to determine robust schedules given a set of contingencies. Unfortunately, the modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs, which are hard to solve. To address this challenge, leveraging the wealth of available historical data, this paper proposes a novel approach that combines deep learning and robust optimization techniques. Unlike recent machine-learning applications where the aim is to mitigate the computational burden of exact solvers, the proposed method predicts directly the SCOPF implementable solution. Feasibility is enforced in two steps. First, during training, a Lagrangian dual method penalizes violations of physical and operations constraints, which are iteratively added as necessary to the machine-learning model by a Column-and-Constraint-Generation Algorithm (CCGA). Second, another different CCGA restores feasibility by finding the closest feasible solution to the prediction. Experiments on large test cases show that the method results in significant time reduction for obtaining feasible solutions with an optimality gap below 0.1%.

math.OC↗

arXiv2020

High-Fidelity Machine Learning Approximations of Large-Scale Optimal Power Flow

The AC Optimal Power Flow (AC-OPF) is a key building block in many power system applications. It determines generator setpoints at minimal cost that meet the power demands while satisfying the underlying physical and operational constraints. It is non-convex and NP-hard, and computationally challenging for large-scale power systems. Motivated by the increased stochasticity in generation schedules and increasing penetration of renewable sources, this paper explores a deep learning approach to deliver highly efficient and accurate approximations to the AC-OPF. In particular, the paper proposes an integration of deep neural networks and Lagrangian duality to capture the physical and operational constraints. The resulting model, called OPF-DNN, is evaluated on real case studies from the French transmission system, with up to 3,400 buses and 4,500 lines. Computational results show that OPF-DNN produces highly accurate AC-OPF approximations whose costs are within 0.01% of optimality. OPF-DNN generates, in milliseconds, solutions that capture the problem constraints with high fidelity.

eess.SP↗

arXiv2020

Differentially Private Convex Optimization with Feasibility Guarantees

This paper develops a novel differentially private framework to solve convex optimization problems with sensitive optimization data and complex physical or operational constraints. Unlike standard noise-additive algorithms, that act primarily on the problem data, objective or solution, and disregard the problem constraints, this framework requires the optimization variables to be a function of the noise and exploits a chance-constrained problem reformulation with formal feasibility guarantees. The noise is calibrated to provide differential privacy for identity and linear queries on the optimization solution. For many applications, including resource allocation problems, the proposed framework provides a trade-off between the expected optimality loss and the variance of optimization results.

cs.CR↗

arXiv2020

Pricing under a multinomial logit model with non linear network effects

We study the problem of pricing under a Multinomial Logit model where we incorporate network effects over the consumer's decisions. We analyse both cases, when sellers compete or collaborate. In particular, we pay special attention to the overall expected revenue and how the behaviour of the no purchase option is affected under variations of a network effect parameter. Where for example we prove that the market share for the no purchase option, is decreasing in terms of the value of the network effect, meaning that stronger communication among costumers increases the expected amount of sales. We also analyse how the customer's utility is altered when network effects are incorporated into the market, comparing the cases where both competitive and monopolistic prices are displayed. We use tools from stochastic approximation algorithms to prove that the probability of purchasing the available products converges to a unique stationary distribution. We model that the sellers can use this stationary distribution to establish their strategies. Finding that under those settings, a pure Nash Equilibrium represents the pricing strategies in the case of competition, and an optimal (that maximises the total revenue) fixed price characterise the case of collaboration.

cs.GT↗