arXiv subjects

Colin N. Jones

Publications and source records attributed to Colin N. Jones.

At least 55 records · Page 3Linked to original sources

arXiv2023

Multi-Agent Bayesian Optimization with Coupled Black-Box and Affine Constraints

This paper studies the problem of distributed multi-agent Bayesian optimization with both coupled black-box constraints and known affine constraints. A primal-dual distributed algorithm is proposed that achieves similar regret/violation bounds as those in the single-agent case for the black-box objective and constraint functions. Additionally, the algorithm guarantees an $\mathcal{O}(N\sqrt{T})$ bound on the cumulative violation for the known affine constraints, where $N$ is the number of agents. Hence, it is ensured that the average of the samples satisfies the affine constraints up to the error $\mathcal{O}({N}/{\sqrt{T}})$. Furthermore, we characterize certain conditions under which our algorithm can bound a stronger metric of cumulative violation and provide best-iterate convergence without affine constraint. The method is then applied to both sampled instances from Gaussian processes and a real-world optimal power allocation problem for wireless communication; the results show that our method simultaneously provides close-to-optimal performance and maintains minor violations on average, corroborating our theoretical analysis.

cs.LG↗

arXiv2023

Primal-Dual Contextual Bayesian Optimization for Control System Online Optimization with Time-Average Constraints

This paper studies the problem of online performance optimization of constrained closed-loop control systems, where both the objective and the constraints are unknown black-box functions affected by exogenous time-varying contextual disturbances. A primal-dual contextual Bayesian optimization algorithm is proposed that achieves sublinear cumulative regret with respect to the dynamic optimal solution under certain regularity conditions. Furthermore, the algorithm achieves zero time-average constraint violation, ensuring that the average value of the constraint function satisfies the desired constraint. The method is applied to both sampled instances from Gaussian processes and a continuous stirred tank reactor parameter tuning problem; simulation results show that the method simultaneously provides close-to-optimal performance and maintains constraint feasibility on average. This contrasts current state-of-the-art methods, which either suffer from large cumulative regret or severe constraint violations for the case studies presented.

cs.LG↗

arXiv2023

PIQP: A Proximal Interior-Point Quadratic Programming Solver

This paper presents PIQP, a high-performance toolkit for solving generic sparse quadratic programs (QP). Combining an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM), the algorithm can handle ill-conditioned convex QP problems without the need for linear independence of the constraints. The open-source implementation is written in C++ with interfaces to C, Python, Matlab, and R leveraging the Eigen3 library. The method uses a pivoting-free factorization routine and allocation-free updates of the problem data, making the solver suitable for embedded applications. The solver is evaluated on the Maros-Mészáros problem set and optimal control problems, demonstrating state-of-the-art performance for both small and large-scale problems, outperforming commercial and open-source solvers.

math.OC↗

arXiv2023

Computationally Efficient Reinforcement Learning: Targeted Exploration leveraging Simple Rules

Model-free Reinforcement Learning (RL) generally suffers from poor sample complexity, mostly due to the need to exhaustively explore the state-action space to find well-performing policies. On the other hand, we postulate that expert knowledge of the system often allows us to design simple rules we expect good policies to follow at all times. In this work, we hence propose a simple yet effective modification of continuous actor-critic frameworks to incorporate such rules and avoid regions of the state-action space that are known to be suboptimal, thereby significantly accelerating the convergence of RL agents. Concretely, we saturate the actions chosen by the agent if they do not comply with our intuition and, critically, modify the gradient update step of the policy to ensure the learning process is not affected by the saturation step. On a room temperature control case study, it allows agents to converge to well-performing policies up to 6-7x faster than classical agents without computational overhead and while retaining good final performance.

cs.LG↗

arXiv2023

A Generalized Stopping Criterion for Real-Time MPC with Guaranteed Stability

Most of the real-time implementations of the stabilizing optimal control actions suffer from the necessity to provide high computational effort. This paper presents a cutting-edge approach for real-time evaluation of linear-quadratic model predictive control (MPC) that employs a novel generalized stopping criterion, achieving asymptotic stability in the presence of input constraints. The proposed method evaluates a fixed number of iterations independent of the initial condition, eliminating the necessity for computationally expensive methods. We demonstrate the effectiveness of the introduced technique by its implementation of two widely-used first-order optimization methods: the projected gradient descent method (PGDM) and the alternating directions method of multipliers (ADMM). The numerical simulation confirmed a significantly reduced number of iterations, resulting in suboptimality rates of less than 2\,\%, while the effort reductions exceeded 80\,\%. These results nominate the proposed criterion for an efficient real-time implementation method of MPC controllers.

eess.SY↗

arXiv2023

Data-Driven Robust Control Using Prediction Error Bounds Based on Perturbation Analysis

For linear systems, many data-driven control methods rely on the behavioral framework, using historical data of the system to predict the future trajectories. However, measurement noise introduces errors in predictions. When the noise is bounded, we propose a method for designing historical experiments that enable the computation of an upper bound on the prediction error. This approach allows us to formulate a minimax control problem where robust constraint satisfaction is enforced. We derive an upper bound on the suboptimality gap of the resulting control input sequence compared to optimal control utilizing accurate measurements. As demonstrated in numerical experiments, the solution derived by our method can achieve constraint satisfaction and a small suboptimality gap despite the measurement noise.

math.OC↗

arXiv2023

Bayesian Optimization of Expensive Nested Grey-Box Functions

We consider the problem of optimizing a grey-box objective function, i.e., nested function composed of both black-box and white-box functions. A general formulation for such grey-box problems is given, which covers the existing grey-box optimization formulations as special cases. We then design an optimism-driven algorithm to solve it. Under certain regularity assumptions, our algorithm achieves similar regret bound as that for the standard black-box Bayesian optimization algorithm, up to a constant multiplicative term depending on the Lipschitz constants of the functions considered. We further extend our method to the constrained case and discuss special cases. For the commonly used kernel functions, the regret bounds allow us to derive a convergence rate to the optimal solution. Experimental results show that our grey-box optimization method empirically improves the speed of finding the global optimal solution significantly, as compared to the standard black-box optimization algorithm.

cs.LG↗

arXiv2023

Hypergraph-Based Fast Distributed AC Power Flow Optimization

This paper presents a novel distributed approach for solving AC power flow (PF) problems. The optimization problem is reformulated into a distributed form using a communication structure corresponding to a hypergraph, by which complex relationships between subgrids can be expressed as hyperedges. Then, a hypergraph-based distributed sequential quadratic programming (HDQ) approach is proposed to handle the reformulated problems, and the hypergraph-based distributed sequential quadratic programming (HDSQP) is used as the inner algorithm to solve the corresponding QP subproblems, which are respectively condensed using Schur complements with respect to coupling variables defined by hyperedges. Furthermore, we rigorously establish the convergence guarantee of the proposed algorithm with a locally quadratic rate and the one-step convergence of the inner algorithm when using the Levenberg-Marquardt regularization. Our analysis also demonstrates that the computational complexity of the proposed algorithm is much lower than the state-of-art distributed algorithm. We implement the proposed algorithm in an open-source toolbox, i.e., rapidPF, and conduct numerical tests that validate the proof and demonstrate the great potential of the proposed distributed algorithm in terms of communication effort and computational speed.

math.OC↗

arXiv2023

A Proximal-Point Lagrangian Based Parallelizable Nonconvex Solver for Bilinear Model Predictive Control

Nonlinear model predictive control has been widely adopted to manipulate bilinear systems with dynamics that include products of the inputs and the states. These systems are ubiquitous in chemical processes, mechanical systems, and quantum physics, to name a few. Running a bilinear MPC controller in real time requires solving a non-convex optimization problem within a limited sampling time. This paper proposes a novel parallel proximal-point Lagrangian based bilinear MPC solver via an interlacing horizon-splitting scheme. The resulting algorithm converts the non-convex MPC control problem into a set of parallelizable small-scale multi-parametric quadratic programs (mpQPs) and an equality-constrained linear-quadratic regulator problem. As a result, the solutions of mpQPs can be pre-computed offline to enable efficient online computation. The proposed algorithm is validated on a simulation of an HVAC system control. It is deployed on a TI LaunchPad XL F28379D microcontroller to execute speed control on a field-controlled DC motor, where the MPC updates at 10 ms and solves the problem in 1.764 ms on average and at most 2.088 ms.

eess.SY↗

arXiv2023

Physically Consistent Multiple-Step Data-Driven Predictions Using Physics-based Filters

(Extended Version) Data-driven control can facilitate the rapid development of controllers, offering an alternative to conventional approaches. In order to maintain consistency between any known underlying physical laws and a data-driven decision-making process, preprocessing of raw data is necessary to account for measurement noise and any inconsistencies it may introduce. In this paper, we present a physics-based filter to achieve this and demonstrate its effectiveness through practical applications, using real-world datasets collected in a building on the Ecole Polytechnique Federale de Lausanne (EPFL) campus. Two distinct use cases are explored: indoor temperature control and demand response bidding.

eess.SY↗

arXiv2023

Stability Verification of Neural Network Controllers using Mixed-Integer Programming

We propose a framework for the stability verification of Mixed-Integer Linear Programming (MILP) representable control policies. This framework compares a fixed candidate policy, which admits an efficient parameterization and can be evaluated at a low computational cost, against a fixed baseline policy, which is known to be stable but expensive to evaluate. We provide sufficient conditions for the closed-loop stability of the candidate policy in terms of the worst-case approximation error with respect to the baseline policy, and we show that these conditions can be checked by solving a Mixed-Integer Quadratic Program (MIQP). Additionally, we demonstrate that an outer and inner approximation of the stability region of the candidate policy can be computed by solving an MILP. The proposed framework is sufficiently general to accommodate a broad range of candidate policies including ReLU Neural Networks (NNs), optimal solution maps of parametric quadratic programs, and Model Predictive Control (MPC) policies. We also present an open-source toolbox in Python based on the proposed framework, which allows for the easy verification of custom NN architectures and MPC formulations. We showcase the flexibility and reliability of our framework in the context of a DC-DC power converter case study and investigate its computational complexity.

eess.SY↗

arXiv2023

A comparison of methods to eliminate regularization weight tuning from data-enabled predictive control

Data-enabled predictive control (DeePC) is a recently established form of Model Predictive Control (MPC), based on behavioral systems theory. While eliminating the need to explicitly identify a model, it requires an additional regularization with a corresponding weight to function well with noisy data. The tuning of this weight is non-trivial and has a significant impact on performance. In this paper, we compare three reformulations of DeePC that either eliminate the regularization, or simplify the tuning to a trivial point. A building simulation study shows a comparable performance for all three reformulations of DeePC. However, a conventional MPC with a black-box model slightly outperforms them, while solving much faster, and yielding smoother optimal trajectories. Two of the DeePC variants also show sensitivity to an unobserved biased input noise, which is not present in the conventional MPC.

eess.SY↗

arXiv2023

Distributed data-driven predictive control for cooperatively smoothing mixed traffic flow

Cooperative control of connected and automated vehicles (CAVs) promises smoother traffic flow. In mixed traffic, where human-driven vehicles with unknown dynamics coexist, data-driven predictive control techniques allow for CAV safe and optimal control with measurable traffic data. However, the centralized control setting in most existing strategies limits their scalability for large-scale mixed traffic flow. To address this problem, this paper proposes a cooperative DeeP-LCC (Data-EnablEd Predictive Leading Cruise Control) formulation and its distributed implementation algorithm. In cooperative DeeP-LCC, the traffic system is naturally partitioned into multiple subsystems with one single CAV, which collects local trajectory data for subsystem behavior predictions based on the Willems' fundamental lemma. Meanwhile, the cross-subsystem interaction is formulated as a coupling constraint. Then, we employ the Alternating Direction Method of Multipliers (ADMM) to design the distributed DeeP-LCC algorithm. This algorithm achieves both computation and communication efficiency, as well as trajectory data privacy, through parallel calculation. Our simulations on different traffic scales verify the real-time wave-dampening potential of distributed DeeP-LCC, which can reduce fuel consumption by over 31.84% in a large-scale traffic system of 100 vehicles with only 5%-20% CAVs.

eess.SY↗

arXiv2023

Comparison of behavioral systems theory and conventional linear models for predicting building zone temperature in long-term in situ measurements

The potential of Model Predictive Control in buildings has been shown many times, being successfully used to achieve various goals, such as minimizing energy consumption or maximizing thermal comfort. However, mass deployment has thus far failed, in part because of the high engineering cost of obtaining and maintaining a sufficiently accurate model. This can be addressed by using adaptive data-driven approaches. The idea of using behavioral systems theory for this purpose has recently found traction in the academic community. In this study, we compare variations thereof with different amounts of data used, different regularization weights, and different methods of data selection. Autoregressive models with exogenous inputs (ARX) are used as a well-established reference. All methods are evaluated by performing iterative system identification on two long-term data sets from real occupied buildings, neither of which include artificial excitation for the purpose of system identification. We find that: (1) Sufficient prediction accuracy is achieved with all methods. (2) The ARX models perform slightly better, while having the additional advantages of fewer tuning parameters and faster computation. (3) Adaptive and non-adaptive schemes perform similarly. (4) The regularization weights of the behavioral systems theory methods show the expected trade-off characteristic with an optimal middle value. (5) Using the most recent data yields better performance than selecting data with similar weather as the day to be predicted. (6) More data improves the model performance.

stat.AP↗

arXiv2022

CONFIG: Constrained Efficient Global Optimization for Closed-Loop Control System Optimization with Unmodeled Constraints

In this paper, the CONFIG algorithm, a simple and provably efficient constrained global optimization algorithm, is applied to optimize the closed-loop control performance of an unknown system with unmodeled constraints. Existing Gaussian process based closed-loop optimization methods, either can only guarantee local convergence (e.g., SafeOPT), or have no known optimality guarantee (e.g., constrained expected improvement) at all, whereas the recently introduced CONFIG algorithm has been proven to enjoy a theoretical global optimality guarantee. In this study, we demonstrate the effectiveness of CONFIG algorithm in the applications. The algorithm is first applied to an artificial numerical benchmark problem to corroborate its effectiveness. It is then applied to a classical constrained steady-state optimization problem of a continuous stirred-tank reactor. Simulation results show that our CONFIG algorithm can achieve performance competitive with the popular CEI (Constrained Expected Improvement) algorithm, which has no known optimality guarantee. As such, the CONFIG algorithm offers a new tool, with both a provable global optimality guarantee and competitive empirical performance, to optimize the closed-loop control performance for a system with soft unmodeled constraints. Last, but not least, the open-source code is available as a python package to facilitate future applications.

math.OC↗

arXiv2022

Physically Consistent Neural ODEs for Learning Multi-Physics Systems

Despite the immense success of neural networks in modeling system dynamics from data, they often remain physics-agnostic black boxes. In the particular case of physical systems, they might consequently make physically inconsistent predictions, which makes them unreliable in practice. In this paper, we leverage the framework of Irreversible port-Hamiltonian Systems (IPHS), which can describe most multi-physics systems, and rely on Neural Ordinary Differential Equations (NODEs) to learn their parameters from data. Since IPHS models are consistent with the first and second principles of thermodynamics by design, so are the proposed Physically Consistent NODEs (PC-NODEs). Furthermore, the NODE training procedure allows us to seamlessly incorporate prior knowledge of the system properties in the learned dynamics. We demonstrate the effectiveness of the proposed method by learning the thermodynamics of a building from the real-world measurements and the dynamics of a simulated gas-piston system. Thanks to the modularity and flexibility of the IPHS framework, PC-NODEs can be extended to learn physically consistent models of multi-physics distributed systems.

cs.LG↗

arXiv2022

Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning

Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order to second-order methods. However, these algorithms cannot be applied in scenarios where the gradient information is not available, e.g., federated black-box attack and federated hyperparameter tuning. To address this issue, in this paper we propose a derivative-free federated zeroth-order optimization (FedZO) algorithm featured by performing multiple local updates based on stochastic gradient estimators in each communication round and enabling partial device participation. Under non-convex settings, we derive the convergence performance of the FedZO algorithm on non-independent and identically distributed data and characterize the impact of the numbers of local iterates and participating edge devices on the convergence. To enable communication-efficient FedZO over wireless networks, we further propose an over-the-air computation (AirComp) assisted FedZO algorithm. With an appropriate transceiver design, we show that the convergence of AirComp-assisted FedZO can still be preserved under certain signal-to-noise ratio conditions. Simulation results demonstrate the effectiveness of the FedZO algorithm and validate the theoretical observations.

cs.LG↗

arXiv2022

Uncertainty-aware Flexibility Envelope Prediction in Buildings with Controller-agnostic Battery Models

Buildings are a promising source of flexibility for the application of demand response. In this work, we introduce a novel battery model formulation to capture the state evolution of a single building. Being fully data-driven, the battery model identification requires one dataset from a period of nominal controller operation, and one from a period with flexibility requests, without making any assumptions on the underlying controller structure. We consider parameter uncertainty in the model formulation and show how to use risk measures to encode risk preferences of the user in robust uncertainty sets. Finally, we demonstrate the uncertainty-aware prediction of flexibility envelopes for a building simulation model from the Python library Energym.

eess.SY↗