Searcharxiv⌕ Search

arXiv subjects

Joram Liebeskind

Publications and source records attributed to Joram Liebeskind.

2 recordsLinked to original sources

Comfort as a Constraint or as a Cost in Reinforcement Learning and Model Predictive Building Control

A building controller that keeps a zone within comfort bounds at minimum energy can treat the bounds as a soft constraint, by adding the weighted violation to its cost, or as a hard constraint. We analyze the effect of the hard and the soft formulation of the same problem and isolate the effect of the formulation from the effect of the algorithm. To this end, we measure the difference in energy use between the two formulations on the BOPTEST bestest_hydronic_heat_pump case, in model predictive control (MPC) and in reinforcement learning (RL). The soft formulations are a weighted MPC and weighted SAC and DDPG learners, each at twelve weights. The hard formulations are an MPC with an exact penalty and the Lagrangian learners SAC-Lagrangian and primal-dual DDPG. We find that the hard formulations use more energy than the soft formulations at the largest weight, but violate the comfort bounds less. For a large weight, the soft formulation approaches the behavior of the hard formulation. The frontiers of the energy-comfort trade-off of the three weighted controllers lie close to each other, and the weight selects an operating point on this frontier. Under the soft formulation, the three algorithms reach very similar operating points, although at somewhat different weights. For this HVAC problem, the formulation and the weight thus affect the energy use more than the choice of algorithm.

eess.SY↗

Differentiable Random Partition Models

Partitioning a set of elements into an unknown number of mutually exclusive subsets is essential in many machine learning problems. However, assigning elements, such as samples in a dataset or neurons in a network layer, to an unknown and discrete number of subsets is inherently non-differentiable, prohibiting end-to-end gradient-based optimization of parameters. We overcome this limitation by proposing a novel two-step method for inferring partitions, which allows its usage in variational inference tasks. This new approach enables reparameterized gradients with respect to the parameters of the new random partition model. Our method works by inferring the number of elements per subset and, second, by filling these subsets in a learned order. We highlight the versatility of our general-purpose approach on three different challenging experiments: variational clustering, inference of shared and independent generative factors under weak supervision, and multitask learning.

cs.LG↗