Input Convex Encoder-Only Transformer for Computationally Efficient Model Predictive Control in Building Demand Response

Abstract

Learning-based Model Predictive Control (MPC) has emerged as a powerful strategy for building demand response (DR). However, its practical deployment is often hindered by the non-convex optimization problems induced by standard neural network models. These problems lead to long solver times and a lack of global optimality guarantees, making long-horizon real-time control challenging. Because forecasting building thermal dynamics and energy consumption relies heavily on historical time-series data, Input Convex Neural Networks (ICNNs) tailored for temporal tasks have been developed to address this issue. Notable examples include Input Convex Recurrent Neural Networks (ICRNNs) and Input Convex Long Short-Term Memory (IC-LSTM). Although IC-LSTM improves the modelling capability of simpler ICRNNs, its recurrent architecture remains computationally expensive and, in the experiments reported here, exhibits numerical training instability at longer historical sequence lengths. To address these limitations, this paper introduces the Input Convex Encoder-Only Transformer (IC-EoT), which combines an encoder-only attention architecture with input convexity guarantees. IC-EoT was evaluated in a co-simulation framework using the Energym Python library and the EnergyPlus building simulator, and compared with IC-LSTM and standard non-convex models in residential and commercial testbeds. IC-EoT maintained predictive accuracy comparable to IC-LSTM, showed no non-finite loss events across 60 multi-seed training runs, and reduced mean per-epoch training time by 26.2% and 29.8% in the two testbeds. At the eight-hour prediction horizon, IC-EoT reduced mean MPC solution time by factors of 5.8 and 5.3 relative to IC-LSTM, while providing comparable closed-loop cost and comfort performance.