arXiv subjects

Jonathan M. Cullen

Publications and source records attributed to Jonathan M. Cullen.

6 recordsLinked to original sources

arXiv2025

Zero-Shot Reinforcement Learning Under Partial Observability

Recent work has shown that, under certain assumptions, zero-shot reinforcement learning (RL) methods can generalise to any unseen task in an environment after reward-free pre-training. Access to Markov states is one such assumption, yet, in many real-world applications, the Markov state is only partially observable. Here, we explore how the performance of standard zero-shot RL methods degrades when subjected to partially observability, and show that, as in single-task RL, memory-based architectures are an effective remedy. We evaluate our memory-based zero-shot RL methods in domains where the states, rewards and a change in dynamics are partially observed, and show improved performance over memory-free baselines. Our code is open-sourced via: https://enjeeneer.io/projects/bfms-with-memory/.

cs.LG↗

arXiv2024

Zero-Shot Reinforcement Learning from Low Quality Data

Zero-shot reinforcement learning (RL) promises to provide agents that can perform any task in an environment after an offline, reward-free pre-training phase. Methods leveraging successor measures and successor features have shown strong performance in this setting, but require access to large heterogenous datasets for pre-training which cannot be expected for most real problems. Here, we explore how the performance of zero-shot RL methods degrades when trained on small homogeneous datasets, and propose fixes inspired by conservatism, a well-established feature of performant single-task offline RL algorithms. We evaluate our proposals across various datasets, domains and tasks, and show that conservative zero-shot RL algorithms outperform their non-conservative counterparts on low quality datasets, and perform no worse on high quality datasets. Somewhat surprisingly, our proposals also outperform baselines that get to see the task during training. Our code is available via https://enjeeneer.io/projects/zero-shot-rl/ .

cs.LG↗

arXiv2023

Low Emission Building Control with Zero-Shot Reinforcement Learning

Heating and cooling systems in buildings account for 31% of global energy use, much of which are regulated by Rule Based Controllers (RBCs) that neither maximise energy efficiency nor minimise emissions by interacting optimally with the grid. Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency, but existing solutions require access to building-specific simulators or data that cannot be expected for every building in the world. In response, we show it is possible to obtain emission-reducing policies without such knowledge a priori--a paradigm we call zero-shot building control. We combine ideas from system identification and model-based RL to create PEARL (Probabilistic Emission-Abating Reinforcement Learning) and show that a short period of active exploration is all that is required to build a performant model. In experiments across three varied building energy simulations, we show PEARL outperforms an existing RBC once, and popular RL baselines in all cases, reducing building emissions by as much as 31% whilst maintaining thermal comfort. Our source code is available online via https://enjeeneer.io/projects/pearl/

cs.LG↗

arXiv2022

Low Emission Building Control with Zero-Shot Reinforcement Learning

Heating and cooling systems in buildings account for 31\% of global energy use, much of which are regulated by Rule Based Controllers (RBCs) that neither maximise energy efficiency nor minimise emissions by interacting optimally with the grid. Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency, but existing solutions require access to building-specific simulators or data that cannot be expected for every building in the world. In response, we show it is possible to obtain emission-reducing policies without such knowledge a priori--a paradigm we call zero-shot building control. We combine ideas from system identification and model-based RL to create PEARL (Probabilistic Emission-Abating Reinforcement Learning) and show that a short period of active exploration is all that is required to build a performant model. In experiments across three varied building energy simulations, we show PEARL outperforms an existing RBC once, and popular RL baselines in all cases, reducing building emissions by as much as 31\% whilst maintaining thermal comfort. Our source code is available online via https://enjeeneer.io/projects/pearl .

cs.LG↗

arXiv2020

Higgs decay to fermion pairs at NLO in SMEFT

The calculation of next-to-leading order (NLO) perturbative corrections at fixed operator dimension in Standard Model Effective Field Theory (SMEFT) has been a topic of much recent interest. In this paper we obtain the NLO corrections from dimension-6 operators to the Higgs boson decays $h\to f\bar{f}$, where the fermions $f \in \{μ,τ,c\}$. This extends previous results for $h\to b\bar{b}$ to all phenomenologically relevant Higgs boson decays into fermions, and provides the basis for future precision analyses of these decays within effective field theory. We point out the benefits of studying ratios of decay rates into different fermions in SMEFT, the most surprising of which is enhanced sensitivity to anomalous $hγγ$ and $hgg$ couplings induced by flavor-universal SMEFT operators, especially in scenarios where flavor-dependent Wilson coefficients are constrained by Minimal Flavor Violation.

hep-ph↗

arXiv2019

NLO corrections to $h\to b\bar b$ decay in SMEFT

We calculate the full set of next-to-leading order (NLO) corrections to $h\to b\bar{b}$ decay in the dimension-6 Standard Model Effective Field Theory (SMEFT). Our calculation forms the basis for precision studies of this decay mode in effective field theory, providing analytic and numerical results for contributions of the 45 dimension-6 operators appearing at NLO. On the technical side, we discuss several complications in NLO SMEFT computations which have not yet been addressed in the literature. These include subtleties in Higgs-$Z$ mixing, electric charge renormalization, and especially the treatment of tadpoles in SMEFT. In particular, we highlight the role of decoupling relations in eliminating potentially large tadpole corrections to the decay rate in hybrid renormalization schemes which employ the $\overline{\hbox{MS}}$~scheme for some Standard Model parameters (such as the $b$-quark mass and electric charge) and the on-shell scheme for others.

hep-ph↗