arXiv subjects

David Ireland

Publications and source records attributed to David Ireland.

6 recordsLinked to original sources

arXiv2024

An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces

Expanding reinforcement learning (RL) to offline domains generates promising prospects, particularly in sectors where data collection poses substantial challenges or risks. Pivotal to the success of transferring RL offline is mitigating overestimation bias in value estimates for state-action pairs absent from data. Whilst numerous approaches have been proposed in recent years, these tend to focus primarily on continuous or small-scale discrete action spaces. Factorised discrete action spaces, on the other hand, have received relatively little attention, despite many real-world problems naturally having factorisable actions. In this work, we undertake a formative investigation into offline reinforcement learning in factorisable action spaces. Using value-decomposition as formulated in DecQN as a foundation, we present the case for a factorised approach and conduct an extensive empirical evaluation of several offline techniques adapted to the factorised setting. In the absence of established benchmarks, we introduce a suite of our own comprising datasets of varying quality and task complexity. Advocating for reproducible research and innovation, we make all datasets available for public use alongside our code base.

stat.ML↗

arXiv2024

REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes

Discrete-action reinforcement learning algorithms often falter in tasks with high-dimensional discrete action spaces due to the vast number of possible actions. A recent advancement leverages value-decomposition, a concept from multi-agent reinforcement learning, to tackle this challenge. This study delves deep into the effects of this value-decomposition, revealing that whilst it curtails the over-estimation bias inherent to Q-learning algorithms, it amplifies target variance. To counteract this, we present an ensemble of critics to mitigate target variance. Moreover, we introduce a regularisation loss that helps to mitigate the effects that exploratory actions in one dimension can have on the value of optimal actions in other dimensions. Our novel algorithm, REValueD, tested on discretised versions of the DeepMind Control Suite tasks, showcases superior performance, especially in the challenging humanoid and dog tasks. We further dissect the factors influencing REValueD's performance, evaluating the significance of the regularisation loss and the scalability of REValueD with increasing sub-actions per dimension.

cs.LG↗

arXiv2023

Deep Learning Level-3 Electron Trigger for CLAS12

Fast, efficient and accurate triggers are a critical requirement for modern high-energy physics experiments given the increasingly large quantities of data that they produce. The CEBAF Large Acceptance Spectrometer (CLAS12) employs a highly efficient electron trigger to filter the amount of recorded data by requiring at least one electron in each event, at the cost of a low purity in electron identification. Machine learning algorithms are increasingly employed for classification tasks such as particle identification due to their high accuracy and fast processing times. In this article, we show how a convolutional neural network could be deployed as a Level 3 electron trigger at CLAS12. We demonstrate that the AI trigger would achieve a significant data reduction compared to the traditional trigger, whilst preserving a 99.5\% electron identification efficiency. The AI trigger purity as a function of increased luminosity is improved relative to the traditional trigger. As a consequence, this AI trigger can achieve a data recording reduction improvement of 0.33\% per nA when compared to the traditional trigger whilst maintaining an efficiency above 99.5\%. A reduction in data output also reduces storage costs and post-processing times, which in turn reduces the time to the publication of new physics measurements.

physics.ins-det↗

arXiv2022

LeNSE: Learning To Navigate Subgraph Embeddings for Large-Scale Combinatorial Optimisation

Combinatorial Optimisation problems arise in several application domains and are often formulated in terms of graphs. Many of these problems are NP-hard, but exact solutions are not always needed. Several heuristics have been developed to provide near-optimal solutions; however, they do not typically scale well with the size of the graph. We propose a low-complexity approach for identifying a (possibly much smaller) subgraph of the original graph where the heuristics can be run in reasonable time and with a high likelihood of finding a global near-optimal solution. The core component of our approach is LeNSE, a reinforcement learning algorithm that learns how to navigate the space of possible subgraphs using an Euclidean subgraph embedding as its map. To solve CO problems, LeNSE is provided with a discriminative embedding trained using any existing heuristics using only on a small portion of the original graph. When tested on three problems (vertex cover, max-cut and influence maximisation) using real graphs with up to $10$ million edges, LeNSE identifies small subgraphs yielding solutions comparable to those found by running the heuristics on the entire graph, but at a fraction of the total run time.

cs.LG↗

arXiv2020

Strange Hadron Spectroscopy with Secondary KL Beam in Hall D

We propose to create a secondary beam of neutral kaons in Hall D at Jefferson Lab to be used with the GlueX experimental setup for strange hadron spectroscopy. The superior CEBAF electron beam will enable a flux on the order of $1\times 10^4~K_L/sec$, which exceeds the flux of that previously attained at SLAC by three orders of magnitude. The use of a deuteron target will provide first measurements ever with neutral kaons on neutrons. The experiment will measure both differential cross sections and self-analyzed polarizations of the produced $\Lambda$, $\Sigma$, $\Xi$, and $\Omega$ hyperons using the GlueX detector at the Jefferson Lab Hall D. The measurements will span CM $\cos\theta$ from $-0.95$ to 0.95 in the range W = 1490 MeV to 2500 MeV. The new data will significantly constrain the partial wave analyses and reduce model-dependent uncertainties in the extraction of the properties and pole positions of the strange hyperon resonances, and establish the orbitally excited multiplets in the spectra of the $\Xi$ and $\Omega$ hyperons. Comparison with the corresponding multiplets in the spectra of the charm and bottom hyperons will provide insight into he accuracy of QCD-based calculations over a large range of masses. The proposed facility will have a defining impact in the strange meson sector through measurements of the final state $K\pi$ system up to 2 GeV invariant mass. This will allow the determination of pole positions and widths of all relevant $K^\ast(K\pi)$ $S$-,$P$-,$D$-,$F$-, and $G$-wave resonances, settle the question of the existence or nonexistence of scalar meson $\kappa/K_0^\ast(700)$ and improve the constrains on their pole parameters. Subsequently improving our knowledge of the low-lying scalar nonet in general.

nucl-ex↗

arXiv2012

Measurement of the generalized form factors near threshold via $\gamma^* p \to n\pi^+$ at high $Q^2$

We report the first extraction of the pion-nucleon multipoles near the production threshold for the $n\pi^+$ channel at relatively high momentum transfer ($Q^2$ up to 4.2 $\rm{GeV^2}$). The dominance of the s-wave transverse multipole ($E_{0+}$), expected in this region, allowed us to access the generalized form factor $G_1$ within the light-cone sum rule (LCSR) framework as well as the axial form factor $G_A$. The data analyzed in this work were collected by the nearly $4\pi$ CEBAF Large Acceptance Spectrometer (CLAS) using a 5.754 $\rm{GeV}$ electron beam on a proton target. The differential cross section and the $\pi-N$-multipole $E_{0+}/G_D$ were measured using two different methods, the LCSR and a direct multipole fit. The results from the two methods are found to be consistent and almost $Q^2$ independent.

nucl-ex↗