arXiv subjects

Sascha Caron

Publications and source records attributed to Sascha Caron.

At least 19 recordsLinked to original sources

arXiv2026

Masked-Token Prediction for Anomaly Detection at the Large Hadron Collider

Anomaly detection in High Energy Physics requires identifying rare signals against overwhelming backgrounds, without prior knowledge of the signal. We present the first application of masked-token prediction, a technique from Large Language Models, to this problem. A lightweight encoder architecture trained solely on background events captures the structure of Standard Model (SM) physics; at inference, sequences deviating from this learned structure are flagged as anomalous. We evaluate the approach on searches for four-top-quark production and supersymmetric gluino pair production, both featuring top-rich final states with substantial missing transverse energy, covering SM and beyond the Standard Model (BSM) scenarios. Strong performance on the four-top signature, which closely resembles background, demonstrates the method's sensitivity to subtle deviations. We further show that the tokenization strategy significantly impacts performance: deep-learned tokenization via vector-quantized variational autoencoders (VQ-VAE) outperforms look-up table tokenization. Comparison with established anomaly detection baselines confirms robustness. These results highlight the potential of token-based collider data representations combined with transformer architectures for new-physics discovery. Once trained on SM background, the model transfers across different BSM searches, enabling scalable, model-independent anomaly detection at reduced computational cost.

hep-ph↗

arXiv2026

On the Codesign of Scientific Experiments and Industrial Systems

The optimization of large experiments in fundamental science, such as detectors for subnuclear physics at particle colliders, shares with the optimization of complex systems for industrial or societal applications the common issue of addressing the inter-relation between parameters describing the hardware used in data production and parameters used to analyse those data. While in many cases this coupling can be ignored -- when the problem can be successfully factored into simpler sub-tasks and the latter addressed serially -- there are situations in which that approach fails to converge to the absolute maximum of expected performance, as it results in a mis-alignment of the optimized hardware and software solutions. In this work we consider a few use cases of interest in fundamental science collected primarily from particle physics and related areas, and a pot-pourri of industrial and societal applications where the matter is similarly of relevance. We discuss the emergence of strong hardware-software coupling in some of those systems, as well as co-design procedures that may be deployed to identify the global maximum of their relevant utility functions. We observe how numerous opportunities exist to advance methods and tools for hardware-software co-design optimization, bridging fundamental science and industry through application- and challenge-driven projects, and shaping the future of scientific experiments and industrial systems.

physics.ins-det↗

arXiv2025

BGRem: A background noise remover for astronomical images based on a diffusion model

Context: Astronomical imaging aims to maximize signal capture while minimizing noise. Enhancing the signal-to-noise ratio directly on detectors is difficult and expensive, leading to extensive research in advanced post-processing techniques. Aims: Removing background noise from images is a valuable pre-processing step catalog-building tasks. We introduce BGRem, a machine learning (ML) based tool to remove background noise from astronomical images. Methods: BGRem uses a diffusion-based model with an attention U-Net as backbone, trained on simulated images for optical and gamma ({\gamma})-ray data from the MeerLICHT and Fermi-LAT telescopes. In a supervised manner, BGRem learns to denoise astronomical images over several diffusion steps. Results: BGRem performance was compared with a widely used tool for cataloging astronomical sources, SourceExtractor (SExtractor). It was shown that the amount of true positive sources using SExtractor increased by about 7% for MeerLICHT data when BGRem was used as a pre-processing step. We also show the generalizability of BGRem by testing it with optical images from different telescopes and also on simulated {\gamma}-ray data representative of the Fermi-LAT telescope. We show that in both cases, BGRem improves the source detection efficiency. Conclusions: BGRem can improve the accuracy in source detection of traditional pixel-based methods by removing complex background noise. Using zero-shot approach, BGRem can generalize well to a wide range of optical images. The successful application of BGRem to simulated {\gamma}-ray images, alongside optical data, demonstrates its adaptability to distinct noise characteristics and observational domains. This cross-wavelength performance highlights its potential as a general-purpose background removal framework for multi-wavelength astronomical surveys.

astro-ph.IM↗

arXiv2025

Event Tokenization and Masked-Token Prediction for Anomaly Detection at the Large Hadron Collider

We propose a novel use of Large Language Models (LLMs) as unsupervised anomaly detectors in particle physics. Using lightweight LLM-like networks with encoder-based architectures trained to reconstruct background events via masked-token prediction, our method identifies anomalies through deviations in reconstruction performance, without prior knowledge of signal characteristics. Applied to searches for simultaneous four-top-quark production, this token-based approach shows competitive performance against established unsupervised methods and effectively captures subtle discrepancies in collider data, suggesting a promising direction for model-independent searches for new physics.

hep-ex↗

arXiv2025

TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction

High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this study, we extend our earlier efforts by conducting detailed investigations into more custom Transformer attention mechanisms, a new design combining geometric projection and lightweight clustering, and a joint model conditioning classification on a regressor's predictions. Furthermore, we discuss new datasets that allow the training on hit level for a range of physics processes. These developments collectively aim to boost both the accuracy and potentially the efficiency of our tracking models, offering a robust solution to meet the demands of next-generation high-energy physics experiments.

hep-ex↗

arXiv2025

Towards a foundation model for astrophysical source detection: An End-to-End Gamma-Ray Data Analysis Pipeline Using Deep Learning

The increasing volume of gamma-ray data demands new analysis approaches that can handle large-scale datasets while providing robustness for source detection. We present a Deep Learning (DL) based pipeline for detection, localization, and characterization of gamma-ray sources. We extend our AutoSourceID (ASID) method, initially tested with \textit{Fermi}-LAT simulated data and optical data (MeerLICHT), to Cherenkov Telescope Array Observatory (CTAO) simulated data. This end-to-end pipeline demonstrates a versatile framework for future application to other surveys and potentially serves as a building block for a foundational model for astrophysical source detection.

astro-ph.IM↗

arXiv2025

Hits to Higgs: Hit-Level Higgs Classification from Raw LHC Detector Data Using Higgsformer

We present Higgsformer, a transformer-based architecture that classifies Higgs events at the Large Hadron Collider directly from raw inner tracker hits, bypassing the traditional reconstruction chain of intermediate physics objects. As a benchmark, we focus on distinguishing $t\bar{t}H$ from $t\bar{t}$ events with $H \to b\bar{b}$, a particularly challenging task due to their similar final state topologies. Our pipeline begins with event generation in Pythia8, fast simulation with ACTS/Fatras, and classification directly from raw detector hits. We show for the first time that a transformer model originally developed for inner tracker hit-to-track assignment can be retrained to classify Higgs signal events directly from raw hits. For comparison, we reconstruct the same events with Delphes and train a Particle Transformer as an object-based classifier. We evaluate both approaches under varying dataset sizes and pileup levels. Despite relying exclusively on inner tracker hits, our large Higgsformer achieves an AUC of $0.855$, matching the performance of the traditional reconstruction pipeline at a $b$-tagging efficiency of $\approx 40\%$ under the same detector constraints.

hep-ph↗

arXiv2025

Towards a Large Physics Benchmark

We introduce a benchmark framework developed by and for the scientific community to evaluate, monitor and steer large language model development in fundamental physics. Building on philosophical concepts of scientific understanding and creativity, we develop a scoring system in which each question is scored by an expert for its correctness, difficulty, and surprise. The questions are of three forms: (i) multiple-choice questions for conceptual understanding, (ii) analytical problems requiring mathematical derivation, and (iii) openended tasks requiring complex problem solving. Our current dataset contains diverse set of examples, including a machine learning challenge to classify high-energy physics events, such as the four top quark signal. To ensure continued relevance, we propose a living benchmark, where physicists contribute questions, for instance alongside new publications. We invite contributions via: http://www.physicsbenchmarks.org/. We hope that this benchmark will enable a targeted AI development that can make a meaningful contribution to fundamental physics research.

physics.data-an↗

arXiv2025

Strategic White Paper on AI Infrastructure for Particle, Nuclear, and Astroparticle Physics: Insights from JENA and EuCAIF

Artificial intelligence (AI) is transforming scientific research, with deep learning methods playing a central role in data analysis, simulations, and signal detection across particle, nuclear, and astroparticle physics. Within the JENA communities-ECFA, NuPECC, and APPEC-and as part of the EuCAIF initiative, AI integration is advancing steadily. However, broader adoption remains constrained by challenges such as limited computational resources, a lack of expertise, and difficulties in transitioning from research and development (R&D) to production. This white paper provides a strategic roadmap, informed by a community survey, to address these barriers. It outlines critical infrastructure requirements, prioritizes training initiatives, and proposes funding strategies to scale AI capabilities across fundamental physics over the next five years.

astro-ph.IM↗

arXiv2025

Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models

This paper explores ideas and provides a potential roadmap for the development and evaluation of physics-specific large-scale AI models, which we call Large Physics Models (LPMs). These models, based on foundation models such as Large Language Models (LLMs) - trained on broad data - are tailored to address the demands of physics research. LPMs can function independently or as part of an integrated framework. This framework can incorporate specialized tools, including symbolic reasoning modules for mathematical manipulations, frameworks to analyse specific experimental and simulated data, and mechanisms for synthesizing theories and scientific literature. We begin by examining whether the physics community should actively develop and refine dedicated models, rather than relying solely on commercial LLMs. We then outline how LPMs can be realized through interdisciplinary collaboration among experts in physics, computer science, and philosophy of science. To integrate these models effectively, we identify three key pillars: Development, Evaluation, and Philosophical Reflection. Development focuses on constructing models capable of processing physics texts, mathematical formulations, and diverse physical data. Evaluation assesses accuracy and reliability by testing and benchmarking. Finally, Philosophical Reflection encompasses the analysis of broader implications of LLMs in physics, including their potential to generate new scientific understanding and what novel collaboration dynamics might arise in research. Inspired by the organizational structure of experimental collaborations in particle physics, we propose a similarly interdisciplinary and collaborative approach to building and refining Large Physics Models. This roadmap provides specific objectives, defines pathways to achieve them, and identifies challenges that must be addressed to realise physics-specific large scale AI models.

physics.data-an↗

arXiv2024

TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era

High-Energy Physics experiments are facing a multi-fold data increase with every new iteration. This is certainly the case for the upcoming High-Luminosity LHC upgrade. Such increased data processing requirements forces revisions to almost every step of the data processing pipeline. One such step in need of an overhaul is the task of particle track reconstruction, a.k.a., tracking. A Machine Learning-assisted solution is expected to provide significant improvements, since the most time-consuming step in tracking is the assignment of hits to particles or track candidates. This is the topic of this paper. We take inspiration from large language models. As such, we consider two approaches: the prediction of the next word in a sentence (next hit point in a track), as well as the one-shot prediction of all hits within an event. In an extensive design effort, we have experimented with three models based on the Transformer architecture and one model based on the U-Net architecture, performing track association predictions for collision event hit points. In our evaluation, we consider a spectrum of simple to complex representations of the problem, eliminating designs with lower metrics early on. We report extensive results, covering both prediction accuracy (score) and computational performance. We have made use of the REDVID simulation framework, as well as reductions applied to the TrackML data set, to compose five data sets from simple to complex, for our experiments. The results highlight distinct advantages among different designs in terms of prediction accuracy and computational performance, demonstrating the efficiency of our methodology. Most importantly, the results show the viability of a one-shot encoder-classifier based Transformer solution as a practical approach for the task of tracking.

hep-ex↗

arXiv2024

Universal Anomaly Detection at the LHC: Transforming Optimal Classifiers and the DDD Method

In this work, we present a novel approach to transform supervised classifiers into effective unsupervised anomaly detectors. The method we have developed, termed Discriminatory Detection of Distortions (DDD), enhances anomaly detection by training a discriminator model on both original and artificially modified datasets. We conducted a comprehensive evaluation of our models on the Dark Machines Anomaly Score Challenge channels and a search for 4-top quark events, demonstrating the effectiveness of our approach across various final states and beyond the Standard Model scenarios. We compare the performance of the DDD method with the Deep Robust One-Class Classification method (DROCC), which incorporates signals in the training process, and the Deep Support Vector Data Description (DeepSVDD) method, a well-established and well-performing method for anomaly detection. Results show that the effectiveness of each model varies by signal and channel, with DDD proving to be a very effective anomaly detector. We recommend the combined use of DeepSVDD and DDD for purely unsupervised applications, with the addition of flow models for improved performance when resources allow. Findings suggest that network architectures that excel in supervised contexts, such as the particle transformer with standard model interactions, also perform well as unsupervised anomaly detectors. We also show that with these methods, it is likely possible to recognize 4-top quark production as an anomaly without prior knowledge of the process. We argue that the Large Hadron Collider community can transform supervised classifiers into anomaly detectors to uncover potential new physical phenomena in each search.

hep-ph↗

arXiv2024

Novel Approaches for ML-Assisted Particle Track Reconstruction and Hit Clustering

Track reconstruction is a vital aspect of High-Energy Physics (HEP) and plays a critical role in major experiments. In this study, we delve into unexplored avenues for particle track reconstruction and hit clustering. Firstly, we enhance the algorithmic design effort by utilising a simplified simulator (REDVID) to generate training data that is specifically composed for simplicity. We demonstrate the effectiveness of this data in guiding the development of optimal network architectures. Additionally, we investigate the application of image segmentation networks for this task, exploring their potential for accurate track reconstruction. Moreover, we approach the task from a different perspective by treating it as a hit sequence to track sequence translation problem. Specifically, we explore the utilisation of Transformer architectures for tracking purposes. Our preliminary findings are covered in detail. By considering this novel approach, we aim to uncover new insights and potential advancements in track reconstruction. This research sheds light on previously unexplored methods and provides valuable insights for the field of particle track reconstruction and hit clustering in HEP.

hep-ex↗

arXiv2023

Reduced Simulations for High-Energy Physics, a Middle Ground for Data-Driven Physics Research

Subatomic particle track reconstruction (tracking) is a vital task in High-Energy Physics experiments. Tracking is exceptionally computationally challenging and fielded solutions, relying on traditional algorithms, do not scale linearly. Machine Learning (ML) assisted solutions are a promising answer. We argue that a complexity-reduced problem description and the data representing it, will facilitate the solution exploration workflow. We provide the REDuced VIrtual Detector (REDVID) as a complexity-reduced detector model and particle collision event simulator combo. REDVID is intended as a simulation-in-the-loop, to both generate synthetic data efficiently and to simplify the challenge of ML model design. The fully parametric nature of our tool, with regards to system-level configuration, while in contrast to physics-accurate simulations, allows for the generation of simplified data for research and education, at different levels. Resulting from the reduced complexity, we showcase the computational efficiency of REDVID by providing the computational cost figures for a multitude of simulation benchmarks. As a simulation and a generative tool for ML-assisted solution design, REDVID is highly flexible, reusable and open-source. Reference data sets generated with REDVID are publicly available. Data generated using REDVID has enabled rapid development of multiple novel ML model designs, which is currently ongoing.

hep-ex↗

arXiv2023

New energy spectra in neutrino and photon detectors to reveal hidden dark matter signals

Neutral particles capable of travelling cosmic distances from a source to detectors on Earth are limited to photons and neutrinos. Examination of the Dark Matter annihilation/decay spectra for these particles reveals the presence of continuum spectra (e.g. due to fragmentation and W or Z decay) and peaks (due to direct annihilations/decays). However, when one explores extensions of the Standard Model (BSM), unexplored spectra emerge that differ significantly from those of the Standard Model (SM) for both neutrinos and photons. In this paper, we argue for the inclusion of important spectra that include peaks as well as previously largely unexplored entities such as boxes and combinations of box, peak and continuum decay spectra.

hep-ph↗

arXiv2023

Towards a Benchmark for Scientific Understanding in Humans and Machines

Scientific understanding is a fundamental goal of science, allowing us to explain the world. There is currently no good way to measure the scientific understanding of agents, whether these be humans or Artificial Intelligence systems. Without a clear benchmark, it is challenging to evaluate and compare different levels of and approaches to scientific understanding. In this Roadmap, we propose a framework to create a benchmark for scientific understanding, utilizing tools from philosophy of science. We adopt a behavioral notion according to which genuine understanding should be recognized as an ability to perform certain tasks. We extend this notion by considering a set of questions that can gauge different levels of scientific understanding, covering information retrieval, the capability to arrange information to produce an explanation, and the ability to infer how things would be different under different circumstances. The Scientific Understanding Benchmark (SUB), which is formed by a set of these tests, allows for the evaluation and comparison of different approaches. Benchmarking plays a crucial role in establishing trust, ensuring quality control, and providing a basis for performance evaluation. By aligning machine and human scientific understanding we can improve their utility, ultimately advancing scientific understanding and helping to discover new insights within machines.

cs.AI↗

arXiv2022

Mind the gap: The discrepancy between simulation and reality drives interpretations of the Galactic Center Excess

The Galactic Center Excess (GCE) in GeV gamma rays has been debated for over a decade, with the possibility that it might be due to dark matter annihilation or undetected point sources such as millisecond pulsars (MSPs). This study investigates how the gamma-ray emission model ($\gamma$EM) used in Galactic center analyses affects the interpretation of the GCE's nature. To address this issue, we construct an ultra-fast and powerful inference pipeline based on convolutional Deep Ensemble Networks. We explore the two main competing hypotheses for the GCE using a set of $\gamma$EMs with increasing parametric freedom. We calculate the fractional contribution ($f_{\mathrm{src}}$) of a dim population of MSPs to the total luminosity of the GCE and analyze its dependence on the complexity of the $\gamma$EM. For the simplest $\gamma$EM, we obtain $f_{\mathrm{src}} = 0.10 \pm 0.07$, while the most complex model yields $f_{\mathrm{src}} = 0.79 \pm 0.24.$ In conclusion, we find that the statement about the nature of the GCE (dark matter or not) strongly depends on the assumed $\gamma$EM. The quoted results for $f_{\mathrm{src}}$ do not account for the additional uncertainty arising from the fact that the observed gamma-ray sky is out-of-distribution concerning the investigated $\gamma$EM iterations. We quantify the reality gap between our $\gamma$EMs using deep-learning-based One-Class Deep Support Vector Data Description networks, revealing that all employed $\gamma$EMs have gaps to reality. Our study casts doubt on the validity of previous conclusions regarding the GCE and dark matter, and underscores the urgent need to account for the reality gap and consider previously overlooked ''out of domain'' uncertainties in future interpretations.

astro-ph.HE↗

arXiv2022

Attention to the strengths of physical interactions: Transformer and graph-based event classification for particle physics experiments

A major task in particle physics is the measurement of rare signal processes. Even modest improvements in background rejection, at a fixed signal efficiency, can significantly enhance the measurement sensitivity. Building on prior research by others that incorporated physical symmetries into neural networks, this work extends those ideas to include additional physics-motivated features. Specifically, we introduce energy-dependent particle interaction strengths, derived from leading-order SM predictions, into modern deep learning architectures, including Transformer Architectures (Particle Transformer), and Graph Neural Networks (Particle Net). These interaction strengths, represented as the SM interaction matrix, are incorporated into the attention matrix (transformers) and edges (graphs). Our results in event classification show that the integration of all physics-motivated features improves background rejection by $10\%-40\%$ over baseline models, with an additional gain of up to $9\%$ due to the SM interaction matrix. This study also provides one of the broadest comparisons of event classifiers to date, demonstrating how various architectures perform across this task. A simplified statistical analysis demonstrates that these enhanced architectures yield significant improvements in signal significance compared to a graph network baseline.

hep-ph↗