Searcharxiv⌕ Search

arXiv subjects

Sascha Caron

Publications and source records attributed to Sascha Caron.

At least 19 recordsLinked to original sources

AlphaHits: Full-Event Tracking and Reconstruction Towards a Hit-Level Foundation Model

We introduce AlphaHits, a Transformer-based approach towards hit-level foundation models for LHC physics, building on the TrackFormer and HiggsFormer architectures. AlphaHits operates directly on detector hits and demonstrates scalable full-event charged-particle reconstruction and multi-task event reconstruction and classification from low-level detector information. AlphaHits-TrackFormer reconstructs the full charged-particle content of complete TrackML events without imposing a transverse-momentum selection on the target tracks. It processes both pixel and strip hits in events containing up to $1.5\times10^5$ hits under HL-LHC-like pileup conditions and assigns hits to individual reconstructed trajectories across its kinematic range. This includes the challenging low-$p_T$ population down to approximately $150\,\mathrm{MeV}$ in TrackML samples. It achieves a TrackML score of 0.959 over all reconstructable tracks, representing, to our knowledge, the highest reported TrackML score for full-detector, inclusive charged-particle reconstruction. Building on the TrackFormer architecture, AlphaHits is pretrained on charged-particle tracking and fine-tuned for multi-task reconstruction and classification. From a shared hit-level representation, it predicts tracks, jets, jet flavour, and event-level observables. AlphaHits achieves an AUC of 0.944 for Higgs event classification, with performance comparable to a Particle Transformer (ParT) baseline using fully reconstructed inputs. It also achieves performance comparable to Delphes-based baselines for jet reconstruction, jet flavour classification, and missing transverse momentum regression. These results highlight the potential for scalable full-event tracking and multi-task learning directly from detector hits, representing a promising step towards transferable hit-level representations and general-purpose foundation models for LHC physics.

hep-ex↗

The First Deep Learning all-sky List of Fermi-LAT Gamma-ray Sources

Aims: We develop a deep-learning-based pipeline to construct a list of γ-ray sources and to quantitatively assess the impact of background uncertainties on source detection and characterization. This work extends and enhances the existing AutoSource-ID algorithm, previously adopted on optical observations, for applications on γ-ray data. Methods: Source detection was performed through image segmentation using a U-Net architecture, while dedicated deep neural networks (DNNs) were employed for source localization, flux estimation, and the classification of detected sources as either genuine or spurious. The training data were generated from Monte Carlo simulations based on 10 years of Fermi-LAT observations. Results: We present the first Fermi-LAT γ-ray all-sky source list derived using a deep-learning-based method (1FDL), containing more than 3800 sources with estimated positions, fluxes, and probability scores that quantify the likelihood of each detection being genuine or spurious. We further evaluated the robustness of the pipeline against variations in the interstellar emission model (IEM) and compared the resulting source list with the 4FGL-DR2 catalog. Conclusion: We find that the our deep-learning-based approach achieves performance comparable to, or better than, the likelihood-based methods employed in the standard Fermi-LAT catalog pipeline when tested against changes in the IEM. The 1FDL source list is broadly consistent with the Fermi-LAT 4FGL-DR2 catalog obtained over the same 10-year observational period, including a few new sources confirmed in the recent Fermi-LAT 16-year source list (FL16Y). These results demonstrate the potential of DNN methods for the detection and characterization of γ-ray sources and provide a foundation for their application in future high-energy astrophysical analyses.

astro-ph.HE↗

Masked-Token Prediction for Anomaly Detection at the Large Hadron Collider

Anomaly detection in High Energy Physics requires identifying rare signals against overwhelming backgrounds, without prior knowledge of the signal. We present the first application of masked-token prediction, a technique from Large Language Models, to this problem. A lightweight encoder architecture trained solely on background events captures the structure of Standard Model (SM) physics; at inference, sequences deviating from this learned structure are flagged as anomalous. We evaluate the approach on searches for four-top-quark production and supersymmetric gluino pair production, both featuring top-rich final states with substantial missing transverse energy, covering SM and beyond the Standard Model (BSM) scenarios. Strong performance on the four-top signature, which closely resembles background, demonstrates the method's sensitivity to subtle deviations. We further show that the tokenization strategy significantly impacts performance: deep-learned tokenization via vector-quantized variational autoencoders (VQ-VAE) outperforms look-up table tokenization. Comparison with established anomaly detection baselines confirms robustness. These results highlight the potential of token-based collider data representations combined with transformer architectures for new-physics discovery. Once trained on SM background, the model transfers across different BSM searches, enabling scalable, model-independent anomaly detection at reduced computational cost.

hep-ph↗

Hits to Higgs: Hit-Level Higgs Classification from Raw LHC Detector Data Using Higgsformer

We present Higgsformer, a transformer-based architecture that classifies Higgs events at the Large Hadron Collider directly from raw inner tracker hits, bypassing the traditional reconstruction chain of intermediate physics objects. As a benchmark, we focus on distinguishing $t\bar{t}H$ from $t\bar{t}$ events with $H \to b\bar{b}$, a particularly challenging task due to their similar final state topologies. Our pipeline begins with event generation in Pythia8, fast simulation with ACTS/Fatras, and classification directly from raw detector hits. We show for the first time that a transformer model originally developed for inner tracker hit-to-track assignment can be retrained to classify Higgs signal events directly from raw hits. For comparison, we reconstruct the same events with Delphes and train a Particle Transformer as an object-based classifier. We evaluate both approaches under varying dataset sizes and pileup levels. Despite relying exclusively on inner tracker hits, our large Higgsformer achieves an AUC of $0.855$, matching the performance of the traditional reconstruction pipeline at a $b$-tagging efficiency of $\approx 40\%$ under the same detector constraints.

hep-ph↗

BGRem: A background noise remover for astronomical images based on a diffusion model

Context: Astronomical imaging aims to maximize signal capture while minimizing noise. Enhancing the signal-to-noise ratio directly on detectors is difficult and expensive, leading to extensive research in advanced post-processing techniques. Aims: Removing background noise from images is a valuable pre-processing step catalog-building tasks. We introduce BGRem, a machine learning (ML) based tool to remove background noise from astronomical images. Methods: BGRem uses a diffusion-based model with an attention U-Net as backbone, trained on simulated images for optical and gamma (γ)-ray data from the MeerLICHT and Fermi-LAT telescopes. In a supervised manner, BGRem learns to denoise astronomical images over several diffusion steps. Results: BGRem performance was compared with a widely used tool for cataloging astronomical sources, SourceExtractor (SExtractor). It was shown that the amount of true positive sources using SExtractor increased by about 7% for MeerLICHT data when BGRem was used as a pre-processing step. We also show the generalizability of BGRem by testing it with optical images from different telescopes and also on simulated γ-ray data representative of the Fermi-LAT telescope. We show that in both cases, BGRem improves the source detection efficiency. Conclusions: BGRem can improve the accuracy in source detection of traditional pixel-based methods by removing complex background noise. Using zero-shot approach, BGRem can generalize well to a wide range of optical images. The successful application of BGRem to simulated γ-ray images, alongside optical data, demonstrates its adaptability to distinct noise characteristics and observational domains. This cross-wavelength performance highlights its potential as a general-purpose background removal framework for multi-wavelength astronomical surveys.

astro-ph.IM↗

On the Codesign of Scientific Experiments and Industrial Systems

The optimization of large experiments in fundamental science, such as detectors for subnuclear physics at particle colliders, shares with the optimization of complex systems for industrial or societal applications the common issue of addressing the inter-relation between parameters describing the hardware used in data production and parameters used to analyse those data. While in many cases this coupling can be ignored -- when the problem can be successfully factored into simpler sub-tasks and the latter addressed serially -- there are situations in which that approach fails to converge to the absolute maximum of expected performance, as it results in a mis-alignment of the optimized hardware and software solutions. In this work we consider a few use cases of interest in fundamental science collected primarily from particle physics and related areas, and a pot-pourri of industrial and societal applications where the matter is similarly of relevance. We discuss the emergence of strong hardware-software coupling in some of those systems, as well as co-design procedures that may be deployed to identify the global maximum of their relevant utility functions. We observe how numerous opportunities exist to advance methods and tools for hardware-software co-design optimization, bridging fundamental science and industry through application- and challenge-driven projects, and shaping the future of scientific experiments and industrial systems.

physics.ins-det↗

TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction

High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this study, we extend our earlier efforts by conducting detailed investigations into more custom Transformer attention mechanisms, a new design combining geometric projection and lightweight clustering, and a joint model conditioning classification on a regressor's predictions. Furthermore, we discuss new datasets that allow the training on hit level for a range of physics processes. These developments collectively aim to boost both the accuracy and potentially the efficiency of our tracking models, offering a robust solution to meet the demands of next-generation high-energy physics experiments.

hep-ex↗

Towards a foundation model for astrophysical source detection: An End-to-End Gamma-Ray Data Analysis Pipeline Using Deep Learning

The increasing volume of gamma-ray data demands new analysis approaches that can handle large-scale datasets while providing robustness for source detection. We present a Deep Learning (DL) based pipeline for detection, localization, and characterization of gamma-ray sources. We extend our AutoSourceID (ASID) method, initially tested with \textit{Fermi}-LAT simulated data and optical data (MeerLICHT), to Cherenkov Telescope Array Observatory (CTAO) simulated data. This end-to-end pipeline demonstrates a versatile framework for future application to other surveys and potentially serves as a building block for a foundational model for astrophysical source detection.

astro-ph.IM↗

Event Tokenization and Masked-Token Prediction for Anomaly Detection at the Large Hadron Collider

We propose a novel use of Large Language Models (LLMs) as unsupervised anomaly detectors in particle physics. Using lightweight LLM-like networks with encoder-based architectures trained to reconstruct background events via masked-token prediction, our method identifies anomalies through deviations in reconstruction performance, without prior knowledge of signal characteristics. Applied to searches for simultaneous four-top-quark production, this token-based approach shows competitive performance against established unsupervised methods and effectively captures subtle discrepancies in collider data, suggesting a promising direction for model-independent searches for new physics.

hep-ex↗

Towards a Large Physics Benchmark

We introduce a benchmark framework developed by and for the scientific community to evaluate, monitor and steer large language model development in fundamental physics. Building on philosophical concepts of scientific understanding and creativity, we develop a scoring system in which each question is scored by an expert for its correctness, difficulty, and surprise. The questions are of three forms: (i) multiple-choice questions for conceptual understanding, (ii) analytical problems requiring mathematical derivation, and (iii) openended tasks requiring complex problem solving. Our current dataset contains diverse set of examples, including a machine learning challenge to classify high-energy physics events, such as the four top quark signal. To ensure continued relevance, we propose a living benchmark, where physicists contribute questions, for instance alongside new publications. We invite contributions via: http://www.physicsbenchmarks.org/. We hope that this benchmark will enable a targeted AI development that can make a meaningful contribution to fundamental physics research.

physics.data-an↗

Attention to the strengths of physical interactions: Transformer and graph-based event classification for particle physics experiments

A major task in particle physics is the measurement of rare signal processes. Even modest improvements in background rejection, at a fixed signal efficiency, can significantly enhance the measurement sensitivity. Building on prior research by others that incorporated physical symmetries into neural networks, this work extends those ideas to include additional physics-motivated features. Specifically, we introduce energy-dependent particle interaction strengths, derived from leading-order SM predictions, into modern deep learning architectures, including Transformer Architectures (Particle Transformer), and Graph Neural Networks (Particle Net). These interaction strengths, represented as the SM interaction matrix, are incorporated into the attention matrix (transformers) and edges (graphs). Our results in event classification show that the integration of all physics-motivated features improves background rejection by $10\%-40\%$ over baseline models, with an additional gain of up to $9\%$ due to the SM interaction matrix. This study also provides one of the broadest comparisons of event classifiers to date, demonstrating how various architectures perform across this task. A simplified statistical analysis demonstrates that these enhanced architectures yield significant improvements in signal significance compared to a graph network baseline.

hep-ph↗

TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era

High-Energy Physics experiments are facing a multi-fold data increase with every new iteration. This is certainly the case for the upcoming High-Luminosity LHC upgrade. Such increased data processing requirements forces revisions to almost every step of the data processing pipeline. One such step in need of an overhaul is the task of particle track reconstruction, a.k.a., tracking. A Machine Learning-assisted solution is expected to provide significant improvements, since the most time-consuming step in tracking is the assignment of hits to particles or track candidates. This is the topic of this paper. We take inspiration from large language models. As such, we consider two approaches: the prediction of the next word in a sentence (next hit point in a track), as well as the one-shot prediction of all hits within an event. In an extensive design effort, we have experimented with three models based on the Transformer architecture and one model based on the U-Net architecture, performing track association predictions for collision event hit points. In our evaluation, we consider a spectrum of simple to complex representations of the problem, eliminating designs with lower metrics early on. We report extensive results, covering both prediction accuracy (score) and computational performance. We have made use of the REDVID simulation framework, as well as reductions applied to the TrackML data set, to compose five data sets from simple to complex, for our experiments. The results highlight distinct advantages among different designs in terms of prediction accuracy and computational performance, demonstrating the efficiency of our methodology. Most importantly, the results show the viability of a one-shot encoder-classifier based Transformer solution as a practical approach for the task of tracking.

hep-ex↗

Strategic White Paper on AI Infrastructure for Particle, Nuclear, and Astroparticle Physics: Insights from JENA and EuCAIF

Artificial intelligence (AI) is transforming scientific research, with deep learning methods playing a central role in data analysis, simulations, and signal detection across particle, nuclear, and astroparticle physics. Within the JENA communities-ECFA, NuPECC, and APPEC-and as part of the EuCAIF initiative, AI integration is advancing steadily. However, broader adoption remains constrained by challenges such as limited computational resources, a lack of expertise, and difficulties in transitioning from research and development (R&D) to production. This white paper provides a strategic roadmap, informed by a community survey, to address these barriers. It outlines critical infrastructure requirements, prioritizes training initiatives, and proposes funding strategies to scale AI capabilities across fundamental physics over the next five years.

astro-ph.IM↗

Universal Anomaly Detection at the LHC: Transforming Optimal Classifiers and the DDD Method

In this work, we present a novel approach to transform supervised classifiers into effective unsupervised anomaly detectors. The method we have developed, termed Discriminatory Detection of Distortions (DDD), enhances anomaly detection by training a discriminator model on both original and artificially modified datasets. We conducted a comprehensive evaluation of our models on the Dark Machines Anomaly Score Challenge channels and a search for 4-top quark events, demonstrating the effectiveness of our approach across various final states and beyond the Standard Model scenarios. We compare the performance of the DDD method with the Deep Robust One-Class Classification method (DROCC), which incorporates signals in the training process, and the Deep Support Vector Data Description (DeepSVDD) method, a well-established and well-performing method for anomaly detection. Results show that the effectiveness of each model varies by signal and channel, with DDD proving to be a very effective anomaly detector. We recommend the combined use of DeepSVDD and DDD for purely unsupervised applications, with the addition of flow models for improved performance when resources allow. Findings suggest that network architectures that excel in supervised contexts, such as the particle transformer with standard model interactions, also perform well as unsupervised anomaly detectors. We also show that with these methods, it is likely possible to recognize 4-top quark production as an anomaly without prior knowledge of the process. We argue that the Large Hadron Collider community can transform supervised classifiers into anomaly detectors to uncover potential new physical phenomena in each search.

hep-ph↗

Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models

This paper explores ideas and provides a potential roadmap for the development and evaluation of physics-specific large-scale AI models, which we call Large Physics Models (LPMs). These models, based on foundation models such as Large Language Models (LLMs) - trained on broad data - are tailored to address the demands of physics research. LPMs can function independently or as part of an integrated framework. This framework can incorporate specialized tools, including symbolic reasoning modules for mathematical manipulations, frameworks to analyse specific experimental and simulated data, and mechanisms for synthesizing theories and scientific literature. We begin by examining whether the physics community should actively develop and refine dedicated models, rather than relying solely on commercial LLMs. We then outline how LPMs can be realized through interdisciplinary collaboration among experts in physics, computer science, and philosophy of science. To integrate these models effectively, we identify three key pillars: Development, Evaluation, and Philosophical Reflection. Development focuses on constructing models capable of processing physics texts, mathematical formulations, and diverse physical data. Evaluation assesses accuracy and reliability by testing and benchmarking. Finally, Philosophical Reflection encompasses the analysis of broader implications of LLMs in physics, including their potential to generate new scientific understanding and what novel collaboration dynamics might arise in research. Inspired by the organizational structure of experimental collaborations in particle physics, we propose a similarly interdisciplinary and collaborative approach to building and refining Large Physics Models. This roadmap provides specific objectives, defines pathways to achieve them, and identifies challenges that must be addressed to realise physics-specific large scale AI models.

physics.data-an↗

Novel Approaches for ML-Assisted Particle Track Reconstruction and Hit Clustering

Track reconstruction is a vital aspect of High-Energy Physics (HEP) and plays a critical role in major experiments. In this study, we delve into unexplored avenues for particle track reconstruction and hit clustering. Firstly, we enhance the algorithmic design effort by utilising a simplified simulator (REDVID) to generate training data that is specifically composed for simplicity. We demonstrate the effectiveness of this data in guiding the development of optimal network architectures. Additionally, we investigate the application of image segmentation networks for this task, exploring their potential for accurate track reconstruction. Moreover, we approach the task from a different perspective by treating it as a hit sequence to track sequence translation problem. Specifically, we explore the utilisation of Transformer architectures for tracking purposes. Our preliminary findings are covered in detail. By considering this novel approach, we aim to uncover new insights and potential advancements in track reconstruction. This research sheds light on previously unexplored methods and provides valuable insights for the field of particle track reconstruction and hit clustering in HEP.

hep-ex↗

Reduced Simulations for High-Energy Physics, a Middle Ground for Data-Driven Physics Research

Subatomic particle track reconstruction (tracking) is a vital task in High-Energy Physics experiments. Tracking is exceptionally computationally challenging and fielded solutions, relying on traditional algorithms, do not scale linearly. Machine Learning (ML) assisted solutions are a promising answer. We argue that a complexity-reduced problem description and the data representing it, will facilitate the solution exploration workflow. We provide the REDuced VIrtual Detector (REDVID) as a complexity-reduced detector model and particle collision event simulator combo. REDVID is intended as a simulation-in-the-loop, to both generate synthetic data efficiently and to simplify the challenge of ML model design. The fully parametric nature of our tool, with regards to system-level configuration, while in contrast to physics-accurate simulations, allows for the generation of simplified data for research and education, at different levels. Resulting from the reduced complexity, we showcase the computational efficiency of REDVID by providing the computational cost figures for a multitude of simulation benchmarks. As a simulation and a generative tool for ML-assisted solution design, REDVID is highly flexible, reusable and open-source. Reference data sets generated with REDVID are publicly available. Data generated using REDVID has enabled rapid development of multiple novel ML model designs, which is currently ongoing.

hep-ex↗

New energy spectra in neutrino and photon detectors to reveal hidden dark matter signals

Neutral particles capable of travelling cosmic distances from a source to detectors on Earth are limited to photons and neutrinos. Examination of the Dark Matter annihilation/decay spectra for these particles reveals the presence of continuum spectra (e.g. due to fragmentation and W or Z decay) and peaks (due to direct annihilations/decays). However, when one explores extensions of the Standard Model (BSM), unexplored spectra emerge that differ significantly from those of the Standard Model (SM) for both neutrinos and photons. In this paper, we argue for the inclusion of important spectra that include peaks as well as previously largely unexplored entities such as boxes and combinations of box, peak and continuum decay spectra.

hep-ph↗