arXiv subjects

Gregor Kasieczka

Publications and source records attributed to Gregor Kasieczka.

At least 19 recordsLinked to original sources

arXiv2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

hep-ph↗

arXiv2026

Point-cloud generative models for fast calorimeter simulation across particles and geometries

Detailed Geant4 simulation of calorimeter showers is the largest single computing cost of collider experiments, and the High-Luminosity LHC will need about ten times more simulated events than are currently produced. We summarise recent progress in generative point cloud fast simulation for highly granular calorimeters. CaloClouds3 generates photon (electromagnetic) showers, is geometry-independent, and runs on average about 120x faster than Geant4 on a single CPU. CaloHadronic uses transformer attention to extend the point cloud diffusion approach to pion (hadronic) showers spanning the electromagnetic and hadronic calorimeters. AllShowers unifies twelve particle types in a single model with far fewer parameters than the specialised baselines while matching or exceeding their fidelity. We close with cross-geometry transfer learning, which needs two to three orders of magnitude fewer training showers while preserving the generative performance.

physics.ins-det↗

arXiv2026

Cross-geometry transfer and model collapse in point cloud calorimeter shower generation

Particle shower simulation is a major computational cost in high-energy physics. Monte Carlo methods such as Geant4 are accurate but expensive, while most machine learning surrogates are tied to specific detector geometries and require retraining for each design change. We study cross-geometry transfer learning with CaloClouds II, a generative model that produces point clouds rather than voxels and can project onto arbitrary detector readouts. We pre-train on photon showers in the International Large Detector (ILD) and adapt to electron showers in the cylindrical CaloChallenge Dataset 3. With only 100 target showers, fine-tuning reduces the geometric mean Wasserstein distance to Geant4 by about 51% over training from scratch. Bias-only fine-tuning (BitFit) stays within 5% of full fine-tuning while updating only 17% of the diffusion network parameters. We also examine model collapse in CaloClouds II by retraining its normalising flow and diffusion model on its own generated showers across successive generations.

physics.ins-det↗

arXiv2026

Jevons' Paradox and Fast Generative Simulation for HEP: Why Realistic Benchmarking is Essential

Simulation is a major computational expense in HEP, and calorimeter simulation in particular drives the overall energy cost of our physics analyses. Future detectors will contain more finely grained calorimeters than ever, and their data analyses will demand unprecedented simulated statistics. Fast generative models redefine what is possible, producing simulations 100 times more efficiently. This article addresses Jevons' paradox in our field and considers the importance of realistic metrics in achieving "true" efficiency.

hep-ex↗

arXiv2026

AllShowers: One model for all calorimeter showers

Accurate and efficient detector simulation is essential for modern collider experiments. To reduce the high computational cost, various fast machine learning surrogate models have been proposed. Traditional surrogate models for calorimeter shower modeling train separate networks for each particle species, limiting scalability and reuse. We introduce AllShowers, a unified generative model that simulates calorimeter showers across multiple particle types using a single generative model. AllShowers is a continuous normalizing flow model with a Transformer architecture, enabling it to generate complex spatial and energy correlations in variable-length point cloud representations of showers. Trained on a diverse dataset of simulated showers in the highly granular ILD detector, the model demonstrates the ability to generate realistic showers for electrons, photons, and charged and neutral hadrons across a wide range of incident energies and angles without retraining. In addition to unifying shower generation for multiple particle types, AllShowers surpasses the fidelity of previous single-particle-type models for hadronic showers. Key innovations include the use of a layer embedding, allowing the model to learn all relevant calorimeter layer properties; a custom attention masking scheme to reduce computational demands and introduce a helpful inductive bias; and a shower- and layer-wise optimal transport mapping to improve training convergence and sample quality. AllShowers marks a significant step towards a universal model for calorimeter shower simulations in collider experiments.

physics.ins-det↗

arXiv2026

Anomaly detection for multijet scenarios

Signals of physics beyond the Standard Model continue to resist discovery at the LHC. Recent years have seen the proliferation of new anomaly detection techniques, promising discovery with significantly fewer model assumptions than traditional approaches. Strategies based on weak supervision have been especially successful, but so far were largely limited in scope by their reliance on a resonance manifesting a decay into a pair of jets. In this work, we demonstrate that a well-established idea from jet substructure physics - recursive soft drop - in combination with the CATHODE technique for anomaly detection can be used to simultaneously perform anomaly detection for signals with an arbitrary number of jets in the final state, greatly increasing the scope of such searches.

hep-ph↗

arXiv2026

Pre-Training for Simulation-Based Science: A Study on Jet Foundation Model Training Objectives

Foundation models (FMs) trained on large datasets and fine-tuned on downstream tasks have emerged as a powerful paradigm in AI for science. Industrial FMs are typically trained using self-supervision with masking due to the lack of labels. In many scientific domains, accurate simulations are plentiful and facilitate large, labeled datasets. This opens up new possibilities for pre-training. We present a systematic comparison of pre-training methods using the OmniLearned High Energy Physics FM framework. We test supervised classification, flow-matching generation, and self-supervised masked particle modeling. All models are pre-trained on the JetClass dataset and fine-tuned on two representative downstream tasks, top jet classification and JetNet conditional generation. Among other observations, for classification tasks, we find that pure classifier pre-training is optimal when downstream labels and model capacity are plentiful, but combining it with self-supervised masked particle modeling (MPM) is uniquely powerful in the low-finetuning label regime. Flow matching-based generative pre-training seems to provide little benefit for downstream classification, and interestingly, for downstream generation, we find that flow matching must be in the pre-training objective to see a significant finetuning advantage, hinting at the orthogonality of classification and generation tasks. That is, for a model to transfer to both generative and classification downstream tasks, it must be pre-trained on both. This study provides a template for controlled scaling analysis of pre-training objectives for foundation models in simulation-based sciences.

hep-ph↗

arXiv2026

Transferable Fast Calorimeter Shower Generation via Multi-Geometry Pre-training

Detailed Geant4 simulation of calorimeter showers dominates the computing budget of high-energy physics experiments. Deep generative surrogates reduce this cost, but they have remained tied to the detector they were trained on, so each new geometry needs a large in-domain dataset. We study whether a single point cloud shower generator can be pre-trained on multiple detectors and transferred to unseen calorimeters. The pre-training geometries come from synthetic geometric variation rather than real-detector data. We introduce SimpleBox, a family of $10^4$ box calorimeters spanning the plane of sampling fraction and longitudinal segmentation, and benchmark it against pre-training on realistic detectors. On a calorimeter unseen in pre-training, with $10^3$ target showers for fine-tuning, the two priors reduce the aggregated sliced Wasserstein distance to Geant4 by factors of 5.2 (synthetic) and 8.0 (realistic) relative to training from scratch. At larger target sizes the synthetic prior performs better than the realistic one. Geometric diversity alone is therefore a practical way to pre-train a transferable shower generator.

physics.ins-det↗

arXiv2026

Cross-Geometry Transfer Learning in Fast Electromagnetic Shower Simulation

Accurate particle shower simulation remains a critical computational bottleneck for high-energy physics. Traditional Monte Carlo methods, such as Geant4, are computationally prohibitive, while existing machine learning surrogates are tied to specific detector geometries and require complete retraining for each design change or alternative detector. We present a transfer learning methodology for generative calorimeter simulation models that enables adaptation across diverse geometries with high data efficiency. Using point cloud representations and pre-training on the International Large Detector, our approach handles new configurations without re-voxelizing showers for each geometry. On the CaloChallenge dataset, transfer learning with only 100 target-domain samples achieves a $51\%$ improvement on the geometric mean of Wasserstein distance over training from scratch. Parameter-efficient fine-tuning with bias-only adaptation achieves competitive performance while updating only $17\%$ of model parameters. Our analysis provides insight into adaptation mechanisms for particle shower development, establishing a baseline for future progress of point cloud approaches in calorimeter simulation.

physics.ins-det↗

arXiv2026

Enhancing next token prediction based pre-training for jet foundation models

Next token prediction is an attractive pre-training task for jet foundation models, in that it is simulation free and enables excellent generative capabilities that can transfer across datasets. Here we study multiple improvements to next token prediction, building on the initial work of OmniJet-$α$. Instead of tokenizing particles and subsequently only using the token-ID as the model input for both the generative and the classification task, we adopt a hybrid setup, which allows us to use continuous feature vectors as model input while only using token-IDs in the next token prediction target. Secondly, we explore a combined pre-training strategy that combines masked particle modeling and generative learning objectives. Taken together, these changes greatly improve the performance in downstream classification tasks without any loss in generative performance.

hep-ph↗

arXiv2026

A Scientific Human-Agent Reproduction Pipeline

Reproducing scientific analyses is essential for preserving knowledge, building extensible codebases, and deepening researcher understanding - yet the effort often outweighs its academic recognition. We argue that the reproduction of scientific data analyses is fundamentally a translation task: converting human-readable knowledge (papers, documentation) into machine-readable analysis code. This makes it uniquely well-suited for AI agents. We present SHARP (Scientific Human-Agent Reproduction Pipeline), a structured framework for reproducing scientific analyses through human-agent collaboration. SHARP decomposes a reproduction task into discrete steps, which an AI agent executes autonomously using specialized subagents for code generation, testing, and quality assurance. At defined checkpoints, the researcher reviews progress, provides feedback, and steers the analysis - keeping the human firmly in control of scientific judgment while the agent handles implementation. We demonstrate SHARP by reproducing a jet classification task in particle physics from a published paper. We evaluate the reproduction along three axes: analysis performance against the original results, code quality and faithfulness, and the nature of the human-agent conversation. The latter is evaluated with a novel framework for characterizing human-agent interactions. Our work highlights a practical model for AI-assisted scientific reproduction where the researcher's role shifts from writing code to understanding, evaluating, and directing - elevating human understanding rather than replacing it.

hep-ph↗

arXiv2026

SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation

We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPADE embeds them independently. Delaying each feature stream relative to the previous one allows intra-token correlations to be learned by the standard self-attention mechanism. Applied to point-cloud calorimeter shower generation in the highly granular ILD detector, SPADE is competitive with the state of the art AllShowers model on photon showers, and substantially outperforms its VQ-VAE-based predecessor OmniJet-$α_C$. The mechanism is applicable to any generative task with multi-feature tokens, enabling LLM-style pretraining workflows for higher-dimensional data.

physics.ins-det↗

arXiv2026

Generative Models and Statistical Validation

Generative machine learning has become an essential tool in theoretical and experimental physics, especially in the context of fast surrogates and density estimators. In this work, we first introduce the underlying framework of modern generative networks and then discuss challenges in quantifying their accuracy, precision, and statistical power.

hep-ph↗

arXiv2026

AI and the Research-Education Environment of Physics

In the current era of AI transforming the research-education environment of physics, variety of issues and concerns arise. The KITP program "Generative AI for High and Low Energy Physics'' offered a discussion session on this, and here presented is a summary of the opinions provided in the discussion. The material is formulated such that it can serve as a starting point for further discussions in readers' research community/institution/group.

physics.ed-ph↗

arXiv2026

Kitchen Sink Anomaly Detection

An enormous amount of R&D effort has resulted in many new resonant anomaly detection methods being proposed in recent years. However, the vast majority of previous R&D studies have suffered from two limitations: they have focused on a very small set of simulated signal benchmark models; and they have either used small sets of carefully crafted high-level jet substructure observables, which can be highly performant but are prone to model dependence, or the full collider event phase space, which is more agnostic but suffers from reduced sensitivity. In this work, we address both limitations: we formulate a number of new simulated signal benchmarks, which we make publicly available in a format fully compatible with the LHCO R&D benchmark; and we explore a high-level, yet highly agnostic, observable set consisting of Energy Flow Polynomials in addition to the usual subjettiness variables. We evaluate this "kitchen sink" observable set for both an idealized anomaly detector and the CWoLa hunting task, along with three baseline observable sets (the Baseline LHC Olympics set, subjettiness observables, and Energy Flow Polynomials). We find that our kitchen sink approach is the most sensitive to a broad range of signal types. Furthermore, we show that an attribute bagging variant, in which each ensemble member is trained on a random subset of substructure observables, yields comparable anomaly detection performance while significantly reducing training cost.

hep-ph↗

arXiv2026

CaloClouds3: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation

We present CaloClouds3, a model for the fast simulation of photon showers in the barrel of a high granularity detector. This iteration demonstrates for the first time how a pointcloud model can employ angular conditioning to replicate photons at all incident angles. Showers produced by this model can be used across the whole detector barrel, due to specially produced position agnostic training data. With this flexibility, the model is usable in a full simulation and reconstruction chain, which offers a further handle for evaluating physics performance of the model. As inference time is a crucial consideration for a generative model, the pre-processing and hyperparameters are aggressively optimised, achieving a speed up factor of two orders of magnitude over Geant4 at inference.

physics.ins-det↗

arXiv2026

Agents of Discovery

The substantial data volumes encountered in modern particle physics and other domains of fundamental physics research allow (and require) the use of increasingly complex data analysis tools and workflows. While the use of machine learning (ML) tools for data analysis has recently proliferated, these tools are typically special-purpose algorithms that rely, for example, on encoded physics knowledge to reach optimal performance. In this work, we investigate a new and orthogonal direction: Using recent progress in large language models (LLMs) to create a team of agents -- instances of LLMs with specific subtasks -- that jointly solve data analysis-based research problems in a way similar to how a human researcher might: by creating code to operate standard tools and libraries (including ML systems) and by building on results of previous iterations. If successful, such agent-based systems could be deployed to automate routine analysis components to counteract the increasing complexity of modern tool chains. To investigate the capabilities of current-generation commercial LLMs, we consider the task of anomaly detection via the publicly available and highly-studied LHC Olympics dataset. Several current models by OpenAI (GPT-4o, o4-mini, GPT-4.1, and GPT-5) are investigated and their stability tested. Overall, we observe the capacity of the agent-based system to solve this data analysis problem. The best agent-created solutions mirror the performance of human state-of-the-art results.

hep-ph↗

arXiv2026

CaloHadronic: a diffusion model for the generation of hadronic showers

Simulating showers of particles in highly-granular calorimeters is a key frontier in the application of machine learning to particle physics. Achieving high accuracy and speed with generative machine learning models can enable them to augment traditional simulations and alleviate a major computing constraint. Recent developments have shown how diffusion based generative shower simulation approaches that do not rely on a fixed structure, but instead generate geometry-independent point clouds, are very efficient. We present a transformer-based extension to previous architectures which were developed for simulating electromagnetic showers in the highly granular electromagnetic calorimeter of the International Large Detector, ILD. The attention mechanism now allows us to generate complex hadronic showers with more pronounced substructure across both the electromagnetic and hadronic calorimeters. This is the first time that machine learning methods are used to holistically generate showers across the electromagnetic and hadronic calorimeter in highly granular imaging calorimeter systems.

physics.ins-det↗