Searcharxiv⌕ Search

arXiv subjects

Lukas Heinrich

Publications and source records attributed to Lukas Heinrich.

At least 19 recordsLinked to original sources

A transformer analysis lowers the energy threshold and improves the sensitivity of a compact imaging atmospheric Cherenkov telescope

Context. A companion study simulated an idealised stand-alone imaging atmospheric Cherenkov telescope with a 5 m diameter mirror. A video vision transformer that processes the raw camera waveforms lowered its energy threshold by a factor of three, compared with the standard analysis based on image parameters. Aims. We use realistic photodetectors, compute the differential flux sensitivity, and study how hardware parameters affect the transformer analysis. Methods. With the simulation tools of the Cherenkov Telescope Array Observatory, we compare three photodetector efficiencies (100 %, 50 % and that of the LST-1 photomultipliers) with the transformer and with a standard analysis optimised for each. We then vary the dish diameter, field of view, pulse width and sampling rate. Results. With the LST-1 photomultipliers, the energy threshold is 0.18 TeV with the transformer and 0.73 TeV with the standard analysis. The standard analysis needs a photodetector efficiency of 100 % to match that threshold. The main reason is that the transformer works with events of about 35 photoelectrons in the camera, whereas the standard analysis needs about 200 before image cleaning. A 4 m dish with the transformer is more sensitive at all energies than a 5 m dish with the standard analysis. Dishes of 4, 5 and 6 m diameter give thresholds of 0.38, 0.18 and 0.13 TeV. Sampling the pixel signals at 2 GS/s instead of 5 GS/s raises the threshold to 0.26 TeV, whereas the single-photoelectron pulse width (1-3 ns) matters little. Conclusions. The analysis matters as much as the mirror area and the photodetector efficiency. With the transformer analysis, the idealised 5 m diameter telescope comes within a factor of about four of the sensitivity of LST-1 and MAGIC above 0.25 TeV, although its mirror area is 21 times smaller than that of the 23 m LST-1. This performance should improve substantially in the coming years.

astro-ph.IM↗

RevolIACT: a longitudinally distributed array of compact AI-driven Cherenkov telescopes for the time-domain sub-TeV sky

Context. The coming decade will be the golden era of time-domain astronomy. In ten years, however, sensitive very-high-energy observations below ~1 TeV may be possible only at the two sites of the Cherenkov Telescope Array Observatory (CTAO). Each can observe a source for only a few hours per night. Aims. We quantify the loss caused by this limited coverage and assess whether RevolIACT, an array of compact stand-alone AI-driven stations, can close the gap. Methods. We estimate the sensitivity that a stand-alone station needs to detect important transients seen by current imaging atmospheric Cherenkov telescopes (IACTs). We also compute the temporal coverage of layouts of ten to fourteen stations distributed in longitude, and how many of the 4401 Fermi-GBM gamma-ray bursts each layout could have observed when they occurred. Results. For each station, a sensitivity of ~15 % of the Crab flux in one hour above 0.2-0.3 TeV, about three times worse than that of current IACT systems, would suffice for an array to detect most of these transients. With the standard analysis, a telescope with a 5 m diameter mirror misses this target by a factor of 17. With a video vision transformer, it comes within a factor of two in an idealised simulation, and a 6 m telescope within 1.5. Ten to fourteen stations would provide three to four times the temporal coverage of CTAO, averaged over the sky and for the GBM bursts. Fourteen stations can observe most sources around the clock for about two months per year. Conclusions. Temporal coverage, not only flux sensitivity, will limit sub-TeV transient astronomy in the next decade. RevolIACT would complement CTAO. The concept relies on the transformer analysis, which is still young and will improve. The decisive next step is to confirm this performance on real data, with a prototype station.

astro-ph.IM↗

It Just Takes Two: Scaling Amortized Inference to Large Sets

Neural posterior estimation has emerged as a powerful tool for amortized inference, with growing adoption across scientific and applied domains. In many of these applications, the conditioning variable is a set of observations whose elements depend not only on the target but also on unknown factors shared across the set. Optimal inference therefore requires treating the set jointly, which in turn requires training the estimator at the deployment set size -- a regime where memory and compute quickly become prohibitive. We introduce a simple, theoretically grounded strategy that decouples representation learning from posterior modeling. Our method trains a mean-pool Deep Set on sets of size at most two, producing an encoder that generalizes to arbitrary set sizes. The inference head is then finetuned on pre-aggregated embeddings, making training cost essentially independent of the deployment set size N. Across scalar, image, multi-view 3D, molecular, and high-dimensional conditional generation benchmarks with N in the thousands, our approach matches or outperforms standard baselines at a fraction of the compute.

cs.LG↗

Gaussian process-based Bayesian imaging of bolometric (sub-)millimetre data

Widefield single-dish telescopes operating at (sub-)mm wavelengths ($λ\sim 4$~mm to 350~$μ$m) tend to feature much faster mapping speeds than radio interferometers at the same wavelengths, and are more suitable for measuring signals on larger spatial scales. However, the data are dominated by atmospheric fluctuations on large scales and by photon and instrumental noise on smaller scales, making it difficult to recover the true astronomical sky. We build on maria-nifty, a Gaussian process-based Bayesian imaging framework for reconstructing sky maps from single-dish telescope data, implemented on top of Numerical Information Field Theory (NIFTy). It uses modular generative models to efficiently separate the astronomical signal from atmospheric emission while providing pixel-wise uncertainty quantification for the reconstructed sky maps. Moreover, our method cleanly separates individual astrophysical signals, even though they are superposed on the sky. Following its successful validation on synthetic time-ordered data generated using the maria software, we apply it to real data observed with the MUSTANG-2 bolometric array on the 100-meter Green Bank Telescope. For the galaxy cluster RXJ1347.5-1145, we separate the extended cluster emission from the compact source at its centre and obtain a smoother map with fewer artefacts compared to a standard maximum-likelihood reconstruction using the same data. The JAX-based implementation and GPU acceleration provide the basis for applying the method to the vastly larger datasets that the next generation of single-dish telescopes will deliver.

astro-ph.IM↗

Precision Tokenization for Multi-Modal Data in High Energy Physics

Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet $p_T$ regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11$\times$ while keeping the jet $p_T$ resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.

hep-ph↗

pylhe: A Lightweight Python interface to Les Houches Event files

Les Houches Event files are a standard format for Monte Carlo event generators in high-energy physics. pylhe is a lightweight pure-Python library for reading and writing LHE event data. It supports .lhe and compressed .lhe.gz files, implements the widely used LHE 3.0 features including multiple event weights and generator metadata, and also supports the recent HDF5-based LHEH5 format. By exposing events through a pythonic iterator and shared data structures, pylhe enables memory-efficient processing of large event samples without loading entire files into memory. The library also supports format conversion and integration with modern analysis workflows, including columnar analysis with Awkward Array and downstream machine-learning applications.

hep-ph↗

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale

Much of the recent progress in machine learning domains such as language models has come from scaling laws that predict performance as a function of training effort. In high-energy physics (HEP) similar behavior has now been observed. To aid further study, we present a systematic procedure to derive robust scaling laws and compare design choices on the relevant budget axes for HEP tasks. We first validate the full scaling trajectory on toy problems and then apply the procedure to multi-task transformers on the ~11 billion-jet ATLAS JetSet2 dataset, in both the compute- and data-constrained regimes. For the latter, we predict, to the best of our knowledge for the first time, the jointly optimal model size, training horizon, learning rate and batch size under early stopping. At compute-optimal scaling, we recover a near-equal $\sqrt{C}$ dependence of model and dataset size, and find that auxiliary objectives lower the primary jet-classification loss at equal compute budget. Expanding the inputs toward lower-level data systematically lowers the loss while leaving the scaling exponent nearly unchanged. The onset of the power-law regime is itself set by scale: below a threshold in dataset size the loss carries little information about high-compute scaling, underscoring the value of large, high-quality full-simulation datasets as a foundation for scaling studies and the development of foundation models in HEP.

hep-ex↗

Fast and Precise Learned Charged-Particle Trajectory Regression at the Large Hadron Collider

We propose a training recipe that treats charged-particle trajectory parameter regression on high-energy physics detector data as a sequence-modeling task. Kalman filters and linearized least-squares fits have been the classical standard approach for this task: they are optimal estimators for sparsely sampled linear-Gaussian data and are commonly used for trajectory parameter regression (fitting). The classical fitting techniques implemented for this domain reach a final precision of one part in $10^5$ through detailed modeling of detector geometry, material, detection effects and precise numerical integration of the equations of motion through the detector's inhomogeneous magnetic field. With this study, we demonstrate that using a bidirectional gated linear recurrent encoder, one is able to reproduce the full precision of classical track fitting techniques. Using a custom kernel, we also achieve significantly higher throughput during GPU inference, compared to classical fitting software running on similarly priced multi-core CPU servers representing typically employed hardware. Such a speedup would lead to considerable cost savings for the pattern recognition at the Large Hadron Collider. To our knowledge, this is the first end-to-end learned track fit to reach the full precision and, at the same time, offer the opportunity to reduce the computing costs.

cs.LG↗

Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks

Reconstruction and simulation at a collider experiment are long chains of specialised algorithms, each tuned to a single step. We explore how one model can serve many of those steps at once, while still producing the intermediate objects (tracks, calorimeter cells, clusters, particles and jets) that make the chain interpretable. To do so, we represent every object in a jet in a single shared token vocabulary and train one model to map any subset of these modalities to any other. A task is then only a choice of which modalities to provide and which to request: particle flow, detector simulation and charged energy subtraction are all directions through the same set of weights. We train over all modality combinations, with a decoder emitting tokens either autoregressively or in parallel. With tokenisation, both architectures train stably with little tuning. In evaluations, both models produce realistic reconstruction and simulation objects, with the autoregressive model particularly faithful to output from Geant4. The autoregressive model also outperforms a state-of-the-art particle flow algorithm on many typical jet reconstruction metrics.

hep-ph↗

The Analysis, not the Aperture: End-to-End Transformer Reconstruction for Imaging Atmospheric Cherenkov Telescopes

Imaging Atmospheric Cherenkov Telescopes (IACTs) detect very-high-energy gamma rays by imaging the nanosecond Cherenkov flash of the air shower they initiate in the Earth's atmosphere. For four decades the first steps of IACT event reconstruction have been essentially unchanged, relying on a heavy parameterisation and dimensionality reduction of the recorded images. This is reasonable when the image is bright, but discards important information when only a few tens of Cherenkov photons are recorded, which is a primary reason why small telescopes perform poorly at sub-TeV energies. We show that this limitation is a property of the analysis rather than of the hardware. We simulate a deliberately simple and idealised compact telescope and treat each event as a short movie that is passed directly to a video vision transformer with a factorised spatio-temporal encoder. A single composite network with a gradient-normalised multi-task loss performs gamma/hadron classification, energy regression and arrival-direction regression at once. This is the first application of a video vision transformer to IACT data. We compare it against an optimised standard analysis on the same dataset. The transformer lowers the energy threshold by a factor of three, from 0.22 to 0.07 TeV, and reconstructs arrival directions down to 0.05 TeV. At 0.2 TeV it increases the effective collection area by a factor of three, and raises the gamma/hadron separation power from an area under the receiver operating characteristic curve of 0.80 to 0.91. At 0.05 TeV, where the standard analysis retains almost nothing, that area grows by nearly two orders of magnitude. These results show promising new opportunities for compact and affordable telescopes operating at sub-TeV energies, paving the way for a broader exploration of time-domain astrophysics.

astro-ph.IM↗

Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough

Machine learning (ML) has become integral to fundamental physics, accelerating statistical workflows from data acquisition through inference and hypothesis testing. As ML systems grow increasingly autonomous, ensuring their reliability for discovery claims becomes critical. This review synthesizes the VERaiPHY (Validation & Evaluation for Robust AI in PHYsics) initiative's frameworks for rigorous ML assessment across particle physics, astrophysics, and cosmology. We establish when verification is essential by contextualizing ML within the statistical discovery workflow. We emphasize fundamental limitations: inductive bias is unavoidable, sample complexity bounds learning, and experimental constraints limit discovery. We reflect on physicists' evolving role as both experimental designers and evaluators whose judgments encode scientific rigor into AI systems. Responsible integration requires understanding ML's transformative potential alongside its intrinsic boundaries.

physics.data-an↗

Gradient estimators for parameter inference in discrete stochastic kinetic models

Stochastic kinetic models are ubiquitous in physics, yet inferring their parameters from experimental data remains challenging. For deterministic models, parameter inference often relies on gradients, which can be obtained efficiently through automatic differentiation (AD). However, AD cannot be applied directly to the Gillespie stochastic simulation algorithm (SSA), since sampling from a discrete set of reactions introduces non-differentiable operations. In this work, we adopt three gradient estimators from machine learning for the Gillespie SSA: the Gumbel-Softmax Straight-Through (GS-ST) estimator, the Score Function estimator, and the Alternative Path estimator. We use the estimators to evaluate gradients of steady-state and time-dependent observables, and compare their performance in representative biophysical systems with relaxation dynamics (bimolecular association) and oscillatory dynamics (repressilator). We find that the GS-ST estimator generally yields well-behaved gradient estimates, but exhibits diverging variance in challenging parameter regimes, which can cause parameter inference to fail. In these cases, other estimators provide more robust, lower variance gradients. Our results demonstrate that gradient-based parameter inference can be effectively combined with the Gillespie SSA, with different estimators offering complementary advantages.

physics.comp-ph↗

Exploring the Boundaries of Differentiable Radiation Transport and Detector Simulation

We present an application of automatic differentiation for particle transport through matter using a Geant4-like radiation transport simulation with a full electromagnetic physics model. When differentiating this step-based transport, we observe exploding gradients driven by rare but extreme sensitivities at material boundaries, which propagate through subsequent transport and shower development. To obtain usable derivatives for optimization, we introduce a targeted mitigation strategy that stops gradient propagation through boundary-crossing operations under identifiable unstable conditions while leaving the forward (primal) simulation unchanged. We demonstrate that this enables stable, optimization-ready gradients in a detector-design problem.

physics.ins-det↗

BRICKS: Compositional Neural Markov Kernels for Zero-Shot Radiation-Matter Simulation

We introduce a new strategy for compositional neural surrogates for radiation-matter interactions, a key task spanning domains from particle physics through nuclear and space engineering to medical physics. Exploiting the locality and the Markov nature of particle interactions, we create a \emph{next-particle prediction} kernel using hybrid discrete-continuous transformer models based on Riemannian Flow Matching on product manifolds. The model generates variable-sized typed sets of particles and radiation side effects that are the result of the interaction of an incident particle with a material volume. The resulting kernel can be composed to simulate unseen large-scale material distributions in a zero-shot manner. Unlike mechanistic simulators, our model is designed to be differentiable, provides tractable likelihoods for future downstream applications. A significant computational speed-up on GPU compared to CPU-bound mechanistic simulation is observed for single-kernel execution. We evaluate the model at the kernel level and demonstrate predictive stability over multi-round autoregressive rollouts. We additionally release a novel 20M-event radiation-matter interaction dataset for further research.

cs.LG↗

maria goes NIFTy: Gaussian Process-Based Reconstruction and Denoising of Simulated (Sub-)Millimetre Single-Dish Telescope Data

(Sub-)millimetre single-dish telescopes feature faster mapping speeds and access larger spatial scales than their interferometric counterparts. However, atmospheric fluctuations tend to dominate their signals and complicate recovery of the astronomical sky. Here we develop a framework for Gaussian process-based sky reconstruction and separation of the atmospheric emission from the astronomical signal based on Numerical Information Field Theory (NIFTy). To validate this novel approach, we use the maria software to generate synthetic time-ordered observational data mimicking the MUSTANG-2 bolometric array. This approach leads to significantly improved sky reconstructions versus traditional methods.

astro-ph.IM↗

On the Codesign of Scientific Experiments and Industrial Systems

The optimization of large experiments in fundamental science, such as detectors for subnuclear physics at particle colliders, shares with the optimization of complex systems for industrial or societal applications the common issue of addressing the inter-relation between parameters describing the hardware used in data production and parameters used to analyse those data. While in many cases this coupling can be ignored -- when the problem can be successfully factored into simpler sub-tasks and the latter addressed serially -- there are situations in which that approach fails to converge to the absolute maximum of expected performance, as it results in a mis-alignment of the optimized hardware and software solutions. In this work we consider a few use cases of interest in fundamental science collected primarily from particle physics and related areas, and a pot-pourri of industrial and societal applications where the matter is similarly of relevance. We discuss the emergence of strong hardware-software coupling in some of those systems, as well as co-design procedures that may be deployed to identify the global maximum of their relevant utility functions. We observe how numerous opportunities exist to advance methods and tools for hardware-software co-design optimization, bridging fundamental science and industry through application- and challenge-driven projects, and shaping the future of scientific experiments and industrial systems.

physics.ins-det↗

Neural Scaling Laws for Boosted Jet Tagging

The success of Large Language Models (LLMs) has established that scaling compute, through joint increases in model capacity and dataset size, is the primary driver of performance in modern machine learning. While machine learning has long been an integral component of High Energy Physics (HEP) data analysis workflows, the compute used to train state-of-the-art HEP models remains orders of magnitude below that of industry foundation models. With scaling laws only beginning to be studied in the field, we investigate neural scaling laws for boosted jet classification using the public JetClass dataset. We derive compute optimal scaling laws and identify an effective performance limit that can be consistently approached through increased compute. We study how data repetition, common in HEP where simulation is expensive, modifies the scaling yielding a quantifiable effective dataset size gain. We then study how the scaling coefficients and asymptotic performance limits vary with the choice of input features and particle multiplicity, demonstrating that increased compute reliably drives performance toward an asymptotic limit, and that more expressive, lower-level features can raise the performance limit and improve results at fixed dataset size.

hep-ex↗

Differentiable quantum-trajectory simulation of Lindblad dynamics for QGP transport-coefficient inference

We study parameter estimation for the transport coefficients of the quark-gluon plasma by differentiating open-quantum-system-based Monte Carlo simulations of quarkonium suppression. The underlying simulator requires solving a Lindblad equation in a large Hilbert space, which makes parameter estimation computationally expensive. We approach the problem using gradient-based optimization. Specifically, we apply the score-function gradient estimator to differentiate through discrete jump sampling in the Monte Carlo wave-function algorithm used to solve the Lindblad equation. The resulting stochastic gradient estimator exhibits sufficiently low variance and can still be estimated in an embarrassingly parallel manner, enabling efficient scaling of the simulations. We implement this gradient estimator in the existing open-source quarkonium suppression code QTraj. To demonstrate its utility for parameter estimation, we infer the two transport coefficients $\hatκ$ and $\hatγ$ using gradient-based optimization on synthetic nuclear modification factor data.

physics.comp-ph↗