SearcharxivSearch

arXiv subjects

Matthew Feickert

Publications and source records attributed to Matthew Feickert.

At least 19 recordsLinked to original sources

The Living Guide of Machine Learning for Particle Physics

We started the Living Review of Machine Learning for Particle Physics (HEP-ML Living Review) in 2020 as a community-maintained, near-comprehensive bibliography of machine learning in particle physics. The field was then growing faster than any single researcher could follow, finding the relevant papers was hard, and a structured, continuously updated reference paid off immediately. Since then the literature has grown by more than an order of magnitude, the methods reach far beyond the classification and generation tasks of the early years, and the community has built its own ecosystem of topic-specific reviews, benchmark papers, and software frameworks. The original model no longer serves this field well, and we can no longer sustain it. We therefore change direction. We freeze the Living Review as an archival reference covering the literature up to 1 June 2026, where it remains a stable record of the first phase of HEP-ML. A new resource, the HEP-ML Living Guide, replaces it. It does not list everything. It curates, it annotates, and it points readers to foundational and representative work, so that researchers can find their way into a mature and rapidly diversifying field. In this article we explain why we make this change and how the new resource works.

hep-ph

pylhe: A Lightweight Python interface to Les Houches Event files

Les Houches Event files are a standard format for Monte Carlo event generators in high-energy physics. pylhe is a lightweight pure-Python library for reading and writing LHE event data. It supports .lhe and compressed .lhe.gz files, implements the widely used LHE 3.0 features including multiple event weights and generator metadata, and also supports the recent HDF5-based LHEH5 format. By exposing events through a pythonic iterator and shared data structures, pylhe enables memory-efficient processing of large event samples without loading entire files into memory. The library also supports format conversion and integration with modern analysis workflows, including columnar analysis with Awkward Array and downstream machine-learning applications.

hep-ph

Reinterpretation and preservation of data and analyses in HEP

Data from particle physics experiments are unique and are often the result of a very large investment of resources. Given the potential scientific impact of these data, which goes far beyond the immediate priorities of the experimental collaborations that obtain them, it is imperative that the collaborations and the wider particle physics community publish and preserve sufficient information to ensure that this impact can be realised, now and into the future. The information to be published and preserved includes the algorithms, statistical information, simulations and the recorded data. This publication and preservation requires significant resources, and should be a strategic priority with commensurate planning and resource allocation from the earliest stages of future facilities and experiments.

hep-ph

The 200 Gbps Challenge: Imagining HL-LHC analysis facilities

The IRIS-HEP software institute, as a contributor to the broader HEP Python ecosystem, is developing scalable analysis infrastructure and software tools to address the upcoming HL-LHC computing challenges with new approaches and paradigms, driven by our vision of what HL-LHC analysis will require. The institute uses a "Grand Challenge" format, constructing a series of increasingly large, complex, and realistic exercises to show the vision of HL-LHC analysis. Recently, the focus has been demonstrating the IRIS-HEP analysis infrastructure at scale and evaluating technology readiness for production. As a part of the Analysis Grand Challenge activities, the institute executed a "200 Gbps Challenge", aiming to show sustained data rates into the event processing of multiple analysis pipelines. The challenge integrated teams internal and external to the institute, including operations and facilities, analysis software tools, innovative data delivery and management services, and scalable analysis infrastructure. The challenge showcases the prototypes - including software, services, and facilities - built to process around 200 TB of data in both the CMS NanoAOD and ATLAS PHYSLITE data formats with test pipelines. The teams were able to sustain the 200 Gbps target across multiple pipelines. The pipelines focusing on event rate were able to process at over 30 MHz. These target rates are demanding; the activity revealed considerations for future testing at this scale and changes necessary for physicists to work at this scale in the future. The 200 Gbps Challenge has established a baseline on today's facilities, setting the stage for the next exercise at twice the scale.

hep-ex

PyHEP.dev 2024 Workshop Summary Report, August 26-30 2024, Aachen, Germany

The second PyHEP.dev workshop, part of the "Python in HEP Developers" series organized by the HEP Software Foundation (HSF), took place in Aachen, Germany, from August 26 to 30, 2024. This gathering brought together nearly 30 Python package developers, maintainers, and power users to engage in informal discussions about current trends in Python, with a primary focus on analysis tools and techniques in High Energy Physics (HEP). The workshop agenda encompassed a range of topics, such as defining the scope of HEP data analysis, exploring the Analysis Grand Challenge project, evaluating statistical models and serialization methods, assessing workflow management systems, examining histogramming practices, and investigating distributed processing tools like RDataFrame, Coffea, and Dask. Additionally, the workshop dedicated time to brainstorming the organization of future PyHEP.dev events, upholding the tradition of alternating between Europe and the United States as host locations. This document, prepared by the session conveners in the weeks following the workshop, serves as a summary of the key discussions, salient points, and conclusions that emerged.

hep-ex

Scalable ATLAS pMSSM computational workflows using containerised REANA reusable analysis platform

In this paper we describe the development of a streamlined framework for large-scale ATLAS pMSSM reinterpretations of LHC Run-2 analyses using containerised computational workflows. The project is looking to assess the global coverage of BSM physics and requires running O(5k) computational workflows representing pMSSM model points. Following ATLAS Analysis Preservation policies, many analyses have been preserved as containerised Yadage workflows, and after validation were added to a curated selection for the pMSSM study. To run the workflows at scale, we utilised the REANA reusable analysis platform. We describe how the REANA platform was enhanced to ensure the best concurrent throughput by internal service scheduling changes. We discuss the scalability of the approach on Kubernetes clusters from 500 to 5000 cores. Finally, we demonstrate a possibility of using additional ad-hoc public cloud infrastructure resources by running the same workflows on the Google Cloud Platform.

cs.DC

Bayesian Methodologies with pyhf

bayesian_pyhf is a Python package that allows for the parallel Bayesian and frequentist evaluation of multi-channel binned statistical models. The Python library pyhf is used to build such models according to the HistFactory framework and already includes many frequentist inference methodologies. The pyhf-built models are then used as data-generating model for Bayesian inference and evaluated with the Python library PyMC. Based on Monte Carlo Chain Methods, PyMC allows for Bayesian modelling and together with the arviz library offers a wide range of Bayesian analysis tools.

stat.CO

Software Citation in HEP: Current State and Recommendations for the Future

In November 2022, the HEP Software Foundation and the Institute for Research and Innovation for Software in High-Energy Physics organized a workshop on the topic of Software Citation and Recognition in HEP. The goal of the workshop was to bring together different types of stakeholders whose roles relate to software citation, and the associated credit it provides, in order to engage the community in a discussion on: the ways HEP experiments handle citation of software, recognition for software efforts that enable physics results disseminated to the public, and how the scholarly publishing ecosystem supports these activities. Reports were given from the publication board leadership of the ATLAS, CMS, and LHCb experiments and HEP open source software community organizations (ROOT, Scikit-HEP, MCnet), and perspectives were given from publishers (Elsevier, JOSS) and related tool providers (INSPIRE, Zenodo). This paper summarizes key findings and recommendations from the workshop as presented at the 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023).

hep-ex

pyhf: pure-Python implementation of HistFactory with tensors and automatic differentiation

The HistFactory p.d.f. template is per-se independent of its implementation in ROOT and it is useful to be able to run statistical analysis outside of the ROOT, RooFit, RooStats framework. pyhf is a pure-Python implementation of that statistical model for multi-bin histogram-based analysis and its interval estimation is based on the asymptotic formulas of "Asymptotic formulae for likelihood-based tests of new physics". pyhf supports modern computational graph libraries such as TensorFlow, PyTorch, and JAX in order to make use of features such as auto-differentiation and GPU acceleration. In addition, pyhf's JSON serialization specification for HistFactory models has been used to publish 23 full probability models from published ATLAS collaboration analyses to HEPData.

hep-ex

Deep Learning for the Matrix Element Method

Extracting scientific results from high-energy collider data involves the comparison of data collected from the experiments with synthetic data produced from computationally-intensive simulations. Comparisons of experimental data and predictions from simulations increasingly utilize machine learning (ML) methods to try to overcome these computational challenges and enhance the data analysis. There is increasing awareness about challenges surrounding interpretability of ML models applied to data to explain these models and validate scientific conclusions based upon them. The matrix element (ME) method is a powerful technique for analysis of particle collider data that utilizes an \textit{ab initio} calculation of the approximate probability density function for a collision event to be due to a physics process of interest. The ME method has several unique and desirable features, including (1) not requiring training data since it is an \textit{ab initio} calculation of event probabilities, (2) incorporating all available kinematic information of a hypothesized process, including correlations, without the need for feature engineering and (3) a clear physical interpretation in terms of transition probabilities within the framework of quantum field theory. These proceedings briefly describe an application of deep learning that dramatically speeds-up ME method calculations and novel cyberinfrastructure developed to execute ME-based analyses on heterogeneous computing platforms.

hep-ex

Reinterpretation and Long-Term Preservation of Data and Code

Careful preservation of experimental data, simulations, analysis products, and theoretical work maximizes their long-term scientific return on investment by enabling new analyses and reinterpretation of the results in the future. Key infrastructure and technical developments needed for some high-value science targets are not in scope for the operations program of the large experiments and are often not effectively funded. Increasingly, the science goals of our projects require contributions that span the boundaries between individual experiments and surveys, and between the theoretical and experimental communities. Furthermore, the computational requirements and technical sophistication of this work is increasing. As a result, it is imperative that the funding agencies create programs that can devote significant resources to these efforts outside of the context of the operations of individual major experiments, including smaller experiments and theory/simulation work. In this Snowmass 2021 Computational Frontier topical group report (CompF7: Reinterpretation and long-term preservation of data and code), we summarize the current state of the field and make recommendations for the future.

physics.comp-ph

Data Science and Machine Learning in Education

The growing role of data science (DS) and machine learning (ML) in high-energy physics (HEP) is well established and pertinent given the complex detectors, large data, sets and sophisticated analyses at the heart of HEP research. Moreover, exploiting symmetries inherent in physics data have inspired physics-informed ML as a vibrant sub-field of computer science research. HEP researchers benefit greatly from materials widely available materials for use in education, training and workforce development. They are also contributing to these materials and providing software to DS/ML-related fields. Increasingly, physics departments are offering courses at the intersection of DS, ML and physics, often using curricula developed by HEP researchers and involving open software and data used in HEP. In this white paper, we explore synergies between HEP research and DS/ML education, discuss opportunities and challenges at this intersection, and propose community activities that will be mutually beneficial.

physics.ed-ph

Data and Analysis Preservation, Recasting, and Reinterpretation

We make the case for the systematic, reliable preservation of event-wise data, derived data products, and executable analysis code. This preservation enables the analyses' long-term future reuse, in order to maximise the scientific impact of publicly funded particle-physics experiments. We cover the needs of both the experimental and theoretical particle physics communities, and outline the goals and benefits that are uniquely enabled by analysis recasting and reinterpretation. We also discuss technical challenges and infrastructure needs, as well as sociological challenges and changes, and give summary recommendations to the particle-physics community.

hep-ph

Publishing statistical models: Getting the most out of particle physics experiments

The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In this paper, we make the scientific case for systematically publishing the full statistical models and discuss the technical developments that make this practical. By means of a variety of physics cases -- including parton distribution functions, Higgs boson measurements, effective field theory interpretations, direct searches for new physics, heavy flavor physics, direct dark matter detection, world averages, and beyond the Standard Model global fits -- we illustrate how detailed information on the statistical modelling can enhance the short- and long-term impact of experimental results.

hep-ph

Learning from the Pandemic: the Future of Meetings in HEP and Beyond

The COVID-19 pandemic has by-and-large prevented in-person meetings since March 2020. While the increasing deployment of effective vaccines around the world is a very positive development, the timeline and pathway to "normality" is uncertain and the "new normal" we will settle into is anyone's guess. Particle physics, like many other scientific fields, has more than a year of experience in holding virtual meetings, workshops, and conferences. A great deal of experimentation and innovation to explore how to execute these meetings effectively has occurred. Therefore, it is an appropriate time to take stock of what we as a community learned from running virtual meetings and discuss possible strategies for the future. Continuing to develop effective strategies for meetings with a virtual component is likely to be important for reducing the carbon footprint of our research activities, while also enabling greater diversity and inclusion for participation. This report summarizes a virtual two-day workshop on Virtual Meetings held May 5-6, 2021 which brought together experts from both inside and outside of high-energy physics to share their experiences and practices with organizing and executing virtual workshops, and to develop possible strategies for future meetings as we begin to emerge from the COVID-19 pandemic. This report outlines some of the practices and tools that have worked well which we hope will serve as a valuable resource for future virtual meeting organizers in all scientific fields.

physics.soc-ph

Distributed statistical inference with pyhf enabled through funcX

In High Energy Physics facilities that provide High Performance Computing environments provide an opportunity to efficiently perform the statistical inference required for analysis of data from the Large Hadron Collider, but can pose problems with orchestration and efficient scheduling. The compute architectures at these facilities do not easily support the Python compute model, and the configuration scheduling of batch jobs for physics often requires expertise in multiple job scheduling services. The combination of the pure-Python libraries pyhf and funcX reduces the common problem in HEP analyses of performing statistical inference with binned models, that would traditionally take multiple hours and bespoke scheduling, to an on-demand (fitting) "function as a service" that can scalably execute across workers in just a few minutes, offering reduced time to insight and inference. We demonstrate execution of a scalable workflow using funcX to simultaneously fit 125 signal hypotheses from a published ATLAS search for new physics using pyhf with a wall time of under 3 minutes. We additionally show performance comparisons for other physics analyses with openly published probability models and argue for a blueprint of fitting as a service systems at HPC centers.

cs.DC

Software Training in HEP

Long term sustainability of the high energy physics (HEP) research software ecosystem is essential for the field. With upgrades and new facilities coming online throughout the 2020s this will only become increasingly relevant throughout this decade. Meeting this sustainability challenge requires a workforce with a combination of HEP domain knowledge and advanced software skills. The required software skills fall into three broad groups. The first is fundamental and generic software engineering (e.g. Unix, version control,C++, continuous integration). The second is knowledge of domain specific HEP packages and practices (e.g., the ROOT data format and analysis framework). The third is more advanced knowledge involving more specialized techniques. These include parallel programming, machine learning and data science tools, and techniques to preserve software projects at all scales. This paper dis-cusses the collective software training program in HEP and its activities led by the HEP Software Foundation (HSF) and the Institute for Research and Innovation in Software in HEP (IRIS-HEP). The program equips participants with an array of software skills that serve as ingredients from which solutions to the computing challenges of HEP can be formed. Beyond serving the community by ensuring that members are able to pursue research goals, this program serves individuals by providing intellectual capital and transferable skills that are becoming increasingly important to careers in the realm of software and computing, whether inside or outside HEP

hep-ex

A Living Review of Machine Learning for Particle Physics

Modern machine learning techniques, including deep learning, are rapidly being applied, adapted, and developed for high energy physics. Given the fast pace of this research, we have created a living review with the goal of providing a nearly comprehensive list of citations for those developing and applying these approaches to experimental, phenomenological, or theoretical analyses. As a living document, it will be updated as often as possible to incorporate the latest developments. A list of proper (unchanging) reviews can be found within. Papers are grouped into a small set of topics to be as useful as possible. Suggestions and contributions are most welcome, and we provide instructions for participating.

hep-ph