Benchmarking Machine Learning Architectures for ttH Multilepton Signal Sensitivity

Abstract

Statistical testing for signal discovery and signal-strength estimation in high-energy physics increasingly relies on machine-learning models trained on simulated data. We present a synthetic dataset for $t\bar t H$ multilepton signal--background classification and perform a systematic evaluation of machine-learning models ranging from the widely used XGBoost for tabular data to LorentzNet, which processes events with built-in Lorentz symmetry. Existing studies often differ in feature definitions, training procedures, and evaluation metrics, making it difficult to isolate the impact of model architecture on performance. To address this, we apply standardized training and hyperparameter-optimization procedures, construct a controlled hierarchy of feature sets, and perform a comprehensive comparison of the models. In addition to the analysis-driven metric of signal-strength uncertainty, we report the commonly used ROC AUC metric and show that under realistically weighted training, it correlates well with the uncertainty-based ranking of models. We further investigate model performance as a function of input feature set, training-set size, and the choice between channel-specific and unified multi-channel training. Our results establish the superiority of Particle Transformer and LorentzNet within the considered setup. We also identify potential avenues for further improvement, including more expressive architectures, joint training across additional analysis channels, and larger simulated datasets. Although this study focuses on a specific Higgs-boson analysis, we expect the main conclusions to generalize to a broader class of searches and measurements at the LHC and HL-LHC.