arXiv subjects

Simon Mak

Publications and source records attributed to Simon Mak.

At least 19 recordsLinked to original sources

arXiv2026

Fast multitask Gaussian processes, with application to surrogate modeling of the quark-gluon plasma

Gaussian processes (GPs) are broadly used for the surrogate modeling of computer experiments with reliable uncertainty quantification. Our motivating application comes from the study of the quark-gluon plasma (QGP), an extreme state of nuclear matter that filled the universe shortly after the Big Bang. To reliably infer properties of the QGP, multiple surrogate models need to be constructed for related particle collision simulation systems (i.e., multiple "tasks"). While there is a body of literature on multitask GPs (MTGPs), such models can be computationally expensive with large datasets: they require $\mathcal{O}(N^3)$ work and $\mathcal{O}(N^2)$ storage for model training, where $N$ is the total number of samples over all tasks. To address this, we propose a new fast MTGP approach, which pairs structured low-discrepancy design points, such as Sobol' points, with special kernel forms for efficient and exact model fitting. This pairing of a kernel and potentially different design points of different sizes across tasks provides a structured Gram matrix, e.g., a circulant block matrix, which we exploit via a novel algorithm for efficient Gram matrix inversion and determinant computation. Our algorithm reduces training costs to $\mathcal{O}(N \log N)$ work and $\mathcal{O}(N)$ storage in the case of equal sample sizes for each tasks. In the worst case of severely unbalanced sample sizes across tasks, our algorithm may require up to $\mathcal{O}(N^2)$ work and storage, with continuous interpolation between these best and worst cases depending on the balance of sample sizes across tasks. An open-source Python implementation is made available in the FastGPs package (https://alegresor.github.io/fastgps/). We demonstrate the effectiveness of our fast MTGP approach on a range of simulation experiments and on our motivating QGP application.

stat.CO↗

arXiv2026

Environmental policy in the context of complex systems: Statistical optimization and sensitivity analysis for ABMs

Coupled human-environment systems are increasingly being understood as complex adaptive systems (CAS), in which micro-level interactions between components lead to emergent behavior. Agent-based models (ABMs) hold great promise for environmental policy design by capturing such complex behavior, enabling a sophisticated understanding of potential interventions. One limitation, however, is that ABMs can be computationally costly to simulate, which hinders their use for policy optimization. To address this, we propose a new statistical framework that exploits machine learning techniques to accelerate policy optimization with costly ABMs. We first develop a statistical approach for sensitivity testing of the optimal policy, then leverage a reinforcement learning method for efficient policy optimization. We test this framework on the classic ``Sugarscape'' model, an ABM for resource harvesting. We show that our approach can quickly identify optimal and interpretable policies that improve upon baseline techniques, with insightful sensitivity and dynamic analyses that connect back to economic theory.

stat.AP↗

arXiv2025

AutoSchA: Automatic Hierarchical Music Representations via Multi-Relational Node Isolation

Hierarchical representations provide powerful and principled approaches for analyzing many musical genres. Such representations have been broadly studied in music theory, for instance via Schenkerian analysis (SchA). Hierarchical music analyses, however, are highly cost-intensive; the analysis of a single piece of music requires a great deal of time and effort from trained experts. The representation of hierarchical analyses in a computer-readable format is a further challenge. Given recent developments in hierarchical deep learning and increasing quantities of computer-readable data, there is great promise in extending such work for an automatic hierarchical representation framework. This paper thus introduces a novel approach, AutoSchA, which extends recent developments in graph neural networks (GNNs) for hierarchical music analysis. AutoSchA features three key contributions: 1) a new graph learning framework for hierarchical music representation, 2) a new graph pooling mechanism based on node isolation that directly optimizes learned pooling assignments, and 3) a state-of-the-art architecture that integrates such developments for automatic hierarchical music analysis. We show, in a suite of experiments, that AutoSchA performs comparably to human experts when analyzing Baroque fugue subjects.

cs.SD↗

arXiv2025

ProGress: Structured Music Generation via Graph Diffusion and Hierarchical Music Analysis

Artificial Intelligence (AI) for music generation is undergoing rapid developments, with recent symbolic models leveraging sophisticated deep learning and diffusion model algorithms. One drawback with existing models is that they lack structural cohesion, particularly on harmonic-melodic structure. Furthermore, such existing models are largely "black-box" in nature and are not musically interpretable. This paper addresses these limitations via a novel generative music framework that incorporates concepts of Schenkerian analysis (SchA) in concert with a diffusion modeling framework. This framework, which we call ProGress (Prolongation-enhanced DiGress), adapts state-of-the-art deep models for discrete diffusion (in particular, the DiGress model of Vignac et al., 2023) for interpretable and structured music generation. Concretely, our contributions include 1) novel adaptations of the DiGress model for music generation, 2) a novel SchA-inspired phrase fusion methodology, and 3) a framework allowing users to control various aspects of the generation process to create coherent musical compositions. Results from human experiments suggest superior performance to existing state-of-the-art methods.

cs.SD↗

arXiv2025

Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design

With advances in scientific computing, computer experiments are increasingly used for optimizing complex systems. However, for modern applications, e.g., the optimization of nuclear physics detectors, each experiment run can require hundreds of CPU hours, making the optimization of its black-box simulator over a high-dimensional space a challenging task. Given limited runs at inputs $\mathbf{x}_1, \cdots, \mathbf{x}_n$, the best solution from these evaluated inputs can be far from optimal, particularly as dimensionality increases. Existing black-box methods, however, largely employ this ''pick-the-winner'' (PW) solution, which leads to mediocre optimization performance. To address this, we propose a new Black-box Optimization via Marginal Means (BOMM) approach. The key idea is a new estimator of a global optimizer $\mathbf{x}^*$ that leverages the so-called marginal mean functions, which can be efficiently inferred with limited runs in high dimensions. Unlike PW, this estimator can select solutions beyond evaluated inputs for improved optimization performance. Assuming the objective function follows a generalized additive model with unknown link function and under mild conditions, we prove that the BOMM estimator not only is consistent for optimization, but also has an optimization rate that tempers the ''curse-of-dimensionality'' faced by existing methods, thus enabling better performance as dimensionality increases. We present a practical framework for implementing BOMM using the transformed additive Gaussian process surrogate model. Finally, we demonstrate the effectiveness of BOMM in numerical experiments and an application on neutrino detector optimization in nuclear physics.

stat.ML↗

arXiv2025

Robust designs for Gaussian process emulation of computer experiments

We study in this paper two classes of experimental designs, support points and projected support points, which can provide robust and effective emulation of computer experiments with Gaussian processes. These designs have two important properties that are appealing for surrogate modeling of computer experiments. First, the proposed designs are robust: they enjoy good emulation performance over a wide class of smooth and rugged response surfaces. Second, they can be efficiently generated for large designs in high dimensions using difference-of-convex programming. In this work, we present a theoretical framework that investigates the above properties, then demonstrate their effectiveness for Gaussian process emulation in a suite of numerical experiments.

stat.ME↗

arXiv2025

The BdryMat\'ern GP: Reliable incorporation of boundary information on irregular domains for Gaussian process modeling

Gaussian processes (GPs) are broadly used as surrogate models for expensive computer simulators of complex phenomena. However, a key bottleneck is that its training data are generated from this expensive simulator and thus can be highly limited. A promising solution is to supplement the learning model with boundary information from scientific knowledge. However, despite recent work on boundary-integrated GPs, such models largely cannot accommodate boundary information on irregular (i.e., non-hypercube) domains, and do not provide sample path smoothness control or approximation error analysis, both of which are important for reliable surrogate modeling. We thus propose a novel BdryMat\'ern GP modeling framework, which can reliably integrate Dirichlet, Neumann and Robin boundaries on an irregular connected domain with a boundary set that is twice-differentiable almost everywhere. Our model leverages a new BdryMat\'ern covariance kernel derived in path integral form via a stochastic partial differential equation formulation. Similar to the GP with Mat\'ern kernel, we prove that sample paths from the BdryMat\'ern GP satisfy the desired boundaries with smoothness control on its derivatives. We further present an efficient approximation procedure for the BdryMat\'ern kernel using finite element modeling with rigorous error analysis. Finally, we demonstrate the effectiveness of the BdryMat\'ern GP in a suite of numerical experiments on incorporating broad boundaries on irregular domains.

stat.ME↗

arXiv2025

Interplay of prompt and non-prompt photons in photon-triggered jet observables

Prompt photons are important yet challenging to observe in relativistic heavy-ion collisions, as they are produced in the early stages and traverse almost the entire QGP medium without interaction. Experimental analyses typically employ isolation cuts, in the hope to identify prompt photons. Most theoretical studies consider only events with actual prompt photons, assuming no contribution from isolated non-prompt photons to reduce computational cost. For the first time, we present a study that compares simulation results generated using inclusive (bremsstrahlung) and prompt-photon events with multiple experimental observables for both $p-p$ and $Pb-Pb$ collisions at $5.02$ TeV. Simulations are carried out using the multi-stage JETSCAPE framework tuned to describe the quenching of jets and hadrons. Isolated non-prompt photons are generated in hard photon bremsstrahlung, where the photon is radiated at a sufficient angle to the jet. Several photon triggered jet and jet substructure observables show significant contributions from inclusive photons, yielding an improvement in comparison with experimental data. Novel photon triggered jet substructure observables are also expected to show new structures, yet to be detected in experiment. This effort examines the significance of isolated non-prompt photons using parameters tuned for a simultaneous description of the leading hadron and jet spectrum, and thus provides an independent verification of the multistage evolution framework.

hep-ph↗

arXiv2025

Effects of hadronic reinteraction on jet fragmentation from small to large systems

We investigate the impact of the hadronic phase on jet quenching in nuclear collider experiments, an open question in heavy-ion physics. Previous studies in a simplified setup suggest that hadronic interactions could have significant effects, but a systematic analysis is needed. Using the X-SCAPE event generator with the SMASH afterburner, we study the role of hadronic rescattering on jet fragmentation hadrons. Applying this framework to $e^++e^-$ collisions, we demonstrate that even in small systems with limited particle production, hadronic interactions lead to measurable modifications in final-state hadronic and jet observables by comparing scenarios with and without afterburner rescattering.

hep-ph↗

arXiv2025

BLAST: Bayesian online change-point detection with structured image data

The prompt online detection of abrupt changes in image data is essential for timely decision-making in broad applications, from video surveillance to manufacturing quality control. Existing methods, however, face three key challenges. First, the high-dimensional nature of image data introduces computational bottlenecks for efficient real-time monitoring. Second, changes often involve structural image features, e.g., edges, blurs and/or shapes, and ignoring such structure can lead to delayed change detection. Third, existing methods are largely non-Bayesian and thus do not provide a quantification of monitoring uncertainty for confident detection. We address this via a novel Bayesian onLine Structure-Aware change deTection (BLAST) method. BLAST first leverages a deep Gaussian Markov random field prior to elicit desirable image structure from offline reference data. With this prior elicited, BLAST employs a new Bayesian online change-point procedure for image monitoring via its so-called posterior run length distribution. This posterior run length distribution can be computed in an online fashion using $\mathcal{O}(p^2)$ work at each time-step, where $p$ is the number of image pixels; this facilitates scalable Bayesian online monitoring of large images. We demonstrate the effectiveness of BLAST over existing methods in a suite of numerical experiments and in two applications, the first on street scene monitoring and the second on real-time process monitoring for metal additive manufacturing.

stat.ME↗

arXiv2025

QuIP: Experimental design for expensive simulators with many Qualitative factors via Integer Programming

The need to explore and/or optimize expensive simulators with many qualitative factors arises in broad scientific and engineering problems. Our motivating application lies in path planning - the exploration of feasible paths for navigation, which plays an important role in robotics, surgical planning and assembly planning. Here, the feasibility of a path is evaluated via expensive virtual experiments, and its parameter space is typically discrete and high-dimensional. A carefully selected experimental design is thus essential for timely decision-making. We propose here a novel framework, called QuIP, for experimental design of Qualitative factors via Integer Programming under a Gaussian process surrogate model with an exchangeable covariance function. For initial design, we show that its asymptotic D-optimal design can be formulated as a variant of the well-known assignment problem in operations research, which can be efficiently solved to global optimality using state-of-the-art integer programming solvers. For sequential design (specifically, for active learning or black-box optimization), we show that its design criterion can similarly be formulated as an assignment problem, thus enabling efficient and reliable optimization with existing solvers. We then demonstrate the effectiveness of QuIP over existing methods in a suite of path planning experiments and an application to rover trajectory optimization.

stat.AP↗

arXiv2024

Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators

A critical bottleneck for scientific progress is the costly nature of computer simulations for complex systems. Surrogate models provide an appealing solution: such models are trained on simulator evaluations, then used to emulate and quantify uncertainty on the expensive simulator at unexplored inputs. In many applications, one often has available data on related systems. For example, in designing a new jet turbine, there may be existing studies on turbines with similar configurations. A key question is how information from such ``source'' systems can be transferred for effective surrogate training on the ``target'' system of interest. We thus propose a new LOcal transfer Learning Gaussian Process (LOL-GP) model, which leverages a carefully-designed Gaussian process to transfer such information for surrogate modeling. The key novelty of the LOL-GP is a latent regularization model, which identifies regions where transfer should be performed and regions where it should be avoided. Such a ``local transfer'' property is present in many scientific systems: at certain parameters, systems may behave similarly and thus transfer is beneficial; at other parameters, they may behave differently and thus transfer is detrimental. By accounting for local transfer, the LOL-GP can temper the risk of ``negative transfer'', i.e., the risk of worsening predictive performance from information transfer. We derive a Gibbs sampling algorithm for efficient posterior predictive sampling on the LOL-GP, for both the multi-source and multi-fidelity transfer settings. We then show, via a suite of numerical experiments and an application for jet turbine design, the improved surrogate performance of the LOL-GP over existing methods.

stat.ML↗

arXiv2024

Expected Diverse Utility (EDU): Diverse Bayesian Optimization of Expensive Computer Simulators

The optimization of expensive black-box simulators arises in a myriad of modern scientific and engineering applications. Bayesian optimization provides an appealing solution, by leveraging a fitted surrogate model to guide the selection of subsequent simulator evaluations. In practice, however, the objective is often not to obtain a single good solution, but rather a ``basket'' of good solutions from which users can choose for downstream decision-making. This need arises in our motivating application for real-time control of internal combustion engines for flight propulsion, where a diverse set of control strategies is essential for stable flight control. There has been little work on this front for Bayesian optimization. We thus propose a new Expected Diverse Utility (EDU) method that searches for diverse ``$\epsilon$-optimal'' solutions: locally-optimal solutions within a tolerance level $\epsilon > 0$ from a global optimum. We show that EDU yields a closed-form acquisition function under a Gaussian process surrogate model, which facilitates efficient sequential queries via automatic differentiation. This closed form further reveals a novel exploration-exploitation-diversity trade-off, which incorporates the desired diversity property within the well-known exploration-exploitation trade-off. We demonstrate the improvement of EDU over existing methods in a suite of numerical experiments, then explore the EDU in two applications on rover trajectory optimization and engine control for flight propulsion.

stat.AP↗

arXiv2024

A New Dataset, Notation Software, and Representation for Computational Schenkerian Analysis

Schenkerian Analysis (SchA) is a uniquely expressive method of music analysis, combining elements of melody, harmony, counterpoint, and form to describe the hierarchical structure supporting a work of music. However, despite its powerful analytical utility and potential to improve music understanding and generation, SchA has rarely been utilized by the computer music community. This is in large part due to the paucity of available high-quality data in a computer-readable format. With a larger corpus of Schenkerian data, it may be possible to infuse machine learning models with a deeper understanding of musical structure, thus leading to more "human" results. To encourage further research in Schenkerian analysis and its potential benefits for music informatics and generation, this paper presents three main contributions: 1) a new and growing dataset of SchAs, the largest in human- and computer-readable formats to date (>140 excerpts), 2) a novel software for visualization and collection of SchA data, and 3) a novel, flexible representation of SchA as a heterogeneous-edge graph data structure.

cs.SD↗

arXiv2024

BayesFLo: Bayesian fault localization of complex software systems

Software testing is essential for the reliable development of complex software systems. A key step in software testing is fault localization, which uses test data to pinpoint failure-inducing combinations for further diagnosis. Existing fault localization methods have two key limitations: they (i) largely do not incorporate domain and/or structural knowledge from test engineers, and (ii) do not provide a probabilistic assessment of risk for potential root causes. Such methods can thus fail to confidently whittle down the combinatorial number of potential root causes in complex systems, resulting in prohibitively high debugging costs. To address this, we propose a novel Bayesian fault localization framework called BayesFLo, which leverages a flexible Bayesian model for identifying potential root causes with probabilistic uncertainties. Using a carefully-specified prior on root cause probabilities, BayesFLo permits the integration of domain and structural knowledge via the principles of combination hierarchy and heredity, which capture the expected structure of failure-inducing combinations. We then develop new algorithms for efficient computation of posterior root cause probabilities, leveraging recent tools from integer programming and graph representations. Finally, we demonstrate the effectiveness of BayesFLo over existing methods in two fault localization case studies, the first on the Traffic Alert and Collision Avoidance System for aircraft collision avoidance, and the second on the Vulnerable Road User protection tests for safe autonomous driving.

cs.SE↗

arXiv2024

Targeted Variance Reduction: Robust Bayesian Optimization of Black-Box Simulators with Noise Parameters

The optimization of a black-box simulator over control parameters $\mathbf{x}$ arises in a myriad of scientific applications. In such applications, the simulator often takes the form $f(\mathbf{x},\boldsymbol{\theta})$, where $\boldsymbol{\theta}$ are parameters that are uncertain in practice. Robust optimization aims to optimize the objective $\mathbb{E}[f(\mathbf{x},\boldsymbol{\Theta})]$, where $\boldsymbol{\Theta} \sim \mathcal{P}$ is a random variable that models uncertainty on $\boldsymbol{\theta}$. For this, existing black-box methods typically employ a two-stage approach for selecting the next point $(\mathbf{x},\boldsymbol{\theta})$, where $\mathbf{x}$ and $\boldsymbol{\theta}$ are optimized separately via different acquisition functions. As such, these approaches do not employ a joint acquisition over $(\mathbf{x},\boldsymbol{\theta})$, and thus may fail to fully exploit control-to-noise interactions for effective robust optimization. To address this, we propose a new Bayesian optimization method called Targeted Variance Reduction (TVR). The TVR leverages a novel joint acquisition function over $(\mathbf{x},\boldsymbol{\theta})$, which targets variance reduction on the objective within the desired region of improvement. Under a Gaussian process surrogate on $f$, the TVR acquisition can be evaluated in closed form, and reveals an insightful exploration-exploitation-precision trade-off for robust black-box optimization. The TVR can further accommodate a broad class of non-Gaussian distributions on $\mathcal{P}$ via a careful integration of normalizing flows. We demonstrate the improved performance of TVR over the state-of-the-art in a suite of numerical experiments and an application to the robust design of automobile brake discs under operational uncertainty.

stat.ML↗

arXiv2023

3D Multi-system Bayesian Calibration with Energy Conservation to Study Rapidity-dependent Dynamics of Nuclear Collisions

Considerable information about the early-stage dynamics of heavy-ion collisions is encoded in the rapidity dependence of measurements. To leverage the large amount of experimental data, we perform a systematic analysis using three-dimensional hydrodynamic simulations of multiple collision systems -- large and small, symmetric and asymmetric. Specifically, we perform fully 3D multi-stage hydrodynamic simulations initialized by a parameterized model for rapidity-dependent energy deposition, which we calibrate on the hadron multiplicity and anisotropic flow coefficients. We utilize Bayesian inference to constrain properties of the early- and late- time dynamics of the system, and highlight the impact of enforcing global energy conservation in our 3D model.

nucl-th↗

arXiv2023

ProSpar-GP: scalable Gaussian process modeling with massive non-stationary datasets

Gaussian processes (GPs) are a popular class of Bayesian nonparametric models, but its training can be computationally burdensome for massive training datasets. While there has been notable work on scaling up these models for big data, existing methods typically rely on a stationary GP assumption for approximation, and can thus perform poorly when the underlying response surface is non-stationary, i.e., it has some regions of rapid change and other regions with little change. Such non-stationarity is, however, ubiquitous in real-world problems, including our motivating application for surrogate modeling of computer experiments. We thus propose a new Product of Sparse GP (ProSpar-GP) method for scalable GP modeling with massive non-stationary data. The ProSpar-GP makes use of a carefully-constructed product-of-experts formulation of sparse GP experts, where different experts are placed within local regions of non-stationarity. These GP experts are fit via a novel variational inference approach, which capitalizes on mini-batching and GPU acceleration for efficient optimization of inducing points and length-scale parameters for each expert. We further show that the ProSpar-GP is Kolmogorov-consistent, in that its generative distribution defines a valid stochastic process over the prediction space; such a property provides essential stability for variational inference, particularly in the presence of non-stationarity. We then demonstrate the improved performance of the ProSpar-GP over the state-of-the-art, in a suite of numerical experiments and an application for surrogate modeling of a satellite drag simulator.

stat.ME↗