arXiv subjects

Difan Song

Publications and source records attributed to Difan Song.

4 recordsLinked to original sources

arXiv2026

Efficient Screening Designs for Expensive Black-box Models with Qualitative and Quantitative Factors

Computationally expensive black-box models often involve a large number of input factors with complex interactions and varying importance. Experimental design techniques can be used for quickly identifying the important factors, which can make the optimization of a complex computer model or the training of an expensive machine learning model more efficient. Existing screening designs for black-box models focus mainly on continuous factors, with the maximum one-factor-at-a-time (MOFAT) design being a recent example. In this work, we extend the design to incorporate multiple types of factors, including nominal, ordinal, and discrete-numeric. We first identify the properties leading to optimal screening, where qualitative and quantitative factors should be treated differently. Based on these properties, we propose practical algorithms to efficiently construct MOFAT designs for all types of factors. The usefulness of the design is demonstrated by both numerical experiments and an application to hyperparameter tuning in machine learning models.

stat.ME↗

arXiv2025

Efficient Active Learning Strategies for Computer Experiments

Active learning in computer experiments aims at allocating resources in an intelligent manner based on the already observed data to satisfy certain objectives such as emulating or optimizing a computationally expensive function. There are two main ingredients for active learning: an initial experimental design, which helps to approximately learn the function, and a surrogate modeling technique, which provides a prediction of the output along with its uncertainty estimates. Space-filling designs are commonly used as initial design and Gaussian processes for surrogate modeling. This article aims at improving the active learning procedure by proposing a new type of initial design and a new correlation function for the Gaussian process. The ideas behind them are known in other fields such as in sensitivity analysis or in kernel theory, but they never seem to have been used for active learning in computer experiments. We show that they provide substantial improvement to the state-of-the-art methods for both emulation and optimization. We support our findings through theory and simulations, and a real experiment on the vapor-phase infiltration process.

stat.ME↗

arXiv2023

ACE: Active Learning for Causal Inference with Expensive Experiments

Experiments are the gold standard for causal inference. In many applications, experimental units can often be recruited or chosen sequentially, and the adaptive execution of such experiments may offer greatly improved inference of causal quantities over non-adaptive approaches, particularly when experiments are expensive. We thus propose a novel active learning method called ACE (Active learning for Causal inference with Expensive experiments), which leverages Gaussian process modeling of the conditional mean functions to guide an informed sequential design of costly experiments. In particular, we develop new acquisition functions for sequential design via the minimization of the posterior variance of a desired causal estimand. Our approach facilitates targeted learning of a variety of causal estimands, such as the average treatment effect (ATE), the average treatment effect on the treated (ATTE), and individualized treatment effects (ITE), and can be used for adaptive selection of an experimental unit and/or the applied treatment. We then demonstrate in a suite of numerical experiments the improved performance of ACE over baseline methods for estimating causal estimands given a limited number of experiments.

stat.ME↗

arXiv2021

On the limits of algorithmic prediction across the globe

The impact of predictive algorithms on people's lives and livelihoods has been noted in medicine, criminal justice, finance, hiring and admissions. Most of these algorithms are developed using data and human capital from highly developed nations. We tested how well predictive models of human behavior trained in a developed country generalize to people in less developed countries by modeling global variation in 200 predictors of academic achievement on nationally representative student data for 65 countries. Here we show that state-of-the-art machine learning models trained on data from the United States can predict achievement with high accuracy and generalize to other developed countries with comparable accuracy. However, accuracy drops linearly with national development due to global variation in the importance of different achievement predictors, providing a useful heuristic for policymakers. Training the same model on national data yields high accuracy in every country, which highlights the value of local data collection.

cs.LG↗