Searcharxiv⌕ Search

arXiv subjects

Michael C. Fu

Publications and source records attributed to Michael C. Fu.

At least 19 recordsLinked to original sources
arXiv2026

Consistent Variance Estimation for Q-Function Estimators in Finite-Horizon MDP Tree Search

We study the variance of Q-function estimators in finite-horizon, finite-state Markov decision process (MDP) tree search. We show that the variance decomposes into three components attributed to the immediate reward collected, probabilistic state transitions, and uncertainty in future state value function estimates. Using this decomposition, we show that the sample variance estimator based on the assumption of i.i.d. paths is biased, underestimating the true variance, and the bias does not vanish in the limit. We then propose a recursive variance estimator that is consistent. To enable efficient storage and computation, we derive an equivalent implementation of the recursive estimator using only node-local statistics that can be iteratively updated. This consistent variance estimator is integrated into two Monte Carlo Tree Search (MCTS) sampling procedures for finite-horizon MDPs. In numerical examples from inventory control and kidney paired donation matching, the new estimator improves the performance of the MCTS algorithm relative to a baseline that uses the i.i.d.-based sample variance estimator.

eess.SY↗
arXiv2026

Sensitivity Analysis and Optimization of Stochastic Epidemic Models under Parameter Uncertainty

To address sensitivity analysis and optimization for a discrete-time stochastic epidemic model, we derive unbiased gradient estimators that accommodate uncertainties represented as distributions over the parameters of interest, such as those arising from Bayesian calibration. Specifically, we estimate the sensitivity of total infections over a finite time horizon with respect to the proportion immunized ($v$) and the contact rate ($\beta$). Comparing the proposed estimators with deterministic limit approximations based on large populations reveals differences due to the finite population and time horizon. The estimators exhibit lower variance than finite-difference estimators for the derivative with respect to $\beta$, but higher variance for the derivative with respect to $v$. Simulation experiments indicate parameter uncertainty reduces sensitivity to the parameters of interest. In particular, indirect effects of vaccination, such as herd immunity, are less pronounced compared to when parameters are known. For optimization problems balancing intervention and infection costs, incorporating parametric uncertainty leads to more conservative policies.

stat.AP↗
arXiv2026

Conditional Leibniz Derivative Estimation with an Application to American Call Min-Options

Leibniz derivative estimation is a Monte Carlo technique for estimating derivatives of a discontinuous sample performance in stochastic models with respect to parameters of interest. By combining the push-out likelihood ratio (LR) method with Leibniz integral rules, it generalizes a broad class of existing LR-based derivative estimators. However, as an LR-based method, its variance is often higher than that of perturbation analysis-based methods and may grow linearly with the dimension of the stochastic input whose distribution depends on the parameter. In this paper, we propose a recursive conditioning approach and combine it with the Leibniz derivative estimation framework. The resulting conditional Leibniz estimator does not involve LR terms and therefore is not subject to variance growth with the input dimension. It also has a simple form and is easy to implement. We apply the method to an American call min-option model, and simulation results show its effectiveness and low-variance performance.

stat.ME↗
arXiv2026

Stochastic Control for Organ Donations: A Review

We review the literature on individual patient organ acceptance decision making by presenting a Markov Decision Process (MDP) model to formulate the organ acceptance decision process as a stochastic control problem. Under the umbrella of the MDP framework, we classify and summarize the major research streams and contributions. In particular, we focus on control limit-type policies, which are shown to be optimal under certain conditions and easy to implement in practice. Finally, we briefly discuss open problems and directions for future research.

eess.SY↗
arXiv2026

Sensitivity analysis for stopping criteria with application to organ transplantations

We consider a stopping problem and its application to the decision-making process regarding the optimal timing of organ transplantation for individual patients. At each decision period, the patient state is inspected and a decision is made whether to transplant. If the organ is transplanted, the process terminates; otherwise, the process continues until a transplant happens or the patient dies. Under suitable conditions, we show that there exists a control limit optimal policy. We propose a smoothed perturbation analysis (SPA) estimator for the gradient of the total expected discounted reward with respect to the control limit. Moreover, we show that the SPA estimator is asymptotically unbiased.

eess.SY↗
arXiv2026

Graph Generation Methods under Partial Information

We study the problem of generating graphs with prescribed degree sequences for bipartite, directed, and undirected networks. We first propose a sequential method for bipartite graph generation and establish a necessary and sufficient interval condition that characterizes the admissible number of connections at each step, thereby guaranteeing global feasibility. Based on this result, we develop bipartite graph enumeration and sampling algorithms suitable for different problem sizes. We then extend these bipartite graph algorithms to the directed and undirected cases by incorporating additional connection constraints, as well as feasibility verification and symmetric connection steps, while preserving the same algorithmic principles. Finally, numerical experiments demonstrate the performance of the proposed algorithms, particularly their scalability to large instances where existing methods become computationally prohibitive.

stat.ME↗
arXiv2025

On Structural Properties of Risk-Averse Optimal Stopping Problems

We establish structural properties of optimal stopping problems under time-consistent dynamic (coherent) risk measures, focusing on value function monotonicity and the existence of control limit (threshold) optimal policies. While such results are well developed for risk-neutral (expected-value) models, they remain underexplored in risk-averse settings. Coherent risk measures typically lack the tower property and are subadditive rather than additive, complicating structural analysis. We show that value function monotonicity mirrors the risk-neutral case. Moreover, if the risk envelope associated with each coherent risk measure admits a minimal element, the risk-averse optimal stopping problem reduces to an equivalent risk-neutral formulation. We also develop a general procedure for identifying control limit optimal policies and use it to derive practical, verifiable conditions on the risk measures and MDP structure that guarantee their existence. We illustrate the theory and verify these conditions through optimal stopping problems arising in operations, marketing, and finance.

eess.SY↗
arXiv2025

Stochastic Derivative Estimation for Discontinuous Sample Performances: A Leibniz Integration Perspective

We develop a novel stochastic derivative estimation framework for sample performance functions that are discontinuous in the parameter of interest, based on the multidimensional Leibniz integral rule. When discontinuities arise from indicator functions, we embed the indicator functions into the sample space, yielding a continuous performance function over a parameter-dependent domain. Applying the Leibniz integral rule in this case produces a single-run, unbiased derivative estimator. For general discontinuous functions, we apply a change of variables to shift parameter dependence into the sample space and the underlying probability measure. Applying the Leibniz integral rule leads to two terms: a standard likelihood ratio (LR) term from differentiating the underlying probability measure and a surface integral from differentiating the boundary of the domain. Evaluating the surface integral may require simulating multiple sample paths. Our proposed Leibniz integration framework generalizes the generalized LR (GLR) method and provides intuition as to when the surface integral vanishes, thereby enabling single-run, easily implementable estimators. Numerical experiments demonstrate the effectiveness and robustness of our methods.

stat.ME↗
arXiv2025

New Bounds and Truncation Boundaries for Importance Sampling

Importance sampling (IS) is a technique that enables statistical estimation of output performance at multiple input distributions from a single nominal input distribution. IS is commonly used in Monte Carlo simulation for variance reduction and in machine learning applications for reusing historical data, but its effectiveness can be challenging to quantify. In this work, we establish a new result showing the tightness of polynomial concentration bounds for classical IS likelihood ratio (LR) estimators in certain settings. Then, to address a practical statistical challenge that IS faces regarding potentially high variance, we propose new truncation boundaries when using a truncated LR estimator, for which we establish upper concentration bounds that imply an exponential convergence rate. Simulation experiments illustrate the contrasting convergence rates of the various LR estimators and the effectiveness of the newly proposed truncation-boundary LR estimators for examples from finance and machine learning.

stat.ME↗
arXiv2025

Generalizing the Generalized Likelihood Ratio Method Through a Push-Out Leibniz Integration Approach

We generalize the generalized likelihood ratio (GLR) method through a novel push-out Leibniz integration approach. Extending the conventional push-out likelihood ratio (LR) method, our approach allows the sample space to be parameter-dependent after the change of variables. Specifically, leveraging the Leibniz integral rule enables differentiation of the parameter-dependent sample space, resulting in a surface integral in addition to the usual LR estimator, which may necessitate additional simulation. Furthermore, our approach extends to cases where the change of variables only locally exists. Notably, the derived estimator includes existing GLR estimators as special cases and is applicable to a broader class of discontinuous sample performances. Moreover, the derivation is streamlined and more straightforward, and the requisite regularity conditions are easier to understand and verify.

stat.ME↗
arXiv2023

Quantile Optimization via Multiple Timescale Local Search for Black-box Functions

We consider quantile optimization of black-box functions that are estimated with noise. We propose two new iterative three-timescale local search algorithms. The first algorithm uses an appropriately modified finite-difference-based gradient estimator that requires $2d$ + 1 samples of the black-box function per iteration of the algorithm, where $d$ is the number of decision variables (dimension of the input vector). For higher-dimensional problems, this algorithm may not be practical if the black-box function estimates are expensive. The second algorithm employs a simultaneous-perturbation-based gradient estimator that uses only three samples for each iteration regardless of problem dimension. Under appropriate conditions, we show the almost sure convergence of both algorithms. In addition, for the class of strongly convex functions, we further establish their (finite-time) convergence rate through a novel fixed-point argument. Simulation experiments indicate that the algorithms work well on a variety of test problems and compare well with recently proposed alternative methods.

math.OC↗
arXiv2022

Optimal Acceptance of Incompatible Kidneys

Incompatibility between patient and donor is a major barrier in kidney transplantation (KT). The increasing shortage of kidney donors has driven the development of desensitization techniques to overcome this immunological challenge. Compared with compatible KT, patients undergoing incompatible KTs are more likely to experience rejection, infection, malignancy, and graft loss. We study the optimal acceptance of possibly incompatible kidneys for individual end-stage kidney disease patients. To capture the effect of incompatibility, we propose a Markov Decision Process (MDP) model that explicitly includes compatibility as a state variable. The resulting higher-dimensional model makes it more challenging to analyze, but under suitable conditions, we derive structural properties including control limit-type optimal policies that are easy to compute and implement. Numerical examples illustrate the behavior of the optimal policy under different mismatch levels and highlight the importance of explicitly incorporating the incompatibility level into the acceptance decision when desensitization therapy is an option.

eess.SY↗
arXiv2021

Adaptive Importance Sampling for Efficient Stochastic Root Finding and Quantile Estimation

In solving simulation-based stochastic root-finding or optimization problems that involve rare events, such as in extreme quantile estimation, running crude Monte Carlo can be prohibitively inefficient. To address this issue, importance sampling can be employed to drive down the sampling error to a desirable level. However, selecting a good importance sampler requires knowledge of the solution to the problem at hand, which is the goal to begin with and thus forms a circular challenge. We investigate the use of adaptive importance sampling to untie this circularity. Our procedure sequentially updates the importance sampler to reach the optimal sampler and the optimal solution simultaneously, and can be embedded in both sample average approximation and stochastic approximation-type algorithms. Our theoretical analysis establishes strong consistency and asymptotic normality of the resulting estimators. We also demonstrate, via a minimax perspective, the key role of using adaptivity in controlling asymptotic errors. Finally, we illustrate the effectiveness of our approach via numerical experiments.

stat.ME↗
arXiv2020

An Optimal Computing Budget Allocation Tree Policy for Monte Carlo Tree Search

We analyze a tree search problem with an underlying Markov decision process, in which the goal is to identify the best action at the root that achieves the highest cumulative reward. We present a new tree policy that optimally allocates a limited computing budget to maximize a lower bound on the probability of correctly selecting the best action at each node. Compared to widely used Upper Confidence Bound (UCB) tree policies, the new tree policy presents a more balanced approach to manage the exploration and exploitation trade-off when the sampling budget is limited. Furthermore, UCB assumes that the support of reward distribution is known, whereas our algorithm relaxes this assumption. Numerical experiments demonstrate the efficiency of our algorithm in selecting the best action at the root.

eess.SY↗
arXiv2020

Supporting Real-Time COVID-19 Medical Management Decisions: The Transition Matrix Model Approach

Since the onset of the COVID-19 outbreak in Wuhan, China, numerous forecasting models have been proposed to project the trajectory of coronavirus infection cases. We propose a new discrete-time Markov chain transition matrix model that directly incorporates stochastic behavior and for which parameter estimation is straightforward from available data. Using such data from China's Hubei province (for which Wuhan is the provincial capital city), the model is shown to be flexible, robust, and accurate. As a result, it has been adopted by the first Shanghai assistance medical team in Wuhan's Jinyintan Hospital, which was the first designated hospital to take COVID-19 patients in the world. The forecast has been used for preparing medical staff, intensive care unit (ICU) beds, ventilators, and other critical care medical resources and for supporting real-time medical management decisions. Empirical data from China's first two months (January/February) of fighting COVID-19 was collected and used to enhance the model by embedding NPI efficiency into the model. We applied the model to forecast Italy, South Korea, and Iran on March 9. Later we made forecasts for Spain, Germany, France, US on March 24. Again, the model has performed very well, proven to be flexible, robust, and accurate for most of these countries/regions outside China.

q-bio.PE↗
arXiv2020

Option Pricing Under a Discrete-Time Markov Switching Stochastic Volatility with Co-Jump Model

We consider option pricing using a discrete-time Markov switching stochastic volatility with co-jump model, which can model volatility clustering and varying mean-reversion speeds of volatility. For pricing European options, we develop a computationally efficient method for obtaining the probability distribution of average integrated variance (AIV), which is key to option pricing under stochastic-volatility-type models. Building upon the efficiency of the European option pricing approach, we are able to price an American-style option, by converting its pricing into the pricing of a portfolio of European options. Our work also provides constructive guidance for analyzing derivatives based on variance, e.g., the variance swap. Numerical results indicate our methods can be implemented very efficiently and accurately.

q-fin.PR↗
arXiv2020

Optimal Unbiased Estimation for Expected Cumulative Cost

We consider estimating an expected infinite-horizon cumulative discounted cost/reward contingent on an underlying stochastic process by Monte Carlo simulation. An unbiased estimator based on truncating the cumulative cost at a random horizon is proposed. Explicit forms for the optimal distributions of the random horizon are given, and explicit expressions for the optimal random truncation level are obtained, leading to a full analysis of the bias-variance tradeoff when comparing this new class of randomized estimators with traditional fixed truncation estimators. Moreover, we characterize when the optimal randomized estimator is preferred over a fixed truncation estimator by considering the tradeoff between bias and variance. This comparison provides guidance on when to choose randomized estimators over fixed truncation estimators in practice. Numerical experiments substantiate the theoretical results.

math.NA↗