arXiv subjects

Timothy C. Y. Chan

Publications and source records attributed to Timothy C. Y. Chan.

At least 19 recordsLinked to original sources

arXiv2026

Estimating Distributions with Failure Rate Properties from Noisy Quantile Data

Estimating an unknown cumulative distribution function (cdf) from data, either as a statistical object of interest or as an input to a downstream optimization problem, is fundamental in operations. In practice, however, distribution estimation is often complicated by incomplete knowledge of the distribution's structure and limited, censored data. To address the first complication, we study distributions satisfying failure-rate shape constraints, especially increasing failure rate (IFR), rather than assuming a fully specified parametric family. To address the second, we consider noisy quantile data: at finitely many prespecified knots, each observation records only whether an independent sample lies below or above the knot. This combination arises naturally in pricing, reliability, and healthcare applications. We formulate the IFR-constrained maximum likelihood estimator and show that the original problem is infinite-dimensional and non-convex. We then develop a tractable two-step approach that solves a finite-dimensional convex optimization problem over transformed knot values and reconstructs a full cdf through shape-preserving interpolation. We establish finite-sample error bounds and convergence rates, yielding practical guidance for offline data collection. We also extend the framework to failure-rate-average, new-better-than-used, and generalized-failure-rate properties. Numerical experiments and case studies in revenue management and reliability demonstrate strong goodness-of-fit and improved downstream decision quality.

stat.AP↗

arXiv2026

Uncertainty Quantification in Data-Driven Inverse Optimization via Bayesian Inference

Inverse optimization (IO) is used to estimate unknown parameters of an optimization model from observed decisions. In the data-driven context, the estimated parameters are inherently uncertain, yet quantifying this uncertainty has received limited attention in the literature, where existing methods return a point estimate. In this paper, we propose a hierarchical Bayesian framework for parameter uncertainty quantification in data-driven inverse optimization. Considering two data-generating processes, we develop two Markov chain Monte Carlo algorithms to estimate the posterior distribution of the unknown parameter vector, which is used to construct credible regions. We establish posterior consistency under standard identifiability conditions. Numerical experiments demonstrate near-nominal empirical coverage of the credible regions and show that the regions shrink as the number of observed decisions increases.

math.OC↗

arXiv2026

Geometry-free prediction of inertial lift forces in microfluidic devices using deep learning

Inertial microfluidic devices (IMDs) offer low-cost, high-throughput alternative techniques for many traditional particle- (or cell-) manipulation tasks, but simulating them requires being able to predict particle migration, and thus particle lift forces, under a variety of possible channel geometries. Recent work has demonstrated that machine learning models can be used to drastically speed up these numerical simulations, but doing so required training individual models for every unique channel cross-section type (e.g., rectangular, triangular) -- shifting the burden from the simulation step to the training step. In this paper, we develop a novel approach for predicting particle lift forces that contains no explicit geometric parameters. We train a neural network model using a new parameter set and show that while it performs comparably to existing models on channel geometries in the training set, it is able to generalize to unseen channel geometries far more effectively. We show that the lift force model developed herein can be easily transferred to particle tracing simulation software, where it is capable of predicting particle migration patterns consistent with the literature across a variety of channel designs.

cs.LG↗

arXiv2025

Dynamic Transfer Policies for Parallel Queues

We consider the problem of load balancing in parallel queues by transferring customers between them at discrete points in time. Holding costs accrue as customers wait in the queue, while transfer decisions incur both fixed (setup) costs and variable costs that increase with the number of transfers and travel distance, and vary by transfer direction. Our work is primarily motivated by inter-facility patient transfers to address imbalanced congestion and inequity in access to care during surges in hospital demand. Analyzing an associated fluid control problem, we show that under general assumptions, including time-varying arrivals and convex holding costs, the optimal policy partitions the state-space into a well-defined $\textit{no-transfer region}$ and its complement, implying that transferring is optimal if and only if the system is sufficiently imbalanced. In the absence of fixed transfer costs, an optimal policy moves the state to the no-transfer region's boundary; in contrast, with fixed costs, the state is moved to its relative interior. Leveraging our structural results, we propose a simulation-based approximate dynamic programming (ADP) algorithm to find effective transfer policies for the stochastic system. We investigate the performance and robustness of the fluid and ADP policies in a case study calibrated using data during the COVID-19 pandemic in the Greater Toronto Area, which demonstrates that transferring patients between hospitals could result in up to 27.7% reduction in total cost with relatively few transfers.

math.OC↗

arXiv2025

Formal Verification of Markov Processes with Learned Parameters

We introduce the problem of formally verifying properties of Markov processes where the parameters are given by the output of machine learning models. For a broad class of machine learning models, including linear models, tree-based models, and neural networks, verifying properties of Markov chains like reachability, hitting time, and total reward can be formulated as a bilinear program. We develop a decomposition and bound propagation scheme for solving the bilinear program and show through computational experiments that our method solves the problem to global optimality up to 100x faster than state-of-the-art solvers. To demonstrate the practical utility of our approach, we apply it to a real-world healthcare case study. Along with the paper, we release markovml, an open-source tool for building Markov processes, integrating pretrained machine learning models, and verifying their properties, available at https://github.com/mmaaz-git/markovml.

cs.LG↗

arXiv2025

Machine Learning-Augmented Optimization of Large Bilevel and Two-stage Stochastic Programs: Application to Cycling Network Design

A wide range of decision problems can be formulated as bilevel programs with independent followers, which as a special case include two-stage stochastic programs. These problems are notoriously difficult to solve especially when a large number of followers present. Motivated by a real-world cycling infrastructure planning application, we present a general approach to solving such problems. We propose an optimization model that explicitly considers a sampled subset of followers and exploits a machine learning model to estimate the objective values of unsampled followers. We prove bounds on the optimality gap of the generated leader decision as measured by the original objective function that considers the full follower set. We then develop follower sampling algorithms to tighten the bounds and a representation learning approach to learn follower features, which are used as inputs to the embedded machine learning model. Through numerical studies, we show that our approach generates leader decisions of higher quality compared to baselines. Finally, in collaboration with the City of Toronto, we perform a real-world case study in Toronto where we solve a cycling network design problem with over one million followers. Compared to the current practice, our approach improves Toronto's cycling accessibility by 19.2%, equivalent to $18M in potential cost savings. Our approach is being used to inform the cycling infrastructure planning in Toronto and outperforms the current practice by a large margin. It can be generalized to any decision problems that are formulated as bilevel programs with independent followers.

math.OC↗

arXiv2024

Exact sensitivity analysis of Markov reward processes via algebraic geometry

We introduce a new approach for deterministic sensitivity analysis of Markov reward processes, commonly used in cost-effectiveness analyses, via reformulation into a polynomial system. Our approach leverages cylindrical algebraic decomposition (CAD), a technique arising from algebraic geometry that provides an exact description of all solutions to a polynomial system. While it is typically intractable to build a CAD for systems with more than a few variables, we show that a special class of polynomial systems, which includes the polynomials arising from Markov reward processes, can be analyzed much more tractably. We establish several theoretical results about such systems and develop a specialized algorithm to construct their CAD, which allows us to perform exact, multi-way sensitivity analysis for common health economic analyses. We develop an open-source software package that implements our algorithm. Finally, we apply it to two case studies, one with synthetic data and one that re-analyzes a previous cost-effectiveness analysis from the literature, demonstrating advantages of our approach over standard techniques. Our software and code are available at: \url{https://github.com/mmaaz-git/markovag}.

math.OC↗

arXiv2024

Learning Risk Preferences in Markov Decision Processes: an Application to the Fourth Down Decision in the National Football League

For decades, National Football League (NFL) coaches' observed fourth down decisions have been largely inconsistent with prescriptions based on statistical models. In this paper, we develop a framework to explain this discrepancy using an inverse optimization approach. We model the fourth down decision and the subsequent sequence of plays in a game as a Markov decision process (MDP), the dynamics of which we estimate from NFL play-by-play data from the 2014 through 2022 seasons. We assume that coaches' observed decisions are optimal but that the risk preferences governing their decisions are unknown. This yields an inverse decision problem for which the optimality criterion, or risk measure, of the MDP is the estimand. Using the quantile function to parameterize risk, we estimate which quantile-optimal policy yields the coaches' observed decisions as minimally suboptimal. In general, we find that coaches' fourth-down behavior is consistent with optimizing low quantiles of the next-state value distribution, which corresponds to conservative risk preferences. We also find that coaches exhibit higher risk tolerances when making decisions in the opponent's half of the field as opposed to their own half, and that league average fourth down risk tolerances have increased over time.

stat.AP↗

arXiv2024

Dynamic Control of Service Systems with Returns: Application to Design of Post-Discharge Hospital Readmission Prevention Programs

We study a control problem for queueing systems where customers may return for additional episodes of service after their initial service completion. At each service completion epoch, the decision maker can choose to reduce the probability of return for the departing customer but at a cost that is convex increasing in the amount of reduction in the return probability. Other costs are incurred as customers wait in the queue and every time they return for service. Our primary motivation comes from post-discharge Quality Improvement (QI) interventions (e.g., follow up phone-calls, appointments) frequently used in a variety of healthcare settings to reduce unplanned hospital readmissions. Our objective is to understand how the cost of interventions should be balanced with the reductions in congestion and service costs. To this end, we consider a fluid approximation of the queueing system and characterize the structure of optimal long-run average and bias-optimal transient control policies for the fluid model. Our structural results motivate the design of intuitive surge protocols whereby different intensities of interventions (corresponding to different levels of reduction in the return probability) are provided based on the congestion in the system. Through extensive simulation experiments, we study the performance of the fluid policy for the stochastic system and identify parameter regimes where it leads to significant cost savings compared to a fixed long-run average optimal policy that ignores holding costs and a simple policy that uses the highest level of intervention whenever the queue is non-empty. In particular, we find that in a parameter regime relevant to our motivating application, dynamically adjusting the intensity of interventions could result in up to 25.4% reduction in long-run average cost and 33.7% in finite-horizon costs compared to the simple aggressive policy.

eess.SY↗

arXiv2024

Conformal Inverse Optimization

Inverse optimization has been increasingly used to estimate unknown parameters in an optimization model based on decision data. We show that such a point estimation is insufficient in a prescriptive setting where the estimated parameters are used to prescribe new decisions. The prescribed decisions may be low-quality and misaligned with human intuition and thus are unlikely to be adopted. To tackle this challenge, we propose conformal inverse optimization, which seeks to learn an uncertainty set for the unknown parameters and then solve a robust optimization model to prescribe new decisions. Under mild assumptions, we show that our method enjoys provable guarantees on solution quality, as evaluated using both the ground-truth parameters and the decision maker's perception of the unknown parameters. Our method demonstrates strong empirical performance compared to classic inverse optimization.

math.OC↗

arXiv2024

Network Flow Models for Robust Binary Optimization with Selective Adaptability

Adaptive robust optimization problems have received significant attention in recent years, but remain notoriously difficult to solve when recourse decisions are discrete in nature. In this paper, we propose new reformulation techniques for adaptive robust binary optimization (ARBO) problems with objective uncertainty. Without loss of generality, we focus on ARBO problems with "selective adaptability", a term we coin to describe a common class of linking constraints between first-stage and second-stage solutions. Our main contribution revolves around a collection of exact and approximate network flow reformulations for the ARBO problem, which we develop by building upon ideas from the decision diagram literature. Our proposed models can generate feasible solutions, primal bounds and dual bounds, while their size and approximation quality can be precisely controlled through user-specified parameters. Furthermore, and in contrast with existing solution methods, these models are easy to implement and can be solved directly with standard off-the-shelf solvers. Through an extensive set of computational experiments, we show that our models can generate high-quality solutions and dual bounds in significantly less time than popular benchmark methods, often by orders of magnitude.

math.OC↗

arXiv2023

Miss It Like Messi: Extracting Value from Off-Target Shots in Soccer

Measuring soccer shooting skill is a challenging analytics problem due to the scarcity and highly contextual nature of scoring events. The introduction of more advanced data surrounding soccer shots has given rise to model-based metrics which better cope with these challenges. Specifically, metrics such as expected goals added, goals above expectation, and post-shot expected goals all use advanced data to offer an improvement over the classical conversion rate. However, all metrics developed to date assign a value of zero to off-target shots, which account for almost two-thirds of all shots, since these shots have no probability of scoring. We posit that there is non-negligible shooting skill signal contained in the trajectories of off-target shots and propose two shooting skill metrics that incorporate the signal contained in off-target shots. Specifically, we develop a player-specific generative model for shot trajectories based on a mixture of truncated bivariate Gaussian distributions. We use this generative model to compute metrics that allow us to attach non-zero value to off-target shots. We demonstrate that our proposed metrics are more stable than current state-of-the-art metrics and have increased predictive power.

stat.AP↗

arXiv2023

AutoLTS: Automating Cycling Stress Assessment via Contrastive Learning and Spatial Post-processing

Cycling stress assessment, which quantifies cyclists' perceived stress imposed by the built environment and motor traffics, increasingly informs cycling infrastructure planning and cycling route recommendation. However, currently calculating cycling stress is slow and data-intensive, which hinders its broader application. In this paper, We propose a deep learning framework to support accurate, fast, and large-scale cycling stress assessments for urban road networks based on street-view images. Our framework features i) a contrastive learning approach that leverages the ordinal relationship among cycling stress labels, and ii) a post-processing technique that enforces spatial smoothness into our predictions. On a dataset of 39,153 road segments collected in Toronto, Canada, our results demonstrate the effectiveness of our deep learning framework and the value of using image data for cycling stress assessment in the absence of high-quality road geometry and motor traffic data.

cs.CV↗

arXiv2023

3D dose prediction for Gamma Knife radiosurgery using deep learning and data modification

Purpose: To develop a machine learning-based, 3D dose prediction methodology for Gamma Knife (GK) radiosurgery. The methodology accounts for cases involving targets of any number, size, and shape. Methods: Data from 322 GK treatment plans was modified by isolating and cropping the contoured MRI and clinical dose distributions based on tumor location, then scaling the resulting tumor spaces to a standard size. An accompanying 3D tensor was created for each instance to account for tumor size. The modified dataset for 272 patients was used to train both a generative adversarial network (GAN-GK) and a 3D U-Net model (U-Net-GK). Unmodified data was used to train equivalent baseline models. All models were used to predict the dose distribution of 50 out-of-sample patients. Prediction accuracy was evaluated using gamma, with criteria of 4%/2mm, 3%/3mm, 3%/1mm and 1%/1mm. Prediction quality was assessed using coverage, selectivity, and conformity indices. Results: The predictions resulting from GAN-GK and U-Net-GK were similar to their clinical counterparts, with average gamma (4%/2mm) passing rates of 84.9 and 83.1, respectively. In contrast, the gamma passing rate of baseline models were significantly worse than their respective GK-specific models (p < 0.001) at all criterion levels. The quality of GK-specific predictions was also similar to that of clinical plans. Conclusion: Deep learning models can use GK-specific data modification to predict 3D dose distributions for GKRS plans with a large range in size, shape, or number of targets. Standard deep learning models applied to unmodified GK data generated poorer predictions.

physics.med-ph↗

arXiv2022

Inverse Optimization: Theory and Applications

Inverse optimization describes a process that is the "reverse" of traditional mathematical optimization. Unlike traditional optimization, which seeks to compute optimal decisions given an objective and constraints, inverse optimization takes decisions as input and determines an objective and/or constraints that render these decisions approximately or exactly optimal. In recent years, there has been an explosion of interest in the mathematics and applications of inverse optimization. This paper provides a comprehensive review of both the methodological and application-oriented literature in inverse optimization.

math.OC↗

arXiv2022

Equity, diversity, and inclusion in sports analytics

This paper presents a landmark study of equity, diversity and inclusion (EDI) in the field of sports analytics. We developed a survey that examined personal and job-related demographics, as well as individual perceptions and experiences about EDI in the workplace. We sent the survey to individuals in the five major North American professional leagues, representatives from the Olympic and Paralympic Committees in Canada and the U.S., the NCAA Division I programs, companies in sports tech/analytics, and university research groups. Our findings indicate the presence of a clear dominant group in sports analytics identifying as: young (72.0%), White (69.5%), heterosexual (89.7%) and male (82.0%). Within professional sports, males in management positions earned roughly 30,000 USD (27%) more on average compared to females. A smaller but equally alarming pay gap of 17,000 USD (14%) was found between White and non-White management personnel. Of concern, females were nearly five times as likely to experience discrimination and twice as likely to have considered leaving their job due to isolation or feeling unwelcome. While they had similar levels of agreement regarding fair processes for rewards and compensation, females "strongly agreed" less often than males regarding equitable support, equitable workload, having a voice, and being taken seriously. Over one third (36.3%) of females indicated that they "strongly agreed" that they must work harder than others to be valued equally, compared to 9.8% of males. We conclude the paper with concrete recommendations that could be considered to create a more equitable, diverse and inclusive environment for individuals working within the sports analytics sector.

stat.AP↗

arXiv2022

Inverse Optimization on Hierarchical Networks: An Application to Breast Cancer Clinical Pathways

Clinical pathways are standardized processes that outline the steps required for managing a specific disease. However, patient pathways often deviate from clinical pathways. Measuring the concordance of patient pathways to clinical pathways is important for health system monitoring and informing quality improvement initiatives. In this paper, we develop an inverse optimization-based approach to measuring pathway concordance in breast cancer, a complex disease. We capture this complexity in a hierarchical network that models the patient's journey through the health system. A novel inverse shortest path model is formulated and solved on this hierarchical network to estimate arc costs, which are used to form a concordance metric to measure the distance between patient pathways and shortest paths (i.e., clinical pathways). Using real breast cancer patient data from Ontario, Canada, we demonstrate that our concordance metric has a statistically significant association with survival for all breast cancer patient subgroups. We also use it to quantify the extent of patient pathway discordances across all subgroups, finding that patients undertaking additional clinical activities constitute the primary driver of discordance in the population.

math.OC↗

arXiv2022

Learning to Optimize Contextually Constrained Problems for Real-Time Decision-Generation

The topic of learning to solve optimization problems has received interest from both the operations research and machine learning communities. In this work, we combine techniques from both fields to address the problem of learning to generate decisions to instances of continuous optimization problems where the feasible set varies with contextual features. We propose a novel framework for training a generative model to estimate optimal decisions by combining interior point methods and adversarial learning, which we further embed within an data generation algorithm. Decisions generated by our model satisfy in-sample and out-of-sample optimality guarantees. Finally, we investigate case studies in portfolio optimization and personalized treatment design, demonstrating that our approach yields advantages over predict-then-optimize and supervised deep learning techniques, respectively.

cs.LG↗