Searcharxiv⌕ Search

arXiv subjects

Pengyi Shi

Publications and source records attributed to Pengyi Shi.

13 recordsLinked to original sources
arXiv2026

Inpatient Overflow Management with Proximal Policy Optimization

Problem Definition: Managing inpatient flow in large hospital systems is challenging due to the complexity of assigning randomly arriving patients -- either waiting for primary units or being overflowed to alternative units. Current practices rely on ad-hoc rules, while prior analytical approaches struggle with the intractably large state and action spaces inherent in patient-unit matching. Scalable decision support is needed to optimize overflow management while accounting for time-periodic fluctuations in patient flow. Methodology/Results: We develop a scalable decision-making framework using Proximal Policy Optimization (PPO) to optimize overflow decisions in a time-periodic, long-run average cost setting. To address the combinatorial complexity, we introduce atomic actions, which decompose multi-patient routing into sequential assignments. We further enhance computational efficiency through a partially-shared policy network designed to balance parameter sharing with time-specific policy adaptations, and a queueing-informed value function approximation to improve policy evaluation. Our method significantly reduces the need for extensive simulation data, a common limitation in reinforcement learning applications. Case studies on hospital systems with up to twenty patient classes and twenty wards demonstrate that our approach matches or outperforms existing benchmarks, including approximate dynamic programming, which is computationally infeasible beyond five wards. Managerial Implications: Our framework offers a scalable, efficient, and explainable solution for managing patient flow in complex hospital systems. More broadly, our results highlight that domain-aware adaptation is more critical to improving algorithm performance than fine-tuning neural network parameters when applying general-purpose algorithms to specific applications.

math.OC↗
arXiv2026

The Dynamic and Endogenous Behavior of Re-Offense Risk: An Agent-Based Simulation Study of Treatment Allocation in Incarceration Diversion Programs

Incarceration-diversion treatment programs aim to improve societal reintegration and reduce recidivism, but limited capacity forces policymakers to make prioritization decisions that often rely on risk assessment tools. While predictive, these tools typically treat risk as a static, individual attribute, which overlooks how risk evolves over time and how treatment decisions shape outcomes through social interactions. In this paper, we develop a new framework that models reoffending risk as a human-system interaction, linking individual behavior with system-level dynamics and endogenous community feedback. Using an agent-based simulation calibrated to U.S. probation data, we evaluate treatment allocation policies under different capacity constraints and incarceration settings. Our results show that no single prioritization policy dominates. Instead, policy effectiveness depends on temporal windows and system parameters: prioritizing low-risk individuals performs better when long-term trajectories matter, while prioritizing high-risk individuals becomes more effective in the short term or when incarceration leads to shorter monitoring periods. These findings highlight the need to evaluate risk-based decision systems as sociotechnical systems with long-term accountability, rather than as isolated predictive tools.

cs.CY↗
arXiv2025

Dynamic Redeployment of Nurses Across Hospitals: A Sample Robust Optimization Approach

Problem definition: We study a workforce redeployment problem in hospital networks, where clinical staff, such as nurses, are temporarily reassigned from overstaffed to understaffed sites to address short-term imbalances. This practice of ``internal travel,'' which gained traction during the COVID-19 pandemic to tackle nurse shortages, presents new operational challenges that require tailored analytical support. Key requirements such as advance notice and short-term secondments must be incorporated. Moreover, in rapidly evolving environments, reliance on historical data leads to unreliable forecasts, limiting the effectiveness of traditional sample-based methods. Methodology: We formulate the problem as a stochastic dynamic program and incorporate demand uncertainty via a sample robust optimization (SRO) framework. Using linear decision rule approximation, we reformulate the problem as a tractable linear program. Results: We evaluate the impact of key network design components on system performance. Network connectivity has the largest effect in reducing the total cost, number of redeployments, and travel distance, but its benefits depend on aligning the secondment duration with the network structure. Full connectivity without proper secondments can be counterproductive. The SRO approach outperforms the traditional sample-average method in the presence of demand surges or under-forecasts by better anticipating emergency redeployments. Managerial implications: Internal travel programs offer a promising strategy to alleviate workforce shortages in healthcare systems. Our results highlight the importance of network design, aligning secondment durations with the network structure, and adopting planning methods that are robust to demand surges or inaccurate predictions.

math.OC↗
arXiv2024

Latent Feature Mining for Predictive Model Enhancement with Large Language Models

Predictive modeling often faces challenges due to limited data availability and quality, especially in domains where collected features are weakly correlated with outcomes and where additional feature collection is constrained by ethical or practical difficulties. Traditional machine learning (ML) models struggle to incorporate unobserved yet critical factors. In this work, we introduce an effective approach to formulate latent feature mining as text-to-text propositional logical reasoning. We propose FLAME (Faithful Latent Feature Mining for Predictive Model Enhancement), a framework that leverages large language models (LLMs) to augment observed features with latent features and enhance the predictive power of ML models in downstream tasks. Our framework is generalizable across various domains with necessary domain-specific adaptation, as it is designed to incorporate contextual information unique to each area, ensuring effective transfer to different areas facing similar data availability challenges. We validate our framework with two case studies: (1) the criminal justice system, a domain characterized by limited and ethically challenging data collection; (2) the healthcare domain, where patient privacy concerns and the complexity of medical data limit comprehensive feature collection. Our results show that inferred latent features align well with ground truth labels and significantly enhance the downstream classifier.

cs.LG↗
arXiv2024

Patient Assignment and Prioritization for Multi-Stage Care with Reentrance

In this paper, we study a queueing model that incorporates patient reentrance to reflect patients' recurring requests for nurse care and their rest periods between these requests. Within this framework, we address two levels of decision-making: the priority discipline decision for each nurse and the nurse-patient assignment problem. We introduce the shortest-first and longest-first rules in the priority discipline decision problem and show the condition under which each policy excels through theoretical analysis and comprehensive simulations. For the nurse-patient assignment problem, we propose two heuristic policies. We show that the policy maximizing the immediate decrease in holding costs outperforms the alternative policy, which considers the long-term aggregate holding cost. Additionally, both proposed policies significantly surpass the benchmark policy, which does not utilize queue length information.

math.OC↗
arXiv2022

Data-pooling Reinforcement Learning for Personalized Healthcare Intervention

Motivated by the emerging needs of personalized preventative intervention in many healthcare applications, we consider a multi-stage, dynamic decision-making problem in the online setting with unknown model parameters. To deal with the pervasive issue of small sample size in personalized planning, we develop a novel data-pooling reinforcement learning (RL) algorithm based on a general perturbed value iteration framework. Our algorithm adaptively pools historical data, with three main innovations: (i) the weight of pooling ties directly to the performance of decision (measured by regret) as opposed to estimation accuracy in conventional methods; (ii) no parametric assumptions are needed between historical and current data; and (iii) requiring data-sharing only via aggregate statistics, as opposed to patient-level data. Our data-pooling algorithm framework applies to a variety of popular RL algorithms, and we establish a theoretical performance guarantee showing that our pooling version achieves a regret bound strictly smaller than that of the no-pooling counterpart. We substantiate the theoretical development with empirically better performance of our algorithm via a case study in the context of post-discharge intervention to prevent unplanned readmissions, generating practical insights for healthcare management. In particular, our algorithm alleviates privacy concerns about sharing health data, which (i) opens the door for individual organizations to levering public datasets or published studies to better manage their own patients; and (ii) provides the basis for public policy makers to encourage organizations to share aggregate data to improve population health outcomes for the broader community.

cs.LG↗
arXiv2021

An Efficient Pessimistic-Optimistic Algorithm for Stochastic Linear Bandits with General Constraints

This paper considers stochastic linear bandits with general nonlinear constraints. The objective is to maximize the expected cumulative reward over horizon $T$ subject to a set of constraints in each round $τ\leq T$. We propose a pessimistic-optimistic algorithm for this problem, which is efficient in two aspects. First, the algorithm yields $\tilde{\cal O}\left(\left(\frac{K^{0.75}}δ+d\right)\sqrtτ\right)$ (pseudo) regret in round $τ\leq T,$ where $K$ is the number of constraints, $d$ is the dimension of the reward feature space, and $δ$ is a Slater's constant; and zero constraint violation in any round $τ>τ',$ where $τ'$ is independent of horizon $T.$ Second, the algorithm is computationally efficient. Our algorithm is based on the primal-dual approach in optimization and includes two components. The primal component is similar to unconstrained stochastic linear bandits (our algorithm uses the linear upper confidence bound algorithm (LinUCB)). The computational complexity of the dual component depends on the number of constraints, but is independent of the sizes of the contextual space, the action space, and the feature space. Thus, the overall computational complexity of our algorithm is similar to that of the linear UCB for unconstrained stochastic linear bandits.

cs.LG↗
arXiv2021

Group-Aware Threshold Adaptation for Fair Classification

The fairness in machine learning is getting increasing attention, as its applications in different fields continue to expand and diversify. To mitigate the discriminated model behaviors between different demographic groups, we introduce a novel post-processing method to optimize over multiple fairness constraints through group-aware threshold adaptation. We propose to learn adaptive classification thresholds for each demographic group by optimizing the confusion matrix estimated from the probability distribution of a classification model output. As we only need an estimated probability distribution of model output instead of the classification model structure, our post-processing model can be applied to a wide range of classification models and improve fairness in a model-agnostic manner and ensure privacy. This even allows us to post-process existing fairness methods to further improve the trade-off between accuracy and fairness. Moreover, our model has low computational cost. We provide rigorous theoretical analysis on the convergence of our optimization algorithm and the trade-off between accuracy and fairness of our method. Our method theoretically enables a better upper bound in near optimality than existing method under same condition. Experimental results demonstrate that our method outperforms state-of-the-art methods and obtains the result that is closest to the theoretical accuracy-fairness trade-off boundary.

cs.LG↗
arXiv2021

POND: Pessimistic-Optimistic oNline Dispatching

This paper considers constrained online dispatching with unknown arrival, reward and constraint distributions. We propose a novel online dispatching algorithm, named POND, standing for Pessimistic-Optimistic oNline Dispatching, which achieves $O(\sqrt{T})$ regret and $O(1)$ constraint violation. Both bounds are sharp. Our experiments on synthetic and real datasets show that POND achieves low regret with minimal constraint violations.

cs.LG↗
arXiv2021

A High-fidelity, Machine-learning Enhanced Queueing Network Simulation Model for Hospital Ultrasound Operations

We collaborate with a large teaching hospital in Shenzhen, China and build a high-fidelity simulation model for its ultrasound center to predict key performance metrics, including the distributions of queue length, waiting time and sojourn time, with high accuracy. The key challenge to build an accurate simulation model is to understanding the complicated patient routing at the ultrasound center. To address the issue, we propose a novel two-level routing component to the queueing network model. We apply machine learning tools to calibrate the key components of the queueing model from data with enhanced accuracy.

cs.LG↗
arXiv2019

Robustness-Driven Exploration with Probabilistic Metric Temporal Logic

The ability to perform autonomous exploration is essential for unmanned aerial vehicles (UAV) operating in unstructured or unknown environments where it is hard or even impossible to describe the environment beforehand. However, algorithms for autonomous exploration often focus on optimizing time and coverage in a greedy fashion. That type of exploration can collect irrelevant data and wastes time navigating areas with no important information. In this paper, we propose a method for exploiting the discovered knowledge about the environment while exploring it by relying on a theory of robustness based on Probabilistic Metric Temporal Logic (P-MTL) as applied to offline verification and online control of hybrid systems. By maximizing the satisfaction of the predefined P-MTL specifications of the exploration problem, the robustness values guide the UAV towards areas with more interesting information to gain. We use Markov Chain Monte Carlo to solve the P-MTL constraints. We demonstrate the effectiveness of the proposed approach by simulating autonomous exploration over Amazonian rainforest where our approach is used to detect areas occupied by illegal Artisanal Small-scale Gold Mining (ASGM) activities. The results show that our approach outperform a greedy exploration approach (Autonomous Exploration Planner) by 38% in terms of ASGM coverage.

cs.RO↗
arXiv2018

Steady-state Diffusion Approximations for Discrete-time Queue in Hospital Inpatient Flow Management

In this paper, we analyze a discrete-time queue that is motivated from studying hospital inpatient flow management, where the customer count process captures the midnight inpatient census. The stationary distribution of the customer count has no explicit form and is difficult to compute in certain parameter regimes. Using the Stein's method framework, we identify a continuous random variable to approximate the steady-state customer count. The continuous random variable corresponds to the stationary distribution of a diffusion process with state-dependent diffusion coefficients. We characterize the error bounds of this approximation under a variety of system load conditions -- from lightly loaded to heavily loaded. We also identify the critical role that the service rate plays in the convergence rate of the error bounds. We perform extensive numerical experiments to support the theoretical findings and to demonstrate the approximation quality. In particular, we show that our approximation performs better than those based on constant diffusion coefficients when the number of servers is small, which is relevant to decision making in a single hospital ward.

math.PR↗