arXiv subjects

Patrick Emedom-Nnamdi

Publications and source records attributed to Patrick Emedom-Nnamdi.

3 recordsLinked to original sources

arXiv2025

Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recovery

We propose a nonparametric additive model for estimating interpretable value functions in reinforcement learning, with an application in optimizing postoperative recovery through personalized, adaptive recommendations. While reinforcement learning has achieved significant success in various domains, recent methods often rely on black-box approaches such as neural networks, which hinder the examination of individual feature contributions to a decision-making policy. Our novel method offers a flexible technique for estimating action-value functions without explicit parametric assumptions, overcoming the limitations of the linearity assumption of classical algorithms. By incorporating local kernel regression and basis expansion, we obtain a sparse, additive representation of the action-value function, enabling local approximation and retrieval of nonlinear, independent contributions of select state features and the interactions between joint feature pairs. We validate our approach through a simulation study and apply it to spine disease recovery, uncovering recommendations aligned with clinical knowledge. This method bridges the gap between flexible machine learning techniques and the interpretability required in healthcare applications, paving the way for more personalized interventions.

stat.ML↗

arXiv2023

Knowledge Transfer from Teachers to Learners in Growing-Batch Reinforcement Learning

Standard approaches to sequential decision-making exploit an agent's ability to continually interact with its environment and improve its control policy. However, due to safety, ethical, and practicality constraints, this type of trial-and-error experimentation is often infeasible in many real-world domains such as healthcare and robotics. Instead, control policies in these domains are typically trained offline from previously logged data or in a growing-batch manner. In this setting a fixed policy is deployed to the environment and used to gather an entire batch of new data before being aggregated with past batches and used to update the policy. This improvement cycle can then be repeated multiple times. While a limited number of such cycles is feasible in real-world domains, the quality and diversity of the resulting data are much lower than in the standard continually-interacting approach. However, data collection in these domains is often performed in conjunction with human experts, who are able to label or annotate the collected data. In this paper, we first explore the trade-offs present in this growing-batch setting, and then investigate how information provided by a teacher (i.e., demonstrations, expert actions, and gradient information) can be leveraged at training time to mitigate the sample complexity and coverage requirements for actor-critic methods. We validate our contributions on tasks from the DeepMind Control Suite.

cs.LG↗

arXiv2017

Queueing Analysis of a Chagas Disease Control Campaign

A critical component of preventing the spread of vector borne diseases such as Chagas disease are door-to-door campaigns by public health officials that implement insecticide application in order to eradicate the vector infestation of households. The success of such campaigns depends on adequate household participation during the active phase as well as on sufficient follow-up during the surveillance phase when newly infested houses or infested houses that had not participated in the active phase will receive treatment. Queueing models which are widely used in operations management give us a mathematical representation of the operational efforts needed to contain the spread of infestation. By modeling the queue as consisting of all infested houses in a given locality, we capture the dynamics of the insect population due to prevalence of infestation and to the additional growth of infestation by redispersion, i.e. by the spread of infestation to previously uninfested houses during the wait time for treatment. In contrast to traditional queueing models, houses waiting for treatment are not known but must be identified through a search process by public health workers. Thus, both the arrival rate of houses to the queue as well as the removal rate from the queue depend on the current level of infestation. We incorporate these dependencies through a load dependent queueing model which allows us to estimate the long run average rate of removing houses from the queue and therefore the cost associated with a given surveillance program. The model is motivated by and applied to an ongoing Chagas disease control campaign in Arequipa, Peru.

q-bio.QM↗