arXiv subjects

Tyler Becker

Publications and source records attributed to Tyler Becker.

6 recordsLinked to original sources

arXiv2025

Simultaneous AlphaZero: Extending Tree Search to Markov Games

Many strategic planning problems require agents to act simultaneously, making turn-based search unsuitable and requiring a normal form game to be solved at each tree state. We introduce Simultaneous AlphaZero, a learning and search method for continuous-state, two-player zero-sum deterministic Markov games. A learned value function bootstraps finite-depth regret-matching search, while approximate regret transfer uses learned regret and average-strategy functions to warm start the local solver at deployment. We establish complementary guarantees for both components. First, we show that even a shallow finite-depth solver contracts errors in its learned frontier values, and we bound how local solution and value-fitting errors propagate through repeated learning and search. Second, imperfect regret and average-strategy transfer yields a finite-time approximate-equilibrium bound decomposing ordinary regret, game mismatch, regret-fitting error, and strategy-fitting error; any fixed finite warm start preserves the solver's asymptotic no-regret guarantee. Experiments in continuous Dubins pursuit-evasion and satellite custody maintenance compare unguided, value-only, and fully transferred search in direct solver cross-play. Across both domains, oracle-guided search generally improves performance over unguided search, demonstrating effective game-theoretic planning in large continuous state spaces.

cs.GT↗

arXiv2024

Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs

When human operators of cyber-physical systems encounter surprising behavior, they often consider multiple hypotheses that might explain it. In some cases, taking information-gathering actions such as additional measurements or control inputs given to the system can help resolve uncertainty and determine the most accurate hypothesis. The task of optimizing these actions can be formulated as a belief-space Markov decision process that we call a hypothesis-driven belief MDP. Unfortunately, this problem suffers from the curse of history similar to a partially observable Markov decision process (POMDP). To plan in continuous domains, an agent needs to reason over countlessly many possible action-observation histories, each resulting in a different belief over the unknown state. The problem is exacerbated in the hypothesis-driven context because each action-observation pair spawns a different belief for each hypothesis, leading to additional branching. This paper considers the case in which each hypothesis corresponds to a different dynamic model in an underlying POMDP. We present a new belief MDP formulation that: (i) enables reasoning over multiple hypotheses, (ii) balances the goals of determining the (most likely) correct hypothesis and performing well in the underlying POMDP, and (iii) can be solved with sparse tree search.

cs.AI↗

arXiv2024

Through the Clutter: Exploring the Impact of Complex Environments on the Legibility of Robot Motion

The environments in which the collaboration of a robot would be the most helpful to a person are frequently uncontrolled and cluttered with many objects present. Legible robot arm motion is crucial in tasks like these in order to avoid possible collisions, improve the workflow and help ensure the safety of the person. Prior work in this area, however, focuses on solutions that are tested only in uncluttered environments and there are not many results taken from cluttered environments. In this research we present a measure for clutteredness based on an entropic measure of the environment, and a novel motion planner based on potential fields. Both our measures and the planner were tested in a cluttered environment meant to represent a more typical tool sorting task for which the person would collaborate with a robot. The in-person validation study with Baxter robots shows a significant improvement in legibility of our proposed legible motion planner compared to the current state-of-the-art legible motion planner in cluttered environments. Further, the results show a significant difference in the performance of the planners in cluttered and uncluttered environments, and the need to further explore legible motion in cluttered environments. We argue that the inconsistency of our results in cluttered environments with those obtained from uncluttered environments points out several important issues with the current research performed in the area of legible motion planners.

cs.RO↗

arXiv2024

Bridging the Gap between Partially Observable Stochastic Games and Sparse POMDP Methods

Many real-world decision problems involve the interaction of multiple self-interested agents with limited sensing ability. The partially observable stochastic game (POSG) provides a mathematical framework for modeling these problems, however solving a POSG requires difficult reasoning over two critical factors: (1) information revealed by partial observations and (2) decisions other agents make. In the single agent case, partially observable Markov decision process (POMDP) planning can efficiently address partial observability with particle filtering. In the multi-agent case, extensive form game solution methods account for other agent's decisions, but preclude belief approximation. We propose a unifying framework that combines POMDP-inspired state distribution approximation and game-theoretic equilibrium search on information sets. This paper lays a theoretical foundation for the approach by bounding errors due to belief approximation, and empirically demonstrates effectiveness with a numerical example. The new approach enables planning in POSGs with very large state spaces, paving the way for reliable autonomous interaction in real-world physical environments and complementing multi-agent reinforcement learning.

cs.GT↗

arXiv2023

Feasibility-Guided Safety-Aware Model Predictive Control for Jump Markov Linear Systems

In this paper, we present a controller framework that synthesizes control policies for Jump Markov Linear Systems subject to stochastic mode switches and imperfect mode estimation. Our approach builds on safe and robust methods for Model Predictive Control (MPC), but in contrast to existing approaches that either optimize without regard to feasibility or utilize soft constraints that increase computational requirements, we employ a safe and robust control approach informed by the feasibility of the optimization problem. We formulate and encode finite horizon safety for multiple model systems in our MPC design using Control Barrier Functions (CBFs). When subject to inaccurate hybrid state estimation, our feasibility-guided MPC generates a control policy that is maximally robust to uncertainty in the system's modes. We evaluate our approach on an orbital rendezvous problem and a six degree-of-freedom hexacopter under several scenarios and benchmarks to demonstrate the utility of the framework. Results indicate that the proposed technique of maximizing the robustness horizon, and the use of CBFs for safety awareness, improve the overall safety and performance of MPC for Jump Markov Linear Systems.

eess.SY↗

arXiv2023

Recursively-Constrained Partially Observable Markov Decision Processes

Many sequential decision problems involve optimizing one objective function while imposing constraints on other objectives. Constrained Partially Observable Markov Decision Processes (C-POMDP) model this case with transition uncertainty and partial observability. In this work, we first show that C-POMDPs violate the optimal substructure property over successive decision steps and thus may exhibit behaviors that are undesirable for some (e.g., safety critical) applications. Additionally, online re-planning in C-POMDPs is often ineffective due to the inconsistency resulting from this violation. To address these drawbacks, we introduce the Recursively-Constrained POMDP (RC-POMDP), which imposes additional history-dependent cost constraints on the C-POMDP. We show that, unlike C-POMDPs, RC-POMDPs always have deterministic optimal policies and that optimal policies obey Bellman's principle of optimality. We also present a point-based dynamic programming algorithm for RC-POMDPs. Evaluations on benchmark problems demonstrate the efficacy of our algorithm and show that policies for RC-POMDPs produce more desirable behaviors than policies for C-POMDPs.

cs.AI↗