arXiv subjects

Dylan Hirsch

Publications and source records attributed to Dylan Hirsch.

11 recordsLinked to original sources

arXiv2026

Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability

Hamilton-Jacobi reachability (HJR) is an important framework in safe control theory. HJR provides theoretical tools and numerical methods to obtain value-functions for tasks involving target-reaching and obstacle-avoidance. A recent work proposed an algebraic framework to scale to complex, composite tasks via decomposing the value functions of the composite tasks into value functions for the fundamental tasks traditionally studied in HJR, all in the one-player setting. Many of these decomposition results, however, do not directly translate to the two-player setting. In this technical note, we nevertheless show that one of the previous work's key results for analyzing various tasks involving reaching multiple targets while obeying constraints still holds in the two-player setting, so long as a critical monotonicity assumption is satisfied. In particular, we detail the analogous result and its proof (which structurally differs from the one-player case), we show via a counter-example what can go wrong if this monotonicity assumption is not satisfied, and we show how the analysis of a variety of tasks can be performed using this result. We note that, whereas the prior work considered discrete-time, infinite-horizon tasks, which are more standard in reinforcement learning, we here consider continuous-time, finite-horizon tasks, which are more standard in HJR.

eess.SY↗

arXiv2026

Extending and Unifying the Fundamental Tasks of Hamilton-Jacobi Reachability Analysis

In this work, we introduce the generalized reach-avoid (GRA) task, which both extends and unifies the canonical tasks of Hamilton-Jacobi Reachability (HJR). We show that the GRA not only serves as a common primitive in this class of fundamental tasks, but also strictly extends the fundamental tasks that can be solved with HJR. Moreover, the GRA formulation enables one to compute the value functions of certain composite tasks, including ones from timed temporal logic, by decomposing the value function of the composite task into value functions of GRA tasks. We additionally show that the GRA is also a natural primitive to consider from a PDE perspective, as it can be used to represent all sufficiently regular solutions of the HJ-PDE that is canonical to HJR. Collectively, the results in this work show the theoretical and practical utility of this task within the increasingly important framework of HJR.

eess.SY↗

arXiv2026

An Update to the Level Set Theorems in Hamilton-Jacobi Reachability Analysis

Hamilton-Jacobi Reachability (HJR) is an important framework for controlling safety-critical systems despite uncertainty. Its theoretical underpinnings are rooted in Hamilton-Jacobi Partial Differential Equations, which provide the value function used for controller synthesis. The Level Set Theorems of HJR allow one to interpret the value function in terms of satisfaction of a qualitative goal (e.g. goal-reaching or obstacle-avoidance). We here provide a technical update regarding additional criteria needed for these theorems to hold.

eess.SY↗

arXiv2026

Exact Decomposition of Adversarial Dual-Objective Value Functions, with Applications to Optimal Drug Dosing

Hamilton-Jacobi Reachability (HJR) is a central framework in safe control theory. While HJR has traditionally focused on a few fundamental tasks, there is increasing interest in scaling to more complex objectives. Recent works have studied the exact decomposition of the value functions for two fundamental dual-objective tasks in the adversary-free setting. However, not all value function decompositions in HJR remain valid with an adversary. In this work, we develop theoretical approaches to certify that for these two composite value functions, the proposed decompositions still hold with an adversary. Finally, we show how these results can solve issues that arise when applying HJR to optimal drug regimen design.

eess.SY↗

arXiv2026

Bellman Value Decomposition for Task Logic in Safe Optimal Control

Real-world tasks involve nuanced combinations of goal and safety specifications. In high dimensions, the challenge is exacerbated: formal automata become cumbersome, and the combination of sparse rewards tends to require laborious tuning. In this work, we consider the innate structure of the Bellman Value as a means to naturally organize the problem for improved automatic performance. Namely, we prove the Bellman Value for a complex task defined in temporal logic can be decomposed into a graph of Bellman Values, connected by a set of well-known Bellman equations (BEs): the Reach-Avoid BE, the Avoid BE, and a novel type, the Reach-Avoid-Loop BE. To solve the Value and optimal policy, we propose VDPPO, which embeds the decomposed Value graph into a two-layer neural net, bootstrapping the implicit dependencies. We conduct a variety of simulated and hardware experiments to test our method on complex, high-dimensional tasks involving heterogeneous teams and nonlinear dynamics. Ultimately, we find this approach greatly improves performance over existing baselines, balancing safety and liveness automatically.

cs.RO↗

arXiv2026

Safe Stochastic Explorer: Enabling Safe Goal Driven Exploration in Stochastic Environments and Safe Interaction with Unknown Objects

Autonomous robots operating in unstructured, safety-critical environments, from planetary exploration to warehouses and homes, must learn to safely navigate and interact with their surroundings despite limited prior knowledge. Current methods for safe control, such as Hamilton-Jacobi Reachability and Control Barrier Functions, assume known system dynamics. Meanwhile existing safe exploration techniques often fail to account for the unavoidable stochasticity inherent when operating in unknown real world environments, such as an exploratory rover skidding over an unseen surface or a household robot pushing around unmapped objects in a pantry. To address this critical gap, we propose Safe Stochastic Explorer (S.S.Explorer) a novel framework for safe, goal-driven exploration under stochastic dynamics. Our approach strategically balances safety and information gathering to reduce uncertainty about safety in the unknown environment. We employ Gaussian Processes to learn the unknown safety function online, leveraging their predictive uncertainty to guide information-gathering actions and provide probabilistic bounds on safety violations. We first present our method for discrete state space environments and then introduce a scalable relaxation to effectively extend this approach to continuous state spaces. Finally we demonstrate how this framework can be naturally applied to ensure safe physical interaction with multiple unknown objects. Extensive validation in simulation and demonstrative hardware experiments showcase the efficacy of our method, representing a step forward toward enabling reliable widespread robot autonomy in complex, uncertain environments.

cs.RO↗

arXiv2026

Learning to Nudge: A Scalable Barrier Function Framework for Safe Robot Interaction in Dense Clutter

Robots operating in everyday environments must navigate and manipulate within densely cluttered spaces, where physical contact with surrounding objects is unavoidable. Traditional safety frameworks treat contact as unsafe, restricting robots to collision avoidance and limiting their ability to function in dense, everyday settings. As the number of objects grows, model-based approaches for safe manipulation become computationally intractable; meanwhile, learned methods typically tie safety to the task at hand, making them hard to transfer to new tasks without retraining. In this work we introduce Dense Contact Barrier Functions(DCBF). Our approach bypasses the computational complexity of explicitly modeling multi-object dynamics by instead learning a composable, object-centric function that implicitly captures the safety constraints arising from physical interactions. Trained offline on interactions with a few objects, the learned DCBFcomposes across arbitrary object sets at runtime, producing a single global safety filter that scales linearly and transfers across tasks without retraining. We validate our approach through simulated experiments in dense clutter, demonstrating its ability to enable collision-free navigation and safe, contact-rich interaction in suitable settings.

cs.RO↗

arXiv2025

Viscosity CBFs: Bridging the Control Barrier Function and Hamilton-Jacobi Reachability Frameworks in Safe Control Theory

Control barrier functions (CBFs) and Hamilton-Jacobi reachability (HJR) are central frameworks in safe control. Traditionally, these frameworks have been viewed as distinct, with the former focusing on optimally safe controller design and the latter providing sufficient conditions for safety. A previous work introduced the notion of a control barrier value function (CB-VF), which is defined similarly to the other value functions studied in HJR but has certain CBF-like properties. In this work, we proceed the other direction by generalizing CBFs to non-differentiable ``viscosity'' CBFs. We show the deep connection between viscosity CBFs and CB-VFs, bridging the CBF and HJR frameworks. Through this bridge, we characterize the viscosity CBFs as precisely those functions which provide CBF-like safety guarantees (control invariance and smooth approach to the boundary). We then further show nice theoretical properties of viscosity CBFs, including their desirable closure under maximum and limit operations. In the process, we also extend CB-VFs to non-exponential anti-discounting and update the corresponding theory for CB-VFs along these lines.

eess.SY↗

arXiv2025

Dual-Objective Reinforcement Learning with Novel Hamilton-Jacobi-Bellman Formulations

Hard constraints in reinforcement learning (RL) often degrade policy performance. Lagrangian methods offer a way to blend objectives with constraints, but require intricate reward engineering and parameter tuning. In this work, we extend recent advances that connect Hamilton-Jacobi (HJ) equations with RL to propose two novel value functions for dual-objective satisfaction. Namely, we address: 1) the Reach-Always-Avoid (RAA) problem -- of achieving distinct reward and penalty thresholds -- and 2) the Reach-Reach (RR) problem -- of achieving thresholds of two distinct rewards. In contrast with temporal logic approaches, which typically involve representing an automaton, we derive explicit, tractable Bellman forms in this context via decomposition. Specifically, we prove that the RAA and RR problems may be rewritten as compositions of previously studied HJ-RL problems. We leverage our analysis to propose a variation of Proximal Policy Optimization (DOHJ-PPO), and demonstrate that it produces distinct behaviors from previous approaches, outcompeting a number of baselines in success, safety and speed across a range of tasks for safe-arrival and multi-target achievement.

cs.AI↗

arXiv2025

Approximate Hamilton-Jacobi Reachability Analysis for a Class of Two-Timescale Systems, with Application to Biological Models

Hamilton-Jacobi reachability (HJR) is an exciting framework used for control of safety-critical systems with nonlinear and possibly uncertain dynamics. However, HJR suffers from the curse of dimensionality, with computation times growing exponentially in the dimension of the system state. Many autonomous and controlled systems involve dynamics that evolve on multiple timescales, and for these systems, singular perturbation methods can be used for model reduction. However, such methods are more challenging to apply in HJR due to the presence of an underlying differential game. In this work, we leverage prior work on singularly perturbed differential games to identify a class of systems which can be readily reduced, and we relate these results to the quantities of interest in HJR. We demonstrate the utility of our results on two examples involving biological systems, where dynamics fitting the identified class are frequently encountered.

eess.SY↗

arXiv2025

Back to Base: Towards Hands-Off Learning via Safe Resets with Reach-Avoid Safety Filters

Designing controllers that accomplish tasks while guaranteeing safety constraints remains a significant challenge. We often want an agent to perform well in a nominal task, such as environment exploration, while ensuring it can avoid unsafe states and return to a desired target by a specific time. In particular we are motivated by the setting of safe, efficient, hands-off training for reinforcement learning in the real world. By enabling a robot to safely and autonomously reset to a desired region (e.g., charging stations) without human intervention, we can enhance efficiency and facilitate training. Safety filters, such as those based on control barrier functions, decouple safety from nominal control objectives and rigorously guarantee safety. Despite their success, constructing these functions for general nonlinear systems with control constraints and system uncertainties remains an open problem. This paper introduces a safety filter obtained from the value function associated with the reach-avoid problem. The proposed safety filter minimally modifies the nominal controller while avoiding unsafe regions and guiding the system back to the desired target set. By preserving policy performance while allowing safe resetting, we enable efficient hands-off reinforcement learning and advance the feasibility of safe training for real world robots. We demonstrate our approach using a modified version of soft actor-critic to safely train a swing-up task on a modified cartpole stabilization problem.

eess.SY↗