arXiv subjects

Ayan Mukhopadhyay

Publications and source records attributed to Ayan Mukhopadhyay.

At least 37 records · Page 2Linked to original sources

arXiv2025

ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs

In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making under uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creating a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty structures such as high-dimensional, multi-modal belief distributions, resulting in estimation errors that lead to suboptimal agent behaviors. To address this challenge, we present ESCORT (Efficient Stein-variational and sliced Consistency-Optimized Representation for Temporal beliefs), a particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures. This approach retains SVGD's attractive-repulsive particle dynamics while enabling accurate modeling of intricate correlation patterns. Unlike particle filters prone to degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling or restrictive distributional assumptions. We demonstrate ESCORT's effectiveness through extensive evaluations on both POMDP domains and synthetic multi-modal distributions of varying dimensionality, where it consistently outperforms state-of-the-art methods in terms of belief approximation accuracy and downstream decision quality.

cs.LG↗

arXiv2025

Combining LLMs with Logic-Based Framework to Explain MCTS

In response to the lack of trust in Artificial Intelligence (AI) for sequential planning, we design a Computational Tree Logic-guided large language model (LLM)-based natural language explanation framework designed for the Monte Carlo Tree Search (MCTS) algorithm. MCTS is often considered challenging to interpret due to the complexity of its search trees, but our framework is flexible enough to handle a wide range of free-form post-hoc queries and knowledge-based inquiries centered around MCTS and the Markov Decision Process (MDP) of the application domain. By transforming user queries into logic and variable statements, our framework ensures that the evidence obtained from the search tree remains factually consistent with the underlying environmental dynamics and any constraints in the actual stochastic control process. We evaluate the framework rigorously through quantitative assessments, where it demonstrates strong performance in terms of accuracy and factual consistency.

cs.AI↗

arXiv2025

Observation Adaptation via Annealed Importance Resampling for Partially Observable Markov Decision Processes

Partially observable Markov decision processes (POMDPs) are a general mathematical model for sequential decision-making in stochastic environments under state uncertainty. POMDPs are often solved \textit{online}, which enables the algorithm to adapt to new information in real time. Online solvers typically use bootstrap particle filters based on importance resampling for updating the belief distribution. Since directly sampling from the ideal state distribution given the latest observation and previous state is infeasible, particle filters approximate the posterior belief distribution by propagating states and adjusting weights through prediction and resampling steps. However, in practice, the importance resampling technique often leads to particle degeneracy and sample impoverishment when the state transition model poorly aligns with the posterior belief distribution, especially when the received observation is highly informative. We propose an approach that constructs a sequence of bridge distributions between the state-transition and optimal distributions through iterative Monte Carlo steps, better accommodating noisy observations in online POMDP solvers. Our algorithm demonstrates significantly superior performance compared to state-of-the-art methods when evaluated across multiple challenging POMDP domains.

cs.AI↗

arXiv2025

Generalized Clausius inequalities and entanglement production in holographic two-dimensional CFTs

Utilizing quantum information theory, it has been shown that irreversible entropy production is bounded from both below and above in physical processes. Both these bounds are positive and generalize the Clausius inequality. Such bounds are, however, obtained from distance measures in the space of states, which are hard to define and compute in quantum field theories. We show that the quantum null energy condition (QNEC) can be utilized to obtain both lower and upper bounds on irreversible entropy production for quenches leading to transitions between thermal states carrying uniform momentum density in two dimensional holographic conformal field theories. We achieve this by refining earlier methods and developing an algebraic procedure for determining HRT surfaces in arbitrary Bañados-Vaidya geometries which are dual to quenches involving transitions between general quantum equilibrium states (e.g. thermal states) where the QNEC is saturated. We also discuss results for the growth and thermalization of entanglement entropy for arbitrary initial and final temperatures and momentum densities. The rate of quadratic growth of entanglement just after the quench depends only on the change in the energy density and is independent of the entangling length. For sufficiently large entangling lengths, the entanglement tsunami phenomenon can be established. Finally, we study recovery of the initial state from the evolving entanglement entropy and argue that the Renyi entropies should give us a refined understanding of scrambling of quantum information.

hep-th↗

arXiv2025

Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstraction

Sequential decision-making in high-dimensional continuous action spaces, particularly in stochastic environments, faces significant computational challenges. We explore this challenge in the traditional offline RL setting, where an agent must learn how to make decisions based on data collected through a stochastic behavior policy. We present Latent Macro Action Planner (L-MAP), which addresses this challenge by learning a set of temporally extended macro-actions through a state-conditional Vector Quantized Variational Autoencoder (VQ-VAE), effectively reducing action dimensionality. L-MAP employs a (separate) learned prior model that acts as a latent transition model and allows efficient sampling of plausible actions. During planning, our approach accounts for stochasticity in both the environment and the behavior policy by using Monte Carlo tree search (MCTS). In offline RL settings, including stochastic continuous control tasks, L-MAP efficiently searches over discrete latent actions to yield high expected returns. Empirical results demonstrate that L-MAP maintains low decision latency despite increased action dimensionality. Notably, across tasks ranging from continuous control with inherently stochastic dynamics to high-dimensional robotic hand manipulation, L-MAP significantly outperforms existing model-based methods and performs on-par with strong model-free actor-critic baselines, highlighting the effectiveness of the proposed approach in planning in complex and stochastic environments with high-dimensional action spaces.

cs.LG↗

arXiv2025

Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards

Strategic aggregation of electric vehicle batteries as energy reservoirs can optimize power grid demand, benefiting smart and connected communities, especially large office buildings that offer workplace charging. This involves optimizing charging and discharging to reduce peak energy costs and net peak demand, monitored over extended periods (e.g., a month), which involves making sequential decisions under uncertainty and delayed and sparse rewards, a continuous action space, and the complexity of ensuring generalization across diverse conditions. Existing algorithmic approaches, e.g., heuristic-based strategies, fall short in addressing real-time decision-making under dynamic conditions, and traditional reinforcement learning (RL) models struggle with large state-action spaces, multi-agent settings, and the need for long-term reward optimization. To address these challenges, we introduce a novel RL framework that combines the Deep Deterministic Policy Gradient approach (DDPG) with action masking and efficient MILP-driven policy guidance. Our approach balances the exploration of continuous action spaces to meet user charging demands. Using real-world data from a major electric vehicle manufacturer, we show that our approach comprehensively outperforms many well-established baselines and several scalable heuristic approaches, achieving significant cost savings while meeting all charging requirements. Our results show that the proposed approach is one of the first scalable and general approaches to solving the V2B energy management challenge.

cs.LG↗

arXiv2025

NS-Gym: Open-Source Simulation Environments and Benchmarks for Non-Stationary Markov Decision Processes

In many real-world applications, agents must make sequential decisions in environments where conditions are subject to change due to various exogenous factors. These non-stationary environments pose significant challenges to traditional decision-making models, which typically assume stationary dynamics. Non-stationary Markov decision processes (NS-MDPs) offer a framework to model and solve decision problems under such changing conditions. However, the lack of standardized benchmarks and simulation tools has hindered systematic evaluation and advance in this field. We present NS-Gym, the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework. In NS-Gym, we segregate the evolution of the environmental parameters that characterize non-stationarity from the agent's decision-making module, allowing for modular and flexible adaptations to dynamic environments. We review prior work in this domain and present a toolkit encapsulating key problem characteristics and types in NS-MDPs. This toolkit is the first effort to develop a set of standardized interfaces and benchmark problems to enable consistent and reproducible evaluation of algorithms under non-stationary conditions. We also benchmark six algorithmic approaches from prior work on NS-MDPs using NS-Gym. Our vision is that NS-Gym will enable researchers to assess the adaptability and robustness of their decision-making algorithms to non-stationary conditions.

cs.AI↗

arXiv2025

Learning holographic horizons

We apply machine learning to understand fundamental aspects of holographic duality, specifically the entropies obtained from the apparent and event horizon areas. We show that simple features of only the time series of the pressure anisotropy, namely the values and half-widths of the maxima and minima, the times these are attained, and the times of the first zeroes can predict the areas of the apparent and event horizons in the dual bulk geometry at all times with a fixed maximum length ($10$) of the input vector. We also argue that the entropy functions are the measures of information that need to be extracted from simple one-point functions to reconstruct specific aspects of correlation functions of the dual state with the best possible approximations.

hep-th↗

arXiv2024

An effective framework for strange metallic transport

Semi-holography, originally proposed as a model for conducting lattice electrons coupled to a holographic critical sector, leads to an effective theory of non-Fermi liquids with only a few relevant interactions on the Fermi surface in the large $N$ limit. A refined version of such theories has only two effective couplings which give holographic and Fermi-liquid-like contributions to the self-energy, respectively. We show that a low co-dimension sub-manifold exists in the space of refined semi-holographic theories in which strange metallic behavior is manifested, and which can be obtained just by tuning the ratio of the two couplings. On this sub-manifold, the product of the spectral function and the temperature is approximately independent of the critical exponent, the Fermi energy, and the temperature at all frequencies and near the Fermi surface when expressed in terms of suitably scaled momentum and frequency variables. This quasi-universal behavior leads to linear-in-$T$ dc resistivity and Planckian dissipation over a large range of temperatures, and we also obtain $T^{-3}$ scaling of the Hall conductivity at higher temperatures. The quasi-universal spectral function also fits well with photoemission spectroscopic data without varying the critical exponent with the doping. Combining with the results for optical conductivity, we construct a generalized version of Drude phenomenology for strange-metallic behavior which satisfies non-trivial consistency tests. Finally, we discuss a possible dynamical mechanism for the fine-tuning of the ratio of the two couplings necessary to realize the strange metallic behavior in a typical state.

hep-th↗

arXiv2024

Shrinking POMCP: A Framework for Real-Time UAV Search and Rescue

Efficient path optimization for drones in search and rescue operations faces challenges, including limited visibility, time constraints, and complex information gathering in urban environments. We present a comprehensive approach to optimize UAV-based search and rescue operations in neighborhood areas, utilizing both a 3D AirSim-ROS2 simulator and a 2D simulator. The path planning problem is formulated as a partially observable Markov decision process (POMDP), and we propose a novel ``Shrinking POMCP'' approach to address time constraints. In the AirSim environment, we integrate our approach with a probabilistic world model for belief maintenance and a neurosymbolic navigator for obstacle avoidance. The 2D simulator employs surrogate ROS2 nodes with equivalent functionality. We compare trajectories generated by different approaches in the 2D simulator and evaluate performance across various belief types in the 3D AirSim-ROS simulator. Experimental results from both simulators demonstrate that our proposed shrinking POMCP solution achieves significant improvements in search times compared to alternative methods, showcasing its potential for enhancing the efficiency of UAV-assisted search and rescue operations.

cs.RO↗

arXiv2024

Enabling MCTS Explainability for Sequential Planning Through Computation Tree Logic

Monte Carlo tree search (MCTS) is one of the most capable online search algorithms for sequential planning tasks, with significant applications in areas such as resource allocation and transit planning. Despite its strong performance in real-world deployment, the inherent complexity of MCTS makes it challenging to understand for users without technical background. This paper considers the use of MCTS in transportation routing services, where the algorithm is integrated to develop optimized route plans. These plans are required to meet a range of constraints and requirements simultaneously, further complicating the task of explaining the algorithm's operation in real-world contexts. To address this critical research gap, we introduce a novel computation tree logic-based explainer for MCTS. Our framework begins by taking user-defined requirements and translating them into rigorous logic specifications through the use of language templates. Then, our explainer incorporates a logic verification and quantitative evaluation module that validates the states and actions traversed by the MCTS algorithm. The outcomes of this analysis are then rendered into human-readable descriptive text using a second set of language templates. The user satisfaction of our approach was assessed through a survey with 82 participants. The results indicated that our explanatory approach significantly outperforms other baselines in user preference.

cs.AI↗

arXiv2024

Generalized conformal quantum mechanics as an ideal observer in two-dimensional gravity

We obtain an action for a generalized conformal mechanics (GCM) coupled to Jackiw-Teitelboim (JT) gravity from a double scaling limit of the motion of a charged massive particle in the near-horizon geometry of a near-extremal spherical black hole. When JT gravity is treated in the classical approximation, the backreaction of the particle's wavefunction on the time-reparametrization mode (and therefore the bulk metric) vanishes while the conformal symmetry in GCM is reparametrized in a state-dependent way. We also construct the semi-classical Hilbert space of the full theory by explicitly solving the general time-dependent normalizable solutions of the Schrödinger equation for GCM, and show that the time-reparametrization mode can be inferred from the measurement of suitable observables. Since the full theory of the GCM coupled to JT gravity is amenable to quantization, it can lead to a solvable model for a detector coupled to quantum gravity.

hep-th↗

arXiv2024

A simple model for strange metallic behavior

A refined semi-holographic non-Fermi liquid model, in which carrier electrons hybridize with operators of a holographic critical sector, has been proposed recently for strange metallic behavior. The model, consistently with effective theory approach, has two couplings whose ratio is related to the doping. We explain the origin of the linear-in-T resistivity and strange metallic behavior as a consequence of the emergence of a universal form of the spectral function which is independent of the model parameters when the ratio of the two couplings take optimal values determined only by the critical exponent. This universal form fits well with photoemission data of copper oxide samples for under/optimal/over-doping with a fixed exponent over a wide range of temperatures. We further obtain a refined Planckian dissipation scenario in which the scattering time $τ= f \cdot \hbar /(k_B T)$, with $f$ being $\mathcal{O}(1)$ at strong coupling, but $\mathcal{O}(10)$ at weak coupling.

cond-mat.str-el↗

arXiv2024

Nambu-Goto equation from three-dimensional gravity

We demonstrate that the solutions of three-dimensional gravity obtained by gluing two copies of a spacetime across a junction constituted of a tensile string are in one-to-one correspondence with the solutions of the Nambu-Goto equation in the same spacetime up to a finite number of rigid deformations related to worldsheet and spacetime isometries. The non-linear Nambu-Goto equation satisfied by the average of the embedding coordinates of the junction emerges directly from the junction conditions along with the rigid deformations and corrections due to the tension. Therefore, the equivalence principle generalizes non-trivially to the string. Our results are valid both in three-dimensional flat and AdS spacetimes. In the context of AdS$_3$/CFT$_2$ correspondence, our setup could be used to describe a class of interfaces in the conformal field theory featuring relative time reparametrization at the interface which encodes the solution of the Nambu-Goto equation corresponding to the bulk junction.

hep-th↗

arXiv2024

Holographic Gubser flow: A combined analytic and numerical study

Gubser flow is an evolution with cylindrical and boost symmetries, which can be best studied by mapping the future wedge of Minkowski space (R$^{3,1}$) to dS$_3$ $\times$ $\mathbb{R}$ in a conformal relativistic theory. Here, we sharpen our previous analytic results and validate them via the first numerical exploration of the Gubser flow in a holographic conformal field theory. Remarkably, the leading generic behavior at large de Sitter time is free-streaming in transverse directions and the sub-leading behavior is that of a color glass condensate. We also show that Gubser flow can be smoothly glued to the vacuum outside the future Minkowski wedge generically given that the energy density vanishes faster than any power when extrapolated to early proper time or to large distances from the central axis. We find that at intermediate times the ratio of both the transverse and longitudinal pressures to the energy density converge approximately to a fixed point which is hydrodynamic only for large initial energy densities. We argue that our results suggest that the Gubser flow is better applied to collective behavior in jets rather than the full medium in the phenomenology of heavy ion collisions and can reveal new clues to the mechanism of confinement.

hep-th↗

arXiv2024

A stabilizer code model with non-invertible symmetries: Strange fractons, confinement, and non-commutative and non-Abelian fusion rules

We introduce a stabilizer code model with a qutrit at every edge on a square lattice and with non-invertible plaquette operators. The degeneracy of the ground state is topological as in the toric code, and it also has the usual deconfined excitations consisting of pairs of electric and magnetic charges. However, there are novel types of confined fractonic excitations composed of a cluster of adjacent faces with vanishing flux. They manifest confinement, and even larger configurations of these fractons are fully immobile although they acquire emergent internal degrees of freedom. Deconfined excitations change their nature in presence of these fractonic defects. As for instance, fractonic defects can absorb magnetic charges making magnetic monopoles exist while electric charges acquire restricted mobility. Furthermore, some generalized symmetries can annihilate any ground state and also the full sector of fully mobile excitations. All these properties can be captured via a novel type of \textit{non-commutative} and \textit{non-Abelian} fusion category in which the product is associative but does not commute, and can be expressed as a sum of (operator) equivalence classes. Generalized non-invertible symmetries give rise to the feature that the fusion products form a non-unital category without a proper identity. We show that a variant of this model features a deconfined fracton liquid phase and a phase where the dual (magnetic) strings have condensed.

hep-th↗

arXiv2024

Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing

An emergency responder management (ERM) system dispatches responders, such as ambulances, when it receives requests for medical aid. ERM systems can also proactively reposition responders between predesignated waiting locations to cover any gaps that arise due to the prior dispatch of responders or significant changes in the distribution of anticipated requests. Optimal repositioning is computationally challenging due to the exponential number of ways to allocate responders between locations and the uncertainty in future requests. The state-of-the-art approach in proactive repositioning is a hierarchical approach based on spatial decomposition and online Monte Carlo tree search, which may require minutes of computation for each decision in a domain where seconds can save lives. We address the issue of long decision times by introducing a novel reinforcement learning (RL) approach, based on the same hierarchical decomposition, but replacing online search with learning. To address the computational challenges posed by large, variable-dimensional, and discrete state and action spaces, we propose: (1) actor-critic based agents that incorporate transformers to handle variable-dimensional states and actions, (2) projections to fixed-dimensional observations to handle complex states, and (3) combinatorial techniques to map continuous actions to discrete allocations. We evaluate our approach using real-world data from two U.S. cities, Nashville, TN and Seattle, WA. Our experiments show that compared to the state of the art, our approach reduces computation time per decision by three orders of magnitude, while also slightly reducing average ambulance response time by 5 seconds.

cs.LG↗

arXiv2024

How Gubser flow ends in a holographic conformal theory

Gubser flow is an axis-symmetric and boost-invariant evolution in a relativistic quantum field theory which is best studied by mapping $\mathbf{R}^{3,1}$ to $dS_{3}\times \mathbf{R}$ when the field theory has conformal symmetry. We show that at late de-Sitter time, which corresponds to large proper time and central region of the future wedge within $\mathbf{R}^{3,1}$, the holographic conformal field theory plasma can reach a state in which $\varepsilon = P_T = - P_L$, with $\varepsilon$, $P_T$ and $P_L$ being the energy density, transverse and longitudinal pressures, respectively. We further determine the full sub-leading behaviour of the energy-momentum tensor at late time. Restricting to flows in which the energy density decays at large transverse distance from the central axis in $\mathbf{R}^{3,1}$, we show that this decay should be faster than any power law. Furthermore, in this case the energy density also vanishes in $\mathbf{R}^{3,1}$ faster than any power as we go back to early proper time. Hydrodynamic behavior can appear in intermediate time.

hep-th↗