arXiv subjects

Jaleh Zand

Publications and source records attributed to Jaleh Zand.

3 recordsLinked to original sources

arXiv2022

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents' strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner's strategy a priori.

cs.LG↗

arXiv2020

Mixture Density Conditional Generative Adversarial Network Models (MD-CGAN)

Generative Adversarial Networks (GANs) have gained significant attention in recent years, with impressive applications highlighted in computer vision in particular. Compared to such examples, however, there have been more limited applications of GANs to time series modelling, including forecasting. In this work, we present the Mixture Density Conditional Generative Adversarial Model (MD-CGAN), with a focus on time series forecasting. We show that our model is capable of estimating a probabilistic posterior distribution over forecasts and that, in comparison to a set of benchmark methods, the MD-CGAN model performs well, particularly in situations where noise is a significant component of the observed time series. Further, by using a Gaussian mixture model as the output distribution, MD-CGAN offers posterior predictions that are non-Gaussian.

cs.LG↗

arXiv2015

On the relevance of q-distribution functions: The return time distribution of restricted random walker

There exist a large literature on the application of $q$-statistics to the out-of-equilibrium non-ergodic systems in which some degree of strong correlations exists. Here we study the distribution of first return times to zero, $P_R(0,t)$, of a random walk on the set of integers $\{0,1,2,...,L\}$ with a position dependent transition probability given by $|n/L|^a$. We find that for all values of $a\in[0,2]$ $P_R(0,t)$ can be fitted by $q$-exponentials, but only for $a=1$ is $P_R(0,t)$ given exactly by a $q$-exponential in the limit $L\rightarrow\infty$. This is a remarkable result since the exact analytical solution of the corresponding continuum model represents $P_R(0,t)$ as a sum of Bessel functions with a smooth dependence on $a$ from which we are unable to identify $a=1$ as of special significance. However, from the high precision numerical iteration of the discrete Master Equation, we do verify that only for $a=1$ is $P_R(0,t)$ exactly a $q$-exponential and that a tiny departure from this parameter value makes the distribution deviate from $q$-exponential. Further research is certainly required to identify the reason for this result and also the applicability of $q$-statistics and its domain.

math-ph↗