arXiv subjects

Tom Bewley

Publications and source records attributed to Tom Bewley.

20 records · Page 2Linked to original sources

arXiv2020

Am I Building a White Box Agent or Interpreting a Black Box Agent?

The rule extraction literature contains the notion of a fidelity-accuracy dilemma: when building an interpretable model of a black box function, optimising for fidelity is likely to reduce performance on the underlying task, and vice versa. I reassert the relevance of this dilemma for the modern field of explainable artificial intelligence, and highlight how it is compounded when the black box is an agent interacting with a dynamic environment. I then discuss two independent research directions - building white box agents and interpreting black box agents - which are both coherent and worthy of attention, but must not be conflated by researchers embarking on projects in the domain of agent interpretability.

cs.AI↗

arXiv2020

Modelling Agent Policies with Interpretable Imitation Learning

As we deploy autonomous agents in safety-critical domains, it becomes important to develop an understanding of their internal mechanisms and representations. We outline an approach to imitation learning for reverse-engineering black box agent policies in MDP environments, yielding simplified, interpretable models in the form of decision trees. As part of this process, we explicitly model and learn agents' latent state representations by selecting from a large space of candidate features constructed from the Markov state. We present initial promising results from an implementation in a multi-agent traffic environment.

cs.AI↗