arXiv subjects
Publications and source records attributed to Daniel Graves.
Dilute Temperley--Lieb algebras are variants of Temperley--Lieb algebras arising in statistical mechanics in the study of solvable lattice models. In this paper we prove that the (co)homology of dilute Temperley--Lieb algebras vanishes in all positive degrees.
This paper studies the (co)homology of rook-Brauer algebras and their subalgebras. Our main results focus on the cohomology of rook-Brauer algebras (which is related to the cohomology of symmetric groups), the cohomology of Motzkin algebras (for which we obtain a vanishing result in positive degrees) and the cohomology of walled Brauer algebras (which is related to the cohomology of products of symmetric groups). Along the way we collect some cohomological analogues of known results for Temperley--Lieb algebras, Brauer algebras and rook algebras.
Coloured partition algebras were introduced by Bloss and exhibit a Schur-Weyl duality with certain complex reflection groups. In this paper we show that these algebras exhibit homological stability by demonstrating that their homology groups are stably isomorphic to the homology groups of a wreath product, generalizing work of Boyd--Hepworth--Patzt and Boyde for the usual partition algebras.
In this paper we study the (co)homology of Tanabe algebras, which are a family of subalgebras of the partition algebras exhibiting a Schur-Weyl duality with certain complex reflection groups. The homology of the partition algebras has been shown to be related to the homology of the symmetric groups by Boyd-Hepworth-Patzt and the results they obtain depend on a parameter. In all known results, the homology of a diagram algebra is dependent on one of two things: the invertibility of a parameter in the ground ring or the parity of the positive integer indexing the number of pairs of vertices. We show that the (co)homology of Tanabe algebras is isomorphic to the (co)homology of the symmetric groups and that this is independent of both the parameter and the parity of the index. To the best of our knowledge, this is the first example of a result of this sort. Along the way we will also study the (co)homology of the uniform block permutation algebra and the totally propagating partition algebra as well collecting cohomological analogues of known results for the homology of partition algebras and Jones annular algebras.
Reflexive homology and dihedral homology are the homology theories associated to the reflexive and dihedral crossed simplicial groups respectively. The former has recently been shown to capture interesting information about $C_2$-equivariant homotopy theory and its structure is related to the study of "real" objects in algebraic topology. The latter has long been of interest for its applications in $O(2)$-equivariant homotopy theory and connections to Hermitian algebraic $K$-theory. In this paper, we show that the reflexive and dihedral homology theories can be interpreted as functor homology over categories of non-commutative sets, after the fashion of Pirashvili and Richter's result for the Hochschild and cyclic homology theories.
Rook-Brauer algebras are a family of diagram algebras. They contain many interesting subalgebras: rook algebras, Brauer algebras, Motzkin algebras, Temperley-Lieb algebras and symmetric group algebras. In this paper, we generalize the rook-Brauer algebras and their subalgebras by allowing more structured diagrams. We introduce equivariance by labelling edges of a diagram with elements of a group $G$. We introduce braiding by insisting that when two strands cross, they do so as either an under-crossing or an over-crossing. We also introduce equivariant, braided diagrams by combining these structures. We then study the homology of our diagram algebras, as pioneered by Boyd and Hepworth, using methods introduced by Boyde. We show that, given certain invertible parameters, we can identify the homology of our generalized diagram algebras with the group homology of the braid groups $B_n$ and the semi-direct products $G^n\rtimes \Sigma_n$ and $G^n\rtimes B_n$. This allows us to deduce homological stability results for our generalized diagram algebras. We also prove that for diagrams with an odd number of edges, the homology of equivariant Brauer algebras and equivariant Temperley-Lieb algebras can be identified with the group homology of $G^n\rtimes \Sigma_n$ and $G^n$ respectively, without any conditions on parameters.
Equivariant monoids are very important objects in many branches of mathematics: they combine the notion of multiplication and the concept of a group action. In this paper we will construct categories which encode the structure borne by monoids with a group action by combining the theory of PROPs and PROBs with the theory of crossed simplicial groups. PROPs and PROBs are categories used to encode structures borne by objects in symmetric and braided monoidal categories respectively, whilst crossed simplicial groups are categories which encode a unital, associative multiplication and a compatible group action. We will produce PROPs and PROBs whose categories of algebras are equivalent to the categories of monoids, comonoids and bimonoids with group action using extensions of the symmetric and braid crossed simplicial groups. We will extend this theory to balanced braided monoidal categories using the ribbon braid crossed simplicial group. Finally, we will use the hyperoctahedral crossed simplicial group to encode the structure of an involutive monoid with a compatible group action.
Reflexive homology is the homology theory associated to the reflexive crossed simplicial group; one of the fundamental crossed simplicial groups. It is the most general way to extend Hochschild homology to detect an order-reversing involution. In this paper we study the relationship between reflexive homology and the $C_2$-equivariant homology of free loop spaces. We define reflexive homology in terms of functor homology. We give a bicomplex for computing reflexive homology together with some calculations, including the reflexive homology of a tensor algebra. We prove that the reflexive homology of a group algebra is isomorphic to the homology of the $C_2$-equivariant Borel construction on the free loop space of the classifying space. We give a direct sum decomposition of the reflexive homology of a group algebra indexed by conjugacy classes of group elements, where the summands are defined in terms of a reflexive analogue of group homology. We define a hyperhomology version of reflexive homology and use it to study the $C_2$-equivariant homology of certain free loop and free loop-suspension spaces. We show that reflexive homology satisfies Morita invariance. We prove that under nice conditions the involutive Hochschild homology studied by Braun and by Fern\`andez-Val\`encia and Giansiracusa coincides with reflexive homology.
Hyperoctahedral homology for involutive algebras is the homology theory associated to the hyperoctahedral crossed simplicial group. It is related to equivariant stable homotopy theory via the homology of equivariant infinite loop spaces. In this paper we show that there is an E-infinity algebra structure on the simplicial module that computes hyperoctahedral homology. We deduce that hyperoctahedral homology admits Dyer-Lashof homology operations. Furthermore, there is a Pontryagin product which gives hyperoctahedral homology the structure of an associative, graded-commutative algebra. We also give an explicit description of hyperoctahedral homology in degree zero. Combining this description and the Pontryagin product we show that hyperoctahedral homology fails to preserve Morita equivalence.
A PROB is a "product and braid" category. Such categories can be used to encode the structure borne by an object in a braided monoidal category. In this paper we provide PROBs whose categories of algebras in a braided monoidal category are equivalent to the categories of monoids and comonoids using the category associated to the braid crossed simplicial group of Fiedorowicz and Loday. We show that PROBs can be composed by generalizing the machinery introduced by Lack for PROPs. We use this to define a PROB for bimonoids in a braided monoidal category as a composite of the PROBs for monoids and comonoids.
We propose a multi-time-scale predictive representation learning method to efficiently learn robust driving policies in an offline manner that generalize well to novel road geometries, and damaged and distracting lane conditions which are not covered in the offline training data. We show that our proposed representation learning method can be applied easily in an offline (batch) reinforcement learning setting demonstrating the ability to generalize well and efficiently under novel conditions compared to standard batch RL methods. Our proposed method utilizes training data collected entirely offline in the real-world which removes the need of intensive online explorations that impede applying deep reinforcement learning on real-world robot training. Various experiments were conducted in both simulator and real-world scenarios for the purpose of evaluation and analysis of our proposed claims.
Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games. A cornerstone of this success is the auto-curriculum framework, which shapes the learning process by continually creating new challenging tasks for agents to adapt to, thereby facilitating the acquisition of new skills. In order to extend MARL methods to real-world domains outside of video games, we envision in this blue sky paper that maintaining a diversity-aware auto-curriculum is critical for successful MARL applications. Specifically, we argue that \emph{behavioural diversity} is a pivotal, yet under-explored, component for real-world multiagent learning systems, and that significant work remains in understanding how to design a diversity-aware auto-curriculum. We list four open challenges for auto-curriculum techniques, which we believe deserve more attention from this community. Towards validating our vision, we recommend modelling realistic interactive behaviours in autonomous driving as an important test bed, and recommend the SMARTS/ULTRA benchmark.
We propose a framework for transferring any existing policy from a potentially unknown source MDP to a target MDP. This framework (1) enables reuse in the target domain of any form of source policy, including classical controllers, heuristic policies, or deep neural network-based policies, (2) attains optimality under suitable theoretical conditions, and (3) guarantees improvement over the source policy in the target MDP. These are achieved by packaging the source policy as a black-box option in the target MDP and providing a theoretically grounded way to learn the option's initiation set through general value functions. Our approach facilitates the learning of new policies by (1) maximizing the target MDP reward with the help of the black-box option, and (2) returning the agent to states in the learned initiation set of the black-box option where it is already optimal. We show that these two variants are equivalent in performance under some conditions. Through a series of experiments in simulated environments, we demonstrate that our framework performs excellently in sparse reward problems given (sub-)optimal source policies and improves upon prior art in transfer methods such as continual learning and progressive networks, which lack our framework's desirable theoretical properties.
We consider real-world reinforcement learning (RL) of robotic manipulation tasks that involve both visuomotor skills and contact-rich skills. We aim to train a policy that maps multimodal sensory observations (vision and force) to a manipulator's joint velocities under practical considerations. We propose to use offline samples to learn a set of general value functions (GVFs) that make counterfactual predictions from the visual inputs. We show that combining the offline learned counterfactual predictions with force feedbacks in online policy learning allows efficient reinforcement learning given only a terminal (success/failure) reward. We argue that the learned counterfactual predictions form a compact and informative representation that enables sample efficiency and provides auxiliary reward signals that guide online explorations towards contact-rich states. Various experiments in simulation and real-world settings were performed for evaluation. Recordings of the real-world robot training can be found via https://sites.google.com/view/realrl.
Hyperoctahedral homology is the homology theory associated to the hyperoctahedral crossed simplicial group. It is defined for involutive algebras over a commutative ring using functor homology and the hyperoctahedral bar construction of Fiedorowicz. The main result of the paper proves that hyperoctahedral homology is related to equivariant stable homotopy theory: for a discrete group of odd order, the hyperoctahedral homology of the group algebra is isomorphic to the homology of the fixed points under the involution of an equivariant infinite loop space built from the classifying space of the group.
General value functions (GVFs) in the reinforcement learning (RL) literature are long-term predictive summaries of the outcomes of agents following specific policies in the environment. Affordances as perceived action possibilities with specific valence may be cast into predicted policy-relative goodness and modelled as GVFs. A systematic explication of this connection shows that GVFs and especially their deep learning embodiments (1) realize affordance prediction as a form of direct perception, (2) illuminate the fundamental connection between action and perception in affordance, and (3) offer a scalable way to learn affordances using RL methods. Through an extensive review of existing literature on GVF applications and representative affordance research in robotics, we demonstrate that GVFs provide the right framework for learning affordances in real-world applications. In addition, we highlight a few new avenues of research opened up by the perspective of "affordance as GVF", including using GVFs for orchestrating complex behaviors.
We study Policy-extended Value Function Approximator (PeVFA) in Reinforcement Learning (RL), which extends conventional value function approximator (VFA) to take as input not only the state (and action) but also an explicit policy representation. Such an extension enables PeVFA to preserve values of multiple policies at the same time and brings an appealing characteristic, i.e., \emph{value generalization among policies}. We formally analyze the value generalization under Generalized Policy Iteration (GPI). From theoretical and empirical lens, we show that generalized value estimates offered by PeVFA may have lower initial approximation error to true values of successive policies, which is expected to improve consecutive value approximation during GPI. Based on above clues, we introduce a new form of GPI with PeVFA which leverages the value generalization along policy improvement path. Moreover, we propose a representation learning framework for RL policy, providing several approaches to learn effective policy embeddings from policy network parameters or state-action pairs. In our experiments, we evaluate the efficacy of value generalization offered by PeVFA and policy representation learning in several OpenAI Gym continuous control tasks. For a representative instance of algorithm implementation, Proximal Policy Optimization (PPO) re-implemented under the paradigm of GPI with PeVFA achieves about 40\% performance improvement on its vanilla counterpart in most environments.
Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse and competent driving interactions. To meet this need, we develop a dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training School). SMARTS supports the training, accumulation, and use of diverse behavior models of road users. These are in turn used to create increasingly more realistic and diverse interactions that enable deeper and broader research on multi-agent interaction. In this paper, we describe the design goals of SMARTS, explain its basic architecture and its key features, and illustrate its use through concrete multi-agent experiments on interactive scenarios. We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving. Our code is available at https://github.com/huawei-noah/SMARTS.