arXiv subjects

Max Heitmann

Publications and source records attributed to Max Heitmann.

2 recordsLinked to original sources

arXiv2024

Diffeomorphism Invariance and General Relativity

Diffeomorphism invariance is often considered to be a hallmark of the theory of general relativity (GR). But closer analysis reveals that this cannot be what makes GR distinctive. The concept of diffeomorphism invariance can be defined in two ways: under the first definition (diff-invariance$_1$), both GR and all other classical spacetime theories turn out to be diffeomorphism invariant, while under the second (diff-invariance$_2$), neither do. Confusion about the matter can be traced to two sources. First, GR is sometimes erroneously thought to embody a "general principle of relativity," which asserts the relativity of all states of motion, and from which it would follow that GR must be diff-invariant$_2$. But GR embodies no such principle, and is easily seen to violate diff-invariance$_2$. Second, GR is unique among spacetime theories in requiring a general-covariant formulation, whereas other classical spacetime theories are typically formulated with respect to a preferred class of global coordinate systems in which their dynamical equations simplify. This makes GR's diffeomorphism invariance (in the sense of diff-invariance$_1$) manifest, while in other spacetime theories it lies latent -- at least in their familiar formulations. I trace this difference back to the fact that the spacetime structure is inhomogeneous within the models of GR, and mutable across its models. I offer a formal criterion for when a spacetime theory possesses immutable spacetime structure, and using this criterion I prove that a theory possesses a preferred class of coordinate systems applicable across its models if and only if it possesses immutable spacetime structure.

physics.gen-ph↗

arXiv2023

On The Expressivity of Objective-Specification Formalisms in Reinforcement Learning

Most algorithms in reinforcement learning (RL) require that the objective is formalised with a Markovian reward function. However, it is well-known that certain tasks cannot be expressed by means of an objective in the Markov rewards formalism, motivating the study of alternative objective-specification formalisms in RL such as Linear Temporal Logic and Multi-Objective Reinforcement Learning. To date, there has not yet been any thorough analysis of how these formalisms relate to each other in terms of their expressivity. We fill this gap in the existing literature by providing a comprehensive comparison of 17 salient objective-specification formalisms. We place these formalisms in a preorder based on their expressive power, and present this preorder as a Hasse diagram. We find a variety of limitations for the different formalisms, and argue that no formalism is both dominantly expressive and straightforward to optimise with current techniques. For example, we prove that each of Regularised RL, (Outer) Nonlinear Markov Rewards, Reward Machines, Linear Temporal Logic, and Limit Average Rewards can express a task that the others cannot. The significance of our results is twofold. First, we identify important expressivity limitations to consider when specifying objectives for policy optimization. Second, our results highlight the need for future research which adapts reward learning to work with a greater variety of formalisms, since many existing reward learning methods assume that the desired objective takes a Markovian form. Our work contributes towards a more cohesive understanding of the costs and benefits of different RL objective-specification formalisms.

cs.LG↗