arXiv subjects

Kalle G. Timperi

Publications and source records attributed to Kalle G. Timperi.

8 recordsLinked to original sources

arXiv2026

Relating Reinforcement Learning to Dynamic Programming-Based Planning

This paper bridges some of the gap between optimal planning and reinforcement learning (RL), both of which share roots in dynamic programming applied to sequential decision making or optimal control. Whereas planning typically favors deterministic models, goal termination, and cost minimization, RL tends to favor stochastic models, infinite-horizon discounting, and reward maximization in addition to learning-related parameters such as the learning rate and greediness factor. A derandomized version of RL is developed, analyzed, and implemented to yield performance comparisons with value iteration and Dijkstra's algorithm using simple planning models. Next, mathematical analysis shows: 1) conditions under which cost minimization and reward maximization are equivalent, 2) conditions for equivalence of single-shot goal termination and infinite-horizon episodic learning, and 3) conditions under which discounting causes goal achievement to fail. The paper then advocates for defining and optimizing truecost, rather than inserting arbitrary parameters to guide operations. Performance studies are then extended to the stochastic case, using planning-oriented criteria and comparing value iteration to RL with learning rates and greediness factors.

cs.RO↗

arXiv2025

$\varepsilon$-neighbourhoods in the Plane with a Nowhere-smooth Boundary

We give an example of a planar set $E\subset \mathbb{R}^2$ for which the boundary $\partial E_\varepsilon$ of its $\varepsilon$-neighbourhood $E_\varepsilon = \{x \in \mathbb{R}^2 \, : \, \textrm{dist}(x, E) \leq \varepsilon \}$ is nowhere $C^1$-smooth, in the sense that the set of singularities on the boundary is countably dense (where we note that the latter set cannot be uncountable). Furthermore, we give an example of a planar set $E$ for which $\partial E_\varepsilon$ has the same properties as above, but in addition contains an uncountable subset, with non-integer Hausdorff dimension, where curvature is not defined. Both constructions make use of a characterisation of those star-shaped sets that are an $\varepsilon$-neighbourhood of one of their subsets.

math.MG↗

arXiv2025

On Boundaries of $\varepsilon$-neighbourhoods of Planar Sets: Singularities, Global Structure, and Curvature

We study the geometry, topological properties and smoothness of the boundaries of closed $\varepsilon$-neighbourhoods $E_\varepsilon = \{x \in \mathbb{R}^2 \, : \, \textrm{dist}(x, E) \leq \varepsilon \}$ of compact planar sets $E \subset \mathbb{R}^2$. We develop a novel technique for analysing the boundary, and use this to obtain a classification of singularities (i.e.~non-smooth points) on $\partial E_\varepsilon$ into eight categories. We show that the set of singularities is either countable or the disjoint union of a countable set and a closed, totally disconnected, nowhere dense set. Furthermore, we characterise, in terms of local geometry, those $\varepsilon$-neighbourhoods whose complement $\overline{\mathbb{R}^2 \setminus E_\varepsilon}$ is a set with positive reach. It is known that for all bounded $E \subset \mathbb{R}^d$ and all $\varepsilon > 0$, the boundary $\partial E_\varepsilon$ is $(d-1)$-rectifiable. Improving on this, we identify a sufficient condition for the boundary to be uniformly rectifiable, and provide an example of a planar $\varepsilon$-neighbourhood that is not Ahlfors regular. In terms of the topological structure, we show that for a compact set $E$ and $\varepsilon > 0$ the boundary $\partial E_\varepsilon$ can be expressed as a disjoint union of an at most countably infinite union of Jordan curves and a possibly uncountable, totally disconnected set of singularities. Finally, we show that curvature is defined almost everywhere on the Jordan curve subsets of the boundary.

math.MG↗

arXiv2025

Minimally sufficient structures for information-feedback policies

In this paper, we consider robotic tasks which require a desirable outcome to be achieved in the physical world that the robot is embedded in and interacting with. Accomplishing this objective requires designing a filter that maintains a useful representation of the physical world and a policy over the filter states. A filter is seen as the robot's perspective of the physical world based on limited sensing, memory, and computation and it is represented as a transition system over a space of information states. To this end, the interactions result from the coupling of an internal and an external system, a filter, and the physical world, respectively, through a sensor mapping and an information-feedback policy. Within this setup, we look for sufficient structures, that is, sufficient internal systems and sensors, for accomplishing a given task. We establish necessary and sufficient conditions for these structures to satisfy for information-feedback policies that can be defined over the states of an internal system to exist. We also show that under mild assumptions, minimal internal systems that can represent a particular plan/policy described over the action-observation histories exist and are unique. Finally, the results are applied to determine sufficient structures for distance-optimal navigation in a polygonal environment.

cs.RO↗

arXiv2024

Universal Plans: One Action Sequence to Solve Them All!

This paper introduces the notion of a universal plan, which when executed, is guaranteed to solve all planning problems in a category, regardless of the obstacles, initial state, and goal set. Such plans are specified as a deterministic sequence of actions that are blindly applied without any sensor feedback. Thus, they can be considered as pure exploration in a reinforcement learning context, and we show that with basic memory requirements, they even yield optimal plans. Building upon results in number theory and theory of automata, we provide universal plans both for discrete and continuous (motion) planning and prove their (semi)completeness. The concepts are applied and illustrated through simulation studies, and several directions for future research are sketched.

cs.RO↗

arXiv2024

An Internal Model Principle For Robots

When designing a robot's internal system, one often makes assumptions about the structure of the intended environment of the robot. One may even assign meaning to various internal components of the robot in terms of expected environmental correlates. In this paper we want to make the distinction between robot's internal and external worlds clear-cut. Can the robot learn about its environment, relying only on internally available information, including the sensor data? Are there mathematical conditions on the internal robot system which can be internally verified and make the robot's internal system mirror the structure of the environment? We prove that sufficiency is such a mathematical principle, and mathematically describe the emergence of the robot's internal structure isomorphic or bisimulation equivalent to that of the environment. A connection to the free-energy principle is established, when sufficiency is interpreted as a limit case of surprise minimization. As such, we show that surprise minimization leads to having an internal model isomorphic to the environment. This also parallels the Good Regulator Principle which states that controlling a system sufficiently well means having a model of it. Unlike the mentioned theories, ours is discrete, and non-probabilistic.

cs.RO↗

arXiv2024

From Virtual Reality to the Emerging Discipline of Perception Engineering

This paper makes the case that a powerful new discipline, which we term perception engineering, is steadily emerging. It follows from a progression of ideas that involve creating illusions, from historical paintings and film, to video games and virtual reality in modern times. Rather than creating physical artifacts such as bridges, airplanes, or computers, perception engineers create illusory perceptual experiences. The scope is defined over any agent that interacts with the physical world, including both biological organisms (humans, animals) and engineered systems (robots, autonomous systems). The key idea is that an agent, called a producer, alters the environment with the intent to alter the perceptual experience of another agent, called a receiver. Most importantly, the paper introduces a precise mathematical formulation of this process, based on the von Neumann-Morgenstern notion of information, to help scope and define the discipline. It is then applied to the cases of engineered and biological agents with discussion of its implications on existing fields such as virtual reality, robotics, and even social media. Finally, open challenges and opportunities for involvement are identified.

cs.HC↗

arXiv2023

A Mathematical Characterization of Minimally Sufficient Robot Brains

This paper addresses the lower limits of encoding and processing the information acquired through interactions between an internal system (robot algorithms or software) and an external system (robot body and its environment) in terms of action and observation histories. Both are modeled as transition systems. We want to know the weakest internal system that is sufficient for achieving passive (filtering) and active (planning) tasks. We introduce the notion of an information transition system for the internal system which is a transition system over a space of information states that reflect a robot's or other observer's perspective based on limited sensing, memory, computation, and actuation. An information transition system is viewed as a filter and a policy or plan is viewed as a function that labels the states of this information transition system. Regardless of whether internal systems are obtained by learning algorithms, planning algorithms, or human insight, we want to know the limits of feasibility for given robot hardware and tasks. We establish, in a general setting, that minimal information transition systems exist up to reasonable equivalence assumptions, and are unique under some general conditions. We then apply the theory to generate new insights into several problems, including optimal sensor fusion/filtering, solving basic planning tasks, and finding minimal representations for modeling a system given input-output relations.

cs.RO↗