arXiv subjects

Tim Schneider

Publications and source records attributed to Tim Schneider.

At least 19 recordsLinked to original sources

arXiv2026

Optical Spectroscopy of Waveguide coupled Er$^{3+}$ ensembles in CaWO$_4$ and YVO$_4$

We present an optical study of near-surface Er$^{3+}$ ensembles in waveguide-integrated CaWO$_4$ and YVO$_4$, investigating how nanophotonic coupling modifies rare-earth spectroscopy. In particular, we compare bulk excitation with evanescently coupled TE and TM waveguide modes. In Er$^{3+}$:CaWO$_4$, we observe a pronounced polarization-dependent surface effect. TE-coupled spectra closely reproduce bulk behavior. In contrast, TM coupling induces strong inhomogeneous broadening and an asymmetric low-energy shoulder of the site S1 Y1Z1 transition, with linewidths exceeding those of the bulk by more than a factor of four. Temperature-dependent measurements and surface termination studies indicate that surface charges are the dominant mechanism. Er$^{3+}$:YVO$_4$ remains largely unaffected by mode polarization, and surface termination leads only to minor spectral shifts. These observations suggest that non-charge-neutral rare-earth systems are more susceptible to surface-induced decoherence sources than charge-neutral hosts.

cond-mat.mtrl-sci↗

arXiv2025

Zeeman Spectroscopy of Vacancy-Charge-Compensated Er3+ Sites in CaWO4 under Vector Magnetic Fields

We present polarization-resolved optical absorption measurements on Er3+ ions in CaWO4 under vector magnetic fields, focusing on charge-compensated sites arising from local Ca2+ vacancies. While the known axial Er3+ site displays a single symmetric Zeeman-split transition pattern consistent with S4 symmetry, two additional sites exhibit more complex spectral behavior, including sets of transitions that interchange under 90{\deg} crystal rotations-evidence of reduced, rhombic-like symmetry. From these polarization- and temperature-dependent spectra, we extract effective g-factors. Our findings are corroborated by electron paramagnetic resonance measurements and support a model of multiple inequivalent Ca2+ vacancies around Er3+ sites in the host lattice. This detailed characterization contributes to understanding defect-engineered rare-earth sites for quantum information applications.

cond-mat.mtrl-sci↗

arXiv2025

Tactile-Conditioned Diffusion Policy for Force-Aware Robotic Manipulation

Contact-rich manipulation depends on applying the correct grasp forces throughout the manipulation task, especially when handling fragile or deformable objects. Most existing imitation learning approaches often treat visuotactile feedback only as an additional observation, leaving applied forces as an uncontrolled consequence of gripper commands. In this work, we present Force-Aware Robotic Manipulation (FARM), an imitation learning framework that integrates high-dimensional tactile data to infer tactile-conditioned force signals, which in turn define a matching force-based action space. We collect human demonstrations using a modified version of the handheld Universal Manipulation Interface (UMI) gripper that integrates a GelSight Mini visual tactile sensor. For deploying the learned policies, we developed an actuated variant of the UMI gripper with geometry matching our handheld version. During policy rollouts, the proposed FARM diffusion policy jointly predicts robot pose, grip width, and grip force. FARM outperforms several baselines across three tasks with distinct force requirements -- high-force, low-force, and dynamic force adaptation -- demonstrating the advantages of its two key components: leveraging force-grounded, high-dimensional tactile observations and a force-based control space. The codebase and design files are open-sourced and available at https://tactile-farm.github.io .

cs.RO↗

arXiv2025

Tactile MNIST: Benchmarking Active Tactile Perception

Tactile perception has the potential to significantly enhance dexterous robotic manipulation by providing rich local information that can complement or substitute for other sensory modalities such as vision. However, because tactile sensing is inherently local, it is not well-suited for tasks that require broad spatial awareness or global scene understanding on its own. A human-inspired strategy to address this issue is to consider active perception techniques instead. That is, to actively guide sensors toward regions with more informative or significant features and integrate such information over time in order to understand a scene or complete a task. Both active perception and different methods for tactile sensing have received significant attention recently. Yet, despite advancements, both fields lack standardized benchmarks. To bridge this gap, we introduce the Tactile MNIST Benchmark Suite, an open-source, Gymnasium-compatible benchmark specifically designed for active tactile perception tasks, including localization, classification, and volume estimation. Our benchmark suite offers diverse simulation scenarios, from simple toy environments all the way to complex tactile perception tasks using vision-based tactile sensors. Furthermore, we also offer a comprehensive dataset comprising 13,500 synthetic 3D MNIST digit models and 153,600 real-world tactile samples collected from 600 3D printed digits. Using this dataset, we train a CycleGAN for realistic tactile simulation rendering. By providing standardized protocols and reproducible evaluation frameworks, our benchmark suite facilitates systematic progress in the fields of tactile sensing and active perception.

cs.RO↗

arXiv2025

Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors

One of the most important object properties that humans and robots perceive through touch is hardness. This paper investigates information-theoretic active sampling strategies for sample-efficient hardness classification with vision-based tactile sensors. We evaluate three probabilistic classifier models and two model-uncertainty-based sampling strategies on a robotic setup as well as on a previously published dataset of samples collected by human testers. Our findings indicate that the active sampling approaches, driven by uncertainty metrics, surpass a random sampling baseline in terms of accuracy and stability. Additionally, while in our human study, the participants achieve an average accuracy of 48.00%, our best approach achieves an average accuracy of 88.78% on the same set of objects, demonstrating the effectiveness of vision-based tactile sensors for object hardness classification.

cs.RO↗

arXiv2025

Towards Safe Robot Foundation Models Using Inductive Biases

Safety is a critical requirement for the real-world deployment of robotic systems. Unfortunately, while current robot foundation models show promising generalization capabilities across a wide variety of tasks, they fail to address safety, an important aspect for ensuring long-term operation. Current robot foundation models assume that safe behavior should emerge by learning from a sufficiently large dataset of demonstrations. However, this approach has two clear major drawbacks. Firstly, there are no formal safety guarantees for a behavior cloning policy trained using supervised learning. Secondly, without explicit knowledge of any safety constraints, the policy may require an unreasonable number of additional demonstrations to even approximate the desired constrained behavior. To solve these key issues, we show how we can instead combine robot foundation models with geometric inductive biases using ATACOM, a safety layer placed after the foundation policy that ensures safe state transitions by enforcing action constraints. With this approach, we can ensure formal safety guarantees for generalist policies without providing extensive demonstrations of safe behavior, and without requiring any specific fine-tuning for safety. Our experiments show that our approach can be beneficial both for classical manipulation tasks, where we avoid unwanted collisions with irrelevant objects, and for dynamic tasks, such as the robot air hockey environment, where we can generate fast trajectories respecting complex tasks and joint space constraints.

cs.RO↗

arXiv2025

Apple: Toward General Active Perception via Reinforcement Learning

Active perception is a fundamental skill that enables us humans to deal with uncertainty in our inherently partially observable environment. For senses such as touch, where the information is sparse and local, active perception becomes crucial. In recent years, active perception has emerged as an important research domain in robotics. However, current methods are often bound to specific tasks or make strong assumptions, which limit their generality. To address this gap, this work introduces APPLE (Active Perception Policy Learning) - a novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective, learning how to actively gather information. By design, APPLE is not limited to a specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate two variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark. Experiments demonstrate the efficacy of APPLE, achieving high accuracies on both regression and classification tasks. These findings underscore the potential of APPLE as a versatile and general framework for advancing active perception in robotics. Project page: https://timschneider42.github.io/apple

cs.RO↗

arXiv2025

On the Importance of Tactile Sensing for Imitation Learning: A Case Study on Robotic Match Lighting

The field of robotic manipulation has advanced significantly in recent years. At the sensing level, several novel tactile sensors have been developed, capable of providing accurate contact information. On a methodological level, learning from demonstrations has proven an efficient paradigm to obtain performant robotic manipulation policies. The combination of both holds the promise to extract crucial contact-related information from the demonstration data and actively exploit it during policy rollouts. However, this integration has so far been underexplored, most notably in dynamic, contact-rich manipulation tasks where precision and reactivity are essential. This work therefore proposes a multimodal, visuotactile imitation learning framework that integrates a modular transformer architecture with a flow-based generative model, enabling efficient learning of fast and dexterous manipulation policies. We evaluate our framework on the dynamic, contact-rich task of robotic match lighting - a task in which tactile feedback influences human manipulation performance. The experimental results highlight the effectiveness of our approach and show that adding tactile information improves policy performance, thereby underlining their combined potential for learning dynamic manipulation from few demonstrations. Project website: https://sites.google.com/view/tactile-il .

cs.RO↗

arXiv2025

ESPARGOS: An Ultra Low-Cost, Realtime-Capable Multi-Antenna WiFi Channel Sounder

Multi-antenna channel sounding is a technique for measuring the propagation characteristics of electromagnetic waves that is commonly employed for parameterizing channel models. Channel sounders are usually custom-built from many Software Defined Radio receivers, making them expensive to procure and difficult to operate, which constrains the set of users to a few specialized scientific institutions and industrial research laboratories. Recent developments in Joint Communications and Sensing (JCaS) extend the possible uses of channel data to applications like human activity recognition, human presence detection, user localization and wireless Channel Charting, all of which are of great interest to security researchers, experts in industrial automation and others. However, due to a lack of affordable, easy-to-use and commercially available multi-antenna channel sounders, those scientific communities can be hindered by their lack of access to wireless channel measurements. To lower the barrier to entry for channel sounding, we develop an ultra low-cost measurement hardware platform based on mass-produced WiFi chips, which is easily affordable to research groups and even hobbyists.

eess.SP↗

arXiv2024

Spectroscopic Investigations of Multiple Environments in Er:CaWO4 through Charge Imbalance

We present a detailed spectroscopic study of the $^4\mathrm{I}_{13/2}$ and $^4\mathrm{I}_{15/2}$ Er$^{3+}$ multiplets of Er:CaWO$_4$ grown without a co-dopant. Using photoluminescence and photoluminescence excitation measurements, we find multiple environments into which the Erbium ions are incorporated in the crystal. For the four most prevalent environments, we provide a complete assignment of the $^4\mathrm{I}_{13/2}$ and $^4\mathrm{I}_{15/2}$ sublevel energies. Polarization-dependent absorption and emission spectroscopy allow us to identify the irreducible representations of the states and determine the dipole nature of transitions. In addition, we compare the different relaxation paths within $^4\mathrm{I}_{13/2}$ and $^4\mathrm{I}_{15/2}$ multiplets via line broadening and lifetime measurements. Finally, spectral, lifetime, and polarization measurements indicate similarities between the studied environments.

cond-mat.mes-hall↗

arXiv2024

TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators

Training robot policies in simulation is becoming increasingly popular; nevertheless, a precise, reliable, and easy-to-use tactile simulator for contact-rich manipulation tasks is still missing. To close this gap, we develop TacEx -- a modular tactile simulation framework. We embed a state-of-the-art soft-body simulator for contacts named GIPC and vision-based tactile simulators Taxim and FOTS into Isaac Sim to achieve robust and plausible simulation of the visuotactile sensor GelSight Mini. We implement several Isaac Lab environments for Reinforcement Learning (RL) leveraging our TacEx simulation, including object pushing, lifting, and pole balancing. We validate that the simulation is stable and that the high-dimensional observations, such as the gel deformation and the RGB images from the GelSight camera, can be used for training. The code, videos, and additional results will be released online https://sites.google.com/view/tacex.

cs.RO↗

arXiv2024

Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks

Robotic insertion tasks remain challenging due to uncertainties in perception and the need for precise control, particularly in unstructured environments. While humans seamlessly combine vision and touch for such tasks, effectively integrating these modalities in robotic systems is still an open problem. Our work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve. These findings provide valuable insights for designing more effective multi-modal robotic control systems and highlight the critical role of tactile feedback in contact-rich manipulation tasks.

cs.RO↗

arXiv2024

Learning Tactile Insertion in the Real World

Humans have exceptional tactile sensing capabilities, which they can leverage to solve challenging, partially observable tasks that cannot be solved from visual observation alone. Research in tactile sensing attempts to unlock this new input modality for robots. Lately, these sensors have become cheaper and, thus, widely available. At the same time, the question of how to integrate them into control loops is still an active area of research, with central challenges being partial observability and the contact-rich nature of manipulation tasks. In this study, we propose to use Reinforcement Learning to learn an end-to-end policy, mapping directly from tactile sensor readings to actions. Specifically, we use Dreamer-v3 on a challenging, partially observable robotic insertion task with a Franka Research 3, both in simulation and on a real system. For the real setup, we built a robotic platform capable of resetting itself fully autonomously, allowing for extensive training runs without human supervision. Our preliminary results indicate that Dreamer is capable of utilizing tactile inputs to solve robotic manipulation tasks in simulation and reality. Furthermore, we find that providing the robot with tactile feedback generally improves task performance, though, in our setup, we do not yet include other sensing modalities. In the future, we plan to utilize our platform to evaluate a wide range of other Reinforcement Learning algorithms on tactile tasks.

cs.RO↗

arXiv2024

Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation

Telerobotics enables humans to overcome spatial constraints and physically interact with the environment in remote locations. However, the sensory feedback provided by the system to the user is often purely visual, limiting the user's dexterity in manipulation tasks. This work addresses this issue by equipping the robot's end-effector with high-resolution visuotactile GelSight sensors. Using low-cost MANUS-Gloves, we provide the user with haptic feedback about forces acting at the points of contact in the form of vibration signals. We employ two different methods for estimating these forces; one based on estimating the movement of markers on the sensor surface and one deep-learning approach. Additionally, we integrate our system into a virtual-reality teleoperation pipeline in which a human user controls both arms of a Tiago robot while receiving visual and haptic feedback. Lastly, we present a novel setup to evaluate normal force, shear force, and slip. We believe that integrating haptic feedback is a crucial step towards dexterous manipulation in teleoperated robotic systems.

cs.RO↗

arXiv2024

What Matters for Active Texture Recognition With Vision-Based Tactile Sensors

This paper explores active sensing strategies that employ vision-based tactile sensors for robotic perception and classification of fabric textures. We formalize the active sampling problem in the context of tactile fabric recognition and provide an implementation of information-theoretic exploration strategies based on minimizing predictive entropy and variance of probabilistic models. Through ablation studies and human experiments, we investigate which components are crucial for quick and reliable texture recognition. Along with the active sampling strategies, we evaluate neural network architectures, representations of uncertainty, influence of data augmentation, and dataset variability. By evaluating our method on a previously published Active Clothing Perception Dataset and on a real robotic system, we establish that the choice of the active exploration strategy has only a minor influence on the recognition accuracy, whereas data augmentation and dropout rate play a significantly larger role. In a comparison study, while humans achieve 66.9% recognition accuracy, our best approach reaches 90.0% in under 5 touches, highlighting that vision-based tactile sensors are highly effective for fabric texture recognition.

cs.RO↗

arXiv2023

Probabilistic Regular Tree Priors for Scientific Symbolic Reasoning

Symbolic Regression (SR) allows for the discovery of scientific equations from data. To limit the large search space of possible equations, prior knowledge has been expressed in terms of formal grammars that characterize subsets of arbitrary strings. However, there is a mismatch between context-free grammars required to express the set of syntactically correct equations, missing closure properties of the former, and a tree structure of the latter. Our contributions are to (i) compactly express experts' prior beliefs about which equations are more likely to be expected by probabilistic Regular Tree Expressions (pRTE), and (ii) adapt Bayesian inference to make such priors efficiently available for symbolic regression encoded as finite state machines. Our scientific case studies show its effectiveness in soil science to find sorption isotherms and for modeling hyper-elastic materials.

cs.LG↗

arXiv2022

Active Exploration for Robotic Manipulation

Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method, on a challenging ball pushing task on tilted tables, where the target ball position is not known to the agent a-priori. Our real-world robot experiment serves as a fundamental application of active exploration in model-based reinforcement learning of complex robotic manipulation tasks.

cs.RO↗

arXiv2022

Active Inference for Robotic Manipulation

Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in the last decades. One of the central challenges of manipulation is partial observability, as the agent usually does not know all physical properties of the environment and the objects it is manipulating in advance. A recently emerging theory that deals with partial observability in an explicit manner is Active Inference. It does so by driving the agent to act in a way that is not only goal-directed but also informative about the environment. In this work, we apply Active Inference to a hard-to-explore simulated robotic manipulation tasks, in which the agent has to balance a ball into a target zone. Since the reward of this task is sparse, in order to explore this environment, the agent has to learn to balance the ball without any extrinsic feedback, purely driven by its own curiosity. We show that the information-seeking behavior induced by Active Inference allows the agent to explore these challenging, sparse environments systematically. Finally, we conclude that using an information-seeking objective is beneficial in sparse environments and allows the agent to solve tasks in which methods that do not exhibit directed exploration fail.

cs.RO↗