arXiv subjects

Karl Van Wyk

Publications and source records attributed to Karl Van Wyk.

32 records · Page 2Linked to original sources

arXiv2021

Generalized Nonlinear and Finsler Geometry for Robotics

Robotics research has found numerous important applications of Riemannian geometry. Despite that, the concept remain challenging to many roboticists because the background material is complex and strikingly foreign. Beyond {\em Riemannian} geometry, there are many natural generalizations in the mathematical literature -- areas such as Finsler geometry and spray geometry -- but those generalizations are largely inaccessible, and as a result there remain few applications within robotics. This paper presents a re-derivation of spray and Finsler geometries we found critical for the development of our recent work on a powerful behavioral design tool we call geometric fabrics. These derivations build from basic tools in advanced calculus and the calculus of variations making them more accessible to a robotics audience than standard presentations. We focus on the pragmatic and calculable results, avoiding the use of tensor notation to appeal to a broader audience, emphasizing geometric path consistency over ideas around connections and curvature. We hope that these derivations will contribute to an increased understanding of generalized nonlinear, and even classical Riemannian, geometry within the robotics community and inspire future research into new applications.

cs.RO↗

arXiv2021

Geometric Fabrics for the Acceleration-based Design of Robotic Motion

This paper describes the pragmatic design and construction of geometric fabrics for shaping a robot's task-independent nominal behavior, capturing behavioral components such as obstacle avoidance, joint limit avoidance, redundancy resolution, global navigation heuristics, etc. Geometric fabrics constitute the most concrete incarnation of a new mathematical formulation for reactive behavior called optimization fabrics. Fabrics generalize recent work on Riemannian Motion Policies (RMPs); they add provable stability guarantees and improve design consistency while promoting the intuitive acceleration-based principles of modular design that make RMPs successful. We describe a suite of mathematical modeling tools that practitioners can employ in practice and demonstrate both how to mitigate system complexity by constructing behaviors layer-wise and how to employ these tools to design robust, strongly-generalizing, policies that solve practical problems one would expect to find in industry applications. Our system exhibits intelligent global navigation behaviors expressed entirely as provably stable fabrics with zero planning or state machine governance.

cs.RO↗

arXiv2021

Optimization Fabrics for Behavioral Design

A common approach to the provably stable design of reactive behavior, exemplified by operational space control, is to reduce the problem to the design of virtual classical mechanical systems (energy shaping). This framework is widely used, and through it we gain stability, but at the price of expressivity. This work presents a comprehensive theoretical framework expanding this approach showing that there is a much larger class of differential equations generalizing classical mechanical systems (and the broader class of Lagrangian systems) and greatly expanding their expressivity while maintaining the same governing stability principles. At the core of our framework is a class of differential equations we call fabrics which constitute a behavioral medium across which we can optimize a potential function. These fabrics shape the system's behavior during optimization but still always provably converge to a local minimum, making them a building block of stable behavioral design. We build the theoretical foundations of our framework here and provide a simple empirical demonstration of a practical class of geometric fabrics, which additionally exhibit a natural geometric path consistency making them convenient for flexible and intuitive behavioral design.

cs.RO↗

arXiv2021

Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories

Imitation learning (IL) is a frequently used approach for data-efficient policy learning. Many IL methods, such as Dataset Aggregation (DAgger), combat challenges like distributional shift by interacting with oracular experts. Unfortunately, assuming access to oracular experts is often unrealistic in practice; data used in IL frequently comes from offline processes such as lead-through or teleoperation. In this paper, we present a novel imitation learning technique called Collocation for Demonstration Encoding (CoDE) that operates on only a fixed set of trajectory demonstrations. We circumvent challenges with methods like back-propagation-through-time by introducing an auxiliary trajectory network, which takes inspiration from collocation techniques in optimal control. Our method generalizes well and more accurately reproduces the demonstrated behavior with fewer guiding trajectories when compared to standard behavioral cloning methods. We present simulation results on a 7-degree-of-freedom (DoF) robotic manipulator that learns to exhibit lifting, target-reaching, and obstacle avoidance behaviors.

cs.RO↗

arXiv2021

DexYCB: A Benchmark for Capturing Hand Grasping of Objects

We introduce DexYCB, a new dataset for capturing hand grasping of objects. We first compare DexYCB with a related one through cross-dataset evaluation. We then present a thorough benchmark of state-of-the-art approaches on three relevant tasks: 2D object and keypoint detection, 6D object pose estimation, and 3D hand pose estimation. Finally, we evaluate a new robotics-relevant task: generating safe robot grasps in human-to-robot object handover. Dataset and code are available at https://dex-ycb.github.io.

cs.CV↗

arXiv2021

RMP2: A Structured Composable Policy Class for Robot Learning

We consider the problem of learning motion policies for acceleration-based robotics systems with a structured policy class specified by RMPflow. RMPflow is a multi-task control framework that has been successfully applied in many robotics problems. Using RMPflow as a structured policy class in learning has several benefits, such as sufficient expressiveness, the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots. However, implementing a system for end-to-end learning RMPflow policies faces several computational challenges. In this work, we re-examine the message passing algorithm of RMPflow and propose a more efficient alternate algorithm, called RMP2, that uses modern automatic differentiation tools (such as TensorFlow and PyTorch) to compute RMPflow policies. Our new design retains the strengths of RMPflow while bringing in advantages from automatic differentiation, including 1) easy programming interfaces to designing complex transformations; 2) support of general directed acyclic graph (DAG) transformation structures; 3) end-to-end differentiability for policy learning; 4) improved computational efficiency. Because of these features, RMP2 can be treated as a structured policy class for efficient robot learning which is suitable encoding domain knowledge. Our experiments show that using structured policy class given by RMP2 can improve policy performance and safety in reinforcement learning tasks for goal reaching in cluttered space.

cs.RO↗

arXiv2021

Interpreting and Predicting Tactile Signals for the SynTouch BioTac

In the human hand, high-density contact information provided by afferent neurons is essential for many human grasping and manipulation capabilities. In contrast, robotic tactile sensors, including the state-of-the-art SynTouch BioTac, are typically used to provide low-density contact information, such as contact location, center of pressure, and net force. Although useful, these data do not convey or leverage the rich information content that some tactile sensors naturally measure. This research extends robotic tactile sensing beyond reduced-order models through 1) the automated creation of a precise experimental tactile dataset for the BioTac over a diverse range of physical interactions, 2) a 3D finite element (FE) model of the BioTac, which complements the experimental dataset with high-density, distributed contact data, 3) neural-network-based mappings from raw BioTac signals to not only low-dimensional experimental data, but also high-density FE deformation fields, and 4) mappings from the FE deformation fields to the raw signals themselves. The high-density data streams can provide a far greater quantity of interpretable information for grasping and manipulation algorithms than previously accessible.

cs.RO↗

arXiv2020

In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation

Tracking the pose of an object while it is being held and manipulated by a robot hand is difficult for vision-based methods due to significant occlusions. Prior works have explored using contact feedback and particle filters to localize in-hand objects. However, they have mostly focused on the static grasp setting and not when the object is in motion, as doing so requires modeling of complex contact dynamics. In this work, we propose using GPU-accelerated parallel robot simulations and derivative-free, sample-based optimizers to track in-hand object poses with contact feedback during manipulation. We use physics simulation as the forward model for robot-object interactions, and the algorithm jointly optimizes for the state and the parameters of the simulations, so they better match with those of the real world. Our method runs in real-time (30Hz) on a single GPU, and it achieves an average point cloud distance error of 6mm in simulation experiments and 13mm in the real-world ones. View experiment videos at https://sites.google.com/view/in-hand-object-pose-tracking/

cs.RO↗

arXiv2020

Optimization Fabrics

This paper presents a theory of optimization fabrics, second-order differential equations that encode nominal behaviors on a space and can be used to define the behavior of a smooth optimizer. Optimization fabrics can encode commonalities among optimization problems that reflect the structure of the space itself, enabling smooth optimization processes to intelligently navigate each problem even when optimizing simple naive potential functions. Importantly, optimization over a fabric is inherently asymptotically stable. The majority of this paper is dedicated to the development of a tool set for the design and use of a broad class of fabrics called geometric fabrics. Geometric fabrics encode behavior as general nonlinear geometries which are covariant second-order differential equations with a special homogeneity property that ensures their behavior is independent of the system's speed through the medium. A class of Finsler Lagrangian energies can be used to both define how these nonlinear geometries combine with one another and how they react when potential functions force them from their nominal paths. Furthermore, these geometric fabrics are closed under the standard operations of pullback and combination on a transform tree. For behavior representation, this class of geometric fabrics constitutes a broad class of spectral semi-sprays (specs), also known as Riemannian Motion Policies (RMPs) in the context of robotic motion generation, that captures both the intuitive separation between acceleration policy and priority metric critical for modular design and are inherently stable. Therefore, geometric fabrics are safe and easier to use by less experienced behavioral designers. Application of this theory to policy representation and generalization in learning are discussed as well.

cs.RO↗

arXiv2020

Interpreting and Predicting Tactile Signals via a Physics-Based and Data-Driven Framework

High-density afferents in the human hand have long been regarded as essential for human grasping and manipulation abilities. In contrast, robotic tactile sensors are typically used to provide low-density contact data, such as center-of-pressure and resultant force. Although useful, this data does not exploit the rich information content that some tactile sensors (e.g., the SynTouch BioTac) naturally provide. This research extends robotic tactile sensing beyond reduced-order models through 1) the automated creation of a precise tactile dataset for the BioTac over diverse physical interactions, 2) a 3D finite element (FE) model of the BioTac, which complements the experimental dataset with high-resolution, distributed contact data, and 3) neural-network-based mappings from raw BioTac signals to low-dimensional experimental data, and more importantly, high-density FE deformation fields. These data streams can provide a far greater quantity of interpretable information for grasping and manipulation algorithms than previously accessible.

cs.RO↗

arXiv2019

DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System

Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, current teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually provide reduced degrees of control. Herein, a low-cost, vision based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enables operators to carry out a variety of complex manipulation tasks that go beyond simple pick-and-place operations. This allows for collection of high dimensional, multi-modality, state-action data that can be leveraged in the future to learn sensorimotor policies for challenging manipulation tasks. The system performance was measured through speed and reliability metrics across two human demonstrators on a variety of tasks. The videos of the experiments can be found at https://sites.google.com/view/dex-pilot.

cs.CV↗

arXiv2019

Efficiently Improving and Quantifying Robot Accuracy In Situ

The advancement of simulation-assisted robot programming, automation of high-tolerance assembly operations, and improvement of real-world performance engender a need for positionally accurate robots. Despite tight machining tolerances, good mechanical design, and careful assembly, robotic arms typically exhibit average Cartesian positioning errors of several millimeters. Fortunately, the vast majority of this error can be removed in software by proper calibration of the so-called "zero-offsets" of a robot's joints. This research developed an automated, inexpensive, highly portable, in situ calibration method that fine tunes these kinematic parameters, thereby, improving a robot's average positioning accuracy four-fold throughout its workspace. In particular, a prospective low-cost motion capture system and a benchmark laser tracker were used as reference sensors for robot calibration. Bayesian inference produced optimized zero-offset parameters alongside their uncertainty for data from both reference sensors. Relative and absolute accuracy metrics were proposed and applied for quantifying robot positioning accuracy. Uncertainty analysis of a validated, probabilistic robot model quantified the absolute positioning accuracy throughout its entire workspace. Altogether, three measures of accuracy conclusively revealed multi-fold improvement in the positioning accuracy of the robotic arm. Bayesian inference on motion capture data yielded zero-offsets and accuracy calculations comparable to those derived from laser tracker data, ultimately proving this method's viability towards robot calibration.

cs.RO↗

arXiv2018

Slip Detection: Analysis and Calibration of Univariate Tactile Signals

The existence of tactile afferents sensitive to slip-related mechanical transients in the human hand augments the robustness of grasping through secondary force modulation protocols. Despite this knowledge and the fact that tactile-based slip detection has been researched for decades, robust slip detection is still not an out-of-the-box capability for any commercially available tactile sensor. This research seeks to bridge this gap with a comprehensive study addressing several aspects of slip detection. Key developments include a systematic data collection process yielding millions of sensory data points, the generalized conversion of multivariate-to-univariate sensor output, an insightful spectral analysis of the univariate sensor outputs, and the application of Long Short-Term Memory (LSTM) neural networks on the univariate signals to produce robust slip detectors from three commercially available sensors capable of tactile sensing. The sensing elements underlying these sensors vary in quantity, spatial arrangement, and mechanics, leveraging principles in electro-mechanical resistance, optics, and hydro-acoustics. Critically, slip detection performance of the tactile technologies is quantified through a measurement methodology that unveils the effects of data window size, sampling rate, material type, slip speed, and sensor manufacturing variability. Results indicate that the investigated commercial tactile sensors are inherently capable of high-quality slip detection.

cs.RO↗

arXiv2016

Multi-Fingered Robotic Grasping: A Primer

This technical report presents an introduction to different aspects of multi-fingered robot grasping. After having introduced relevant mathematical background for modeling, form and force closure are discussed. Next, we present an overview of various grasp planning algorithms with the objective of illustrating different approaches to solve this problem. Finally, we discuss grasp performance benchmarking.

cs.RO↗