arXiv subjects

Dmitry Kalashnikov

Publications and source records attributed to Dmitry Kalashnikov.

At least 37 records · Page 2Linked to original sources

arXiv2022

Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation

Despite decades of research, existing navigation systems still face real-world challenges when deployed in the wild, e.g., in cluttered home environments or in human-occupied public spaces. To address this, we present a new class of implicit control policies combining the benefits of imitation learning with the robust handling of system constraints from Model Predictive Control (MPC). Our approach, called Performer-MPC, uses a learned cost function parameterized by vision context embeddings provided by Performers -- a low-rank implicit-attention Transformer. We jointly train the cost function and construct the controller relying on it, effectively solving end-to-end the corresponding bi-level optimization problem. We show that the resulting policy improves standard MPC performance by leveraging a few expert demonstrations of the desired navigation behavior in different challenging real-world scenarios. Compared with a standard MPC policy, Performer-MPC achieves >40% better goal reached in cluttered environments and >65% better on social metrics when navigating around humans.

cs.RO↗

arXiv2022

Sgoldstino signal at FASER: prospects in searches for supersymmetry

We investigate FASER@LHC perspectives in searches for light ($0.1-5$ GeV) sgoldstinos in models with low energy ($10-10^4$ TeV) supersymmetry breaking. We consider flavor conserving and flavor violating couplings of sgoldstinos to Standard Model fermions and find the both options to be testable at FASER. Even the first FASER run allows one to probe interesting patches in the model parameter space, while the second run, FASER-II, with significantly larger detector fiducial volume, gives a possibility to thoroughly explore a wide class of supersymmetric extensions of particle physics complementary to those probed at LHC with ATLAS and CMS detectors.

hep-ph↗

arXiv2022

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances

Large language models can encode a wealth of semantic knowledge about the world. Such knowledge could be extremely useful to robots aiming to act upon high-level, temporally extended instructions expressed in natural language. However, a significant weakness of language models is that they lack real-world experience, which makes it difficult to leverage them for decision making within a given embodiment. For example, asking a language model to describe how to clean a spill might result in a reasonable narrative, but it may not be applicable to a particular agent, such as a robot, that needs to perform this task in a particular environment. We propose to provide real-world grounding by means of pretrained skills, which are used to constrain the model to propose natural language actions that are both feasible and contextually appropriate. The robot can act as the language model's "hands and eyes," while the language model supplies high-level semantic knowledge about the task. We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment. We evaluate our method on a number of real-world robotic tasks, where we show the need for real-world grounding and that this approach is capable of completing long-horizon, abstract, natural language instructions on a mobile manipulator. The project's website and the video can be found at https://say-can.github.io/.

cs.RO↗

arXiv2022

Hybrid Random Features

We propose a new class of random feature methods for linearizing softmax and Gaussian kernels called hybrid random features (HRFs) that automatically adapt the quality of kernel estimation to provide most accurate approximation in the defined regions of interest. Special instantiations of HRFs lead to well-known methods such as trigonometric (Rahimi and Recht, 2007) or (recently introduced in the context of linear-attention Transformers) positive random features (Choromanski et al., 2021). By generalizing Bochner's Theorem for softmax/Gaussian kernels and leveraging random features for compositional kernels, the HRF-mechanism provides strong theoretical guarantees - unbiased approximation and strictly smaller worst-case relative errors than its counterparts. We conduct exhaustive empirical evaluation of HRF ranging from pointwise kernel estimation experiments, through tests on data admitting clustering structure to benchmarking implicit-attention Transformers (also for downstream Robotics applications), demonstrating its quality in a wide spectrum of machine learning problems.

cs.LG↗

arXiv2021

AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scale

Robotic skills can be learned via imitation learning (IL) using user-provided demonstrations, or via reinforcement learning (RL) using large amountsof autonomously collected experience.Both methods have complementarystrengths and weaknesses: RL can reach a high level of performance, but requiresexploration, which can be very time consuming and unsafe; IL does not requireexploration, but only learns skills that are as good as the provided demonstrations.Can a single method combine the strengths of both approaches? A number ofprior methods have aimed to address this question, proposing a variety of tech-niques that integrate elements of IL and RL. However, scaling up such methodsto complex robotic skills that integrate diverse offline data and generalize mean-ingfully to real-world scenarios still presents a major challenge. In this paper, ouraim is to test the scalability of prior IL + RL algorithms and devise a system basedon detailed empirical experimentation that combines existing components in themost effective and scalable way. To that end, we present a series of experimentsaimed at understanding the implications of each design decision, so as to develop acombined approach that can utilize demonstrations and heterogeneous prior datato attain the best performance on a range of real-world and realistic simulatedrobotic problems. Our complete method, which we call AW-Opt, combines ele-ments of advantage-weighted regression [1, 2] and QT-Opt [3], providing a unifiedapproach for integrating demonstrations and offline data for robotic manipulation.Please see https://awopt.github.io for more details.

cs.RO↗

arXiv2021

Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills

We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data. In particular, we propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We employ goal-conditioned Q-learning with hindsight relabeling and develop several techniques that enable training in a particularly challenging offline setting. We find that our method can operate on high-dimensional camera images and learn a variety of skills on real robots that generalize to previously unseen scenes and objects. We also show that our method can learn to reach long-horizon goals across multiple episodes through goal chaining, and learn rich representations that can help with downstream tasks through pre-training or auxiliary objectives. The videos of our experiments can be found at https://actionable-models.github.io

cs.RO↗

arXiv2021

MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

General-purpose robotic systems must master a large repertoire of diverse skills to be useful in a range of daily tasks. While reinforcement learning provides a powerful framework for acquiring individual behaviors, the time needed to acquire each skill makes the prospect of a generalist robot trained with RL daunting. In this paper, we study how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously, sharing exploration, experience, and representations across tasks. In this framework new tasks can be continuously instantiated from previously learned tasks improving overall performance and capabilities of the system. To instantiate this system, we develop a scalable and intuitive framework for specifying new tasks through user-provided examples of desired outcomes, devise a multi-robot collective learning system for data collection that simultaneously collects experience for multiple tasks, and develop a scalable and generalizable multi-task deep reinforcement learning method, which we call MT-Opt. We demonstrate how MT-Opt can learn a wide range of skills, including semantic picking (i.e., picking an object from a particular category), placing into various fixtures (e.g., placing a food item onto a plate), covering, aligning, and rearranging. We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots, and demonstrate the performance of our system both in terms of its ability to generalize to structurally similar new tasks, and acquire distinct new tasks more quickly by leveraging past experience. We recommend viewing the videos at https://karolhausman.github.io/mt-opt/

cs.RO↗

arXiv2021

Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras

In this work, we present an effective multi-view approach to closed-loop end-to-end learning of precise manipulation tasks that are 3D in nature. Our method learns to accomplish these tasks using multiple statically placed but uncalibrated RGB camera views without building an explicit 3D representation such as a pointcloud or voxel grid. This multi-camera approach achieves superior task performance on difficult stacking and insertion tasks compared to single-view baselines. Single view robotic agents struggle from occlusion and challenges in estimating relative poses between points of interest. While full 3D scene representations (voxels or pointclouds) are obtainable from registered output of multiple depth sensors, several challenges complicate operating off such explicit 3D representations. These challenges include imperfect camera calibration, poor depth maps due to object properties such as reflective surfaces, and slower inference speeds over 3D representations compared to 2D images. Our use of static but uncalibrated cameras does not require camera-robot or camera-camera calibration making the proposed approach easy to setup and our use of \textit{sensor dropout} during training makes it resilient to the loss of camera-views after deployment.

cs.RO↗

arXiv2021

Visionary: Vision architecture discovery for robot learning

We propose a vision-based architecture search algorithm for robot manipulation learning, which discovers interactions between low dimension action inputs and high dimensional visual inputs. Our approach automatically designs architectures while training on the task - discovering novel ways of combining and attending image feature representations with actions as well as features from previous layers. The obtained new architectures demonstrate better task success rates, in some cases with a large margin, compared to a recent high performing baseline. Our real robot experiments also confirm that it improves grasping performance by 6%. This is the first approach to demonstrate a successful neural architecture search and attention connectivity search for a real-robot task.

cs.RO↗

arXiv2021

Continuous wave second harmonic generation enabled by quasi-bound-states in the continuum on gallium phosphide metasurfaces

Resonant metasurfaces are an attractive platform for enhancing the non-linear optical processes, such as second harmonic generation (SHG), since they can generate very large local electromagnetic fields while relaxing the phase-matching requirements. Here, we take this platform a step closer to the practical applications by demonstrating visible range, continuous wave (CW) SHG. We do so by combining the attractive material properties of gallium phosphide with engineered, high quality-factor photonic modes enabled by bound states in the continuum. For the optimum case, we obtain efficiencies around 5e-5 % W$^{-1}$ when the system is pumped at 1200 nm wavelength with CW intensities of 1 kW/cm$^2$. Moreover, we measure external efficiencies as high as 0.1 % W$^{-1}$ with pump intensities of only 10 MW/cm$^2$ for pulsed irradiation. This efficiency is higher than the values previously reported for dielectric metasurfaces, but achieved here with pump intensities that are two orders of magnitude lower.

physics.optics↗

arXiv2020

Disentangled Planning and Control in Vision Based Robotics via Reward Machines

In this work we augment a Deep Q-Learning agent with a Reward Machine (DQRM) to increase speed of learning vision-based policies for robot tasks, and overcome some of the limitations of DQN that prevent it from converging to good-quality policies. A reward machine (RM) is a finite state machine that decomposes a task into a discrete planning graph and equips the agent with a reward function to guide it toward task completion. The reward machine can be used for both reward shaping, and informing the policy what abstract state it is currently at. An abstract state is a high level simplification of the current state, defined in terms of task relevant features. These two supervisory signals of reward shaping and knowledge of current abstract state coming from the reward machine complement each other and can both be used to improve policy performance as demonstrated on several vision based robotic pick and place tasks. Particularly for vision based robotics applications, it is often easier to build a reward machine than to try and get a policy to learn the task without this structure.

cs.RO↗

arXiv2020

Integrated Single Photon Emitters

The realization of scalable systems for quantum information processing and networking is of utmost importance to the quantum information community. However, building such systems is difficult because of challenges in achieving all the necessary functionalities on a unified platform while maintaining stringent performance requirements of the individual elements. A promising approach which addresses this challenge is based on the consolidation of experimental and theoretical capabilities in quantum physics and integrated photonics. Integrated quantum photonics devices allow efficient control and read-out of quantum information while being scalable and cost effective. Here we review recent developments in solid-state single photon emitters coupled with various integrated photonic structures, which form a critical component of future scalable quantum devices. Our work contributes to the further development and realization of quantum networking protocols and quantum logic on a scalable and fabrication-friendly platform.

physics.optics↗

arXiv2020

Demonstration of second harmonic generationin gallium phosphide nano-waveguides

We designed, fabricated and tested gallium phosphide (GaP) nano-waveguides for second harmonic generation (SHG). We demonstrate SHG in the visible range around 655 nm using low power continuous-wave pump in the optical communication O-band. Our structures utilize modal phase matching, such that lower order eigenmodes of the pump are phase matched to higher order eigenmodes of the second harmonic. We observe phase matched SHG for different combinations of interacting modes by varying the widths of the waveguides and tuning the wavelength of the pump. The presented results contribute to the development of integrated photonic platforms with efficient nonlinear wave-mixing processes for classical and quantum applications.

physics.optics↗

arXiv2020

Thinking While Moving: Deep Reinforcement Learning with Concurrent Control

We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system, such as when a robot must decide on the next action while still performing the previous action. Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed. In order to develop an algorithmic framework for such concurrent control problems, we start with a continuous-time formulation of the Bellman equations, and then discretize them in a way that is aware of system delays. We instantiate this new class of approximate dynamic programming methods via a simple architectural extension to existing value-based deep reinforcement learning algorithms. We evaluate our methods on simulated benchmark tasks and a large-scale robotic grasping task where the robot must "think while moving".

cs.LG↗

arXiv2019

Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks

Real world data, especially in the domain of robotics, is notoriously costly to collect. One way to circumvent this can be to leverage the power of simulation to produce large amounts of labelled data. However, training models on simulated images does not readily transfer to real-world ones. Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power. In this paper, we present Randomized-to-Canonical Adaptation Networks (RCANs), a novel approach to crossing the visual reality gap that uses no real-world data. Our method learns to translate randomized rendered images into their equivalent non-randomized, canonical versions. This in turn allows for real images to also be translated into canonical sim images. We demonstrate the effectiveness of this sim-to-real approach by training a vision-based closed-loop grasping reinforcement learning agent in simulation, and then transferring it to the real world to attain 70% zero-shot grasp success on unseen objects, a result that almost doubles the success of learning the same task directly on domain randomization alone. Additionally, by joint finetuning in the real-world with only 5,000 real-world grasps, our method achieves 91%, attaining comparable performance to a state-of-the-art system trained with 580,000 real-world grasps, resulting in a reduction of real-world data by more than 99%.

cs.RO↗

arXiv2018

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execute the desired grasp, our method enables closed-loop vision-based control, whereby the robot continuously updates its grasp strategy based on the most recent observations to optimize long-horizon grasp success. To that end, we introduce QT-Opt, a scalable self-supervised vision-based reinforcement learning framework that can leverage over 580k real-world grasp attempts to train a deep neural network Q-function with over 1.2M parameters to perform closed-loop, real-world grasping that generalizes to 96% grasp success on unseen objects. Aside from attaining a very high success rate, our method exhibits behaviors that are quite distinct from more standard grasping systems: using only RGB vision-based perception from an over-the-shoulder camera, our method automatically learns regrasping strategies, probes objects to find the most effective grasps, learns to reposition objects and perform other non-prehensile pre-grasp manipulations, and responds dynamically to disturbances and perturbations.

cs.LG↗

arXiv2018

Polarization effects in the nonlinear interference of down-converted photons

We study polarization effects in the nonlinear interference of photons generated via frequency non-degenerate spontaneous parametric down conversion. Signal and idler photons generated in the visible and infrared (IR) range, are split in different arms of a nonlinear Michelson interferometer. The interference pattern for signal photons is detected, and it is shown to be dependent on the polarization rotation of idler photons, introduced by a birefringent sample. Based on this concept, we realize two new methods for measurement of sample retardation in the IR range by using well-developed and inexpensive components for visible light. The accuracy of the methods meets current industry standards. The developed IR polarimetry technique is relevant to material research, optical inspection, and quality control.

quant-ph↗

arXiv2018

Measurement of infrared optical constants with visible photons

We demonstrate a new scheme of infrared spectroscopy with visible light sources and detectors. The technique relies on the nonlinear interference of correlated photons, produced via spontaneous parametric down conversion in a nonlinear crystal. Visible and infrared photons are split into two paths and the infrared photons interact with the sample under study. The photons are reflected back to the crystal, resembling a conventional Michelson interferometer. Interference of the visible photons is observed and it is dependent on the phases of all three interacting photons: pump, visible and infrared. The transmission coefficient and the refractive index of the sample in the infrared range can be inferred from the interference pattern of visible photons. The method does not require the use of potentially expensive and inefficient infrared detectors and sources, it can be applied to a broad variety of samples, and it does not require a priori knowledge of sample properties in the visible range.

physics.optics↗