arXiv subjects

Sangbae Kim

Publications and source records attributed to Sangbae Kim.

At least 37 records · Page 2Linked to original sources

arXiv2024

CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control

The parallelism afforded by GPUs presents significant advantages in training controllers through reinforcement learning (RL). However, integrating model-based optimization into this process remains challenging due to the complexity of formulating and solving optimization problems across thousands of instances. In this work, we present CusADi, an extension of the CasADi symbolic framework to support the parallelization of arbitrary closed-form expressions on GPUs with CUDA. We also formulate a closed-form approximation for solving general optimal control problems, enabling large-scale parallelization and evaluation of MPC controllers. Our results show a ten-fold speedup relative to similar MPC implementation on the CPU, and we demonstrate the use of CusADi for various applications, including parallel simulation, parameter sweeps, and policy training.

cs.RO↗

arXiv2024

Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion

In this work, we introduce a control framework that combines model-based footstep planning with Reinforcement Learning (RL), leveraging desired footstep patterns derived from the Linear Inverted Pendulum (LIP) dynamics. Utilizing the LIP model, our method forward predicts robot states and determines the desired foot placement given the velocity commands. We then train an RL policy to track the foot placements without following the full reference motions derived from the LIP model. This partial guidance from the physics model allows the RL policy to integrate the predictive capabilities of the physics-informed dynamics and the adaptability characteristics of the RL controller without overfitting the policy to the template model. Our approach is validated on the MIT Humanoid, demonstrating that our policy can achieve stable yet dynamic locomotion for walking and turning. We further validate the adaptability and generalizability of our policy by extending the locomotion task to unseen, uneven terrain. During the hardware deployment, we have achieved forward walking speeds of up to 1.5 m/s on a treadmill and have successfully performed dynamic locomotion maneuvers such as 90-degree and 180-degree turns.

cs.RO↗

arXiv2024

Reducing of the Uncertainty Product of Coherent Light through Multi-Photon Interference

We demonstrate theoretically and experimentally how the diffraction and interferometric resolution limit for single-mode coherent cw laser light can be overcome by multi-photon interference. By use of a Mach-Zehnder interferometer, operated in the single input and single or double output port geometries, we observe a fringe width reduction of the conventional interference pattern, predicted by the wave or single photon quantum theory, by a factor of up to $1/\sqrt{2N}$ through coincident detection of $N=2,3,4$ photons. Our scheme does not require squeezed or entangled light to overcome the standard quantum limit and greatly facilitates precision interferometry experiments.

quant-ph↗

arXiv2024

FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning

Motion trajectories offer reliable references for physics-based motion learning but suffer from sparsity, particularly in regions that lack sufficient data coverage. To address this challenge, we introduce a self-supervised, structured representation and generation method that extracts spatial-temporal relationships in periodic or quasi-periodic motions. The motion dynamics in a continuously parameterized latent space enable our method to enhance the interpolation and generalization capabilities of motion learning algorithms. The motion learning controller, informed by the motion parameterization, operates online tracking of a wide range of motions, including targets unseen during training. With a fallback mechanism, the controller dynamically adapts its tracking strategy and automatically resorts to safe action execution when a potentially risky target is proposed. By leveraging the identified spatial-temporal structure, our work opens new possibilities for future advancements in general motion representation and learning algorithms.

cs.LG↗

arXiv2024

Learning Emergent Gaits with Decentralized Phase Oscillators: on the role of Observations, Rewards, and Feedback

We present a minimal phase oscillator model for learning quadrupedal locomotion. Each of the four oscillators is coupled only to itself and its corresponding leg through local feedback of the ground reaction force, which can be interpreted as an observer feedback gain. We interpret the oscillator itself as a latent contact state-estimator. Through a systematic ablation study, we show that the combination of phase observations, simple phase-based rewards, and the local feedback dynamics induces policies that exhibit emergent gait preferences, while using a reduced set of simple rewards, and without prescribing a specific gait. The code is open-source, and a video synopsis available at https://youtu.be/1NKQ0rSV3jU.

cs.RO↗

arXiv2023

Observations of super-resolution using phase-controlled coherent photons in a delayed-choice quantum eraser scheme

Super-resolution overcoming the standard quantum limit has been intensively studied for quantum sensing applications of precision target detection over the last decades. Not only higher-order entangled photons but also phase-controlled coherent photons have been used to demonstrate the super-resolution. Due to the extreme inefficiency of higher-order entangled photon-pair generation and ultralow signal-to-noise ratio, however, quantum sensing has been severely limited. Here, we report observations of coherently excited super-resolution using phase-controlled coherent photons in a delayed-choice quantum eraser scheme. Using phase manipulations of the quantum erasers, super-resolution has been observed for higher-order intensity correlations between them, satisfying the Heisenberg limit in phase resolution. This new type of precision phase-detection technique opens the door to practical applications of quantum sensing compatible with current technologies based on coherence optics.

quant-ph↗

arXiv2023

How to Model Brushless Electric Motors for the Design of Lightweight Robotic Systems

A key step in the development of lightweight, high performance robotic systems is the modeling and selection of permanent magnet brushless direct current (BLDC) electric motors. Typical modeling analyses are completed a priori, and provide insight for properly sizing a motor for an application, specifying the required operating voltage and current, as well as assessing the thermal response and other design attributes (e.g.transmission ratio). However, to perform these modeling analyses, proper information about the motor's characteristics are needed, which are often obtained from manufacturer datasheets. Through our own experience and communications with manufacturers, we have noticed a lack of clarity and standardization in modeling BLDC motors, compounded by vague or inconsistent terminology used in motor datasheets. The purpose of this tutorial is to concisely describe the governing equations for BLDC motor analyses used in the design process, as well as highlight potential errors that can arise from incorrect usage. We present a power-invariant conversion from phase and line-to-line reference frames to a familiar q-axis DC motor representation, which provides a ``brushed'' analogue of a three phase BLDC motor that is convenient for analysis and design. We highlight potential errors including incorrect calculations of winding resistive heat loss, improper estimation of motor torque via the motor's torque constant, and incorrect estimation of the required bus voltage or resulting angular velocity limitations. A unified and condensed set of governing equations is available for designers in the Appendix. The intent of this work is to provide a consolidated mathematical foundation for modeling BLDC motors that addresses existing confusion and fosters high performance designs of future robotic systems.

cs.RO↗

arXiv2023

Benchmarking Potential Based Rewards for Learning Humanoid Locomotion

The main challenge in developing effective reinforcement learning (RL) pipelines is often the design and tuning the reward functions. Well-designed shaping reward can lead to significantly faster learning. Naively formulated rewards, however, can conflict with the desired behavior and result in overfitting or even erratic performance if not properly tuned. In theory, the broad class of potential based reward shaping (PBRS) can help guide the learning process without affecting the optimal policy. Although several studies have explored the use of potential based reward shaping to accelerate learning convergence, most have been limited to grid-worlds and low-dimensional systems, and RL in robotics has predominantly relied on standard forms of reward shaping. In this paper, we benchmark standard forms of shaping with PBRS for a humanoid robot. We find that in this high-dimensional system, PBRS has only marginal benefits in convergence speed. However, the PBRS reward terms are significantly more robust to scaling than typical reward shaping approaches, and thus easier to tune.

cs.RO↗

arXiv2023

Reinforcement Learning for Legged Robots: Motion Imitation from Model-Based Optimal Control

We propose MIMOC: Motion Imitation from Model-Based Optimal Control. MIMOC is a Reinforcement Learning (RL) controller that learns agile locomotion by imitating reference trajectories from model-based optimal control. MIMOC mitigates challenges faced by other motion imitation RL approaches because the references are dynamically consistent, require no motion retargeting, and include torque references. Hence, MIMOC does not require fine-tuning. MIMOC is also less sensitive to modeling and state estimation inaccuracies than model-based controllers. We validate MIMOC on the Mini-Cheetah in outdoor environments over a wide variety of challenging terrain, and on the MIT Humanoid in simulation. We show cases where MIMOC outperforms model-based optimal controllers, and show that imitating torque references improves the policy's performance.

cs.RO↗

arXiv2023

Observations of the delayed-choice quantum eraser using coherent photons

Quantum superposition is the cornerstone of quantum mechanics, where interference fringes originate in the self-interference of a single photon via indistinguishable photon characteristics. Wheeler delayed-choice experiments have been extensively studied for the wave-particle duality over the last several decades to understand the complementarity theory of quantum mechanics. The heart of the delayed-choice quantum eraser is in the mutually exclusive quantum feature violating the cause-effect relation. Here, we experimentally demonstrate the quantum eraser using coherent photon pairs by the delayed choice of a polarizer placed out of the interferometer. Coherence solutions of the observed quantum eraser are derived from a typical Mach-Zehnder interferometer, where the violation of the cause-effect relation is due to selective measurements of basis choice.

quant-ph↗

arXiv2023

Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation

Modern robotic manipulation systems fall short of human manipulation skills partly because they rely on closing feedback loops exclusively around vision data, which reduces system bandwidth and speed. By developing autonomous grasping reflexes that rely on high-bandwidth force, contact, and proximity data, the overall system speed and robustness can be increased while reducing reliance on vision data. We are developing a new system built around a low-inertia, high-speed arm with nimble fingers that combines a high-level trajectory planner operating at less than 1 Hz with low-level autonomous reflex controllers running upwards of 300 Hz. We characterize the reflex system by comparing the volume of the set of successful grasps for a naive baseline controller and variations of our reflexive grasping controller, finding that our controller expands the set of successful grasps by 55% relative to the baseline. We also deploy our reflexive grasping controller with a simple vision-based planner in an autonomous clutter clearing task, achieving a grasp success rate above 90% while clearing over 100 items.

cs.RO↗

arXiv2023

Design of a Multimodal Fingertip Sensor for Dynamic Manipulation

We introduce a spherical fingertip sensor for dynamic manipulation. It is based on barometric pressure and time-of-flight proximity sensors and is low-latency, compact, and physically robust. The sensor uses a trained neural network to estimate the contact location and three-axis contact forces based on data from the pressure sensors, which are embedded within the sensor's sphere of polyurethane rubber. The time-of-flight sensors face in three different outward directions, and an integrated microcontroller samples each of the individual sensors at up to 200 Hz. To quantify the effect of system latency on dynamic manipulation performance, we develop and analyze a metric called the collision impulse ratio and characterize the end-to-end latency of our new sensor. We also present experimental demonstrations with the sensor, including measuring contact transitions, performing coarse mapping, maintaining a contact force with a moving object, and reacting to avoid collisions.

cs.RO↗

arXiv2022

Humanoid Self-Collision Avoidance Using Whole-Body Control with Control Barrier Functions

This work combines control barrier functions (CBFs) with a whole-body controller to enable self-collision avoidance for the MIT Humanoid. Existing reactive controllers for self-collision avoidance cannot guarantee collision-free trajectories as they do not leverage the robot's full dynamics, thus compromising kinematic feasibility. In comparison, the proposed CBF-WBC controller can reason about the robot's underactuated dynamics in real-time to guarantee collision-free motions. The effectiveness of this approach is validated in simulation. First, a simple hand-reaching experiment shows that the CBF-WBC enables the robot's hand to deviate from an infeasible reference trajectory to avoid self-collisions. Second, the CBF-WBC is combined with a linear model predictive controller (LMPC) designed for dynamic locomotion, and the CBF-WBC is used to track the LMPC predictions. Walking experiments show that adding CBFs avoids leg self-collisions when the footstep location or swing trajectory provided by the high-level planner are infeasible for the real robot, and generates feasible arm motions that improve disturbance recovery.

cs.RO↗

arXiv2022

Fast Reflexive Grasping with a Proprioceptive Teleoperation Platform

We present a proprioceptive teleoperation system that uses a reflexive grasping algorithm to enhance the speed and robustness of pick-and-place tasks. The system consists of two manipulators that use quasi-direct-drive actuation to provide highly transparent force feedback. The end-effector has bimodal force sensors that measure 3-axis force information and 2-dimensional contact location. This information is used for anti-slip and re-grasping reflexes. When the user makes contact with the desired object, the re-grasping reflex aligns the gripper fingers with antipodal points on the object to maximize the grasp stability. The reflex takes only 150ms to correct for inaccurate grasps chosen by the user, so the user's motion is only minimally disturbed by the execution of the re-grasp. Once antipodal contact is established, the anti-slip reflex ensures that the gripper applies enough normal force to prevent the object from slipping out of the grasp. The combination of proprioceptive manipulators and reflexive grasping allows the user to complete teleoperated tasks with precision at high speed.

cs.RO↗

arXiv2022

Orientation-Aware Model Predictive Control with Footstep Adaptation for Dynamic Humanoid Walking

This paper proposes a novel orientation-aware model predictive control (MPC) for dynamic humanoid walking that can plan footstep locations online. Instead of a point-mass model, this work uses the augmented single rigid body model (aSRBM) to enable the MPC to leverage orientation dynamics and stepping strategy within a unified optimization framework. With the footstep location as part of the decision variables in the aSRBM, the MPC can reason about stepping within the kinematic constraints. A task-space controller (TSC) tracks the body pose and swing leg references output from the MPC, while exploiting the full-order dynamics of the humanoid. The proposed control framework is suitable for real-time applications since both MPC and TSC are formulated as quadratic programs. Simulation investigations show that the orientation-aware MPC-based framework is more robust against external torque disturbance compared to state-of-the-art controllers using the point mass model, especially when the torso undergoes large angular excursion. The same control framework can also enable the MIT Humanoid to overcome uneven terrains, such as traversing a wave field.

cs.RO↗

arXiv2022

Rapid and Reliable Quadruped Motion Planning with Omnidirectional Jumping

Dynamic jumping with legged robots poses a challenging problem in planning and control. Formulating the jump optimization to allow fast online execution is difficult; efficiently using this capability to generate long-horizon motion plans further complicates the problem. In this work, we present a hierarchical planning framework to address this problem. We first formulate a real-time tractable trajectory optimization for performing omnidirectional jumping. We then embed the results of this optimization into a low dimensional jump feasibility classifier. This classifier is leveraged to produce geometric motion plans that select dynamically feasible jumps while mitigating the effects of the process noise. We deploy our framework on the Mini Cheetah Vision quadruped, demonstrating the robot's ability to generate and execute reliable, goal-oriented plans that involve forward, lateral, and rotational jumps onto surfaces as tall as the robot's nominal hip height. The ability to plan through omnidirectional jumping greatly expands the robot's mobility relative to planners that restrict jumping to the sagittal or frontal planes.

cs.RO↗

arXiv2021

Learning to Jump from Pixels

Today's robotic quadruped systems can robustly walk over a diverse range of rough but continuous terrains, where the terrain elevation varies gradually. Locomotion on discontinuous terrains, such as those with gaps or obstacles, presents a complementary set of challenges. In discontinuous settings, it becomes necessary to plan ahead using visual inputs and to execute agile behaviors beyond robust walking, such as jumps. Such dynamic motion results in significant motion of onboard sensors, which introduces a new set of challenges for real-time visual processing. The requirement for agility and terrain awareness in this setting reinforces the need for robust control. We present Depth-based Impulse Control (DIC), a method for synthesizing highly agile visually-guided locomotion behaviors. DIC affords the flexibility of model-free learning but regularizes behavior through explicit model-based optimization of ground reaction forces. We evaluate the proposed method both in simulation and in the real world.

cs.RO↗

arXiv2021

Observations of the nonclassical feature of photon bunching on a beam splitter using coherent photons along the same input port

One of the most striking quantum phenomena is photon bunching resulting from coincidently impinging two-indistinguishable photons on a beam splitter (BS) from two different input ports. Such a nonclassical feature has also been observed even between two independent light sources through either coherence optics resulting in phase locking or post-selected measurements such as quantum beating-based gating. Recently, BS physics regarding quantum features has been discussed using pure coherence optics based on phase basis superposition of the BS. Here, we experimentally demonstrate coherent photon bunching on a BS, where coherent photons come from the same input port. Although the mean values of both output photons are uniform and equal to each other, the mean value of the coincidence measurements between two output photons results in the nonclassical feature of photon bunching at a 50% rate. For this unprecedented result, we discuss the origin of indistinguishability for this quantum feature using the wave nature of a photon to understand the role of a BS in quantum mechanics.

quant-ph↗