arXiv subjects

Kihwan Kim

Publications and source records attributed to Kihwan Kim.

At least 55 records · Page 3Linked to original sources

arXiv2019

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical flow, and segmentation of a video into the static scene and moving regions. Our key insight is that these four fundamental vision problems are coupled through geometric constraints. Consequently, learning to solve them together simplifies the problem because the solutions can reinforce each other. We go beyond previous work by exploiting geometry more explicitly and segmenting the scene into static and moving regions. To that end, we introduce Competitive Collaboration, a framework that facilitates the coordinated training of multiple specialized neural networks to solve complex problems. Competitive Collaboration works much like expectation-maximization, but with neural networks that act as both competitors to explain pixels that correspond to static or moving regions, and as collaborators through a moderator that assigns pixels to be either static or independently moving. Our novel method integrates all these problems in a common framework and simultaneously reasons about the segmentation of the scene into moving objects and the static background, the camera motion, depth of the static scene structure, and the optical flow of moving objects. Our model is trained without any supervision and achieves state-of-the-art performance among joint unsupervised methods on all sub-problems.

cs.CV↗

arXiv2019

Focus on Shortcuts to Adiabaticity

Shortcuts to Adiabaticity (STA) constitute driving schemes that provide an alternative to adiabatic protocols to control and guide the dynamics of classical and quantum systems without the requirement of slow driving. Research on STA advances swiftly with theoretical progress being accompanied by experiments on a wide variety of platforms. We summarize recent developments emphasizing advances reported in this focus issue while providing an outlook with open problems and prospects for future research.

quant-ph↗

arXiv2019

Probing Quantum Fluctuations of Work with a Trapped Ion

In this chapter, we illustrate how a trapped ion system can be used for the experimental study of quantum thermodynamics, in particular, quantum fluctuation of work. As technology of nano/micro scale develops, it becomes critical to understand thermodynamics at the quantum mechanical level. The trapped ion system is a representative physical platform to experimentally demonstrate quantum phenomena with excellent control and precision. We provide a basic introduction of the trapped ion system and present the theoretical framework for the experimental study of quantum thermodynamics. Then we bring out two concrete examples of the experimental demonstrations. Finally, we discuss the results and the future of the experimental study of quantum thermodynamics with trapped ion systems.

quant-ph↗

arXiv2019

Randomness expansion secured by quantum contextuality

The output randomness from a random number generator can be certified by observing the violation of quantum contextuality inequalities based on the Kochen-Specker theorem. Contextuality can be tested in a single quantum system, which significantly simplifies the experimental requirements to observe the violation comparing to the ones based on nonlocality tests. However, it is not yet resolved how to ensure compatibilities for sequential measurements that is required in contextuality tests. Here, we employ a modified Klyachko-Can-Binicioğlu-Shumovsky contextuality inequality, which can ease the strict compatibility requirement on measurements. On a trapped single \Ba ion system, we experimentally demonstrate violation of the contextuality inequality and realize self-testing quantum random number expansion by closing detection loopholes. We perform $1.29 \times 10^8$ trials of experiments and extract the randomness of $8.06 \times 10^5$ bits with a speed of 270 bits s$^{-1}$. Our demonstration paves the way for the practical high-speed spot-checking quantum random number expansion and other secure information processing applications.

quant-ph↗

arXiv2019

NRMVS: Non-Rigid Multi-View Stereo

Scene reconstruction from unorganized RGB images is an important task in many computer vision applications. Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: dense 3D reconstruction of scenes with non-rigid changes observed from arbitrary, sparse, and wide-baseline views. We formulate the problem as a joint optimization of deformation and depth estimation, using deformation graphs as the underlying representation. We propose a new sparse 3D to 2D matching technique, together with a dense patch-match evaluation scheme to estimate deformation and depth with photometric consistency. We show that creating a dense 4D structure from a few RGB images with non-rigid changes is possible, and demonstrate that our method can be used to interpolate novel deformed scenes from various combinations of these deformation estimates derived from the sparse views.

cs.CV↗

arXiv2019

Scalable global entangling gates on arbitrary ion qubits

A quantum algorithm can be decomposed into a sequence consisting of single qubit and 2-qubit entangling gates. To optimize the decomposition and achieve more efficient construction of the quantum circuit, we can replace multiple 2-qubit gates with a single global entangling gate. Here, we propose and implement a scalable scheme to realize the global entangling gates on multiple $\yb$ ion qubits by coupling to multiple motional modes through external fields. Such global gates require simultaneously decoupling of multiple motional modes and balancing of the coupling strengths for all the qubit-pairs at the gate time. To satisfy the complicated requirements, we develop a trapped-ion system with fully-independent control capability on each ion, and experimentally realize the global entangling gates. As examples, we utilize them to prepare the Greenberger-Horne-Zeilinger (GHZ) states in a single entangling operation, and successfully show the genuine multi-partite entanglements up to four qubits with the state fidelities over $93.4\%$.

quant-ph↗

arXiv2019

Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera

Depth sensing is crucial for 3D reconstruction and scene understanding. Active depth sensors provide dense metric measurements, but often suffer from limitations such as restricted operating ranges, low spatial resolution, sensor interference, and high power consumption. In this paper, we propose a deep learning (DL) method to estimate per-pixel depth and its uncertainty continuously from a monocular video stream, with the goal of effectively turning an RGB camera into an RGB-D camera. Unlike prior DL-based methods, we estimate a depth probability distribution for each pixel rather than a single depth value, leading to an estimate of a 3D depth probability volume for each input frame. These depth probability volumes are accumulated over time under a Bayesian filtering framework as more incoming frames are processed sequentially, which effectively reduces depth uncertainty and improves accuracy, robustness, and temporal stability. Compared to prior work, the proposed approach achieves more accurate and stable results, and generalizes better to new datasets. Experimental results also show the output of our approach can be directly fed into classical RGB-D based 3D scanning methods for 3D scene reconstruction.

cs.CV↗

arXiv2019

PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image

This paper proposes a deep neural architecture, PlaneRCNN, that detects and reconstructs piecewise planar surfaces from a single RGB image. PlaneRCNN employs a variant of Mask R-CNN to detect planes with their plane parameters and segmentation masks. PlaneRCNN then jointly refines all the segmentation masks with a novel loss enforcing the consistency with a nearby view during training. The paper also presents a new benchmark with more fine-grained plane segmentations in the ground-truth, in which, PlaneRCNN outperforms existing state-of-the-art methods with significant margins in the plane detection, segmentation, and reconstruction metrics. PlaneRCNN makes an important step towards robust plane extraction, which would have an immediate impact on a wide range of applications including Robotics, Augmented Reality, and Virtual Reality.

cs.CV↗

arXiv2018

Probabilistic Eigensolver with a Trapped-Ion Quantum Processor

Quantum simulation of complex quantum systems and their properties often requires the ability to prepare initial states in an eigenstate of the Hamiltonian to be simulated. In addition, to compute the eigenvalues of a Hamiltonian is in general a non-trivial problem. Here, we propose a hybrid quantum-classical probabilistic method to compute eigenvalues and prepare eigenstates of Hamiltonians which are simulatable with a trapped-ion quantum processor.

quant-ph↗

arXiv2018

EOE: Expected Overlap Estimation over Unstructured Point Cloud Data

We present an iterative overlap estimation technique to augment existing point cloud registration algorithms that can achieve high performance in difficult real-world situations where large pose displacement and non-overlapping geometry would otherwise cause traditional methods to fail. Our approach estimates overlapping regions through an iterative Expectation Maximization procedure that encodes the sensor field-of-view into the registration process. The proposed technique, Expected Overlap Estimation (EOE), is derived from the observation that differences in field-of-view violate the iid assumption implicitly held by all maximum likelihood based registration techniques. We demonstrate how our approach can augment many popular registration methods with minimal computational overhead. Through experimentation on both synthetic and real-world datasets, we find that adding an explicit overlap estimation step can aid robust outlier handling and increase the accuracy of both ICP-based and GMM-based registration methods, especially in large unstructured domains and where the amount of overlap between point clouds is very small.

cs.CV↗

arXiv2018

Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation

Estimation of 3D motion in a dynamic scene from a temporal pair of images is a core task in many scene understanding problems. In real world applications, a dynamic scene is commonly captured by a moving camera (i.e., panning, tilting or hand-held), increasing the task complexity because the scene is observed from different view points. The main challenge is the disambiguation of the camera motion from scene motion, which becomes more difficult as the amount of rigidity observed decreases, even with successful estimation of 2D image correspondences. Compared to other state-of-the-art 3D scene flow estimation methods, in this paper we propose to \emph{learn} the rigidity of a scene in a supervised manner from a large collection of dynamic scene data, and directly infer a rigidity mask from two sequential images with depths. With the learned network, we show how we can effectively estimate camera motion and projected scene flow using computed 2D optical flow and the inferred rigidity mask. For training and testing the rigidity network, we also provide a new semi-synthetic dynamic scene dataset (synthetic foreground objects with a real background) and an evaluation split that accounts for the percentage of observed non-rigid pixels. Through our evaluation we show the proposed framework outperforms current state-of-the-art scene flow estimation methods in challenging dynamic scenes.

cs.CV↗

arXiv2018

Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures

Point cloud registration sits at the core of many important and challenging 3D perception problems including autonomous navigation, SLAM, object/scene recognition, and augmented reality. In this paper, we present a new registration algorithm that is able to achieve state-of-the-art speed and accuracy through its use of a hierarchical Gaussian Mixture Model (GMM) representation. Our method constructs a top-down multi-scale representation of point cloud data by recursively running many small-scale data likelihood segmentations in parallel on a GPU. We leverage the resulting representation using a novel PCA-based optimization criterion that adaptively finds the best scale to perform data association between spatial subsets of point cloud data. Compared to previous Iterative Closest Point and GMM-based techniques, our tree-based point association algorithm performs data association in logarithmic-time while dynamically adjusting the level of detail to best match the complexity and spatial distribution characteristics of local scene geometry. In addition, unlike other GMM methods that restrict covariances to be isotropic, our new PCA-based optimization criterion well-approximates the true MLE solution even when fully anisotropic Gaussian covariances are used. Efficient data association, multi-scale adaptability, and a robust MLE approximation produce an algorithm that is up to an order of magnitude both faster and more accurate than current state-of-the-art on a wide variety of 3D datasets captured from LiDAR to structured light.

cs.CV↗

arXiv2018

Frequency stabilization of a 650 nm laser to I$_{2}$ spectrum for trapped $^{138}$Ba$^{+}$ ions

The optical manipulation of Ba$^{+}$ ions is mainly performed by a 493 nm laser for the S$_{1/2}$-P$_{1/2}$ transition and a 650 nm laser for the P$_{1/2}$-D$_{3/2}$ transition. Since the branching ratio between the 493 nm and 650 nm transitions of a single Ba$^{+}$ ion is comparable, stabilization systems of both lasers are equally important for Doppler cooling, sub-Doppler cooling, optical pumping and state detection. The stabilization system of a 493 nm laser to an absolute Te$_2$ reference has been well established. However, the stabilization of a 650 nm laser has not been presented before. Here we report twenty spectral lines of I$_{2}$ in the range of 0.9 GHz above the resonance of the P$_{1/2}$-D$_{3/2}$ transition. We stabilize the 650 nm laser through the optical cavity to the lowest one among these lines, which is about 350 MHz apart, as the absolute frequency reference. Furthermore, we measure the frequency differences between these iodine lines and the Ba$^+$ resonance through fluorescence excitation spectrum with well-resolved dark states, which is in agreement with the theoretical expectation. The presented stabilization scheme enables us to perform precise experiments with Ba$^{+}$ ions.

physics.atom-ph↗

arXiv2018

A Lightweight Approach for On-the-Fly Reflectance Estimation

Estimating surface reflectance (BRDF) is one key component for complete 3D scene capture, with wide applications in virtual reality, augmented reality, and human computer interaction. Prior work is either limited to controlled environments (\eg gonioreflectometers, light stages, or multi-camera domes), or requires the joint optimization of shape, illumination, and reflectance, which is often computationally too expensive (\eg hours of running time) for real-time applications. Moreover, most prior work requires HDR images as input which further complicates the capture process. In this paper, we propose a lightweight approach for surface reflectance estimation directly from $8$-bit RGB images in real-time, which can be easily plugged into any 3D scanning-and-fusion system with a commodity RGBD sensor. Our method is learning-based, with an inference time of less than 90ms per scene and a model size of less than 340K bytes. We propose two novel network architectures, HemiCNN and Grouplet, to deal with the unstructured input data from multiple viewpoints under unknown illumination. We further design a loss function to resolve the color-constancy and scale ambiguity. In addition, we have created a large synthetic dataset, SynBRDF, which comprises a total of $500$K RGBD images rendered with a physically-based ray tracer under a variety of natural illumination, covering $5000$ materials and $5000$ shapes. SynBRDF is the first large-scale benchmark dataset for reflectance estimation. Experiments on both synthetic data and real data show that the proposed method effectively recovers surface reflectance, and outperforms prior work for reflectance estimation in uncontrolled environments.

cs.CV↗

arXiv2018

Geometry-Aware Learning of Maps for Camera Localization

Maps are a key component in image-based camera localization and visual SLAM systems: they are used to establish geometric constraints between images, correct drift in relative pose estimation, and relocalize cameras after lost tracking. The exact definitions of maps, however, are often application-specific and hand-crafted for different scenarios (e.g. 3D landmarks, lines, planes, bags of visual words). We propose to represent maps as a deep neural net called MapNet, which enables learning a data-driven map representation. Unlike prior work on learning maps, MapNet exploits cheap and ubiquitous sensory inputs like visual odometry and GPS in addition to images and fuses them together for camera localization. Geometric constraints expressed by these inputs, which have traditionally been used in bundle adjustment or pose-graph optimization, are formulated as loss terms in MapNet training and also used during inference. In addition to directly improving localization accuracy, this allows us to update the MapNet (i.e., maps) in a self-supervised manner using additional unlabeled video sequences from the scene. We also propose a novel parameterization for camera rotation which is better suited for deep-learning based camera pose regression. Experimental results on both the indoor 7-Scenes dataset and the outdoor Oxford RobotCar dataset show significant performance improvement over prior work. The MapNet project webpage is https://goo.gl/mRB3Au.

cs.CV↗

arXiv2017

Operational effects of the UNOT gate on classical and quantum correlations

The NOT gate that flips a classical bit is ubiquitous in classical information processing. However its quantum analogue, the universal NOT (UNOT) gate that flips a quantum spin in any alignment into its antipodal counterpart is strictly forbidden. Here we explore the connection between this discrepancy and how UNOT gates affect classical and quantum correlations. We show that while a UNOT gate always preserves classical correlations between two spins, it can non-locally increase or decrease their shared discord in ways that allow violation of the data processing inequality. We experimentally illustrate this using a multi-level trapped \Yb ion that allows simulation of anti-unitary operations.

quant-ph↗

arXiv2017

Quantum simulation of the quantum Rabi model in a trapped ion

The quantum Rabi model, involving a two-level system and a bosonic field mode, is arguably the simplest and most fundamental model describing quantum light-matter interactions. Historically, due to the restricted parameter regimes of natural light-matter processes, the richness of this model has been elusive in the lab. Here, we experimentally realize a quantum simulation of the quantum Rabi model in a single trapped ion, where the coupling strength between the simulated light mode and atom can be tuned at will. The versatility of the demonstrated quantum simulator enables us to experimentally explore the quantum Rabi model in detail, including a wide range of otherwise unaccessible phenomena, as those happening in the ultrastrong and deep strong coupling regimes. In this sense, we are able to adiabatically generate the ground state of the quantum Rabi model in the deep strong coupling regime, where we are able to detect the nontrivial entanglement between the bosonic field mode and the two-level system. Moreover, we observe the breakdown of the rotating-wave approximation when the coupling strength is increased, and the generation of phonon wave packets that bounce back and forth when the coupling reaches the deep strong coupling regime. Finally, we also measure the energy spectrum of the quantum Rabi model in the ultrastrong coupling regime.

quant-ph↗

arXiv2017

Multiframe Scene Flow with Piecewise Rigid Motion

We introduce a novel multiframe scene flow approach that jointly optimizes the consistency of the patch appearances and their local rigid motions from RGB-D image sequences. In contrast to the competing methods, we take advantage of an oversegmentation of the reference frame and robust optimization techniques. We formulate scene flow recovery as a global non-linear least squares problem which is iteratively solved by a damped Gauss-Newton approach. As a result, we obtain a qualitatively new level of accuracy in RGB-D based scene flow estimation which can potentially run in real-time. Our method can handle challenging cases with rigid, piecewise rigid, articulated and moderate non-rigid motion, and does not rely on prior knowledge about the types of motions and deformations. Extensive experiments on synthetic and real data show that our method outperforms state-of-the-art.

cs.CV↗