arXiv subjects

Ariel Gordon

Publications and source records attributed to Ariel Gordon.

15 recordsLinked to original sources

arXiv2020

Unsupervised Monocular Depth Learning in Dynamic Scenes

We present a method for jointly training the estimation of depth, ego-motion, and a dense 3D translation field of objects relative to the scene, with monocular photometric consistency being the sole source of supervision. We show that this apparently heavily underdetermined problem can be regularized by imposing the following prior knowledge about 3D translation fields: they are sparse, since most of the scene is static, and they tend to be constant for rigid moving objects. We show that this regularization alone is sufficient to train monocular depth prediction models that exceed the accuracy achieved in prior work for dynamic scenes, including methods that require semantic input. Code is at https://github.com/google-research/google-research/tree/master/depth_and_motion_learning .

cs.CV↗

arXiv2020

What Matters in Unsupervised Optical Flow

We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective. Alongside this investigation we construct a number of novel improvements to unsupervised flow models, such as cost volume normalization, stopping the gradient at the occlusion mask, encouraging smoothness before upsampling the flow field, and continual self-supervision with image resizing. By combining the results of our investigation with our improved model components, we are able to present a new unsupervised flow technique that significantly outperforms the previous unsupervised state-of-the-art and performs on par with supervised FlowNet2 on the KITTI 2015 dataset, while also being significantly simpler than related approaches.

cs.CV↗

arXiv2020

Taskology: Utilizing Task Relations at Scale

Many computer vision tasks address the problem of scene understanding and are naturally interrelated e.g. object classification, detection, scene segmentation, depth estimation, etc. We show that we can leverage the inherent relationships among collections of tasks, as they are trained jointly, supervising each other through their known relationships via consistency losses. Furthermore, explicitly utilizing the relationships between tasks allows improving their performance while dramatically reducing the need for labeled data, and allows training with additional unsupervised or simulated data. We demonstrate a distributed joint training algorithm with task-level parallelism, which affords a high degree of asynchronicity and robustness. This allows learning across multiple tasks, or with large amounts of input data, at scale. We demonstrate our framework on subsets of the following collection of tasks: depth and normal prediction, semantic segmentation, 3D motion and ego-motion estimation, and object tracking and 3D detection in point clouds. We observe improved performance across these tasks, especially in the low-label regime.

cs.CV↗

arXiv2020

Improving Semantic Segmentation through Spatio-Temporal Consistency Learned from Videos

We leverage unsupervised learning of depth, egomotion, and camera intrinsics to improve the performance of single-image semantic segmentation, by enforcing 3D-geometric and temporal consistency of segmentation masks across video frames. The predicted depth, egomotion, and camera intrinsics are used to provide an additional supervision signal to the segmentation model, significantly enhancing its quality, or, alternatively, reducing the number of labels the segmentation model needs. Our experiments were performed on the ScanNet dataset.

cs.CV↗

arXiv2020

Detecting Deficient Coverage in Colonoscopies

Colonoscopy is the tool of choice for preventing Colorectal Cancer, by detecting and removing polyps before they become cancerous. However, colonoscopy is hampered by the fact that endoscopists routinely miss 22-28% of polyps. While some of these missed polyps appear in the endoscopist's field of view, others are missed simply because of substandard coverage of the procedure, i.e. not all of the colon is seen. This paper attempts to rectify the problem of substandard coverage in colonoscopy through the introduction of the C2D2 (Colonoscopy Coverage Deficiency via Depth) algorithm which detects deficient coverage, and can thereby alert the endoscopist to revisit a given area. More specifically, C2D2 consists of two separate algorithms: the first performs depth estimation of the colon given an ordinary RGB video stream; while the second computes coverage given these depth estimates. Rather than compute coverage for the entire colon, our algorithm computes coverage locally, on a segment-by-segment basis; C2D2 can then indicate in real-time whether a particular area of the colon has suffered from deficient coverage, and if so the endoscopist can return to that area. Our coverage algorithm is the first such algorithm to be evaluated in a large-scale way; while our depth estimation technique is the first calibration-free unsupervised method applied to colonoscopies. The C2D2 algorithm achieves state of the art results in the detection of deficient coverage. On synthetic sequences with ground truth, it is 2.4 times more accurate than human experts; while on real sequences, C2D2 achieves a 93.0% agreement with experts.

cs.CV↗

arXiv2019

Computationally Efficient Neural Image Compression

Image compression using neural networks have reached or exceeded non-neural methods (such as JPEG, WebP, BPG). While these networks are state of the art in ratedistortion performance, computational feasibility of these models remains a challenge. We apply automatic network optimization techniques to reduce the computational complexity of a popular architecture used in neural image compression, analyze the decoder complexity in execution runtime and explore the trade-offs between two distortion metrics, rate-distortion performance and run-time performance to design and research more computationally efficient neural image compression. We find that our method decreases the decoder run-time requirements by over 50% for a stateof-the-art neural architecture.

eess.IV↗

arXiv2019

Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras

We present a novel method for simultaneous learning of depth, egomotion, object motion, and camera intrinsics from monocular videos, using only consistency across neighboring video frames as supervision signal. Similarly to prior work, our method learns by applying differentiable warping to frames and comparing the result to adjacent ones, but it provides several improvements: We address occlusions geometrically and differentiably, directly using the depth maps as predicted during training. We introduce randomized layer normalization, a novel powerful regularizer, and we account for object motion relative to the scene. To the best of our knowledge, our work is the first to learn the camera intrinsic parameters, including lens distortion, from video in an unsupervised manner, thereby allowing us to extract accurate depth and motion from arbitrary videos of unknown origin at scale. We evaluate our results on the Cityscapes, KITTI and EuRoC datasets, establishing new state of the art on depth prediction and odometry, and demonstrate qualitatively that depth prediction can be learned from a collection of YouTube videos.

cs.CV↗

arXiv2017

MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks

We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per inference), and capable of increasing the network's performance. When applied to standard network architectures on a wide variety of datasets, our approach discovers novel structures in each domain, obtaining higher performance while respecting the resource constraint.

cs.LG↗

arXiv2016

Scalable Learning of Non-Decomposable Objectives

Modern retrieval systems are often driven by an underlying machine learning model. The goal of such systems is to identify and possibly rank the few most relevant items for a given query or context. Thus, such systems are typically evaluated using a ranking-based performance metric such as the area under the precision-recall curve, the $F_\beta$ score, precision at fixed recall, etc. Obviously, it is desirable to train such systems to optimize the metric of interest. In practice, due to the scalability limitations of existing approaches for optimizing such objectives, large-scale retrieval systems are instead trained to maximize classification accuracy, in the hope that performance as measured via the true objective will also be favorable. In this work we present a unified framework that, using straightforward building block bounds, allows for highly scalable optimization of a wide range of ranking-based objectives. We demonstrate the advantage of our approach on several real-life retrieval problems that are significantly larger than those considered in the literature, while achieving substantial improvement in performance over the accuracy-objective baseline.

stat.ML↗

arXiv2006

First-order strong field approximation for high-order harmonic generation

Recently it was shown [A. Gordon and F. X. Kaertner, Phys. Rev. Lett. 95, 223901 (2005)] that the strong field approximation (SFA) for high-order harmonic generation (HHG) is significantly improved when the SFA wave function is used with the acceleration rather than the length form of the dipole operator. In this work it is shown that using the acceleration form upgrades the SFA from zeroth-order to first-order accuracy in the binding potential. The first-order correct three-step model (1st-order TSM) obtained thereby is systematically compared to its standard zeroth-order counterpart (0th-order TSM) and it is found that they differ significantly even for energetic electrons. For molecules (in the single-electron approximation), the 0th-order and the 1st-order TSMs in general disagree about the connection between the orbital symmetry and the positions of the minima in the HHG spectrum. At last, we briefly comment on gauge and translation invariance issues of the SFA.

physics.optics↗

arXiv2005

High order stimulated Brillouin scattering in single-mode fibers with strong feedback

We present an experimental and theoretical study of cascaded high order Stimulated Brillouin Scatterings (SBS) in single mode fibers. It is shown that because of the back-scattering nature of the process, feedback in the input port is needed for obtaining a significant cascaded effect in nonresonant systems. We also discuss similarities to nonlinear photorefractive processes.

physics.optics↗

arXiv2005

Critical Behavior of Light

Light is shown to exhibit critical and tricritical behavior in passive mode-locked lasers with externally injected pulses. It is a first and unique example of critical phenomena in a one-dimensional many body light-mode system. The phase diagrams consist of regimes with continuous wave, driven para-pulses, spontaneous pulses via mode condensation, and heterogeneous pulses, separated by phase transition lines which terminate with critical or tricritical points. Enhanced nongaussian fluctuations and collective dynamics are observed at the critical and tricritical points, showing a mode system analog of the critical opalescence phenomenon. The critical exponents are calculated and shown to comply with the mean field theory, which is rigorous in the light system.

cond-mat.stat-mech↗

arXiv2004

Formation and annihilation of laser light pulse quanta in thermodynamic-like pathway

We present a theoretical and experimental study of multiple pulse formation in passively mode-locked (PML) lasers. Following a statistical mechanics approach, the study yields a thermodynamic-like "phase diagram" with boundaries representing cascaded first order phase transitions. They correspond to abrupt creation or annihilation of pulses and a quantized RF power behavior, as system parameters (noise and/or pumping levels) are varied, in excellent accordance with the experiments. Remarkably, individual pulses carry an almost constant quantum of energy.

cond-mat.stat-mech↗

arXiv2004

Exact solution of the nonlinear laser passive mode locking transition

We present the first statistical mechanics study of a passively mode locked laser which includes all the main physical processes, saturable absorption, Kerr nonlinearity, parabolic gain filtering and group velocity dispersion, assuming the soliton condition. We achieve an exact solution in the thermodynamic limit, where the ratio of the cavity length to the pulse width, the duty cycle, tends to infinity. The thermodynamics depends on a single dimensionless parameter $γ$, the ratio of the correlation length to the pulse width. The phase diagram consists of one ordered, mode-locked phase and one disordered, continuous wave phase, separated by a first order phase transition at $γ=9$. The model belongs to a new class of solvable statistical mechanics models with a non-trivial phase diagram. The results are obtained with a fully controlled transfer matrix calculation, showing rigorously that passive mode locking is a thermodynamic phase transition.

cond-mat.stat-mech↗

arXiv2003

Solution of a statistical mechanics model for pulse formation in lasers

We present a rigorous statistical-mechanics theory of nonlinear many mode laser systems. An important example is the passively mode-locked laser that promotes pulse operation when a saturable absorber is placed in the cavity. It was shown by Gordon and Fischer [1] that pulse formation is a first-order phase transition of spontaneous ordering of modes in an effective "thermodynamic" system, in which intracavity noise level is the effective temperature. In this paper we present a rigorous solution of a model of passive mode locking. We show that the thermodynamics depends on a single parameter, and calculate exactly the mode-locking point. We find the phase diagram and calculate statistical quantities, including the dependence of the intracavity power on the gain saturation function, and finite size corrections near the transition point. We show that the thermodynamics is independent of the gain saturation mechanism and that it is correctly reproduced by a mean field calculation. The outcome is a new solvable statistical mechanics system with an unstable self-interaction accompanied by a natural global power constraint, and an exact description of an important many mode laser system.

cond-mat.stat-mech↗