arXiv subjects

Yi Zhu

Publications and source records attributed to Yi Zhu.

At least 235 records · Page 13Linked to original sources

arXiv2019

Using Conditional Generative Adversarial Networks to Generate Ground-Level Views From Overhead Imagery

This paper develops a deep-learning framework to synthesize a ground-level view of a location given an overhead image. We propose a novel conditional generative adversarial network (cGAN) in which the trained generator generates realistic looking and representative ground-level images using overhead imagery as auxiliary information. The generator is an encoder-decoder network which allows us to compare low- and high-level features as well as their concatenation for encoding the overhead imagery. We also demonstrate how our framework can be used to perform land cover classification by modifying the trained cGAN to extract features from overhead imagery. This is interesting because, although we are using this modified cGAN as a feature extractor for overhead imagery, it incorporates knowledge of how locations look from the ground.

cs.CV↗

arXiv2019

Ferroelectric driven exciton and trion modulation in monolayer MoSe2 and WSe2

In this work, we show how domain engineered lithium niobate can be used to selectively dope monolayer MoSe2 and WSe2 and demonstrate that these ferroelectric domains can significantly enhance or inhibit photoluminescence (PL) with the most dramatic modulation occurring at the heterojunction interface between two domains. A micro-PL and Raman system is used to obtain spatially resolved images of the differently doped transition metal dichalcogenides (TMDs). The domain inverted lithium niobate causes changes in the TMDs due to electrostatic doping as a result of the remnant polarization from the substrate. Moreover, the differently doped TMDs (n-type MoSe2 and p-type WSe2) exhibit opposite PL modulation. Distinct oppositely charged domains were obtained with a 9-fold PL enhancement for the same single MoSe2 sheet when adhered to the positive (P+) and negative (P-) domains. This sharp PL modulation on the ferroelectric domain results from different free electron or hole concentrations in the materials conduction band or valence band. Moreover, excitons dissociate rapidly at the interface between the P+ and P- domains due to the built-in electric field. We are able to adjust the charge on the P+ and P- domains using temperature via the pyroelectric effect and observe rapid PL quenching over a narrow temperature range illustrating the observed PL modulation is electronic in nature. This observation creates an opportunity to harness the direct bandgap TMD 2D materials as an active optical component for the lithium niobate platform using domain engineering of the lithium niobate substrate to create optically active heterostructures that could be used for photodetectors or even electrically driven optical sources on-chip.

cond-mat.mtrl-sci↗

arXiv2018

Bloch theory-based gradient recovery method for computing topological edge modes in photonic graphene

Photonic graphene, a photonic crystal with honeycomb structures, has been intensively studied in both theoretical and applied fields. Similar to graphene which admits Dirac Fermions and topological edge states, photonic graphene supports novel and subtle propagating modes (edge modes) of electromagnetic waves. These modes have wide applications in many optical systems. In this paper, we propose a novel gradient recovery method based on Bloch theory for the computation of topological edge modes in the honeycomb structure. Compared to standard finite element methods, this method provides higher order accuracy with the help of gradient recovery technique. This high order accuracy is highly desired for constructing the propagating electromagnetic modes in applications. We analyze the accuracy and prove the superconvergence of this method. Numerical examples are presented to show the efficiency by computing the edge mode for the $\mathcal{P}$-symmetry and $\mathcal{C}$-symmetry breaking cases in honeycomb structures.

math.NA↗

arXiv2018

Hidden Two-Stream Convolutional Networks for Action Recognition

Analyzing videos of human actions involves understanding the temporal relationships among video frames. State-of-the-art action recognition approaches rely on traditional optical flow estimation methods to pre-compute motion information for CNNs. Such a two-stage approach is computationally expensive, storage demanding, and not end-to-end trainable. In this paper, we present a novel CNN architecture that implicitly captures motion information between adjacent frames. We name our approach hidden two-stream CNNs because it only takes raw video frames as input and directly predicts action classes without explicitly computing optical flow. Our end-to-end approach is 10x faster than its two-stage baseline. Experimental results on four challenging action recognition datasets: UCF101, HMDB51, THUMOS14 and ActivityNet v1.2 show that our approach significantly outperforms the previous best real-time approaches.

cs.CV↗

arXiv2018

Gated Transfer Network for Transfer Learning

Deep neural networks have led to a series of breakthroughs in computer vision given sufficient annotated training datasets. For novel tasks with limited labeled data, the prevalent approach is to transfer the knowledge learned in the pre-trained models to the new tasks by fine-tuning. Classic model fine-tuning utilizes the fact that well trained neural networks appear to learn cross domain features. These features are treated equally during transfer learning. In this paper, we explore the impact of feature selection in model fine-tuning by introducing a transfer module, which assigns weights to features extracted from pre-trained models. The proposed transfer module proves the importance of feature selection for transferring models from source to target domains. It is shown to significantly improve upon fine-tuning results with only marginal extra computational cost. We also incorporate an auxiliary classifier as an extra regularizer to avoid over-fitting. Finally, we build a Gated Transfer Network (GTN) based on our transfer module and achieve state-of-the-art results on six different tasks.

cs.CV↗

arXiv2018

Random Temporal Skipping for Multirate Video Analysis

Current state-of-the-art approaches to video understanding adopt temporal jittering to simulate analyzing the video at varying frame rates. However, this does not work well for multirate videos, in which actions or subactions occur at different speeds. The frame sampling rate should vary in accordance with the different motion speeds. In this work, we propose a simple yet effective strategy, termed random temporal skipping, to address this situation. This strategy effectively handles multirate videos by randomizing the sampling rate during training. It is an exhaustive approach, which can potentially cover all motion speed variations. Furthermore, due to the large temporal skipping, our network can see video clips that originally cover over 100 frames. Such a time range is enough to analyze most actions/events. We also introduce an occlusion-aware optical flow learning method that generates improved motion maps for human action recognition. Our framework is end-to-end trainable, runs in real-time, and achieves state-of-the-art performance on six widely adopted video benchmarks.

cs.CV↗

arXiv2018

What Is It Like Down There? Generating Dense Ground-Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks

This paper investigates conditional generative adversarial networks (cGANs) to overcome a fundamental limitation of using geotagged media for geographic discovery, namely its sparse and uneven spatial distribution. We train a cGAN to generate ground-level views of a location given overhead imagery. We show the "fake" ground-level images are natural looking and are structurally similar to the real images. More significantly, we show the generated images are representative of the locations and that the representations learned by the cGANs are informative. In particular, we show that dense feature maps generated using our framework are more effective for land-cover classification than approaches which spatially interpolate features extracted from sparse ground-level images. To our knowledge, ours is the first work to use cGANs to generate ground-level views given overhead imagery and to explore the benefits of the learned representations.

cs.CV↗

arXiv2018

Generalized hydrodynamics and the classical hydrodynamic limit

This work is concerned with our recently developed formalism of non-equilibrium thermodynamics. This formalism extends the classical irreversible thermodynamics which leads to classical thermodynamics and can not describe physical phenomena with long relaxations. With the extended theory, we obtain a generalized hydrodynamic system which can be used to describe the non-Newtonian fluids, heat transfer beyond Fourier's law and fluid flow in very short temporal and spatial scales. We study the mathematical structure of the generalized hydrodynamic system and show that it possesses a nice conservation-dissipation structure and therefore is symmetrizable hyperbolic. Moreover, we rigorously justify that this generalized hydrodynamic system tends to the classical hydrodynamics when the relaxation times approach to zero. This shows that the classical hydrodynamics can be derived as an approximation of our generalized one.

math.AP↗

arXiv2018

The arithmetic puncturing problem and integral points

In this paper, we provide evidence to support a positive answer to a question of Hassett and Tschinkel. In particular, if an algebraic variety V has a dense set of rational points, they ask whether or not the set of D-integral points is potentially dense, where D is a set of codimension at least two. We give a positive answer to this question in many cases, including varieties whose generic linear section is a smooth rational curve, and certain K3 surfaces. We also discuss some stronger notions of integrality of points, and give some positive answers to some cases of the analogous question in the stronger context.

math.AG↗

arXiv2018

Learning Optical Flow via Dilated Networks and Occlusion Reasoning

Despite the significant progress that has been made on estimating optical flow recently, most estimation methods, including classical and deep learning approaches, still have difficulty with multi-scale estimation, real-time computation, and/or occlusion reasoning. In this paper, we introduce dilated convolution and occlusion reasoning into unsupervised optical flow estimation to address these issues. The dilated convolution allows our network to avoid upsampling via deconvolution and the resulting gridding artifacts. Dilated convolution also results in a smaller memory footprint which speeds up interference. The occlusion reasoning prevents our network from learning incorrect deformations due to occluded image regions during training. Our proposed method outperforms state-of-the-art unsupervised approaches on the KITTI benchmark. We also demonstrate its generalization capability by applying it to action recognition in video.

cs.CV↗

arXiv2018

Spatial Morphing Kernel Regression For Feature Interpolation

In recent years, geotagged social media has become popular as a novel source for geographic knowledge discovery. Ground-level images and videos provide a different perspective than overhead imagery and can be applied to a range of applications such as land use mapping, activity detection, pollution mapping, etc. The sparse and uneven distribution of this data presents a problem, however, for generating dense maps. We therefore investigate the problem of spatially interpolating the high-dimensional features extracted from sparse social media to enable dense labeling using standard classifiers. Further, we show how prior knowledge about region boundaries can be used to improve the interpolation through spatial morphing kernel regression. We show that an interpolate-then-classify framework can produce dense maps from sparse observations but that care must be taken in choosing the interpolation method. We also show that the spatial morphing kernel improves the results.

cs.CV↗

arXiv2018

Parsing Tweets into Universal Dependencies

We study the problem of analyzing tweets with Universal Dependencies. We extend the UD guidelines to cover special constructions in tweets that affect tokenization, part-of-speech tagging, and labeled dependencies. Using the extended guidelines, we create a new tweet treebank for English (Tweebank v2) that is four times larger than the (unlabeled) Tweebank v1 introduced by Kong et al. (2014). We characterize the disagreements between our annotators and show that it is challenging to deliver consistent annotation due to ambiguity in understanding and explaining tweets. Nonetheless, using the new treebank, we build a pipeline system to parse raw tweets into UD. To overcome annotation noise without sacrificing computational efficiency, we propose a new method to distill an ensemble of 20 transition-based parsers into a single one. Our parser achieves an improvement of 2.2 in LAS over the un-ensembled baseline and outperforms parsers that are state-of-the-art on other treebanks in both accuracy and speed.

cs.CL↗

arXiv2018

Weakly Supervised Instance Segmentation using Class Peak Response

Weakly supervised instance segmentation with image-level labels, instead of expensive pixel-level masks, remains unexplored. In this paper, we tackle this challenging problem by exploiting class peak responses to enable a classification network for instance mask extraction. With image labels supervision only, CNN classifiers in a fully convolutional manner can produce class response maps, which specify classification confidence at each image location. We observed that local maximums, i.e., peaks, in a class response map typically correspond to strong visual cues residing inside each instance. Motivated by this, we first design a process to stimulate peaks to emerge from a class response map. The emerged peaks are then back-propagated and effectively mapped to highly informative regions of each object instance, such as instance boundaries. We refer to the above maps generated from class peak responses as Peak Response Maps (PRMs). PRMs provide a fine-detailed instance-level representation, which allows instance masks to be extracted even with some off-the-shelf methods. To the best of our knowledge, we for the first time report results for the challenging image-level supervised instance segmentation task. Extensive experiments show that our method also boosts weakly supervised pointwise localization as well as semantic segmentation performance, and reports state-of-the-art results on popular benchmarks, including PASCAL VOC 2012 and MS COCO.

cs.CV↗

arXiv2018

Towards Universal Representation for Unseen Action Recognition

Unseen Action Recognition (UAR) aims to recognise novel action categories without training examples. While previous methods focus on inner-dataset seen/unseen splits, this paper proposes a pipeline using a large-scale training source to achieve a Universal Representation (UR) that can generalise to a more realistic Cross-Dataset UAR (CD-UAR) scenario. We first address UAR as a Generalised Multiple-Instance Learning (GMIL) problem and discover 'building-blocks' from the large-scale ActivityNet dataset using distribution kernels. Essential visual and semantic components are preserved in a shared space to achieve the UR that can efficiently generalise to new datasets. Predicted UR exemplars can be improved by a simple semantic adaptation, and then an unseen action can be directly recognised using UR during the test. Without further training, extensive experiments manifest significant improvements over the UCF101 and HMDB51 benchmarks.

cs.CV↗

arXiv2018

Global small solutions of 3D incompressible Oldroyd-B model without damping mechanism

In this paper, we prove the global existence of small smooth solutions to the three-dimensional incompressible Oldroyd-B model without damping on the stress tensor. The main difficulty is the lack of full dissipation in stress tensor. To overcome it, we construct some time-weighted energies based on the special coupled structure of system. Such type energies show the partial dissipation of stress tensor and the strongly full dissipation of velocity. In the view of treating "nonlinear term" as a "linear term", we also apply this result to 3D incompressible viscoelastic system with Hookean elasticity and then prove the global existence of small solutions without the physical assumption (div-curl structure) as previous works.

math.AP↗

arXiv2018

Global classical solutions of 3D viscous MHD system without magnetic diffusion on periodic boxes

In this paper, we study the global existence of classical solutions to the three dimensional incompressible viscous magneto-hydrodynamical system without magnetic diffusion on periodic boxes, i.e., with periodic boundary conditions. We work in Eulerian coordinate and employ a time-weighted energy estimate to prove the global existence result, under the assumptions that the initial magnetic field is close enough to an equilibrium state and the initial data have some symmetries.

math.AP↗

arXiv2018

Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images

We perform fine-grained land use mapping at the city scale using ground-level images. Mapping land use is considerably more difficult than mapping land cover and is generally not possible using overhead imagery as it requires close-up views and seeing inside buildings. We postulate that the growing collections of georeferenced, ground-level images suggest an alternate approach to this geographic knowledge discovery problem. We develop a general framework that uses Flickr images to map 45 different land-use classes for the City of San Francisco. Individual images are classified using a novel convolutional neural network containing two streams, one for recognizing objects and another for recognizing scenes. This network is trained in an end-to-end manner directly on the labeled training images. We propose several strategies to overcome the noisiness of our user-generated data including search-based training set augmentation and online adaptive training. We derive a ground truth map of San Francisco in order to evaluate our method. We demonstrate the effectiveness of our approach through geo-visualization and quantitative analysis. Our framework achieves over 29% recall at the individual land parcel level which represents a strong baseline for the challenging 45-way land use classification problem especially given the noisiness of the image data.

cs.CV↗

arXiv2018

Radio-frequency capacitive gate-based sensing

Developing fast, accurate and scalable techniques for quantum state readout is an active area in semiconductor-based quantum computing. Here, we present results on dispersive sensing of silicon corner state quantum dots coupled to lumped-element electrical resonators via the gate. The gate capacitance of the quantum device is configured in parallel with a superconducting spiral inductor resulting in resonators with loaded Q-factors in the 400-800 range. For a resonator operating at 330 MHz, we achieve a charge sensitivity of 7.7 $μ$e$/\sqrt{\text{Hz}}$ and, when operating at 616 MHz, we get 1.3 $μ$e$/\sqrt{\text{Hz}}$. We perform a parametric study of the resonator to reveal its optimal operation points and perform a circuit analysis to determine the best resonator design. The results place gate-based sensing at par with the best reported radio-frequency single-electron transistor sensitivities while providing a fast and compact method for quantum state readout.

physics.app-ph↗