arXiv subjects

Feng Wu

Publications and source records attributed to Feng Wu.

At least 235 records · Page 13Linked to original sources

arXiv2019

Learning to Assemble Neural Module Tree Networks for Visual Grounding

Visual grounding, a task to ground (i.e., localize) natural language in images, essentially requires composite visual reasoning. However, existing methods over-simplify the composite nature of language into a monolithic sentence embedding or a coarse composition of subject-predicate-object triplet. In this paper, we propose to ground natural language in an intuitive, explainable, and composite fashion as it should be. In particular, we develop a novel modular network called Neural Module Tree network (NMTree) that regularizes the visual grounding along the dependency parsing tree of the sentence, where each node is a neural module that calculates visual attention according to its linguistic feature, and the grounding score is accumulated in a bottom-up direction where as needed. NMTree disentangles the visual grounding from the composite reasoning, allowing the former to only focus on primitive and easy-to-generalize patterns. To reduce the impact of parsing errors, we train the modules and their assembly end-to-end by using the Gumbel-Softmax approximation and its straight-through gradient estimator, accounting for the discrete nature of module assembly. Overall, the proposed NMTree consistently outperforms the state-of-the-arts on several benchmarks. Qualitative results show explainable grounding score calculation in great detail.

cs.CV↗

arXiv2019

LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network

In this paper, we explore the task of generating photo-realistic face images from lines. Previous methods based on conditional generative adversarial networks (cGANs) have shown their power to generate visually plausible images when a conditional image and an output image share well-aligned structures. However, these models fail to synthesize face images with a whole set of well-defined structures, e.g. eyes, noses, mouths, etc., especially when the conditional line map lacks one or several parts. To address this problem, we propose a conditional self-attention generative adversarial network (CSAGAN). We introduce a conditional self-attention mechanism to cGANs to capture long-range dependencies between different regions in faces. We also build a multi-scale discriminator. The large-scale discriminator enforces the completeness of global structures and the small-scale discriminator encourages fine details, thereby enhancing the realism of generated face images. We evaluate the proposed model on the CelebA-HD dataset by two perceptual user studies and three quantitative metrics. The experiment results demonstrate that our method generates high-quality facial images while preserving facial structures. Our results outperform state-of-the-art methods both quantitatively and qualitatively.

cs.CV↗

arXiv2019

Carrier recombination mechanism at defects in wide band gap two-dimensional materials from first principles

Identification and design of defects in two-dimensional (2D) materials as promising single photon emitters (SPE) requires a deep understanding of underlying carrier recombination mechanisms. Yet, the dominant mechanism of carrier recombination at defects in 2D materials has not been well understood, and some outstanding questions remain: How do recombination processes at defects differ between 2D and 3D systems? What factors determine defects in 2D materials as excellent SPE at room temperature? In order to address these questions, we developed first-principles methods to accurately calculate the radiative and non-radiative recombination rates at defects in 2D materials, using h-BN as a prototypical example. We reveal the carrier recombination mechanism at defects in 2D materials being mostly dominated by defect-defect state recombination in contrast to defect-bulk state recombination in most 3D semiconductors. In particular, we disentangle the non-radiative recombination mechanism into key physical quantities: zero-phonon line (ZPL) and Huang-Rhys factor. At the end, we identified strain can effectively tune the electron-phonon coupling at defect centers and drastically change non-radiative recombination rates. Our theoretical development serves as a general platform for understanding carrier recombination at defects in 2D materials, while providing pathways for engineering of quantum efficiency of SPE.

cond-mat.mtrl-sci↗

arXiv2019

Generalized Fault-Tolerance Topology Generation for Application Specific Network-on-Chips

The Network-on-Chips is a promising candidate for addressing communication bottlenecks in many-core processors and neural network processors. In this work, we consider the generalized fault-tolerance topology generation problem, where the link or switch failures can happen, for application-specific network-on-chips (ASNoC). With a user-defined number, K, we propose an integer linear programming (ILP) based method to generate ASNoC topologies, which can tolerate at most K faults in switches or links. Given the communication requirements between cores and their floorplan, we first propose a convex-cost-flow based method to solve a core mapping problem for building connections between the cores and switches. Second, an ILP based method is proposed to allocate K+1 switch-disjoint routing paths for every communication flow between the cores. Finally, to reduce switch sizes, we propose sharing the switch ports for the connections between the cores and switches and formulate the port sharing problem as a clique-partitioning problem Additionally, we propose an ILP-based method to simultaneously solve the core mapping and routing path allocation problems when considering physical link failures only. Experimental results show that the power consumptions of fault-tolerance topologies increase almost linearly with K because of the routing path redundancy. When both switch faults and link faults are considered, port sharing can reduce the average power consumption of fault-tolerance topologies with K = 1, K = 2 and K = 3 by 18.08%, 28.88%, and 34.20%, respectively. When considering only the physical link faults, the experimental results show that compared to the FTTG algorithm, the proposed method reduces power consumption and hop count by 10.58% and 6.25%, respectively; compared to the DBG based method, the proposed method reduces power consumption and hop count by 21.72% and 9.35%, respectively.

cs.AR↗

arXiv2019

Context-Aware Visual Policy Network for Fine-Grained Image Captioning

With the maturity of visual detection techniques, we are more ambitious in describing visual content with open-vocabulary, fine-grained and free-form language, i.e., the task of image captioning. In particular, we are interested in generating longer, richer and more fine-grained sentences and paragraphs as image descriptions. Image captioning can be translated to the task of sequential language prediction given visual content, where the output sequence forms natural language description with plausible grammar. However, existing image captioning methods focus only on language policy while not visual policy, and thus fail to capture visual context that are crucial for compositional reasoning such as object relationships (e.g., "man riding horse") and visual comparisons (e.g., "small(er) cat"). This issue is especially severe when generating longer sequences such as a paragraph. To fill the gap, we propose a Context-Aware Visual Policy network (CAVP) for fine-grained image-to-language generation: image sentence captioning and image paragraph captioning. During captioning, CAVP explicitly considers the previous visual attentions as context, and decides whether the context is used for the current word/sentence generation given the current visual attention. Compared against traditional visual attention mechanism that only fixes a single visual region at each step, CAVP can attend to complex visual compositions over time. The whole image captioning model -- CAVP and its subsequent language policy network -- can be efficiently optimized end-to-end by using an actor-critic policy gradient method. We have demonstrated the effectiveness of CAVP by state-of-the-art performances on MS-COCO and Stanford captioning datasets, using various metrics and sensible visualizations of qualitative visual context.

cs.CV↗

arXiv2019

Deep Learning-Based Video Coding: A Review and A Case Study

The past decade has witnessed great success of deep learning technology in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. This paper reviews the representative works about using deep learning for image/video coding, which has been an actively developing research area since the year of 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks (deep schemes), and deep network-based coding tools (deep tools) that shall be used within traditional coding schemes or together with traditional coding tools. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively. For deep tools, there have been several proposed techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, namely Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter (CNN-ILF) and CNN-based block adaptive resolution coding (CNN-BARC). Both tools help improve the compression efficiency by a significant margin. With the two deep tools as well as other non-deep coding tools, DLVC is able to achieve on average 39.6\% and 33.0\% bits saving than HEVC, under random-access and low-delay configurations, respectively. The source code of DLVC has been released for future researches.

cs.MM↗

arXiv2019

Camera Lens Super-Resolution

Existing methods for single image super-resolution (SR) are typically evaluated with synthetic degradation models such as bicubic or Gaussian downsampling. In this paper, we investigate SR from the perspective of camera lenses, named as CameraSR, which aims to alleviate the intrinsic tradeoff between resolution (R) and field-of-view (V) in realistic imaging systems. Specifically, we view the R-V degradation as a latent model in the SR process and learn to reverse it with realistic low- and high-resolution image pairs. To obtain the paired images, we propose two novel data acquisition strategies for two representative imaging systems (i.e., DSLR and smartphone cameras), respectively. Based on the obtained City100 dataset, we quantitatively analyze the performance of commonly-used synthetic degradation models, and demonstrate the superiority of CameraSR as a practical solution to boost the performance of existing SR methods. Moreover, CameraSR can be readily generalized to different content and devices, which serves as an advanced digital zoom tool in realistic imaging systems. Codes and datasets are available at https://github.com/ngchc/CameraSR.

cs.CV↗

arXiv2019

Dimensionality and Anisotropicity Dependence of Radiative Recombination in Nanostructured Phosphorene

The interplay between dimensionality and anisotropicity leads to intriguing optoelectronic properties and exciton dynamics in low dimensional semiconductors. In this study we use nanostructured phosphorene as a prototypical example to unfold such complex physics and develop a general first-principles framework to study exciton dynamics in low dimensional systems. Specifically we derived the radiative lifetime and light emission intensity from 2D to 0D systems based on many-body perturbation theory, and investigated the dimensionality and anisotropicity effects on radiative recombination lifetime both at 0 K and finite temperature, as well as polarization and angle dependence of emitted light. We show that the radiative lifetime at 0 K increases by an order of $10^3$ with the lowering of one dimension (i.e. from 2D to 1D nanoribbons or from 1D to 0D quantum dots). We also show that obtaining the radiative lifetime at finite temperature requires accurate exciton dispersion beyond the effective mass approximation. Finally, we demonstrate that monolayer phosphorene and its nanostructures always emit linearly polarized light consistent with experimental observations, different from in-plane isotropic 2D materials like MoS2 and h-BN that can emit light with arbitrary polarization, which may have important implications for quantum information applications.

cond-mat.mes-hall↗

arXiv2019

Towards Optimal Power Control via Ensembling Deep Neural Networks

A deep neural network (DNN) based power control method is proposed, which aims at solving the non-convex optimization problem of maximizing the sum rate of a multi-user interference channel. Towards this end, we first present PCNet, which is a multi-layer fully connected neural network that is specifically designed for the power control problem. PCNet takes the channel coefficients as input and outputs the transmit power of all users. A key challenge in training a DNN for the power control problem is the lack of ground truth, i.e., the optimal power allocation is unknown. To address this issue, PCNet leverages the unsupervised learning strategy and directly maximizes the sum rate in the training phase. Observing that a single PCNet does not globally outperform the existing solutions, we further propose ePCNet, a network ensemble with multiple PCNets trained independently. Simulation results show that for the standard symmetric multi-user Gaussian interference channel, ePCNet can outperform all state-of-the-art power control methods by 1.2%-4.6% under a variety of system configurations. Furthermore, the performance improvement of ePCNet comes with a reduced computational complexity.

eess.SP↗

arXiv2019

Learning Deterministic Policy with Target for Power Control in Wireless Networks

Inter-Cell Interference Coordination (ICIC) is a promising way to improve energy efficiency in wireless networks, especially where small base stations are densely deployed. However, traditional optimization based ICIC schemes suffer from severe performance degradation with complex interference pattern. To address this issue, we propose a Deep Reinforcement Learning with Deterministic Policy and Target (DRL-DPT) framework for ICIC in wireless networks. DRL-DPT overcomes the main obstacles in applying reinforcement learning and deep learning in wireless networks, i.e. continuous state space, continuous action space and convergence. Firstly, a Deep Neural Network (DNN) is involved as the actor to obtain deterministic power control actions in continuous space. Then, to guarantee the convergence, an online training process is presented, which makes use of a dedicated reward function as the target rule and a policy gradient descent algorithm to adjust DNN weights. Experimental results show that the proposed DRL-DPT framework consistently outperforms existing schemes in terms of energy efficiency and throughput under different wireless interference scenarios. More specifically, it improves up to 15% of energy efficiency with faster convergence rate.

cs.NI↗

arXiv2019

Learning for Video Compression

One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis, binarization, etc. Experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.

cs.MM↗

arXiv2018

Receiver-driven Video Multicast over NOMA Systems in Heterogeneous Environments

Non-orthogonal multiple access (NOMA) has shown potential for scalable multicast of video data. However, one key drawback for NOMA-based video multicast is the limited number of layers allowed by the embedded successive interference cancellation algorithm, failing to meet satisfaction of heterogeneous receivers. We propose a novel receiver-driven superposed video multicast (Supcast) scheme by integrating Softcast, an analog-like transmission scheme, into the NOMA-based system to achieve high bandwidth efficiency as well as gradual decoding quality proportional to channel conditions at receivers. Although Softcast allows gradual performance by directly transmitting power-scaled transformation coefficients of frames, it suffers performance degradation due to discarding coefficients under insufficient bandwidth and its power allocation strategy cannot be directly applied in NOMA due to interference. In Supcast, coefficients are grouped into chunks, which are basic units for power allocation and superposition scheduling. By bisecting chunks into base-layer chunks and enhanced-layer chunks, the joint power allocation and chunk scheduling is formulated as a distortion minimization problem. A two-stage power allocation strategy and a near-optimal low-complexity algorithm for chunk scheduling based on the matching theory are proposed. Simulation results have shown the advantage of Supcast against Softcast as well as the reference scheme in NOMA under various practical scenarios.

cs.MM↗

arXiv2018

A Two-Stream Mutual Attention Network for Semi-supervised Biomedical Segmentation with Noisy Labels

\begin{abstract} Learning-based methods suffer from a deficiency of clean annotations, especially in biomedical segmentation. Although many semi-supervised methods have been proposed to provide extra training data, automatically generated labels are usually too noisy to retrain models effectively. In this paper, we propose a Two-Stream Mutual Attention Network (TSMAN) that weakens the influence of back-propagated gradients caused by incorrect labels, thereby rendering the network robust to unclean data. The proposed TSMAN consists of two sub-networks that are connected by three types of attention models in different layers. The target of each attention model is to indicate potentially incorrect gradients in a certain layer for both sub-networks by analyzing their inferred features using the same input. In order to achieve this purpose, the attention models are designed based on the propagation analysis of noisy gradients at different layers. This allows the attention models to effectively discover incorrect labels and weaken their influence during the parameter updating process. By exchanging multi-level features within the two-stream architecture, the effects of noisy labels in each sub-network are reduced by decreasing the updating gradients. Furthermore, a hierarchical distillation is developed to provide more reliable pseudo labels for unlabelded data, which further boosts the performance of our retrained TSMAN. The experiments using both the HVSMR 2016 and BRATS 2015 benchmarks demonstrate that our semi-supervised learning framework surpasses the state-of-the-art fully-supervised results.

cs.CV↗

arXiv2018

Sequential Gating Ensemble Network for Noise Robust Multi-Scale Face Restoration

Face restoration from low resolution and noise is important for applications of face analysis recognition. However, most existing face restoration models omit the multiple scale issues in face restoration problem, which is still not well-solved in research area. In this paper, we propose a Sequential Gating Ensemble Network (SGEN) for multi-scale noise robust face restoration issue. To endow the network with multi-scale representation ability, we first employ the principle of ensemble learning for SGEN network architecture designing. The SGEN aggregates multi-level base-encoders and base-decoders into the network, which enables the network to contain multiple scales of receptive field. Instead of combining these base-en/decoders directly with non-sequential operations, the SGEN takes base-en/decoders from different levels as sequential data. Specifically, it is visualized that SGEN learns to sequentially extract high level information from base-encoders in bottom-up manner and restore low level information from base-decoders in top-down manner. Besides, we propose to realize bottom-up and top-down information combination and selection with Sequential Gating Unit (SGU). The SGU sequentially takes information from two different levels as inputs and decides the output based on one active input. Experiment results on benchmark dataset demonstrate that our SGEN is more effective at multi-scale human face restoration with more image details and less noise than state-of-the-art image restoration models. Further utilizing adversarial training scheme, SGEN also produces more visually preferred results than other models under subjective evaluation.

cs.CV↗

arXiv2018

Fundamental Principles for Calculating Charged Defect Ionization Energies in Ultrathin Two-Dimensional Materials

Defects in 2D materials are becoming prominent candidates for quantum emitters and scalable optoelectronic applications. However, several physical properties that characterize their behavior, such as charged defect ionization energies, are difficult to simulate with conventional first-principles methods, mainly because of the weak and anisotropic dielectric screening caused by the reduced dimensionality. We establish fundamental principles for accurate and efficient calculations of charged defect ionization energies and electronic structure in ultrathin 2D materials. We propose to use the vacuum level as the reference for defect charge transition levels (CTLs) because it gives robust results insensitive to the level of theory, unlike commonly used band edge positions. Furthermore, we determine the fraction of Fock exchange in hybrid functionals for accurate band gaps and band edge positions of 2D materials by enforcing the generalized Koopmans' condition of localized defect states. We found the obtained fractions of Fock exchange vary significantly from 0.2 for bulk $h$-BN to 0.4 for monolayer $h$-BN, whose band gaps are also in good agreement with experimental results and calculated GW results. The combination of these methods allows for reliable and efficient prediction of defect ionization energies (difference between CTLs and band edge positions). We motivate and generalize these findings with several examples including different defects in monolayer to few-layer hexagonal boron nitride ($h$-BN), monolayer MoS$_2$ and graphane. Finally, we show that increasing the number of layers of $h$-BN systematically lowers defect ionization energies, mainly through CTLs shifting towards vacuum, with conduction band minima kept almost unchanged.

cond-mat.mtrl-sci↗

arXiv2018

A Deep Tree-Structured Fusion Model for Single Image Deraining

We propose a simple yet effective deep tree-structured fusion model based on feature aggregation for the deraining problem. We argue that by effectively aggregating features, a relatively simple network can still handle tough image deraining problems well. First, to capture the spatial structure of rain we use dilated convolutions as our basic network block. We then design a tree-structured fusion architecture which is deployed within each block (spatial information) and across all blocks (content information). Our method is based on the assumption that adjacent features contain redundant information. This redundancy obstructs generation of new representations and can be reduced by hierarchically fusing adjacent features. Thus, the proposed model is more compact and can effectively use spatial and content information. Experiments on synthetic and real-world datasets show that our network achieves better deraining results with fewer parameters.

cs.CV↗

arXiv2018

FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions

Machine learning as a service has been widely deployed to utilize deep neural network models to provide prediction services. However, this raises privacy concerns since clients need to send sensitive information to servers. In this paper, we focus on the scenario where clients want to classify private images with a convolutional neural network model hosted in the server, while both parties keep their data private. We present FALCON, a fast and secure approach for CNN predictions based on Fourier Transform. Our solution enables linear layers of a CNN model to be evaluated simply and efficiently with fully homomorphic encryption. We also introduce the first efficient and privacy-preserving protocol for softmax function, which is an indispensable component in CNNs and has not yet been evaluated in previous works due to its high complexity. We implemented the FALCON and evaluated the performance on real-world CNN models. The experimental results show that FALCON outperforms the best known works in both computation and communication cost.

cs.CR↗

arXiv2018

Combining Landau Zener Theory and Kinetic Monte Carlo Sampling for Small Polaron Mobility of Doped BiVO4

Transition metal oxides such as \ce{BiVO4} are promising materials as photoelectrodes in solar-to-fuel conversion applications. However, their performance is limited by the low carrier mobility (especially electron mobility) due to the formation of small polarons. Recent experimental studies show improved carrier mobility and conductivity by atomic dopings. We studied the small polaron hopping mobility in pristine and doped \ce{BiVO4} by combining Landau-Zener theory and kinetic Monte Carlo (kMC) simulation fully from first-principles, and investigated the effect of dopant-polaron interactions on the mobility. We found polarons are spontaneously formed at V in both pristine and Mo/W doped \ce{BiVO4}, which can only be described correctly by density function theory (DFT) with the Hubbard correction (DFT+U) or hybrid exchange-correlation functional but not local or semi-local functionals. We found DFT+U and dielectric dependant hybrid functional (DDH) give similar electron hopping barriers, which are also similar between the room temperature monoclinic phase and the tetragonal phase. The calculated electron mobility agrees well with experimental values, which is around $10^{-4}$ cm$^2$V$^{-1}$s$^{-1}$. We found the electron polaron transport in \ce{BiVO4} is neither fully adiabatic nor nonadiabatic, and the first and second nearest neighbor hoppings have significantly different adiabacity in spite of similar hopping barriers. We further computed polaron mobility in the presence of different dopants and showed that Cr substitution of V is an electron trap while Mo and W are "repulsive" centers, mainly due to the minimization of local lattice expansion by dopants and electron polarons. The dopants with "repulsive" interactions to polarons are promising for mobility improvement due to larger wavefunction overlap and delocalization of locally concentrated polarons.

cond-mat.mtrl-sci↗