arXiv subjects

Jing Liu

Publications and source records attributed to Jing Liu.

At least 559 records · Page 31Linked to original sources

arXiv2021

Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations

This paper tackles the problem of training a deep convolutional neural network of both low-bitwidth weights and activations. Optimizing a low-precision network is very challenging due to the non-differentiability of the quantizer, which may result in substantial accuracy loss. To address this, we propose three practical approaches, including (i) progressive quantization; (ii) stochastic precision; and (iii) joint knowledge distillation to improve the network training. First, for progressive quantization, we propose two schemes to progressively find good local minima. Specifically, we propose to first optimize a net with quantized weights and subsequently quantize activations. This is in contrast to the traditional methods which optimize them simultaneously. Furthermore, we propose a second progressive quantization scheme which gradually decreases the bit-width from high-precision to low-precision during training. Second, to alleviate the excessive training burden due to the multi-round training stages, we further propose a one-stage stochastic precision strategy to randomly sample and quantize sub-networks while keeping other parts in full-precision. Finally, we adopt a novel learning scheme to jointly train a full-precision model alongside the low-precision one. By doing so, the full-precision model provides hints to guide the low-precision model training and significantly improves the performance of the low-precision network. Extensive experiments on various datasets (e.g., CIFAR-100, ImageNet) show the effectiveness of the proposed methods.

cs.CV↗

arXiv2021

Boosting the Performance of Video Compression Artifact Reduction with Reference Frame Proposals and Frequency Domain Information

Many deep learning based video compression artifact removal algorithms have been proposed to recover high-quality videos from low-quality compressed videos. Recently, methods were proposed to mine spatiotemporal information via utilizing multiple neighboring frames as reference frames. However, these post-processing methods take advantage of adjacent frames directly, but neglect the information of the video itself, which can be exploited. In this paper, we propose an effective reference frame proposal strategy to boost the performance of the existing multi-frame approaches. Besides, we introduce a loss based on fast Fourier transformation~(FFT) to further improve the effectiveness of restoration. Experimental results show that our method achieves better fidelity and perceptual performance on MFQE 2.0 dataset than the state-of-the-art methods. And our method won Track 1 and Track 2, and was ranked the 2nd in Track 3 of NTIRE 2021 Quality enhancement of heavily compressed videos Challenge.

eess.IV↗

arXiv2021

Large-Scale Data-Driven Airline Market Influence Maximization

We present a prediction-driven optimization framework to maximize the market influence in the US domestic air passenger transportation market by adjusting flight frequencies. At the lower level, our neural networks consider a wide variety of features, such as classical air carrier performance features and transportation network features, to predict the market influence. On top of the prediction models, we define a budget-constrained flight frequency optimization problem to maximize the market influence over 2,262 routes. This problem falls into the category of the non-linear optimization problem, which cannot be solved exactly by conventional methods. To this end, we present a novel adaptive gradient ascent (AGA) method. Our prediction models show two to eleven times better accuracy in terms of the median root-mean-square error (RMSE) over baselines. In addition, our AGA optimization method runs 690 times faster with a better optimization result (in one of our largest scale experiments) than a greedy algorithm.

cs.LG↗

arXiv2021

InAs nanocrystals with robust p-type doping

Robust control over the carrier type is fundamental for the fabrication of nanocrystal-based optoelectronic devices, such as the p-n homojunction, but effective incorporation of impurities in semiconductor nanocrystals and its characterization is highly challenging due to their small size. Herein, InAs nanocrystals, post-synthetically doped with Cd, serve as a model system for successful p-type doping of originally n-type InAs nanocrystals, as demonstrated in field-effect transistors (FETs). Advanced structural analysis, using atomic resolution electron microscopy and synchrotron X-ray absorption fine structure spectroscopy reveal that Cd impurities reside near and on the nanocrystal surface acting as substitutional p-dopants replacing Indium. Commensurately, Cd-doped InAs FETs exhibited remarkable stability of their hole conduction, mobility, and hysteretic behavior over time when exposed to air, while intrinsic InAs NCs FETs were easily oxidized and their performance quickly declined. Therefore, Cd plays a dual role acting as a p-type dopant, and also protects the nanocrystals from oxidation, as evidenced directly by Xray photoelectron spectroscopy measurements of air-exposed samples of intrinsic and Cd doped InAs NCs films. This study demonstrates robust p-type doping of InAs nanocrystals, setting the stage for implementation of such doped nanocrystal systems in printed electronic devices.

cond-mat.mtrl-sci↗

arXiv2021

RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever.

cs.CL↗

arXiv2021

Orbital driven two-dome superconducting phases in multiorbital superconductors

We theoretically study the superconductivity in multiorbital superconductors based on a three-orbital tight-banding model. With appropriate values of the nearest-neighbour exchange $J_{1}^{αβ}$ and the next-nearest-neighbour exchange $J_{2}^{αβ}$, we find a two-dome structure in the $T_{c}-n$ phase diagram: one dome in the doping range $n<3.9$ where the superconducting (SC) state is mainly $s_{x^{2} y^{2}}$ component contributed by inter-orbital pairing, the other dome in the doping range $3.9<n<4.46$ where the SC state is mainly $s_{x^{2} y^{2}}+s_{x^{2}+y^{2}}$ components contributed by intra-orbital pairing. We find that the competition between different orbital pairing leads to two-dome SC phase diagrams in multiorbital superconductors, and different matrix elements of $J_{1}$ and $J_{2}$ considerably affect the boundary of two SC domes.

cond-mat.supr-con↗

arXiv2021

Discrimination-aware Network Pruning for Deep Model Compression

We study network pruning which aims to remove redundant channels/kernels and hence speed up the inference of deep networks. Existing pruning methods either train from scratch with sparsity constraints or minimize the reconstruction error between the feature maps of the pre-trained models and the compressed ones. Both strategies suffer from some limitations: the former kind is computationally expensive and difficult to converge, while the latter kind optimizes the reconstruction error but ignores the discriminative power of channels. In this paper, we propose a simple-yet-effective method called discrimination-aware channel pruning (DCP) to choose the channels that actually contribute to the discriminative power. Note that a channel often consists of a set of kernels. Besides the redundancy in channels, some kernels in a channel may also be redundant and fail to contribute to the discriminative power of the network, resulting in kernel level redundancy. To solve this, we propose a discrimination-aware kernel pruning (DKP) method to further compress deep networks by removing redundant kernels. To prevent DCP/DKP from selecting redundant channels/kernels, we propose a new adaptive stopping condition, which helps to automatically determine the number of selected channels/kernels and often results in more compact models with better performance. Extensive experiments on both image classification and face recognition demonstrate the effectiveness of our methods. For example, on ILSVRC-12, the resultant ResNet-50 model with 30% reduction of channels even outperforms the baseline model by 0.36% in terms of Top-1 accuracy. The pruned MobileNetV1 and MobileNetV2 achieve 1.93x and 1.42x inference acceleration on a mobile device, respectively, with negligible performance degradation. The source code and the pre-trained models are available at https://github.com/SCUT-AILab/DCP.

cs.CV↗

arXiv2021

Resonance instability of primordial gravitational waves during inflation in Chern-Simons gravity

We investigate axion inflation where the gravitational Chern-Simons term is coupled to a periodic function of the inflaton. We find that tensor perturbations with different polarizations are amplified in different ways by the Chern-Simons coupling. Depending on the model parameters, the resonance amplification results in a parity-violating peak or a board plateau in the energy spectrum of gravitational waves, and the sharp cutoff in the infrared region constitutes a characteristic distinguishable from stochastic gravitational wave backgrounds produced by matter fields in Einstein gravity.

gr-qc↗

arXiv2021

Momentum-Resolved Visualization of Electronic Evolution in Doping a Mott Insulator

High temperature superconductivity in cuprates arises from doping a parent Mott insulator by electrons or holes. A central issue is how the Mott gap evolves and the low-energy states emerge with doping. Here we report angle-resolved photoemission spectroscopy measurements on a cuprate parent compound by sequential in situ electron doping. The chemical potential jumps to the bottom of the upper Hubbard band upon a slight electron doping, making it possible to directly visualize the charge transfer band and the full Mott gap region. With increasing doping, the Mott gap rapidly collapses due to the spectral weight transfer from the charge transfer band to the gapped region and the induced low-energy states emerge in a wide energy range inside the Mott gap. These results provide key information on the electronic evolution in doping a Mott insulator and establish a basis for developing microscopic theories for cuprate superconductivity.

cond-mat.str-el↗

arXiv2021

Generation and storage of spin squeezing via learning-assisted optimal control

The generation and storage of spin squeezing is an attracting topic in quantum metrology and the foundations of quantum mechanics. The major models to realize the spin squeezing are the one- and two-axis twisting models. Here, we consider a collective spin system coupled to a bosonic field, and show that proper constant-value controls in this model can simulate the dynamical behaviors of these two models. More interestingly, a better performance of squeezing can be obtained when the control is time-varying, which is generated via a reinforcement learning algorithm. However, this advantage becomes limited if the collective noise is involved. To deal with it, we propose a four-step strategy for the construction of a new type of combined controls, which include both constant-value and time-varying controls, but performed at different time intervals. Compared to the full time-varying controls, the combined controls not only give a comparable minimum value of the squeezing parameter over time, but also provides a better lifetime and larger full amount of squeezing. Moreover, the amplitude form of a combined control is simpler and more stable than the full time-varying control. Therefore, our scheme is very promising to be applied in practice to improve the generation and storage performance of squeezing.

quant-ph↗

arXiv2021

CPTR: Full Transformer Network for Image Captioning

In this paper, we consider the image captioning task from a new sequence-to-sequence prediction perspective and propose CaPtion TransformeR (CPTR) which takes the sequentialized raw images as the input to Transformer. Compared to the "CNN+Transformer" design paradigm, our model can model global context at every encoder layer from the beginning and is totally convolution-free. Extensive experiments demonstrate the effectiveness of the proposed model and we surpass the conventional "CNN+Transformer" methods on the MSCOCO dataset. Besides, we provide detailed visualizations of the self-attention between patches in the encoder and the "words-to-patches" attention in the decoder thanks to the full Transformer architecture.

cs.CV↗

arXiv2021

Global-Local Propagation Network for RGB-D Semantic Segmentation

Depth information matters in RGB-D semantic segmentation task for providing additional geometric information to color images. Most existing methods exploit a multi-stage fusion strategy to propagate depth feature to the RGB branch. However, at the very deep stage, the propagation in a simple element-wise addition manner can not fully utilize the depth information. We propose Global-Local propagation network (GLPNet) to solve this problem. Specifically, a local context fusion module(L-CFM) is introduced to dynamically align both modalities before element-wise fusion, and a global context fusion module(G-CFM) is introduced to propagate the depth information to the RGB branch by jointly modeling the multi-modal global context features. Extensive experiments demonstrate the effectiveness and complementarity of the proposed fusion modules. Embedding two fusion modules into a two-stream encoder-decoder structure, our GLPNet achieves new state-of-the-art performance on two challenging indoor scene segmentation datasets, i.e., NYU-Depth v2 and SUN-RGBD dataset.

cs.CV↗

arXiv2021

Fast Sequence Generation with Multi-Agent Reinforcement Learning

Autoregressive sequence Generation models have achieved state-of-the-art performance in areas like machine translation and image captioning. These models are autoregressive in that they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. Recently, non-autoregressive decoding has been proposed in machine translation to speed up the inference time by generating all words in parallel. Typically, these models use the word-level cross-entropy loss to optimize each word independently. However, such a learning process fails to consider the sentence-level consistency, thus resulting in inferior generation quality of these non-autoregressive models. In this paper, we propose a simple and efficient model for Non-Autoregressive sequence Generation (NAG) with a novel training paradigm: Counterfactuals-critical Multi-Agent Learning (CMAL). CMAL formulates NAG as a multi-agent reinforcement learning system where element positions in the target sequence are viewed as agents that learn to cooperatively maximize a sentence-level reward. On MSCOCO image captioning benchmark, our NAG method achieves a performance comparable to state-of-the-art autoregressive models, while brings 13.9x decoding speedup. On WMT14 EN-DE machine translation dataset, our method outperforms cross-entropy trained baseline by 6.0 BLEU points while achieves the greatest decoding speedup of 17.46x.

cs.CL↗

arXiv2021

On the gravitational wave sources from the NANOGrav 12.5-yr data

The NANOGrav Collaboration recently reported a strong evidence for a stochastic common-spectrum process in the pulsar-timing data. We evaluate the evidence of interpreting this process as mergers of super massive black hole binaries and/or various stochastic gravitational wave background sources in the early Universe, including first-order phase transitions, cosmic strings, domain walls, and large amplitude curvature perturbations. We discuss the implications of the constraints on these possible sources. It is found that the cosmic string is the most favored source against other gravitational wave sources based on the Bayes factor analysis.

astro-ph.CO↗

arXiv2021

Current-induced magnetization switching in a chemically disordered A1 CoPt single layer

We report the first demonstration of the current-induced magnetization switching in a perpendicularly magnetized A1 CoPt single layer. We show that good perpendicular magnetic anisotropy can be obtained in a wide composition range of the A1 Co1-xPtx single layers, which allows to fabricate perpendicularly magnetized CoPt single layer with composition gradient to break the inversion symmetry of the structure. By fabricating the gradient CoPt single layer, we have evaluated the SOT efficiency and successfully realized the SOT-induced magnetization switching. Our study provides an approach to realize the current-induced magnetization in the ferromagnetic single layers without attaching SOT source materials.

cond-mat.mtrl-sci↗

arXiv2020

Large anisotropies of the stochastic gravitational wave background from cosmic domain walls

We investigate the stochastic gravitational wave background (SGWB) from cosmic domain walls (DWs) caused by quantum fluctuations of a light scalar field $ϕ$ during inflation. Perturbations of $ϕ$ remain almost constant after leaving the Hubble horizon. The probabilities of the two domains depend on the averaged value of $ϕ$ at each large scale region, leading to large scale perturbations of DW energy density and large anisotropies in the SGWB. We find that the angular power spectrum is scale-invariant and at least of the order of $10^{-2}$, which is expected to be detected by future gravitational-wave (GW) detectors. Since we have not detected primordial GWs yet, anisotropies of the SGWB could help us to verify the rationality of inflation and determine the energy scale of inflation.

astro-ph.CO↗

arXiv2020

Conditional Automated Channel Pruning for Deep Neural Networks

Model compression aims to reduce the redundancy of deep networks to obtain compact models. Recently, channel pruning has become one of the predominant compression methods to deploy deep models on resource-constrained devices. Most channel pruning methods often use a fixed compression rate for all the layers of the model, which, however, may not be optimal. To address this issue, given a target compression rate for the whole model, one can search for the optimal compression rate for each layer. Nevertheless, these methods perform channel pruning for a specific target compression rate. When we consider multiple compression rates, they have to repeat the channel pruning process multiple times, which is very inefficient yet unnecessary. To address this issue, we propose a Conditional Automated Channel Pruning(CACP) method to obtain the compressed models with different compression rates through single channel pruning process. To this end, we develop a conditional model that takes an arbitrary compression rate as input and outputs the corresponding compressed model. In the experiments, the resultant models with different compression rates consistently outperform the models compressed by existing methods with a channel pruning process for each target compression rate.

cs.CV↗

arXiv2020

Robust Mean Estimation in High Dimensions via $\ell_0$ Minimization

We study the robust mean estimation problem in high dimensions, where $α<0.5$ fraction of the data points can be arbitrarily corrupted. Motivated by compressive sensing, we formulate the robust mean estimation problem as the minimization of the $\ell_0$-`norm' of the outlier indicator vector, under second moment constraints on the inlier data points. We prove that the global minimum of this objective is order optimal for the robust mean estimation problem, and we propose a general framework for minimizing the objective. We further leverage the $\ell_1$ and $\ell_p$ $(0<p<1)$, minimization techniques in compressive sensing to provide computationally tractable solutions to the $\ell_0$ minimization problem. Both synthetic and real data experiments demonstrate that the proposed algorithms significantly outperform state-of-the-art robust mean estimation methods.

stat.ML↗