arXiv subjects

Jianfeng Wang

Publications and source records attributed to Jianfeng Wang.

At least 127 records · Page 7Linked to original sources

arXiv2021

RSG: A Simple but Effective Module for Learning Imbalanced Datasets

Imbalanced datasets widely exist in practice and area great challenge for training deep neural models with agood generalization on infrequent classes. In this work, wepropose a new rare-class sample generator (RSG) to solvethis problem. RSG aims to generate some new samplesfor rare classes during training, and it has in particularthe following advantages: (1) it is convenient to use andhighly versatile, because it can be easily integrated intoany kind of convolutional neural network, and it works wellwhen combined with different loss functions, and (2) it isonly used during the training phase, and therefore, no ad-ditional burden is imposed on deep neural networks duringthe testing phase. In extensive experimental evaluations, weverify the effectiveness of RSG. Furthermore, by leveragingRSG, we obtain competitive results on Imbalanced CIFARand new state-of-the-art results on Places-LT, ImageNet-LT, and iNaturalist 2018. The source code is available at https://github.com/Jianf-Wang/RSG.

cs.CV↗

arXiv2021

Convolutional Neural Networks with Gated Recurrent Connections

The convolutional neural network (CNN) has become a basic model for solving many computer vision problems. In recent years, a new class of CNNs, recurrent convolution neural network (RCNN), inspired by abundant recurrent connections in the visual systems of animals, was proposed. The critical element of RCNN is the recurrent convolutional layer (RCL), which incorporates recurrent connections between neurons in the standard convolutional layer. With increasing number of recurrent computations, the receptive fields (RFs) of neurons in RCL expand unboundedly, which is inconsistent with biological facts. We propose to modulate the RFs of neurons by introducing gates to the recurrent connections. The gates control the amount of context information inputting to the neurons and the neurons' RFs therefore become adaptive. The resulting layer is called gated recurrent convolution layer (GRCL). Multiple GRCLs constitute a deep model called gated RCNN (GRCNN). The GRCNN was evaluated on several computer vision tasks including object recognition, scene text recognition and object detection, and obtained much better results than the RCNN. In addition, when combined with other adaptive RF techniques, the GRCNN demonstrated competitive performance to the state-of-the-art models on benchmark datasets for these tasks. The codes are released at \href{https://github.com/Jianf-Wang/GRCNN}{https://github.com/Jianf-Wang/GRCNN}.

cs.CV↗

arXiv2021

SEED: Self-supervised Distillation For Visual Representation

This paper is concerned with self-supervised learning for small models. The problem is motivated by our empirical studies that while the widely used contrastive self-supervised learning method has shown great progress on large model training, it does not work well for small models. To address this problem, we propose a new learning paradigm, named SElf-SupErvised Distillation (SEED), where we leverage a larger network (as Teacher) to transfer its representational knowledge into a smaller architecture (as Student) in a self-supervised fashion. Instead of directly learning from unlabeled data, we train a student encoder to mimic the similarity score distribution inferred by a teacher over a set of instances. We show that SEED dramatically boosts the performance of small networks on downstream tasks. Compared with self-supervised baselines, SEED improves the top-1 accuracy from 42.2% to 67.6% on EfficientNet-B0 and from 36.3% to 68.2% on MobileNet-v3-Large on the ImageNet-1k dataset.

cs.CV↗

arXiv2021

Compressing Visual-linguistic Model via Knowledge Distillation

Despite exciting progress in pre-training for visual-linguistic (VL) representations, very few aspire to a small VL model. In this paper, we study knowledge distillation (KD) to effectively compress a transformer-based large VL model into a small VL model. The major challenge arises from the inconsistent regional visual tokens extracted from different detectors of Teacher and Student, resulting in the misalignment of hidden representations and attention distributions. To address the problem, we retrain and adapt the Teacher by using the same region proposals from Student's detector while the features are from Teacher's own object detector. With aligned network inputs, the adapted Teacher is capable of transferring the knowledge through the intermediate representations. Specifically, we use the mean square error loss to mimic the attention distribution inside the transformer block and present a token-wise noise contrastive loss to align the hidden state by contrasting with negative representations stored in a sample queue. To this end, we show that our proposed distillation significantly improves the performance of small VL models on image captioning and visual question answering tasks. It reaches 120.8 in CIDEr score on COCO captioning, an improvement of 5.1 over its non-distilled counterpart; and an accuracy of 69.8 on VQA 2.0, a 0.8 gain from the baseline. Our extensive experiments and ablations confirm the effectiveness of VL distillation in both pre-training and fine-tuning stages.

cs.CV↗

arXiv2021

DAP: Detection-Aware Pre-training with Weak Supervision

This paper presents a detection-aware pre-training (DAP) approach, which leverages only weakly-labeled classification-style datasets (e.g., ImageNet) for pre-training, but is specifically tailored to benefit object detection tasks. In contrast to the widely used image classification-based pre-training (e.g., on ImageNet), which does not include any location-related training tasks, we transform a classification dataset into a detection dataset through a weakly supervised object localization method based on Class Activation Maps to directly pre-train a detector, making the pre-trained model location-aware and capable of predicting bounding boxes. We show that DAP can outperform the traditional classification pre-training in terms of both sample efficiency and convergence speed in downstream detection tasks including VOC and COCO. In particular, DAP boosts the detection accuracy by a large margin when the number of examples in the downstream task is small.

cs.CV↗

arXiv2021

End-to-End Object Detection with Fully Convolutional Network

Mainstream object detectors based on the fully convolutional network has achieved impressive performance. While most of them still need a hand-designed non-maximum suppression (NMS) post-processing, which impedes fully end-to-end training. In this paper, we give the analysis of discarding NMS, where the results reveal that a proper label assignment plays a crucial role. To this end, for fully convolutional detectors, we introduce a Prediction-aware One-To-One (POTO) label assignment for classification to enable end-to-end detection, which obtains comparable performance with NMS. Besides, a simple 3D Max Filtering (3DMF) is proposed to utilize the multi-scale features and improve the discriminability of convolutions in the local region. With these techniques, our end-to-end framework achieves competitive performance against many state-of-the-art detectors with NMS on COCO and CrowdHuman datasets. The code is available at https://github.com/Megvii-BaseDetection/DeFCN .

cs.CV↗

arXiv2021

Adversarial Feature Augmentation and Normalization for Visual Recognition

Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings, instead of relying on computationally-expensive pixel-level perturbations. We propose Adversarial Feature Augmentation and Normalization (A-FAN), which (i) first augments visual recognition models with adversarial features that integrate flexible scales of perturbation strengths, (ii) then extracts adversarial feature statistics from batch normalization, and re-injects them into clean features through feature normalization. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks, including ResNets and EfficientNets for classification, Faster-RCNN for detection, and Deeplab V3+ for segmentation. Extensive experiments show that A-FAN yields consistent generalization improvement over strong baselines across various datasets for classification, detection and segmentation tasks, such as CIFAR-10, CIFAR-100, ImageNet, Pascal VOC2007, Pascal VOC2012, COCO2017, and Cityspaces. Comprehensive ablation studies and detailed analyses also demonstrate that adding perturbations to specific modules and layers of classification/detection/segmentation backbones yields optimal performance. Codes and pre-trained models will be made available at: https://github.com/VITA-Group/CV_A-FAN.

cs.LG↗

arXiv2021

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection

Label assignment has been widely studied in general object detection because of its great impact on detectors' performance. However, none of these works focus on label assignment in dense pedestrian detection. In this paper, we propose a simple yet effective assigning strategy called Loss-aware Label Assignment (LLA) to boost the performance of pedestrian detectors in crowd scenarios. LLA first calculates classification (cls) and regression (reg) losses between each anchor and ground-truth (GT) pair. A joint loss is then defined as the weighted summation of cls and reg losses as the assigning indicator. Finally, anchors with top K minimum joint losses for a certain GT box are assigned as its positive anchors. Anchors that are not assigned to any GT box are considered negative. Loss-aware label assignment is based on an observation that anchors with lower joint loss usually contain richer semantic information and thus can better represent their corresponding GT boxes. Experiments on CrowdHuman and CityPersons show that such a simple label assigning strategy can boost MR by 9.53% and 5.47% on two famous one-stage detectors - RetinaNet and FCOS, respectively, demonstrating the effectiveness of LLA.

cs.CV↗

arXiv2021

A best bound for $λ_2(G)$ to guarantee $κ(G) \geq 2$

Let $G$ be a connected $d$-regular graph with a given order and the second largest eigenvalue $λ_2(G)$. Mohar and O (private communication) asked a challenging problem: what is the best upper bound for $λ_2(G)$ which guarantees that $κ(G) \geq t+1$, where $1 \leq t \leq d-1$ and $κ(G)$ is the vertex-connectivity of $G$, which was also mentioned by Cioabă. As a starting point, we solve this problem in the case $t =1$, and characterize all families of extremal graphs.

math.CO↗

arXiv2021

Galleon: Reshaping the Square Peg of NFV

Software is often used for Network Functions (NFs) -- such as firewalls, NAT, deep packet inspection, and encryption -- that are applied to traffic in the network. The community has hoped that NFV would enable rapid development of new NFs and leverage commodity computing infrastructure. However, the challenge for researchers and operators has been to align the square peg of high-speed packet processing with the round hole of cloud computing infrastructures and abstractions, all while delivering performance, scalability, and isolation. Past work has led to the belief that NFV is different enough that it requires novel, custom approaches that deviate from today's norms. To the contrary, we show that we can achieve performance, scalability, and isolation in NFV judiciously using mechanisms and abstractions of FaaS, the Linux kernel, NIC hardware, and OpenFlow switches. As such, with our system Galleon, NFV can be practically-deployable today in conventional cloud environments while delivering up to double the performance per core compared to the state of the art.

cs.DC↗

arXiv2020

A Hoffman's Theorem: a revisit with new discovery

In 1972, A. J. Hoffman proved his celebrated theorem concerning the limit points of spectral radii of non-negative symmetric integral matrices less than $\sqrt{2+\sqrt{5}}$. In this paper, after giving a new version of Hoffman's theorem, we get two generalized versions of it applicable to non-negative symmetric matrices with fractional elements. As a corollary, we obtain another alternative version about the limit points of spectral radii of (signless) Laplacian matrices of graphs less than $2+ {\tiny \frac{{\;}1{\;}}{3}\left((54 - 6\sqrt{33})^{\frac{{\;}1{\;}}{3}} + (54 + 6\sqrt{33})^{\frac{{\;} 1{\;}}{3}} \right)}$. We also discuss how our approach could be fruitfully employed to investigate equiangular lines.

math.CO↗

arXiv2020

The Hoffman program of graphs: old and new

The Hoffman program with respect to any real or complex square matrix $M$ associated to a graph $G$ stems from A. J. Hoffman's pioneering work on the limit points for the spectral radius of adjacency matrices of graphs less than $\sqrt{2+\sqrt{5}}$. The program consists of two aspects: finding all the possible limit points of $M$-spectral radii of graphs and detecting all the connected graphs whose $M$-spectral radius does not exceed a fixed limit point. In this paper, we summarize the results on this topic concerning several graph matrices, including the adjacency, the Laplacian, the signless Laplacian, the Hermitian adjacency and skew-adjacency matrix of graphs. As well, the tensors of hypergraphs are discussed. Moreover, we obtain new results about the Hoffman program with relation to the $A_α$-matrix. Some further problems on this topic are also proposed.

math.CO↗

arXiv2020

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

In this paper, we propose Text-Aware Pre-training (TAP) for Text-VQA and Text-Caption tasks. These two tasks aim at reading and understanding scene text in images for question answering and image caption generation, respectively. In contrast to the conventional vision-language pre-training that fails to capture scene text and its relationship with the visual and text modalities, TAP explicitly incorporates scene text (generated from OCR engines) in pre-training. With three pre-training tasks, including masked language modeling (MLM), image-text (contrastive) matching (ITM), and relative (spatial) position prediction (RPP), TAP effectively helps the model learn a better aligned representation among the three modalities: text word, visual object, and scene text. Due to this aligned representation learning, even pre-trained on the same downstream task dataset, TAP already boosts the absolute accuracy on the TextVQA dataset by +5.4%, compared with a non-TAP baseline. To further improve the performance, we build a large-scale dataset based on the Conceptual Caption dataset, named OCR-CC, which contains 1.4 million scene text-related image-text pairs. Pre-trained on this OCR-CC dataset, our approach outperforms the state of the art by large margins on multiple tasks, i.e., +8.3% accuracy on TextVQA, +8.6% accuracy on ST-VQA, and +10.2 CIDEr score on TextCaps.

cs.CV↗

arXiv2020

AutoAssign: Differentiable Label Assignment for Dense Object Detection

Determining positive/negative samples for object detection is known as label assignment. Here we present an anchor-free detector named AutoAssign. It requires little human knowledge and achieves appearance-aware through a fully differentiable weighting mechanism. During training, to both satisfy the prior distribution of data and adapt to category characteristics, we present Center Weighting to adjust the category-specific prior distributions. To adapt to object appearances, Confidence Weighting is proposed to adjust the specific assign strategy of each instance. The two weighting modules are then combined to generate positive and negative weights to adjust each location's confidence. Extensive experiments on the MS COCO show that our method steadily surpasses other best sampling strategies by large margins with various backbones. Moreover, our best model achieves 52.1% AP, outperforming all existing one-stage detectors. Besides, experiments on other datasets, e.g., PASCAL VOC, Objects365, and WiderFace, demonstrate the broad applicability of AutoAssign.

cs.CV↗

arXiv2020

Density-independent plasmons for terahertz-stable topological metamaterials

To efficiently integrate cutting-edge terahertz technology into compact devices, the highly confined terahertz plasmons are attracting intensive attentions. Compared to plasmons at visible frequencies in metals, terahertz plasmons, typically in lightly doped semiconductors or graphene, are sensitive to carrier density (n) and thus have an easy tunability, which, however, leads to unstable or imprecise terahertz spectra. By deriving a simplified but universal form of plasmon frequencies, here we reveal a unified mechanism for generating unusual n-independent plasmons (DIPs) in all topological states with different dimensions. Remarkably, we predict that terahertz DIPs can be excited in 2D nodal-line and 1D nodal-point systems, confirmed by the first-principles calculations on almost all existing topological semimetals with diverse lattice symmetries. Besides of n independence, the feature of Fermi-velocity and degeneracy-factor dependences in DIPs can be applied to design topological superlattice and multi-walled carbon nanotube metamaterials for broadband terahertz spectroscopy and quantized terahertz plasmons, respectively. Surprisingly, high spatial confinement and quality factor, also insensitive to n, can be simultaneously achieved in these terahertz DIPs. Our findings pave the way to developing topological plasmonic devices for stable terahertz applications.

physics.optics↗

arXiv2020

Effects of winter climate on high speed passenger trains in Botnia-Atlantica region

Harsh winter climate can cause various problems for both public and private sectors in Sweden, especially in the northern part for railway industry. To have a better understanding of winter climate impacts, this study investigates effects of the winter climate including atmospheric icing on the performance of high speed passenger trains in the Botnia-Atlantica region. The investigation is done with train operational data together with simulated weather data from the Weather Research and Forecast model over January - February 2017. Two different measurements of the train performance are analysed. One is cumulative delay which measures the increment in delay in terms of running time within two consecutive measuring spots, the other is current delay which is the delay in terms of arrival time at each measuring spot compared to the schedule. Cumulative delay is investigated through a Cox model and the current delay is studied using a Markov chain model. The results show that the weather factors have impacts on the train performance. Therein temperature and humidity have significant impacts on both the occurrence of cumulative delay and the transition probabilities between (current) delayed and non-delayed states.

stat.AP↗

arXiv2020

Realization of Semiconducting Layered Multiferroic Heterojunctions via Asymmetrical Magnetoelectric Coupling

Two-dimensional (2D) semiconducting multiferroics that can effectively couple magnetic and polarization (P) orders have great interest for both fundamental research and technological applications in nanoscale, which are, however, rare in nature. In this study, we propose a general mechanism to realize semiconducting 2D multiferroics via vdW heterojunction engineering, as demonstrated in a typical heterostructure consisting of magnetic bilayer CrI3 (bi-CrI3) and ferroelectric monolayer In2Se3. Interestingly, the novel indirect orbital coupling between Se 4p and Cr 3d orbitals, intermediated by the interfacial I 5p orbitals, are switchable in the opposite P configurations, resulting in an unexpected mechanism of strong asymmetrical magnetoelectric coupling. Therefore, along with the noticeable ferroelectric energy barrier induced by In2Se3, the realization of opposite magnetic orders in opposite P configurations can eventually result in the novel multiferroicity in bi-CrI3/In2Se3. Finally, we demonstrate that our mechanism can generally be applied to design other vdW multiferroics even with tunable layer thickness.

cond-mat.mtrl-sci↗

arXiv2020

Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfer

In this paper, we propose an effective knowledge transfer framework to boost the weakly supervised object detection accuracy with the help of an external fully-annotated source dataset, whose categories may not overlap with the target domain. This setting is of great practical value due to the existence of many off-the-shelf detection datasets. To more effectively utilize the source dataset, we propose to iteratively transfer the knowledge from the source domain by a one-class universal detector and learn the target-domain detector. The box-level pseudo ground truths mined by the target-domain detector in each iteration effectively improve the one-class universal detector. Therefore, the knowledge in the source dataset is more thoroughly exploited and leveraged. Extensive experiments are conducted with Pascal VOC 2007 as the target weakly-annotated dataset and COCO/ImageNet as the source fully-annotated dataset. With the proposed solution, we achieved an mAP of $59.7\%$ detection performance on the VOC test set and an mAP of $60.2\%$ after retraining a fully supervised Faster RCNN with the mined pseudo ground truths. This is significantly better than any previously known results in related literature and sets a new state-of-the-art of weakly supervised object detection under the knowledge transfer setting. Code: \url{https://github.com/mikuhatsune/wsod_transfer}.

cs.CV↗