Searcharxiv⌕ Search

arXiv subjects

Wen Liu

Publications and source records attributed to Wen Liu.

66 records · Page 4Linked to original sources
arXiv2020

Channel Distillation: Channel-Wise Attention for Knowledge Distillation

Knowledge distillation is to transfer the knowledge from the data learned by the teacher network to the student network, so that the student has the advantage of less parameters and less calculations, and the accuracy is close to the teacher. In this paper, we propose a new distillation method, which contains two transfer distillation strategies and a loss decay strategy. The first transfer strategy is based on channel-wise attention, called Channel Distillation (CD). CD transfers the channel information from the teacher to the student. The second is Guided Knowledge Distillation (GKD). Unlike Knowledge Distillation (KD), which allows the student to mimic each sample's prediction distribution of the teacher, GKD only enables the student to mimic the correct output of the teacher. The last part is Early Decay Teacher (EDT). During the training process, we gradually decay the weight of the distillation loss. The purpose is to enable the student to gradually control the optimization rather than the teacher. Our proposed method is evaluated on ImageNet and CIFAR100. On ImageNet, we achieve 27.68% of top-1 error with ResNet18, which outperforms state-of-the-art methods. On CIFAR100, we achieve surprising result that the student outperforms the teacher. Code is available at https://github.com/zhouzaida/channel-distillation.

cs.LG↗
arXiv2019

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

We tackle the human motion imitation, appearance transfer, and novel view synthesis within a unified framework, which means that the model once being trained can be used to handle all these tasks. The existing task-specific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only expresses the position information with no abilities to characterize the personalized shape of the individual person and model the limbs rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape, which can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose a Liquid Warping GAN with Liquid Warping Block (LWB) that propagates the source information in both image and feature spaces, and synthesizes an image with respect to the reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method is able to support a more flexible warping from multiple sources. In addition, we build a new dataset, namely Impersonator (iPER) dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis. Extensive experiments demonstrate the effectiveness of our method in several aspects, such as robustness in occlusion case and preserving face identity, shape consistency and clothes details. All codes and datasets are available on https://svip-lab.github.io/project/impersonator.html

cs.CV↗
arXiv2018

Multi-Cell Multi-Task Convolutional Neural Networks for Diabetic Retinopathy Grading

Diabetic Retinopathy (DR) is a non-negligible eye disease among patients with Diabetes Mellitus, and automatic retinal image analysis algorithm for the DR screening is in high demand. Considering the resolution of retinal image is very high, where small pathological tissues can be detected only with large resolution image and large local receptive field are required to identify those late stage disease, but directly training a neural network with very deep architecture and high resolution image is both time computational expensive and difficult because of gradient vanishing/exploding problem, we propose a \textbf{Multi-Cell} architecture which gradually increases the depth of deep neural network and the resolution of input image, which both boosts the training time but also improves the classification accuracy. Further, considering the different stages of DR actually progress gradually, which means the labels of different stages are related. To considering the relationships of images with different stages, we propose a \textbf{Multi-Task} learning strategy which predicts the label with both classification and regression. Experimental results on the Kaggle dataset show that our method achieves a Kappa of 0.841 on test set which is the 4-th rank of all state-of-the-arts methods. Further, our Multi-Cell Multi-Task Convolutional Neural Networks (M$^2$CNN) solution is a general framework, which can be readily integrated with many other deep neural network architectures.

cs.CV↗
arXiv2018

A Data-Aided Channel Estimation Scheme for Decoupled Systems in Heterogeneous Networks

Uplink/downlink (UL/DL) decoupling promises more flexible cell association and higher throughput in heterogeneous networks (HetNets), however, it hampers the acquisition of DL channel state information (CSI) in time-division-duplex (TDD) systems due to different base stations (BSs) connected in UL/DL. In this paper, we propose a novel data-aided (DA) channel estimation scheme to address this problem by utilizing decoded UL data to exploit CSI from received UL data signal in decoupled HetNets where a massive multiple-input multiple-output BS and dense small cell BSs are deployed. We analytically estimate BER performance of UL decoded data, which are used to derive an approximated normalized mean square error (NMSE) expression of the DA minimum mean square error (MMSE) estimator. Compared with the conventional least square (LS) and MMSE, it is shown that NMSE performances of all estimators are determined by their signal-to-noise ratio (SNR)-like terms and there is an increment consisting of UL data power, UL data length and BER values in the SNR-like term of DA method, which suggests DA method outperforms the conventional ones in any scenarios. Higher UL data power, longer UL data length and better BER performance lead to more accurate estimated channels with DA method. Numerical results verify that the analytical BER and NMSE results are close to the simulated ones and a remarkable gain in both NMSE and DL rate can be achieved by DA method in multiple scenarios with different modulations.

cs.IT↗
arXiv2018

Future Frame Prediction for Anomaly Detection -- A New Baseline

Anomaly detection in videos refers to the identification of events that do not conform to expected behavior. However, almost all existing methods tackle the problem by minimizing the reconstruction errors of training data, which cannot guarantee a larger reconstruction error for an abnormal event. In this paper, we propose to tackle the anomaly detection problem within a video prediction framework. To the best of our knowledge, this is the first work that leverages the difference between a predicted future frame and its ground truth to detect an abnormal event. To predict a future frame with higher quality for normal events, other than the commonly used appearance (spatial) constraints on intensity and gradient, we also introduce a motion (temporal) constraint in video prediction by enforcing the optical flow between predicted frames and ground truth frames to be consistent, and this is the first work that introduces a temporal constraint into the video prediction task. Such spatial and motion constraints facilitate the future frame prediction for normal events, and consequently facilitate to identify those abnormal events that do not conform the expectation. Extensive experiments on both a toy dataset and some publicly available datasets validate the effectiveness of our method in terms of robustness to the uncertainty in normal events and the sensitivity to abnormal events.

cs.CV↗
arXiv2018

Eigenspaces of symmetric graphs are not typically irreducible

We construct rich families of Schrödinger operators on symmetric graphs, both quantum and combinatorial, whose spectral degeneracies are persistently larger than the maximal dimension of an irreducible representations of the symmetry group.

math.SP↗
arXiv2016

Simplicity of eigenvalues and non-vanishing of eigenfunctions of a quantum graph

We prove that after an arbitrarily small adjustment of edge lengths, the spectrum of a compact quantum graph with $δ$-type vertex conditions can be simple. We also show that the eigenfunctions, with the exception of those living entirely on a looping edge, can be made to be non-vanishing on all vertices of the graph. As an application of the above result, we establish that the secular manifold (also called "determinant manifold") of a large family of graphs has exactly two smooth connected components.

math-ph↗
arXiv2016

The Attenuated Space Poset $\mathcal{A}_q(N, M)$

In this paper, we study the incidence algebra $T$ of the attenuated space poset $\mathcal{A}_q(N, M)$. We consider the following topics. We consider some generators of $T$: the raising matrix $R$, the lowering matrix $L$, and a certain diagonal matrix $K$. We describe some relations among $R, L, K$. We put these relations in an attractive form using a certain matrix $S$ in $T$. We characterize the center $Z(T)$. Using $Z(T)$, we relate $T$ to the quantum group $U_τ({\mathfrak{sl}}_2)$ with $τ^2=q$. We consider two elements $A, A^*$ in $T$ of a certain form. We find necessary and sufficient conditions for $A, A^*$ to satisfy the tridiagonal relations. Let $W$ denote an irreducible $T$-module. We find necessary and sufficient conditions for the above $A, A^*$ to act on $W$ as a Leonard pair.

math.CO↗
arXiv2013

Full automorphism groups of association schemes based on isotropic subspaces

The set of all subspaces of a given dimension in a finite classical polar space has a structure of a symmetric association scheme. If the dimension is zero, this is the scheme of the collinearity graph of the space; If the dimension is maximum, it is the dual polar scheme. In this note, we determine the full automorphism group of this scheme.

math.CO↗
arXiv2013

Transformations of polar Grassmannians preserving certain intersecting relations

Let $Π$ be a polar space of rank $n\ge 3$. Denote by ${\mathcal G}_{k}(Π)$ the polar Grassmannian formed by singular subspaces of $Π$ whose projective dimension is equal to $k$. Suppose that $k$ is an integer not greater than $n-2$ and consider the relation ${\mathfrak R}_{i,j}$, $0\le i\le j\le k+1$ formed by all pairs $(X,Y)\in {\mathcal G}_{k}(Π)\times {\mathcal G}_{k}(Π)$ such that $\dim_{p}(X^{\perp}\cap Y)=k-i$ and $\dim_{p} (X\cap Y)=k-j$ ($X^{\perp}$ consists of all points of $Π$ collinear to every point of $X$). We show that every bijective transformation of ${\mathcal G}_{k}(Π)$ preserving ${\mathfrak R}_{1,1}$ is induced by an automorphism of $Π$ and the same holds for the relation ${\mathfrak R}_{0,t}$ if $n\ge 2t\ge 4$ and $k=n-t-1$. In the case when $Π$ is a finite classical polar space, we establish that the valencies of ${\mathfrak R}_{i,j}$ and ${\mathfrak R}_{i',j'}$ are distinct if $(i,j)\ne (i',j')$.

math.AG↗
arXiv2012

Full automorphism groups of association schemes based on attenuated spaces

The set of subspaces with a given dimension in an attenuated space has a structure of a symmetric association scheme, which is a generalization of both Grassmann schemes and bilinear forms schemes. In [K. Wang, J. Guo, F. Li, Association schemes based on attenuated space, European J. Combin. 31 (2010) 297--305], its intersection numbers were computed. In this paper, we determine its full automorphism group.

math.CO↗