arXiv subjects

Fanxu Meng

Publications and source records attributed to Fanxu Meng.

43 records · Page 3Linked to original sources

arXiv2021

An Empirical Study and Analysis on Open-Set Semi-Supervised Learning

Pseudo-labeling (PL) and Data Augmentation-based Consistency Training (DACT) are two approaches widely used in Semi-Supervised Learning (SSL) methods. These methods exhibit great power in many machine learning tasks by utilizing unlabeled data for efficient training. But in a more realistic setting (termed as open-set SSL), where unlabeled dataset contains out-of-distribution (OOD) samples, the traditional SSL methods suffer severe performance degradation. Recent approaches mitigate the negative influence of OOD samples by filtering them out from the unlabeled data. However, it is not clear whether directly removing the OOD samples is the best choice. Furthermore, why PL and DACT could perform differently in open-set SSL remains a mystery. In this paper, we thoroughly analyze various SSL methods (PL and DACT) on open-set SSL and discuss pros and cons of these two approaches separately. Based on our analysis, we propose Style Disturbance to improve traditional SSL methods on open-set SSL and experimentally show our approach can achieve state-of-the-art results on various datasets by utilizing OOD samples properly. We believe our study can bring new insights for SSL research.

cs.CV↗

arXiv2021

Quantum Algorithm for DOA Estimation in Hybrid Massive MIMO

The direction of arrival (DOA) estimation in array signal processing is an important research area. The effectiveness of the direction of arrival greatly determines the performance of multi-input multi-output (MIMO) antenna systems. The multiple signal classification (MUSIC) algorithm, which is the most canonical and widely used subspace-based method, has a moderate estimation performance of DOA. However, in hybrid massive MIMO systems, the received signals at the antennas are not sent to the receiver directly, and spatial covariance matrix, which is essential in MUSIC algorithm, is thus unavailable. Therefore, the spatial covariance matrix reconstruction is required for the application of MUSIC in hybrid massive MIMO systems. In this article, we present a quantum algorithm for MUSIC-based DOA estimation in hybrid massive MIMO systems. Compared with the best-known classical algorithm, our quantum algorithm can achieve an exponential speedup on some parameters and a polynomial speedup on others under some mild conditions. In our scheme, we first present the quantum subroutine for the beam sweeping based spatial covariance matrix reconstruction, where we implement a quantum singular vector transition process to avoid extending the steering vectors matrix into the Hermitian form. Second, a variational quantum density matrix eigensolver (VQDME) is proposed for obtaining signal and noise subspaces, where we design a novel objective function in the form of the trace of density matrices product. Finally, a quantum labeling operation is proposed for the direction of arrival estimation of the signal.

cs.IT↗

arXiv2021

Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation

Filter is the key component in modern convolutional neural networks (CNNs). However, since CNNs are usually over-parameterized, a pre-trained network always contain some invalid (unimportant) filters. These filters have relatively small $l_{1}$ norm and contribute little to the output (\textbf{Reason}). While filter pruning removes these invalid filters for efficiency consideration, we tend to reactivate them to improve the representation capability of CNNs. In this paper, we introduce filter grafting (\textbf{Method}) to achieve this goal. The activation is processed by grafting external information (weights) into invalid filters. To better perform the grafting, we develop a novel criterion to measure the information of filters and an adaptive weighting strategy to balance the grafted information among networks. After the grafting operation, the network has fewer invalid filters compared with its initial state, enpowering the model with more representation capacity. Meanwhile, since grafting is operated reciprocally on all networks involved, we find that grafting may lose the information of valid filters when improving invalid filters. To gain a universal improvement on both valid and invalid filters, we compensate grafting with distillation (\textbf{Cultivation}) to overcome the drawback of grafting . Extensive experiments are performed on the classification and recognition tasks to show the superiority of our method. Code is available at \textcolor{black}{\emph{https://github.com/fxmeng/filter-grafting}}.

cs.LG↗

arXiv2020

Pruning Filter in Filter

Pruning has become a very powerful and effective technique to compress and accelerate modern neural networks. Existing pruning methods can be grouped into two categories: filter pruning (FP) and weight pruning (WP). FP wins at hardware compatibility but loses at the compression ratio compared with WP. To converge the strength of both methods, we propose to prune the filter in the filter. Specifically, we treat a filter $F \in \mathbb{R}^{C\times K\times K}$ as $K \times K$ stripes, i.e., $1\times 1$ filters $\in \mathbb{R}^{C}$, then by pruning the stripes instead of the whole filter, we can achieve finer granularity than traditional FP while being hardware friendly. We term our method as SWP (\emph{Stripe-Wise Pruning}). SWP is implemented by introducing a novel learnable matrix called Filter Skeleton, whose values reflect the shape of each filter. As some recent work has shown that the pruned architecture is more crucial than the inherited important weights, we argue that the architecture of a single filter, i.e., the shape, also matters. Through extensive experiments, we demonstrate that SWP is more effective compared to the previous FP-based methods and achieves the state-of-art pruning ratio on CIFAR-10 and ImageNet datasets without obvious accuracy drop. Code is available at https://github.com/fxmeng/Pruning-Filter-in-Filter

cs.CV↗

arXiv2020

A Novel RIS-Assisted Modulation Scheme

In this work, in order to achieve higher spectrum efficiency, we propose a reconfigurable intelligent surface (RIS)-assisted multi-user communication uplink system. Different from previous work in which the RIS only optimizes the phase of the incident users's signal, we propose the use of the RIS to create a virtual constellation diagram to transmit the data of an additional user signal. We focus on the two-user case and develop a tight approximation for the cumulative distribution function (CDF) of the received signal-to-noise ratio of both users. Then, based on the proposed statistical distribution, we derive the analytical expressions of the average bit error rate of the considered two users. The paper shows the trade off between the performance of the two users against each other as a function of the proposed phase shift at the RIS.

cs.IT↗

arXiv2020

Accurate Closed-Form Approximations to Channel Distributions of RIS-Aided Wireless Systems

This paper proposes highly accurate closed-form approximations to channel distributions of two different reconfigurable intelligent surface (RIS)-based wireless system setups, namely, dual-hop RIS-aided (RIS-DH) scheme and RIS-aided transmit (RIS-T) scheme. Differently from previous works, the proposed approximations reveal to be very tight for arbitrary number $N$ of reflecting metasurface's elements. Our findings are then applied to the performance analysis of the considered systems, in which the outage probability, bit error rate, and average channel capacity are derived. Results show that the achievable diversity orders $G_d$ for RIS-DH and RIS-T schemes are $N-1<G_d<N$ and $N$, respectively. Furthermore, it is revealed that both schemes can not provide the multiplexing gain and only diversity gains are achieved. For the RIS-DH scheme, the channels are similar to the keyhole multiple-input multiple-output (MIMO) channels with only one degree of freedom, while the RIS-T scheme is like the transmit diversity structure.

cs.IT↗

arXiv2020

Filter Grafting for Deep Neural Networks

This paper proposes a new learning paradigm called filter grafting, which aims to improve the representation capability of Deep Neural Networks (DNNs). The motivation is that DNNs have unimportant (invalid) filters (e.g., l1 norm close to 0). These filters limit the potential of DNNs since they are identified as having little effect on the network. While filter pruning removes these invalid filters for efficiency consideration, filter grafting re-activates them from an accuracy boosting perspective. The activation is processed by grafting external information (weights) into invalid filters. To better perform the grafting process, we develop an entropy-based criterion to measure the information of filters and an adaptive weighting strategy for balancing the grafted information among networks. After the grafting operation, the network has very few invalid filters compared with its untouched state, enpowering the model with more representation capacity. We also perform extensive experiments on the classification and recognition tasks to show the superiority of our method. For example, the grafted MobileNetV2 outperforms the non-grafted MobileNetV2 by about 7 percent on CIFAR-100 dataset. Code is available at https://github.com/fxmeng/filter-grafting.git.

cs.CV↗