arXiv subjects

Yang Sui

Publications and source records attributed to Yang Sui.

41 records · Page 3Linked to original sources

arXiv2022

CHIP: CHannel Independence-based Pruning for Compact Neural Networks

Filter pruning has been widely used for neural network compression because of its enabled practical acceleration. To date, most of the existing filter pruning works explore the importance of filters via using intra-channel information. In this paper, starting from an inter-channel perspective, we propose to perform efficient filter pruning using Channel Independence, a metric that measures the correlations among different feature maps. The less independent feature map is interpreted as containing less useful information$/$knowledge, and hence its corresponding filter can be pruned without affecting model capacity. We systematically investigate the quantification metric, measuring scheme and sensitiveness$/$reliability of channel independence in the context of filter pruning. Our evaluation results for different models on various datasets show the superior performance of our approach. Notably, on CIFAR-10 dataset our solution can bring $0.90\%$ and $0.94\%$ accuracy increase over baseline ResNet-56 and ResNet-110 models, respectively, and meanwhile the model size and FLOPs are reduced by $42.8\%$ and $47.4\%$ (for ResNet-56) and $48.3\%$ and $52.1\%$ (for ResNet-110), respectively. On ImageNet dataset, our approach can achieve $40.8\%$ and $44.8\%$ storage and computation reductions, respectively, with $0.15\%$ accuracy increase over the baseline ResNet-50 model. The code is available at https://github.com/Eclipsess/CHIP_NeurIPS2021.

cs.CV↗

arXiv2021

Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework

Advanced tensor decomposition, such as Tensor train (TT) and Tensor ring (TR), has been widely studied for deep neural network (DNN) model compression, especially for recurrent neural networks (RNNs). However, compressing convolutional neural networks (CNNs) using TT/TR always suffers significant accuracy loss. In this paper, we propose a systematic framework for tensor decomposition-based model compression using Alternating Direction Method of Multipliers (ADMM). By formulating TT decomposition-based model compression to an optimization problem with constraints on tensor ranks, we leverage ADMM technique to systemically solve this optimization problem in an iterative way. During this procedure, the entire DNN model is trained in the original structure instead of TT format, but gradually enjoys the desired low tensor rank characteristics. We then decompose this uncompressed model to TT format and fine-tune it to finally obtain a high-accuracy TT-format DNN model. Our framework is very general, and it works for both CNNs and RNNs, and can be easily modified to fit other tensor decomposition approaches. We evaluate our proposed framework on different DNN models for image classification and video recognition tasks. Experimental results show that our ADMM-based TT-format models demonstrate very high compression performance with high accuracy. Notably, on CIFAR-100, with 2.3X and 2.4X compression ratios, our models have 1.96% and 2.21% higher top-1 accuracy than the original ResNet-20 and ResNet-32, respectively. For compressing ResNet-18 on ImageNet, our model achieves 2.47X FLOPs reduction without accuracy loss.

cs.CV↗

arXiv2011

Substrate Gating of Contact Resistance in Graphene Transistors

Metal contacts have been identified to be a key technological bottleneck for the realization of viable graphene electronics. Recently, it was observed that for structures that possess both a top and a bottom gate, the electron-hole conductance asymmetry can be modulated by the bottom gate. In this letter, we explain this observation by postulating the presence of an effective thin interfacial dielectric layer between the metal contact and the underlying graphene. Electrical results from quantum transport calculations accounting for this modified electrostatics corroborate well with the experimentally measured contact resistances. Our study indicates that the engineering of metal- graphene interface is a crucial step towards reducing the contact resistance for high performance graphene transistors.

cond-mat.mes-hall↗

arXiv2011

Signatures of disorder in the minimum conductivity of graphene

Graphene has been proposed as a promising material for future nanoelectronics because of its unique electronic properties. Understanding the scaling behavior of this new nanomaterial under common experimental conditions is of critical importance for developing graphene-based nanoscale devices. We present a comprehensive experimental and theoretical study on the influence of edge disorder and bulk disorder on the minimum conductivity of graphene ribbons. For the first time, we discovered a strong non-monotonic size scaling behavior featuring a peak and saturation minimum conductivity. Through extensive numerical simulations and analysis, we are able to attribute these features to the amount of edge and bulk disorder in graphene devices. This study elucidates the quantum transport mechanisms in realistic experimental graphene systems, which can be used as a guideline for designing graphene-based nanoscale devices with improved performance.

cond-mat.mes-hall↗

arXiv2009

Screening and interlayer coupling in multilayer graphene field-effect transistors

With the motivation of improving the performance and reliability of aggressively scaled nano-patterned graphene field-effect transistors, we present the first systematic experimental study on charge and current distribution in multilayer graphene field-effect transistors. We find a very particular thickness dependence for Ion, Ioff, and the Ion/Ioff ratio, and propose a resistor network model including screening and interlayer coupling to explain the experimental findings. In particular, our model does not invoke modification of the linear energy-band structure of graphene for the multilayer case. Noise reduction in nano-scale few-layer graphene transistors is experimentally demonstrated and can be understood within this model as well.

cond-mat.mes-hall↗