Exploring the Effects of Channel Sparsity on Neural Network Pruning for Acoustic Scene Classification

Abstract

Acoustic Scene Classification (ASC) algorithms are usually expected to be deployed in resource-constrained systems. Existing works reduce the complexity of ASC algorithms by pruning some components, e.g. pruning channels in neural network. In practice, neural networks are often trained with sparsification such that unimportant channels can be found and further pruned. However, little efforts have been made to explore the the impact of channel sparsity on neural network pruning. To fully utilize the benefits of pruning for ASC, and to make sure the model performs consistently, we need a more profound comprehension of channel sparsification and its effects. This paper examines the internal weights acquired by convolutional neural networks that will undergone pruning. The study discusses how these weights can be utilized to create a novel metric, Weight Skewness (WS), for quantifying the sparsity of channels. We also provide a new approach to compare the performance of different pruning methods, which balances the trade-off between accuracy and complexity. The experiment results demonstrate that 1) applying higher channel sparsity to models can achieve greater compression rates while maintaining acceptable levels of accuracy; 2) the selection of pruning method has little influence on result 1); 3) MobileNets exhibit more significant benefits from channel sparsification than VGGNets and ResNets.