arXiv subjects

Pu Li

Publications and source records attributed to Pu Li.

At least 37 records · Page 2Linked to original sources

arXiv2023

Group channel pruning and spatial attention distilling for object detection

Due to the over-parameterization of neural networks, many model compression methods based on pruning and quantization have emerged. They are remarkable in reducing the size, parameter number, and computational complexity of the model. However, most of the models compressed by such methods need the support of special hardware and software, which increases the deployment cost. Moreover, these methods are mainly used in classification tasks, and rarely directly used in detection tasks. To address these issues, for the object detection network we introduce a three-stage model compression method: dynamic sparse training, group channel pruning, and spatial attention distilling. Firstly, to select out the unimportant channels in the network and maintain a good balance between sparsity and accuracy, we put forward a dynamic sparse training method, which introduces a variable sparse rate, and the sparse rate will change with the training process of the network. Secondly, to reduce the effect of pruning on network accuracy, we propose a novel pruning method called group channel pruning. In particular, we divide the network into multiple groups according to the scales of the feature layer and the similarity of module structure in the network, and then we use different pruning thresholds to prune the channels in each group. Finally, to recover the accuracy of the pruned network, we use an improved knowledge distillation method for the pruned network. Especially, we extract spatial attention information from the feature maps of specific scales in each group as knowledge for distillation. In the experiments, we use YOLOv4 as the object detection network and PASCAL VOC as the training dataset. Our method reduces the parameters of the model by 64.7 % and the calculation by 34.9%.

cs.CV↗

arXiv2023

Learning Knowledge-Rich Sequential Model for Planar Homography Estimation in Aerial Video

This paper presents an unsupervised approach that leverages raw aerial videos to learn to estimate planar homographic transformation between consecutive video frames. Previous learning-based estimators work on pairs of images to estimate their planar homographic transformations but suffer from severe over-fitting issues, especially when applying over aerial videos. To address this concern, we develop a sequential estimator that directly processes a sequence of video frames and estimates their pairwise planar homographic transformations in batches. We also incorporate a set of spatial-temporal knowledge to regularize the learning of such a sequence-to-sequence model. We collect a set of challenging aerial videos and compare the proposed method to the alternative algorithms. Empirical studies suggest that our sequential model achieves significant improvement over alternative image-based methods and the knowledge-rich regularization further boosts our system performance. Our codes and dataset could be found at https://github.com/Paul-LiPu/DeepVideoHomography

cs.CV↗

arXiv2023

Learning Stage-wise GANs for Whistle Extraction in Time-Frequency Spectrograms

Whistle contour extraction aims to derive animal whistles from time-frequency spectrograms as polylines. For toothed whales, whistle extraction results can serve as the basis for analyzing animal abundance, species identity, and social activities. During the last few decades, as long-term recording systems have become affordable, automated whistle extraction algorithms were proposed to process large volumes of recording data. Recently, a deep learning-based method demonstrated superior performance in extracting whistles under varying noise conditions. However, training such networks requires a large amount of labor-intensive annotation, which is not available for many species. To overcome this limitation, we present a framework of stage-wise generative adversarial networks (GANs), which compile new whistle data suitable for deep model training via three stages: generation of background noise in the spectrogram, generation of whistle contours, and generation of whistle signals. By separating the generation of different components in the samples, our framework composes visually promising whistle data and labels even when few expert annotated data are available. Regardless of the amount of human-annotated data, the proposed data augmentation framework leads to a consistent improvement in performance of the whistle extraction model, with a maximum increase of 1.69 in the whistle extraction mean F1-score. Our stage-wise GAN also surpasses one single GAN in improving whistle extraction models with augmented data. The data and code will be available at https://github.com/Paul-LiPu/CompositeGAN\_WhistleAugment.

cs.CV↗

arXiv2023

SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations

Reverse engineering CAD models from raw geometry is a classic but strenuous research problem. Previous learning-based methods rely heavily on labels due to the supervised design patterns or reconstruct CAD shapes that are not easily editable. In this work, we introduce SECAD-Net, an end-to-end neural network aimed at reconstructing compact and easy-to-edit CAD models in a self-supervised manner. Drawing inspiration from the modeling language that is most commonly used in modern CAD software, we propose to learn 2D sketches and 3D extrusion parameters from raw shapes, from which a set of extrusion cylinders can be generated by extruding each sketch from a 2D plane into a 3D body. By incorporating the Boolean operation (i.e., union), these cylinders can be combined to closely approximate the target geometry. We advocate the use of implicit fields for sketch representation, which allows for creating CAD variations by interpolating latent codes in the sketch latent space. Extensive experiments on both ABC and Fusion 360 datasets demonstrate the effectiveness of our method, and show superiority over state-of-the-art alternatives including the closely related method for supervised CAD reconstruction. We further apply our approach to CAD editing and single-view CAD reconstruction. The code is released at https://github.com/BunnySoCrazy/SECAD-Net.

cs.CV↗

arXiv2022

RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantization

We introduce a Power-of-Two low-bit post-training quantization(PTQ) method for deep neural network that meets hardware requirements and does not call for long-time retraining. Power-of-Two quantization can convert the multiplication introduced by quantization and dequantization to bit-shift that is adopted by many efficient accelerators. However, the Power-of-Two scale factors have fewer candidate values, which leads to more rounding or clipping errors. We propose a novel Power-of-Two PTQ framework, dubbed RAPQ, which dynamically adjusts the Power-of-Two scales of the whole network instead of statically determining them layer by layer. It can theoretically trade off the rounding error and clipping error of the whole network. Meanwhile, the reconstruction method in RAPQ is based on the BN information of every unit. Extensive experiments on ImageNet prove the excellent performance of our proposed method. Without bells and whistles, RAPQ can reach accuracy of 65% and 48% on ResNet-18 and MobileNetV2 respectively with weight INT2 activation INT4. We are the first to propose the more constrained but hardware-friendly Power-of-Two quantization scheme for low-bit PTQ specially and prove that it can achieve nearly the same accuracy as SOTA PTQ method. The code was released.

cs.CV↗

arXiv2022

MVP-Net: Multiple View Pointwise Semantic Segmentation of Large-Scale Point Clouds

Semantic segmentation of 3D point cloud is an essential task for autonomous driving environment perception. The pipeline of most pointwise point cloud semantic segmentation methods includes points sampling, neighbor searching, feature aggregation, and classification. Neighbor searching method like K-nearest neighbors algorithm, KNN, has been widely applied. However, the complexity of KNN is always a bottleneck of efficiency. In this paper, we propose an end-to-end neural architecture, Multiple View Pointwise Net, MVP-Net, to efficiently and directly infer large-scale outdoor point cloud without KNN or any complex pre/postprocessing. Instead, assumption-based space filling curves and multi-rotation of point cloud methods are introduced to point feature aggregation and receptive field expanding. Numerical experiments show that the proposed MVP-Net is 11 times faster than the most efficient pointwise semantic segmentation method RandLA-Net and achieves the same accuracy on the large-scale benchmark SemanticKITTI dataset.

cs.CV↗

arXiv2022

Fast all-optical random number generator

We propose a simple and all-optical method for fast random number generation based on the laser mode hopping. Through periodically restarting a two-mode laser operating in the bistable state, a random number stream can be generated due to the spontaneous emission noise. To validate the feasibility of this method, we perform a theoretical simulation using the common vertical-cavity surface-emitting laser (VCSEL) with two polarization modes. Numerical results demonstrate that fast 2.5 Gb/s random number streams can be continuously obtained with verified randomness. This scheme provide a fully monolithic solution for random number generator, due to its simple and all-optical structure.

physics.optics↗

arXiv2021

Fabry-Perot Lasers as Enablers for Parallel Reservoir Computing

We introduce the use of Fabry-Perot (FP) lasers as potential neuromorphic computing machines with parallel processing capabilities. With the use of optical injection between a master FP laser and a slave FP laser under feedback we demonstrate the potential for scaling up the processing power at longitudinal mode granularity and perform real-time processing for signal equalization in 25 Gbaud intensity modulation direct detection optical communication systems. We demonstrate the improvement of classification performance as the number of nodes increases and the capability of simultaneous processing of arbitrary data streams. Extensive numerical simulations show that up to 8 longitudinal modes in typical Fabry-Perot lasers can be leveraged so as to enhance classification performance.

physics.app-ph↗

arXiv2020

Review of Text Style Transfer Based on Deep Learning

Text style transfer is a hot issue in recent natural language processing,which mainly studies the text to adapt to different specific situations, audiences and purposes by making some changes. The style of the text usually includes many aspects such as morphology, grammar, emotion, complexity, fluency, tense, tone and so on. In the traditional text style transfer model, the text style is generally relied on by experts knowledge and hand-designed rules, but with the application of deep learning in the field of natural language processing, the text style transfer method based on deep learning Started to be heavily researched. In recent years, text style transfer is becoming a hot issue in natural language processing research. This article summarizes the research on the text style transfer model based on deep learning in recent years, and summarizes, analyzes and compares the main research directions and progress. In addition, the article also introduces public data sets and evaluation indicators commonly used for text style transfer. Finally, the existing characteristics of the text style transfer model are summarized, and the future development trend of the text style transfer model based on deep learning is analyzed and forecasted.

cs.CL↗

arXiv2020

Chance Constrained Optimization for Energy Management in Electric Vehicles

E-powertrain of future electric vehicles could consist of energy generation units (e.g., fuel cells and photovoltaic modules), energy storage systems (e.g., batteries and supercapacitors), energy conversion units (e.g., bidirectional DC/DC converters and DC/AC inverters) and an electric machine, which can work in both generating and motoring modes [1- 6]. An energy management system is responsible to operate the above-mentioned components in a way that the technical constraints are satisfied. This task should be accomplished by solving an optimization problem, which could aim at minimizing the total operation costs [5]. The optimization problem has been widely addressed by deterministic approaches [7], which take into account the forecasted values of active-reactive load profile. However, as shown in Figure 1 (a), it is impossible to accurately forecast the values, meaning that the solutions coming from deterministic approaches could lead to infeasible operations (i.e., constraint violations). Therefore, stochastic optimization approaches [8] should be utilized to fi nd optimal solution strategies while considering uncertain parameters.

math.OC↗

arXiv2020

Real-Time Dynamic Optimal Power Flow in Electric Vehicles Considering the Lifetime of the Components in the E-Powertrain

Different types of energy sources (e.g., batteries, supercapacitors, fuel cells) can be utilized in electric vehicles to store and provide energy in the e-powertrain through power electronic devices [1-6]. The lifetime of the components in the e-powertrain depends on their load profile [7,8]. For instance, the lifetime of a battery highly depends on the depth of discharge and the number of charge/discharge cycles [9-13]. The lifetime of an inverter mostly depends on the variations in the active-reactive power passing through it. This means that the expended life cost of the components can be decreased by allocating an optimal share of the total power to each energy source and power electronic device at an optimal time instance. In addition, the driving range of a vehicle can be prolonged by decreasing the energy loss i.e., operating the components in their high-efficiency region. Therefore, it is necessary to perform Optimal Power Flow (OPF) in the operation of the e-powertrain.

eess.SY↗

arXiv2020

An Once-for-All Budgeted Pruning Framework for ConvNets Considering Input Resolution

We propose an efficient once-for-all budgeted pruning framework (OFARPruning) to find many compact network structures close to winner tickets in the early training stage considering the effect of input resolution during the pruning process. In structure searching stage, we utilize cosine similarity to measure the similarity of the pruning mask to get high-quality network structures with low energy and time consumption. After structure searching stage, our proposed method randomly sample the compact structures with different pruning rates and input resolution to achieve joint optimization. Ultimately, we can obtain a cohort of compact networks adaptive to various resolution to meet dynamic FLOPs constraints on different edge devices with only once training. The experiments based on image classification and object detection show that OFARPruning has a higher accuracy than the once-for-all compression methods such as US-Net and MutualNet (1-2% better with less FLOPs), and achieve the same even higher accuracy as the conventional pruning methods (72.6% vs. 70.5% on MobileNetv2 under 170 MFLOPs) with much higher efficiency.

cs.CV↗

arXiv2020

Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

In order to deploy deep convolutional neural networks (CNNs) on resource-limited devices, many model pruning methods for filters and weights have been developed, while only a few to layer pruning. However, compared with filter pruning and weight pruning, the compact model obtained by layer pruning has less inference time and run-time memory usage when the same FLOPs and number of parameters are pruned because of less data moving in memory. In this paper, we propose a simple layer pruning method using fusible residual convolutional block (ResConv), which is implemented by inserting shortcut connection with a trainable information control parameter into a single convolutional layer. Using ResConv structures in training can improve network accuracy and train deep plain networks, and adds no additional computation during inference process because ResConv is fused to be an ordinary convolutional layer after training. For layer pruning, we convert convolutional layers of network into ResConv with a layer scaling factor. In the training process, the L1 regularization is adopted to make the scaling factors sparse, so that unimportant layers are automatically identified and then removed, resulting in a model of layer reduction. Our pruning method achieves excellent performance of compression and acceleration over the state-of-the-arts on different datasets, and needs no retraining in the case of low pruning rate. For example, with ResNet-110, we achieve a 65.5%-FLOPs reduction by removing 55.5% of the parameters, with only a small loss of 0.13% in top-1 accuracy on CIFAR-10.

cs.CV↗

arXiv2020

FenceMask: A Data Augmentation Approach for Pre-extracted Image Features

We propose a novel data augmentation method named 'FenceMask' that exhibits outstanding performance in various computer vision tasks. It is based on the 'simulation of object occlusion' strategy, which aim to achieve the balance between object occlusion and information retention of the input data. By enhancing the sparsity and regularity of the occlusion block, our augmentation method overcome the difficulty of small object augmentation and notably improve performance over baselines. Sufficient experiments prove the performance of our method is better than other simulate object occlusion approaches. We tested it on CIFAR10, CIFAR100 and ImageNet datasets for Coarse-grained classification, COCO2017 and VisDrone datasets for detection, Oxford Flowers, Cornel Leaf and Stanford Dogs datasets for Fine-Grained Visual Categorization. Our method achieved significant performance improvement on Fine-Grained Visual Categorization task and VisDrone dataset.

cs.CV↗

arXiv2020

Learning Deep Models from Synthetic Data for Extracting Dolphin Whistle Contours

We present a learning-based method for extracting whistles of toothed whales (Odontoceti) in hydrophone recordings. Our method represents audio signals as time-frequency spectrograms and decomposes each spectrogram into a set of time-frequency patches. A deep neural network learns archetypical patterns (e.g., crossings, frequency modulated sweeps) from the spectrogram patches and predicts time-frequency peaks that are associated with whistles. We also developed a comprehensive method to synthesize training samples from background environments and train the network with minimal human annotation effort. We applied the proposed learn-from-synthesis method to a subset of the public Detection, Classification, Localization, and Density Estimation (DCLDE) 2011 workshop data to extract whistle confidence maps, which we then processed with an existing contour extractor to produce whistle annotations. The F1-score of our best synthesis method was 0.158 greater than our baseline whistle extraction algorithm (~25% improvement) when applied to common dolphin (Delphinus spp.) and bottlenose dolphin (Tursiops truncatus) whistles.

q-bio.QM↗

arXiv2019

Parallel real-time quantum random number generator

Quantum random number generation exploits inherent randomness of quantum mechanical processes and measurements. Real-time generation rate of quantum random numbers is usually limited by electronic bandwidth and data processing rates. Here we use a multiplexing scheme to create a fast real-time quantum random number generator based on continuous variable vacuum fluctuations. Multiple sideband frequency modes of a quantum vacuum state within a homodyne detection bandwidth are concurrently extracted as the randomness source. Parallel post-processing of raw data from three sub-entropy sources is realized in one field-programmable gate array (FPGA) based on Toeplitz-hashing extractors. A cumulative generation rate of 8.25 Gbps in real-time is achieved. The system relies on optoelectronic components and circuits that could be integrated in a compact, economical package.

quant-ph↗

arXiv2019

An Efficient Interval Uncertainty Optimization Approach Based on Quasi-sparse Response Surface

The structure uncertainty optimization problem is usually treated as double-loop optimization process, which is computation-intensive. In this paper, an efficient interval uncertainty optimization approach based on Quasi-sparse response surface (QSRS) is proposed for structure uncertainty optimization. In which, 1) with l_1 norm and l_2 norm penalty method, a few appropriate basis functions are selected form a large number of basis functions to construct QSRS accurately and only a few sampling points is required, 2) as the orthogonal chebyshev polynomials is employed as QSRS basis functions, the local uncertainty can be evaluated by the combination of QSRS and interval arithmetic. Hence, the inner optimization process is eliminated. One mathematical problem and one engineering problem are used to validate the efficiency of the proposed approach. The results show that only 25% sampling points is needed than recently published approach.

math.OC↗

arXiv2018

Enhancing squeezing and nonclassicality of light in atom-optomechanical systems

Quadrature squeezing of light is investigated in a hybrid atom-optomechanical system comprising a cloud of two-level atoms and a movable mirror mediated by a single-mode cavity field. When the system is at high temperatures with quadrature fluctuations of light much above the standard quantum limit (SQL), excitation counting on the collective atomic state can effectively reduce the light noise close to the SQL. When the system is at low temperatures, considerable squeezing of light below the SQL is found at steady state. The squeezing is enhanced by simply increasing the atom-light coupling strength with the laser power optimized close to the unstable regime, and further noise reduction is achieved by decreasing various losses in the system. The presence of atoms and excitation counting on the atoms lessen the limitation of thermal noise, and the squeezing can be achieved at environment temperature of the order K. The nonclassicality of the light, embodied by the negative distributions of the Wigner function, is also studied by making non-Gaussian measurements on the atoms. It is shown that with feasible parameters excitation counting on the atoms is effective in inducing strongly optical nonclassicality.

quant-ph↗