arXiv subjects

Pengfei Xu

Publications and source records attributed to Pengfei Xu.

69 records · Page 4Linked to original sources

arXiv2020

Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference

State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications. We propose a dual dynamic inference (DDI) framework that highlights the following aspects: 1) we integrate both input-dependent and resource-dependent dynamic inference mechanisms under a unified framework in order to fit the varying IoT resource requirements in practice. DDI is able to both constantly suppress unnecessary costs for easy samples, and to halt inference for all samples to meet hard resource constraints enforced; 2) we propose a flexible multi-grained learning to skip (MGL2S) approach for input-dependent inference which allows simultaneous layer-wise and channel-wise skipping; 3) we extend DDI to complex CNN backbones such as DenseNet and show that DDI can be applied towards optimizing any specific resource goals including inference latency or energy cost. Extensive experiments demonstrate the superior inference accuracy-resource trade-off achieved by DDI, as well as the flexibility to control such trade-offs compared to existing peer methods. Specifically, DDI can achieve up to 4 times computational savings with the same or even higher accuracy as compared to existing competitive baselines.

cs.LG↗

arXiv2019

E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings

Convolutional neural networks (CNNs) have been increasingly deployed to edge devices. Hence, many efforts have been made towards efficient CNN inference in resource-constrained platforms. This paper attempts to explore an orthogonal direction: how to conduct more energy-efficient training of CNNs, so as to enable on-device training. We strive to reduce the energy cost during training, by dropping unnecessary computations from three complementary levels: stochastic mini-batch dropping on the data level; selective layer update on the model level; and sign prediction for low-cost, low-precision back-propagation, on the algorithm level. Extensive simulations and ablation studies, with real energy measurements from an FPGA board, confirm the superiority of our proposed strategies and demonstrate remarkable energy savings for training. For example, when training ResNet-74 on CIFAR-10, we achieve aggressive energy savings of >90% and >60%, while incurring a top-1 accuracy loss of only about 2% and 1.2%, respectively. When training ResNet-110 on CIFAR-100, an over 84% training energy saving is achieved without degrading inference accuracy.

cs.LG↗

arXiv2019

Multi-source Domain Adaptation for Semantic Segmentation

Simulation-to-real domain adaptation for semantic segmentation has been actively studied for various applications such as autonomous driving. Existing methods mainly focus on a single-source setting, which cannot easily handle a more practical scenario of multiple sources with different distributions. In this paper, we propose to investigate multi-source domain adaptation for semantic segmentation. Specifically, we design a novel framework, termed Multi-source Adversarial Domain Aggregation Network (MADAN), which can be trained in an end-to-end manner. First, we generate an adapted domain for each source with dynamic semantic consistency while aligning at the pixel-level cycle-consistently towards the target. Second, we propose sub-domain aggregation discriminator and cross-domain cycle discriminator to make different adapted domains more closely aggregated. Finally, feature-level alignment is performed between the aggregated domain and target domain while training the segmentation network. Extensive experiments from synthetic GTA and SYNTHIA to real Cityscapes and BDDS datasets demonstrate that the proposed MADAN model outperforms state-of-the-art approaches. Our source code is released at: https://github.com/Luodian/MADAN.

cs.CV↗

arXiv2019

Blind source separation using Fast-ICA with a novel nonlinear function

Blind source separation(BSS) is a hotspot in signal processing, and independent component analysis (ICA) is a very effective tool for solving the BSS problem. In order to improve the performance of the separation, a new nonlinear function sin was introduced. It can replace the commonly used classical functions (tanh, gauss and pow3) and does not need to select different nonlinear functions according to the Gauss property of signals. The two Matlab simulation results show that the improved Fast-ICA algorithm with the proposed nonlinearity can not only improve the separation accuracy but also speed up the convergence of blind source separation.

eess.SP↗

arXiv2019

Blind separation of rotor vibration signals in high-noise environments

During the operation of the engine rotor, the vibration signal measured by the sensor is the mixed signal of each vibration source, and contains strong noise at the same time. In this paper, a new separation method for mixed vibration signals in strong noise environment(SNR=-5) is proposed. Firstly, the time-delay auto-correlation de-noising method is used to de-noise the mixed signals, and then the common blind separation algorithm (MSNR algorithm is used here) is used to separate the mixed vibration signals, which improves the separation performance. The simulation results verify the validity of the method. The proposed method provides a new idea for health monitoring and fault diagnosis of engine rotor vibration signals.

eess.SP↗

arXiv2019

Blind Source Separation of Optical Wireless Communications in Noisy Environments

Blind source separation is a research hotspot in the field of signal processing because it aims to separate unknown source signals from observed mixtures through an unknown transmission channel. A low computational complexity instantaneous linear mixture signals blind separation algorithm was introduced and improved. There is only one variable parameter named the length of moving average in the algorithm, which has a significant impact on the separation effect. This paper gives some suggestions on the reasonable value through experiments. The algorithm is extended to the separation of visible light communication signals in different noise environments, and has achieved certain results, which further expands the applicability of the algorithm.

eess.SP↗

arXiv2019

Multiple Set Matching and Pre-Filtering with Bloom Multifilters

Bloom filter is a space-efficient probabilistic data structure for checking elements' membership in a set. Given multiple sets, however, a standard Bloom filter is not sufficient when looking for the items to which an element or a set of input elements belong to. In this article, we solve multiple set matching problem by proposing two efficient Bloom Multifilters called Bloom Matrix and Bloom Vector. Both of them are space efficient and answer queries with a set of identifiers for multiple set matching problems. We show that the space efficiency can be optimized further according to the distribution of labels among multiple sets: Uniform and Zipf. While both of them are space efficient, Bloom Vector can efficiently exploit Zipf distribution of data for further space reduction. Our results also highlight that basic ADD and LOOKUP operations on Bloom Matrix are faster than on Bloom Vector. However, Bloom Matrix does not meet the theoretical false positive rate of less than $10^{-2}$ for LOOKUP operations if the represented data or the labels are not uniformly distributed among the multiple sets. Consequently, we introduce \textit{Bloom Test} which uses Bloom Matrix as the pre-filter structure to determine which structure is suitable for improved performance with an arbitrary input dataset.

cs.DS↗

arXiv2018

Efficient Taxonomic Similarity Joins with Adaptive Overlap Constraint

A similarity join aims to find all similar pairs between two collections of records. Established approaches usually deal with synthetic differences like typos and abbreviations, but neglect the semantic relations between words. Such relations, however, are helpful for obtaining high-quality joining results. In this paper, we leverage the taxonomy knowledge (i.e., a set of IS-A hierarchical relations) to define a similarity measure which finds semantic-similar records from two datasets. Based on this measure, we develop a similarity join algorithm with prefix filtering framework to prune away irrelevant pairs effectively. Our technical contribution here is an algorithm that judiciously selects critical parameters in a prefix filter to maximise its filtering power, supported by an estimation technique and Monte Carlo simulation process. Empirical experiments show that our proposed methods exhibit high efficiency and scalability, outperforming the state-of-art by a large margin.

cs.IR↗

arXiv2017

Demonstration of a genetic-algorithm-optimized cavity-based waveguide crossing

A compact, single-etch silicon photonics waveguide crossing based on low quality factor cavity and optimized by a genetic algorithm is demonstrated. The device has 0.1~0.3 dB insertion loss and < -35 dB crosstalk per crossing for fundamental TE mode in the 1550-1600 nm wavelength band, with a footprint of 5 x 5 μm$^2$.

physics.app-ph↗

arXiv2017

Performance Evaluation of Deep Learning Tools in Docker Containers

With the success of deep learning techniques in a broad range of application domains, many deep learning software frameworks have been developed and are being updated frequently to adapt to new hardware features and software libraries, which bring a big challenge for end users and system administrators. To address this problem, container techniques are widely used to simplify the deployment and management of deep learning software. However, it remains unknown whether container techniques bring any performance penalty to deep learning applications. The purpose of this work is to systematically evaluate the impact of docker container on the performance of deep learning applications. We first benchmark the performance of system components (IO, CPU and GPU) in a docker container and the host system and compare the results to see if there's any difference. According to our results, we find that computational intensive jobs, either running on CPU or GPU, have small overhead indicating docker containers can be applied to deep learning programs. Then we evaluate the performance of some popular deep learning tools deployed in a docker container and the host system. It turns out that the docker container will not cause noticeable drawbacks while running those deep learning tools. So encapsulating deep learning tool in a container is a feasible solution.

cs.DC↗

arXiv2017

Supervised Learning Based Algorithm Selection for Deep Neural Networks

Many recent deep learning platforms rely on third-party libraries (such as cuBLAS) to utilize the computing power of modern hardware accelerators (such as GPUs). However, we observe that they may achieve suboptimal performance because the library functions are not used appropriately. In this paper, we target at optimizing the operations of multiplying a matrix with the transpose of another matrix (referred to as NT operation hereafter), which contribute about half of the training time of fully connected deep neural networks. Rather than directly calling the library function, we propose a supervised learning based algorithm selection approach named MTNN, which uses a gradient boosted decision tree to select one from two alternative NT implementations intelligently: (1) calling the cuBLAS library function; (2) calling our proposed algorithm TNN that uses an efficient out-of-place matrix transpose. We evaluate the performance of MTNN on two modern GPUs: NVIDIA GTX 1080 and NVIDIA Titan X Pascal. MTNN can achieve 96\% of prediction accuracy with very low computational overhead, which results in an average of 54\% performance improvement on a range of NT operations. To further evaluate the impact of MTNN on the training process of deep neural networks, we have integrated MTNN into a popular deep learning platform Caffe. Our experimental results show that the revised Caffe can outperform the original one by an average of 28\%. Both MTNN and the revised Caffe are open-source.

cs.DC↗

arXiv2017

Benchmarking State-of-the-Art Deep Learning Software Tools

Deep learning has been shown as a successful machine learning method for a variety of tasks, and its popularity results in numerous open-source deep learning software tools. Training a deep network is usually a very time-consuming process. To address the computational challenge in deep learning, many tools exploit hardware features such as multi-core CPUs and many-core GPUs to shorten the training time. However, different tools exhibit different features and running performance when training different types of deep networks on different hardware platforms, which makes it difficult for end users to select an appropriate pair of software and hardware. In this paper, we aim to make a comparative study of the state-of-the-art GPU-accelerated deep learning software tools, including Caffe, CNTK, MXNet, TensorFlow, and Torch. We first benchmark the running performance of these tools with three popular types of neural networks on two CPU platforms and three GPU platforms. We then benchmark some distributed versions on multiple GPUs. Our contribution is two-fold. First, for end users of deep learning tools, our benchmarking results can serve as a guide to selecting appropriate hardware platforms and software tools. Second, for software developers of deep learning tools, our in-depth analysis points out possible future directions to further optimize the running performance.

cs.DC↗

arXiv2016

UDBMS: Road to Unification for Multi-model Data Management

A traditional database systems is organized around a single data model that determines how data can be organized, stored and manipulated. But the vision of this paper is to develop new principles and techniques to manage multiple data models against a single, integrated backend. For example, semi-structured, graph and relational models are examples of data models that may be supported by a new system. Having a single data platform for managing both well-structured data and NoSQL data is beneficial to users; this approach significantly reduces integration, migration, development, maintenance and operational issues. The problem is challenging: the existing database principles mainly work for a single model and the research on multi-model data management is still at an early stage. In this paper, we envision a UDBMS (Unified Database Management System) for multi-model data management in one platform. UDBMS will provide several new features such as unified data model and flexible schema, unified query processing, unified index structure and cross-model transaction guarantees. We discuss our vision as well as present multiple research challenges that we need to address.

cs.DB↗

arXiv2016

Top-k String Auto-Completion with Synonyms

Auto-completion is one of the most prominent features of modern information systems. The existing solutions of auto-completion provide the suggestions based on the beginning of the currently input character sequence (i.e. prefix). However, in many real applications, one entity often has synonyms or abbreviations. For example, "DBMS" is an abbreviation of "Database Management Systems". In this paper, we study a novel type of auto-completion by using synonyms and abbreviations. We propose three trie-based algorithms to solve the top-k auto-completion with synonyms; each one with different space and time complexity trade-offs. Experiments on large-scale datasets show that it is possible to support effective and efficient synonym-based retrieval of completions of a million strings with thousands of synonyms rules at about a microsecond per-completion, while taking small space overhead (i.e. 160-200 bytes per string). The source code of our experiments can be download at: http://udbms.cs.helsinki.fi/?projects/autocompletion/download .

cs.IR↗

arXiv2016

Design and optimization of optical modulators based on graphene-on-silicon nitride microring resonators

In order to overcome the challenge of obtaining high modulation depth due to weak graphene-light interaction, a graphene-on-silicon nitride (SiNx) microring resonator based on graphene's gate-tunable optical conductivity is proposed and studied. Geometrical parameters of graphene-on-SiNx waveguide are systematically analyzed and optimized, yielding a loss tunability of 0.04 dB/μm and an effective index variation of 0.0022. We explicitly study the interaction between graphene and a 40-μm-radius microring resonator, where electro-absorptive and electro-refractive modulation are both taken into account. By choosing appropriate graphene coverage and coupling coefficient, a high modulation depth of over 40 dB with large fabrication tolerance is obtained.

physics.optics↗