arXiv subjects

Meng Li

Publications and source records attributed to Meng Li.

At least 361 records · Page 20Linked to original sources

arXiv2020

SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks

Machine learning techniques have been widely applied in Internet companies for various tasks, acting as an essential driving force, and feature engineering has been generally recognized as a crucial tache when constructing machine learning systems. Recently, a growing effort has been made to the development of automatic feature engineering methods, so that the substantial and tedious manual effort can be liberated. However, for industrial tasks, the efficiency and scalability of these methods are still far from satisfactory. In this paper, we proposed a staged method named SAFE (Scalable Automatic Feature Engineering), which can provide excellent efficiency and scalability, along with requisite interpretability and promising performance. Extensive experiments are conducted and the results show that the proposed method can provide prominent efficiency and competitive effectiveness when comparing with other methods. What's more, the adequate scalability of the proposed method ensures it to be deployed in large scale industrial tasks.

cs.LG↗

arXiv2020

TimingCamouflage+: Netlist Security Enhancement with Unconventional Timing (with Appendix)

With recent advances in reverse engineering, attackers can reconstruct a netlist to counterfeit chips by opening the die and scanning all layers of authentic chips. This relatively easy counterfeiting is made possible by the use of the standard simple clocking scheme, where all combinational blocks function within one clock period, so that a netlist of combinational logic gates and flip-flops is sufficient to duplicate a design. In this paper, we propose to invalidate the assumption that a netlist completely represents the function of a circuit with unconventional timing. With the introduced wave-pipelining paths, attackers have to capture gate and interconnect delays during reverse engineering, or to test a huge number of combinational paths to identify the wave-pipelining paths. To hinder the test-based attack, we construct false paths with wave-pipelining to increase the counterfeiting challenge. Experimental results confirm that wave-pipelining true paths and false paths can be constructed in benchmark circuits successfully with only a negligible cost, thus thwarting the potential attack techniques.

cs.CR↗

arXiv2020

Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization

Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric. The data-path energy, including the computation energy and the data movement energy among the arithmetic units, claims a significant part of the total accelerator energy. By revisiting the basic physics of the arithmetic logic circuits, we show that the data-path energy is highly correlated with the bit flips when streaming the input operands into the arithmetic units, defined as the hamming distance of the input operand matrices. Based on the insight, we propose a post-training optimization algorithm and a hamming-distance-aware training algorithm to co-design and co-optimize the accelerator and the network synergistically. The experimental results based on post-layout simulation with MobileNetV2 demonstrate on average 2.85X data-path energy reduction and up to 8.51X data-path energy reduction for certain layers.

cs.CV↗

arXiv2020

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs). However, it remains an open problem, how to integrate NAS with Application-Specific Integrated Circuits (ASICs), despite them being the most powerful AI accelerating platforms. The major bottleneck comes from the large design freedom associated with ASIC designs. Moreover, with the consideration that multiple DNNs will run in parallel for different workloads with diverse layer operations and sizes, integrating heterogeneous ASIC sub-accelerators for distinct DNNs in one design can significantly boost performance, and at the same time further complicate the design space. To address these challenges, in this paper we build ASIC template set based on existing successful designs, described by their unique dataflows, so that the design space is significantly reduced. Based on the templates, we further propose a framework, namely NASAIC, which can simultaneously identify multiple DNN architectures and the associated heterogeneous ASIC accelerator design, such that the design specifications (specs) can be satisfied, while the accuracy can be maximized. Experimental results show that compared with successive NAS and ASIC design optimizations which lead to design spec violations, NASAIC can guarantee the results to meet the design specs with 17.77%, 2.49x, and 2.32x reductions on latency, energy, and area and with 0.76% accuracy loss. To the best of the authors' knowledge, this is the first work on neural architecture and ASIC accelerator design co-exploration.

cs.LG↗

arXiv2020

Scalable Double Regularization for 3D Nano-CT Reconstruction

Nano-CT (computerized tomography) has emerged as a non-destructive high-resolution cross-sectional imaging technique to effectively study the sub-$μ$m pore structure of shale, which is of fundamental importance to the evaluation and development of shale oil and gas. Nano-CT poses unique challenges to the inverse problem of reconstructing the 3D structure due to the lower signal-to-noise ratio (than Micro-CT) at the nano-scale, increased sensitivity to the misaligned geometry caused by the movement of object manipulator, limited sample size, and a larger volume of data at higher resolution. In this paper, we propose a scalable double regularization (SDR) method to utilize the entire dataset for simultaneous 3D structural reconstruction across slices through total variation regularization within slices and $L_1$ regularization between adjacent slices. SDR allows information borrowing both within and between slices, contrasting with the traditional methods that usually build on slice by slice reconstruction. We develop a scalable and memory-efficient algorithm by exploiting the systematic sparsity and consistent geometry induced by such Nano-CT data. We illustrate the proposed method using synthetic data and two Nano-CT imaging datasets of Jiulaodong (JLD) shale and Longmaxi (LMX) shale acquired in the Sichuan Basin. These numerical experiments show that the proposed method substantially outperforms selected alternatives both visually and quantitatively.

eess.IV↗

arXiv2019

RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing

Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a fundamental challenge to accelerate. This paper proposes a lightweight, commodity DRAM compliant, near-memory processing solution to accelerate personalized recommendation inference. The in-depth characterization of production-grade recommendation models shows that embedding operations with high model-, operator- and data-level parallelism lead to memory bandwidth saturation, limiting recommendation inference performance. We propose RecNMP which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models. RecNMP is specifically tailored to production environments with heavy co-location of operators on a single server. Several hardware/software co-optimization techniques such as memory-side caching, table-aware packet scheduling, and hot entry profiling are studied, resulting in up to 9.8x memory latency speedup over a highly-optimized baseline. Overall, RecNMP offers 4.2x throughput improvement and 45.8% memory energy savings.

cs.DC↗

arXiv2019

Aggregate Power Control of Heterogeneous TCL Populations Governed by Fokker-Planck Equations

This paper addresses the modeling and control of heterogeneous populations of thermostatically controlled loads (TCLs) operated by model predictive control (MPC) schemes at level of each TCL. It is shown that the dynamics of such TCLs populations can be described by a pair of Fokker-Planck equations coupled via the actions in the domain. The technique of input-output feedback linearization is used in the design of aggregate power control, which leads to a nonlinear system in closed loop. Well-posedness analysis is carried out to validate the developed control scheme. The closed-loop stability of the system is assessed by a rigorous analysis. A simulation study is contacted, and the obtained results confirm the validity and the effectiveness of the proposed approach.

math.OC↗

arXiv2019

Experimental realization of state transfer by quantum walks with two coins

Quantum state transfer between different sites is a significant problem for quantum networks and quantum computers. By selecting quantum walks with two coins as the basic model and two coin spaces as the communication carriers, we successfully implement quantum state transfer on various graphs (EPL, \textbf{124} (2018) 60009) \cite{Shang_2019}. Here, we demonstrate the experimental implementation of this scheme using IBM quantum experience platform. In particular, we show the transfer of Bell state, GHZ state and W state on complete graph on the quantum device. Also, we observe that our protocol has high fidelity by preforming quantum state tomography.

quant-ph↗

arXiv2019

PGU-net+: Progressive Growing of U-net+ for Automated Cervical Nuclei Segmentation

Automated cervical nucleus segmentation based on deep learning can effectively improve the quantitative analysis of cervical cancer. However, accurate nuclei segmentation is still challenging. The classic U-net has not achieved satisfactory results on this task, because it mixes the information of different scales that affect each other, which limits the segmentation accuracy of the model. To solve this problem, we propose a progressive growing U-net (PGU-net+) model, which uses two paradigms to extract image features at different scales in a more independent way. First, we add residual modules between different scales of U-net, which enforces the model to learn the approximate shape of the annotation in the coarser scale, and to learn the residual between the annotation and the approximate shape in the finer scale. Second, we start to train the model with the coarsest part and then progressively add finer part to the training until the full model is included. When we train a finer part, we will reduce the learning rate of the previous coarser part, which further ensures that the model independently extracts information from different scales. We conduct several comparative experiments on the Herlev dataset. The experimental results show that the PGU-net+ has superior accuracy than the previous state-of-the-art methods on cervical nuclei segmentation.

eess.IV↗

arXiv2019

Function-on-Scalar Quantile Regression with Application to Mass Spectrometry Proteomics Data

Mass spectrometry proteomics, characterized by spiky, spatially heterogeneous functional data, can be used to identify potential cancer biomarkers. Existing mass spectrometry analyses utilize mean regression to detect spectral regions that are differentially expressed across groups. However, given the inter-patient heterogeneity that is a key hallmark of cancer, many biomarkers are only present at aberrant levels for a subset of, not all, cancer samples. Differences in these biomarkers can easily be missed by mean regression, but might be more easily detected by quantile-based approaches. Thus, we propose a unified Bayesian framework to perform quantile regression on functional responses. Our approach utilizes an asymmetric Laplace working likelihood, represents the functional coefficients with basis representations which enable borrowing of strength from nearby locations, and places a global-local shrinkage prior on the basis coefficients to achieve adaptive regularization. Different types of basis transform and continuous shrinkage priors can be used in our framework. A scalable Gibbs sampler is developed to generate posterior samples that can be used to perform Bayesian estimation and inference while accounting for multiple testing. Our framework performs quantile regression and coefficient regularization in a unified manner, allowing them to inform each other and leading to improvement in performance over competing methods as demonstrated by simulation studies. We also introduce an adjustment procedure to the model to improve its frequentist properties of posterior inference. We apply our model to identify proteomic biomarkers of pancreatic cancer that are differentially expressed for a subset of cancer patients compared to the normal controls, which were missed by previous mean-regression based approaches. Supplementary materials for this article are available online.

stat.ME↗

arXiv2019

Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images

Automatic and accurate segmentation for retinal and choroidal layers of Optical Coherence Tomography (OCT) is crucial for detection of various ocular diseases. However, because of the variations in different equipments, OCT data obtained from different manufacturers might encounter appearance discrepancy, which could lead to performance fluctuation to a deep neural network. In this paper, we propose an uncertainty-guided domain alignment method to aim at alleviating this problem to transfer discriminative knowledge across distinct domains. We disign a novel uncertainty-guided cross-entropy loss for boosting the performance over areas with high uncertainty. An uncertainty-guided curriculum transfer strategy is developed for the self-training (ST), which regards uncertainty as efficient and effective guidance to optimize the learning process in target domain. Adversarial learning with feature recalibration module (FRM) is applied to transfer informative knowledge from the domain feature spaces adaptively. The experiments on two OCT datasets show that the proposed methods can obtain significant segmentation improvements compared with the baseline models.

cs.CV↗

arXiv2019

Deep Instance-Level Hard Negative Mining Model for Histopathology Images

Histopathology image analysis can be considered as a Multiple instance learning (MIL) problem, where the whole slide histopathology image (WSI) is regarded as a bag of instances (i.e, patches) and the task is to predict a single class label to the WSI. However, in many real-life applications such as computational pathology, discovering the key instances that trigger the bag label is of great interest because it provides reasons for the decision made by the system. In this paper, we propose a deep convolutional neural network (CNN) model that addresses the primary task of a bag classification on a WSI and also learns to identify the response of each instance to provide interpretable results to the final prediction. We incorporate the attention mechanism into the proposed model to operate the transformation of instances and learn attention weights to allow us to find key patches. To perform a balanced training, we introduce adaptive weighing in each training bag to explicitly adjust the weight distribution in order to concentrate more on the contribution of hard samples. Based on the learned attention weights, we further develop a solution to boost the classification performance by generating the bags with hard negative instances. We conduct extensive experiments on colon and breast cancer histopathology data and show that our framework achieves state-of-the-art performance.

cs.CV↗

arXiv2019

Limits of Sums for Binomial and Eulerian Numbers and their Associated Distributions

We provide a unified, probabilistic approach using renewal theory to derive some novel limits of sums for the normalized binomial coefficients and for the normalized Eulerian numbers. We also investigate some corresponding results for their associated distributions -- the binomial distributions for the binomial coefficients and the Irwin-Hall distributions (uniform B-splines) for the Eulerian numbers.

math.CO↗

arXiv2018

Automated segmentaiton and classification of arterioles and venules using Cascading Dilated Convolutional Neural Networks

The change of retinal vasculature is an early sign of many vascular and systematic diseases, such as diabetes and hypertension. Different behaviors of retinal arterioles and venules form an important metric to measure the disease severity. Therefore, an accurate classification of arterioles and venules is of great necessity. In this work, we propose a novel architecture of deep convolutional neural network for segmenting and classifying arterioles and venules on retinal fundus images. This network takes the original color fundus image as inputs and multi-class labels as outputs. We adopt the encoding-decoding structure (Unet) as the backbone network of our proposed model. To improve the classification accuracy, we develop a special encoding path that couples InceptionV4 modules and Cascading Dilated Convolutions (CDCs) on top of the backbone network. The model is thus able to extract and fuse high-level semantic features from multi-scale receptive fields. The proposed method has outperformed the previous state-of-the-art method on DRIVE dataset with an accuracy of 0.955 $\pm$ 0.002.

cs.CV↗

arXiv2018

Multistep Speed Prediction on Traffic Networks: A Graph Convolutional Sequence-to-Sequence Learning Approach with Attention Mechanism

Multistep traffic forecasting on road networks is a crucial task in successful intelligent transportation system applications. To capture the complex non-stationary temporal dynamics and spatial dependency in multistep traffic-condition prediction, we propose a novel deep learning framework named attention graph convolutional sequence-to-sequence model (AGC-Seq2Seq). In the proposed deep learning framework, spatial and temporal dependencies are modeled through the Seq2Seq model and graph convolution network separately, and the attention mechanism along with a newly designed training method based on the Seq2Seq architecture is proposed to overcome the difficulty in multistep prediction and further capture the temporal heterogeneity of traffic pattern. We conduct numerical tests to compare AGC-Seq2Seq with other benchmark models using a real-world dataset. The results indicate that our model yields the best prediction performance in terms of various prediction error measures. Furthermore, the variation of spatiotemporal correlation of traffic conditions under different perdition steps and road segments is revealed through sensitivity analyses.

cs.LG↗

arXiv2018

Streaming Algorithms for Planar Convex Hulls

Many classical algorithms are known for computing the convex hull of a set of $n$ point in $\mathbb{R}^2$ using $O(n)$ space. For large point sets, whose size exceeds the size of the working space, these algorithms cannot be directly used. The current best streaming algorithm for computing the convex hull is computationally expensive, because it needs to solve a set of linear programs. In this paper, we propose simpler and faster streaming and W-stream algorithms for computing the convex hull. Our streaming algorithm has small pass complexity, which is roughly a square root of the current best bound, and it is simpler in the sense that our algorithm mainly relies on computing the convex hulls of smaller point sets. Our W-stream algorithms, one of which is deterministic and the other of which is randomized, have nearly-optimal tradeoff between the pass complexity and space usage, as we established by a new unconditional lower bound.

cs.CG↗

arXiv2018

Deep Learning for Quality Control of Subcortical Brain 3D Shape Models

We present several deep learning models for assessing the morphometric fidelity of deep grey matter region models extracted from brain MRI. We test three different convolutional neural net architectures (VGGNet, ResNet and Inception) over 2D maps of geometric features. Further, we present a novel geometry feature augmentation technique based on a parametric spherical mapping. Finally, we present an approach for model decision visualization, allowing human raters to see the areas of subcortical shapes most likely to be deemed of failing quality by the machine. Our training data is comprised of 5200 subjects from the ENIGMA Schizophrenia MRI cohorts, and our test dataset contains 1500 subjects from the ENIGMA Major Depressive Disorder cohorts. Our final models reduce human rater time by 46-70%. ResNet outperforms VGGNet and Inception for all of our predictive tasks.

q-bio.NC↗

arXiv2018

Computed Tomography Image Enhancement using 3D Convolutional Neural Network

Computed tomography (CT) is increasingly being used for cancer screening, such as early detection of lung cancer. However, CT studies have varying pixel spacing due to differences in acquisition parameters. Thick slice CTs have lower resolution, hindering tasks such as nodule characterization during computer-aided detection due to partial volume effect. In this study, we propose a novel 3D enhancement convolutional neural network (3DECNN) to improve the spatial resolution of CT studies that were acquired using lower resolution/slice thicknesses to higher resolutions. Using a subset of the LIDC dataset consisting of 20,672 CT slices from 100 scans, we simulated lower resolution/thick section scans then attempted to reconstruct the original images using our 3DECNN network. A significant improvement in PSNR (29.3087dB vs. 28.8769dB, p-value < 2.2e-16) and SSIM (0.8529dB vs. 0.8449dB, p-value < 2.2e-16) compared to other state-of-art deep learning methods is observed.

cs.CV↗