arXiv subjects

Cheng Li

Publications and source records attributed to Cheng Li.

At least 379 records · Page 21Linked to original sources

arXiv2020

Structure/property relationship of semi-crystalline polymer during tensile deformation: A molecular dynamics approach

A coarse-grained molecular dynamics model of linear polyethylene-like polymer chain system was built to investigate the responds of structure and mechanical properties during uniaxial deformation. The influence of chain length, temperature, and strain rate were studied. The molecular dynamic tests showed that yielding may governed by different mechanisms at temperatures above and below Tg. Melt-recrystallization was observed at higher temperature, and destruction of crystal structures was observed at lower temperatures beyond yield point. While the higher temperature and lower strain rate have similar effects on mechanical properties. The correlated influences of time and temperature in the microscopic structures are more complicated. The evolution of microscopic characteristics such as the orientation parameter, the bond length, and the content of trans-trans conformation were calculated from the simulation. The results showed that the temperature have double effects on polymer chains. Higher temperature on one hand makes the chains more flexible, while on the other hand shortens the relaxation time of polymers. It is the interaction of these two aspects that determine the orientation parameter. During deformation, the trans conformation has experienced a rising process after the first drop process. And these microscopic structure parameters exhibit critical transaction, which are closely related to the yield point. A hypothetical model was thus proposed to describe the micro-structure and property relations based the investigations of this study.

cond-mat.soft↗

arXiv2020

Estimating dust attenuation from galactic spectra. I. methodology and tests

We develop a method to estimate the dust attenuation curve of galaxies from full spectral fitting of their optical spectra. Motivated from previous studies, we separate the small-scale features from the large-scale spectral shape, by performing a moving average method to both the observed spectrum and the simple stellar population model spectra. The intrinsic dust-free model spectrum is then derived by fitting the observed ratio of the small-scale to large-scale (S/L) components with the S/L ratios of the SSP models. The selective dust attenuation curve is then determined by comparing the observed spectrum with the dust-free model spectrum. One important advantage of this method is that the estimated dust attenuation curve is independent of the shape of theoretical dust attenuation curves. We have done a series of tests on a set of mock spectra covering wide ranges of stellar age and metallicity. We show that our method is able to recover the input dust attenuation curve accurately, although the accuracy depends slightly on signal-to-noise ratio of the spectra. We have applied our method to a number of edge-on galaxies with obvious dust lanes from the ongoing MaNGA survey, deriving their dust attenuation curves and $E(B-V)$ maps, as well as dust-free images in $g$, $r$, and $i$ bands. These galaxies show obvious dust lane features in their original images, which largely disappear after we have corrected the effect of dust attenuation. The vertical brightness profiles of these galaxies become axis-symmetric and can well be fitted by a simple model proposed for the disk vertical structure. Comparing the estimated dust attenuation curve with the three commonly-adopted model curves, we find that the Calzetti curve provides the best description of the estimated curves for the inner region of galaxies, while the Milky Way and SMC curves work better for the outer region.

astro-ph.GA↗

arXiv2020

Hierarchically Fair Federated Learning

When the federated learning is adopted among competitive agents with siloed datasets, agents are self-interested and participate only if they are fairly rewarded. To encourage the application of federated learning, this paper employs a management strategy, i.e., more contributions should lead to more rewards. We propose a novel hierarchically fair federated learning (HFFL) framework. Under this framework, agents are rewarded in proportion to their pre-negotiated contribution levels. HFFL+ extends this to incorporate heterogeneous models. Theoretical analysis and empirical evaluation on several datasets confirm the efficacy of our frameworks in upholding fairness and thus facilitating federated learning in the competitive settings.

cs.LG↗

arXiv2020

Downstream Model Design of Pre-trained Language Model for Relation Extraction Task

Supervised relation extraction methods based on deep neural network play an important role in the recent information extraction field. However, at present, their performance still fails to reach a good level due to the existence of complicated relations. On the other hand, recently proposed pre-trained language models (PLMs) have achieved great success in multiple tasks of natural language processing through fine-tuning when combined with the model of downstream tasks. However, original standard tasks of PLM do not include the relation extraction task yet. We believe that PLMs can also be used to solve the relation extraction problem, but it is necessary to establish a specially designed downstream task model or even loss function for dealing with complicated relations. In this paper, a new network architecture with a special loss function is designed to serve as a downstream model of PLMs for supervised relation extraction. Experiments have shown that our method significantly exceeded the current optimal baseline models across multiple public datasets of relation extraction.

cs.CL↗

arXiv2020

Optimal Iterative Threshold-Kernel Estimation of Jump Diffusion Processes

In this paper, we propose a new threshold-kernel jump-detection method for jump-diffusion processes, which iteratively applies thresholding and kernel methods in an approximately optimal way to achieve improved finite-sample performance. We use the expected number of jump misclassifications as the objective function to optimally select the threshold parameter of the jump detection scheme. We prove that the objective function is quasi-convex and obtain a new second-order infill approximation of the optimal threshold in closed form. The approximate optimal threshold depends not only on the spot volatility, but also the jump intensity and the value of the jump density at the origin. Estimation methods for these quantities are then developed, where the spot volatility is estimated by a kernel estimator with thresholding and the value of the jump density at the origin is estimated by a density kernel estimator applied to those increments deemed to contain jumps by the chosen thresholding criterion. Due to the interdependency between the model parameters and the approximate optimal estimators built to estimate them, a type of iterative fixed-point algorithm is developed to implement them. Simulation studies for a prototypical stochastic volatility model show that it is not only feasible to implement the higher-order local optimal threshold scheme but also that this is superior to those based only on the first order approximation and/or on average values of the parameters over the estimation time period.

math.ST↗

arXiv2020

Incorporating Expert Prior in Bayesian Optimisation via Space Warping

Bayesian optimisation is a well-known sample-efficient method for the optimisation of expensive black-box functions. However when dealing with big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function. Since the function evaluations are expensive in terms of both money and time, it may be desirable to alleviate this problem. One approach to subside this cold start phase is to use prior knowledge that can accelerate the optimisation. In its standard form, Bayesian optimisation assumes the likelihood of any point in the search space being the optimum is equal. Therefore any prior knowledge that can provide information about the optimum of the function would elevate the optimisation performance. In this paper, we represent the prior knowledge about the function optimum through a prior distribution. The prior distribution is then used to warp the search space in such a way that space gets expanded around the high probability region of function optimum and shrinks around low probability region of optimum. We incorporate this prior directly in function model (Gaussian process), by redefining the kernel matrix, which allows this method to work with any acquisition function, i.e. acquisition agnostic approach. We show the superiority of our method over standard Bayesian optimisation method through optimisation of several benchmark functions and hyperparameter tuning of two algorithms: Support Vector Machine (SVM) and Random forest.

cs.LG↗

arXiv2020

DLBricks: Composable Benchmark Generation to Reduce Deep Learning Benchmarking Effort on CPUs (Extended)

The past few years have seen a surge of applying Deep Learning (DL) models for a wide array of tasks such as image classification, object detection, machine translation, etc. While DL models provide an opportunity to solve otherwise intractable tasks, their adoption relies on them being optimized to meet latency and resource requirements. Benchmarking is a key step in this process but has been hampered in part due to the lack of representative and up-to-date benchmarking suites. This is exacerbated by the fast-evolving pace of DL models. This paper proposes DLBricks, a composable benchmark generation design that reduces the effort of developing, maintaining, and running DL benchmarks on CPUs. DLBricks decomposes DL models into a set of unique runnable networks and constructs the original model's performance using the performance of the generated benchmarks. DLBricks leverages two key observations: DL layers are the performance building blocks of DL models and layers are extensively repeated within and across DL models. Since benchmarks are generated automatically and the benchmarking time is minimized, DLBricks can keep up-to-date with the latest proposed models, relieving the pressure of selecting representative DL models. Moreover, DLBricks allows users to represent proprietary models within benchmark suites. We evaluate DLBricks using $50$ MXNet models spanning $5$ DL tasks on $4$ representative CPU systems. We show that DLBricks provides an accurate performance estimate for the DL models and reduces the benchmarking time across systems (e.g. within $95\%$ accuracy and up to $4.4\times$ benchmarking time speedup on Amazon EC2 c5.xlarge).

cs.LG↗

arXiv2020

Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling

Scientific experiments are usually expensive due to complex experimental preparation and processing. Experimental design is therefore involved with the task of finding the optimal experimental input that results in the desirable output by using as few experiments as possible. Experimenters can often acquire the knowledge about the location of the global optimum. However, they do not know how to exploit this knowledge to accelerate experimental design. In this paper, we adopt the technique of Bayesian optimization for experimental design since Bayesian optimization has established itself as an efficient tool for optimizing expensive black-box functions. Again, it is unknown how to incorporate the expert prior knowledge about the global optimum into Bayesian optimization process. To address it, we represent the expert knowledge about the global optimum via placing a prior distribution on it and we then derive its posterior distribution. An efficient Bayesian optimization approach has been proposed via posterior sampling on the posterior distribution of the global optimum. We theoretically analyze the convergence of the proposed algorithm and discuss the robustness of incorporating expert prior. We evaluate the efficiency of our algorithm by optimizing synthetic functions and tuning hyperparameters of classifiers along with a real-world experiment on the synthesis of short polymer fiber. The results clearly demonstrate the advantages of our proposed method.

stat.ML↗

arXiv2020

DLSpec: A Deep Learning Task Exchange Specification

Deep Learning (DL) innovations are being introduced at a rapid pace. However, the current lack of standard specification of DL tasks makes sharing, running, reproducing, and comparing these innovations difficult. To address this problem, we propose DLSpec, a model-, dataset-, software-, and hardware-agnostic DL specification that captures the different aspects of DL tasks. DLSpec has been tested by specifying and running hundreds of DL tasks.

cs.LG↗

arXiv2020

MLModelScope: A Distributed Platform for Model Evaluation and Benchmarking at Scale

Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that researchers are hard-pressed to analyze and study them. The complicated procedures for evaluating innovations, along with the lack of standard and efficient ways of specifying and provisioning ML/DL evaluation, is a major "pain point" for the community. This paper proposes MLModelScope, an open-source, framework/hardware agnostic, extensible and customizable design that enables repeatable, fair, and scalable model evaluation and benchmarking. We implement the distributed design with support for all major frameworks and hardware, and equip it with web, command-line, and library interfaces. To demonstrate MLModelScope's capabilities we perform parallel evaluation and show how subtle changes to model evaluation pipeline affects the accuracy and HW/SW stack choices affect performance.

cs.DC↗

arXiv2020

Benanza: Automatic $μ$Benchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUs

As Deep Learning (DL) models have been increasingly used in latency-sensitive applications, there has been a growing interest in improving their response time. An important venue for such improvement is to profile the execution of these models and characterize their performance to identify possible optimization opportunities. However, the current profiling tools lack the highly desired abilities to characterize ideal performance, identify sources of inefficiency, and quantify the benefits of potential optimizations. Such deficiencies have led to slow characterization/optimization cycles that cannot keep up with the fast pace at which new DL models are introduced. We propose Benanza, a sustainable and extensible benchmarking and analysis design that speeds up the characterization/optimization cycle of DL models on GPUs. Benanza consists of four major components: a model processor that parses models into an internal representation, a configurable benchmark generator that automatically generates micro-benchmarks given a set of models, a database of benchmark results, and an analyzer that computes the "lower-bound" latency of DL models using the benchmark data and informs optimizations of model execution. The "lower-bound" latency metric estimates the ideal model execution on a GPU system and serves as the basis for identifying optimization opportunities in frameworks or system libraries. We used Benanza to evaluate 30 ONNX models in MXNet, ONNX Runtime, and PyTorch on 7 GPUs ranging from Kepler to the latest Turing, and identified optimizations in parallel layer execution, cuDNN convolution algorithm selection, framework inefficiency, layer fusion, and using Tensor Cores.

cs.LG↗

arXiv2020

Accelerated Bayesian Optimization throughWeight-Prior Tuning

Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance is selected from a set of standard covariance functions. From a weight-space view, this models the objective as a linear function in a feature space implied by the given covariance K, with an arbitrary Gaussian weight prior ${\bf w} \sim \mathcal{N} ({\bf 0}, {\bf I})$. In many practical applications there is data available that has a similar (covariance) structure to the objective, but which, having different form, cannot be used directly in standard transfer learning. In this paper we show how such auxiliary data may be used to construct a GP covariance corresponding to a more appropriate weight prior for the objective function. Building on this, we show that we may accelerate BO by modeling the objective function using this (learned) weight prior, which we demonstrate on both test functions and a practical application to short-polymer fibre manufacture.

stat.ML↗

arXiv2019

Non-rigid Registration Method between 3D CT Liver Data and 2D Ultrasonic Images based on Demons Model

The non-rigid registration between CT data and ultrasonic images of liver can facilitate the diagnosis and treatment, which has been widely studied in recent years. To improve the registration accuracy of the Demons model on the non-rigid registration between 3D CT liver data and 2D ultrasonic images, a novel boundary extraction and enhancement method based on radial directional local intuitionistic fuzzy entropy in the polar coordinates has been put forward, and a new registration workflow has been provided. Experiments show that our method can acquire high-accuracy registration results. Experiments also show that the accuracy of the results of our method is higher than that of the original Demons method and the Demons method using simulated ultrasonic image by Field II. The operation time of our registration workflow is about 30 seconds, and it can be used in the surgery.

eess.IV↗

arXiv2019

X-Net: Brain Stroke Lesion Segmentation Based on Depthwise Separable Convolution and Long-range Dependencies

The morbidity of brain stroke increased rapidly in the past few years. To help specialists in lesion measurements and treatment planning, automatic segmentation methods are critically required for clinical practices. Recently, approaches based on deep learning and methods for contextual information extraction have served in many image segmentation tasks. However, their performances are limited due to the insufficient training of a large number of parameters, which sometimes fail in capturing long-range dependencies. To address these issues, we propose a depthwise separable convolution based X-Net that designs a nonlocal operation namely Feature Similarity Module (FSM) to capture long-range dependencies. The adopted depthwise convolution allows to reduce the network size, while the developed FSM provides a more effective, dense contextual information extraction and thus facilitates better segmentation. The effectiveness of X-Net was evaluated on an open dataset Anatomical Tracings of Lesions After Stroke (ATLAS) with superior performance achieved compared to other six state-of-the-art approaches. We make our code and models available at https://github.com/Andrewsher/X-Net.

eess.IV↗

arXiv2019

Accelerating Reduction and Scan Using Tensor Core Units

Driven by deep learning, there has been a surge of specialized processors for matrix multiplication, referred to as TensorCore Units (TCUs). These TCUs are capable of performing matrix multiplications on small matrices (usually 4x4 or 16x16) to accelerate the convolutional and recurrent neural networks in deep learning workloads. In this paper we leverage NVIDIA's TCU to express both reduction and scan with matrix multiplication and show the benefits -- in terms of program simplicity, efficiency, and performance. Our algorithm exercises the NVIDIA TCUs which would otherwise be idle, achieves 89%-98% of peak memory copy bandwidth, and is orders of magnitude faster (up to 100x for reduction and 3x for scan) than state-of-the-art methods for small segment sizes -- common in machine learning and scientific applications. Our algorithm achieves this while decreasing the power consumption by up to 22% for reduction and16%for scan.

cs.PF↗

arXiv2019

The Design and Implementation of a Scalable DL Benchmarking Platform

The current Deep Learning (DL) landscape is fast-paced and is rife with non-uniform models, hardware/software (HW/SW) stacks, but lacks a DL benchmarking platform to facilitate evaluation and comparison of DL innovations, be it models, frameworks, libraries, or hardware. Due to the lack of a benchmarking platform, the current practice of evaluating the benefits of proposed DL innovations is both arduous and error-prone - stifling the adoption of the innovations. In this work, we first identify $10$ design features which are desirable within a DL benchmarking platform. These features include: performing the evaluation in a consistent, reproducible, and scalable manner, being framework and hardware agnostic, supporting real-world benchmarking workloads, providing in-depth model execution inspection across the HW/SW stack levels, etc. We then propose MLModelScope, a DL benchmarking platform design that realizes the $10$ objectives. MLModelScope proposes a specification to define DL model evaluations and techniques to provision the evaluation workflow using the user-specified HW/SW stack. MLModelScope defines abstractions for frameworks and supports board range of DL models and evaluation scenarios. We implement MLModelScope as an open-source project with support for all major frameworks and hardware architectures. Through MLModelScope's evaluation and automated analysis workflows, we performed case-study analyses of $37$ models across $4$ systems and show how model, hardware, and framework selection affects model accuracy and performance under different benchmarking scenarios. We further demonstrated how MLModelScope's tracing capability gives a holistic view of model execution and helps pinpoint bottlenecks.

cs.DC↗

arXiv2019

Estimating the molecular gas mass of low-redshift galaxies from a combination of mid-infrared luminosity and optical properties

We present CO(J=1-0) and/or CO(J=2-1) spectroscopy for 31 galaxies selected from the ongoing MaNGA survey, obtained with multiple telescopes. This sample is combined with CO observations from the literature to study the correlation of the CO luminosities ($L_{\rm CO(1-0)}$) with the mid-infrared luminosities at 12 ($L_{12 μm}$) and 22 $μ$m ($L_{\rm 22 μm}$), as well as the dependence of the residuals on a variety of galaxy properties. The correlation with $L_{\rm 12 μm}$ is tighter and more linear, but galaxies with relatively low stellar masses and blue colors fall significantly below the mean $L_{\rm CO(1-0)}-L_{\rm 12μm}$ relation. We propose a new estimator of the CO(1-0) luminosity (and thus the total molecular gas mass) that is a linear combination of three parameters: $L_{\rm 12 μm}$, $M_\ast$ and $g-r$. We show that, with a scatter of only 0.18 dex in log $(L_{\rm CO(1-0)})$, this estimator provides unbiased estimates for galaxies of different properties and types. An immediate application of this estimator to a compiled sample of galaxies with only CO(J=2-1) observations yields a distribution of the CO(J=2-1) to CO(J=1-0) luminosity ratios ($R21$) that agrees well with the distribution of real observations, in terms of both the median and the shape. Application of our estimator to the current MaNGA sample reveals a gas-poor population of galaxies that are predominantly early-type and show no correlation between molecular gas-to-stellar mass ratio and star formation rate, in contrast to gas-rich galaxies. We also provide alternative estimators with similar scatters, based on $r$ and/or $z$ band luminosities instead of $M_\ast$. These estimators serve as cheap and convenient $M_{\rm mol}$ proxies to be potentially applied to large samples of galaxies, thus allowing statistical studies of gas-related processes of galaxies.

astro-ph.GA↗

arXiv2019

Self-Adaptive Network Pruning

Deep convolutional neural networks have been proved successful on a wide range of tasks, yet they are still hindered by their large computation cost in many industrial scenarios. In this paper, we propose to reduce such cost for CNNs through a self-adaptive network pruning method (SANP). Our method introduces a general Saliency-and-Pruning Module (SPM) for each convolutional layer, which learns to predict saliency scores and applies pruning for each channel. Given a total computation budget, SANP adaptively determines the pruning strategy with respect to each layer and each sample, such that the average computation cost meets the budget. This design allows SANP to be more efficient in computation, as well as more robust to datasets and backbones. Extensive experiments on 2 datasets and 3 backbones show that SANP surpasses state-of-the-art methods in both classification accuracy and pruning rate.

cs.LG↗