arXiv subjects

Chao Li

Publications and source records attributed to Chao Li.

At least 469 records · Page 26Linked to original sources

arXiv2020

DLFusion: An Auto-Tuning Compiler for Layer Fusion on Deep Neural Network Accelerator

Many hardware vendors have introduced specialized deep neural networks (DNN) accelerators owing to their superior performance and efficiency. As such, how to generate and optimize the code for the hardware accelerator becomes an important yet less explored problem. In this paper, we perform the compiler-stage optimization study using a novel and representative Cambricon DNN accelerator and demonstrate that the code optimization knobs play an important role in unleashing the potential of hardware computational horsepower. However, even only two studied code optimization knobs, namely the number of cores and layer fusion scheme, present an enormous search space that prevents the naive brute-force search. This work introduces a joint, auto-tuning optimization framework to address this challenge. We first use a set of synthesized DNN layers to study the interplay between the hardware performance and layer characteristics. Based on the insights, we extract the operation count and feature map channel size as each layer's characteristics and derive a joint optimization strategy to decide the performance-optimal core number and fusion scheme. We evaluate the performance of the proposed approach using a set of representative DNN models and show that it achieves the minimal of 3.6x and the maximal of 7.9x performance speedup compared to no optimization baseline. We also show that the achieved speedup is close to the oracle case that is based on a reduced brute-force search but with much less search time.

cs.DC↗

arXiv2020

An Empirical-cum-Statistical Approach to Power-Performance Characterization of Concurrent GPU Kernels

Growing deployment of power and energy efficient throughput accelerators (GPU) in data centers demands enhancement of power-performance co-optimization capabilities of GPUs. Realization of exascale computing using accelerators requires further improvements in power efficiency. With hardwired kernel concurrency enablement in accelerators, inter- and intra-workload simultaneous kernels computation predicts increased throughput at lower energy budget. To improve Performance-per-Watt metric of the architectures, a systematic empirical study of real-world throughput workloads (with concurrent kernel execution) is required. To this end, we propose a multi-kernel throughput workload generation framework that will facilitate aggressive energy and performance management of exascale data centers and will stimulate synergistic power-performance co-optimization of throughput architectures. Also, we demonstrate a multi-kernel throughput benchmark suite based on the framework that encapsulates symmetric, asymmetric and co-existing (often appears together) kernel based workloads. On average, our analysis reveals that spatial and temporal concurrency within kernel execution in throughput architectures saves energy consumption by 32%, 26% and 33% in GTX470, Tesla M2050 and Tesla K20 across 12 benchmarks. Concurrency and enhanced utilization are often correlated but do not imply significant deviation in power dissipation. Diversity analysis of proposed multi-kernels confirms characteristic variation and power-profile diversity within the suite. Besides, we explain several findings regarding power-performance co-optimization of concurrent throughput workloads.

cs.DC↗

arXiv2020

How Far Does BERT Look At:Distance-based Clustering and Analysis of BERT$'$s Attention

Recent research on the multi-head attention mechanism, especially that in pre-trained models such as BERT, has shown us heuristics and clues in analyzing various aspects of the mechanism. As most of the research focus on probing tasks or hidden states, previous works have found some primitive patterns of attention head behavior by heuristic analytical methods, but a more systematic analysis specific on the attention patterns still remains primitive. In this work, we clearly cluster the attention heatmaps into significantly different patterns through unsupervised clustering on top of a set of proposed features, which corroborates with previous observations. We further study their corresponding functions through analytical study. In addition, our proposed features can be used to explain and calibrate different attention heads in Transformer models.

cs.CL↗

arXiv2020

Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration

Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) restoration, which includes the tasks of denoising, compressed HSI reconstruction and inpainting. Unfortunately, while its restoration performance benefits from more spectral bands, its runtime also substantially increases. In this paper, we claim that the HSI lies in a global spectral low-rank subspace, and the spectral subspaces of each full band patch group should lie in this global low-rank subspace. This motivates us to propose a unified paradigm combining the spatial and spectral properties for HSI restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity from the low-rank orthogonal basis exploration. An efficient alternating minimization algorithm with rank adaptation is developed. It is done by first solving a fidelity term-related problem for the update of a latent input image, and then learning a low-dimensional orthogonal basis and the related reduced image from the latent input image. Subsequently, non-local low-rank denoising is developed to refine the reduced image and orthogonal basis iteratively. Finally, the experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority with respect to state-of-the-art HSI restoration methods.

eess.IV↗

arXiv2020

Dihedral rigidity of parabolic polyhedrons in hyperbolic spaces

In this note, we establish the dihedral rigidity phenomenon for a collection of parabolic polyhedrons enclosed by horospheres in hyperbolic manifolds, extending Gromov's comparison theory to metrics with negative scalar curvature lower bounds. Our result is a localization of the positive mass theorem for asymptotically hyperbolic manifolds. We also motivate and formulate some open questions concerning related rigidity phenomenon and convergence of metrics with scalar curvature lower bounds.

math.DG↗

arXiv2020

Further Study of Planar Functions in Characteristic Two

Planar functions are of great importance in the constructions of DES-like iterated ciphers, error-correcting codes, signal sets and the area of mathematics. They are defined over finite fields of odd characteristic originally and generalized by Y. Zhou \cite{Zhou} in even characteristic. In 2016, L. Qu \cite{Q} proposed a new approach to constructing quadratic planar functions over $\F_{2^n}$. Very recently, D. Bartoli and M. Timpanella \cite{Bartoli} characterized the condition on coefficients $a,b$ such that the function $f_{a,b}(x)=ax^{2^{2m}+1}+bx^{2^m+1} \in\F_{2^{3m}}[x]$ is a planar function over $\F_{2^{3m}}$ by the Hasse-Weil bound. In this paper, using the Lang-Weil bound, a generalization of the Hasse-Weil bound, and the new approach introduced in \cite{Q}, we completely characterize the necessary and sufficient conditions on coefficients of four classes of planar functions over $\F_{q^k}$, where $q=2^m$ with $m$ sufficiently large (see Theorem \ref{main}). The first and last classes of them are over $\F_{q^2}$ and $\F_{q^4}$ respectively, while the other two classes are over $\F_{q^3}$. One class over $\F_{q^3}$ is an extension of $f_{a,b}(x)$ investigated in \cite{Bartoli}, while our proofs seem to be much simpler. In addition, although the planar binomial over $\F_{q^2}$ of our results is finally a known planar monomial, we also answer the necessity at the same time and solve partially an open problem for the binomial case proposed in \cite{Q}.

math.AG↗

arXiv2020

Joint Generative Learning and Super-Resolution For Real-World Camera-Screen Degradation

In real-world single image super-resolution (SISR) task, the low-resolution image suffers more complicated degradations, not only downsampled by unknown kernels. However, existing SISR methods are generally studied with the synthetic low-resolution generation such as bicubic interpolation (BI), which greatly limits their performance. Recently, some researchers investigate real-world SISR from the perspective of the camera and smartphone. However, except the acquisition equipment, the display device also involves more complicated degradations. In this paper, we focus on the camera-screen degradation and build a real-world dataset (Cam-ScreenSR), where HR images are original ground truths from the previous DIV2K dataset and corresponding LR images are camera-captured versions of HRs displayed on the screen. We conduct extensive experiments to demonstrate that involving more real degradations is positive to improve the generalization of SISR models. Moreover, we propose a joint two-stage model. Firstly, the downsampling degradation GAN(DD-GAN) is trained to model the degradation and produces more various of LR images, which is validated to be efficient for data augmentation. Then the dual residual channel attention network (DuRCAN) learns to recover the SR image. The weighted combination of L1 loss and proposed Laplacian loss are applied to sharpen the high-frequency edges. Extensive experimental results in both typical synthetic and complicated real-world degradations validate the proposed method outperforms than existing SOTA models with less parameters, faster speed and better visual results. Moreover, in real captured photographs, our model also delivers best visual quality with sharper edge, less artifacts, especially appropriate color enhancement, which has not been accomplished by previous methods.

eess.IV↗

arXiv2020

NF-Crowd: Nearly-free Blockchain-based Crowdsourcing

Advancements in distributed ledger technologies are rapidly driving the rise of decentralized crowdsourcing systems on top of open smart contract platforms like Ethereum. While decentralized blockchain-based crowdsourcing provides numerous benefits compared to centralized solutions, current implementations of decentralized crowdsourcing suffer from fundamental scalability limitations by requiring all participants to pay a small transaction fee every time they interact with the blockchain. This increases the cost of using decentralized crowdsourcing solutions, resulting in a total payment that could be even higher than the price charged by centralized crowdsourcing platforms. This paper proposes a novel suite of protocols called NF-Crowd that resolves the scalability issue by reducing the lower bound of the total cost of a decentralized crowdsourcing project to O(1). NF-Crowd is a highly reliable solution for scaling decentralized crowdsourcing. We prove that as long as participants of a project powered by NF-Crowd are rational, the O(1) lower bound of cost could be reached regardless of the scale of the crowd. We also demonstrate that as long as at least one participant of a project powered by NF-Crowd is honest, the project cannot be aborted and the results are guaranteed to be correct. We design NF-Crowd protocols for a representative type of project named crowdsourcing contest with open community review (CC-OCR). We implement the protocols over the Ethereum official test network. Our results demonstrate that NF-Crowd protocols can reduce the cost of running a CC-OCR project to less than $2 regardless of the scale of the crowd, providing a significant cost benefit in adopting decentralized crowdsourcing solutions.

cs.CR↗

arXiv2020

User independent Emotion Recognition with Residual Signal-Image Network

User independent emotion recognition with large scale physiological signals is a tough problem. There exist many advanced methods but they are conducted under relatively small datasets with dozens of subjects. Here, we propose Res-SIN, a novel end-to-end framework using Electrodermal Activity(EDA) signal images to classify human emotion. We first apply convex optimization-based EDA (cvxEDA) to decompose signals and mine the static and dynamic emotion changes. Then, we transform decomposed signals to images so that they can be effectively processed by CNN frameworks. The Res-SIN combines individual emotion features and external emotion benchmarks to accelerate convergence. We evaluate our approach on the PMEmo dataset, the currently largest emotional dataset containing music and EDA signals. To the best of author's knowledge, our method is the first attempt to classify large scale subject-independent emotion with 7962 pieces of EDA signals from 457 subjects. Experimental results demonstrate the reliability of our model and the binary classification accuracy of 73.65% and 73.43% on arousal and valence dimension can be used as a baseline.

cs.CV↗

arXiv2020

Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition

Graph Convolutional Networks (GCNs) have attracted increasing interests for the task of skeleton-based action recognition. The key lies in the design of the graph structure, which encodes skeleton topology information. In this paper, we propose Dynamic GCN, in which a novel convolutional neural network named Contextencoding Network (CeN) is introduced to learn skeleton topology automatically. In particular, when learning the dependency between two joints, contextual features from the rest joints are incorporated in a global manner. CeN is extremely lightweight yet effective, and can be embedded into a graph convolutional layer. By stacking multiple CeN-enabled graph convolutional layers, we build Dynamic GCN. Notably, as a merit of CeN, dynamic graph topologies are constructed for different input samples as well as graph convolutional layers of various depths. Besides, three alternative context modeling architectures are well explored, which may serve as a guideline for future research on graph topology learning. CeN brings only ~7% extra FLOPs for the baseline model, and Dynamic GCN achieves better performance with $2\times$~$4\times$ fewer FLOPs than existing methods. By further combining static physical body connections and motion modalities, we achieve state-of-the-art performance on three large-scale benchmarks, namely NTU-RGB+D, NTU-RGB+D 120 and Skeleton-Kinetics.

cs.CV↗

arXiv2020

FeCAM: A Universal Compact Digital and Analog Content Addressable Memory Using Ferroelectric

Ferroelectric field effect transistors (FeFETs) are being actively investigated with the potential for in-memory computing (IMC) over other non-volatile memories (NVMs). Content Addressable Memories (CAMs) are a form of IMC that performs parallel searches for matched entries over a memory array for a given input query. CAMs are widely used for data-centric applications that involve pattern matching and search functionality. To accommodate the ever expanding data, it is attractive to resort to analog CAM for memory density improvement. However, the digital CAM design nowadays based on standard CMOS or emerging nonvolatile memories (e.g., resistive storage devices) is already challenging due to area, power, and cost penalties. Thus, it can be extremely expensive to achieve analog CAM with those technologies due to added cell components. As such, we propose, for the first time, a universal compact FeFET based CAM design, FeCAM, with search and storage functionality enabled in digital and analog domain simultaneously. By exploiting the multi-level-cell (MLC) states of FeFET, FeCAM can store and search inputs in either digital or analog domain. We perform a device-circuit co-design of the proposed FeCAM and validate its functionality and performance using an experimentally calibrated FeFET model. Circuit level simulation results demonstrate that FeCAM can either store continuous matching ranges or encode 3-bit data in a single CAM cell. When compared with the existing digital CMOS based CAM approaches, FeCAM is found to improve both memory density by 22.4X and energy saving by 8.6/3.2X for analog/digital modes, respectively. In the CAM-related application, our evaluations show that FeCAM can achieve 60.5X/23.1X saving in area/search energy compared with conventional CMOS based CAMs.

cs.ET↗

arXiv2020

Topology-Change-Aware Volumetric Fusion for Dynamic Scene Reconstruction

Topology change is a challenging problem for 4D reconstruction of dynamic scenes. In the classic volumetric fusion-based framework, a mesh is usually extracted from the TSDF volume as the canonical surface representation to help estimating deformation field. However, the surface and Embedded Deformation Graph (EDG) representations bring conflicts under topology changes since the surface mesh has fixed-connectivity but the deformation field can be discontinuous. In this paper, the classic framework is re-designed to enable 4D reconstruction of dynamic scene under topology changes, by introducing a novel structure of Non-manifold Volumetric Grid to the re-design of both TSDF and EDG, which allows connectivity updates by cell splitting and replication. Experiments show convincing reconstruction results for dynamic scenes of topology changes, as compared to the state-of-the-art methods.

cs.CV↗

arXiv2020

A 20-Gbps Beam-steered Infrared Wireless Link Enabled by a Passively Field-programmable Metasurface

Beam steering is one of the main challenges in energy-efficient and high-speed infrared light communication. To date, active beam-steering schemes based on a spatial light modulator (SLM) or micro-electrical mechanical system (MEMS) mirror, as well as the passive ones based on diffractive gratings, have been demonstrated for infrared light communication. Here, for the first time to our knowledge, an infrared beam is steered by 35° on one side empowered by a passively field-programmable metasurface. By combining the centralized control of wavelength and polarization, a remote passive metasurface can steer the infrared beam in a remote access point. The proposed system keeps scalability to support multiple beams, flexibility to steer the beam, high optical efficiency, simple and cheap devices on remote sides, and centralized control (low maintenance cost), while it avoids disadvantages such as grating loss, a small coverage area, and a bulky size. Based on the proposed beam-steering technology, we also demonstrated a proof-of-concept experiment system with a data rate of 20 Gbps.

physics.app-ph↗

arXiv2020

NTIRE 2020 Challenge on Video Quality Mapping: Methods and Results

This paper reviews the NTIRE 2020 challenge on video quality mapping (VQM), which addresses the issues of quality mapping from source video domain to target video domain. The challenge includes both a supervised track (track 1) and a weakly-supervised track (track 2) for two benchmark datasets. In particular, track 1 offers a new Internet video benchmark, requiring algorithms to learn the map from more compressed videos to less compressed videos in a supervised training manner. In track 2, algorithms are required to learn the quality mapping from one device to another when their quality varies substantially and weakly-aligned video pairs are available. For track 1, in total 7 teams competed in the final test phase, demonstrating novel and effective solutions to the problem. For track 2, some existing methods are evaluated, showing promising solutions to the weakly-supervised video quality mapping problem.

eess.IV↗

arXiv2020

Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

The research interest in specialized hardware accelerators for deep neural networks (DNN) spikes recently owing to their superior performance and efficiency. However, today's DNN accelerators primarily focus on accelerating specific "kernels" such as convolution and matrix multiplication, which are vital but only part of an end-to-end DNN-enabled application. Meaningful speedups over the entire application often require supporting computations that are, while massively parallel, ill-suited to DNN accelerators. Integrating a general-purpose processor such as a CPU or a GPU incurs significant data movement overhead and leads to resource under-utilization on the DNN accelerators. We propose Simultaneous Multi-mode Architecture (SMA), a novel architecture design and execution model that offers general-purpose programmability on DNN accelerators in order to accelerate end-to-end applications. The key to SMA is the temporal integration of the systolic execution model with the GPU-like SIMD execution model. The SMA exploits the common components shared between the systolic-array accelerator and the GPU, and provides lightweight reconfiguration capability to switch between the two modes in-situ. The SMA achieves up to 63% performance improvement while consuming 23% less energy than the baseline Volta architecture with TensorCore.

cs.DC↗

arXiv2020

EventWarden: A Decentralized Event-driven Proxy Service for Outsourcing Arbitrary Transactions in Ethereum-like Blockchains

Transactions represent a fundamental component in blockchains as they are the primary means for users to change the blockchain state. Current blockchain systems such as Bitcoin and Ethereum require users to constantly observe the state changes of interest or the events taking place in a blockchain and requires the user to explicitly release the required transactions to respond to the observed events in the blockchain. This paper proposes EventWarden, a decentralized event-driven proxy service for users to outsource transactions in Ethereum-like blockchains. EventWarden employs a novel combination of smart contracts and blockchain logs. EventWarden allows a user to create a proxy smart contract that specifies an interested event and also reserves an arbitrary transaction to release. Upon observing the occurrence of the prescribed event, anyone in the Blockchain network can call the proxy contract to earn the service fee reserved in the contract by proving to the contract that the event has been recorded into blockchain logs, which then automatically triggers the proxy contract to release the reserved transaction. We show that the reserved transaction can only get released from the proxy contract when the prescribed event has taken place. We also demonstrate that as long as a single member in the Blockchain network is incentivized by the service fee to call the proxy contract after the prescribed event has taken place, the reserved transaction is guaranteed to get released. We implement EventWarden over the Ethereum official test network. The results demonstrate that EventWarden is effective and is ready-to-use in practice.

cs.CR↗

arXiv2020

A discrete chemo-dynamical model of M87's globular clusters: Kinematics extending to ~ 400 kpc

We study the mass distribution and kinematics of the giant elliptical galaxy M87 (NGC 4486) using discrete chemo-dynamical, axisymmetric Jeans equation modelling. Our catalogue comprises 894 globular clusters (GCs) extending to a projected radius of $\sim 430$ kpc with line-of-sight velocities and colours, and Multi Unit Spectroscopic Explorer (MUSE) integral field unit data within the central $2.4$ kpc of the main galaxy. The gravitational potential for our models is a combination of a luminous matter potential with a varying mass-to-light ratio for the main galaxy, a supermassive black hole and a dark matter (DM) potential with a cusped or cored DM halo. The best-fitting models with either a cusped or a cored DM halo show no significant differences and both are acceptable. We obtain a total mass of $(2.16 \pm 0.38) \times 10^{13} M_{\odot}$ within $\sim$ 400 kpc. By including the stellar mass-to-light ratio gradient, the DM fraction increases from $\sim$ 26 percent (with no gradient) to $\sim$ 73 percent within $1\,R_e^{\rm maj}$ (major axis of half-light isophote, 14.2 kpc), and from $\sim$ 84 percent to $\sim$ 94 percent within $5\,R_e^{\rm maj}$ (71.2 kpc). Red GCs have moderate rotation with $V_{\rm max}/σ\sim$ 0.4, and blue GCs have weak rotation with $V_{\rm max}/σ\sim$ 0.1. Red GCs have tangential velocity dispersion anisotropy, while blue GCs are consistent with being nearly isotropic. Our results suggest that red GCs are more likely to be born in-situ, while blue GCs are more likely to be accreted.

astro-ph.GA↗

arXiv2020

Towards QoS-Aware and Resource-Efficient GPU Microservices Based on Spatial Multitasking GPUs In Datacenters

While prior researches focus on CPU-based microservices, they are not applicable for GPU-based microservices due to the different contention patterns. It is challenging to optimize the resource utilization while guaranteeing the QoS for GPU microservices. We find that the overhead is caused by inter microservice communication, GPU resource contention and imbalanced throughput within microservice pipeline. We propose Camelot, a runtime system that manages GPU micorservices considering the above factors. In Camelot, a global memory-based communication mechanism enables onsite data sharing that significantly reduces the end-to-end latencies of user queries. We also propose two contention aware resource allocation policies that either maximize the peak supported service load or minimize the resource usage at low load while ensuring the required QoS. The two policies consider the microservice pipeline effect and the runtime GPU resource contention when allocating resources for the microservices. Compared with state-of-the-art work, Camelot increases the supported peak load by up to 64.5% with limited GPUs, and reduces 35% resource usage at low load while achieving the desired 99%-ile latency target.

cs.DC↗