arXiv subjects

Xin Liang

Publications and source records attributed to Xin Liang.

97 records · Page 6Linked to original sources

arXiv2020

Convergence Analysis of Extended LOBPCG for Computing Extreme Eigenvalues

This paper is concerned with the convergence analysis of an extended variation of the locally optimal preconditioned conjugate gradient method (LOBPCG) for the extreme eigenvalue of a Hermitian matrix polynomial which admits some extended form of Rayleigh quotient. This work is a generalization of the analysis by Ovtchinnikov (SIAM J. Numer. Anal., 46(5):2567-2592, 2008). As instances, the algorithms for definite matrix pairs and hyperbolic quadratic matrix polynomials are shown to be globally convergent and to have an asymptotically local convergence rate. Also, numerical examples are given to illustrate the convergence.

math.NA↗

arXiv2019

DeepSZ: A Novel Framework to Compress Deep Neural Networks by Using Error-Bounded Lossy Compression

DNNs have been quickly and broadly exploited to improve the data analysis quality in many complex science and engineering applications. Today's DNNs are becoming deeper and wider because of increasing demand on the analysis quality and more and more complex applications to resolve. The wide and deep DNNs, however, require large amounts of resources, significantly restricting their utilization on resource-constrained systems. Although some network simplification methods have been proposed to address this issue, they suffer from either low compression ratios or high compression errors, which may introduce a costly retraining process for the target accuracy. In this paper, we propose DeepSZ: an accuracy-loss bounded neural network compression framework, which involves four key steps: network pruning, error bound assessment, optimization for error bound configuration, and compressed model generation, featuring a high compression ratio and low encoding time. The contribution is three-fold. (1) We develop an adaptive approach to select the feasible error bounds for each layer. (2) We build a model to estimate the overall loss of accuracy based on the accuracy degradation caused by individual decompressed layers. (3) We develop an efficient optimization algorithm to determine the best-fit configuration of error bounds in order to maximize the compression ratio under the user-set accuracy constraint. Experiments show that DeepSZ can compress AlexNet and VGG-16 on the ImageNet by a compression ratio of 46X and 116X, respectively, and compress LeNet-300-100 and LeNet-5 on the MNIST by a compression ratio of 57X and 56X, respectively, with only up to 0.3% loss of accuracy. Compared with other state-of-the-art methods, DeepSZ can improve the compression ratio by up to 1.43X, the DNN encoding performance by up to 4.0X (with four Nvidia Tesla V100 GPUs), and the decoding performance by up to 6.2X.

cs.CV↗

arXiv2019

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by HPC applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are the two leading lossy compressors available to compress scientific data sets. However, their performance is not consistent across different data sets and across different fields of some data sets: for some fields SZ provides better compression performance, while other fields are better compressed with ZFP. This situation raises the need for an automatic online (during compression) selection between SZ and ZFP, with a minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. We implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70% with the same level of data distortion because of very accurate selection (around 99%) of the best-fit compressor, with little overhead (less than 7% in the experiments).

cs.DC↗

arXiv2018

Fixed-PSNR Lossy Compression for Scientific Data

Error-controlled lossy compression has been studied for years because of extremely large volumes of data being produced by today's scientific simulations. None of existing lossy compressors, however, allow users to fix the peak signal-to-noise ratio (PSNR) during compression, although PSNR has been considered as one of the most significant indicators to assess compression quality. In this paper, we propose a novel technique providing a fixed-PSNR lossy compression for scientific data sets. We implement our proposed method based on the SZ lossy compression framework and release the code as an open-source toolkit. We evaluate our fixed-PSNR compressor on three real-world high-performance computing data sets. Experiments show that our solution has a high accuracy in controlling PSNR, with an average deviation of 0.1 ~ 5.0 dB on the tested data sets.

cs.IT↗

arXiv2018

Improving Performance of Iterative Methods by Lossy Checkponting

Iterative methods are commonly used approaches to solve large, sparse linear systems, which are fundamental operations for many modern scientific simulations. When the large-scale iterative methods are running with a large number of ranks in parallel, they have to checkpoint the dynamic variables periodically in case of unavoidable fail-stop errors, requiring fast I/O systems and large storage space. To this end, significantly reducing the checkpointing overhead is critical to improving the overall performance of iterative methods. Our contribution is fourfold. (1) We propose a novel lossy checkpointing scheme that can significantly improve the checkpointing performance of iterative methods by leveraging lossy compressors. (2) We formulate a lossy checkpointing performance model and derive theoretically an upper bound for the extra number of iterations caused by the distortion of data in lossy checkpoints, in order to guarantee the performance improvement under the lossy checkpointing scheme. (3) We analyze the impact of lossy checkpointing (i.e., extra number of iterations caused by lossy checkpointing files) for multiple types of iterative methods. (4)We evaluate the lossy checkpointing scheme with optimal checkpointing intervals on a high-performance computing environment with 2,048 cores, using a well-known scientific computation package PETSc and a state-of-the-art checkpoint/restart toolkit. Experiments show that our optimized lossy checkpointing scheme can significantly reduce the fault tolerance overhead for iterative methods by 23%~70% compared with traditional checkpointing and 20%~58% compared with lossless-compressed checkpointing, in the presence of system failures.

cs.DC↗

arXiv2017

Single Layer In-O Atomic Sheets as Phonon and Electron and Barriers in ZnO-In2O3 Natural Superlattices: Implications for Thermoelectricity

The phases in the ZnO half of the ZnO-In2O3 binary system are natural superlattices consisting of periodic stacking of single InO2 sheets separated by indium doped ZnO blocks with a spacing that depends on the composition according to the relationship In2O3(ZnO)k. Characterization by combined, atomic resolution, aberration-corrected scanning transmission electron microscopy (STEM) and electron energy loss spectroscopy (EELS) analysis, indicates that the atomic structure of each InO2 layer consists of a single continuous sheet of octahedrally-coordinated InO2. The sheets also are crystallographic inversion boundaries. Analysis of the electrical conductivity, thermal conductivity and Seebeck coefficient data at 800 oC indicates that the InO2 sheets not only decrease thermal conductivity by phonon scattering but also create potential barriers to electron transport. The origin of the potential barriers, the role of piezoelectric effects and their dependence on superlattice spacing is discussed qualitatively. It is also argued that the crystallographically-aligned InO2 sheets within individual grains are also transport barriers in randomly oriented polycrystalline materials.

cond-mat.mtrl-sci↗

arXiv2016

Morphology of Fly Larval Class IV Dendrites Accords with a Random Branching and Contact Based Branch Deletion Model

Dendrites are branched neuronal processes that receive input signals from other neurons or the outside world [1]. To maintain connectivity as the organism grows, dendrites must also continue to grow. For example, the dendrites in the peripheral nervous system continue to grow and branch to maintain proper coverage of their receptor fields [2, 3, 4, 5]. One such neuron is the Drosophila melanogaster class IV dendritic arborization neuron [6]. The dendritic arbors of these neurons tile the larval surface [7], where they detect localized noxious stimuli, such as jabs from parasitic wasps [8]. In the present study, we used a novel measure, the hitting probability, to show that the class IV neuron forms a tight mesh that covers the larval surface. Furthermore, we found that the mesh size remains largely unchanged during the larval stages, despite a dramatic increase in overall size of the neuron and the larva. We also found that the class IV dendrites are dense (assayed with the fractal dimension) and uniform (assayed with the lacunarity) throughout the larval stages. To understand how the class IV neuron maintains its morphology during larval development, we constructed a mathematical model based on random branching and self-avoidance. We found that if the branching rate is uniform in space and time and that if all contacting branches are deleted, we can reproduce the branch length distribution, mesh size and density of the class IV dendrites throughout the larval stages. Thus, a simple set of statistical rules can generate and maintain a complex branching morphology during growth.

q-bio.NC↗