arXiv subjects

Evangelos Georganas

Publications and source records attributed to Evangelos Georganas.

21 records · Page 2Linked to original sources

arXiv2018

Mixed Precision Training of Convolutional Neural Networks using Integer Operations

The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular, FP16 accumulating into FP32 Micikevicius et al. (2017). On the other hand, while a lot of research has also happened in the domain of low and mixed-precision Integer training, these works either present results for non-SOTA networks (for instance only AlexNet for ImageNet-1K), or relatively small datasets (like CIFAR-10). In this work, we train state-of-the-art visual understanding neural networks on the ImageNet-1K dataset, with Integer operations on General Purpose (GP) hardware. In particular, we focus on Integer Fused-Multiply-and-Accumulate (FMA) operations which take two pairs of INT16 operands and accumulate results into an INT32 output.We propose a shared exponent representation of tensors and develop a Dynamic Fixed Point (DFP) scheme suitable for common neural network operations. The nuances of developing an efficient integer convolution kernel is examined, including methods to handle overflow of the INT32 accumulator. We implement CNN training for ResNet-50, GoogLeNet-v1, VGG-16 and AlexNet; and these networks achieve or exceed SOTA accuracy within the same number of iterations as their FP32 counterparts without any change in hyper-parameters and with a 1.8X improvement in end-to-end training throughput. To the best of our knowledge these results represent the first INT16 training results on GP hardware for ImageNet-1K dataset using SOTA CNNs and achieve highest reported accuracy using half-precision

cs.NE↗

arXiv2017

Extreme-Scale De Novo Genome Assembly

De novo whole genome assembly reconstructs genomic sequence from short, overlapping, and potentially erroneous DNA segments and is one of the most important computations in modern genomics. This work presents HipMER, a high-quality end-to-end de novo assembler designed for extreme scale analysis, via efficient parallelization of the Meraculous code. Genome assembly software has many components, each of which stresses different components of a computer system. This chapter explains the computational challenges involved in each step of the HipMer pipeline, the key distributed data structures, and communication costs in detail. We present performance results of assembling the human genome and the large hexaploid wheat genome on large supercomputers up to tens of thousands of cores.

cs.DC↗

arXiv2014

Constructing Performance Models for Dense Linear Algebra Algorithms on Cray XE Systems

Hiding or minimizing the communication cost is key in order to obtain good performance on large-scale systems. While communication overlapping attempts to hide communications cost, 2.5D communication avoiding algorithms improve performance scalability by reducing the volume of data transfers at the cost of extra memory usage. Both approaches can be used together or separately and the best choice depends on the machine, the algorithm and the problem size. Thus, the development of performance models is crucial to determine the best option for each scenario. In this paper, we present a methodology for constructing performance models for parallel numerical routines on Cray XE systems. Our models use portable benchmarks that measure computational cost and network characteristics, as well as performance degradation caused by simultaneous accesses to the network. We validate our methodology by constructing the performance models for the 2D and 2.5D approaches, with and without overlapping, of two matrix multiplication algorithms (Cannon's and SUMMA), triangular solve (TRSM) and Cholesky. We compare the estimations provided by these models with the experimental results using up to 24,576 cores of a Cray XE6 system and predict the performance of the algorithms on larger systems. Results prove that the estimations significantly improve when taking into account network contention.

cs.DC↗