arXiv subjects

Ariful Azad

Publications and source records attributed to Ariful Azad.

48 records · Page 3Linked to original sources

arXiv2020

FastSV: A Distributed-Memory Connected Component Algorithm with Fast Convergence

This paper presents a new distributed-memory algorithm called FastSV for finding connected components in an undirected graph. Our algorithm simplifies the classic Shiloach-Vishkin algorithm and employs several novel and efficient hooking strategies for faster convergence. We map different steps of FastSV to linear algebraic operations and implement them with the help of scalable graph libraries. FastSV uses sparse operations to avoid redundant work and optimized MPI communication to avoid bottlenecks. The resultant algorithm shows high-performance and scalability as it can find the connected components of a hyperlink graph with over 134B edges in 30 seconds using 262K cores on a Cray XC40 supercomputer. FastSV outperforms the state-of-the-art algorithm by an average speedup of 2.21x (max 4.27x) on a variety of real-world graphs.

cs.DS↗

arXiv2019

A GraphBLAS Approach for Subgraph Counting

Subgraph counting aims to count the occurrences of a subgraph template T in a given network G. The basic problem of computing structural properties such as counting triangles and other subgraphs has found applications in diverse domains. Recent biological, social, cybersecurity and sensor network applications have motivated solving such problems on massive networks with billions of vertices. The larger subgraph problem is known to be memory bounded and computationally challenging to scale; the complexity grows both as a function of T and G. In this paper, we study the non-induced tree subgraph counting problem, propose a novel layered softwarehardware co-design approach, and implement a shared-memory multi-threaded algorithm: 1) reducing the complexity of the parallel color-coding algorithm by identifying and pruning redundant graph traversal; 2) achieving a fully-vectorized implementation upon linear algebra kernels inspired by GraphBLAS, which significantly improves cache usage and maximizes memory bandwidth utilization. Experiments show that our implementation improves the overall performance over the state-of-the-art work by orders of magnitude and up to 660x for subgraph templates with size over 12 on a dual-socket Intel(R) Xeon(R) Platinum 8160 server. We believe our approach using GraphBLAS with optimized sparse linear algebra can be applied to other massive subgraph counting problems and emerging high-memory bandwidth hardware architectures.

cs.DC↗

arXiv2018

High-performance sparse matrix-matrix products on Intel KNL and multicore architectures

Sparse matrix-matrix multiplication (SpGEMM) is a computational primitive that is widely used in areas ranging from traditional numerical applications to recent big data analysis and machine learning. Although many SpGEMM algorithms have been proposed, hardware specific optimizations for multi- and many-core processors are lacking and a detailed analysis of their performance under various use cases and matrices is not available. We firstly identify and mitigate multiple bottlenecks with memory management and thread scheduling on Intel Xeon Phi (Knights Landing or KNL). Specifically targeting multi- and many-core processors, we develop a hash-table-based algorithm and optimize a heap-based shared-memory SpGEMM algorithm. We examine their performance together with other publicly available codes. Different from the literature, our evaluation also includes use cases that are representative of real graph algorithms, such as multi-source breadth-first search or triangle counting. Our hash-table and heap-based algorithms are showing significant speedups from libraries in the majority of the cases while different algorithms dominate the other scenarios with different matrix size, sparsity, compression factor and operation type. We wrap up in-depth evaluation results and make a recipe to give the best SpGEMM algorithm for target scenario. A critical finding is that hash-table-based SpGEMM gets a significant performance boost if the nonzeros are not required to be sorted within each row of the output matrix.

cs.DC↗

arXiv2018

Integrated Model, Batch and Domain Parallelism in Training Neural Networks

We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD). Our goal is to find an efficient parallelization strategy for a fixed batch size using $P$ processes. Our method is inspired by the communication-avoiding algorithms in numerical linear algebra. We see $P$ processes as logically divided into a $P_r \times P_c$ grid where the $P_r$ dimension is implicitly responsible for model/domain parallelism and the $P_c$ dimension is implicitly responsible for batch parallelism. In practice, the integrated matrix-based parallel algorithm encapsulates these types of parallelism automatically. We analyze the communication complexity and analytically demonstrate that the lowest communication costs are often achieved neither with pure model nor with pure data parallelism. We also show how the domain parallel approach can help in extending the theoretical scaling limit of the typical batch parallel method.

cs.LG↗

arXiv2018

Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ~819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.

stat.ML↗

arXiv2016

Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication

Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algorithms have been proposed and theoretically analyzed in the flat MPI model on Erdos-Renyi matrices, those algorithms had not been implemented in practice and their complexities had not been analyzed for the general case. In this work, we present the first ever implementation of the 3D SpGEMM formulation that also exploits multiple (intra-node and inter-node) levels of parallelism, achieving significant speedups over the state-of-the-art publicly available codes at all levels of concurrencies. We extensively evaluate our implementation and identify bottlenecks that should be subject to further research.

cs.DC↗

arXiv2016

The Reverse Cuthill-McKee Algorithm in Distributed-Memory

Ordering vertices of a graph is key to minimize fill-in and data structure size in sparse direct solvers, maximize locality in iterative solvers, and improve performance in graph algorithms. Except for naturally parallelizable ordering methods such as nested dissection, many important ordering methods have not been efficiently mapped to distributed-memory architectures. In this paper, we present the first-ever distributed-memory implementation of the reverse Cuthill-McKee (RCM) algorithm for reducing the profile of a sparse matrix. Our parallelization uses a two-dimensional sparse matrix decomposition. We achieve high performance by decomposing the problem into a small number of primitives and utilizing optimized implementations of these primitives. Our implementation shows strong scaling up to 1024 cores for smaller matrices and up to 4096 cores for larger matrices.

cs.DC↗

arXiv2016

A work-efficient parallel sparse matrix-sparse vector multiplication algorithm

We design and develop a work-efficient multithreaded algorithm for sparse matrix-sparse vector multiplication (SpMSpV) where the matrix, the input vector, and the output vector are all sparse. SpMSpV is an important primitive in the emerging GraphBLAS standard and is the workhorse of many graph algorithms including breadth-first search, bipartite graph matching, and maximal independent set. As thread counts increase, existing multithreaded SpMSpV algorithms can spend more time accessing the sparse matrix data structure than doing arithmetic. Our shared-memory parallel SpMSpV algorithm is work efficient in the sense its total work is proportional to the number of arithmetic operations required. The key insight is to avoid each thread individually scan the list of matrix columns. Our algorithm is simple to implement and operates on existing column-based sparse matrix formats. It performs well on diverse matrices and vectors with heterogeneous sparsity patterns. A high-performance implementation of the algorithm attains up to 15x speedup on a 24-core Intel Ivy Bridge processor and up to 49x speedup on a 64-core Intel KNL manycore processor. In contrast to implementations of existing algorithms, the performance of our algorithm is sustained on a variety of different input types include matrices representing scale-free and high-diameter graphs.

cs.DC↗

arXiv2015

An Algorithmic Pipeline for Analyzing Multi-parametric Flow Cytometry Data

Flow cytometry (FC) is a single-cell profiling platform for measuring the phenotypes of individual cells from millions of cells in biological samples. FC employs high-throughput technologies and generates high-dimensional data, and hence algorithms for analyzing the data represent a bottleneck. This dissertation addresses several computational challenges arising in modern cytometry while mining information from high-dimensional and high-content biological data. A collection of combinatorial and statistical algorithms for locating, matching, prototyping, and classifying cellular populations from multi-parametric FC data is developed. The algorithmic pipeline, flowMatch, developed in this dissertation consists of five well-defined algorithmic modules to (1) transform data to stabilize within-population variance, (2) identify cell populations by robust clustering algorithms, (3) register cell populations across samples, (4) encapsulate a class of samples with templates, and (5) classify samples based on their similarity with the templates. Components of flowMatch can work independently or collaborate with each other to perform the complete data analysis. flowMatch is made available as an open-source R package in Bioconductor. We have employed flowMatch for classifying leukemia samples, evaluating the phosphorylation effects on T cells, classifying healthy immune profiles, and classifying the vaccination status of HIV patients. In these analyses, the pipeline is able to reach biologically meaningful conclusions quickly and efficiently with the automated algorithms. The algorithms included in flowMatch can also be applied to problems outside of flow cytometry such as in microarray data analysis and image recognition. Therefore, this dissertation contributes to the solution of fundamental problems in computational cytometry and related domains.

q-bio.QM↗

arXiv2014

Immunophenotypes of Acute Myeloid Leukemia From Flow Cytometry Data Using Templates

Motivation: We investigate whether a template-based classification pipeline could be used to identify immunophenotypes in (and thereby classify) a heterogeneous disease with many subtypes. The disease we consider here is Acute Myeloid Leukemia, which is heterogeneous at the morphologic, cytogenetic and molecular levels, with several known subtypes. The prognosis and treatment for AML depends on the subtype. Results: We apply flowMatch, an algorithmic pipeline for flow cytometry data created in earlier work, to compute templates succinctly summarizing classes of AML and healthy samples. We develop a scoring function that accounts for features of the AML data such as heterogeneity to identify immunophenotypes corresponding to various AML subtypes, including APL. All of the AML samples in the test set are classified correctly with high confidence. Availability: flowMatch is available at www.bioconductor.org/packages/devel/bioc/html/flowMatch.html; programs specific to immunophenotyping AML are at www.cs.purdue.edu/homes/aazad/software.html.

q-bio.QM↗

arXiv2010

Resampling Residuals on Phylogenetic Trees: Extended Results

In this article the results of Waddell and Azad (2009) are extended. In particular, the geometric percentage mean standard deviation measure of the fit of distances to a phylogenetic tree is adjusted for the number of parameters fitted to the model. The formulae are also presented in their general form for any weight that is a function of the distance. The cell line gene expression data set of Ross et al. (2000) is reanalyzed. It is shown that ordinary least squares (OLS) is a much better fit to the data than a Neighbor Joining or BME tree. Residual resampling shows that cancer cell lines do indeed fit a tree fairly well and that the tree does have strong internal structure. Simulations show that least squares tree building methods, including OLS, are strong competitors with BME type methods for fitting model data, while real world examples often suggest the same conclusion.

q-bio.PE↗

arXiv2009

Resampling Residuals: Robust Estimators of Error and Fit for Evolutionary Trees and Phylogenomics

Phylogenomics, even more so than traditional phylogenetics, needs to represent the uncertainty in evolutionary trees due to systematic error. Here we illustrate the analysis of genome-scale alignments of yeast, using robust measures of the additivity of the fit of distances to tree when using flexi Weighted Least Squares. A variety of DNA and protein distances are used. We explore the nature of the residuals, standardize them, and then create replicate data sets by resampling these residuals. Under the model, the results are shown to be very similar to the conventional sequence bootstrap. With real data they show up uncertainty in the tree that is either due to underestimating the stochastic error (hence massively overestimating the effective sequence length) and/or systematic error. The methods are extended to the very fast BME criterion with similarly promising results.

q-bio.PE↗