arXiv subjects

Rajgopal Kannan

Publications and source records attributed to Rajgopal Kannan.

At least 91 records · Page 5Linked to original sources

arXiv2020

RECEIPT: REfine CoarsE-grained IndePendent Tasks for Parallel Tip decomposition of Bipartite Graphs

Tip decomposition is a crucial kernel for mining dense subgraphs in bipartite networks, with applications in spam detection, analysis of affiliation networks etc. It creates a hierarchy of vertex-induced subgraphs with varying densities determined by the participation of vertices in butterflies (2,2-bicliques). To build the hierarchy, existing algorithms iteratively follow a delete-update(peeling) process: deleting vertices with the minimum number of butterflies and correspondingly updating the butterfly count of their 2-hop neighbors. The need to explore 2-hop neighborhood renders tip-decomposition computationally very expensive. Furthermore, the inherent sequentiality in peeling only minimum butterfly vertices makes derived parallel algorithms prone to heavy synchronization. In this paper, we propose a novel parallel tip-decomposition algorithm -- REfine CoarsE-grained Independent Tasks (RECEIPT) that relaxes the peeling order restrictions by partitioning the vertices into multiple independent subsets that can be concurrently peeled. This enables RECEIPT to simultaneously achieve a high degree of parallelism and dramatic reduction in synchronizations. Further, RECEIPT employs a hybrid peeling strategy along with other optimizations that drastically reduce the amount of wedge exploration and execution time. We perform detailed experimental evaluation of RECEIPT on a shared-memory multicore server. It can process some of the largest publicly available bipartite datasets orders of magnitude faster than the state-of-the-art algorithms -- achieving up to 1100x and 64x reduction in the number of thread synchronizations and traversed wedges, respectively. Using 36 threads, RECEIPT can provide up to 17.1x self-relative speedup. Our implementation of RECEIPT is available at https://github.com/kartiklakhotia/RECEIPT.

cs.DC↗

arXiv2020

Accurate, Efficient and Scalable Training of Graph Neural Networks

Graph Neural Networks (GNNs) are powerful deep learning models to generate node embeddings on graphs. When applying deep GNNs on large graphs, it is still challenging to perform training in an efficient and scalable way. We propose a novel parallel training framework. Through sampling small subgraphs as minibatches, we reduce training workload by orders of magnitude compared with state-of-the-art minibatch methods. We then parallelize the key computation steps on tightly-coupled shared memory systems. For graph sampling, we exploit parallelism within and across sampler instances, and propose an efficient data structure supporting concurrent accesses from samplers. The parallel sampler theoretically achieves near-linear speedup with respect to number of processing units. For feature propagation within subgraphs, we improve cache utilization and reduce DRAM traffic by data partitioning. Our partitioning is a 2-approximation strategy for minimizing the communication cost compared to the optimal. We further develop a runtime scheduler to reorder the training operations and adjust the minibatch subgraphs to improve parallel performance. Finally, we generalize the above parallelization strategies to support multiple types of GNN models and graph samplers. The proposed training outperforms the state-of-the-art in scalability, efficiency and accuracy simultaneously. On a 40-core Xeon platform, we achieve 60x speedup (with AVX) in the sampling step and 20x speedup in the feature propagation step, compared to the serial implementation. Our algorithm enables fast training of deeper GNNs, as demonstrated by orders of magnitude speedup compared to the Tensorflow implementation. We open-source our code at https://github.com/GraphSAINT/GraphSAINT.

cs.LG↗

arXiv2020

Towards High Performance, Portability, and Productivity: Lightweight Augmented Neural Networks for Performance Prediction

Writing high-performance code requires significant expertise in the programming language, compiler optimizations, and hardware knowledge. This often leads to poor productivity and portability and is inconvenient for a non-programmer domain-specialist such as a Physicist. More desirable is a high-level language where the domain-specialist simply specifies the workload in terms of high-level operations (e.g., matrix-multiply(A, B)), and the compiler identifies the best implementation fully utilizing the heterogeneous platform. For creating a compiler that supports productivity, portability, and performance simultaneously, it is crucial to predict the performance of various available implementations (variants) of the dominant operations (kernels) contained in the workload on various hardware to decide (a) which variant should be chosen for each kernel in the workload, and (b) on which hardware resource the variant should run. To enable the performance prediction, we propose lightweight augmented neural networks for arbitrary combinations of kernel-variant-hardware. A key innovation is utilizing the mathematical complexity of the kernels as a feature to achieve higher accuracy. These models are compact to reduce training time and fast inference during compile-time and run-time. Using models with less than 75 parameters, and only 250 training data instances, we are able to obtain a low MAPE of 3%, significantly outperforming traditional feed-forward neural networks on 48 kernel-variant-hardware combinations. We further demonstrate that our variant-selection approach can be used in Halide implementations to obtain up to 1.7x speedup over Halide's auto-scheduler.

cs.PF↗

arXiv2020

Accurate, Efficient and Scalable Graph Embedding

The Graph Convolutional Network (GCN) model and its variants are powerful graph embedding tools for facilitating classification and clustering on graphs. However, a major challenge is to reduce the complexity of layered GCNs and make them parallelizable and scalable on very large graphs -- state-of the art techniques are unable to achieve scalability without losing accuracy and efficiency. In this paper, we propose novel parallelization techniques for graph sampling-based GCNs that achieve superior scalable performance on very large graphs without compromising accuracy. Specifically, our GCN guarantees work-efficient training and produces order of magnitude savings in computation and communication. To scale GCN training on tightly-coupled shared memory systems, we develop parallelization strategies for the key steps in training: For the graph sampling step, we exploit parallelism within and across multiple sampling instances, and devise an efficient data structure for concurrent accesses that provides theoretical guarantee of near-linear speedup with number of processing units. For the feature propagation step within the sampled graph, we improve cache utilization and reduce DRAM communication by data partitioning. We prove that our partitioning strategy is a 2-approximation for minimizing the communication time compared to the optimal strategy. We demonstrate that our parallel graph embedding outperforms state-of-the-art methods in scalability (with respect to number of processors, graph size and GCN model size), efficiency and accuracy on several large datasets. On a 40-core Xeon platform, our parallel training achieves $64\times$ speedup (with AVX) in the sampling step and $25\times$ speedup in the feature propagation step, compared to the serial implementation, resulting in a net speedup of $21\times$.

cs.LG↗

arXiv2020

GraphSAINT: Graph Sampling Based Inductive Learning Method

Graph Convolutional Networks (GCNs) are powerful models for learning representations of attributed graphs. To scale GCNs to large graphs, state-of-the-art methods use various layer sampling techniques to alleviate the "neighbor explosion" problem during minibatch training. We propose GraphSAINT, a graph sampling based inductive learning method that improves training efficiency and accuracy in a fundamentally different way. By changing perspective, GraphSAINT constructs minibatches by sampling the training graph, rather than the nodes or edges across GCN layers. Each iteration, a complete GCN is built from the properly sampled subgraph. Thus, we ensure fixed number of well-connected nodes in all layers. We further propose normalization technique to eliminate bias, and sampling algorithms for variance reduction. Importantly, we can decouple the sampling from the forward and backward propagation, and extend GraphSAINT with many architecture variants (e.g., graph attention, jumping connection). GraphSAINT demonstrates superior performance in both accuracy and training time on five large graphs, and achieves new state-of-the-art F1 scores for PPI (0.995) and Reddit (0.970).

cs.LG↗

arXiv2019

GPOP: A cache- and work-efficient framework for Graph Processing Over Partitions

Past decade has seen the development of many shared-memory graph processing frameworks, intended to reduce the effort of developing high performance parallel applications. However many of these frameworks, based on Vertex-centric or Edge-centric paradigms suffer from several issues, such as poor cache utilization, irregular memory accesses, heavy use of synchronization primitives and theoretical inefficiency, that deteriorate overall performance and scalability. Recently, we proposed a cache and memory efficient partition-centric paradigm for computing PageRank. In this paper, we generalize this approach to develop a novel Graph Processing Over Partitions (GPOP) framework that is cache-efficient, scalable and work-efficient. GPOP induces locality in memory accesses by increasing granularity of execution to vertex subsets called 'partitions', thereby dramatically improving the cache performance of a variety of graph algorithms. It achieves high scalability by enabling completely lock and atomic free computation. GPOP's built-in analytical performance model enables it to use a hybrid of source and partitioncentric communication modes in a way that ensures work-efficiency each iteration, while simultaneously boosting high bandwidth sequential memory accesses. We extensively evaluate the performance of GPOP for a variety of graph algorithms, using several large datasets. We observe that GPOP incurs up to 9x, 6.8x and 5.5x less L2 cache misses compared to Ligra, GraphMat and Galois, respectively. In terms of execution time, GPOP is upto 19x, 9.3x and 3.6x faster than Ligra, GraphMat and Galois respectively.

cs.DC↗

arXiv2019

Planting Trees for scalable and efficient Canonical Hub Labeling

Point-to-Point Shortest Distance (PPSD) query is a crucial primitive in graph database applications. Hub labeling algorithms compute a labeling that converts a PPSD query into a list intersection problem (over a pre-computed indexing) enabling swift query response. However, constructing hub labeling is computationally challenging. Even state-of-the-art parallel algorithms based on Pruned Landmark Labeling (PLL) [3], are plagued by large label size, violation of given network hierarchy, poor scalability and inability to process large graphs. In this paper, we develop novel parallel shared-memory and distributed-memory algorithms for constructing the Canonical Hub Labeling (CHL) that is minimal in size for a given network hierarchy. To the best of our knowledge, none of the existing parallel algorithms guarantee canonical labeling. Our key contribution, the PLaNT algorithm, scales well beyond the limits of current practice by completely avoiding inter-node communication. PLaNT also enables the design of a collaborative label partitioning scheme across multiple nodes for completely in-memory processing of massive graphs whose labels cannot fit on a single machine. Compared to the sequential PLL, we empirically demonstrate upto 47.4x speedup on a 72 thread shared-memory platform. On a 64-node cluster, PLaNT achieves an average 42x speedup over single node execution. Finally, we show how our approach demonstrates superior scalability - we can process 14x larger graphs (in terms of label size) and construct hub labeling orders of magnitude faster compared to state-of-the-art distributed paraPLL algorithm.

cs.DC↗

arXiv2018

Accelerating PageRank using Partition-Centric Processing

PageRank is a fundamental link analysis algorithm that also functions as a key representative of the performance of Sparse Matrix-Vector (SpMV) multiplication. The traditional PageRank implementation generates fine granularity random memory accesses resulting in large amount of wasteful DRAM traffic and poor bandwidth utilization. In this paper, we present a novel Partition-Centric Processing Methodology (PCPM) to compute PageRank, that drastically reduces the amount of DRAM communication while achieving high sustained memory bandwidth. PCPM uses a Partition-centric abstraction coupled with the Gather-Apply-Scatter (GAS) programming model. By carefully examining how a PCPM based implementation impacts communication characteristics of the algorithm, we propose several system optimizations that improve the execution time substantially. More specifically, we develop (1) a new data layout that significantly reduces communication and random DRAM accesses, and (2) branch avoidance mechanisms to get rid of unpredictable data-dependent branches. We perform detailed analytical and experimental evaluation of our approach using 6 large graphs and demonstrate an average 2.7x speedup in execution time and 1.7x reduction in communication volume, compared to the state-of-the-art. We also show that unlike other GAS based implementations, PCPM is able to further reduce main memory traffic by taking advantage of intelligent node labeling that enhances locality. Although we use PageRank as the target application in this paper, our approach can be applied to generic SpMV computation.

cs.DC↗

arXiv2018

Optimal Net-Load Balancing in Smart Grids with High PV Penetration

Mitigating Supply-Demand mismatch is critical for smooth power grid operation. Traditionally, load curtailment techniques such as Demand Response (DR) have been used for this purpose. However, these cannot be the only component of a net-load balancing framework for Smart Grids with high PV penetration. These grids can sometimes exhibit supply surplus causing over-voltages. Supply curtailment techniques such as Volt-Var Optimizations are complex and computationally expensive. This increases the complexity of net-load balancing systems used by the grid operator and limits their scalability. Recently new technologies have been developed that enable the rapid and selective connection of PV modules of an installation to the grid. Taking advantage of these advancements, we develop a unified optimal net-load balancing framework which performs both load and solar curtailment. We show that when the available curtailment values are discrete, this problem is NP-hard and develop bounded approximation algorithms for minimizing the curtailment cost. Our algorithms produce fast solutions, given the tight timing constraints required for grid operation. We also incorporate the notion of fairness to ensure that curtailment is evenly distributed among all the nodes. Finally, we develop an online algorithm which performs net-load balancing using only data available for the current interval. Using both theoretical analysis and practical evaluations, we show that our net-load balancing algorithms provide solutions which are close to optimal in a small amount of time.

cs.DS↗

arXiv2013

The Price of Anarchy is Unbounded for Congestion Games with Superpolynomial Latency Costs

We consider non-cooperative unsplittable congestion games where players share resources, and each player's strategy is pure and consists of a subset of the resources on which it applies a fixed weight. Such games represent unsplittable routing flow games and also job allocation games. The congestion of a resource is the sum of the weights of the players that use it and the player's cost function is the sum of the utilities of the resources on its strategy. The social cost is the total weighted sum of the player's costs. The quality of Nash equilibria is determined by the price of anarchy ($PoA$) which expresses how much worse is the social outcome in the worst equilibrium versus the optimal coordinated solution. In the literature the predominant work has only been on games with polynomial utility costs, where it has been proven that the price of anarchy is bounded by the degree of the polynomial. However, no results exist on general bounds for non-polynomial utility functions. Here, we consider general versions of these games in which the utility of each resource is an arbitrary non-decreasing function of the congestion. In particular, we consider a large family of superpolynomial utility functions which are asymptotically larger than any polynomial. We demonstrate that for every such function there exist games for which the price of anarchy is unbounded and increasing with the number of players (even if they have infinitesimal weights) while network resources remain fixed. We give tight lower and upper bounds which show this dependence on the number of players. Furthermore we provide an exact characterization of the $PoA$ of all congestion games whose utility costs are bounded above by a polynomial function. Heretofore such results existed only for games with polynomial cost functions.

cs.GT↗

arXiv2010

Polynomial Bottleneck Congestion Games with Optimal Price of Anarchy

We study {\em bottleneck congestion games} where the social cost is determined by the worst congestion of any resource. These games directly relate to network routing problems and also job-shop scheduling problems. In typical bottleneck congestion games, the utility costs of the players are determined by the worst congested resources that they use. However, the resulting Nash equilibria are inefficient, since the price of anarchy is proportional on the number of resources which can be high. Here we show that we can get smaller price of anarchy with the bottleneck social cost metric. We introduce the {\em polynomial bottleneck games} where the utility costs of the players are polynomial functions of the congestion of the resources that they use. In particular, the delay function for any resource $r$ is $C_{r}^\M$, where $C_r$ is the congestion measured as the number of players that use $r$, and $\M \geq 1$ is an integer constant that defines the degree of the polynomial. The utility cost of a player is the sum of the individual delays of the resources that it uses. The social cost of the game remains the same, namely, it is the worst bottleneck resource congestion: $\max_{r} C_r$. We show that polynomial bottleneck games are very efficient and give price of anarchy $O(|R|^{1/(\M+1)})$, where $R$ is the set of resources. This price of anarchy is tight, since we demonstrate a game with price of anarchy $Ω(|R|^{1/(\M+1)})$, for any $\M \geq 1$. We obtain our tight bounds by using two proof techniques: {\em transformation}, which we use to convert arbitrary games to simpler games, and {\em expansion}, which we use to bound the price of anarchy in a simpler game.

cs.GT↗

arXiv2010

Bottleneck Routing Games with Low Price of Anarchy

We study {\em bottleneck routing games} where the social cost is determined by the worst congestion on any edge in the network. In the literature, bottleneck games assume player utility costs determined by the worst congested edge in their paths. However, the Nash equilibria of such games are inefficient since the price of anarchy can be very high and proportional to the size of the network. In order to obtain smaller price of anarchy we introduce {\em exponential bottleneck games} where the utility costs of the players are exponential functions of their congestions. We find that exponential bottleneck games are very efficient and give a poly-log bound on the price of anarchy: $O(\log L \cdot \log |E|)$, where $L$ is the largest path length in the players' strategy sets and $E$ is the set of edges in the graph. By adjusting the exponential utility costs with a logarithm we obtain games whose player costs are almost identical to those in regular bottleneck games, and at the same time have the good price of anarchy of exponential games.

cs.GT↗

arXiv2009

The Impact of Exponential Utility Costs in Bottleneck Routing Games

We study bottleneck routing games where the social cost is determined by the worst congestion on any edge in the network. Bottleneck games have been studied in the literature by having the player's utility costs to be determined by the worst congested edge in their paths. However, the Nash equilibria of such games are inefficient since the price of anarchy can be very high with respect to the parameters of the game. In order to obtain smaller price of anarchy we explore {\em exponential bottleneck games} where the utility costs of the players are exponential functions on the congestion of the edges in their paths. We find that exponential bottleneck games are very efficient giving a poly-log bound on the price of anarchy: O(log L log |E|), where L is the largest path length in the players strategy sets and E is the set of edges in the graph.

cs.GT↗

arXiv2009

Optimal Strategies for Gaussian Jamming in Block-Fading Channels under Delay and Power Constraints

Without assuming any knowledge on source's codebook and its output signals, we formulate a Gaussian jamming problem in block fading channels as a two-player zero sum game. The outage probability is adopted as an objective function, over which transmitter aims at minimization and jammer aims at maximization by selecting their power control strategies. Optimal power control strategies for each player are obtained under both short-term and long-term power constraints. For the latter case, we first prove the non-existence of a Nash equilibrium, and then provide a complete solution for both maxmin and minimax problems. Numerical results demonstrate a sharp difference between the outage probabilities of the minimax and maxmin solutions.

cs.IT↗

arXiv2008

Jamming in Fixed-Rate Wireless Systems with Power Constraints - Part II: Parallel Slow Fading Channels

This is the second part of a two-part paper that studies the problem of jamming in a fixed-rate transmission system with fading. In the first part, we studied the scenario with a fast fading channel, and found Nash equilibria of mixed strategies for short term power constraints, and for average power constraints with and without channel state information (CSI) feedback. We also solved the equally important maximin and minimax problems with pure strategies. Whenever we dealt with average power constraints, we decomposed the problem into two levels of power control, which we solved individually. In this second part of the paper, we study the scenario with a parallel, slow fading channel, which usually models multi-carrier transmissions, such as OFDM. Although the framework is similar as the one in Part I \cite{myself3}, dealing with the slow fading requires more intricate techniques. Unlike in the fast fading scenario, where the frames supporting the transmission of the codewords were equivalent and completely characterized by the channel statistics, in our present scenario the frames are unique, and characterized by a specific set of channel realizations. This leads to more involved inter-frame power allocation strategies, and in some cases even to the need for a third level of power control. We also show that for parallel slow fading channels, the CSI feedback helps in the battle against jamming, as evidenced by the significant degradation to system performance when CSI is not sent back. We expect this degradation to decrease as the number of parallel channels $M$ increases, until it becomes marginal for $M\to \infty$ (which can be considered as the case in Part I).

cs.IT↗

arXiv2008

Bicretieria Optimization in Routing Games

Two important metrics for measuring the quality of routing paths are the maximum edge congestion $C$ and maximum path length $D$. Here, we study bicriteria in routing games where each player $i$ selfishly selects a path that simultaneously minimizes its maximum edge congestion $C_i$ and path length $D_i$. We study the stability and price of anarchy of two bicriteria games: - {\em Max games}, where the social cost is $\max(C,D)$ and the player cost is $\max(C_i, D_i)$. We prove that max games are stable and convergent under best-response dynamics, and that the price of anarchy is bounded above by the maximum path length in the players' strategy sets. We also show that this bound is tight in worst-case scenarios. - {\em Sum games}, where the social cost is $C+D$ and the player cost is $C_i+D_i$. For sum games, we first show the negative result that there are game instances that have no Nash-equilibria. Therefore, we examine an approximate game called the {\em sum-bucket game} that is always convergent (and therefore stable). We show that the price of anarchy in sum-bucket games is bounded above by $C^* \cdot D^* / (C^* + D^*)$ (with a poly-log factor), where $C^*$ and $D^*$ are the optimal coordinated congestion and path length. Thus, the sum-bucket game has typically superior price of anarchy bounds than the max game. In fact, when either $C^*$ or $D^*$ is small (e.g. constant) the social cost of the Nash-equilibria is very close to the coordinated optimal $C^* + D^*$ (within a poly-log factor). We also show that the price of anarchy bound is tight for cases where both $C^*$ and $D^*$ are large.

cs.GT↗

arXiv2007

Energy Efficient Estimation of Gaussian Sources Over Inhomogeneous Gaussian MAC Channels

It has been shown lately the optimality of uncoded transmission in estimating Gaussian sources over homogeneous/symmetric Gaussian multiple access channels (MAC) using multiple sensors. It remains, however, unclear whether it still holds for any arbitrary networks and/or with high channel signal-to-noise ratio (SNR) and high signal-to-measurement-noise ratio (SMNR). In this paper, we first provide a joint source and channel coding approach in estimating Gaussian sources over Gaussian MAC channels, as well as its sufficient and necessary condition in restoring Gaussian sources with a prescribed distortion value. An interesting relationship between our proposed joint approach with a more straightforward separate source and channel coding scheme is then established. We then formulate constrained power minimization problems and transform them to relaxed convex geometric programming problems, whose numerical results exhibit that either separate or uncoded scheme becomes dominant over a linear topology network. In addition, we prove that the optimal decoding order to minimize the total transmission powers for both source and channel coding parts is solely subject to the ranking of MAC channel qualities, and has nothing to do with the ranking of measurement qualities. Finally, asymptotic results for homogeneous networks are obtained which not only confirm the existing optimality of the uncoded approach, but also show that the asymptotic SNR exponents of these three approaches are all the same. Moreover, the proposed joint approach share the same asymptotic ratio with respect to high SNR and high SMNR as the uncoded scheme.

cs.IT↗

arXiv2005

Analysis of Relationship between Strategic and Aggregate Energy Minimization in Delay-Constrained Wireless Networks

We formulate two versions of the power control problem for wireless networks with latency constraints arising from duty cycle allocations In the first version, strategic power optimization, wireless nodes are modeled as rational agents in a power game, who strategically adjust their powers to minimize their own energy. In the other version, joint power optimization, wireless nodes jointly minimize the aggregate energy expenditure. Our analysis of these models yields insights into the different energy outcomes of strategic versus joint power optimization. We derive analytical solutions for power allocation under both models and study how they are affected by data loads and channel quality. We derive simple necessary conditions for the existence of Nash equilibria in the power game and also provide numerical examples of optimal power allocation under both models. Finally, we show that joint optimization can (sometimes) be Pareto-optimal and dominate strategic optimization, i.e the energy expenditure of all nodes is lower than if they were using strategic optimization.

cs.NI↗