arXiv subjects

Vijaykrishnan Narayanan

Publications and source records attributed to Vijaykrishnan Narayanan.

43 records · Page 3Linked to original sources

arXiv2022

Seeker: Synergizing Mobile and Energy Harvesting Wearable Sensors for Human Activity Recognition

There is an increasing demand for intelligent processing on emerging ultra-low-power internet of things (IoT) devices, and recent works have shown substantial efficiency boosts by executing inference tasks directly on the IoT device (node) rather than merely transmitting sensor data. However, the computation and power demands of Deep Neural Network (DNN)-based inference pose significant challenges for nodes in an energy-harvesting wireless sensor network (EH-WSN). Moreover, these tasks often require responses from multiple physically distributed EH sensor nodes, which imposes crucial system optimization challenges in addition to per-node constraints. To address these challenges, we propose \emph{Seeker}, a novel approach to efficiently execute DNN inferences for Human Activity Recognition (HAR) tasks, using both an EH-WSN and a host mobile device. Seeker minimizes communication overheads and maximizes computation at each sensor without violating the quality of service. \emph{Seeker} uses a \emph{store-and-execute} approach to complete a subset of inferences on the EH sensor node, reducing communication with the mobile host. Further, for those inferences unfinished because of harvested energy constraints, it leverages an \emph{activity aware coreset} (AAC) construction to efficiently communicate compact features to the host device where ensemble techniques are used to efficiently finish the inferences. \emph{Seeker} performs HAR with $86.8\%$ accuracy, surpassing the $81.2\%$ accuracy of a state of the art approach. Moreover, by using AAC, it lowers the communication data volume by $8.9\times$.

eess.SP↗

arXiv2022

Communication-efficient k-Means for Edge-based Machine Learning

We consider the problem of computing the k-means centers for a large high-dimensional dataset in the context of edge-based machine learning, where data sources offload machine learning computation to nearby edge servers. k-Means computation is fundamental to many data analytics, and the capability of computing provably accurate k-means centers by leveraging the computation power of the edge servers, at a low communication and computation cost to the data sources, will greatly improve the performance of these analytics. We propose to let the data sources send small summaries, generated by joint dimensionality reduction (DR), cardinality reduction (CR), and quantization (QT), to support approximate k-means computation at reduced complexity and communication cost. By analyzing the complexity, the communication cost, and the approximation error of k-means algorithms based on carefully designed composition of DR/CR/QT methods, we show that: (i) it is possible to compute near-optimal k-means centers at a near-linear complexity and a constant or logarithmic communication cost, (ii) the order of applying DR and CR significantly affects the complexity and the communication cost, and (iii) combining DR/CR methods with a properly configured quantizer can further reduce the communication cost without compromising the other performance metrics. Our theoretical analysis has been validated through experiments based on real datasets.

cs.LG↗

arXiv2021

An Oscillator-based MaxSAT solver

The quest to solve hard combinatorial optimization problems efficiently -- still a longstanding challenge for traditional digital computers -- has inspired the exploration of many alternate computing models and platforms. As a case in point, oscillator networks offer a potentially promising energy efficient and scalable option. However, prior oscillator-based combinatorial optimization solvers have primarily focused on quadratic combinatorial optimization problems that consider only pairwise interaction among the oscillators. In this work, we propose a new computational model based on the maximum entropy production (MEP) principle that exploits higher order interactions among the oscillators, and demonstrate its application in solving the non-quadratic maximum satisfiability (MaxSAT) problem. We demonstrate that the solution to the MaxSAT problem can be directly mapped to the entropy production rate in the oscillator network, and subsequently, propose an area-efficient hardware implementation that leverages Compute-in-Memory (CiM) primitives. Using experiments along with analytical and circuit simulations, we elucidate the performance of the proposed approach in computing high-quality optimal / near-optimal solutions to the MaxSAT problem. Our work not only reveals how oscillators can solve non-quadratic combinatorial optimization problems such as MaxSAT but also extends the application of this dynamical system-based approach to a broader class of problems that can be easily decomposed to the MaxSAT solution.

cs.ET↗

arXiv2021

Exploiting Activation based Gradient Output Sparsity to Accelerate Backpropagation in CNNs

Machine/deep-learning (ML/DL) based techniques are emerging as a driving force behind many cutting-edge technologies, achieving high accuracy on computer vision workloads such as image classification and object detection. However, training these models involving large parameters is both time-consuming and energy-hogging. In this regard, several prior works have advocated for sparsity to speed up the of DL training and more so, the inference phase. This work begins with the observation that during training, sparsity in the forward and backward passes are correlated. In that context, we investigate two types of sparsity (input and output type) inherent in gradient descent-based optimization algorithms and propose a hardware micro-architecture to leverage the same. Our experimental results use five state-of-the-art CNN models on the Imagenet dataset, and show back propagation speedups in the range of 1.69$\times$ to 5.43$\times$, compared to the dense baseline execution. By exploiting sparsity in both the forward and backward passes, speedup improvements range from 1.68$\times$ to 3.30$\times$ over the sparsity-agnostic baseline execution. Our work also achieves significant reduction in training iteration time over several previously proposed dense as well as sparse accelerator based platforms, in addition to achieving order of magnitude energy efficiency improvements over GPU based execution.

cs.LG↗

arXiv2021

STAR: Sparse Transformer-based Action Recognition

The cognitive system for human action and behavior has evolved into a deep learning regime, and especially the advent of Graph Convolution Networks has transformed the field in recent years. However, previous works have mainly focused on over-parameterized and complex models based on dense graph convolution networks, resulting in low efficiency in training and inference. Meanwhile, the Transformer architecture-based model has not yet been well explored for cognitive application in human action and behavior estimation. This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data. Our model can also process the variable length of video clips grouped as a single batch. Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference. Experiments show that our model achieves 4~18x speedup and 1/7~1/15 model size compared with the baseline models at competitive accuracy.

cs.CV↗

arXiv2021

Transformer-based Machine Learning for Fast SAT Solvers and Logic Synthesis

CNF-based SAT and MaxSAT solvers are central to logic synthesis and verification systems. The increasing popularity of these constraint problems in electronic design automation encourages studies on different SAT problems and their properties for further computational efficiency. There has been both theoretical and practical success of modern Conflict-driven clause learning SAT solvers, which allows solving very large industrial instances in a relatively short amount of time. Recently, machine learning approaches provide a new dimension to solving this challenging problem. Neural symbolic models could serve as generic solvers that can be specialized for specific domains based on data without any changes to the structure of the model. In this work, we propose a one-shot model derived from the Transformer architecture to solve the MaxSAT problem, which is the optimization version of SAT where the goal is to satisfy the maximum number of clauses. Our model has a scale-free structure which could process varying size of instances. We use meta-path and self-attention mechanism to capture interactions among homogeneous nodes. We adopt cross-attention mechanisms on the bipartite graph to capture interactions among heterogeneous nodes. We further apply an iterative algorithm to our model to satisfy additional clauses, enabling a solution approaching that of an exact-SAT problem. The attention mechanisms leverage the parallelism for speedup. Our evaluation indicates improved speedup compared to heuristic approaches and improved completion rate compared to machine learning approaches.

cs.NE↗

arXiv2020

Robust Coreset Construction for Distributed Machine Learning

Coreset, which is a summary of the original dataset in the form of a small weighted set in the same sample space, provides a promising approach to enable machine learning over distributed data. Although viewed as a proxy of the original dataset, each coreset is only designed to approximate the cost function of a specific machine learning problem, and thus different coresets are often required to solve different machine learning problems, increasing the communication overhead. We resolve this dilemma by developing robust coreset construction algorithms that can support a variety of machine learning problems. Motivated by empirical evidence that suitably-weighted k-clustering centers provide a robust coreset, we harden the observation by establishing theoretical conditions under which the coreset provides a guaranteed approximation for a broad range of machine learning problems, and developing both centralized and distributed algorithms to generate coresets satisfying the conditions. The robustness of the proposed algorithms is verified through extensive experiments on diverse datasets with respect to both supervised and unsupervised learning problems.

cs.LG↗