arXiv subjects

Radu Marculescu

Publications and source records attributed to Radu Marculescu.

64 records · Page 4Linked to original sources

arXiv2019

Work-in-Progress: A Simulation Framework for Domain-Specific System-on-Chips

Heterogeneous system-on-chips (SoCs) have become the standard embedded computing platforms due to their potential to deliver superior performance and energy efficiency compared to homogeneous architectures. They can be particularly suited to target a specific domain of applications. However, this potential is contingent upon optimizing the SoC for the target domain and utilizing its resources effectively at run-time. Cycle-accurate instruction set simulators are not suitable for this optimization, since meaningful temperature and power consumption evaluations require simulating seconds, if not minutes, of workloads. This paper presents a system-level domain-specific SoC simulation (DS3) framework to address this need. DS3 enables both design space exploration and dynamic resource management for power-performance optimization for domain applications with$~600\times$ speedup compared to commonly used gem5 simulator. We showcase DS3 using five applications from wireless communications and radar processing domain. DS3, as well as the reference applications, will be shared as open-source software to stimulate research in this area.

cs.AR↗

arXiv2019

Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT

Model compression has emerged as an important area of research for deploying deep learning models on Internet-of-Things (IoT). However, for extremely memory-constrained scenarios, even the compressed models cannot fit within the memory of a single device and, as a result, must be distributed across multiple devices. This leads to a distributed inference paradigm in which memory and communication costs represent a major bottleneck. Yet, existing model compression techniques are not communication-aware. Therefore, we propose Network of Neural Networks (NoNN), a new distributed IoT learning paradigm that compresses a large pretrained 'teacher' deep network into several disjoint and highly-compressed 'student' modules, without loss of accuracy. Moreover, we propose a network science-based knowledge partitioning algorithm for the teacher model, and then train individual students on the resulting disjoint partitions. Extensive experimentation on five image classification datasets, for user-defined memory/performance budgets, show that NoNN achieves higher accuracy than several baselines and similar accuracy as the teacher model, while using minimal communication among students. Finally, as a case study, we deploy the proposed model for CIFAR-10 dataset on edge devices and demonstrate significant improvements in memory footprint (up to 24x), performance (up to 12x), and energy per node (up to 14x) compared to the large teacher model. We further show that for distributed inference on multiple edge devices, our proposed NoNN model results in up to 33x reduction in total latency w.r.t. a state-of-the-art model compression baseline.

stat.ML↗

arXiv2019

Dream Distillation: A Data-Independent Model Compression Framework

Model compression is eminently suited for deploying deep learning on IoT-devices. However, existing model compression techniques rely on access to the original or some alternate dataset. In this paper, we address the model compression problem when no real data is available, e.g., when data is private. To this end, we propose Dream Distillation, a data-independent model compression framework. Our experiments show that Dream Distillation can achieve 88.5% accuracy on the CIFAR-10 test set without actually training on the original data!

stat.ML↗

arXiv2018

Climate Anomalies vs Air Pollution: Carbon Emissions and Anomaly Networks

This project aims to shed light on how man-made carbon emissions are affecting global wind patterns by looking for temporal and geographical correlations between carbon emissions, surface temperatures anomalies, and wind speed anomalies at high altitude. We use a networks-based approach and daily data from 1950 to 2010 [1-3] to model and draw correlations between disparate regions of the globe.

cs.OH↗

arXiv2018

A Dynamic Network and Representation LearningApproach for Quantifying Economic Growth fromSatellite Imagery

Quantifying the improvement in human living standard, as well as the city growth in developing countries, is a challenging problem due to the lack of reliable economic data. Therefore, there is a fundamental need for alternate, largely unsupervised, computational methods that can estimate the economic conditions in the developing regions. To this end, we propose a new network science- and representation learning-based approach that can quantify economic indicators and visualize the growth of various regions. More precisely, we first create a dynamic network drawn out of high-resolution nightlight satellite images. We then demonstrate that using representation learning to mine the resulting network, our proposed approach can accurately predict spatial gross economic expenditures over large regions. Our method, which requires only nightlight images and limited survey data, can capture city-growth, as well as how people's living standard is changing; this can ultimately facilitate the decision makers' understanding of growth without heavily relying on expensive and time-consuming surveys.

cs.SI↗

arXiv2018

PGLasso: Microbial Community Detection through Phylogenetic Graphical Lasso

Due to the recent advances in high-throughput sequencing technologies, it becomes possible to directly analyze microbial communities in the human body and in the environment. Knowledge of how microbes interact with each other and form functional communities can provide a solid foundation to understand microbiome related diseases; this can serve as a key step towards precision medicine. In order to understand how microbes form communities, we propose a two step approach: First, we infer the microbial co-occurrence network by integrating a graph inference algorithm with phylogenetic information obtained directly from metagenomic data. Next, we utilize a network-based community detection algorithm to cluster microbes into functional groups where microbes in each group are highly correlated. We also curate a "gold standard" network based on the microbe-metabolic relationships which are extracted directly from the metagenomic data. Utilizing community detection on the resulting microbial metabolic pathway bipartite graph, the community membership for each microbe can be viewed as the true label when evaluating against other existing methods. Overall, our proposed framework Phylogenetic Graphical Lasso (PGLasso) outperforms existing methods with gains larger than 100% in terms of Adjusted Rand Index (ARI) which is commonly used to quantify the goodness of clusterings.

q-bio.QM↗

arXiv2017

On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8x reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25% savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

cs.DC↗

arXiv2017

Machine Learning and Manycore Systems Design: A Serendipitous Symbiosis

Tight collaboration between experts of machine learning and manycore system design is necessary to create a data-driven manycore design framework that integrates both learning and expert knowledge. Such a framework will be necessary to address the rising complexity of designing large-scale manycore systems and machine learning techniques.

cs.LG↗

arXiv2007

Energy- and Performance-Driven NoC Communication Architecture Synthesis Using a Decomposition Approach

In this paper, we present a methodology for customized communication architecture synthesis that matches the communication requirements of the target application. This is an important problem, particularly for network-based implementations of complex applications. Our approach is based on using frequently encountered generic communication primitives as an alphabet capable of characterizing any given communication pattern. The proposed algorithm searches through the entire design space for a solution that minimizes the system total energy consumption, while satisfying the other design constraints. Compared to the standard mesh architecture, the customized architecture generated by the newly proposed approach shows about 36% throughput increase and 51% reduction in the energy required to encrypt 128 bits of data with a standard encryption algorithm.

cs.AR↗

arXiv2007

Energy-Aware Routing for E-Textile Applications

As the scale of electronic devices shrinks, "electronic textiles" (e-textiles) will make possible a wide variety of novel applications which are currently unfeasible. Due to the wearability concerns, low-power techniques are critical for e-textile applications. In this paper, we address the issue of the energy-aware routing for e-textile platforms and propose an efficient algorithm to solve it. The platform we consider consists of dedicated components for e-textiles, including computational modules, dedicated transmission lines and thin-film batteries on fiber substrates. Furthermore, we derive an analytical upper bound for the achievable number of jobs completed over all possible routing strategies. From a practical standpoint, for the Advanced Encryption Standard (AES) cipher, the routing technique we propose achieves about fifty percent of this analytical upper bound. Moreover, compared to the non-energy-aware counterpart, our routing technique increases the number of encryption jobs completed by one order of magnitude.

cs.AR↗