arXiv subjects

Chandra Thapa

Publications and source records attributed to Chandra Thapa.

46 records · Page 3Linked to original sources

arXiv2021

Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things

Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning training without accessing raw data on clients or end devices. However, their \emph{comparative training performance} under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioners. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding (i) learning performance with heterogeneous data distributions and (ii) on-device execution overhead. Our analyses in this work demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form splitfed learning (SFL) to leverage each of their benefits (e.g., parallel training of FL and lightweight on-device computation requirement of SL). This work then considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance, including training time, communication overhead, power consumption, and memory usage. Besides evaluations, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of a hybrid type of model training at the server-side. The generalized SFL merges sequential (dependent) and parallel (independent) processes of model training and is thus beneficial for a system with large-scaled IoT devices, specifically at the server-side operations. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead by up to four times for the SL and (generalized) SFL.

cs.LG↗

arXiv2020

Advancements of federated learning towards privacy preservation: from federated learning to split learning

In the distributed collaborative machine learning (DCML) paradigm, federated learning (FL) recently attracted much attention due to its applications in health, finance, and the latest innovations such as industry 4.0 and smart vehicles. FL provides privacy-by-design. It trains a machine learning model collaboratively over several distributed clients (ranging from two to millions) such as mobile phones, without sharing their raw data with any other participant. In practical scenarios, all clients do not have sufficient computing resources (e.g., Internet of Things), the machine learning model has millions of parameters, and its privacy between the server and the clients while training/testing is a prime concern (e.g., rival parties). In this regard, FL is not sufficient, so split learning (SL) is introduced. SL is reliable in these scenarios as it splits a model into multiple portions, distributes them among clients and server, and trains/tests their respective model portions to accomplish the full model training/testing. In SL, the participants do not share both data and their model portions to any other parties, and usually, a smaller network portion is assigned to the clients where data resides. Recently, a hybrid of FL and SL, called splitfed learning, is introduced to elevate the benefits of both FL (faster training/testing time) and SL (model split and training). Following the developments from FL to SL, and considering the importance of SL, this chapter is designed to provide extensive coverage in SL and its variants. The coverage includes fundamentals, existing findings, integration with privacy measures such as differential privacy, open problems, and code implementation.

cs.LG↗

arXiv2020

Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy

Precision health leverages information from various sources, including omics, lifestyle, environment, social media, medical records, and medical insurance claims to enable personalized care, prevent and predict illness, and precise treatments. It extensively uses sensing technologies (e.g., electronic health monitoring devices), computations (e.g., machine learning), and communication (e.g., interaction between the health data centers). As health data contain sensitive private information, including the identity of patient and carer and medical conditions of the patient, proper care is required at all times. Leakage of these private information affects the personal life, including bullying, high insurance premium, and loss of job due to the medical history. Thus, the security, privacy of and trust on the information are of utmost importance. Moreover, government legislation and ethics committees demand the security and privacy of healthcare data. Herein, in the light of precision health data security, privacy, ethical and regulatory requirements, finding the best methods and techniques for the utilization of the health data, and thus precision health is essential. In this regard, firstly, this paper explores the regulations, ethical guidelines around the world, and domain-specific needs. Then it presents the requirements and investigates the associated challenges. Secondly, this paper investigates secure and privacy-preserving machine learning methods suitable for the computation of precision health data along with their usage in relevant health projects. Finally, it illustrates the best available techniques for precision health data security and privacy with a conceptual system model that enables compliance, ethics clearance, consent management, medical innovations, and developments in the health domain.

cs.CR↗

arXiv2020

End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

This work is the first attempt to evaluate and compare felderated learning (FL) and split neural networks (SplitNN) in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For learning performance, which is specified by the model accuracy and convergence speed metrics, we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution, but worse than FL under an extreme non-IID data distribution. For implementation overhead, we end-to-end mount both FL and SplitNN on Raspberry Pis, and comprehensively evaluate overheads including training time, communication overhead under the real LAN setting, power consumption and memory usage. Our key observations are that under IoT scenario where the communication traffic is the main concern, the FL appears to perform better over SplitNN because FL has the significantly lower communication overhead compared with SplitNN, which empirically corroborate previous statistical analysis. In addition, we reveal several unrecognized limitations about SplitNN, forming the basis for future research.

cs.CR↗

arXiv2020

Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

A new collaborative learning, called split learning, was recently introduced, aiming to protect user data privacy without revealing raw input data to a server. It collaboratively runs a deep neural network model where the model is split into two parts, one for the client and the other for the server. Therefore, the server has no direct access to raw data processed at the client. Until now, the split learning is believed to be a promising approach to protect the client's raw data; for example, the client's data was protected in healthcare image applications using 2D convolutional neural network (CNN) models. However, it is still unclear whether the split learning can be applied to other deep learning models, in particular, 1D CNN. In this paper, we examine whether split learning can be used to perform privacy-preserving training for 1D CNN models. To answer this, we first design and implement an 1D CNN model under split learning and validate its efficacy in detecting heart abnormalities using medical ECG data. We observed that the 1D CNN model under split learning can achieve the same accuracy of 98.9\% like the original (non-split) model. However, our evaluation demonstrates that split learning may fail to protect the raw data privacy on 1D CNN models. To address the observed privacy leakage in split learning, we adopt two privacy leakage mitigation techniques: 1) adding more hidden layers to the client side and 2) applying differential privacy. Although those mitigation techniques are helpful in reducing privacy leakage, they have a significant impact on model accuracy. Hence, based on those results, we conclude that split learning alone would not be sufficient to maintain the confidentiality of raw sequential data in 1D CNN models.

cs.CR↗

arXiv2019

Structural Characteristics of Two-Sender Index Coding

This paper studies index coding with two senders. In this setup, source messages are distributed among the senders possibly with common messages. In addition, there are multiple receivers, with each receiver having some messages a priori, known as side-information, and requesting one unique message such that each message is requested by only one receiver. Index coding in this setup is called two-sender unicast index coding (TSUIC). The main goal is to find the shortest aggregate normalized codelength, which is expressed as the optimal broadcast rate. In this work, firstly, for a given TSUIC problem, we form three independent sub-problems each consisting of the only subset of the messages, based on whether the messages are available only in one of the senders or in both senders. Then we express the optimal broadcast rate of the TSUIC problem as a function of the optimal broadcast rates of those independent sub-problems. In this way, we discover the structural characteristics of TSUIC. For the proofs of our results, we utilize confusion graphs and coding techniques used in single-sender index coding. To adapt the confusion graph technique in TSUIC, we introduce a new graph-coloring approach that is different from the normal graph coloring, which we call two-sender graph coloring, and propose a way of grouping the vertices to analyze the number of colors used. We further determine a class of TSUIC instances where a certain type of side-information can be removed without affecting their optimal broadcast rates. Finally, we generalize the results of a class of TSUIC problems to multiple senders.

cs.IT↗

arXiv2018

Interlinked Cycles for Index Coding: Generalizing Cycles and Cliques

We consider a graphical approach to index coding. While cycles have been shown to provide coding gain, only disjoint cycles and cliques (a specific type of overlapping cycles) have been exploited in existing literature. In this paper, we define a more general form of overlapping cycles, called the interlinked-cycle (IC) structure, that generalizes cycles and cliques. We propose a scheme, called the interlinked-cycle-cover (ICC) scheme, that leverages IC structures in digraphs to construct scalar linear index codes. We characterize a class of infinitely many digraphs where our proposed scheme is optimal over all linear and non-linear index codes. Consequently, for this class of digraphs, we indirectly prove that scalar linear index codes are optimal. Furthermore, we show that the ICC scheme can outperform all existing graph-based schemes (including partial-clique-cover and fractional-local-chromatic number schemes), and a random-coding scheme (namely, composite coding) for certain graphs.

cs.IT↗

arXiv2016

Graph-Theoretic Approaches to Two-Sender Index Coding

Consider a communication scenario over a noiseless channel where a sender is required to broadcast messages to multiple receivers, each having side information about some messages. In this scenario, the sender can leverage the receivers' side information during the encoding of messages in order to reduce the required transmissions. This type of encoding is called index coding. In this paper, we study index coding with two cooperative senders, each with some subset of messages, and multiple receivers, each requesting one unique message. The index coding in this setup is called two-sender unicast index coding (TSUIC). The main aim of TSUIC is to minimize the total number of transmissions required by the two senders. Based on graph-theoretic approaches, we prove that TSUIC is equivalent to single-sender unicast index coding (SSUIC) for some special cases. Moreover, we extend the existing schemes for SSUIC, viz., the cycle-cover scheme, the clique-cover scheme, and the local-chromatic scheme to the corresponding schemes for TSUIC.

cs.IT↗

arXiv2015

Generalized Interlinked Cycle Cover for Index Coding

A source coding problem over a noiseless broadcast channel where the source is pre-informed about the contents of the cache of all receivers, is an index coding problem. Furthermore, if each message is requested by one receiver, then we call this an index coding problem with a unicast message setting. This problem can be represented by a directed graph. In this paper, we first define a structure (we call generalized interlinked cycles (GIC)) in directed graphs. A GIC consists of cycles which are interlinked in some manner (i.e., not disjoint), and it turns out that the GIC is a generalization of cliques and cycles. We then propose a simple scalar linear encoding scheme with linear time encoding complexity. This scheme exploits GICs in the digraph. We prove that our scheme is optimal for a class of digraphs with message packets of any length. Moreover, we show that our scheme can outperform existing techniques, e.g., partial clique cover, local chromatic number, composite-coding, and interlinked cycle cover.

cs.IT↗

arXiv2015

A New Index Coding Scheme Exploiting Interlinked Cycles

We study the index coding problem in the unicast message setting, i.e., where each message is requested by one unique receiver. This problem can be modeled by a directed graph. We propose a new scheme called interlinked cycle cover, which exploits interlinked cycles in the directed graph, for designing index codes. This new scheme generalizes the existing clique cover and cycle cover schemes. We prove that for a class of infinitely many digraphs with messages of any length, interlinked cycle cover provides an optimal index code. Furthermore, the index code is linear with linear time encoding complexity.

cs.IT↗