arXiv subjects

Leandros Tassiulas

Publications and source records attributed to Leandros Tassiulas.

At least 91 records · Page 5Linked to original sources

arXiv2022

Model Pruning Enables Efficient Federated Learning on Edge Devices

Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a datacenter. To overcome this challenge, we propose PruneFL -- a novel FL approach with adaptive and distributed parameter pruning, which adapts the model size during FL to reduce both communication and computation overhead and minimize the overall training time, while maintaining a similar accuracy as the original model. PruneFL includes initial pruning at a selected client and further pruning as part of the FL process. The model size is adapted during this process, which includes maximizing the approximate empirical risk reduction divided by the time of one FL round. Our experiments with various datasets on edge devices (e.g., Raspberry Pi) show that: (i) we significantly reduce the training time compared to conventional FL and various other pruning-based methods; (ii) the pruned model with automatically determined size converges to an accuracy that is very similar to the original model, and it is also a lottery ticket of the original model.

cs.LG↗

arXiv2022

KerGNNs: Interpretable Graph Neural Networks with Graph Kernels

Graph kernels are historically the most widely-used technique for graph classification tasks. However, these methods suffer from limited performance because of the hand-crafted combinatorial features of graphs. In recent years, graph neural networks (GNNs) have become the state-of-the-art method in downstream graph-related tasks due to their superior performance. Most GNNs are based on Message Passing Neural Network (MPNN) frameworks. However, recent studies show that MPNNs can not exceed the power of the Weisfeiler-Lehman (WL) algorithm in graph isomorphism test. To address the limitations of existing graph kernel and GNN methods, in this paper, we propose a novel GNN framework, termed \textit{Kernel Graph Neural Networks} (KerGNNs), which integrates graph kernels into the message passing process of GNNs. Inspired by convolution filters in convolutional neural networks (CNNs), KerGNNs adopt trainable hidden graphs as graph filters which are combined with subgraphs to update node embeddings using graph kernels. In addition, we show that MPNNs can be viewed as special cases of KerGNNs. We apply KerGNNs to multiple graph-related tasks and use cross-validation to make fair comparisons with benchmarks. We show that our method achieves competitive performance compared with existing state-of-the-art methods, demonstrating the potential to increase the representation ability of GNNs. We also show that the trained graph filters in KerGNNs can reveal the local graph structures of the dataset, which significantly improves the model interpretability compared with conventional GNN models.

cs.LG↗

arXiv2021

Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling

Federated learning (FL) algorithms usually sample a fraction of clients in each round (partial participation) when the number of participants is large and the server's communication bandwidth is limited. Recent works on the convergence analysis of FL have focused on unbiased client sampling, e.g., sampling uniformly at random, which suffers from slow wall-clock time for convergence due to high degrees of system heterogeneity and statistical heterogeneity. This paper aims to design an adaptive client sampling algorithm that tackles both system and statistical heterogeneity to minimize the wall-clock convergence time. We obtain a new tractable convergence bound for FL algorithms with arbitrary client sampling probabilities. Based on the bound, we analytically establish the relationship between the total learning time and sampling probabilities, which results in a non-convex optimization problem for training time minimization. We design an efficient algorithm for learning the unknown parameters in the convergence bound and develop a low-complexity algorithm to approximately solve the non-convex problem. Experimental results from both hardware prototype and simulation demonstrate that our proposed sampling scheme significantly reduces the convergence time compared to several baseline sampling schemes. Notably, our scheme in hardware prototype spends 73% less time than the uniform sampling baseline for reaching the same target loss.

cs.LG↗

arXiv2021

Cost-Effective Federated Learning in Mobile Edge Networks

Federated learning (FL) is a distributed learning paradigm that enables a large number of mobile devices to collaboratively learn a model under the coordination of a central server without sharing their raw data. Despite its practical efficiency and effectiveness, the iterative on-device learning process (e.g., local computations and global communications with the server) incurs a considerable cost in terms of learning time and energy consumption, which depends crucially on the number of selected clients and the number of local iterations in each training round. In this paper, we analyze how to design adaptive FL in mobile edge networks that optimally chooses these essential control variables to minimize the total cost while ensuring convergence. We establish the analytical relationship between the total cost and the control variables with the convergence upper bound. To efficiently solve the cost minimization problem, we develop a low-cost sampling-based algorithm to learn the convergence related unknown parameters. We derive important solution properties that effectively identify the design principles for different optimization metrics. Practically, we evaluate our theoretical results both in a simulated environment and on a hardware prototype. Experimental evidence verifies our derived properties and demonstrates that our proposed solution achieves near-optimal performance for different optimization metrics for various datasets and heterogeneous system and statistical settings.

cs.LG↗

arXiv2021

Federated Learning with Spiking Neural Networks

As neural networks get widespread adoption in resource-constrained embedded devices, there is a growing need for low-power neural systems. Spiking Neural Networks (SNNs)are emerging to be an energy-efficient alternative to the traditional Artificial Neural Networks (ANNs) which are known to be computationally intensive. From an application perspective, as federated learning involves multiple energy-constrained devices, there is a huge scope to leverage energy efficiency provided by SNNs. Despite its importance, there has been little attention on training SNNs on a large-scale distributed system like federated learning. In this paper, we bring SNNs to a more realistic federated learning scenario. Specifically, we propose a federated learning framework for decentralized and privacy-preserving training of SNNs. To validate the proposed federated learning framework, we experimentally evaluate the advantages of SNNs on various aspects of federated learning with CIFAR10 and CIFAR100 benchmarks. We observe that SNNs outperform ANNs in terms of overall accuracy by over 15% when the data is distributed across a large number of clients in the federation while providing up to5.3x energy efficiency. In addition to efficiency, we also analyze the sensitivity of the proposed federated SNN framework to data distribution among the clients, stragglers, and gradient noise and perform a comprehensive comparison with ANNs.

cs.LG↗

arXiv2021

Optimal Bidding Strategy for Maker Auctions

The Maker Protocol is a decentralized finance application that enables collateralized lending. The application uses open-bid, second-price auctions to complete its loan liquidation process. In this paper, we develop a bidding function for these auctions, focusing on the costs incurred to participate in the auctions. We then optimize these costs using parameters from historical auction data, and compare our optimal bidding prices to the historical auction prices. We find that the majority of auctions end at higher prices than our recommended optimal prices, and we propose several theories for these results.

q-fin.TR↗

arXiv2021

State-Dependent Processing in Payment Channel Networks for Throughput Optimization

Payment channel networks (PCNs) have emerged as a scalability solution for blockchains built on the concept of a payment channel: a setting that allows two nodes to safely transact between themselves in high frequencies based on pre-committed peer-to-peer balances. Transaction requests in these networks may be declined because of unavailability of funds due to temporary uneven distribution of the channel balances. In this paper, we investigate how to alleviate unnecessary payment blockage via proper prioritization of the transaction execution order. Specifically, we consider the scheduling problem in PCNs: as transactions continuously arrive on both sides of a channel, nodes need to decide which ones to process and when in order to maximize their objective, which in our case is the channel throughput. We introduce a stochastic model to capture the dynamics of a payment channel under random arrivals, and propose that channels can hold incoming transactions in buffers up to some deadline in order to enable more elaborate processing decisions. We describe a policy that maximizes the channel success rate/throughput for uniform transaction requests of fixed amounts, both in the presence and absence of buffering capabilities, and formally prove its optimality. We also develop a discrete event simulator of a payment channel, and evaluate different heuristic scheduling policies in the more general heterogeneous amounts case, with the results showing superiority of the heuristic extension of our policy in this case as well. Our work opens the way for more formal research on improving PCN performance via joint consideration of routing and scheduling decisions.

eess.SY↗

arXiv2020

Cost-Effective Federated Learning Design

Federated learning (FL) is a distributed learning paradigm that enables a large number of devices to collaboratively learn a model without sharing their raw data. Despite its practical efficiency and effectiveness, the iterative on-device learning process incurs a considerable cost in terms of learning time and energy consumption, which depends crucially on the number of selected clients and the number of local iterations in each training round. In this paper, we analyze how to design adaptive FL that optimally chooses these essential control variables to minimize the total cost while ensuring convergence. Theoretically, we analytically establish the relationship between the total cost and the control variables with the convergence upper bound. To efficiently solve the cost minimization problem, we develop a low-cost sampling-based algorithm to learn the convergence related unknown parameters. We derive important solution properties that effectively identify the design principles for different metric preferences. Practically, we evaluate our theoretical results both in a simulated environment and on a hardware prototype. Experimental evidence verifies our derived properties and demonstrates that our proposed solution achieves near-optimal performance for various datasets, different machine learning models, and heterogeneous system settings.

cs.LG↗

arXiv2020

Birkhoff's Decomposition Revisited: Sparse Scheduling for High-Speed Circuit Switches

Data centers are increasingly using high-speed circuit switches to cope with the growing demand and reduce operational costs. One of the fundamental tasks of circuit switches is to compute a sparse collection of switching configurations to support a traffic demand matrix. Such a problem has been addressed in the literature with variations of the approach proposed by Birkhoff in 1946 to decompose a doubly stochastic matrix exactly. However, the existing methods are heuristic and do not have theoretical guarantees on how well a collection of switching configurations (i.e., permutations) can approximate a traffic matrix (i.e., a scaled doubly stochastic matrix). In this paper, we revisit Birkhoff's approach and make three contributions. First, we establish the first theoretical bound on the sparsity of Birkhoff's algorithm (i.e., the number of switching configurations necessary to approximate a traffic matrix). In particular, we show that by using a subset of the admissible permutation matrices, Birkhoff's algorithm obtains an $ε$-approximate decomposition with at most $O( \log(1 / ε))$ permutations. Second, we propose a new algorithm, Birkhoff+, which combines the wealth of Frank-Wolfe with Birkhoff's approach to obtain sparse decompositions in a fast manner. And third, we evaluate the performance of the proposed algorithm numerically and study how this affects the performance of a circuit switch. Our results show that Birkhoff+ is superior to previous algorithms in terms of throughput, running time, and number of switching configurations.

math.OC↗

arXiv2020

A Blockchain-based Decentralized Data Sharing Infrastructure for Off-grid Networking

Off-grid networks are recently emerging as a solution to connect the unconnected or provide alternative services to networks of possibly untrusted participants. The systems currently used, however, exhibit limitations due to their centralized nature and thus prove inadequate to secure trust. Blockchain technology can be the tool that will enable trust and transparency in such networks. In this paper, we introduce a platform for secure and privacy-respecting decentralized data sharing among untrusted participants in off-grid networks. The proposed architecture realizes this goal via the integration of existing blockchain frameworks (Hyperledger Fabric, Indy, Aries) with an off-grid network device and a distributed file system. We evaluate the proposed platform through experiments and show results for its throughput and latency, which indicate its adequate performance for supporting off-grid decentralized applications.

cs.DC↗

arXiv2019

Online Convex Optimization with Perturbed Constraints

This paper addresses Online Convex Optimization (OCO) problems where the constraints have additive perturbations that (i) vary over time and (ii) are not known at the time to make a decision. Perturbations may not be i.i.d. generated and can be used to model a time-varying budget or commodity in resource allocation problems. The problem is to design a policy that obtains sublinear regret while ensuring that the constraints are satisfied on average. To solve this problem, we present a primal-dual proximal gradient algorithm that has $O(T^ε\vee T^{1-ε})$ regret and $O(T^ε)$ constraint violation, where $ε\in [0,1)$ is a parameter in the learning rate. Our results match the bounds of previous work on OCO with time-varying constraints when $ε= 1/2$; however, we (i) define the regret using a time-varying set of best fixed decisions; (ii) can balance between regret and constraint violation; and (iii) use an adaptive learning rate that allows us to run the algorithm for any time horizon.

math.OC↗

arXiv2019

Learning the Optimal Synchronization Rates in Distributed SDN Control Architectures

Since the early development of Software-Defined Network (SDN) technology, researchers have been concerned with the idea of physical distribution of the control plane to address scalability and reliability challenges of centralized designs. However, having multiple controllers managing the network while maintaining a "logically-centralized" network view brings additional challenges. One such challenge is how to coordinate the management decisions made by the controllers which is usually achieved by disseminating synchronization messages in a peer-to-peer manner. While there exist many architectures and protocols to ensure synchronized network views and drive coordination among controllers, there is no systematic methodology for deciding the optimal frequency (or rate) of message dissemination. In this paper, we fill this gap by introducing the SDN synchronization problem: how often to synchronize the network views for each controller pair. We consider two different objectives; first, the maximization of the number of controller pairs that are synchronized, and second, the maximization of the performance of applications of interest which may be affected by the synchronization rate. Using techniques from knapsack optimization and learning theory, we derive algorithms with provable performance guarantees for each objective. Evaluation results demonstrate significant benefits over baseline schemes that synchronize all controller pairs at equal rate.

cs.NI↗

arXiv2019

Joint Service Placement and Request Routing in Multi-cell Mobile Edge Computing Networks

The proliferation of innovative mobile services such as augmented reality, networked gaming, and autonomous driving has spurred a growing need for low-latency access to computing resources that cannot be met solely by existing centralized cloud systems. Mobile Edge Computing (MEC) is expected to be an effective solution to meet the demand for low-latency services by enabling the execution of computing tasks at the network-periphery, in proximity to end-users. While a number of recent studies have addressed the problem of determining the execution of service tasks and the routing of user requests to corresponding edge servers, the focus has primarily been on the efficient utilization of computing resources, neglecting the fact that non-trivial amounts of data need to be stored to enable service execution, and that many emerging services exhibit asymmetric bandwidth requirements. To fill this gap, we study the joint optimization of service placement and request routing in MEC-enabled multi-cell networks with multidimensional (storage-computation-communication) constraints. We show that this problem generalizes several problems in literature and propose an algorithm that achieves close-to-optimal performance using randomized rounding. Evaluation results demonstrate that our approach can effectively utilize the available resources to maximize the number of requests served by low-latency edge cloud servers.

cs.NI↗

arXiv2018

SDN-enabled Tactical Ad Hoc Networks: Extending Programmable Control to the Edge

Modern tactical operations have complex communication and computing requirements, often involving different coalition teams, that cannot be supported by today's mobile ad hoc networks. To this end, the emerging Software Defined Networking (SDN) paradigm has the potential to enable the redesign and successful deployment of these systems. In this paper, we propose a set of novel architecture designs for SDN-enabled mobile ad hoc networks in the tactical field. We discuss in detail the challenges raised by the ad hoc and coalition network environment, and we present specific solutions to address them. The proposed approaches build on evidence from experimental evaluation of such architectures and leverage recent theoretical results from SDN deployments in large backbone networks.

cs.NI↗

arXiv2017

How Better is Distributed SDN? An Analytical Approach

Distributed software-defined networks (SDN), consisting of multiple inter-connected network domains, each managed by one SDN controller, is an emerging networking architecture that offers balanced centralized control and distributed operations. Under such networking paradigm, most existing works focus on designing sophisticated controller-synchronization strategies to improve joint controller-decision-making for inter-domain routing. However, there is still a lack of fundamental understanding of how the performance of distributed SDN is related to network attributes, thus impossible to justify the necessity of complicated strategies. In this regard, we analyze and quantify the performance enhancement of distributed SDN architectures, influenced by intra-/inter-domain synchronization levels and network structural properties. Based on a generic weighted network model, we establish analytical methods for performance estimation under four synchronization scenarios with increasing synchronization cost. Moreover, two of these synchronization scenarios correspond to extreme cases, i.e., minimum/maximum synchronization, which are, therefore, capable of bounding the performance of distributed SDN with any given synchronization levels. Our theoretical results reveal how network performance is related to synchronization levels and inter-domain connections, the accuracy of which are confirmed by simulations based on both real and synthetic networks. To the best of our knowledge, this is the first work quantifying the performance of distributed SDN analytically, which provides fundamental guidance for future SDN protocol designs and performance estimation.

cs.DC↗

arXiv2017

Routing entanglement in the quantum internet

Remote quantum entanglement can enable numerous applications including distributed quantum computation, secure communication, and precision sensing. In this paper, we consider how a quantum network-nodes equipped with limited quantum processing capabilities connected via lossy optical links-can distribute high-rate entanglement simultaneously between multiple pairs of users (multiple flows). We develop protocols for such quantum "repeater" nodes, which enable a pair of users to achieve large gains in entanglement rates over using a linear chain of quantum repeaters, by exploiting the diversity of multiple paths in the network. Additionally, we develop repeater protocols that enable multiple user pairs to generate entanglement simultaneously at rates that can far exceed what is possible with repeaters time sharing among assisting individual entanglement flows. Our results suggest that the early-stage development of quantum memories with short coherence times and implementations of probabilistic Bell-state measurements can have a much more profound impact on quantum networks than may be apparent from analyzing linear repeater chains. This framework should spur the development of a general quantum network theory, bringing together quantum memory physics, quantum information theory, and computer network theory.

quant-ph↗

arXiv2017

Dynamic Policies for Cooperative Networked Systems

A set of economic entities embedded in a network graph collaborate by opportunistically exchanging their resources to satisfy their dynamically generated needs. Under what conditions their collaboration leads to a sustainable economy? Which online policy can ensure a feasible resource exchange point will be attained, and what information is needed to implement it? Furthermore, assuming there are different resources and the entities have diverse production capabilities, which production policy each entity should employ in order to maximize the economy's sustainability? Importantly, can we design such policies that are also incentive compatible even when there is no a priori information about the entities' needs? We introduce a dynamic production scheduling and resource exchange model to capture this fundamental problem and provide answers to the above questions. Applications range from infrastructure sharing, trade and organisation management, to social networks and sharing economy services.

cs.GT↗

arXiv2017

Bringing SDN to the Mobile Edge

Nowadays, Software Defined Network (SDN) architectures and applications are revolutionizing the way wired networks are built and operate. However, little is known about the potential of this disruptive technology in wireless mobile networks. In fact, SDN is based on a centralized network control principle, while existing mobile network protocols give emphasis on the distribution of network resources and their management. Therefore, it is challenging to apply SDN ideas in the context of mobile networks. In this paper, we propose methods to overcome these challenges and make SDN more suitable for the mobile environment. Our main idea is to combine centralized SDN and distributed control in a hybrid design that takes the best of the two paradigms; (i) global network view and control programmability of SDN and (ii) robustness of distributed protocols. We discuss the pros and cons of each method and highlight them in an SDN prototype implementation built using off-the-shelf mobile devices.

cs.NI↗