arXiv subjects

Hulya Seferoglu

Publications and source records attributed to Hulya Seferoglu.

At least 19 recordsLinked to original sources

arXiv2025

Efficient and Privacy-Preserving Binary Dot Product via Multi-Party Computation

Striking a balance between protecting data privacy and enabling collaborative computation is a critical challenge for distributed machine learning. While privacy-preserving techniques for federated learning have been extensively developed, methods for scenarios involving bitwise operations, such as tree-based vertical federated learning (VFL), are still underexplored. Traditional mechanisms, including Shamir's secret sharing and multi-party computation (MPC), are not optimized for bitwise operations over binary data, particularly in settings where each participant holds a different part of the binary vector. This paper addresses the limitations of existing methods by proposing a novel binary multi-party computation (BiMPC) framework. The BiMPC mechanism facilitates privacy-preserving bitwise operations, with a particular focus on dot product computations of binary vectors, ensuring the privacy of each individual bit. The core of BiMPC is a novel approach called Dot Product via Modular Addition (DoMA), which uses regular and modular additions for efficient binary dot product calculation. To ensure privacy, BiMPC uses random masking in a higher field for linear computations and a three-party oblivious transfer (triot) protocol for non-linear binary operations. The privacy guarantees of the BiMPC framework are rigorously analyzed, demonstrating its efficiency and scalability in distributed settings.

cs.CR↗

arXiv2025

Model-Distributed Inference for Large Language Models at the Edge

We introduce Model-Distributed Inference for Large-Language Models (MDI-LLM), a novel framework designed to facilitate the deployment of state-of-the-art large-language models (LLMs) across low-power devices at the edge. This is accomplished by dividing the model into multiple partitions, which are then assigned to different devices/nodes within the network. These nodes exchange intermediate activation vectors via device-to-device links, enabling collaborative computation. To enhance the efficiency of this process, we propose the "recurrent pipeline parallelism" technique, which reduces idle time on each device and facilitates parallel inference during the generation of multiple text sequences. By leveraging the combined computational resources of multiple edge devices, MDI-LLM enables the deployment of LLMs that exceed the memory capacity of individual devices, making it possible to perform inference on low-cost hardware. Furthermore, as the number of participating devices increases, MDI-LLM boosts token generation throughput and reduces memory consumption per device.

cs.LG↗

arXiv2025

A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip

Although gossip and random walk-based learning algorithms are widely known for decentralized learning, there has been limited theoretical and experimental analysis to understand their relative performance for different graph topologies and data heterogeneity. We first design and analyze a random walk-based learning algorithm with multiple streams (walks), which we name asynchronous "Multi-Walk (MW)". We provide a convergence analysis for MW w.r.t iteration (computation), wall-clock time, and communication. We also present a convergence analysis for "Asynchronous Gossip", noting the lack of a comprehensive analysis of its convergence, along with the computation and communication overhead, in the literature. Our results show that MW has better convergence in terms of iterations as compared to Asynchronous Gossip in graphs with large diameters (e.g., cycles), while its relative performance, as compared to Asynchronous Gossip, depends on the number of walks and the data heterogeneity in graphs with small diameters (e.g., complete graphs). In wall-clock time analysis, we observe a linear speed-up with the number of walks and nodes in MW and Asynchronous Gossip, respectively. Finally, we show that MW outperforms Asynchronous Gossip in communication overhead, except in small-diameter topologies with extreme data heterogeneity. These results highlight the effectiveness of each algorithm in different graph topologies and data heterogeneity. Our codes are available for reproducibility.

cs.LG↗

arXiv2024

Priority-Aware Model-Distributed Inference at Edge Networks

Distributed inference techniques can be broadly classified into data-distributed and model-distributed schemes. In data-distributed inference (DDI), each worker carries the entire Machine Learning (ML) model but processes only a subset of the data. However, feeding the data to workers results in high communication costs, especially when the data is large. An emerging paradigm is model-distributed inference (MDI), where each worker carries only a subset of ML layers. In MDI, a source device that has data processes a few layers of ML model and sends the output to a neighboring device, i.e., offloads the rest of the layers. This process ends when all layers are processed in a distributed manner. In this paper, we investigate the design and development of MDI when multiple data sources co-exist. We consider that each data source has a different importance and, hence, a priority. We formulate and solve a priority-aware model allocation optimization problem. Based on the structure of the optimal solution, we design a practical Priority-Aware Model- Distributed Inference (PA-MDI) algorithm that determines model allocation and distribution over devices by taking into account the priorities of different sources. Experiments were conducted on a real-life testbed of NVIDIA Jetson Xavier and Nano edge devices as well as in the Colosseum testbed with ResNet-50, ResNet- 56, and GPT-2 models. The experimental results show that PA-MDI performs priority-aware model allocation successfully while reducing the inference time as compared to baselines.

cs.DC↗

arXiv2024

Early-Exit meets Model-Distributed Inference at Edge Networks

Distributed inference techniques can be broadly classified into data-distributed and model-distributed schemes. In data-distributed inference (DDI), each worker carries the entire deep neural network (DNN) model but processes only a subset of the data. However, feeding the data to workers results in high communication costs, especially when the data is large. An emerging paradigm is model-distributed inference (MDI), where each worker carries only a subset of DNN layers. In MDI, a source device that has data processes a few layers of DNN and sends the output to a neighboring device, i.e., offloads the rest of the layers. This process ends when all layers are processed in a distributed manner. In this paper, we investigate the design and development of MDI with early-exit, which advocates that there is no need to process all the layers of a model for some data to reach the desired accuracy, i.e., we can exit the model without processing all the layers if target accuracy is reached. We design a framework MDI-Exit that adaptively determines early-exit and offloading policies as well as data admission at the source. Experimental results on a real-life testbed of NVIDIA Nano edge devices show that MDI-Exit processes more data when accuracy is fixed and results in higher accuracy for the fixed data rate.

cs.DC↗

arXiv2024

Privacy-Preserving Hierarchical Model-Distributed Inference

This paper focuses on designing a privacy-preserving Machine Learning (ML) inference protocol for a hierarchical setup, where clients own/generate data, model owners (cloud servers) have a pre-trained ML model, and edge servers perform ML inference on clients' data using the cloud server's ML model. Our goal is to speed up ML inference while providing privacy to both data and the ML model. Our approach (i) uses model-distributed inference (model parallelization) at the edge servers and (ii) reduces the amount of communication to/from the cloud server. Our privacy-preserving hierarchical model-distributed inference, privateMDI design uses additive secret sharing and linearly homomorphic encryption to handle linear calculations in the ML inference, and garbled circuit and a novel three-party oblivious transfer are used to handle non-linear functions. privateMDI consists of offline and online phases. We designed these phases in a way that most of the data exchange is done in the offline phase while the communication overhead of the online phase is reduced. In particular, there is no communication to/from the cloud server in the online phase, and the amount of communication between the client and edge servers is minimized. The experimental results demonstrate that privateMDI significantly reduces the ML inference time as compared to the baselines.

cs.CR↗

arXiv2024

Differentiated Aggregation to Improve Generalization in Federated Learning

This paper focuses on reducing the communication cost of federated learning by exploring generalization bounds and representation learning. We first characterize a tighter generalization bound for one-round federated learning based on local clients' generalizations and heterogeneity of data distribution (non-iid scenario). We also characterize a generalization bound in R-round federated learning and its relation to the number of local updates (local stochastic gradient descents (SGDs)). Then, based on our generalization bound analysis and its interpretation through representation learning, we infer that less frequent aggregations for the representation extractor (typically corresponds to initial layers) compared to the head (usually the final layers) leads to the creation of more generalizable models, particularly in non-iid scenarios. We design a novel Federated Learning with Adaptive Local Steps (FedALS) algorithm based on our generalization bound and representation learning analysis. FedALS employs varying aggregation frequencies for different parts of the model, so reduces the communication cost. The paper is followed with experimental results showing the effectiveness of FedALS. Our codes are available at for reproducibility.

cs.LG↗

arXiv2023

DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates

Two widely considered decentralized learning algorithms are Gossip and random walk-based learning. Gossip algorithms (both synchronous and asynchronous versions) suffer from high communication cost, while random-walk based learning experiences increased convergence time. In this paper, we design a fast and communication-efficient asynchronous decentralized learning mechanism DIGEST by taking advantage of both Gossip and random-walk ideas, and focusing on stochastic gradient descent (SGD). DIGEST is an asynchronous decentralized algorithm building on local-SGD algorithms, which are originally designed for communication efficient centralized learning. We design both single-stream and multi-stream DIGEST, where the communication overhead may increase when the number of streams increases, and there is a convergence and communication overhead trade-off which can be leveraged. We analyze the convergence of single- and multi-stream DIGEST, and prove that both algorithms approach to the optimal solution asymptotically for both iid and non-iid data distributions. We evaluate the performance of single- and multi-stream DIGEST for logistic regression and a deep neural network ResNet20. The simulation results confirm that multi-stream DIGEST has nice convergence properties; i.e., its convergence time is better than or comparable to the baselines in iid setting, and outperforms the baselines in non-iid setting.

cs.LG↗

arXiv2023

Efficient Coded Multi-Party Computation at Edge Networks

Multi-party computation (MPC) is promising for designing privacy-preserving machine learning algorithms at edge networks. An emerging approach is coded-MPC (CMPC), which advocates the use of coded computation to improve the performance of MPC in terms of the required number of workers involved in computations. The current approach for designing CMPC algorithms is to merely combine efficient coded computation constructions with MPC. We show that this approach fails short of being efficient; e.g., entangled polynomial codes are not necessarily better than PolyDot codes in MPC setting, while they are always better for coded computation. Motivated by this observation, we propose a new construction; Adaptive Gap Entangled (AGE) polynomial codes for MPC. We show through analysis and simulations that MPC with AGE codes always perform better than existing CMPC algorithms in terms of the required number of workers as well as computation, storage, and communication overhead.

cs.DC↗

arXiv2022

Adaptive Gap Entangled Polynomial Coding for Multi-Party Computation at the Edge

Multi-party computation (MPC) is promising for designing privacy-preserving machine learning algorithms at edge networks. An emerging approach is coded-MPC (CMPC), which advocates the use of coded computation to improve the performance of MPC in terms of the required number of workers involved in computations. The current approach for designing CMPC algorithms is to merely combine efficient coded computation constructions with MPC. Instead, we propose a new construction; Adaptive Gap Entangled polynomial (AGE) codes, where the degrees of polynomials used in computations are optimized for MPC. We show that MPC with AGE codes (AGE-CMPC) performs better than existing CMPC algorithms in terms of the required number of workers as well as storage, communication and computation load.

cs.DC↗

arXiv2021

Capacity and Stability Regions for Layered Packet Erasure Broadcast Channels with Feedback

This paper focuses on the Layered Packet Erasure Broadcast Channel (LPE-BC) with Channel Output Feedback (COF) available at the transmitter. The LPE-BC is a high-SNR approximation of the fading Gaussian BC recently proposed by Tse and Yates, who characterized the capacity region for any number of users and any number of layers when there is no COF. This paper provides a comparative overview of this channel model along the following lines: First, inner and outer bounds to the capacity region (set of achievable rates with backlogged arrivals) are presented: a) a new outer bound based on the idea of the physically degraded broadcast channel, and b) an inner bound of the LPE-BC with COF for the case of two users and any number of layers. Next, an inner bound on the stability region (set of exogenous arrival rates for which packet arrival queues are stable) for the same model is derived. The capacity region inner bound generalizes past results for the two-user erasure BC, which is a special case of the LPE-BC with COF with only one layer. The novelty lies in the use of inter-user and inter-layer network coding retransmissions (for those packets that have only been received by the unintended user), where each random linear combination may involve packets intended for any user originally sent on any of the layers. For the case of $K = 2$ users and $Q \geq 1$ layers, the inner bounds to the capacity region and the stability region coincide; both strategically employ the novel retransmission protocol. For the case of $Q = 2$ layers, sufficient conditions are derived by Fourier-Motzkin elimination for the inner bound on the stability region to coincide with the capacity outer bound, thus showing that in those cases the capacity and stability regions coincide.

cs.IT↗

arXiv2021

PolyDot Coded Privacy Preserving Multi-Party Computation at the Edge

We investigate the problem of privacy preserving distributed matrix multiplication in edge networks using multi-party computation (MPC). Coded multi-party computation (CMPC) is an emerging approach to reduce the required number of workers in MPC by employing coded computation. Existing CMPC approaches usually combine coded computation algorithms designed for efficient matrix multiplication with MPC. We show that this approach is not efficient. We design a novel CMPC algorithm; PolyDot coded MPC (PolyDot-CMPC) by using a recently proposed coded computation algorithm; PolyDot codes. We exploit "garbage terms" that naturally arise when polynomials are constructed in the design of PolyDot-CMPC to reduce the number of workers needed for privacy-preserving computation. We show that entangled polynomial codes, which are consistently better than PolyDot codes in coded computation setup, are not necessarily better than PolyDot-CMPC in MPC setting.

cs.IT↗

arXiv2021

Adaptive Coding for Matrix Multiplication at Edge Networks

Edge computing is emerging as a new paradigm to allow processing data at the edge of the network, where data is typically generated and collected, by exploiting multiple devices at the edge collectively. However, exploiting the potential of edge computing is challenging mainly due to the heterogeneous and time-varying nature of edge devices. Coded computation, which advocates mixing data in sub-tasks by employing erasure codes and offloading these sub-tasks to other devices for computation, is recently gaining interest, thanks to its higher reliability, smaller delay, and lower communication cost. In this paper, our focus is on characterizing the cost-benefit trade-offs of coded computation for practical edge computing systems, and develop an adaptive coded computation framework. In particular, we focus on matrix multiplication as a computationally intensive task, and develop an adaptive coding for matrix multiplication (ACM^2) algorithm by taking into account the heterogeneous and time varying nature of edge devices. ACM^2 dynamically selects the best coding policy by taking into account the computing time, storage requirements as well as successful decoding probability. We show that ACM^2 improves the task completion delay significantly as compared to existing coded matrix multiplication algorithms.

cs.IT↗

arXiv2020

Packet Completion Time Minimization via Joint D2D and Cellular Communication: A Unified Network Coding Approach

This paper tackles the problem of transmitting a common content to a number of cellular users by means of instantly decodable network coding (IDNC) with the help of intermittently connected D2D links. Of particular interest are broadcasting real-time applications such as video-on-demand, where common contents may be partially received by cellular users due to packet erasures over cellular links. Specifically, we investigate the problem of packet completion time, defined as the number of transmission slots necessary to deliver a common content to all users. Drawing on graph theory, we develop an optimal packet completion time strategy by constructing a two-layer IDNC conflict graph. The higher-layer graph permits us to determine all feasible packet combinations that can be transmitted over the cellular link, while the lower-layer graph enables us to find all feasible network coded packets and identify the set of users that can generate and transmit these packets via intermittently connected D2D links. By combining the higher-layer and the lower-layer IDNC conflict graphs, we demonstrate that finding the optimal IDNC packets to minimize the packet completion time problem is equivalent to finding the maximum independent set of the two-layer IDNC conflict graph, which is known to be an NP-hard problem. We design a scheme that invokes the Bron-Kerbosch algorithm to find the optimal policy. To circumvent the high computational complexity required to reach the global optimum, we establish a polynomial-time solvable low-complexity heuristic to find an efficient sub-optimal solution. The effectiveness of our proposed scheme is verified through extensive numerical results which indicate substantial performance improvement in comparison with existing methods.

cs.NI↗

arXiv2019

Robust and Computationally-Efficient Anomaly Detection using Powers-of-Two Networks

Robust and computationally efficient anomaly detection in videos is a problem in video surveillance systems. We propose a technique to increase robustness and reduce computational complexity in a Convolutional Neural Network (CNN) based anomaly detector that utilizes the optical flow information of video data. We reduce the complexity of the network by denoising the intermediate layer outputs of the CNN and by using powers-of-two weights, which replaces the computationally expensive multiplication operations with bit-shift operations. Denoising operation during inference forces small valued intermediate layer outputs to zero. The number of zeros in the network significantly increases as a result of denoising, we can implement the CNN about 10% faster than a comparable network while detecting all the anomalies in the testing set. It turns out that denoising operation also provides robustness because the contribution of small intermediate values to the final result is negligible. During training we also generate motion vector images by a Generative Adversarial Network (GAN) to improve the robustness of the overall system. We experimentally observe that the resulting system is robust to background motion.

cs.LG↗

arXiv2019

Private and Rateless Adaptive Coded Matrix-Vector Multiplication

Edge computing is emerging as a new paradigm to allow processing data near the edge of the network, where the data is typically generated and collected. This enables critical computations at the edge in applications such as Internet of Things (IoT), in which an increasing number of devices (sensors, cameras, health monitoring devices, etc.) collect data that needs to be processed through computationally intensive algorithms with stringent reliability, security and latency constraints. Our key tool is the theory of coded computation, which advocates mixing data in computationally intensive tasks by employing erasure codes and offloading these tasks to other devices for computation. Coded computation is recently gaining interest, thanks to its higher reliability, smaller delay, and lower communication costs. In this paper, we develop a private and rateless adaptive coded computation (PRAC) algorithm for distributed matrix-vector multiplication by taking into account (i) the privacy requirements of IoT applications and devices, and (ii) the heterogeneous and time-varying resources of edge devices. We show that PRAC outperforms known secure coded computing methods when resources are heterogeneous. We provide theoretical guarantees on the performance of PRAC and its comparison to baselines. Moreover, we confirm our theoretical results through simulations and implementations on Android-based smartphones.

cs.IT↗

arXiv2019

Secure Coded Cooperative Computation at the Heterogeneous Edge against Byzantine Attacks

Edge computing is emerging as a new paradigm to allow processing data at the edge of the network, where data is typically generated and collected, by exploiting multiple devices at the edge collectively. However, offloading tasks to other devices leaves the edge computing applications at the complete mercy of an attacker. One of the attacks, which is also the focus of this work, is Byzantine attacks, where one or more devices can corrupt the offloaded tasks. Furthermore, exploiting the potential of edge computing is challenging mainly due to the heterogeneous and time-varying nature of the devices at the edge. In this paper, we develop a secure coded cooperative computation mechanism (SC3) that provides both security and computation efficiency guarantees by gracefully combining homomorphic hash functions and coded cooperative computation. Homomorphic hash functions are used against Byzantine attacks and coded cooperative computation is used to improve computation efficiency when edge resources are heterogeneous and time-varying. Simulations results show that SC3 improves task completion delay significantly.

cs.DC↗

arXiv2018

On Erasure Broadcast Channels with Hard Deadlines

This paper considers packet scheduling over a broadcast channel with packet erasures to multiple receivers with different messages (multiple uni-cast) each with possibly different hard deadline constraints. A novel metric is proposed and evaluated: the global deadline outage probability, which gives the probability that the hard communication deadline is not met for at least one of the receivers. The cut-set upper bound is derived and a scheduling policy is proposed to determine which receiver's packets should be sent in each time slot. This policy is shown to be optimal among all scheduling policies, i.e., it achieves all boundary points of cut-set upper bounds when the transmitter knows the erasure patterns for all the receivers ahead of making the scheduling decision. An expression for the global deadline outage probability is obtained for two receivers and is plotted and interpreted for various system parameters. These plots are not Monte-Carlo simulations, and hence the obtained expression may be used in the design of future downlink broadcast networks. Future extensions to per-user deadline outage probabilities as well as to scenarios with causal knowledge of the channel states are briefly discussed.

cs.NI↗