arXiv subjects

Dong-Jun Han

Publications and source records attributed to Dong-Jun Han.

44 records · Page 3Linked to original sources

arXiv2023

Test-Time Style Shifting: Handling Arbitrary Styles in Domain Generalization

In domain generalization (DG), the target domain is unknown when the model is being trained, and the trained model should successfully work on an arbitrary (and possibly unseen) target domain during inference. This is a difficult problem, and despite active studies in recent years, it remains a great challenge. In this paper, we take a simple yet effective approach to tackle this issue. We propose test-time style shifting, which shifts the style of the test sample (that has a large style gap with the source domains) to the nearest source domain that the model is already familiar with, before making the prediction. This strategy enables the model to handle any target domains with arbitrary style statistics, without additional model update at test-time. Additionally, we propose style balancing, which provides a great platform for maximizing the advantage of test-time style shifting by handling the DG-specific imbalance issues. The proposed ideas are easy to implement and successfully work in conjunction with various other DG schemes. Experimental results on different datasets show the effectiveness of our methods.

cs.CV↗

arXiv2023

SplitGP: Achieving Both Generalization and Personalization in Federated Learning

A fundamental challenge to providing edge-AI services is the need for a machine learning (ML) model that achieves personalization (i.e., to individual clients) and generalization (i.e., to unseen data) properties concurrently. Existing techniques in federated learning (FL) have encountered a steep tradeoff between these objectives and impose large computational requirements on edge devices during training and inference. In this paper, we propose SplitGP, a new split learning solution that can simultaneously capture generalization and personalization capabilities for efficient inference across resource-constrained clients (e.g., mobile/IoT devices). Our key idea is to split the full ML model into client-side and server-side components, and impose different roles to them: the client-side model is trained to have strong personalization capability optimized to each client's main task, while the server-side model is trained to have strong generalization capability for handling all clients' out-of-distribution tasks. We analytically characterize the convergence behavior of SplitGP, revealing that all client models approach stationary points asymptotically. Further, we analyze the inference time in SplitGP and provide bounds for determining model split ratios. Experimental results show that SplitGP outperforms existing baselines by wide margins in inference time and test accuracy for varying amounts of out-of-distribution samples.

cs.LG↗

arXiv2021

Communication-Computation Efficient Secure Aggregation for Federated Learning

Federated learning has been spotlighted as a way to train neural networks using distributed data with no need for individual nodes to share data. Unfortunately, it has also been shown that adversaries may be able to extract local data contents off model parameters transmitted during federated learning. A recent solution based on the secure aggregation primitive enabled privacy-preserving federated learning, but at the expense of significant extra communication/computational resources. In this paper, we propose a low-complexity scheme that provides data privacy using substantially reduced communication/computational resources relative to the existing secure solution. The key idea behind the suggested scheme is to design the topology of secret-sharing nodes as a sparse random graph instead of the complete graph corresponding to the existing solution. We first obtain the necessary and sufficient condition on the graph to guarantee both reliability and privacy. We then suggest using the Erdős-Rényi graph in particular and provide theoretical guarantees on the reliability/privacy of the proposed scheme. Through extensive real-world experiments, we demonstrate that our scheme, using only $20 \sim 30\%$ of the resources required in the conventional scheme, maintains virtually the same levels of reliability and data privacy in practical federated learning systems.

cs.LG↗

arXiv2020

Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks

Recent advances in large-scale distributed learning algorithms have enabled communication-efficient training via SignSGD. Unfortunately, a major issue continues to plague distributed learning: namely, Byzantine failures may incur serious degradation in learning accuracy. This paper proposes Election Coding, a coding-theoretic framework to guarantee Byzantine-robustness for SignSGD with Majority Vote, which uses minimum worker-master communication in both directions. The suggested framework explores new information-theoretic limits of finding the majority opinion when some workers could be malicious, and paves the road to implement robust and efficient distributed learning algorithms. Under this framework, we construct two types of explicit codes, random Bernoulli codes and deterministic algebraic codes, that can tolerate Byzantine attacks with a controlled amount of computational redundancy. For the Bernoulli codes, we provide upper bounds on the error probability in estimating the majority opinion, which give useful insights into code design for tolerating Byzantine attacks. As for deterministic codes, we construct an explicit code which perfectly tolerates Byzantines, and provide tight upper/lower bounds on the minimum required computational redundancy. Finally, the Byzantine-tolerance of the suggested coding schemes is confirmed by deep learning experiments on Amazon EC2 using Python with MPI4py package.

cs.IT↗

arXiv2020

Cache Allocations for Consecutive Requests of Categorized Contents: Service Provider's Perspective

In wireless caching networks, a user generally has a concrete purpose of consuming contents in a certain preferred category, and requests multiple contents in sequence. While most existing research on wireless caching and delivery has focused only on one-shot requests, the popularity distribution of contents requested consecutively is definitely different from the one-shot request and has been not considered. Also, especially from the perspective of the service provider, it is advantageous for users to consume as many contents as possible. Thus, this paper proposes two cache allocation policies for categorized contents and consecutive user demands, which maximize 1) the cache hit rate and 2) the number of consecutive content consumption, respectively. Numerical results show how categorized contents and consecutive content requests have impacts on the cache allocation.

cs.NI↗

arXiv2019

Coded Distributed Computing over Packet Erasure Channels

Coded computation is a framework which provides redundancy in distributed computing systems to speed up largescale tasks. Although most existing works assume an error-free scenarios in a master-worker setup, the link failures are common in current wired/wireless networks. In this paper, we consider the straggler problem in coded distributed computing with link failures, by modeling the links between the master node and worker nodes as packet erasure channels. When the master fails to detect the received signal, retransmission is required for each worker which increases the overall run-time to finish the task. We first investigate the expected overall run-time in this setting using an (n; k) maximum distance separable (MDS) code. We obtain the lower and upper bounds on the latency in closed-forms and give guidelines to design MDS code depending on the packet erasure probability. Finally, we consider a setup where the number of retransmissions is limited due to the bandwidth constraint. By formulating practical optimization problems related to latency, transmission bandwidth and probability of successful computation, we obtain achievable performance curves as a function of packet erasure probability.

cs.IT↗

arXiv2018

Bi-Directional Cooperative NOMA without Full CSIT

In this paper, we propose bi-directional cooperative non-orthogonal multiple access (NOMA). Compared to conventional NOMA, the main contributions of bi-directional cooperative NOMA can be explained in two directions: 1) The proposed NOMA system is still efficient when the channel gains of scheduled users are almost the same. 2) The proposed NOMA system operates well without accurate channel state information (CSI) at the base station (BS). In a two-user scenario, the closed-form ergodic capacity of bi-directional cooperative NOMA is derived and it is proven to be better than those of other techniques. Based on the ergodic capacity, the algorithms to find optimal power allocations maximizing user fairness and sum-rate are presented. Outage probability is also derived, and we show that bi-directional cooperative NOMA achieves a power gain over uni-directional cooperative NOMA and a diversity gain over non-cooperative NOMA and orthogonal multiple access (OMA). We finally extend the bi-directional cooperative NOMA to a multi-user model. The analysis of ergodic capacity and outage probability in two-user scenario is numerically verified. Also, simulation results show that bi-directional cooperative NOMA provdes better data rates than the existing NOMA schemes as well as OMA in multi-user scenario.

cs.IT↗

arXiv2017

NOMA in Distributed Antenna System for Max-Min Fairness and Max-Sum-Rate

Distributed antenna system (DAS) has been deployed for over a decade. DAS has advantages in capacity especially for the cell edge users, in both single-cell and multi-cell environments. In this paper, non-orthogonal multiple access (NOMA) is suggested in single-cell DAS to maximize user fairness and sumrate. Two transmission strategies are considered: NOMA with single selection transmission and NOMA with blanket transmission. In a two-user scenario, the center base station (BS) uses NOMA to serve both users and the remote radio units (RRUs) serve only the weak user based on the transmission scheme. The signals sent from the RRUs are not considered as interference at both user sides. At one side, the signals from the RRUs are used to detect the required data and at the other side, the signals are used to perform successive interference cancellation (SIC). The max-min performance is studied when instantaneous channel gain information (CGI) or channel distribution information (CDI) is known at the transmitter. A closed-form expression of the upper bound is derived for the data rate of each user with only CDI. In addition, the sum-rate is studied with a minimum rate constraint when instantaneous CGI known at the transmitter. The results show that NOMA with blanket transmission gives the best performance compared to other schemes; conventional-NOMA, conventional single selection scheme and joint-transmission (JT) NOMA. It is also shown that with less transmit power, NOMA with single selection transmission gives better performance than conventional-NOMA.

cs.IT↗