arXiv subjects

Qianrui Li

Publications and source records attributed to Qianrui Li.

6 recordsLinked to original sources

arXiv2024

Communication-Efficient Federated Learning via Regularized Sparse Random Networks

This work presents a new method for enhancing communication efficiency in stochastic Federated Learning that trains over-parameterized random networks. In this setting, a binary mask is optimized instead of the model weights, which are kept fixed. The mask characterizes a sparse sub-network that is able to generalize as good as a smaller target network. Importantly, sparse binary masks are exchanged rather than the floating point weights in traditional federated learning, reducing communication cost to at most 1 bit per parameter (Bpp). We show that previous state of the art stochastic methods fail to find sparse networks that can reduce the communication and storage overhead using consistent loss objectives. To address this, we propose adding a regularization term to local objectives that acts as a proxy of the transmitted masks entropy, therefore encouraging sparser solutions by eliminating redundant features across sub-networks. Extensive empirical experiments demonstrate significant improvements in communication and memory efficiency of up to five magnitudes compared to the literature, with minimal performance degradation in validation accuracy in some instances

cs.LG↗

arXiv2023

User-Centric Federated Learning: Trading off Wireless Resources for Personalization

Statistical heterogeneity across clients in a Federated Learning (FL) system increases the algorithm convergence time and reduces the generalization performance, resulting in a large communication overhead in return for a poor model. To tackle the above problems without violating the privacy constraints that FL imposes, personalized FL methods have to couple statistically similar clients without directly accessing their data in order to guarantee a privacy-preserving transfer. In this work, we design user-centric aggregation rules at the parameter server (PS) that are based on readily available gradient information and are capable of producing personalized models for each FL client. The proposed aggregation rules are inspired by an upper bound of the weighted aggregate empirical risk minimizer. Secondly, we derive a communication-efficient variant based on user clustering which greatly enhances its applicability to communication-constrained systems. Our algorithm outperforms popular personalized FL baselines in terms of average accuracy, worst node performance, and training communication overhead.

cs.LG↗

arXiv2022

UAV-Aided Multi-Community Federated Learning

In this work, we investigate the problem of an online trajectory design for an Unmanned Aerial Vehicle (UAV) in a Federated Learning (FL) setting where several different communities exist, each defined by a unique task to be learned. In this setting, spatially distributed devices belonging to each community collaboratively contribute towards training their community model via wireless links provided by the UAV. Accordingly, the UAV acts as a mobile orchestrator coordinating the transmissions and the learning schedule among the devices in each community, intending to accelerate the learning process of all tasks. We propose a heuristic metric as a proxy for the training performance of the different tasks. Capitalizing on this metric, a surrogate objective is defined which enables us to jointly optimize the UAV trajectory and the scheduling of the devices by employing convex optimization techniques and graph theory. The simulations illustrate the out-performance of our solution when compared to other handpicked static and mobile UAV deployment baselines.

cs.IT↗

arXiv2021

User-Centric Federated Learning

Data heterogeneity across participating devices poses one of the main challenges in federated learning as it has been shown to greatly hamper its convergence time and generalization capabilities. In this work, we address this limitation by enabling personalization using multiple user-centric aggregation rules at the parameter server. Our approach potentially produces a personalized model for each user at the cost of some extra downlink communication overhead. To strike a trade-off between personalization and communication efficiency, we propose a broadcast protocol that limits the number of personalized streams while retaining the essential advantages of our learning scheme. Through simulation results, our approach is shown to enjoy higher personalization capabilities, faster convergence, and better communication efficiency compared to other competing baseline solutions.

cs.LG↗

arXiv2017

Cooperative Channel Estimation for Coordinated Transmission with Limited Backhaul

Obtaining accurate global channel state information (CSI) at multiple transmitter devices is critical to the performance of many coordinated transmission schemes. Practical CSI local feedback often leads to noisy and partial CSI estimates at each transmitter. With rate-limited bi-directional backhaul, transmitters have the opportunity to exchange few CSI-related bits to establish global channel state information at transmitter (CSIT). This work investigates possible strategies towards this goal. We propose a novel decentralized algorithm that produces minimum mean square error (MMSE)-optimal global channel estimates at each device from combining local feedback and information exchanged through backhauls. The method adapts to arbitrary initial information topologies and feedback noise statistics and can do that with a combination of closed-form and convex approaches. Simulations for coordinated multi-point (CoMP) transmission systems with two or three transmitters exhibit the advantage of the proposed algorithm over conventional CSI exchange mechanisms when the coordination backhauls are limited.

cs.IT↗

arXiv2017

Robust Regularized ZF in Cooperative Broadcast Channel under Distributed CSIT

In this work, we consider the sum rate performance of joint processing coordinated multi-point transmission network (JP-CoMP, a.k.a Network MIMO) in a so-called distributed channel state information (D-CSI) setting. In the D-CSI setting, the various transmitters (TXs) acquire a local, TX-dependent, estimate of the global multi-user channel state matrix obtained via terminal feedback and limited backhauling. The CSI noise across TXs can be independent or correlated, so as to reflect the degree to which TXs can exchange information over the backhaul, hence allowing to model a range of situations bridging fully distributed and fully centralized CSI settings. In this context we aim to study the price of CSI distributiveness in terms of sum rate at finite SNR when compared with conventional centralized scenarios. We consider the family of JP-CoMP precoders known as regularized zero-forcing (RZF). We conduct our study in the large scale antenna regime, as it is currently envisioned to be used in real 5G deployments. It is then possible to obtain accurate approximations on so-called deterministic equivalents of the signal to interference and noise ratios. Guided by the obtained deterministic equivalents, we propose an approach to derive a RZF scheme that is robust to the distributed aspect of the CSI, whereby the key idea lies in the optimization of a TX-dependent power level and regularization factor. Our analysis confirms the improved robustness of the proposed scheme with respect to CSI inconsistency at different TXs, even with moderate number of antennas and receivers (RXs).

cs.IT↗