arXiv subjects

Sheng Sun

Publications and source records attributed to Sheng Sun.

At least 55 records · Page 3Linked to original sources

arXiv2024

Knowledge Distillation in Federated Edge Learning: A Survey

The increasing demand for intelligent services and privacy protection of mobile and Internet of Things (IoT) devices motivates the wide application of Federated Edge Learning (FEL), in which devices collaboratively train on-device Machine Learning (ML) models without sharing their private data. Limited by device hardware, diverse user behaviors and network infrastructure, the algorithm design of FEL faces challenges related to resources, personalization and network environments. Fortunately, Knowledge Distillation (KD) has been leveraged as an important technique to tackle the above challenges in FEL. In this paper, we investigate the works that KD applies to FEL, discuss the limitations and open problems of existing KD-based FEL approaches, and provide guidance for their real deployment.

cs.LG↗

arXiv2024

Spin and Orbital Angular Momenta of Electromagnetic Waves: From Classical to Quantum Forms

Angular momenta of electromagnetic waves are important both in concepts and applications. In this work, we systematically discuss two types of angular momenta, i.e., spin angular momentum and orbital angular momentum in various cases, e.g., with source and without source, in classical and quantum forms. Numerical results demonstrating how to extract the topological charge of a classical vortex beam by spectral method are also presented.

physics.optics↗

arXiv2024

Breaking the Degrees-of-Freedom Limit of Holographic MIMO Communications: A 3-D Antenna Array Topology

The performance of holographic multiple-input multiple-output (MIMO) communications, employing two-dimensional (2-D) planar antenna arrays, is typically compromised by finite degrees-of-freedom (DOF) stemming from limited array size. The DOF constraint becomes significant when the element spacing approaches approximately half a wavelength, thereby restricting the overall performance of MIMO systems. To break this inherent limitation, we propose a novel three-dimensional (3-D) antenna array that strategically explores the untapped vertical dimension. We investigate the performance of MIMO systems utilizing 3-D arrays across different multi-path scenarios, encompassing Rayleigh channels with varying angular spreads and the 3rd generation partnership project (3GPP) channels. We subsequently showcase the advantages of these 3-D arrays over their 2-D counterparts with the same aperture sizes. As a proof of concept, a practical dipole-based 3-D array, facilitated by an electromagnetic band-gap (EBG) reflecting surface, is conceived, constructed, and evaluated. The experimental results align closely with full-wave simulations, and channel simulations substantiate that the DOF and capacity constraints of traditional holographic MIMO systems can be surpassed by adopting such a 3-D array configuration.

cs.IT↗

arXiv2024

FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence

Edge Intelligence (EI) allows Artificial Intelligence (AI) applications to run at the edge, where data analysis and decision-making can be performed in real-time and close to data sources. To protect data privacy and unify data silos among end devices in EI, Federated Learning (FL) is proposed for collaborative training of shared AI models across devices without compromising data privacy. However, the prevailing FL approaches cannot guarantee model generalization and adaptation on heterogeneous clients. Recently, Personalized Federated Learning (PFL) has drawn growing awareness in EI, as it enables a productive balance between local-specific training requirements inherent in devices and global-generalized optimization objectives for satisfactory performance. However, most existing PFL methods are based on the Parameters Interaction-based Architecture (PIA) represented by FedAvg, which causes unaffordable communication burdens due to large-scale parameters transmission between devices and the edge server. In contrast, Logits Interaction-based Architecture (LIA) allows to update model parameters with logits transfer and gains the advantages of communication lightweight and heterogeneous on-device model allowance compared to PIA. Nevertheless, previous LIA methods attempt to achieve satisfactory performance either relying on unrealistic public datasets or increasing communication overhead for additional information transmission other than logits. To tackle this dilemma, we propose a knowledge cache-driven PFL architecture, named FedCache, which reserves a knowledge cache on the server for fetching personalized knowledge from the samples with similar hashes to each given on-device sample. During the training phase, ensemble distillation is applied to on-device models for constructive optimization with personalized knowledge transferred from the server-side knowledge cache.

cs.DC↗

arXiv2024

Logits Poisoning Attack in Federated Distillation

Federated Distillation (FD) is a novel and promising distributed machine learning paradigm, where knowledge distillation is leveraged to facilitate a more efficient and flexible cross-device knowledge transfer in federated learning. By optimizing local models with knowledge distillation, FD circumvents the necessity of uploading large-scale model parameters to the central server, simultaneously preserving the raw data on local clients. Despite the growing popularity of FD, there is a noticeable gap in previous works concerning the exploration of poisoning attacks within this framework. This can lead to a scant understanding of the vulnerabilities to potential adversarial actions. To this end, we introduce FDLA, a poisoning attack method tailored for FD. FDLA manipulates logit communications in FD, aiming to significantly degrade model performance on clients through misleading the discrimination of private samples. Through extensive simulation experiments across a variety of datasets, attack scenarios, and FD configurations, we demonstrate that LPA effectively compromises client model accuracy, outperforming established baseline algorithms in this regard. Our findings underscore the critical need for robust defense mechanisms in FD settings to mitigate such adversarial threats.

cs.LG↗

arXiv2023

Non-iterative Methods in Inhomogeneous Background Inverse Scattering Imaging Problem Assisted by Swin Transformer Network

A deep learning-assisted inversion method is proposed to solve the inhomogeneous background imaging problem. Three non-iterative methods, namely the distorted-Born (DB) major current coefficients method, the DB modified Born approximation method, and the DB connection method, are introduced to address the inhomogeneous background inverse scattering problem. These methods retain the multiple scattering information by utilizing the major current obtained through singular value decomposition of the Green's function and the scattered field, without resourcing to optimization techniques. As a result, the proposed methods offer improved reconstruction resolution and accuracy for unknown objects embedded in inhomogeneous backgrounds, surpassing the backpropagation scheme (BPS) and Born approximation (BA) method that disregard the multiple scattering effect. To further enhance the resolution and accuracy of the reconstruction, a Shifted-Window (Swin) transformer network is employed for capturing super-resolution information in the images. The attention mechanism incorporated in the shifted window facilitates global interactions between objects, thereby enhancing the performance of the inhomogeneous background imaging algorithm while reducing computational complexity. Moreover, an adaptive training method is proposed to enhance the generalization ability of the network. The effectiveness of the proposed methods is demonstrated through both synthetic data and experimental data. Notably, super-resolution imaging is achieved with quasi real-time speed, indicating promising application potential for the proposed algorithms.

physics.app-ph↗

arXiv2023

Improving Communication Efficiency of Federated Distillation via Accumulating Local Updates

As an emerging federated learning paradigm, federated distillation enables communication-efficient model training by transmitting only small-scale knowledge during the learning process. To further improve the communication efficiency of federated distillation, we propose a novel technique, ALU, which accumulates multiple rounds of local updates before transferring the knowledge to the central server. ALU drastically decreases the frequency of communication in federated distillation, thereby significantly reducing the communication overhead during the training process. Empirical experiments demonstrate the substantial effect of ALU in improving the communication efficiency of federated distillation.

cs.LG↗

arXiv2023

Federated Skewed Label Learning with Logits Fusion

Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. Data heterogeneity is a critical challenge in realistic FL settings, as it causes significant performance deterioration due to discrepancies in optimization among local models. In this work, we focus on label distribution skew, a common scenario in data heterogeneity, where the data label categories are imbalanced on each client. To address this issue, we propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Specifically, we introduce an extra private weak learner on the client side, which forms an ensemble model with the local model. By fusing the logits of the two models, the private weak learner can capture the variance of different data, regardless of their category. Therefore, the optimization direction of local models can be improved by increasing the penalty for misclassifying minority classes and reducing the attention to majority classes, resulting in a better global model. Extensive experiments show that our method can gain 13\% higher average accuracy compared with state-of-the-art methods.

cs.LG↗

arXiv2023

Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation

Federated learning (FL) is a privacy-preserving machine learning paradigm in which the server periodically aggregates local model parameters from clients without assembling their private data. Constrained communication and personalization requirements pose severe challenges to FL. Federated distillation (FD) is proposed to simultaneously address the above two problems, which exchanges knowledge between the server and clients, supporting heterogeneous local models while significantly reducing communication overhead. However, most existing FD methods require a proxy dataset, which is often unavailable in reality. A few recent proxy-data-free FD approaches can eliminate the need for additional public data, but suffer from remarkable discrepancy among local knowledge due to client-side model heterogeneity, leading to ambiguous representation on the server and inevitable accuracy degradation. To tackle this issue, we propose a proxy-data-free FD algorithm based on distributed knowledge congruence (FedDKC). FedDKC leverages well-designed refinement strategies to narrow local knowledge differences into an acceptable upper bound, so as to mitigate the negative effects of knowledge incongruence. Specifically, from perspectives of peak probability and Shannon entropy of local knowledge, we design kernel-based knowledge refinement (KKR) and searching-based knowledge refinement (SKR) respectively, and theoretically guarantee that the refined-local knowledge can satisfy an approximately-similar distribution and be regarded as congruent. Extensive experiments conducted on three common datasets demonstrate that our proposed FedDKC significantly outperforms the state-of-the-art on various heterogeneous settings while evidently improving the convergence speed.

cs.LG↗

arXiv2023

Unraveling the Complexity of Metal Ion Dissolution: Insights from Hybrid First-Principles/Continuum Calculations

The study of ion dissolution from metal surfaces has a long-standing history, wherein the gradual dissolution of solute atoms with increasing electrode potential, leading to their existence as ions in the electrolyte with integer charges, is well-known. However, our present work reveals a more intricate and nuanced physical perspective based on comprehensive first-principles/continuum calculations. We investigate the dissolution and deposition processes of 22 metal elements across a range of applied electrode potentials, unveiling diverse dissolution models. By analyzing the energy profiles and valence states of solute atoms as a function of the distance between the solute atom and metal surface, we identify three distinct dissolution models for different metals. Firstly, solute atoms exhibit an integer valence state following an integer-valence jump, aligning with classical understandings. Secondly, solute atoms attain an eventual integer valence, yet their valence state increases in a non-integer manner during dissolution. Lastly, we observe solute atoms exhibiting a non-integer valence state, challenging classical understandings. Furthermore, we propose a theoretical criterion for determining the selection of ion valence during electrode dissolution under applied potential. These findings not only contribute to a deeper understanding of the dissolution process but also offer valuable insights into the complex dynamics governing metal ion dissolution at the atomic level. Such knowledge has the potential to advance the design of more efficient electrochemical systems and open new avenues for controlling dissolution processes in various applications.

cond-mat.mtrl-sci↗

arXiv2023

FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout

Federated Learning (FL) emerges as a distributed machine learning paradigm without end-user data transmission, effectively avoiding privacy leakage. Participating devices in FL are usually bandwidth-constrained, and the uplink is much slower than the downlink in wireless networks, which causes a severe uplink communication bottleneck. A prominent direction to alleviate this problem is federated dropout, which drops fractional weights of local models. However, existing federated dropout studies focus on random or ordered dropout and lack theoretical support, resulting in unguaranteed performance. In this paper, we propose Federated learning with Bayesian Inference-based Adaptive Dropout (FedBIAD), which regards weight rows of local models as probability distributions and adaptively drops partial weight rows based on importance indicators correlated with the trend of local training loss. By applying FedBIAD, each client adaptively selects a high-quality dropping pattern with accurate approximations and only transmits parameters of non-dropped weight rows to mitigate uplink costs while improving accuracy. Theoretical analysis demonstrates that the convergence rate of the average generalization error of FedBIAD is minimax optimal up to a squared logarithmic factor. Extensive experiments on image classification and next-word prediction show that compared with status quo approaches, FedBIAD provides 2x uplink reduction with an accuracy increase of up to 2.41% even on non-Independent and Identically Distributed (non-IID) data, which brings up to 72% decrease in training time.

cs.DC↗

arXiv2023

FedTrip: A Resource-Efficient Federated Learning Method with Triplet Regularization

In the federated learning scenario, geographically distributed clients collaboratively train a global model. Data heterogeneity among clients significantly results in inconsistent model updates, which evidently slow down model convergence. To alleviate this issue, many methods employ regularization terms to narrow the discrepancy between client-side local models and the server-side global model. However, these methods impose limitations on the ability to explore superior local models and ignore the valuable information in historical models. Besides, although the up-to-date representation method simultaneously concerns the global and historical local models, it suffers from unbearable computation cost. To accelerate convergence with low resource consumption, we innovatively propose a model regularization method named FedTrip, which is designed to restrict global-local divergence and decrease current-historical correlation for alleviating the negative effects derived from data heterogeneity. FedTrip helps the current local model to be close to the global model while keeping away from historical local models, which contributes to guaranteeing the consistency of local updates among clients and efficiently exploring superior local models with negligible additional computation cost on attaching operations. Empirically, we demonstrate the superiority of FedTrip via extensive evaluations. To achieve the target accuracy, FedTrip outperforms the state-of-the-art baselines in terms of significantly reducing the total overhead of client-server communication and local computation.

cs.DC↗

arXiv2023

Resource-aware Probability-based Collaborative Odor Source Localization Using Multiple UAVs

Benefitting from UAVs' characteristics of flexible deployment and controllable movement in 3D space, odor source localization with multiple UAVs has been a hot research area in recent years. Considering the limited resources and insufficient battery capacities of UAVs, it is necessary to fast locate the odor source with low-complexity computation and minimal interaction under complicated environmental states. To this end, we propose a multi-UAV collaboration based odor source localization (\textit{MUC-OSL}) method, where source estimation and UAV navigation are iteratively performed, aiming to accelerate the searching process and reduce the resource consumption of UAVs. Specifically, in the source estimation phase, we present a collaborative particle filter algorithm on the basis of UAVs' cognitive difference and Gaussian fitting to improve source estimation accuracy. In the following navigation phase, an adaptive path planning algorithm is designed based on Partially Observable Markov Decision Process (POMDP) to distributedly determine the subsequent flying direction and moving steps of each UAV. The results of experiments conducted on two simulation platforms demonstrate that \textit{MUC-OSL} outperforms existing efforts in terms of mean search time and success rate, and effectively reduces the resource consumption of UAVs.

cs.MA↗

arXiv2023

Online Spatio-Temporal Correlation-Based Federated Learning for Traffic Flow Forecasting

Traffic flow forecasting (TFF) is of great importance to the construction of Intelligent Transportation Systems (ITS). To mitigate communication burden and tackle with the problem of privacy leakage aroused by centralized forecasting methods, Federated Learning (FL) has been applied to TFF. However, existing FL-based approaches employ batch learning manner, which makes the pre-trained models inapplicable to subsequent traffic data, thus exhibiting subpar prediction performance. In this paper, we perform the first study of forecasting traffic flow adopting Online Learning (OL) manner in FL framework and then propose a novel prediction method named Online Spatio-Temporal Correlation-based Federated Learning (FedOSTC), aiming to guarantee performance gains regardless of traffic fluctuation. Specifically, clients employ Gated Recurrent Unit (GRU)-based encoders to obtain the internal temporal patterns inside traffic data sequences. Then, the central server evaluates spatial correlation among clients via Graph Attention Network (GAT), catering to the dynamic changes of spatial closeness caused by traffic fluctuation. Furthermore, to improve the generalization of the global model for upcoming traffic data, a period-aware aggregation mechanism is proposed to aggregate the local models which are optimized using Online Gradient Descent (OGD) algorithm at clients. We perform comprehensive experiments on two real-world datasets to validate the efficiency and effectiveness of our proposed method and the numerical results demonstrate the superiority of FedOSTC.

cs.LG↗

arXiv2022

Towards 5G Zero Trusted Air Interface Architecture

5G is destined to be supporting large deployment of Industrial IoT (IIoT) with the characteristics of ultra-high densification and low latency. 5G utilizes a more intelligent architecture, with Radio Access Networks (RANs) no longer constrained by base station proximity or proprietary infrastructure. The 3rd Generation Partnership Project (3GPP) covers telecommunication technologies including RAN, core transport networks and service capabilities. Open RAN Alliance (O-RAN) aims to define implementation and deployment architectures, focusing on open-source interfaces and functional units to further reduce the cost and complexity. O-RAN based 5G networks could use components from different hardware and software vendors, promoting vendor diversity, interchangeability and 5G supply chain resiliency. Both 3GPP and O-RAN 5G have to manage the security and privacy challenges that arose from the deployment. Many existing research studies have addressed the threats and vulnerabilities within each system. 5G also has the overwhelming challenges in compliance with privacy regulations and requirements which mandate the user identifiable information need to be protected. In this paper, we look into the 3GPP and O-RAN 5G security and privacy designs and the identified threats and vulnerabilities. We also discuss how to extend the Zero Trust Model to provide advanced protection over 5G air interfaces and network components.

cs.CR↗

arXiv2022

Electromagnetic Effective-Degree-of-Freedom Limit of a MIMO System in 2-D Inhomogeneous Environment

Compared with a single-input-single-output (SISO) wireless communication system, the benefit of multiple-input-multiple-output (MIMO) technology originates from its extra degree of freedom (DOF), also referred as scattering channels or spatial electromagnetic (EM) modes, brought by spatial multiplexing. When the physical sizes of transmitting and receiving arrays are fixed, and there are sufficient antennas (typically with half-wavelength spacings), the DOF limit is only dependent on the propagating environment. Analytical methods can be used to estimate this limit in free space, and some approximate models are adopted in stochastic environments, such as Clarke's model and Ray-tracing methods. However, this DOF limit in an certain inhomogeneous environment has not been well discussed with rigorous full-wave numerical methods. In this work, volume integral equation (VIE) is implemented for investigating the limit of MIMO effective degree of freedom (EDOF) in three representative two-dimensional (2-D) inhomogeneous environments. Moreover, we clarify the relation between the performance of a MIMO system and the scattering characteristics of its propagating environment.

cs.IT↗

arXiv2022

Towards Federated Learning against Noisy Labels via Local Self-Regularization

Federated learning (FL) aims to learn joint knowledge from a large scale of decentralized devices with labeled data in a privacy-preserving manner. However, since high-quality labeled data require expensive human intelligence and efforts, data with incorrect labels (called noisy labels) are ubiquitous in reality, which inevitably cause performance degradation. Although a lot of methods are proposed to directly deal with noisy labels, these methods either require excessive computation overhead or violate the privacy protection principle of FL. To this end, we focus on this issue in FL with the purpose of alleviating performance degradation yielded by noisy labels meanwhile guaranteeing data privacy. Specifically, we propose a Local Self-Regularization method, which effectively regularizes the local training process via implicitly hindering the model from memorizing noisy labels and explicitly narrowing the model output discrepancy between original and augmented instances using self distillation. Experimental results demonstrate that our proposed method can achieve notable resistance against noisy labels in various noise levels on three benchmark datasets. In addition, we integrate our method with existing state-of-the-arts and achieve superior performance on the real-world dataset Clothing1M. The code is available at https://github.com/Sprinter1999/FedLSR.

cs.LG↗

arXiv2022

Approaching the Fundamental Limit of Orbital Angular Momentum Multiplexing Through a Hologram Metasurface

Establishing and approaching the fundamental limit of orbital angular momentum (OAM) multiplexing are necessary and increasingly urgent for current multiple-input multiple-output research. In this work, we elaborate the fundamental limit in terms of independent scattering channels (or degrees of freedom of scattered fields) through angular-spectral analysis, in conjunction with a rigorous Green function method. The scattering channel limit is universal for arbitrary spatial mode multiplexing, which is launched by a planar electromagnetic device, such as antenna, metasurface, etc, with a predefined physical size. As a proof of concept, we demonstrate both theoretically and experimentally the limit by a metasurface hologram that transforms orthogonal OAM modes to plane-wave modes scattered at critically separated angular-spectral regions. Particularly, a minimax optimization algorithm is applied to suppress angular spectrum aliasing, achieving good performances in both full-wave simulation and experimental measurement at microwave frequencies. This work offers a theoretical upper bound and corresponding approach route for engineering designs of OAM multiplexing.

physics.app-ph↗