arXiv subjects

Haibo Yang

Publications and source records attributed to Haibo Yang.

47 records · Page 3Linked to original sources

arXiv2022

CHARLES: Channel-Quality-Adaptive Over-the-Air Federated Learning over Wireless Networks

Over-the-air federated learning (OTA-FL) has emerged as an efficient mechanism that exploits the superposition property of the wireless medium and performs model aggregation for federated learning in the air. OTA-FL is naturally sensitive to wireless channel fading, which could significantly diminish its learning accuracy. To address this challenge, in this paper, we propose an OTA-FL algorithm called CHARLES (channel-quality-aware over-the-air local estimating and scaling). Our CHARLES algorithm performs channel state information (CSI) estimation and adaptive scaling to mitigate the impacts of wireless channel fading. We establish the theoretical convergence rate performance of CHARLES and analyze the impacts of CSI error on the convergence of CHARLES. We show that the adaptive channel inversion scaling scheme in CHARLES is robust under imperfect CSI scenarios. We also demonstrate through numerical results that CHARLES outperforms existing OTA-FL algorithms with heterogeneous data under imperfect CSI.

cs.IT↗

arXiv2022

Over-the-Air Federated Learning with Joint Adaptive Computation and Power Control

This paper considers over-the-air federated learning (OTA-FL). OTA-FL exploits the superposition property of the wireless medium, and performs model aggregation over the air for free. Thus, it can greatly reduce the communication cost incurred in communicating model updates from the edge devices. In order to fully utilize this advantage while providing comparable learning performance to conventional federated learning that presumes model aggregation via noiseless channels, we consider the joint design of transmission scaling and the number of local iterations at each round, given the power constraint at each edge device. We first characterize the training error due to such channel noise in OTA-FL by establishing a fundamental lower bound for general functions with Lipschitz-continuous gradients. Then, by introducing an adaptive transceiver power scaling scheme, we propose an over-the-air federated learning algorithm with joint adaptive computation and power control (ACPC-OTA-FL). We provide the convergence analysis for ACPC-OTA-FL in training with non-convex objective functions and heterogeneous data. We show that the convergence rate of ACPC-OTA-FL matches that of FL with noise-free communications.

cs.LG↗

arXiv2022

Topmetal-M: a novel pixel sensor for compact tracking applications

The Topmetal-M is a large area pixel sensor (18 mm * 23 mm) prototype fabricated in a new 130 nm high-resistivity CMOS process in 2019. It contains 400 rows * 512 columns square pixels with the pitch of 40 μm. In Topmetal-M, a novel charge collection method combing the Monolithic Active Pixel Sensor (MAPS) and the Topmetal sensor has been proposed for the first time. Both the ionized charge deposited by the particle in the sensor and along the track over the sensor can be collected. The in-pixel circuit mainly consists of a low-noise charge sensitive amplifier to establish the signal for the energy reconstruction, and a discriminator with a Time-to-Amplitude Converter (TAC) for the Time of Arrival (TOA) measurement. With this mechanism, the trajectory, particle hit position, energy and arrival time of the particle can be measured. The analog signal from each pixel is accessible through time-shared multiplexing over the entire pixel array. This paper will discuss the design and preliminary test results of the Topmetal-M sensor.

physics.ins-det↗

arXiv2021

A remark on ill-posedness

Norm inflation implies certain discontinuous dependence of the solution on the initial value. The well-posedness of the mild solution means the existence and uniqueness of the fixed points of the corresponding integral equation. For ${\rm BMO}^{-1}$, Auscher-Dubois-Tchamitchian proved that Koch-Tataru's solution is stable. In this paper, we construct a non-Gauss flow function to show that, for classic Navier-Stokes equations, wellposedness and norm inflation may have no conflict and stability may have meaning different to $L^{\infty}(({\rm BMO}^{-1})^{n})$.

math.AP↗

arXiv2021

STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning

Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achieve the desired solution. This work addresses the above question and considers a class of stochastic algorithms where the WNs perform a few local updates before communication. We show that when both the WN's and the server's directions are chosen based on a stochastic momentum estimator, the algorithm requires $\tilde{\mathcal{O}}(ε^{-3/2})$ samples and $\tilde{\mathcal{O}}(ε^{-1})$ communication rounds to compute an $ε$-stationary solution. To the best of our knowledge, this is the first FL algorithm that achieves such {\it near-optimal} sample and communication complexities simultaneously. Further, we show that there is a trade-off curve between local update frequencies and local minibatch sizes, on which the above sample and communication complexities can be maintained. Finally, we show that for the classical FedAvg (a.k.a. Local SGD, which is a momentum-less special case of the STEM), a similar trade-off curve exists, albeit with worse sample and communication complexities. Our insights on this trade-off provides guidelines for choosing the four important design elements for FL algorithms, the update frequency, directions, and minibatch sizes to achieve the best performance.

cs.LG↗

arXiv2021

CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning

Federated learning (FL) is a prevailing distributed learning paradigm, where a large number of workers jointly learn a model without sharing their training data. However, high communication costs could arise in FL due to large-scale (deep) learning models and bandwidth-constrained connections. In this paper, we introduce a communication-efficient algorithmic framework called CFedAvg for FL with non-i.i.d. datasets, which works with general (biased or unbiased) SNR-constrained compressors. We analyze the convergence rate of CFedAvg for non-convex functions with constant and decaying learning rates. The CFedAvg algorithm can achieve an $\mathcal{O}(1 / \sqrt{mKT} + 1 / T)$ convergence rate with a constant learning rate, implying a linear speedup for convergence as the number of workers increases, where $K$ is the number of local steps, $T$ is the number of total communication rounds, and $m$ is the total worker number. This matches the convergence rate of distributed/federated learning without compression, thus achieving high communication efficiency while not sacrificing learning accuracy in FL. Furthermore, we extend CFedAvg to cases with heterogeneous local steps, which allows different workers to perform a different number of local steps to better adapt to their own circumstances. The interesting observation in general is that the noise/variance introduced by compressors does not affect the overall convergence rate order for non-i.i.d. FL. We verify the effectiveness of our CFedAvg algorithm on three datasets with two gradient compression schemes of different compression ratios.

cs.LG↗

arXiv2021

Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning

Federated learning (FL) is a distributed machine learning architecture that leverages a large number of workers to jointly learn a model with decentralized data. FL has received increasing attention in recent years thanks to its data privacy protection, communication efficiency and a linear speedup for convergence in training (i.e., convergence performance increases linearly with respect to the number of workers). However, existing studies on linear speedup for convergence are only limited to the assumptions of i.i.d. datasets across workers and/or full worker participation, both of which rarely hold in practice. So far, it remains an open question whether or not the linear speedup for convergence is achievable under non-i.i.d. datasets with partial worker participation in FL. In this paper, we show that the answer is affirmative. Specifically, we show that the federated averaging (FedAvg) algorithm (with two-sided learning rates) on non-i.i.d. datasets in non-convex settings achieves a convergence rate $\mathcal{O}(\frac{1}{\sqrt{mKT}} + \frac{1}{T})$ for full worker participation and a convergence rate $\mathcal{O}(\frac{\sqrt{K}}{\sqrt{nT}} + \frac{1}{T})$ for partial worker participation, where $K$ is the number of local steps, $T$ is the number of total communication rounds, $m$ is the total worker number and $n$ is the worker number in one communication round if for partial worker participation. Our results also reveal that the local steps in FL could help the convergence and show that the maximum number of local steps can be improved to $T/m$ in full worker participation. We conduct extensive experiments on MNIST and CIFAR-10 to verify our theoretical results.

cs.LG↗

arXiv2020

Ill-posedness of Naiver-Stokes equations and critical Besov-Morrey spaces

The blow up phenomenon in the first step of the classical Picard's scheme was proved in this paper. For certain initial spaces, Bourgain-Pavlović and Yoneda proved the ill-posedness of the Navier-Stokes equations by showing the norm inflation in certain solution spaces. But Chemin and Gallagher said the space $\dot{B}^{-1,\infty}_{\infty}$ seems to be optimal for some solution spaces best chosen. In this paper, we consider more general initial spaces than Bourgain-Pavlović and Yoneda did and establish ill-posedness result independent of the choice of solution space. Our result is a complement of the previous ill-posedness results on Navier-Stokes equations.

math-ph↗

arXiv2020

Development of readout electronics a novel beam monitoring system for ion research facility accelerator

This article presents the readout electronics of a novel beam monitoring system for ion research facility accelerator. The readout electronics are divided into Front-end Card (FEC) and Readout Control Unit (RCU). FEC uses Topmetal II minus to processes the energy of the hitting particles and convert it into a voltage signal. The main function of RCU is to digitize the analog output signal of FEC and format the raw data. On the other hand, the RCU also processes the control commands from the host and distributes the commands according to the mapping. The readout electronic has been characterized and calibrated in the laboratory, and have been installed with the detector. Implementation and testing of readout electronics have been discussed.

physics.ins-det↗

arXiv2019

Byzantine-Resilient Stochastic Gradient Descent for Distributed Learning: A Lipschitz-Inspired Coordinate-wise Median Approach

In this work, we consider the resilience of distributed algorithms based on stochastic gradient descent (SGD) in distributed learning with potentially Byzantine attackers, who could send arbitrary information to the parameter server to disrupt the training process. Toward this end, we propose a new Lipschitz-inspired coordinate-wise median approach (LICM-SGD) to mitigate Byzantine attacks. We show that our LICM-SGD algorithm can resist up to half of the workers being Byzantine attackers, while still converging almost surely to a stationary region in non-convex settings. Also, our LICM-SGD method does not require any information about the number of attackers and the Lipschitz constant, which makes it attractive for practical implementations. Moreover, our LICM-SGD method enjoys the optimal $O(md)$ computational time-complexity in the sense that the time-complexity is the same as that of the standard SGD under no attacks. We conduct extensive experiments to show that our LICM-SGD algorithm consistently outperforms existing methods in training multi-class logistic regression and convolutional neural networks with MNIST and CIFAR-10 datasets. In our experiments, LICM-SGD also achieves a much faster running time thanks to its low computational time-complexity.

cs.LG↗

arXiv2017

The Plastic Scintillator Detector at DAMPE

he DArk Matter Particle Explorer (DAMPE) is a general purposed satellite-borne high energy $γ-$ray and cosmic ray detector, and among the scientific objectives of DAMPE are the searches for the origin of cosmic rays and an understanding of Dark Matter particles. As one of the four detectors in DAMPE, the Plastic Scintillator Detector (PSD) plays an important role in the particle charge measurement and the photons/electrons separation. The PSD has 82 modules, each consists of a long organic plastic scintillator bar and two PMTs at both ends for readout, in two layers and covers an overall active area larger than 82 cm $\times$ 82 cm. It can identify the charge states for relativistic ions from H to Fe, and the detector efficiency for Z=1 particles can reach 0.9999. The PSD has been successfully launched with DAMPE on Dec. 17, 2015. In this paper, the design, the assembly, the qualification tests of the PSD and some of the performance measured on the ground have been described in detail.

astro-ph.IM↗