arXiv subjects

John Paul Walters

Publications and source records attributed to John Paul Walters.

3 recordsLinked to original sources

arXiv2025

FedPaI: Achieving Extreme Sparsity in Federated Learning via Pruning at Initialization

Federated Learning (FL) enables distributed training on edge devices but faces significant challenges due to resource constraints in edge environments, impacting both communication and computational efficiency. Existing iterative pruning techniques improve communication efficiency but are limited by their centralized design, which struggles with FL's decentralized and data-imbalanced nature, resulting in suboptimal sparsity levels. To address these issues, we propose FedPaI, a novel efficient FL framework that leverages Pruning at Initialization (PaI) to achieve extreme sparsity. FedPaI identifies optimal sparse connections at an early stage, maximizing model capacity and significantly reducing communication and computation overhead by fixing sparsity patterns at the start of training. To adapt to diverse hardware and software environments, FedPaI supports both structured and unstructured pruning. Additionally, we introduce personalized client-side pruning mechanisms for improved learning capacity and sparsity-aware server-side aggregation for enhanced efficiency. Experimental results demonstrate that FedPaI consistently outperforms existing efficient FL that applies conventional iterative pruning with significant leading in efficiency and model accuracy. For the first time, our proposed FedPaI achieves an extreme sparsity level of up to 98% without compromising the model accuracy compared to unpruned baselines, even under challenging non-IID settings. By employing our FedPaI with joint optimization of model learning capacity and sparsity, FL applications can benefit from faster convergence and accelerate the training by 6.4 to 7.9 times.

cs.LG↗

arXiv2022

QuantPipe: Applying Adaptive Post-Training Quantization for Distributed Transformer Pipelines in Dynamic Edge Environments

Pipeline parallelism has achieved great success in deploying large-scale transformer models in cloud environments, but has received less attention in edge environments. Unlike in cloud scenarios with high-speed and stable network interconnects, dynamic bandwidth in edge systems can degrade distributed pipeline performance. We address this issue with QuantPipe, a communication-efficient distributed edge system that introduces post-training quantization (PTQ) to compress the communicated tensors. QuantPipe uses adaptive PTQ to change bitwidths in response to bandwidth dynamics, maintaining transformer pipeline performance while incurring limited inference accuracy loss. We further improve the accuracy with a directed-search analytical clipping for integer quantization method (DS-ACIQ), which bridges the gap between estimated and real data distributions. Experimental results show that QuantPipe adapts to dynamic bandwidth to maintain pipeline performance while achieving a practical model accuracy using a wide range of quantization bitwidths, e.g., improving accuracy under 2-bit quantization by 15.85\% on ImageNet compared to naive quantization.

cs.DC↗

arXiv2021

RDAnet: A Deep Learning Based Approach for Synthetic Aperture Radar Image Formation

Synthetic Aperture Radar (SAR) imaging systems operate by emitting radar signals from a moving object, such as a satellite, towards the target of interest. Reflected radar echoes are received and later used by image formation algorithms to form a SAR image. There is great interest in using SAR images in computer vision tasks such as classification or automatic target recognition. Today, however, SAR applications consist of multiple operations: image formation followed by image processing. In this work, we train a deep neural network that performs both the image formation and image processing tasks, integrating the SAR processing pipeline. Results show that our integrated pipeline can output accurately classified SAR imagery with image quality comparable to those formed using a traditional algorithm. We believe that this work is the first demonstration of an integrated neural network based SAR processing pipeline using real data.

eess.IV↗