arXiv subjects

Jingjing Zhang

Publications and source records attributed to Jingjing Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2024

ABS: Adaptive Bounded Staleness Converges Faster and Communicates Less

Wall-clock convergence time and communication rounds are critical performance metrics in distributed learning with parameter-server setting. While synchronous methods converge fast but are not robust to stragglers; and asynchronous ones can reduce the wall-clock time per round but suffers from degraded convergence rate due to the staleness of gradients, it is natural to combine the two methods to achieve a balance. In this work, we develop a novel asynchronous strategy that leverages the advantages of both synchronous methods and asynchronous ones, named adaptive bounded staleness (ABS). The key enablers of ABS are two-fold. First, the number of workers that the PS waits for per round for gradient aggregation is adaptively selected to strike a straggling-staleness balance. Second, the workers with relatively high staleness are required to start a new round of computation to alleviate the negative effect of staleness. Simulation results are provided to demonstrate the superiority of ABS over state-of-the-art schemes in terms of wall-clock time and communication rounds.

cs.DC↗

arXiv2023

Highly Anisotropic Elastic Properties of Suspended Black Arsenic Nanoribbons

Anisotropy, as an exotic degree of freedom, enables us to discover the emergent two-dimensional (2D) layered nanomaterials with low in-plane symmetry and to explore their outstanding properties and promising applications. 2D black arsenic (b-As) with puckered structure has garnered increasing attention these years owing to its extreme anisotropy with respect to the electrical, thermal, and optical properties. However, the investigation on mechanical properties of 2D b-As is still lacking, despite much effort on theoretical simulations. Herein, we report the highly anisotropic elastic properties of suspended b-As nanoribbons via atomic force microscope-based nanoindentation. It was found that the extracted Young's modulus of b-As nanoribbons exhibits remarkable anisotropy, which approximates to 72.2 +- 5.4 and 44.3 +- 1.4 GPa along zigzag and armchair directions, respectively. The anisotropic ratio reaches up to ~ 1.6. We expect that these results could lay a solid foundation for the potential applications of 2D anisotropic nanomaterials in the next-generation nanomechanics and optoelectronics.

cond-mat.mtrl-sci↗

arXiv2023

Experimental observation of highly anisotropic elastic properties of two-dimensional black arsenic

Anisotropic two-dimensional layered materials with low-symmetric lattices have attracted increasing attention due to their unique orientation-dependent mechanical properties. Black arsenic (b-As), with the puckered structure, exhibits extreme in-plane anisotropy in optical, electrical and thermal properties. However, experimental research on mechanical properties of b-As is very rare, although theoretical calculations predicted the exotic elastic properties of b-As, such as anisotropic Young's modulus and negative Poisson's ratio. Herein, experimental observations on highly anisotropic elastic properties of b-As were demonstrated using our developed in situ tensile straining setup based on the effective microelectromechanical system. The cyclic and repeatable load-displacement curves proved that Young's modulus along zigzag direction was ~1.6 times greater than that along armchair direction, while the anisotropic ratio of ultimate strain reached ~2.5, attributed to hinge structure in armchair direction. This study could provide significant insights to design novel anisotropic materials and explore their potential applications in nanomechanics and nanodevices.

cond-mat.mtrl-sci↗

arXiv2023

Intralayer Negative Poisson's Ratio in Two-Dimensional Black Arsenic by Strain Engineering

Negative Poisson's ratio as the anomalous characteristic generally exists in artificial architectures, such as re-entrant and honeycomb structures. The structures with negative Poisson's ratio have attracted intensive attention due to their unique auxetic effect and many promising applications in shear resistant and energy absorption fields. However, experimental observation of negative Poisson's ratio in natural materials barely happened, although various two-dimensional layered materials are predicted in theory. Herein, we report the anisotropic Raman response and the intrinsic intralayer negative Poisson's ratio of two-dimensional natural black arsenic (b-As) via strain engineering strategy. The results were evident by the detailed Raman spectrum of b-As under uniaxial strain together with density functional theory calculations. It is found that b-As was softer along the armchair than zigzag direction. The anisotropic mechanical features and van der Waals interactions play essential roles in strain-dependent Raman shifts and negative Poisson's ratio in the natural b-As along zigzag direction. This work may shed a light on the mechanical properties and potential applications of two-dimensional puckered materials.

cond-mat.mtrl-sci↗

arXiv2023

DRAG: Divergence-based Adaptive Aggregation in Federated learning on Non-IID Data

Local stochastic gradient descent (SGD) is a fundamental approach in achieving communication efficiency in Federated Learning (FL) by allowing individual workers to perform local updates. However, the presence of heterogeneous data distributions across working nodes causes each worker to update its local model towards a local optimum, leading to the phenomenon known as ``client-drift" and resulting in slowed convergence. To address this issue, previous works have explored methods that either introduce communication overhead or suffer from unsteady performance. In this work, we introduce a novel metric called ``degree of divergence," quantifying the angle between the local gradient and the global reference direction. Leveraging this metric, we propose the divergence-based adaptive aggregation (DRAG) algorithm, which dynamically ``drags" the received local updates toward the reference direction in each round without requiring extra communication overhead. Furthermore, we establish a rigorous convergence analysis for DRAG, proving its ability to achieve a sublinear convergence rate. Compelling experimental results are presented to illustrate DRAG's superior performance compared to state-of-the-art algorithms in effectively managing the client-drift phenomenon. Additionally, DRAG exhibits remarkable resilience against certain Byzantine attacks. By securely sharing a small sample of the client's data with the FL server, DRAG effectively counters these attacks, as demonstrated through comprehensive experiments.

cs.LG↗

arXiv2023

Adaptive Multi-Armed Bandit Learning for Task Offloading in Edge Computing

The widespread adoption of edge computing has emerged as a prominent trend for alleviating task processing delays and reducing energy consumption. However, the dynamic nature of network conditions and the varying computation capacities of edge servers (ESs) can introduce disparities between computation loads and available computing resources in edge computing networks, potentially leading to inadequate service quality. To address this challenge, this paper investigates a practical scenario characterized by dynamic task offloading. Initially, we examine traditional Multi-armed Bandit (MAB) algorithms, namely the $\varepsilon$-greedy algorithm and the UCB1-based algorithm. However, both algorithms exhibit certain weaknesses in effectively addressing the tidal data traffic patterns. Consequently, based on MAB, we propose an adaptive task offloading algorithm (ATOA) that overcomes these limitations. By conducting extensive simulations, we demonstrate the superiority of our ATOA solution in reducing task processing latency compared to conventional MAB methods. This substantiates the effectiveness of our approach in enhancing the performance of edge computing networks and improving overall service quality.

cs.NI↗

arXiv2023

Frequency-dependent Switching Control for Disturbance Attenuation of Linear Systems

The generalized Kalman-Yakubovich-Popov lemma as established by Iwasaki and Hara in 2005 marks a milestone in the analysis and synthesis of linear systems from a finite-frequency perspective. Given a pre-specified frequency band, it allows us to produce passive controllers with excellent in-band disturbance attenuation performance at the expense of some of the out-of-band performance. This paper focuses on control design of linear systems in the presence of disturbances with non-strictly or non-stationary limited frequency spectrum. We first propose a class of frequency-dependent excited energy functions (FD-EEF) as well as frequency-dependent excited power functions (FD-EPF), which possess a desirable frequency-selectiveness property with regard to the in-band and out-of-band excited energy as well as excited power of the system. Based upon a group of frequency-selective passive controllers, we then develop a frequency-dependent switching control (FDSC) scheme that selects the most appropriate controller at runtime. We show that our FDSC scheme is capable to approximate the solid in-band performance while maintaining acceptable out-of-band performance with regard to global time horizons as well as localized time horizons. The method is illustrated by a commonly used benchmark model.

eess.SY↗

arXiv2023

STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization

Synchronous local stochastic gradient descent (local SGD) suffers from some workers being idle and random delays due to slow and straggling workers, as it waits for the workers to complete the same amount of local updates. In this paper, to mitigate stragglers and improve communication efficiency, a novel local SGD strategy, named STSyn, is developed. The key point is to wait for the $K$ fastest workers, while keeping all the workers computing continually at each synchronization round, and making full use of any effective (completed) local update of each worker regardless of stragglers. An analysis of the average wall-clock time, average number of local updates and average number of uploading workers per round is provided to gauge the performance of STSyn. The convergence of STSyn is also rigorously established even when the objective function is nonconvex. Experimental results show the superiority of the proposed STSyn against state-of-the-art schemes through utilization of the straggler-tolerant technique and additional effective local updates at each worker, and the influence of system parameters is studied. By waiting for faster workers and allowing heterogeneous synchronization with different numbers of local updates across workers, STSyn provides substantial improvements both in time and communication efficiency.

cs.LG↗

arXiv2023

Deep Reinforcement Learning Based Resource Allocation for Cloud Native Wireless Network

Cloud native technology has revolutionized 5G beyond and 6G communication networks, offering unprecedented levels of operational automation, flexibility, and adaptability. However, the vast array of cloud native services and applications presents a new challenge in resource allocation for dynamic cloud computing environments. To tackle this challenge, we investigate a cloud native wireless architecture that employs container-based virtualization to enable flexible service deployment. We then study two representative use cases: network slicing and Multi-Access Edge Computing. To optimize resource allocation in these scenarios, we leverage deep reinforcement learning techniques and introduce two model-free algorithms capable of monitoring the network state and dynamically training allocation policies. We validate the effectiveness of our algorithms in a testbed developed using Free5gc. Our findings demonstrate significant improvements in network efficiency, underscoring the potential of our proposed techniques in unlocking the full potential of cloud native wireless networks.

cs.NI↗

arXiv2023

FedGSM: Efficient Federated Learning for LEO Constellations with Gradient Staleness Mitigation

Recent advancements in space technology have equipped low Earth Orbit (LEO) satellites with the capability to perform complex functions and run AI applications. Federated Learning (FL) on LEO satellites enables collaborative training of a global ML model without the need for sharing large datasets. However, intermittent connectivity between satellites and ground stations can lead to stale gradients and unstable learning, thereby limiting learning performance. In this paper, we propose FedGSM, a novel asynchronous FL algorithm that introduces a compensation mechanism to mitigate gradient staleness. FedGSM leverages the deterministic and time-varying topology of the orbits to offset the negative effects of staleness. Our simulation results demonstrate that FedGSM outperforms state-of-the-art algorithms for both IID and non-IID datasets, underscoring its effectiveness and advantages. We also investigate the effect of system parameters.

eess.SP↗

arXiv2022

Communication-Efficient Local SGD with Age-Based Worker Selection

A major bottleneck of distributed learning under parameter-server (PS) framework is communication cost due to frequent bidirectional transmissions between the PS and workers. To address this issue, local stochastic gradient descent (SGD) and worker selection have been exploited by reducing the communication frequency and the number of participating workers at each round, respectively. However, partial participation can be detrimental to convergence rate, especially for heterogeneous local datasets. In this paper, to improve communication efficiency and speed up the training process, we develop a novel worker selection strategy named AgeSel. The key enabler of AgeSel is utilization of the ages of workers to balance their participation frequencies. The convergence of local SGD with the proposed age-based partial worker participation is rigorously established. Simulation results demonstrate that the proposed AgeSel strategy can significantly reduce the number of training rounds needed to achieve a targeted accuracy, as well as the communication cost. The influence of the algorithm hyper-parameter is also explored to manifest the benefit of age-based worker selection.

cs.IT↗

arXiv2022

Improving Human Sperm Head Morphology Classification with Unsupervised Anatomical Feature Distillation

With rising male infertility, sperm head morphology classification becomes critical for accurate and timely clinical diagnosis. Recent deep learning (DL) morphology analysis methods achieve promising benchmark results, but leave performance and robustness on the table by relying on limited and possibly noisy class labels. To address this, we introduce a new DL training framework that leverages anatomical and image priors from human sperm microscopy crops to extract useful features without additional labeling cost. Our core idea is to distill sperm head information with reliably-generated pseudo-masks and unsupervised spatial prediction tasks. The predicted foreground masks from this distillation step are then leveraged to regularize and reduce image and label noise in the tuning stage. We evaluate our new approach on two public sperm datasets and achieve state-of-the-art performances (e.g. 65.9% SCIAN accuracy and 96.5% HuSHeM accuracy).

cs.CV↗

arXiv2022

Adaptive Worker Grouping For Communication-Efficient and Straggler-Tolerant Distributed SGD

Wall-clock convergence time and communication load are key performance metrics for the distributed implementation of stochastic gradient descent (SGD) in parameter server settings. Communication-adaptive distributed Adam (CADA) has been recently proposed as a way to reduce communication load via the adaptive selection of workers. CADA is subject to performance degradation in terms of wall-clock convergence time in the presence of stragglers. This paper proposes a novel scheme named grouping-based CADA (G-CADA) that retains the advantages of CADA in reducing the communication load, while increasing the robustness to stragglers at the cost of additional storage at the workers. G-CADA partitions the workers into groups of workers that are assigned the same data shards. Groups are scheduled adaptively at each iteration, and the server only waits for the fastest worker in each selected group. We provide analysis and experimental results to elaborate the significant gains on the wall-clock time, as well as communication load and computation load, of G-CADA over other benchmark schemes.

cs.IT↗

arXiv2021

Understanding Longitudinal Dynamics of Recommender Systems with Agent-Based Modeling and Simulation

Today's research in recommender systems is largely based on experimental designs that are static in a sense that they do not consider potential longitudinal effects of providing recommendations to users. In reality, however, various important and interesting phenomena only emerge or become visible over time, e.g., when a recommender system continuously reinforces the popularity of already successful artists on a music streaming site or when recommendations that aim at profit maximization lead to a loss of consumer trust in the long run. In this paper, we discuss how Agent-Based Modeling and Simulation (ABM) techniques can be used to study such important longitudinal dynamics of recommender systems. To that purpose, we provide an overview of the ABM principles, outline a simulation framework for recommender systems based on the literature, and discuss various practical research questions that can be addressed with such an ABM-based simulation framework.

cs.IR↗

arXiv2021

Coded Alternating Least Squares for Straggler Mitigation in Distributed Recommendations

Matrix factorization is an important representation learning algorithm, e.g., recommender systems, where a large matrix can be factorized into the product of two low dimensional matrices termed as latent representations. This paper investigates the problem of matrix factorization in distributed computing systems with stragglers, those compute nodes that are slow to return computation results. A computation procedure, called coded Alternative Least Square (ALS), is proposed for mitigating the effect of stragglers in such systems. The coded ALS algorithm iteratively computes two low dimensional latent matrices by solving various linear equations, with the Entangled Polynomial Code (EPC) as a building block. We theoretically characterize the maximum number of stragglers that the algorithm can tolerate (or the recovery threshold) in relation to the redundancy of coding (or the code rate). In addition, we theoretically show the computation complexity for the coded ALS algorithm and conduct numerical experiments to validate our design.

cs.IT↗

arXiv2021

Coded Computing and Cooperative Transmission for Wireless Distributed Matrix Multiplication

Consider a multi-cell mobile edge computing network, in which each user wishes to compute the product of a user-generated data matrix with a network-stored matrix. This is done through task offloading by means of input uploading, distributed computing at edge nodes (ENs), and output downloading. Task offloading may suffer long delay since servers at some ENs may be straggling due to random computation time, and wireless channels may experience severe fading and interference. This paper aims to investigate the interplay among upload, computation, and download latencies during the offloading process in the high signal-to-noise ratio regime from an information-theoretic perspective. A policy based on cascaded coded computing and on coordinated and cooperative interference management in uplink and downlink is proposed and proved to be approximately optimal for a sufficiently large upload time. By investing more time in uplink transmission, the policy creates data redundancy at the ENs, which can reduce the computation time, by enabling the use of coded computing, as well as the download time via transmitter cooperation. Moreover, the policy allows computation time to be traded for download time. Numerical examples demonstrate that the proposed policy can improve over existing schemes by significantly reducing the end-to-end execution time.

cs.IT↗

arXiv2020

Asymptotic stability of Landau solutions to Navier-Stokes system under $L^p$-perturbations

In this paper, we show that Landau solutions to the Navier-Stokes system are asymptotically stable under $L^3$-perturbations. We give the local well-posedness of solutions to the perturbed system with initial data in $L_σ^3$ space and the global well-posedness with small initial data in $L_σ^3$ space, together with a study of the $L^q$ decay for all $q>3.$ Moreover, we have also studied the local well-posedness, global well-posedness and stability in $L^p$ spaces for $3<p<\infty$.

math.AP↗

arXiv2020

Automated Prostate Cancer Diagnosis Based on Gleason Grading Using Convolutional Neural Network

The Gleason grading system using histological images is the most powerful diagnostic and prognostic predictor of prostate cancer. The current standard inspection is evaluating Gleason H&E-stained histopathology images by pathologists. However, it is complicated, time-consuming, and subject to observers. Deep learning (DL) based-methods that automatically learn image features and achieve higher generalization ability have attracted significant attention. However, challenges remain especially using DL to train the whole slide image (WSI), a predominant clinical source in the current diagnostic setting, containing billions of pixels, morphological heterogeneity, and artifacts. Hence, we proposed a convolutional neural network (CNN)-based automatic classification method for accurate grading of PCa using whole slide histopathology images. In this paper, a data augmentation method named Patch-Based Image Reconstruction (PBIR) was proposed to reduce the high resolution and increase the diversity of WSIs. In addition, a distribution correction (DC) module was developed to enhance the adaption of pretrained model to the target dataset by adjusting the data distribution. Besides, a Quadratic Weighted Mean Square Error (QWMSE) function was presented to reduce the misdiagnosis caused by equal Euclidean distances. Our experiments indicated the combination of PBIR, DC, and QWMSE function was necessary for achieving superior expert-level performance, leading to the best results (0.8885 quadratic-weighted kappa coefficient).

eess.IV↗