arXiv subjects

Li Chai

Publications and source records attributed to Li Chai.

At least 19 recordsLinked to original sources

arXiv2026

Optimal Parameter Design for DIGing on Minimizing Unweighted Sum of Squares

There is no general method for designing proper parameters to achieve faster convergence in distributed optimization algorithms. In this paper, we consider the distributed inexact gradient tracking (DIGing) algorithm with the objective function being the unweighted sum of squares. By representing the iteration algorithm as a dynamical linear system, we decompose it into different graph frequencies and obtain a set of decoupled subsystems, on which we can easily analyze the convergence rate. By using Routh stability criterion from control theory, we derive the explicit formula of the optimal worst-case convergence rate and the corresponding parameters. We can see that the convergence rate of DIGing is slow even for the simplest objective functions, thus acceleration is necessary for general application. The proposed method can be viewed as the first step toward optimal parameter design of DIGing algorithm in solving general objective functions.

math.OC↗

arXiv2026

Exact Worst-case Convergence Rates of Distributed Gradient Tracking Methods

Different from most existing literature in the analysis of distributed optimization algorithms that reports sufficient convergence conditions leading to a conservative convergence rate, this work provides the exact worst-case convergence rates for two typical gradient tracking algorithms, DIGing and AugDGM. By eigen-decomposition, we show that two algorithms share the same average-state dynamics, while they differ from each other in the gradient tracking subsystems, which entirely govern algorithm convergence. Exploiting the diagonal structure of this decomposition, we reduce the stability analysis of MIMO systems to that of a set of parameter-varying SISO systems, from which explicit formulas for the exact worst-case convergence rate can be derived. These formulas clearly show that the optimal worst-case convergence rate of DIGing is larger than that of AugDGM under the same assumptions on objective functions and communication networks. Furthermore, we find that there is an inflection point in the graph connectivity, which is $\sigma =1/3$. For graphs with connectivity better than this inflection point, the optimal convergence rate of centralized gradient descent can be achieved by AugDGM provided the condition number of objective functions is worse enough. On the other hand, for graphs with connectivity worse than this inflection point, the centralized optimal rate can never be achieved. Numerical experiments validate the theoretical results.

math.OC↗

arXiv2025

A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization

This paper investigates the privacy-preserving distributed optimization problem, aiming to protect agents' private information from potential attackers during the optimization process. Gradient tracking, an advanced technique for improving the convergence rate in distributed optimization, has been applied to most first-order algorithms in recent years. We first reveal the inherent privacy leakage risk associated with gradient tracking. Building upon this insight, we propose a weighted gradient tracking distributed privacy-preserving algorithm, eliminating the privacy leakage risk in gradient tracking using decaying weight factors. Then, we characterize the convergence of the proposed algorithm under time-varying heterogeneous step sizes. We prove the proposed algorithm converges precisely to the optimal solution under mild assumptions. Finally, numerical simulations validate the algorithm's effectiveness through a classical distributed estimation problem and the distributed training of a convolutional neural network.

cs.LG↗

arXiv2025

Pareto-optimal Trade-offs Between Communication and Computation with Flexible Gradient Tracking

This paper addresses distributed stochastic optimization problems under non-i.i.d. data, focusing on the inherent trade-offs between communication and computational efficiency. To this end, we propose FlexGT, a flexible snapshot gradient tracking method that enables tunable numbers of local updates and neighbor communications per round, thereby adapting efficiently to diverse system resource conditions. Leveraging a unified convergence analysis framework, we derive tight communication and computational complexity for FlexGT with explicit dependence on objective properties and certain tunable parameters. Moreover, we introduce an accelerated variant, termed Acc-FlexGT, and prove that, with prior knowledge of the graph, it achieves Pareto-optimal trade-offs between communication and computation. Particularly, in the nonconvex case, Acc-FlexGT achieves the optimal iteration complexity of $\tilde{\mathcal{O}}\left( \left( L\sigma ^2 \right) /\left( n\epsilon ^2 \right) +L/\left( \epsilon \sqrt{1-\sqrt{\rho _W}} \right) \right) $ and optimal communication complexity of $\tilde{\mathcal{O}}\left( L/\left( \epsilon \sqrt{1-\sqrt{\rho _W}} \right) \right)$ for appropriately chosen numbers of local updates, matching existing lower bounds up to logarithmic factors. And, it improves the existing results for the strongly convex case by a factor of $\tilde{\mathcal{O}} \left( 1/\sqrt{\epsilon} \right)$, where $\epsilon$ is the targeted accuracy, $n$ the number of nodes, $L$ the Lipschitz constant, $\rho_W$ the connectivity of the graph, and $\sigma$ the stochastic gradient variance. Numerical experiments corroborate the theoretical results and demonstrate the effectiveness of the proposed methods.

math.OC↗

arXiv2025

ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning

Jointly addressing Byzantine attacks and privacy leakage in distributed machine learning (DML) has become an important issue. A common strategy involves integrating Byzantine-resilient aggregation rules with differential privacy mechanisms. However, the incorporation of these techniques often results in a significant degradation in model accuracy. To address this issue, we propose a decentralized DML framework, named ImprovDML, that achieves high model accuracy while simultaneously ensuring privacy preservation and resilience to Byzantine attacks. The framework leverages a kind of resilient vector consensus algorithms that can compute a point within the normal (non-Byzantine) agents' convex hull for resilient aggregation at each iteration. Then, multivariate Gaussian noises are introduced to the gradients for privacy preservation. We provide convergence guarantees and derive asymptotic learning error bounds under non-convex settings, which are tighter than those reported in existing works. For the privacy analysis, we adopt the notion of concentrated geo-privacy, which quantifies privacy preservation based on the Euclidean distance between inputs. We demonstrate that it enables an improved trade-off between privacy preservation and model accuracy compared to differential privacy. Finally, numerical simulations validate our theoretical results.

cs.LG↗

arXiv2025

Enhancing Accuracy in Differentially Private Distributed Optimization Through Sensitivity Reduction

In this paper, we investigate the problem of differentially private distributed optimization. Recognizing that lower sensitivity leads to higher accuracy, we analyze the key factors influencing the sensitivity of differentially private distributed algorithms. Building on these insights, we propose a novel differentially private distributed algorithm for undirected graphs that enhances optimization accuracy by reducing sensitivity. To ensure practical applicability, we derive an explicit closed-form expression for the noise parameter as a function of the privacy budget. Moreover, we rigorously prove that the proposed algorithm can achieve arbitrarily rigorous $\epsilon$-differential privacy, establish its convergence in the mean square sense, and provide an upper bound on its optimization accuracy. Finally, extensive comparisons with various privacy-preserving methods validate the effectiveness of our algorithm.

math.OC↗

arXiv2025

Adaptive Extensive Cancellation Algorithm and Harmonic Enhanced Heart Rate Estimation based on MMWave Radar

Heart rate (HR) monitoring is crucial for assessing physical fitness, cardiovascular health, and stress management. Millimeter-wave radar offers a promising noncontact solution for long-term monitoring. However, accurate HR estimation remains challenging in low signal-tonoise ratio (SNR) conditions. To deal with both respiration harmonics and intermodulation interference, this paper proposes a cancellation-before-estimation strategy. Firstly, we present the adaptive extensive cancellation algorithm (ECA) to suppress respiratory and its low-order harmonics. Then, we propose an adaptive harmonic enhanced trace (AHET) method to avoid intermodulation interference by refining the HR search region. Various experimental results validate the effectiveness of the proposed methods, demonstrating improvements in accuracy, robustness, and computational efficiency compared to conventional approaches based on the FMCW (Frequency Modulated Continuous Wave) system

eess.SP↗

arXiv2024

On Random Sampling of Diffused Graph Signals with Sparse Inputs on Vertex Domain

The sampling of graph signals has recently drawn much attention due to the wide applications of graph signal processing. While a lot of efficient methods and interesting results have been reported to the sampling of band-limited or smooth graph signals, few research has been devoted to non-smooth graph signals, especially to sparse graph signals, which are also of importance in many practical applications. This paper addresses the random sampling of non-smooth graph signals generated by diffusion of sparse inputs. We aim to present a solid theoretical analysis on the random sampling of diffused sparse graph signals, which can be parallel to that of band-limited graph signals, and thus present a sufficient condition to the number of samples ensuring the unique recovery for uniform random sampling. Then, we focus on two classes of widely used binary graph models, and give explicit and tighter estimations on the sampling numbers ensuring unique recovery. We also propose an adaptive variable-density sampling strategy to provide a better performance than uniform random sampling. Finally, simulation experiments are presented to validate the effectiveness of the theoretical results.

eess.SP↗

arXiv2024

Lifting Scheme-Based Implicit Disentanglement of Emotion-Related Facial Dynamics in the Wild

In-the-wild dynamic facial expression recognition (DFER) encounters a significant challenge in recognizing emotion-related expressions, which are often temporally and spatially diluted by emotion-irrelevant expressions and global context. Most prior DFER methods directly utilize coupled spatiotemporal representations that may incorporate weakly relevant features with emotion-irrelevant context bias. Several DFER methods highlight dynamic information for DFER, but following explicit guidance that may be vulnerable to irrelevant motion. In this paper, we propose a novel Implicit Facial Dynamics Disentanglement framework (IFDD). Through expanding wavelet lifting scheme to fully learnable framework, IFDD disentangles emotion-related dynamic information from emotion-irrelevant global context in an implicit manner, i.e., without exploit operations and external guidance. The disentanglement process contains two stages. The first is Inter-frame Static-dynamic Splitting Module (ISSM) for rough disentanglement estimation, which explores inter-frame correlation to generate content-aware splitting indexes on-the-fly. We utilize these indexes to split frame features into two groups, one with greater global similarity, and the other with more unique dynamic features. The second stage is Lifting-based Aggregation-Disentanglement Module (LADM) for further refinement. LADM first aggregates two groups of features from ISSM to obtain fine-grained global context features by an updater, and then disentangles emotion-related facial dynamic features from the global context by a predictor. Extensive experiments on in-the-wild datasets have demonstrated that IFDD outperforms prior supervised DFER methods with higher recognition accuracy and comparable efficiency. Code is available at https://github.com/CyberPegasus/IFDD.

cs.CV↗

arXiv2024

Hessian-Free Distributed Bilevel Optimization via Penalization with Time-Scale Separation

This paper considers a class of distributed bilevel optimization (DBO) problems with a coupled inner-level subproblem. Existing approaches typically rely on hypergradient estimations involving computationally expensive Hessian evaluation. To address this, we approximate the DBO problem as a minimax problem by properly designing a penalty term that enforces both the constraint imposed by the inner-level subproblem and the consensus among the decision variables of agents. Moreover, we propose a loopless distributed algorithm, AHEAD, that employs multiple-timescale updates to solve the approximate problem asymptotically without requiring Hessian computation. Theoretically, we establish sharp convergence rates for nonconvex-strongly-convex settings and for distributed minimax problems as special cases. Our analysis reveals a clear dependence of convergence performance on node heterogeneity, penalty parameters, and network connectivity, with a weaker assumption on heterogeneity that only requires bounded gradients at the optimum. Numerical experiments corroborate our theoretical results.

math.OC↗

arXiv2024

CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls

Lyric-to-melody generation is a highly challenging task in the field of AI music generation. Due to the difficulty of learning strict yet weak correlations between lyrics and melodies, previous methods have suffered from weak controllability, low-quality and poorly structured generation. To address these challenges, we propose CSL-L2M, a controllable song-level lyric-to-melody generation method based on an in-attention Transformer decoder with fine-grained lyric and musical controls, which is able to generate full-song melodies matched with the given lyrics and user-specified musical attributes. Specifically, we first introduce REMI-Aligned, a novel music representation that incorporates strict syllable- and sentence-level alignments between lyrics and melodies, facilitating precise alignment modeling. Subsequently, sentence-level semantic lyric embeddings independently extracted from a sentence-wise Transformer encoder are combined with word-level part-of-speech embeddings and syllable-level tone embeddings as fine-grained controls to enhance the controllability of lyrics over melody generation. Then we introduce human-labeled musical tags, sentence-level statistical musical attributes, and learned musical features extracted from a pre-trained VQ-VAE as coarse-grained, fine-grained and high-fidelity controls, respectively, to the generation process, thereby enabling user control over melody generation. Finally, an in-attention Transformer decoder technique is leveraged to exert fine-grained control over the full-song melody generation with the aforementioned lyric and musical conditions. Experimental results demonstrate that our proposed CSL-L2M outperforms the state-of-the-art models, generating melodies with higher quality, better controllability and enhanced structure. Demos and source code are available at https://lichaiustc.github.io/CSL-L2M/.

eess.AS↗

arXiv2024

Privacy-Preserving Resilient Vector Consensus

This paper studies privacy-preserving resilient vector consensus in multi-agent systems against faulty agents, where normal agents can achieve consensus within the convex hull of their initial states while protecting state vectors from being disclosed. Specifically, we consider a modification of an existing algorithm known as Approximate Distributed Robust Convergence Using Centerpoints (ADRC), i.e., Privacy-Preserving ADRC (PP-ADRC). Under PP-ADRC, each normal agent introduces multivariate Gaussian noise to its state during each iteration. We first provide sufficient conditions to ensure that all normal agents' states can achieve mean square convergence under PP-ADRC. Then, we analyze convergence accuracy from two perspectives, i.e., the Mahalanobis distance of the final value from its expectation and the Hausdorff distance-based alteration of the convex hull caused by noise when only partial dimensions are added with noise. Then, we employ concentrated geo-privacy to characterize privacy preservation and conduct a thorough comparison with differential privacy. Finally, numerical simulations demonstrate the theoretical results.

eess.SY↗

arXiv2024

Frequency-Domain Refinement with Multiscale Diffusion for Super Resolution

The performance of single image super-resolution depends heavily on how to generate and complement high-frequency details to low-resolution images. Recently, diffusion-based DDPM models exhibit great potential in generating high-quality details for super-resolution tasks. They tend to directly predict high-frequency information of wide bandwidth by solely utilizing the high-resolution ground truth as the target for all sampling timesteps. However, as a result, they encounter hallucination problem that they generate mismatching artifacts. To tackle this problem and achieve higher-quality super-resolution, we propose a novel Frequency Domain-guided multiscale Diffusion model (FDDiff), which decomposes the high-frequency information complementing process into finer-grained steps. In particular, a wavelet packet-based frequency degradation pyramid is developed to provide multiscale intermediate targets with increasing bandwidth. Based on these targets, FDDiff guides reverse diffusion process to progressively complement missing high-frequency details over timesteps. Moreover, a multiscale frequency refinement network is designed to predict the required high-frequency components at multiple scales within one unified network. Comprehensive evaluations on popular benchmarks are conducted, and demonstrate that FDDiff outperforms prior generative methods with higher-fidelity super-resolution results.

cs.CV↗

arXiv2024

Cryptography-Based Privacy-Preserving Method for Distributed Optimization over Time-Varying Directed Graphs with Enhanced Efficiency

In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm can be applied to time-varying directed networks. We show that the proposed algorithm can protect an agent's privacy if the agent has at least one legitimate neighbor at the initial iteration. Under the assumption that the objective function is strongly convex and Lipschitz smooth, we rigorously prove that the proposed algorithm has a linear convergence rate. Finally, the effectiveness of the proposed algorithm is demonstrated by numerical simulations of the canonical sensor fusion problem.

math.OC↗

arXiv2023

Distributed Stochastic Bilevel Optimization: Improved Complexity and Heterogeneity Analysis

This paper consider solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized yet. To address these issues, we propose LoPA, a loopless personalized distributed algorithm that leverages a tracking mechanism for iterative approximation of inner-level solutions and Hessian-inverse matrices without relying on extra computation loops. Our theoretical analysis explicitly characterizes the heterogeneity across nodes (denoted by $b$), and establishes a sublinear rate of $\mathcal{O}( {\frac{1}{{{{\left( {1 - \rho } \right)}}K}} \!+ \!\frac{{(\frac{b}{\sqrt{m}})^{\frac{2}{3}} }}{{\left( {1 - \rho } \right)^{\frac{2}{3}} K^{\frac{2}{3}} }} \!+ \!\frac{1}{\sqrt{ K }}( {\sigma _{\operatorname{p} }} + \frac{1}{\sqrt{m}}{\sigma _{\operatorname{c} }} ) } )$ without the boundedness of local hypergradients, where ${\sigma _{\operatorname{p} }}$ and ${\sigma _{\operatorname{c} }}$ represent the gradient sampling variances associated with the inner- and outer-level variables, respectively. We also integrate LoPA with a gradient tracking scheme to eliminate the impact of data heterogeneity, yielding an improved rate of ${{\mathcal{O}}}(\frac{{1}}{{ (1-\rho)^2K }} \!+\! \frac{1}{{\sqrt{K}}}( \sigma_{\rm{p}} \!+\! \frac{1}{\sqrt{m}}\sigma_{\rm{c}} ) )$. The computational complexity of LoPA is of ${{\mathcal{O}}}({\epsilon^{-2}})$ to an $\epsilon$-stationary point, matching the communication complexity due to the loopless structure, which outperforms existing counterparts for DSBO. Numerical experiments validate the effectiveness of the proposed algorithm.

math.OC↗

arXiv2023

WCCNet: Wavelet-context Cooperative Network for Efficient Multispectral Pedestrian Detection

Multispectral pedestrian detection is essential to various tasks especially autonomous driving, for which both the accuracy and computational cost are of paramount importance. Most existing approaches treat RGB and infrared modalities equally. They typically adopt two symmetrical backbones for multimodal feature extraction, which ignore the substantial differences between modalities and bring great difficulty for the reduction of the computational cost as well as effective crossmodal fusion. In this work, we propose a novel and efficient framework named Wavelet-context Cooperative Network (WCCNet), which differentially extracts complementary features across spectra with low computational cost and further fuses these diverse features based on their spatially relevant cross-modal semantics. WCCNet explores an asymmetric but cooperative dual-stream backbone, in which WCCNet utilizes generic neural layers for texture-rich feature extraction from RGB modality, while proposing Mixture of Wavelet Experts (MoWE) to capture complementary frequency patterns of infrared modality. By assessing multispectral environmental context, MoWE generates routing scores to selectively activate specific learnable Adaptive DWT (ADWT) layers, alongside shared static DWT, which are both considerible lightwight and efficient to significantly reduce computational overhead and facilitate subsequent fusion. To further fuse these multispectral features with significant semantic differences, we elaborately design the crossmodal rearranging fusion module (CMRF), which aims to mitigate misalignment and merge semantically complementary features in spatially-related local regions to amplify the crossmodal reciprocal information. Results from comprehensive evaluations on KAIST and FLIR benchmarks indicate that WCCNet outperforms state-of-the-art methods with considerable computational efficiency and competitive accuracy.

cs.CV↗

arXiv2023

Accelerating the Convergence Rate of Consensus for Second-Order Multi-Agent Systems by Memory Information

This paper utilizes the agent's memory in accelerated consensus for second-order multi-agent systems (MASs). In the case of one-tap memory, explicit formulas for the optimal consensus convergence rate and control parameters are derived by applying the Jury stability criterion. It is proved that the optimal consensus convergence rate with one-tap memory is faster than that without memory. In the case of M-tap memory, an iterative algorithm is given to derive the control parameters to accelerate the convergence rate. Moreover, the accelerated consensus with one-tap memory is extended to the formation control, and the control parameters to achieve the fastest formation are obtained. Numerical examples further illustrate the theoretical results.

math.OC↗

arXiv2023

A Novel Bilateral Energy Trading Mechanism for Electricity Markets with Numerous Prosumers

With the rapid development of distributed energy resources, increasing number of residential and commercial users have been switched from pure electricity consumers to prosumers that can both consume and produce energy. To properly manage these emerging prosumers, a peer-to-peer electricity market has been explored and extensively studied. In such an electricity market, each prosumer trades energy directly with other prosumers, posing a serious challenge to the scalability of the market. Therefore, a bilateral energy trading mechanism with good scalability is proposed for electricity markets with numerous prosumers in this paper. First, the multi-bilateral economic dispatch problem that maximizes the social welfare is formulated, taking into account product differentiation and network constraints. Then, an energy trading mechanism is devised to improve the scalability from two aspects: (i) an accelerated distributed clearing algorithm with less exchanged information and faster convergence rate. (ii) a novel selection strategy to reduce the amount of computation and communication per prosumer. Finally, the convergence proof of the proposed accelerated algorithm is given, and the proposed selection strategy is illustrated through a Monte Carlo simulation experiment.

cs.DC↗