arXiv subjects

Juncheng Wang

Publications and source records attributed to Juncheng Wang.

33 records · Page 2Linked to original sources

arXiv2025

Doubly-Bounded Queue for Constrained Online Learning: Keeping Pace with Dynamics of Both Loss and Constraint

We consider online convex optimization with time-varying constraints and conduct performance analysis using two stringent metrics: dynamic regret with respect to the online solution benchmark, and hard constraint violation that does not allow any compensated violation over time. We propose an efficient algorithm called Constrained Online Learning with Doubly-bounded Queue (COLDQ), which introduces a novel virtual queue that is both lower and upper bounded, allowing tight control of the constraint violation without the need for the Slater condition. We prove via a new Lyapunov drift analysis that COLDQ achieves $O(T^\frac{1+V_x}{2})$ dynamic regret and $O(T^{V_g})$ hard constraint violation, where $V_x$ and $V_g$ capture the dynamics of the loss and constraint functions. For the first time, the two bounds smoothly approach to the best-known $O(T^\frac{1}{2})$ regret and $O(1)$ violation, as the dynamics of the losses and constraints diminish. For strongly convex loss functions, COLDQ matches the best-known $O(\log{T})$ static regret while maintaining the $O(T^{V_g})$ hard constraint violation. We further introduce an expert-tracking variation of COLDQ, which achieves the same performance bounds without any prior knowledge of the system dynamics. Simulation results demonstrate that COLDQ outperforms the state-of-the-art approaches.

cs.LG↗

arXiv2025

Constrained Over-the-Air Model Updating for Wireless Online Federated Learning with Delayed Information

We study online federated learning over a wireless network, where the central server updates an online global model sequence to minimize the time-varying loss of multiple local devices over time. The server updates the global model through over-the-air model-difference aggregation from the local devices over a noisy multiple-access fading channel. We consider the practical scenario where information on both the local loss functions and the channel states is delayed, and each local device is under a time-varying power constraint. We propose Constrained Over-the-air Model Updating with Delayed infOrmation (COMUDO), where a new lower-and-upper-bounded virtual queue is introduced to counter the delayed information and control the hard constraint violation. We show that its local model updates can be efficiently computed in closed-form expressions. Furthermore, through a new Lyapunov drift analysis, we show that COMUDO provides bounds on the dynamic regret, static regret, and hard constraint violation. Simulation results on image classification tasks under practical wireless network settings show substantial accuracy gain of COMUDO over state-of-the-art approaches, especially in the low-power region.

cs.DC↗

arXiv2024

Adaptive Federated Learning in Heterogeneous Wireless Networks with Independent Sampling

Federated Learning (FL) algorithms commonly sample a random subset of clients to address the straggler issue and improve communication efficiency. While recent works have proposed various client sampling methods, they have limitations in joint system and data heterogeneity design, which may not align with practical heterogeneous wireless networks. In this work, we advocate a new independent client sampling strategy to minimize the wall-clock training time of FL, while considering data heterogeneity and system heterogeneity in both communication and computation. We first derive a new convergence bound for non-convex loss functions with independent client sampling and then propose an adaptive bandwidth allocation scheme. Furthermore, we propose an efficient independent client sampling algorithm based on the upper bounds on the convergence rounds and the expected per-round training time, to minimize the wall-clock time of FL, while considering both the data and system heterogeneity. Experimental results under practical wireless network settings with real-world prototype demonstrate that the proposed independent sampling scheme substantially outperforms the current best sampling schemes under various training models and datasets.

cs.LG↗

arXiv2024

Joint Beamforming Design and Stream Allocation for Non-Coherent Joint Transmission in Cell-Free MIMO Networks

We consider joint beamforming and stream allocation to maximize the weighted sum rate (WSR) for non-coherent joint transmission (NCJT) in user-centric cell-free MIMO networks, where distributed access points (APs) are organized in clusters to transmit different signals to serve each user equipment (UE). We for the first time consider the common limits of maximum number of receive streams at UEs in practical networks, and formulate a joint beamforming and transmit stream allocation problem for WSR maximization under per-AP transmit power constraints. Since the integer number of transmit streams determines the dimension of the beamformer, the joint optimization problem is mixed-integer and nonconvex with coupled decision variables that is inherently NP-hard. In this paper, we first propose a distributed low-interaction reduced weighted minimum mean square error (RWMMSE) beamforming algorithm for WSR maximization with fixed streams. Our proposed RWMMSE algorithm requires significantly less interaction across the network and has the current lowest computational complexity that scales linearly with the number of transmit antennas, without any compromise on WSR. We draw insights on the joint beamforming and stream allocation problem to decouple the decision variables and relax the mixed-integer constraints. We then propose a joint beamforming and linear stream allocation algorithm, termed as RWMMSE-LSA, which yields closed-form updates with linear stream allocation complexity and is guaranteed to converge to the stationary points of the original joint optimization problem. Simulation results demonstrate substantial performance gain of our proposed algorithms over the current best alternatives in both WSR performance and convergence time.

eess.SP↗

arXiv2024

Energy-Efficient Data Offloading for Earth Observation Satellite Networks

In Earth Observation Satellite Networks (EOSNs) with a large number of battery-carrying satellites, proper power allocation and task scheduling are crucial to improving the data offloading efficiency. As such, we jointly optimize power allocation and task scheduling to achieve energy-efficient data offloading in EOSNs, aiming to balance the objectives of reducing the total energy consumption and increasing the sum weights of tasks. First, we derive the optimal power allocation solution to the joint optimization problem when the task scheduling policy is given. Second, leveraging the conflict graph model, we transform the original joint optimization problem into a maximum weight independent set problem when the power allocation strategy is given. Finally, we utilize the genetic framework to combine the above special solutions as a two-layer solution for the joint optimization problem. Simulation results demonstrate that our proposed solution can properly balance the sum weights of tasks and the total energy consumption, achieving superior system performance over the current best alternatives.

math.OC↗

arXiv2023

Frustratingly Easy Model Generalization by Dummy Risk Minimization

Empirical risk minimization (ERM) is a fundamental machine learning paradigm. However, its generalization ability is limited in various tasks. In this paper, we devise Dummy Risk Minimization (DuRM), a frustratingly easy and general technique to improve the generalization of ERM. DuRM is extremely simple to implement: just enlarging the dimension of the output logits and then optimizing using standard gradient descent. Moreover, we validate the efficacy of DuRM on both theoretical and empirical analysis. Theoretically, we show that DuRM derives greater variance of the gradient, which facilitates model generalization by observing better flat local minima. Empirically, we conduct evaluations of DuRM across different datasets, modalities, and network architectures on diverse tasks, including conventional classification, semantic segmentation, out-of-distribution generalization, adverserial training, and long-tailed recognition. Results demonstrate that DuRM could consistently improve the performance under all tasks with an almost free lunch manner. Furthermore, we show that DuRM is compatible with existing generalization techniques and we discuss possible limitations. We hope that DuRM could trigger new interest in the fundamental research on risk minimization.

cs.LG↗

arXiv2023

Crowd Localization from Gaussian Mixture Scoped Knowledge and Scoped Teacher

Crowd localization is to predict each instance head position in crowd scenarios. Since the distance of instances being to the camera are variant, there exists tremendous gaps among scales of instances within an image, which is called the intrinsic scale shift. The core reason of intrinsic scale shift being one of the most essential issues in crowd localization is that it is ubiquitous in crowd scenes and makes scale distribution chaotic. To this end, the paper concentrates on access to tackle the chaos of the scale distribution incurred by intrinsic scale shift. We propose Gaussian Mixture Scope (GMS) to regularize the chaotic scale distribution. Concretely, the GMS utilizes a Gaussian mixture distribution to adapt to scale distribution and decouples the mixture model into sub-normal distributions to regularize the chaos within the sub-distributions. Then, an alignment is introduced to regularize the chaos among sub-distributions. However, despite that GMS is effective in regularizing the data distribution, it amounts to dislodging the hard samples in training set, which incurs overfitting. We assert that it is blamed on the block of transferring the latent knowledge exploited by GMS from data to model. Therefore, a Scoped Teacher playing a role of bridge in knowledge transform is proposed. What' s more, the consistency regularization is also introduced to implement knowledge transform. To that effect, the further constraints are deployed on Scoped Teacher to derive feature consistence between teacher and student end. With proposed GMS and Scoped Teacher implemented on five mainstream datasets of crowd localization, the extensive experiments demonstrate the superiority of our work. Moreover, comparing with existing crowd locators, our work achieves state-of-the-art via F1-meansure comprehensively on five datasets.

cs.CV↗

arXiv2022

Counting Like Human: Anthropoid Crowd Counting on Modeling the Similarity of Objects

The mainstream crowd counting methods regress density map and integrate it to obtain counting results. Since the density representation to one head accords to its adjacent distribution, it embeds the same category objects with variant values, while human beings counting models the invariant features namely similarity to objects. Inspired by this, we propose a rational and anthropoid crowd counting framework. To begin with, we leverage counting scalar as supervision signal, which provides global and implicit guidance to similar matters. Then, the large kernel CNN is utilized to imitate the paradigm of human beings which models invariant knowledge firstly and slides to compare similarity. Later, re-parameterization on pre-trained paralleled parameters is presented to cater to the inner-class variance on similarity comparison. Finally, the Random Scaling patches Yield (RSY) is proposed to facilitate similarity modeling on long distance dependencies. Extensive experiments on five challenging benchmarks in crowd counting show the proposed framework achieves state-of-the-art.

cs.CV↗

arXiv2022

Online Distributed Coordinated Precoding for Virtualized MIMO Networks with Delayed CSI

We consider online wireless network virtualization (WNV) in a multi-cell multiple-input multiple output (MIMO) system with delayed feedback of channel state information (CSI). Multiple service providers (SPs) simultaneously share the base station resources of an infrastructure provider (InP). We aim at minimizing the accumulated precoding deviation of the InP's actual precoder from the SPs' virtualization demands via managing both inter-SP and inter-cell interference, subject to both long-term and short-term per-cell transmit power constraints. We develop an online coordinated precoding solution and show that it provides provable performance bounds. Our precoding solution is fully distributed at each cell, based only on delayed local CSI. Furthermore, it has a closed-form expression with low computational complexity. Finally, simulation results demonstrate the substantial performance gain of our precoding solution over the current best alternative.

cs.IT↗

arXiv2021

Hierarchical Online Convex Optimization

We consider online convex optimization (OCO) over a heterogeneous network with communication delay, where multiple workers together with a master execute a sequence of decisions to minimize the accumulation of time-varying global costs. The local data may not be independent or identically distributed, and the global cost functions may not be locally separable. Due to communication delay, neither the master nor the workers have in-time information about the current global cost function. We propose a new algorithm, termed Hierarchical OCO (HiOCO), which takes full advantage of the network heterogeneity in information timeliness and computation capacity to enable multi-step gradient descent at both the workers and the master. We analyze the impacts of the unique hierarchical architecture, multi-slot delay, and gradient estimation error to derive upper bounds on the dynamic regret of HiOCO, which measures the gap of costs between HiOCO and an offline globally optimal performance benchmark.

cs.IT↗

arXiv2021

Periodic Updates for Constrained OCO with Application to Large-Scale Multi-Antenna Systems

In many dynamic systems, decisions on system operation are updated over time, and the decision maker requires an online learning approach to optimize its strategy in response to the changing environment. When the loss and constraint functions are convex, this belongs to the general family of online convex optimization (OCO). In existing OCO works, the environment is assumed to vary in a time-slotted fashion, while the decisions are updated at each time slot. However, many wireless communication systems permit only periodic decision updates, i.e., each decision is fixed over multiple time slots, while the environment changes between the decision epochs. The standard OCO model is inadequate for these systems. Therefore, in this work, we consider periodic decision updates for OCO. We aim to minimize the accumulation of time-varying convex loss functions, subject to both short-term and long-term constraints. Information about the loss functions within the current update period may be incomplete and is revealed to the decision maker only after the decision is made. We propose an efficient algorithm, termed Periodic Queueing and Gradient Aggregation (PQGA), which employs novel periodic queues together with possibly multi-step aggregated gradient descent to update the decisions over time. We derive upper bounds on the dynamic regret, static regret, and constraint violation of PQGA. As an example application, we study the performance of PQGA in a large-scale multi-antenna system shared by multiple wireless service providers. Simulation results show that PQGA converges fast and substantially outperforms the known best alternative.

cs.IT↗

arXiv2021

Online Multi-Cell Coordinated MIMO Wireless Network Virtualization with Imperfect CSI

We consider online coordinated precoding design for downlink wireless network virtualization (WNV) in a multi-cell multiple-input multiple-output (MIMO) network with imperfect channel state information (CSI). In our WNV framework, an infrastructure provider (InP) owns each base station that is shared by several service providers (SPs) oblivious of each other. The SPs design their precoders as virtualization demands for user services, while the InP designs the actual precoding solution to meet the service demands from the SPs. Our aim is to minimize the long-term time-averaged expected precoding deviation over MIMO fading channels, subject to both per-cell long-term and short-term transmit power limits. We propose an online coordinated precoding algorithm for virtualization, which provides a fully distributed semi-closed-form precoding solution at each cell, based only on the current imperfect CSI without any CSI exchange across cells. Taking into account the two-fold impact of imperfect CSI on both the InP and the SPs, we show that our proposed algorithm is within an $O(δ)$ gap from the optimum over any time horizon, where $δ$ is a CSI inaccuracy indicator. Simulation results validate the performance of our proposed algorithm under two commonly used precoding techniques in a typical urban micro-cell network environment.

cs.IT↗

arXiv2021

Delay-Tolerant Constrained OCO with Application to Network Resource Allocation

We consider online convex optimization (OCO) with multi-slot feedback delay, where an agent makes a sequence of online decisions to minimize the accumulation of time-varying convex loss functions, subject to short-term and long-term constraints that are possibly time-varying. The current convex loss function and the long-term constraint function are revealed to the agent only after the decision is made, and they may be delayed for multiple time slots. Existing work on OCO under this general setting has focused on the static regret, which measures the gap of losses between the online decision sequence and an offline benchmark that is fixed over time. In this work, we consider both the static regret and the more practically meaningful dynamic regret, where the benchmark is a time-varying sequence of per-slot optimizers. We propose an efficient algorithm, termed Delay-Tolerant Constrained-OCO (DTC-OCO), which uses a novel constraint penalty with double regularization to tackle the asynchrony between information feedback and decision updates. We derive upper bounds on its dynamic regret, static regret, and constraint violation, proving them to be sublinear under mild conditions. We further apply DTC-OCO to a general network resource allocation problem, which arises in many systems such as data networks and cloud computing. Simulation results demonstrate substantial performance gain of DTC-OCO over the known best alternative.

cs.IT↗

arXiv2021

Distributed coordinated precoding for MIMO cellular network virtualization

This paper presents a new virtualization method for the downlink of a multi-cell multiple-input multiple-output (MIMO) network, to achieve service isolation among multiple Service Providers (SPs) that share the base station resources of an Infrastructure Provider (InP). Each SP designs a virtual precoder for its users in each cell, as its service demand to the InP, without the need to be aware of the existence of the other SPs or to know the channel state information (CSI) outside the cell. The InP performs network virtualization to meet the SPs' service demands while managing both the inter-SP and inter-cell interference. We consider coordinated multi-cell precoding at the InP and formulate an optimization problem to minimize a weighted sum of signal leakage and precoding deviation, with per-cell transmit power constraints. We propose a fully distributed semi-closed-form solution at each cell, without any CSI exchange across cells. We further propose a low-complexity scheme to allocate the virtual transmit power, for the InP to regulate between interference elimination and virtual demand maximization. Simulation results demonstrate that our precoding solution for network virtualization substantially outperforms the traditional spectrum isolation alternative. It can approach the performance of fully cooperative precoding when the number of antennas is large.

cs.IT↗

arXiv2017

Logically reversible measurements: Construction and application

We show that for any von Neumann measurement, we can construct a logically reversible measurement such that Shannon entropies and quantum discords induced by the two measurements have compact connections. In particular, we prove that quantum discord for the logically reversible measurement is never less than that for the von Neumann measurement.

quant-ph↗