arXiv subjects

Jingge Zhu

Publications and source records attributed to Jingge Zhu.

At least 55 records · Page 3Linked to original sources

arXiv2023

Stability Bounds for Learning-Based Adaptive Control of Discrete-Time Multi-Dimensional Stochastic Linear Systems with Input Constraints

We consider the problem of adaptive stabilization for discrete-time, multi-dimensional linear systems with bounded control input constraints and unbounded stochastic disturbances, where the parameters of the true system are unknown. To address this challenge, we propose a certainty-equivalent control scheme which combines online parameter estimation with saturated linear control. We establish the existence of a high probability stability bound on the closed-loop system, under additional assumptions on the system and noise processes. Finally, numerical examples are presented to illustrate our results.

eess.SY↗

arXiv2023

Learning-Based Adaptive Control for Stochastic Linear Systems with Input Constraints

We propose a certainty-equivalence scheme for adaptive control of scalar linear systems subject to additive, i.i.d. Gaussian disturbances and bounded control input constraints, without requiring prior knowledge of the bounds of the system parameters, nor the control direction. Assuming that the system is at-worst marginally stable, mean square boundedness of the closed-loop system states is proven. Lastly, numerical examples are presented to illustrate our results.

eess.SY↗

arXiv2023

On the Value of Stochastic Side Information in Online Learning

We study the effectiveness of stochastic side information in deterministic online learning scenarios. We propose a forecaster to predict a deterministic sequence where its performance is evaluated against an expert class. We assume that certain stochastic side information is available to the forecaster but not the experts. We define the minimax expected regret for evaluating the forecasters performance, for which we obtain both upper and lower bounds. Consequently, our results characterize the improvement in the regret due to the stochastic side information. Compared with the classical online learning problem with regret scales with O(\sqrt(n)), the regret can be negative when the stochastic side information is more powerful than the experts. To illustrate, we apply the proposed bounds to two concrete examples of different types of side information.

cs.LG↗

arXiv2023

Design and Analysis of Hardware-limited Non-uniform Task-based Quantizers

Hardware-limited task-based quantization is a new design paradigm for data acquisition systems equipped with serial scalar analog-to-digital converters using a small number of bits. By taking into account the underlying system task, task-based quantizers can efficiently recover the desired parameters from the low-bit quantized observation. Current design and analysis frameworks for hardware-limited task-based quantization are only applicable to inputs with bounded support and uniform quantizers with non-subtractive dithering. Here, we propose a new framework based on generalized Bussgang decomposition that enables the design and analysis of hardware-limited task-based quantizers that are equipped with non-uniform scalar quantizers or that have inputs with unbounded support. We first consider the scenario in which the task is linear. Under this scenario, we derive new pre-quantization and post-quantization linear mappings for task-based quantizers with mean squared error (MSE) that closely matches the theoretical MSE. Next, we extend the proposed analysis framework to quadratic tasks. We demonstrate that our derived analytical expression for the MSE accurately predicts the performance of task-based quantizers with quadratic tasks.

eess.SP↗

arXiv2022

Capacity Bounds for One-Bit MIMO Gaussian Channels with Analog Combining

The use of 1-bit analog-to-digital converters (ADCs) is seen as a promising approach to significantly reduce the power consumption and hardware cost of multiple-input multiple-output (MIMO) receivers. However, the nonlinear distortion due to 1-bit quantization fundamentally changes the optimal communication strategy and also imposes a capacity penalty to the system. In this paper, the capacity of a Gaussian MIMO channel in which the antenna outputs are processed by an analog linear combiner and then quantized by a set of zero threshold ADCs is studied. A new capacity upper bound for the zero threshold case is established that is tighter than the bounds available in the literature. In addition, we propose an achievability scheme which configures the analog combiner to create parallel Gaussian channels with phase quantization at the output. Under this class of analog combiners, an algorithm is presented that identifies the analog combiner and input distribution that maximize the achievable rate. Numerical results are provided showing that the rate of the achievability scheme is tight in the low signal-to-noise ratio (SNR) regime. Finally, a new 1-bit MIMO receiver architecture which employs analog temporal and spatial processing is proposed. The proposed receiver attains the capacity in the high SNR regime.

cs.IT↗

arXiv2022

On the Capacity-Achieving Input of the Gaussian Channel with Polar Quantization

The polar receiver architecture is a receiver design that captures the envelope and phase information of the signal rather than its in-phase and quadrature components. Several studies have demonstrated the robustness of polar receivers to phase noise and other nonlinearities. Yet, the information-theoretic limits of polar receivers with finite-precision quantizers have not been investigated in the literature. The main contribution of this work is to identify the optimal signaling strategy for the additive white Gaussian noise (AWGN) channel with polar quantization at the output. More precisely, we show that the capacity-achieving modulation scheme has an amplitude phase shift keying (APSK) structure. Using this result, the capacity of the AWGN channel with polar quantization at the output is established by numerically optimizing the probability mass function of the amplitude. The capacity of the polar-quantized AWGN channel with $b_1$-bit phase quantizer and optimized single-bit magnitude quantizer is also presented. Our numerical findings suggest the existence of signal-to-noise ratio (SNR) thresholds, above which the number of amplitude levels of the optimal APSK scheme and their respective probabilities change abruptly. Moreover, the manner in which the capacity-achieving input evolves with increasing SNR depends on the number of phase quantization bits.

cs.IT↗

arXiv2022

A Learning-Based Approach to Approximate Coded Computation

Lagrange coded computation (LCC) is essential to solving problems about matrix polynomials in a coded distributed fashion; nevertheless, it can only solve the problems that are representable as matrix polynomials. In this paper, we propose AICC, an AI-aided learning approach that is inspired by LCC but also uses deep neural networks (DNNs). It is appropriate for coded computation of more general functions. Numerical simulations demonstrate the suitability of the proposed approach for the coded computation of different matrix functions that are often utilized in digital signal processing.

cs.IT↗

arXiv2022

Fast Rate Generalization Error Bounds: Variations on a Theme

A recent line of works, initiated by Russo and Xu, has shown that the generalization error of a learning algorithm can be upper bounded by information measures. In most of the relevant works, the convergence rate of the expected generalization error is in the form of O(sqrt{lambda/n}) where lambda is some information-theoretic quantities such as the mutual information between the data sample and the learned hypothesis. However, such a learning rate is typically considered to be "slow", compared to a "fast rate" of O(1/n) in many learning scenarios. In this work, we first show that the square root does not necessarily imply a slow rate, and a fast rate (O(1/n)) result can still be obtained using this bound under appropriate assumptions. Furthermore, we identify the key conditions needed for the fast rate generalization error, which we call the (eta,c)-central condition. Under this condition, we give information-theoretic bounds on the generalization error and excess risk, with a convergence rate of O(λ/{n}) for specific learning algorithms such as empirical risk minimization. Finally, analytical examples are given to show the effectiveness of the bounds.

cs.IT↗

arXiv2022

On the Capacity-Achieving Input of Channels with Phase Quantization

Several information-theoretic studies on channels with output quantization have identified the capacity-achieving input distributions for different fading channels with 1-bit in-phase and quadrature (I/Q) output quantization. However, an exact characterization of the capacity-achieving input distribution for channels with multi-bit phase quantization has not been provided. In this paper, we consider four different channel models with multi-bit phase quantization at the output and identify the optimal input distribution for each channel model. We first consider a complex Gaussian channel with $b$-bit phase-quantized output and prove that the capacity-achieving distribution is a rotated $2^b$-phase shift keying (PSK). The analysis is then extended to multiple fading scenarios. We show that the optimality of rotated $2^b$-PSK continues to hold under noncoherent fast fading Rician channels with $b$-bit phase quantization when line-of-sight (LoS) is present. When channel state information (CSI) is available at the receiver, we identify $\frac{2π}{2^b}$-symmetry and constant amplitude as the necessary and sufficient conditions for the ergodic capacity-achieving input distribution; which a $2^b$-PSK satisfies. Finally, an optimum power control scheme is presented which achieves ergodic capacity when CSI is also available at the transmitter.

cs.IT↗

arXiv2021

A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms

Transfer learning is a machine learning paradigm where knowledge from one problem is utilized to solve a new but related problem. While conceivable that knowledge from one task could be useful for solving a related task, if not executed properly, transfer learning algorithms can impair the learning performance instead of improving it -- commonly known as negative transfer. In this paper, we study transfer learning from a Bayesian perspective, where a parametric statistical model is used. Specifically, we study three variants of transfer learning problems, instantaneous, online, and time-variant transfer learning. For each problem, we define an appropriate objective function, and provide either exact expressions or upper bounds on the learning performance using information-theoretic quantities, which allow simple and explicit characterizations when the sample size becomes large. Furthermore, examples show that the derived bounds are accurate even for small sample sizes. The obtained bounds give valuable insights into the effect of prior knowledge for transfer learning, at least with respect to our Bayesian formulation of the transfer learning problem. In particular, we formally characterize the conditions under which negative transfer occurs. Lastly, we devise two (online) transfer learning algorithms that are amenable to practical implementations, one of which does not require the parametric assumption. We demonstrate the effectiveness of our algorithms with real data sets, focusing primarily on when the source and target data have strong similarities.

cs.LG↗

arXiv2021

On Minimizing Symbol Error Rate Over Fading Channels with Low-Resolution Quantization

We analyze the symbol error probability (SEP) of $M$-ary pulse amplitude modulation ($M$-PAM) receivers equipped with optimal low-resolution quantizers. We first show that the optimum detector can be reduced to a simple decision rule. Using this simplification, an exact SEP expression for quantized $M$-PAM receivers is obtained when Nakagami-$m$ fading channel is considered. The derived expression enables the optimization of the quantizer and/or constellation under the minimum SEP criterion. Our analysis of optimal quantization for equidistant $M$-PAM receiver reveals the existence of error floor which decays at a double exponential rate with increasing quantization bits, $b$. Moreover, by also allowing the transmitter to optimize the constellation based on the statistics of the fading channel, we prove that the error floor can be eliminated but at a lower decay exponent than the unquantized case. Characterization of this decay exponent is provided in this paper. We also expose the outage performance limitations of SEP-optimal uniform quantizers. To be more precise, its decay exponent does not improve with $b$. Lastly, we demonstrate that the decay exponent of a quantized receiver can be complemented by receive antenna diversity techniques.

eess.SP↗

arXiv2021

Is Phase Shift Keying Optimal for Channels with Phase-Quantized Output?

This paper establishes the capacity of additive white Gaussian noise (AWGN) channels with phase-quantized output. We show that a rotated $2^b$-phase shift keying scheme is the capacity-achieving input distribution for a complex AWGN channel with $b$-bit phase quantization. The result is then used to establish the expression for the channel capacity as a function of average power constraint $P$ and quantization bits $b$. The outage performance of phase-quantized system is also investigated for the case of Rayleigh fading when the channel state information (CSI) is only known at the receiver. Our findings suggest the existence of a threshold in the rate $R$, above which the outage exponent of the outage probability changes abruptly. In fact, this threshold effect in the outage exponent causes $2^b$-PSK to have suboptimal outage performance at high SNR.

cs.IT↗

arXiv2021

Online Transfer Learning: Negative Transfer and Effect of Prior Knowledge

Transfer learning is a machine learning paradigm where the knowledge from one task is utilized to resolve the problem in a related task. On the one hand, it is conceivable that knowledge from one task could be useful for solving a related problem. On the other hand, it is also recognized that if not executed properly, transfer learning algorithms could in fact impair the learning performance instead of improving it - commonly known as "negative transfer". In this paper, we study the online transfer learning problems where the source samples are given in an offline way while the target samples arrive sequentially. We define the expected regret of the online transfer learning problem and provide upper bounds on the regret using information-theoretic quantities. We also obtain exact expressions for the bounds when the sample size becomes large. Examples show that the derived bounds are accurate even for small sample sizes. Furthermore, the obtained bounds give valuable insight on the effect of prior knowledge for transfer learning in our formulation. In particular, we formally characterize the conditions under which negative transfer occurs.

cs.LG↗

arXiv2020

Customized Local Differential Privacy for Multi-Agent Distributed Optimization

Real-time data-driven optimization and control problems over networks may require sensitive information of participating users to calculate solutions and decision variables, such as in traffic or energy systems. Adversaries with access to coordination signals may potentially decode information on individual users and put user privacy at risk. We develop local differential privacy, which is a strong notion that guarantees user privacy regardless of any auxiliary information an adversary may have, for a larger family of convex distributed optimization problems. The mechanism allows agent to customize their own privacy level based on local needs and parameter sensitivities. We propose a general sampling based approach for determining sensitivity and derive analytical bounds for specific quadratic problems. We analyze inherent trade-offs between privacy and suboptimality and propose allocation schemes to divide the maximum allowable noise, a privacy budget, among all participating agents. Our algorithm is implemented to enable privacy in distributed optimal power flow for electric grids.

math.OC↗

arXiv2020

Information-theoretic analysis for transfer learning

Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different distributions (denoted as $μ$ and $μ'$, respectively). In this work, we give an information-theoretic analysis on the generalization error and the excess risk of transfer learning algorithms, following a line of work initiated by Russo and Zhou. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence $D(mu||mu')$ plays an important role in characterizing the generalization error in the settings of domain adaptation. Specifically, we provide generalization error upper bounds for general transfer learning algorithms and extend the results to a specific empirical risk minimization (ERM) algorithm where data from both distributions are available in the training phase. We further apply the method to iterative, noisy gradient descent algorithms, and obtain upper bounds which can be easily calculated, only using parameters from the learning algorithms. A few illustrative examples are provided to demonstrate the usefulness of the results. In particular, our bound is tighter in specific classification problems than the bound derived using Rademacher complexity.

cs.LG↗

arXiv2019

Analysis of Nakamoto Consensus, Revisited

In the Bitcoin white paper, Nakamoto proposed a very simple Byzantine fault tolerant consensus algorithm that is also known as Nakamoto consensus. Despite its simplicity, some existing analysis of Nakamoto consensus appears to be long and involved. In this technical report, we aim to make such analysis simple and transparent so that we can teach senior undergraduate students and graduate students in our institutions. This report is largely based on a 3-hour tutorial given by one of the authors in June 2019.

cs.CR↗

arXiv2017

Compute--Forward Multiple Access (CFMA): Practical Code Design

We present a practical strategy that aims to attain rate points on the dominant face of the multiple access channel capacity using a standard low complexity decoder. This technique is built upon recent theoretical developments of Zhu and Gastpar on compute-forward multiple access (CFMA) which achieves the capacity of the multiple access channel using a sequential decoder. We illustrate this strategy with off-the-shelf LDPC codes. In the first stage of decoding, the receiver first recovers a linear combination of the transmitted codewords using the sum-product algorithm (SPA). In the second stage, by using the recovered sum-of-codewords as side information, the receiver recovers one of the two codewords using a modified SPA, ultimately recovering both codewords. The main benefit of recovering the sum-of-codewords instead of the codeword itself is that it allows to attain points on the dominant face of the multiple access channel capacity without the need of rate-splitting or time sharing while maintaining a low complexity in the order of a standard point-to-point decoder. This property is also shown to be crucial for some applications, e.g., interference channels. For all the simulations with single-layer binary codes, our proposed practical strategy is shown to be within \SI{1.7}{\decibel} of the theoretical limits, without explicit optimization on the off-the-self LDPC codes.

cs.IT↗

arXiv2017

A Sequential Approximation Framework for Coded Distributed Optimization

Building on the previous work of Lee et al. and Ferdinand et al. on coded computation, we propose a sequential approximation framework for solving optimization problems in a distributed manner. In a distributed computation system, latency caused by individual processors ("stragglers") usually causes a significant delay in the overall process. The proposed method is powered by a sequential computation scheme, which is designed specifically for systems with stragglers. This scheme has the desirable property that the user is guaranteed to receive useful (approximate) computation results whenever a processor finishes its subtask, even in the presence of uncertain latency. In this paper, we give a coding theorem for sequentially computing matrix-vector multiplications, and the optimality of this coding scheme is also established. As an application of the results, we demonstrate solving optimization problems using a sequential approximation approach, which accelerates the algorithm in a distributed system with stragglers.

cs.IT↗