Searcharxiv⌕ Search

arXiv subjects

Waqar Mirza

Publications and source records attributed to Waqar Mirza.

3 recordsLinked to original sources

Finite Time Analysis of Risk-Sensitive RL via Noisy Power Iteration

Estimating the principal eigenpair of a positive operator from stochastic evaluations gives rise to a noisy form of normalized power iteration: each step applies a sampled operator estimate and then renormalizes. We establish finite-time stochastic-approximation guarantees for this procedure, working in Hilbert's projective metric, which is well-suited to the multiplicative geometry of Perron--Frobenius eigenproblems. Our main application is risk-sensitive average-cost reinforcement learning (RL) with exponential utility. In this setting the Bellman equations are multiplicative, and value functions are characterized by nonlinear eigenvalue problems rather than additive fixed points. We cast both policy evaluation and control in this framework, obtaining risk-sensitive TD- and Q-learning algorithms that learn from a single Markovian trajectory. Under explicit positivity, mixing, and linear function approximation assumptions, we prove finite-time bounds on the recovered eigenvector and eigenvalue with $\tilde{O}(ε^{-2})$ sample complexity up to problem-dependent constants. The results cover both tabular and linear function approximation regimes, and through the duality between exponential utility and KL-robust control yield finite-time guarantees for KL-robust average-cost RL as well.

cs.LG↗

Cascaded Group Testing

In this paper, we introduce a variation of the group testing problem where each test is specified by an ordered subset of items and returns the first defective item in the specified order or returns null if there are no defectives. We refer to this as cascaded group testing and the goal is to identify a small set of $K$ defective items amongst a collection of size $N$, using as few tests as possible for perfect recovery. For the adaptive testing regime, we show that a simple scheme can find all defective items in at most $K$ tests, which is optimal. For the non-adaptive setting, we first come up with a necessary and sufficient condition for any collection of tests to be feasible for recovering all the defectives. Using this, we show that any feasible non-adaptive strategy requires at least $Ω(K^2)$ tests. In terms of achievability, it is easy to show the existence of a feasible collection of $O(K^2 \log (N/K))$ tests. We show via carefully constructed explicit designs that one can do significantly better for constant $K$. While the cases $K = 1, 2$ are straightforward, the case $K=3$ is already non-trivial and we come up with an iterative design that is asymptotically optimal and requires $Θ(\log \log N)$ tests. Note that this is in contrast to standard binary group testing, where at least $Ω(\log N)$ tests are required. For constant $K \ge 3$, our iterative design requires only poly$(\log \log N)$ tests.

cs.IT↗

Estimating Joint Probability Distribution With Low-Rank Tensor Decomposition, Radon Transforms and Dictionaries

In this paper, we describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components. Prior works have used such a decomposition to estimate the joint density from lower-dimensional marginals, which can be estimated more reliably with the same number of samples. We combine two key ideas: dictionaries to represent 1-D densities, and random projections to estimate the joint distribution from 1-D marginals, explored separately in prior work. Our algorithm benefits from improved sample complexity over the previous dictionary-based approach by using 1-D marginals for reconstruction. We evaluate the performance of our method on estimating synthetic probability densities and compare it with the previous dictionary-based approach and Gaussian Mixture Models (GMMs). Our algorithm outperforms these other approaches in all the experimental settings.

stat.ML↗