Searcharxiv⌕ Search

arXiv subjects

Keqin Liu

Publications and source records attributed to Keqin Liu.

At least 19 recordsLinked to original sources
arXiv2026

From Relaxed Indexability to Exact Indexability: A $t$-Step Approach for Partially Observable Restless Bandits

Whittle index policies offer a scalable method for restless multi-armed bandits, but under partial observability even determining the indifference subsidy at a single belief requires solving an infinite-horizon belief-state problem with no closed-form value function. Liu [10] addresses this difficulty by linearizing the unknown decision boundary, leading to a linear system and a closed-form approximate Whittle index. However, the resulting threshold uses only a one-step active--passive comparison and does not account for longer-horizon continuation values. We extend this framework to a \emph{$t$-step lookahead threshold policy}. For each subsidy $m$, the threshold is defined by the active-minus-passive advantage under $t$-step finite-horizon value iteration. At $t=1$, the threshold is $m$-independent and recovers the linear threshold of Liu [10]; for $t>1$, it becomes subsidy-dependent through the induced first-crossing structure and tracks the exact decision boundary more closely. The proposed algorithmic framework does not require indexability as an input and includes an indexability verification. Under the original Whittle indexability, we prove that the $t$-step approximate Whittle index converges geometrically to the exact Whittle index, \[ |\widehat W_t(\omega)-W(\omega)|=O(\beta^t). \] Numerically, all 2,715 tested three-state instances are verified with computable priority index functions according to the proposed criterion. The P95 index error decreases from $2.18\times10^{-2}$ at $t=1$ to $8.93\times10^{-4}$ at $t=8$. In an exact-comparable instance with $\beta=0.9999$, $t=2$ already recovers the exact Whittle-index ordering. Moderate-depth threshold policies also outperform the one-step baseline and remain close to the optimal dynamic-programming benchmark, while runtime grows mildly with $t$.

cs.LG↗
arXiv2026

On a determinant involving linear combinations of Legendre symbols

In this paper, we prove a conjecture of the second author by evaluating the determinant $$\det\left[x + \left(\frac{i-j}p\right) + \left(\frac ip\right)y + \left(\frac jp\right)z + \left(\frac{ij}p\right)w\right]_{0\le i,j\le(p-3)/2}$$ for any odd prime $p$, where $(\frac{\cdot}p)$ denotes the Legendre symbol. In particular, the determinant is equal to $x$ when $p\equiv 3\pmod4$.

math.NT↗
arXiv2026

Basic Zero Divisor Mathematics (I)

We develop refinements of inner products, the Gram-Schmidt process, root system, Hilbert space and Maschke's Theorem in the context of zero divisor mathematics. We then introduce the concept of respecting nilpotent step representations of Lie algebras and give the new refinement of Ado's Theorem.

math.RA↗
arXiv2026

Nonlinear Bandit

In this paper we first study the problem of generalized linear bandit (GLB) under heavy-tailed noise. The characteristics of heavy-tailed distributions are widely observed in real-world applications such as personalized recommendation, financial markets, and medical treatments. Based on the online mirror descent (OMD) method, we propose an algorithm EHM that extends the adaptive Huber loss method (Wang et al., 2025) with one-pass update ($\mathcal{O}(1)$ computational complexity with respect to current round $t$ and the time horizon $T$), which simultaneously achieves an almost optimal regret of $\widetilde{\mathcal{O}}(T^{\frac{1}{1+ε}})$ where $T$ is the time horizon. In addition, by utilizing a special property of some link function (Sawarni et al., 2025), our algorithm eliminates the need to know a commonly used parameter. Next, we study the GLB problem under the case when contextual characteristic becomes piecewise constant, and we slightly revised former algorithm to obtain the PGLB-EHM algorithm. After theoretical analysis, we prove that the regret upper bound order stays the same. Furthermore, we look deeper into a special case of nonlinear bandit (NB) and present the NB-EHM algorithm with bisection method and special restriction. Eventually we utilize the affine lifting approach and show that the general NB problem can be applied with NB-EHM to achieve a sublinear regret bound.

cs.LG↗
arXiv2025

General Formulation and PCL-Analysis for Restless Bandits with Limited Observability

In this paper, we consider a general observation model for restless multi-armed bandit problems. The operation of the player is based on the past observation history that is limited (partial) and error-prone due to resource constraints or environmental or intrinsic noises. By establishing a general probabilistic model for dynamics of the observation process, we formulate the problem as a restless bandit with an infinite high-dimensional belief state space. We apply the achievable region method with partial conservation law (PCL) to the infinite-state problem and analyze its indexability and priority index (Whittle index). Finally, we propose an approximation process to transform the problem into which the AG algorithm of Niño-Mora (2001) for finite-state problems can be applied. Numerical experiments show that our algorithm has excellent performance.

stat.ML↗
arXiv2025

Extended UCB Policies for Multi-armed Bandit Problems

The multi-armed bandit (MAB) problems are widely studied in fields of operations research, stochastic optimization, and reinforcement learning. In this paper, we consider the classical MAB model with heavy-tailed reward distributions and introduce the extended robust UCB policy, which is an extension of the results of Bubeck et al. [5] and Lattimore [22] that are further based on the pioneering idea of UCB policies [e.g. Auer et al. 3]. The previous UCB policies require some strict conditions on reward distributions, which can be difficult to guarantee in practical scenarios. Our extended robust UCB generalizes Lattimore's seminary work (for moments of orders $p=4$ and $q=2$) to arbitrarily chosen $p>q>1$ as long as the two moments have a known controlled relationship, while still achieving the optimal regret growth order $O(log T)$, thus providing a broadened application area of UCB policies for heavy-tailed reward distributions. Furthermore, we achieve a near-optimal regret order without any knowledge of the reward distributions as long as their $p$-th moments exist for some $p>1$. Finally, we briefly present our earlier work on light-tailed reward distributions for a complete illustration of the amazing simplicity and power of UCB policies.

cs.LG↗
arXiv2025

Zero Divisor Manifolds

We develop the basic properties of $R^{(2)}$-modules, introduce the concept of zero divisor manifolds, construct projective $R^{(2)}$-space which generalizes the real projective space, and initiate the study of the counterpart of symplectic spaces

math.SG↗
arXiv2025

Generalization and Alternative Proof of Two Identities Posed by Sun

We study two identities involving roots of unity and determinants of Hermitian matrices which have been recently proved by using the famous eigenvector-eigenvalue identity for normal matrices. In this paper, we extend these identities to a more general form by considering the class of circulant matrices. Furthermore, we give an alternative proof of Sun's identities independent of the eigenvector-eigenvalue identity, where our strategy is built upon the similarity of an unnecessarily normal matrix to a particular matrix with integer eigenvalues, derived from the Fourier transform vectors.

math.GM↗
arXiv2025

Relaxed Indexability and Index Policy for Partially Observable Restless Bandits

This paper addresses an important class of restless multi-armed bandit (RMAB) problems that finds broad application in operations research, stochastic optimization, and reinforcement learning. There are $N$ independent Markov processes that may be operated, observed and offer rewards. Due to the resource constraint, we can only choose a subset of $M~(M<N)$ processes to operate and accrue reward determined by the states of selected processes. We formulate the problem as a partially observable RMAB with an infinite state space and design an algorithm that achieves a near-optimal performance with low complexity. Our algorithm is based on a generalization of Whittle's original idea of indexability. Referred to as the relaxed indexability, the extended definition leads to the efficient online verifications and computations of the approximate Whittle index under the proposed algorithmic framework.

math.OC↗
arXiv2024

Efficient Algorithm Design of Dynamic Spectrum Access by Whittle Index

Dynamic spectrum access problem is an important problem that allows a wireless sub-network to use channels temporarily unoccupied by the parent network for minimizing the spectrum waste. Previous work has shown that the sequential channel allocation problem for the sub-network can be formulated within the restless multi-armed bandits (RMAB) framework. The objective is to maximize the expected long-term return over an infinite horizon while minimizing interference to the parent network. Different from the previous work that exploits a binary feedback (e.g., ACK/NAK) to compensate for sensing errors, we leverage the finer and more robust channel quality indicator (CQI) feedback to update the information state (belief vector) of the sub-network. However, the implementation of CQI-based observation model yields significantly more complex belief transition behaviors in an infinite state space and worsens the curse of dimensionality of dynamic programming. To overcome this challenge, we dive into the rich structures of the value functions and obtain tight bounds on their derivatives. These results lead to the proof of optimality of threshold policies on a single-channel problem with subsidy and subsequently a closed-form channel index function using an iterative method to approximate the well-known Whittle index policy, which offers a low-complexity solution for ranking the currently available channels whose states are never directly observable. Through extensive numerical studies, we demonstrate the superior performance and robustness of our proposed algorithm.

math.OC↗
arXiv2024

Low-Complexity Algorithm for Restless Bandits with Imperfect Observations

We consider a class of restless bandit problems that finds a broad application area in reinforcement learning and stochastic optimization. We consider $N$ independent discrete-time Markov processes, each of which had two possible states: 1 and 0 (`good' and `bad'). Only if a process is both in state 1 and observed to be so does reward accrue. The aim is to maximize the expected discounted sum of returns over the infinite horizon subject to a constraint that only $M$ $(<N)$ processes may be observed at each step. Observation is error-prone: there are known probabilities that state 1 (0) will be observed as 0 (1). From this one knows, at any time $t$, a probability that process $i$ is in state 1. The resulting system may be modeled as a restless multi-armed bandit problem with an information state space of uncountable cardinality. Restless bandit problems with even finite state spaces are PSPACE-HARD in general. We propose a novel approach for simplifying the dynamic programming equations of this class of restless bandits and develop a low-complexity algorithm that achieves a strong performance and is readily extensible to the general restless bandit model with observation errors. Under certain conditions, we establish the existence (indexability) of Whittle index and its equivalence to our algorithm. When those conditions do not hold, we show by numerical experiments the near-optimal performance of our algorithm in the general parametric space. Furthermore, we theoretically prove the optimality of our algorithm for homogeneous systems.

cs.LG↗
arXiv2023

Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training

Regularization in modern machine learning is crucial, and it can take various forms in algorithmic design: training set, model family, error function, regularization terms, and optimizations. In particular, the learning rate, which can be interpreted as a temperature-like parameter within the statistical mechanics of learning, plays a crucial role in neural network training. Indeed, many widely adopted training strategies basically just define the decay of the learning rate over time. This process can be interpreted as decreasing a temperature, using either a global learning rate (for the entire model) or a learning rate that varies for each parameter. This paper proposes TempBalance, a straightforward yet effective layer-wise learning rate method. TempBalance is based on Heavy-Tailed Self-Regularization (HT-SR) Theory, an approach which characterizes the implicit self-regularization of different layers in trained models. We demonstrate the efficacy of using HT-SR-motivated metrics to guide the scheduling and balancing of temperature across all network layers during model training, resulting in improved performance during testing. We implement TempBalance on CIFAR10, CIFAR100, SVHN, and TinyImageNet datasets using ResNets, VGGs, and WideResNets with various depths and widths. Our results show that TempBalance significantly outperforms ordinary SGD and carefully-tuned spectral norm regularization. We also show that TempBalance outperforms a number of state-of-the-art optimizers and learning rate schedulers.

cs.LG↗
arXiv2022

Noncommutative Partial Derivative

We introduce the axiomatic definition of the point-derivative for noncommutative algebras and present the counterparts of the ordinary multi-variable chain rule and Clairaut's Theorem in the context of partial point-derivatives.

math.RA↗
arXiv2021

A Data Analysis Study on Human Liver Blood Circulation

The liver has a unique blood supply system and plays an important role in the human blood circulatory system. Thus, hemodynamic problems related to the liver serve as an important part in clinical diagnosis and treatment. Although estimating parameters in these hemodynamic models is essential to the study of liver models, due to the limitations of medical measurement methods and constraints of ethics on clinical studies, it is impossible to directly measure the parameters of blood vessels in livers. Furthermore, as an important part of the systemic blood circulation, livers' studies are supposed to be in conjunction with other blood vessels. In this article, we present an innovative method to fix parameters of an individual liver in a human blood circulation using non-invasive clinical measurements. The method consists of a 1-D blood flow model of human arteries and veins, a 0-D model reflecting the peripheral resistance of capillaries and a lumped parameter circuit model for human livers. We apply the finite element method in fluid mechanics of these models to a numerical study, based on non-invasive blood related measures of 33 individuals. The estimated results of human blood vessel characteristic and liver model parameters are verified from the perspective of Stroke Value Variation, which shows the effectiveness of our estimation method.

math.NA↗
arXiv2020

Automatic Integration

The purpose of this paper is to introduce the concept of the automatic integration and present a new way of approximating definite integrals using the automatic integration based on an associative algebra with zero divisors.

math.GM↗
arXiv2020

How to Define Automatic Differentiation

Based on a class of associative algebras with zero-divisors which are called real-like algebras by us, we introduce a way of defining automatic differentiation and present different ways of doing automatic differentiation to compute the first, the second and the third derivatives of a function exactly and simultaneously.

math.NA↗
arXiv2018

Calculus on Dual Real Numbers

We present the basic theory of calculus on dual real numbers, and prove the counterpart of the ordinary fundamental theorem of calculus in the context of dual real numbers.

math.CA↗