Searcharxiv⌕ Search

arXiv subjects

Zhong-Wei Liao

Publications and source records attributed to Zhong-Wei Liao.

9 recordsLinked to original sources
arXiv2023

Viscosity solutions approach to finite-horizon continuous-time Markov decision process

This paper investigates the optimal control problems for the finite-horizon continuous-time Markov decision processes with delay-dependent control policies. We develop compactification methods in decision processes, and show that the existence of optimal policies. Subsequently, through the dynamic programming principle of the delay-dependent control policies, the differential-difference Hamilton-Jacobi-Bellman (HJB) equation in the setting of discrete space was established. Under certain conditions, we give the comparison principle and further prove that the value function is the unique viscosity solution to this HJB equation. Based on this, we show that among the class of delay-dependent control policies, there is an optimal one which is Markovian.

math.PR↗
arXiv2021

Discounted semi-Markov games with incomplete information on one side

This work considers two-player zero-sum semi-Markov games with incomplete information on one side and perfect observation. At the beginning, the system selects a game type according to a given probability distribution and informs to Player 1 only. After each stage, the actions chosen are observed by both players before proceeding to the next stage. Firstly, we show the existence of the value function under the expected discount criterion and the optimality equation. Secondly, the existence and iterative algorithm of the optimal policy for Player 1 are introduced through the optimality equation of value function. Moreove, About the optimal policy for the uninformed Player 2, we define the auxiliary dual games and construct a new optimality equation for the value function in the dual games, which implies the existence of the optimal policy for Player 2 in the dual game. Finally, the existence and iterative algorithm of the optimal policy for Player 2 in the original game is given by the results of the dual game.

math.OC↗
arXiv2021

Optimal stopping time on semi-Markov processes with finite horizon

In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main results of finite horizon semi-Markov decision processes (SMDPs) to the case with additional terminal costs, introduce an explicit construction of SMDPs, and prove the equivalence between the optimal stopping problems on SMPs and SMDPs. Then, using the equivalence and the results on SMDPs developed here, we not only show the existence of optimal stopping time of SMPs, but also provide an algorithm for computing optimal stopping time on SMPs. Moreover, we show that the optimal and "-optimal stopping time can be characterized by the hitting time of some special sets, respectively.

math.PR↗
arXiv2021

Optimal stopping time on discounted semi-Markov processes

This paper attempts to study the optimal stopping time for semi-Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the value functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value function and develop an effective iterative algorithm for computing it. Moreover, we show that the optimal and ε-optimal stopping time can be characterized by the hitting time of the special sets. Finally, to illustrate the validity of our results, an example of a maintenance system is presented in the end.

math.PR↗
arXiv2020

Estimate the exponential convergence rate of f-ergodicity via spectral gap

This paper studies the f-ergodicity and its exponential convergence rate for continuous-time Markov chain. Assume f is square integrable, for reversible Markov chain, it is proved that the exponential convergence of f-ergodicity holds if and only if the spectral gap of the generator is positive. Moreover, the convergence rate is equal to the spectral gap. For irreversible case, the positivity of spectral gap remains a sufficient condition of f-ergodicity. The effectiveness of these results are illustrated by some typical examples.

math.PR↗
arXiv2019

Long time behavior of Levy-driven Ornstein-Uhlenbeck process with regime-switchin

In this work we investigate the long time behavior of the Ornstein-Uhlenbeck process driven by Levy noise with regime-switching. We provide explicit criteria on the transience and recurrence of this process. Contrasted with the Ornstein-Uhlenbeck process driven simply by Brownian motion, whose stationary distribution must be light-tailed, both the jumps caused by the Levy noise and regime-switching described by Markov chain can derive the heavy-tailed property of the stationary distribution. In this work, the different role played by Levy measure and regime-switching process is clearly characterized.

math.PR↗
arXiv2015

Discrete Weighted Hardy Inequalities with Different Kinds of Boundary Conditions

This paper studies the weighted Hardy inequalities on the discrete intervals with four different kinds of boundary conditions. The main result is the uniform expression of the basic estimate of the optimal constant with the corresponding boundary condition. Firstly, one-side boundary condition is considered, which means that the sequences vanish at the right endpoint (ND-case). Based on the dual method, it can be translated into the case vanishing at left endpoint (DN-case). Secondly, the condition is the case that the sequences vanish at two endpoints (DD-case). The third type of condition is the generality of the mean zero condition (NN-case), which is motivated from probability theory. To deal with the second and the third kinds of inequalities, the splitting technique is presented. Finally, as typical applications, some examples are included.

math.FA↗
arXiv2014

Discrete Hardy-type Inequalities

This paper studies the Hardy-type inequalities on the discrete intervals. The first result is the variational formulas of the optimal constants. Using these formulas, one may obtain an approximating procedure and the known basic estimates of the optimal constants. The second result, which is the main innovation of this paper, is about the factor of basic upper estimates. An improved factor is presented, which is smaller than the known one and is best possible. Some comparison results are included for comparing the optimal constants on different intervals.

math.FA↗