Two-sided estimates for the transition densities of symmetric Markov processes dominated by stable-like processes in $C^{1,\eta}$ open sets

Abstract

In this paper, we study sharp Dirichlet heat kernel estimates for a large class of symmetric Markov processes in $C^{1,\eta}$ open sets. The processes are symmetric pure jump Markov processes with jumping intensity $\kappa(x,y) \psi_1 (|x-y|)^{-1} |x-y|^{-d-\alpha}$, where $\alpha \in (0,2)$. Here, $\psi_1$ is an increasing function on $[ 0, \infty )$, with $\psi_1(r)=1$ on $0 1$ for $\beta \in [0,\infty]$, and $ \kappa( x, y)$ is a symmetric function confined between two positive constants, with $|\kappa(x,y)-\kappa(x,x)|\leq c_5|x-y|^{\rho}$ for $|x-y|<1$ and $\rho>\alpha/2$. We establish two-sided estimates for the transition densities of such processes in $C^{1,\eta}$ open sets when $\eta \in (\alpha/2, 1]$. In particular, our result includes (relativistic) symmetric stable processes and finite-range stable processes in $C^{1,\eta}$ open sets when $\eta \in (\alpha/2, 1]$.