Generalized 3G theorem and application to relativistic stable process on non-smooth open sets

Abstract

Let G ( x , y ) and G D ( x , y ) be the Green functions of rotationally invariant symmetric α-stable process in R d and in an open set D, respectively, where 0 < α < 2 . The inequality G D ( x , y ) G D ( y , z ) / G D ( x , z ) ⩽ c ( G ( x , y ) + G ( y , z ) ) is a very useful tool in studying (local) Schrödinger operators. When the above inequality is true with c = c ( D ) ∈ ( 0 , ∞ ) , then we say that the 3G theorem holds in D. In this paper, we establish a generalized version of 3G theorem when D is a bounded κ-fat open set, which includes a bounded John domain. The 3G we consider is of the form G D ( x , y ) G D ( z , w ) / G D ( x , w ) , where y may be different from z. When y = z , we recover the usual 3G. The 3G form G D ( x , y ) G D ( z , w ) / G D ( x , w ) appears in non-local Schrödinger operator theory. Using our generalized 3G theorem, we give a concrete class of functions belonging to the non-local Kato class, introduced by Chen and Song, on κ-fat open sets. As an application, we discuss relativistic α-stable processes (relativistic Hamiltonian when α = 1 ) in κ-fat open sets. We identify the Martin boundary and the minimal Martin boundary with the Euclidean boundary for relativistic α-stable processes in κ-fat open sets. Furthermore, we show that relative Fatou type theorem is true for relativistic stable processes in κ-fat open sets. The main results of this paper hold for a large class of symmetric Markov processes, as are illustrated in the last section of this paper. We also discuss the generalized 3G theorem for a large class of symmetric stable Lévy processes.