Fractional Laplacians on domains, a development of Hörmander's theory of μ-transmission pseudodifferential operators

Abstract

Let P be a classical pseudodifferential operator of order m∈C on an n-dimensional C∞ manifold Ω1. For the truncation PΩ to a smooth subset Ω there is a well-known theory of boundary value problems when PΩ has the transmission property (preserves C∞(Ω¯)) and is of integer order; the calculus of Boutet de Monvel. Many interesting operators, such as for example complex powers of the Laplacian (−Δ)μ with μ∉Z, are not covered. They have instead the μ-transmission property defined in Hörmander's books, mapping xnμC∞(Ω¯) into C∞(Ω¯). In an unpublished lecture note from 1965, Hörmander described an L2-solvability theory for μ-transmission operators, departing from Vishik and Eskin's results. We here develop the theory in Lp Sobolev spaces (1<p<∞) in a modern setting. It leads to not only Fredholm solvability statements but also regularity results in full scales of Sobolev spaces (s→∞). The solution spaces have a singularity at the boundary that we describe in detail. We moreover obtain results in Hölder spaces, which radically improve recent regularity results for fractional Laplacians.