An application of Holmgren’s theorem and convexity with respect to differential operators with flat characteristic cones

Abstract

At a fixed point x? E Rn the (real) zeroes of Pm(xO, 6) form a cone in Rn which is called the (real) characteristic cone of P(x, D) at x?. In this paper we study only differential operators with flat characteristic cones. Specifically we assume that at each point of an open set Q the characteristic cone of P(x, D) is contained in the orthogonal complement of a fixed subspace W of Rn different from {O}. In ?2, by a repeated application of llolmgren's theorem (as extended to distribution solutions by Hormander [1]), we show that if P(x, D) has analytic coefficients in Q and if u is a distribution solution of P(x, D)u = 0 in Q which vanishes in an open subset QO of Q, then u must also vanish in the Wextension of QO in Q consisting of all points of Q which can be connected to points of QO by polygonal paths in Q with sides defining vectors in W. We consider next the problem of characterizing geometrically the open sets in Rn which are P-convex with respect to a differential operator P(D) with constant coefficients. An open set Q in Rn is called P-convex if to every compact subset K1 of Q there exists a compact subset K2 of Q such that for every u E &'(Q) (distribution with compact support in Q),