Symmetry of solutions to nonlocal nonlinear boundary value problems in radial sets

Abstract

For open radial sets $${\Omega \subset {\mathbb {R}}^N}$$ , $${N\geq 2}$$ we consider the nonlinear problem $$(P)\qquad\left\{\begin{array}{ll}I u = f(|x|,u)& \;\, \rm{ in } ~\Omega,\\ u \equiv 0 &\,\,\, \text{on}~ \mathbb{R}^{N}{\setminus} \Omega,\\ \lim_{|x|\to\infty}u(x) = 0,&\end{array}\right.$$ where I is a nonlocal operator and f is a nonlinearity. Under mild symmetry and monotonicity assumptions on I, f and $${\Omega}$$ we show that any continuous bounded solution of (P) is axial symmetric once it satisfies a simple reflection inequality with respect to a hyperplane. In the special case where f does not depend on |x|, we show that any nonnegative nontrivial continuous bounded solution of (P) in $${\mathbb {R}^N}$$ is radially symmetric (up to translation) and strictly decreasing in its radial direction. Our proves rely on different variants of maximum principles for antisymmetric supersolutions which can be seen as extensions of the results in Jarohs and Weth (Ann Mat Pura Appl 195:273–291, 2016). As an application, we prove an axial symmetry result for minimizers of an energy functional associated to (P).