In this paper we introduce new characterizations of spectral fractional Laplacian to incorporate nonhomogeneous Dirichlet and Neumann boundary conditions. The classical cases with homogeneous boundary conditions arise as a special case. We apply our definition to fractional elliptic equations of order $s \in (0,1)$ with nonzero Dirichlet and Neumann boundary condition. Here the domain $\Omega$ is assumed to be a bounded, quasi-convex Lipschitz domain. To impose the nonzero boundary conditions, we construct fractional harmonic extensions of the boundary data. It is shown that solving for the fractional harmonic extension is equivalent to solving for the standard harmonic extension in the very-weak form. The latter result is of independent interest as well. The remaining fractional elliptic problem (with homogeneous boundary data) can be realized using the existing techniques. We introduce finite element discretizations and derive discretization error estimates in natural norms, which are confirmed by numerical experiments. We also apply our characterizations to Dirichlet and Neumann boundary optimal control problems with fractional elliptic equation as constraints.