In generalized inner product Sobolev spaces we investigate elliptic differential problems with additional unknown functions or distributions in boundary conditions. These spaces are parametrized with a function OR-varying at infinity. This characterizes the regularity of distributions more finely than the number parameter used for the Sobolev spaces. We prove that these problems induce Fredholm bounded operators on appropriate pairs of the above spaces. Investigating generalized solutions to the problems, we prove theorems on their regularity and a priori estimates in these spaces. As an application, we find new sufficient conditions under which components of these solutions have continuous classical derivatives of given orders. We assume that the orders of boundary differential operators may be equal to or greater than the order of the relevant elliptic equation.