For $3$-dimensional convex polytopes, inscribability is a classical property that is relatively well-understood due to its relation with Delaunay subdivisions of the plane and hyperbolic geometry. In particular, inscribability can be tested in polynomial time, and for every $f$-vector of $3$-polytopes, there exists an inscribable polytope with that $f$-vector. For higher-dimensional polytopes, much less is known. Of course, for any inscribable polytope, all of its lower-dimensional faces need to be inscribable, but this condition does not appear to be very strong. We observe non-trivial new obstructions to the inscribability of polytopes that arise when imposing that a certain inscribable face be inscribed. Using this obstruction, we show that the duals of the $4$-dimensional cyclic polytopes with at least $8$ vertices---all of whose faces are inscribable---are not inscribable. This result is optimal in the following sense: We prove that the duals of the cyclic $4$-polytopes with up to $7$ vertices are, in fact, inscribable. Moreover, we interpret this obstruction combinatorially as a forbidden subposet of the face lattice of a polytope, show that $d$-dimensional cyclic polytopes with at least $d+4$ vertices are not circumscribable, and that no dual of a neighborly $4$-polytope with $8$ vertices, that is, no polytope with $f$-vector $(20,40,28,8)$, is inscribable.