We introduce the notion of a von Neumann category, as a multi-object version of von Neumann algebra. A von Neumann category is a premonoidal category with compatible ∗-structure which embeds as a double commutant into a suitable premonoidal category of Hilbert spaces. The notion was inspired by algebraic quantum field theory, and we do explain that basic idea and indicate how quantum teleportation could be encoded in this setting. But we focus here on the structure of von Neumann categories. After giving the definitions and examples, we consider constructions typically associated to von Neumann algebras, and examine their extensions to the category-theoretic setting. In particular, we present a crossed product construction for ∗-premonoidal categories.