Deep learning models achieve state-of-the-art performance in many applications, but their prediction decisions are difficult to explain. Various solutions exist in the area of explainable AI, for example to understand individual predictions or to approximate complex models using simpler interpretable ones. We contribute to this body of work with POEM: a tool that produces pattern-oriented explanations of image classification models. POEM explains models that learn hierarchies of concepts, such as Convolutional Neural Networks that detect shapes and objects in images. For example, POEM may identify a pattern of the form "if bed then bedroom", indicating that if an image contains a bed and the model pays attention to this region of the image during inference, then the model classifies the image as a bedroom. We present the modular design of POEM, followed by examples of POEM's use in model auditing and detecting errors in training data.