Evaluating creativity in humans, computers, and collectively intelligent systems

Abstract

Creativity studies focus on the processes that produce creative artifacts and how we evaluate an artifact to determine if it is creative. This paper focuses on the essential criteria in evaluating if a potentially creative artifact is creative. Evaluating creativity is still largely subjective and not well supported with computational tools. An evaluation metric is presented as a way of measuring three essential criteria for creativity: novelty, value, and unexpectedness. The metric is independent of the domain or discipline and does not depend on whether the system producing the creative artifact is a person, a computer, or a combination of human and computer agents. Novelty is a measure of the distance from other artifacts in the space, characterizing the artifact as similar but different. To distinguish this from novelty, value is a measure of the artifact's performance or acceptance rather than a measure of how the artifact's description differs from other artifacts in its class. A metric for value has to accommodate that a creative artifact can change the value system by introducing a performance or function that did not exist in the class of known artifacts. Unexpectedness is measured by how far the artifact is from the expected next artifact.