The Judgment Game: The Turing Test as a General Research Framework

Abstract

Turing’s three-participant “Imitation Game,” is revisited and the probabilistic and temporal nature of the game is formalized. It is argued that Turing-like games can be used as general tests of distinguishability between levels of knowledge/performance in learned subject areas along the expertise and interpersonal dimensions. A modification of the Imitation Game called the “Judgment Game,” is introduced, which also has three participants—an interrogator (Judge), who must distinguish between two players A & B. The players, despite inherent differences, attempt to be indistinguishable to the Judge. However, in the Judgment Game, the focus is on the success of the Judge, which can be scored and compared to other judges. Because the Judge’s “job” is mostly interpersonal, such a game provides a method to assess knowledgebases/skillsets, along the interpersonal dimension, which has been problematic in the education of humans and machines. A machine in the role of the Judge in a Judgement Game, might be a higher standard for machine cognition than a machine in the role of Player A, as in Turing’s original Imitation Game. However, these tests provide a restrictive definition of “thinking” because thought processes involved with learning must take place before these games can be played.