Scoring Divergent Thinking Tests by Computer With a Semantics-Based Algorithm.

Kenes Beketayev,Mark A Runco

doi:10.5964/ejop.v12i2.1127

Abstract

Divergent thinking (DT) tests are useful for the assessment of creative potentials. This article reports the semantics-based algorithmic (SBA) method for assessing DT. This algorithm is fully automated: Examinees receive DT questions on a computer or mobile device and their ideas are immediately compared with norms and semantic networks. This investigation compared the scores generated by the SBA method with the traditional methods of scoring DT (i.e., fluency, originality, and flexibility). Data were collected from 250 examinees using the “Many Uses Test” of DT. The most important finding involved the flexibility scores from both scoring methods. This was critical because semantic networks are based on conceptual structures, and thus a high SBA score should be highly correlated with the traditional flexibility score from DT tests. Results confirmed this correlation (r = .74). This supports the use of algorithmic scoring of DT. The nearly-immediate computation time required by SBA method may make it the method of choice, especially when it comes to moderate- and large-scale DT assessment investigations. Correlations between SBA scores and GPA were insignificant, providing evidence of the discriminant and construct validity of SBA scores. Limitations of the present study and directions for future research are offered.

Highlights

Divergent thinking (DT) tests are useful for the assessment of creative potentials
Divergent thinking (DT) is not synonymous with creativity but tests of DT do provide useful information about creative potential. They provide scores for ideational fluency, which represents the number of ideas an individual gives, ideational flexibility, which represents the number of different conceptual categories used by the individual, and ideational originality, which represents the statistical infrequency or uniqueness of ideas
As more data are processed over time, the underlying semantic statistics of semantics-based algorithmic (SBA) method are expected to become more robust, which will no doubt lead to improved reliability of the SBA scores

Summary

Introduction

Divergent thinking (DT) tests are useful for the assessment of creative potentials. This article reports the semantics-based algorithmic (SBA) method for assessing DT. The most important finding involved the flexibility scores from both scoring methods This was critical because semantic networks are based on conceptual structures, and a high SBA score should be highly correlated with the traditional flexibility score from DT tests. Acar and Runco (2014), for example, gave divergent thinking tests to a group of individuals via computer and scored these tests using three semantic networks They were especially interested in associative distance. By far the most comprehensive computerbased method for scoring divergent thinking tests is the semantic-based algorithm (SBA) developed by SparcIt (http://cit.sparcit.com). It uses 12 semantic networks when coding ideas and continually improves over time as it processes more data. It is analogous to having a larger normative sample when interpreting some test result or score

Methods

Results

Conclusion