A modified Delphi approach to a new card sorting methodology

Abstract

Open card sorting is used by information architects to gather insights from users to incorporate feedback into an information architecture. In theory, it is one of the more inexpensive, user-centered design methods available to practitioners, but hidden costs make it less likely to be conducted properly and affect the quality of results produced. The following proposes a new card sorting method called the Modified-Delphi card sort to replace the Open card sort. The Modified-Delphi card sort is based on a well-known forecasting technique called the Delphi method. Instead of producing individual models that are then analyzed as a whole, participants work with a single model that is proposed and modified throughout the study. The Modified-Delphi card sorting method produces more useful results to aid in the design of an information architecture than the Open card sorting method. A series of studies were conducted to directly compare the Modified-Delphi and Open cart sorting methods. First, two parallel studies using both methods were conducted with the same dataset and number of participants. Then, two studies were conducted on the results of the parallel studies: a heuristic review and ranking with information design experts and an Inverse card sort with additional users of the proposed architecture. The Modified-Delphi card sorting method produced results that were found to be at least as good as the Open card sorting method results and in some cases, better.