A multiple-response chi-square framework for the analysis of Free-Comment and Check-All-That-Apply data

Abstract

Free-Comment (FC) and Check-All-That-Apply (CATA) provide a contingency table containing citation counts of descriptors by products. The analyses performed on this table are most often related to the chi-square statistic. However, such practices are not well suited because they consider experimental units as being the citations (one descriptor for one product by one subject) while the evaluations (vector of citations for one product by one subject) should be considered instead. This results in incorrect expected frequencies under the null hypothesis of independence between products and descriptors and thus in an incorrect chi-square statistic. Thus, analyses related to this incorrect chi-square statistic, which include Correspondence Analysis, can lead to wrong interpretations. This paper presents a modified chi-square square framework dedicated to the analysis of multiple-response data in which experimental units are the evaluations and which is, therefore, better suited to FC and CATA data. This new framework includes a multiple-response dimensionality test of dependence, a multiple-response Correspondence Analysis, and a multiple-response hypergeometric test to investigate which descriptors are significantly associated with which product. The benefits of the multiple-response chi-square framework over the usual chi-square framework are exhibited on real CATA data. An R package called “MultiResponseR” is available upon request to the authors and on GitHub to perform the multiple-response chi-square analyses.