Biases in paired preference tests: Cross‐cultural comparison of Japanese and American consumers

Abstract

Listen

Translate article

AbstractThis research examined potential biases associated with paired preference tests and how these biases might differ between Japanese and American cultures. The three biases that were investigated were the potential differential use of the no preference option, a predisposition in choosing the first stimulus in a pair, and finally a potential tendency of picking the stimulus with the numerically higher code in the pair. In this investigation, 96 American and 192 Japanese subjects performed six paired preference tests involving three types of grapefruit juices, three of the pairs involving identical samples. Of the three biases, only the differential use of the “no preference” option was significantly observed, with American subjects exhibiting higher “no preference” frequencies than Japanese for identical pairs (24% and 9%, respectively). Interestingly, no clear position or code bias effects were uncovered, a result possibly driven by the experimental approach used in this research. Potential explanations are discussed.Practical applicationsBiases when executing paired preference tests have been documented. Addressing them, if not controlling them, is necessary to ensure reliable interpretation of test results. One important bias involves the use of the “no preference” option for identical samples. This use directly impacts how an identicality norm can be utilized to interpret test results. Assuming the wrong identicality norm will increase the likelihood of reaching an incorrect conclusion. This research establishes the identicality norm for Japanese subjects and confirms that such norm would be different from that of American subjects and more like that of Korean consumers. This will be of value for companies conducting research across cultures to ensure that more reliable conclusions are reached. No clear position or code biases were found in this research. Nevertheless, sample position balancing and the use of two sets of labeling codes in any paired preference test design are recommended since these effects have been previously reported elsewhere.