Reliability of a Correlational Similarity Matrix

Abstract

The new methodological revision (cf. Stone & Coles, 1970a, 1970b; Stone, Coles, & Lindem, 1970)= of the multidimensional scaling method-model (similarity analysis) originally described by Ekman (1954) proposes that similarity estimauons (which Ekman factor analyzes) should be considered only as averaged (across k judges) raw judgmental data. With the Stone-Coles revision each column, in a similarity marrlx, can be considered as a column vector of v row elements ( v = number of stimuli). Product-moment correlations are then computed between all such column vectors of such a matrix. In other words, similarity correlations are computed from the mean similarity ratings which each stimulus shared with all of the other stimuli in the group-set. It is the resultant correlational similarity matrices which are factor analyzed with the Stone-Coles method-model. [The several advantages of such an approach are clearly spelled-out in the Stone, Coles, and Lindem (1970) monograph.] Since the key statistical construct, with the Stone-Coles method-model, is that of correlational similarity, it was wondered whether a logic for estimating the reliability of such could be formulated. W e have succeeded (or so we believe) in devising a scheme whereby the reliability of any set (a matrix) of correlational similarities can be estimated. The accomplishment of such an estimation is rather simple and straightforward. A correlational similarity matrix is computed based only on the similarity estimations pertaining to even numbered stimulus comparisons (even-numbered elements in each of the u column vectors of the mean similarity matrix). Another correlational similarity matrix is computed based only on the scimulus estimations pertaining to odd-numbered stimulus comparisons (odd-numbered elements in each column vector). The correlation between these two resultant correlational matrices is computed and provides a reliability estimate. The df associated with such a reliability estimation are ([v(v-1)]/2)-2. All indications are that such a reliability estimation approach is conservative. First, all data points used in the calculation of correlational similarities are reasonably stable mean values; and second, the problem of correlated error, which usually influences a split-half estimate, is not expected to be present.