Is a meta-analysis a foundation, or just another brick? Comment on Meltzer, McNulty, Jackson, and Karney (2014).

Abstract

In a longitudinal data set of married couples, Meltzer, McNulty, Jackson, and Karney (2014) reported that partner physical attractiveness is more strongly associated with relationship satisfaction for men than for women. Although a recent meta-analysis (Eastwick, Luchies, Finkel, & Hunt, in press) provided no support for this sex difference across 97 samples and ∼30,000 participants, Meltzer et al. (2014) responded by outlining 7 criteria required for an appropriate test of the sex difference; these criteria eliminate all but 1 study from the meta-analysis. In this commentary, we raise 3 concerns about Meltzer et al.'s contribution. First, there is weak theoretical and empirical support for the criteria they used to dismiss the relevance of the meta-analysis studies. Second, if one adds Meltzer et al.'s data to the meta-analysis, all the sex differences remain extremely small and nonsignificant, even if one focuses only on studies that best conform to Meltzer et al.'s criteria (i.e., married samples, objective attractiveness measures). Third, a new data set meeting all 7 criteria fails to replicate the Meltzer et al. sex difference; in contrast, data revealed that physical attractiveness is, if anything, more strongly associated with the trajectory of relationship satisfaction for women than for men. As noted by Eastwick, Luchies, et al. (in press), in paradigms where participants evaluate partners they have (at a minimum) met face-to-face, the sex difference in the association of physical attractiveness with romantic evaluations is (a) extremely small on average and (b) unlinked to all cross-study characteristics identified to date.