Tips for Readers of Research: Beware the ‘Classic Study’

Abstract

EVERY SO often, someone proposes that research-oriented graduate students earn their master's degrees by trying to replicate a study. Replication is de rigueur in the natural sciences. A fact isn't a fact until a lot of people around the world confirm that it is so. However, replication carries little cachet in education and psychology because it doesn't lead to promotion or tenure. The master's degree proposal is invariably rejected, usually on the ground that students need to work on original problems. For various reasons, though, sometimes someone tries to repeat a classic study -- and finds it wanting. So it was with Judith Kleinfeld of the of Alaska, Fairbanks, and Stanley Milgram's Six Degrees of Separation studies. Milgram's research gave rise to the idea (which became a play and a movie and a game) that everyone in the world is separated from everyone else by usually no more than six intervening links. Milgram's initial study supposedly used a random sample of people in Wichita who were asked to get a document to the wife of a divinity student in Cambridge, Massachusetts. Milgram reported that the average number of links between the sender and the receiver was six. Kleinfeld, seeking a way to convince her students that social science is more than the systematic study of the obvious, had her students attempt to replicate the study. They couldn't. She decided to find out why. Combing the Yale archives, she found that some of Milgram's own unpublished data were starkly at odds with the myth. In the first study, Milgram used newspaper ads to invite people into the experiment. In another, he bought a mailing list. The newspaper ad presented the task as something of a challenge: Could you, as a typical American, contact another citizen, regardless of his walk of life? People who read newspapers and people whose names are thought to be worth selling are not likely to be representative of all of America. In addition, the document to be mailed was an impressive-looking royal blue passport with Harvard University embossed in gold letters. Even so, in the unpublished study, only three out of 60 documents made it from Wichita to Cambridge. And those usually required eight or nine connections, not six. In the technical report related to another attempt, Milgram reported only a 29% success rate. When researchers in Ohio attempted a replication using a target in Los Angeles, they also found a low completion rate, 18%. Moreover, they had divided both the senders and the receivers into high-, low-, and middle-income people. Low-income senders had little success and were able to reach only low-income receivers. Middle- and high-income senders did manage to complete chains, not only to their own social class but to the other classes as well. In these studies, only two locations in the U.S. served as sending or receiving locations. Even if this research had pointed to six degrees of separation, it would hardly be sufficient to establish that it's a small world after all. A new attempt to test the six degrees hypothesis is now under way at Columbia University. Unfortunately, the research team there is also soliciting participants and, worse, is conducting the experiment by e- mail, thereby preventing most of the world from participating. If you want to participate, go to http://smallworld.sociology.columbia.edu. Meanwhile, Milgram's obedience have been replicated worldwide. They show that, under the right circumstances, anyone can become an Adolf Eichmann, which provides some empirical evidence for Hannah Arendt's banality of evil theory. Germans have the highest proportion of people who obey a scientist and administer (they are led to believe) increasingly dangerous electric shocks to another person. Other studies are widely misunderstood. Some have taken the Hawthorne Effect to show that workers respond well to workplace changes when they think management cares about them and is acting for the workers' benefit. …