Michigan Survey Research Center Research Articles

Proxy Survey Cost Indicators in Interviewer-Administered Surveys: Are they Actually Correlated with Costs?

Survey design decisions are-by their very nature-tradeoffs between costs and errors. However, measuring costs is often difficult. Furthermore, surveys are growing more complex. Many surveys require that cost information be available to make decisions during data collection. These complexities create new challenges for monitoring and understanding survey costs. Often, survey cost information lags behind reporting of paradata. Furthermore, in some situations, the measurement of costs at the case level is difficult. Given the time lag in reporting cost information and the difficulty of assigning costs directly to cases, survey designers and managers have frequently turned to proxy indicators for cost. These proxy measures are often based upon level-of-effort paradata. An example of such a proxy cost indicator is the number of attempts per interview. Unfortunately, little is known about how accurately these proxy indicators actually mirror the true costs of the survey. In this article, we examine a set of these proxy indicators across several surveys with different designs, including different modes of interview. We examine the strength of correlation between these indicators and two different measures of costs-the total project cost and total interviewer hours. This article provides some initial evidence about the quality of these proxies as surrogates for the true costs using data from several different surveys with interviewer-administered modes (telephone, face to face) across three organizations (University of Michigan's Survey Research Center, Westat, US Census Bureau). We find that some indicators (total attempts, total contacts, total completes, sample size) are correlated (average correlation ∼0.60) with total costs across several surveys. These same indicators are strongly correlated (average correlation ∼0.82) with total interviewer hours. For survey components, three indicators (total attempts, sample size, and total miles) are strongly correlated with both total costs (average correlation ∼0.77) and with total interviewer hours (average correlation ∼0.86).

“PROPAGANDISTS FOR THE BEHAVIORAL SCIENCES”: THE OVERLOOKED PARTNERSHIP BETWEEN THE CARNEGIE CORPORATION AND SSRC IN THE MID‐TWENTIETH CENTURY

The Carnegie Corporation's role as a patron of the behavioral sciences has been overlooked; its support for the behavioral sciences not only began earlier than the Ford Foundation's but was also at least equally important to their success. I show how the close postwar collaboration between the Carnegie Corporation and the Social Science Research Council (SSRC) to promote the behavioral sciences emerged after a struggle between Carnegie and the Rockefeller Foundation over the direction and leadership of the SSRC. I then focus on three postwar projects Carnegie helped conceive and fund that were publicized as the work of the SSRC: Chase's The Proper Study of Mankind (1948), Stouffer et al.'s The American Soldier (), and the Michigan's Survey Research Center 1952 election study. In each of these projects, Carnegie deliberately muted its own role and promoted the remade SSRC as a major advocate for the behavioral sciences.

Precision and Disclosure in Text and Voice Interviews on Smartphones.

As people increasingly communicate via asynchronous non-spoken modes on mobile devices, particularly text messaging (e.g., SMS), longstanding assumptions and practices of social measurement via telephone survey interviewing are being challenged. In the study reported here, 634 people who had agreed to participate in an interview on their iPhone were randomly assigned to answer 32 questions from US social surveys via text messaging or speech, administered either by a human interviewer or by an automated interviewing system. 10 interviewers from the University of Michigan Survey Research Center administered voice and text interviews; automated systems launched parallel text and voice interviews at the same time as the human interviews were launched. The key question was how the interview mode affected the quality of the response data, in particular the precision of numerical answers (how many were not rounded), variation in answers to multiple questions with the same response scale (differentiation), and disclosure of socially undesirable information. Texting led to higher quality data—fewer rounded numerical answers, more differentiated answers to a battery of questions, and more disclosure of sensitive information—than voice interviews, both with human and automated interviewers. Text respondents also reported a strong preference for future interviews by text. The findings suggest that people interviewed on mobile devices at a time and place that is convenient for them, even when they are multitasking, can give more trustworthy and accurate answers than those in more traditional spoken interviews. The findings also suggest that answers from text interviews, when aggregated across a sample, can tell a different story about a population than answers from voice interviews, potentially altering the policy implications from a survey.

Observing Attitudes, Intentions and Expectations (1945-1973)

Although involved in projects of influent institutions like the Cowles Commission, the NBER, and the Michigan Survey Research Center (SRC), George Katona, the “pioneer student and chief collector of consumer anticipations data” (Tobin, 1959, p. 1) is virtually absent from accounts of the topics he explored, including the study of the consumption function and the development of behavioral economics. This essay argues that such an absence is partly explained by the theoretical underpinnings of Katona’s project, which were incompatible with the economic views of behavior that dominated from the mid-1940s to the mid-1970s. It compares alternative survey programs funded by the Federal Reserve during that period, and analyzes the ensuing controversy on the purposes of the observation of attitudes, intentions and expectations. It claims that understanding Katona’s approach “required a real restructuring of thought – a genuine paradigm shift” (Simon, 1979, p. 12), which gives specific interest to this historical episode.

Observing Attitudes, Intentions, and Expectations (1945–73)

Although involved in projects of influential institutions like the Cowles Commission, the NBER, and the Michigan Survey Research Center, George Katona, the “pioneer student and chief collector of consumer anticipations data” (Tobin 1959, 1), is virtually absent from accounts of the topics he worked on, including the study of aggregate consumption and the development of behavioral economics. This essay argues that such an absence is partly explained by the theoretical underpinnings of Katona’s project, which were incompatible with the economic views of consumer behavior that dominated from the mid-1940s to the mid-1970s. It proceeds by comparing alternative survey programs funded by the Federal Reserve during that period, and analyzing the ensuing controversy on the purposes of the observation of attitudes, intentions, and expectations. It claims that understanding Katona’s approach “required a real restructuring of thought—a genuine paradigm shift” (Simon 1979, 12). That such a shift was required gives interest to this historical episode.

Consumer power and market control: Exploring consumer behaviour in affluent contexts (1946–1980)

The aim of this essay is to present and explain the emergence and decay of two unorthodox views of consumer behaviour that developed from the mid-1940s to the mid-1970s: the view of the powerful consumer and the view of market control by producers. It begins by presenting their common origins in empirical studies that opposed the Keynesian-type analysis of consumption. While the first developed into the program of behavioural economics defended by George Katona of the Michigan Survey Research Center, the second nourished the contributions of authors like Galbraith (1958, 1967, 1977), Scitovsky (1954, 1962, 1976) and Mishan (1960, 1967).

George Katona: A founding father of old behavioral economics

Assuming the division of behavioral economics into old and new, the paper begins to argue that old behavioral economics began with the works of two giants – George Katuna and Herbert Simon during the 1950s and early 1960s. The contributors of Herbert Simon are well established, thanks to the popularity of bounded rationality and satisficing, and his being award Noble Prize in economics. However, economists are much less familiar with the contributions of George Katona that can be viewed as the father of behavioral economics. Furthermore, the author argues that Katona was also misunderstood by various economists when he was attempting to create a psychologically based economics that rejected the mechanistic psychology of neoclassical economics and introducing the survey method to economic research that he had been using in his experimental psychology research previously. He also had influenced various economists during their debates in the 1950s without given the credit for. Many historians of behavioral economics limit Katona's contributions to the start of behavioral economics only to his contributions to macroeconomics. However, the paper demonstrates that Katona's behavioral economics included his contributions to macroeconomics (bringing realism to Keynesian consumption function and consumption behavior), micro-economics (business behavior, the rationality assumption, etc.), public finance and economic policy, and his introduction of the survey method. To demonstrate these contributions, the author argues that Katona attempted to bring realism to economic analysis – through psychological concepts – beginning with his early days of research in Germany which coincided with German hyper inflation- and continued whether working at New school for Social Research, Chicago University's Cowles Commission, the U.S. Department of Agriculture, or the University of Michigan's Survey Research Center. The author also argues that Katona's contributions went through stages, depending upon what economic problem persisted at the time, what advertises he was facing, and what institution/organization he was associated with.

Heterogeneity, learning and information stickiness in inflation expectations

In this paper we propose novel techniques for the empirical analysis of adaptive learning and sticky information in inflation expectations. These methodologies are applied to the distribution of households’ inflation expectations collected by the University of Michigan Survey Research Center. To account for the evolution of the cross-section of inflation forecasts over time and measure the degree of heterogeneity in private agents’ forecasts, we explore time series of percentiles from the empirical distribution. Our results show that heterogeneity is pervasive in the process of inflation expectation formation. We identify three regions of the distribution that correspond to different underlying mechanisms of expectation formation: a static or highly autoregressive region on the left hand side of the median, a nearly rational region around the median and a fraction of forecasts on the right hand side of the median formed in accordance with adaptive learning and sticky information.

The WHO World Mental Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders

The WHO World Mental Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders

Does Consumer Sentiment Foretell Revolving Credit Use?

The rising level of consumer debt in the U.S. is well documented. Revolving credit (credit cards) has experienced this growth, with the level of outstanding revolving credit increasing by over 600% in inflation-adjusted dollars over the past three decades. The goal here is to gauge the extent to which consumer sentiment; namely, the University of Michigan Survey Research Center Index of Consumer Sentiment, has predictive power in explaining the aggregate use of revolving credit using time-series data. The results generally show that changes in the consumer sentiment measure are related to subsequent changes in revolving credit use.

The Imperialism of Categories: Situating Knowledge in a Globalizing World

In February 1957 Lloyd Rudolph and I set forth into the “heat and dust” villages of Thanjavur district, South India, with 10 Indian graduate students from Madras Christian College. Our objective was to conduct a survey on political consciousness. Six hundred urban and rural Tamils scattered across three districts constituted the random sample we had selected from the first electoral rolls of recently freed India. V. O. Key, that witty and groundbreaking doyen of electoral behavior analysis, had enticed us into survey research. Upon our return, the Michigan Survey Research Center provided a methodologically intense summer.Susanne Rudolph is the William Benton Distinguished Service Professor Emerita of Political Science at the University of Chicago and past president of the American Political Science Association (srudolph@midway.uchicago.edu). She studies comparative politics with special interest in the political economy and political sociology of South Asia, state formation, Max Weber, and the politics of category and culture. An earlier version of this address was presented at the annual meeting of the association on September 2, 2004.

Sample designs and sampling methods for the Collaborative Psychiatric Epidemiology Studies (CPES)

This paper provides an overview of the probability sample designs and sampling methods for the Collaborative Psychiatric Epidemiology Studies (CPES): the National Comorbidity Survey Replication (NCS-R), the National Study of American Life (NSAL) and the National Latino and Asian American Study of Mental Health (NLAAS). The multi-stage sample design and respondent selection procedures used in these three studies are based on the University of Michigan Survey Research Center's National Sample designs and operations. The paper begins with a general overview of these designs and procedures and then turns to a more detailed discussion of the adaptation of these general methods to the three specific study designs. The detailed discussions of the individual study samples focus on design characteristics and outcomes that are important to analysts of the CPES data sets and to researchers and statisticians who are planning future studies. The paper describes how the expected survey cost and error structure for each of these surveys influenced the original design of the samples and how actual field experience led to changes and adaptations to arrive at the final samples of each survey population.

Communication Researchers and Policy-Making

Braman, S. (Ed.) (2003). Communication Researchers and Policy-Making. Cambridge, MA: MIT Press Sourcebooks. 576 pages. This volume, skillfully compiled by Sandra Braman, explores the confluence of communication research and policy making. Contributors range from communication pioneers Harold Lasswell and Paul Lazarsfeld to seminal policy figures like Woodrow Wilson and John Dewey. The volume includes, as well, current contributions from a host of academicians and policy analysts ranging from Columbia Professor James Carey to the President of Research Solutions, Cecilie Gaziano. As rich as the collection of articles is in its own right, the introduction, conclusion, and transitional chapters by editor Sandra Braman provide the indispensable links among the nearly two dozen contributions. Examining historical accounts of the rise of communication research in the forties through its role today, one is struck by the distance that now separates communication research and policy making. As Braman writes, while examining the policy implications of research results and bringing those insights to the attention of policy-makers may have great value to society, doing so can impede [italics added] the career of an academic by taking time and energy away from the peer-reviewed publications that are the primary criterion for tenure and promotion (p. 415). The irony in this confounding admission is that the field of communication research was developed largely through the support of the federal government. As Christopher Simpson (pp. 259-60) explains, World War II shaped the emerging field of communication research as a reaction to the fears of Nazi warfare, a term that entered the English language in 1941 as an imperfect translation of Weltanschauungkrieg (worldview warfare). Six centers of psychological warfare emerged within the federal government, including the Office of War Information led by Elmer Davis and the Office of Strategic Services led by William Donovan. It was within these organizations that many social scientists began their careers in communication research and went on to establish research institutions outside of government. As communication research crystallized in the decade after the war, Simpson reports that 96% of all federal funding at that time was provided by the U.S. military. Several dozen studies in Public Opinion Quarterly were supported by the Office of Naval Research, the U.S. Air Force, the CIA, and the U.S. Information Agency. The most important communication research centers--ranging from the University of Michigan's Survey Research Center to MIT's Center for international Studies--depended for their survival on funding from national security agencies. Without the funding from the military, CIA, and USIA, Simpson contends, mass communication research would not have matured into a distinct scholarly field (pp. 271-275). Simpson worries that the network of scientists and state sponsors for psychological warfare a pivotal role in creating a new branch of social science, euphemistically termed 'communication research,' whose framework played an indirect role in shaping our view of U.S. society. He approvingly quotes James Carey: We first produce the world by symbolic work, and then take up residence in the world we have produced (p. 283). In a separate chapter by Carey, he takes Elihu Katz to task for his 1977 report for the BBC on broadcast research priorities. While acknowledging the importance of the study, he dwells on the problems the report creates rather than on the more obvious opportunities it presents (p. 438). Carey questions the value of the silent embrace between the scholarly community and the broadcast industry: do not think it produces any necessary or potential benefits for scholarship. In fact, I think a better case can be made that scholarship, like many of the arts, flourishes when it stands in determined opposition to the established order (p. …

Reality Bites: News Exposure and Economic Opinion

Reality Bites: News Exposure and Economic Opinion

Taxing and Spending Politics: A Generational Perspective

The aging of the American population has important ramifications for public policy. Economists have long been predicting intergenerational economic warfare based upon their close look at the nation's changing demographics. It is time for political scientists to focus on this issue, especially since taxing and spending issues are increasingly at the forefront of most elections, whether at the national or state and local levels. This research examines some of the existing myths about different age groups' taxing and spending preferences. It is based on data generated from annual surveys of people 18 years and older by the U.S. Advisory Commission on Intergovernmental Relations, the Florida Survey Research Laboratory, and perennial surveys by the Times Mirror Center for The People & The Press, Gallup, Roper, and the University of Michigan's Survey Research Center, among others. The results suggest that as the intergenerational competition for scarce resources intensifies, the older generations are likely to ...

Question Form and Context Effects in the Measurement of Partisanship: Experimental Tests of the Artifact Hypothesis

Previous articles in this Review, including a Controversy in 1992, debated the comparability of alternative forms of the question about partisanship asked in Gallup and Michigan SRC surveys. Bishop, Tuchfarber and Smith contribute to this debate by reporting and analyzing evidence from 15 experimental surveys in Ohio in 1991–1993. They conclude that the distribution of partisan loyalties will generally be the same whether one uses the Gallup or Michigan Survey Research Center question and that, contrary to findings of Abramson and Ostrom, the Gallup form is no more responsive to short-term political forces than its SRC counterpart. In response, Abramson and Ostrom agree that during many time periods there will be little difference between aggregate levels of macropartisanship regardless of which measure is used. But they argue that during periods of political volatility the Gallup approach will accentuate differences, while the SRC version will attenuate them.

The indexes of consumer sentiment and confidence: Leading or misleading guides to future buyer behavior

The University of Michigan Survey Research Center's indexes of consumer sentiment and expectations and the Conference Board's index of consumer confidence and expectations are analyzed using time-series methods. The indexes were examined for a priori information content about a number of important variables associated with forecasting product demand and aggregate buyer behavior patterns. The empirical findings suggest that the indexes are useful in predicting future values of aggregate consumer spending and business and economic activity. The Michigan measure appears to outperform the Conference Board measure when predicting durable good purchases, whereas the reverse is true for general economic activity. Both measures are useful for predicting business activity.

Question Wording and Partisanship: Change and Continuity in Party Loyalties During the 1992 Election Campaign

Gallup macropartisanship varies more over time than aggregate measures of partisanship employing the standard Michigan Survey Research Center (SRC) party identification measure, but previous analyses do not provide direct evidence about why Gallup macropartisanship is more variable. Although these differences could result from the short-term focus of the Gallup party affiliation question, aggregate-level analyses cannot test the effects of question wording directly. Between March and October 1992, we conducted a series of question-wording experiments, employing six statewide computer-assisted telephone interview (CATI) surveys of Michigan adults, including a four-wave panel study. Our analyses strongly suggest that the Gallup measure responds more to short-term political conditions and clearly demonstrate that the Gallup measure is less stable over time. These individual-level results help explain why Gallup macropartisanship varies more over time than aggregate measures of partisanship employing the standard SRC measure and raise questions about the degree to which one can generalize from analyses using the Gallup data to the research literature on party identification.

Measurement errors in survey forecasts of expected inflation and the rationality of inflation expectations

Studies that use survey data to test the rationality of inflationary expectations usually test for unbiasedness by regressing actual inflation on expected inflation and testing the joint hypothesis that the intercept is zero and the slope coefficient is one. Such studies are fatally flawed because they incorrectly assume that expected inflation is measured without error. A procedure that allows for the influence of observation errors in expected inflation is developed and applied to the University of Michigan Survey Research Center data. Contrary to some recent studies, unbiasedness is rejected.

Macropartisanship: An Empirical Reassessment

To evaluate the comparability of the Gallup and Michigan Survey Research Center measures for studying levels of partisanship among the U.S. electorate we compare the overtime distribution of partisanship and the correlates of partisanship using the results of Gallup surveys, the National Election Studies, and the General Social Surveys. Compared with the Gallup results, both the other two surveys reveal less short-term variation and also less total variation. Compared with the Gallup results, the National Election Studies partisanship results are less related to short-term electoral outcomes and do not appear to be strongly driven by short-term economic and political evaluations. Our analyses suggest that scholars should be cautious about using Gallup results to revise conclusions based upon analyses that employ the Michigan Survey Research Center party identification measure.