Social Research and Market Research: A Critique of a Critique

Abstract

From a methodological standpoint, sample surveys organised by academics are generally less satisfactory than those conducted outside academia. Particularly in comparison with surveys sponsored by private companies, academics are likely to have less research money, less technical skill, less experience at both conducting and commissioning surveys and not least are less likely than company executives to be in a position where their future careers depend on the production of reliable results. This may be a depressing judgment but it is nonetheless unavoidable. It is derived from a year's experience of dealing with clients, both academic and commercial, as an executive with a market research company and amply confirmed by an analysis of research reports appearing in the British Journal of Sociology and Sociology for the period 1968-78. Although 24% of the 510 reports appearing in this decade could be included under the heading of 'survey research', broadly defined as any study in which a number of sampling units were drawn from a specified universe, the technical quality of the surveys reported is usually poor.1 The most frequent problem is the failure to sample from a theoretically significant population, a failing which places severe limits not only on the general relevance of the study but even on the value of the descriptive statistics it generates. One is often reminded of the researcher described by Peel and Skidworth in their unfavourable comparison of British sociological surveys with the American equivalents, a researcher whose sample consisted of 4 100 employees of a large insurance firm, 100 unskilled workers from British Rail, and 80 housewives in a suburb - half in owner-occupied housing, half in council housing'.2 Only 28% of survey reports are based on national studies of the population in question, whether that population is Catholic priests in New Zealand, Scottish firms quoted on the Stock Exchange or white-collar trade unions in Britain. Convenience rather than critical case methodology seems to have been the main criterion in determining the methodology of most of the subnational studies. None of the national surveys that are reported are based on samples drawn from the adult British population, the standard universe for many market research studies. Although the appropriate universe varies with the research problem, it is important for the development of sociology that surveys of the general population should be based on some standardized universe. As things stand, the sociology journals present us with a haphazard patchwork of survey reports which because of the enormous heterogeneity of populations studied, sampling methods employed and questions asked offer strictly limited opportunities for cumulation, replication and trending.3 Given too that as many as 3 1% of survey reports are based on studies conducted outside Britain, one can only infer that British sociology has signally failed to exploit the opportunity made available to it by the development of sampling techniques to produce precise, reliable and informative descriptions of Britain's social structure - and of those people within that structure who would otherwise be most likely to pass by unnoticed and unheeded. Whatever one's views on the importance of survey research to the development of the discipline, it is surely unarguable that those sociological surveys which are conducted should satisfy more exacting standards than is currently the case.