Dual-frame Design Research Articles

Are mobile phones predominant in dual-frame computer assisted telephone interviewing surveys?

The effectiveness of Computer Assisted Telephone Interviewing (CATI) surveys relying on fixed telephone numbers has been eroded by the rise in mobile phone usage and the growing population reachable exclusively through mobile phones. As a result, telephone survey methodologists have had to come up with research designs capable of dealing with the coverage bias in CATI surveys caused by incomplete fixed phone directories. The methodological and operational issues of dual-frame surveys, namely those combining fixed phones and mobile phones, have been extensively investigated, but a comprehensive overview of the magnitude of mobile phone use in CATI surveys is lacking. The present study explores this research gap by analysing data from 675 CATI surveys conducted in European countries. Although a generalised scenario of single frame mobile phone designs is not yet visible, the predominance of mobile phone units over fixed phone units in dual-frame CATI surveys is evident. The sampling method and the target population are the survey-related features most strongly associated to the percentage of mobile phone units in the sample. Survey organisations in countries which continue to have a high number of fixed phone subscriptions tend to give less weight to mobile phone frames in their dual-frame designs.

Dual frame design in agricultural surveys: reviewing roots and methodological perspectives

Dual frame design in agricultural surveys: reviewing roots and methodological perspectives

Moving towards a single-frame cell phone design in random digit dialing surveys: considerations from a French general population health survey

BackgroundOver the last two decades, telephone surveys based on random digit dialing have developed considerably. At the same time, however, the proportion of the population with a cell phone has increased, whereas landline frame coverage has declined, thus raising the possibility of discontinuing landline phone surveys. This paper aims to assess the impact of using a single-frame (SF) cell phone design instead of a dual-frame (DF) design with landlines and cell phones in the context of repeated health surveillance surveys in the general population. We analyze data from a random digit dialing health survey of the French population and assess differences between the DF and the counterfactual SF design that excludes the landline phone sample from the DF design. We evaluate the quality of the two survey designs in terms of survey productivity, response rates, representativeness, balancing of external covariates, and prevalence estimates of key health behavior indicators.ResultsOur results show that a SF cell phone survey has several advantages over a combined DF landline and cell phone survey. Cell numbers require fewer call attempts to complete an interview, leading to a substantial reduction in the mean data collection duration and weight dispersion. The global representativeness of the SF design was slightly better than its DF counterpart, although the elderly were underrepresented. After calibration, differences in health behavior estimates were small for the seven health indicators analyzed.ConclusionsSwitching from a DF random telephone survey to a SF cell phone design has a number of practical advantages and would have a minimal impact on general population health surveys for monitoring health behavior at the population level. However, the different aspects of the survey quality had to be studied to make a decision. Further studies are needed to explore the scope of possibilities.

Using a dual-frame design to improve phone surveys on political attitudes: developing a weighting strategy for limited external information in Hong Kong

Using a dual-frame design to improve phone surveys on political attitudes: developing a weighting strategy for limited external information in Hong Kong

Estimating the Breeding Population of Great Blue Herons (Ardea herodias) in Maine, USA, Using Dual-Frame Sampling

The Great Blue Heron (Ardea herodias) is a Species of Special Concern in the State of Maine, USA, due to declines along the Atlantic coast since the mid-1980s. To determine if this decline was statewide, an aerial survey was conducted from 1 May to 18 June 2015 to estimate the breeding population using a stratified dualframe design accounting for imperfect detection of colonies. The strata were based on habitat and known colony densities. The area frame consisted of 10-km x 10-km plots. Stratum samples were searched by air independently by front and rear observers for known and new colonies. The list frame consisted of plots containing known colonies, a sample of which was checked for nesting activity. The dual-frame design combined the strengths of both list- and area-only surveys. Flying directly to known colonies in the list frame was very efficient, and accounting for new colonies in the area frame, allowed unbiased, statistically valid estimation. Estimates were calculated for a Maine statewide total of active Great Blue Heron colonies (n = 243, 90% credible interval (CI) = 193–352), total nests (n = 2,019, CI = 1,778–2,434), and active nests (n = 1,593, CI = 1,416–1,872). Observer detection ranged from 25 to 99%. The percentage of list frame colony coverage in the dual-frame estimate was 63%, but varied by stratum. The percent CV of the dual-frame statewide total active nests was 8.9%, but also varied by stratum. By repeating these methods, strata and statewide population trends could be obtained.

Frame-Based Sparse Analysis and Synthesis Signal Representations and Parseval K-SVD

Frames are the foundation of the linear operators used in the decomposition and reconstruction of signals, such as the discrete Fourier transform, Gabor, wavelets, and curvelet transforms. The emergence of sparse representation models has shifted the emphasis in frame theory toward sparse-norm minimization problems. In this paper, we apply frame theory to the sparse representation of signals, wherein a frame is used for synthesis dictionary and a dual frame is used for analysis dictionary. Our objective was to formulate a novel dual-frame design in which the sparse vector obtained through the decomposition of any signal is also the sparse solution representing signals based on a reconstruction frame. Our findings demonstrate that this type of dual frame cannot be constructed for overcomplete frames, thereby precluding the use of any linear analysis operator to derive the sparse synthesis coefficient for signal representation. We also developed a novel dictionary learning algorithm (called Parseval K-SVD) to learn a tight-frame dictionary with normalized atoms. We then dealt with the problem of signal representation using frames from the analysis and synthesis perspective to derive optimization formulations for problems pertaining to image recovery. Our preliminary results demonstrate that the images recovered using this approach are correlated with the frame bounds of dictionaries, thereby demonstrating the importance of using different dictionaries for different applications.

Darkness made visible: Field management and nonresponse in the 2004 SCF

Nonresponse in field surveys is the joint outcome of the decision of survey staff to apply effort to inform and persuade respondents, and the evaluation of such inputs by respondents. In most such surveys, the field staff are under great pressure to produce completed interviews. Thus, as discussed in Kennickell [2004], they have an incentive to apply effort to cases that are most likely, in their view, to be completed with least effort. To the extent that interviewers’ perceptions are unbiased, such behavior would tend to amplify latent patterns of nonresponse. When the characteristics of respondents that affect the likelihood of participation are correlated with variables of analytical interest in the survey, bias results, unless a means can be found of discovering and adjusting for the underlying behavioral structures. But, absent constraints on the behavior of interviewers, the observed outcomes are contaminated by the endogeneity of effort, and only strong a priori assumptions could disentangle the interviewer effects from the respondent effects. To address the problem of endogenous effort, the 2004 Survey of Consumer Finances introduced a phased plan of sample management to make effort more nearly exogenous through the first two of three phases of field work. Thus, nonresponse in these controlled stages should largely reflect respondent characteristics, not a mixture of respondent and interviewer characteristics. The dual frame design of the SCF offers two classes of sample cases for modeling nonresponse. For the area-probability sample, tract-level data are available from the 2000 Census of Population. For the list sample, frame case-specific data based on statistical records derived from tax returns are available. For both set of cases, some interviewer observations are also available. This paper presents estimates of nonresponse models based on these data. Nonresponse in field surveys is the joint outcome of the decision of survey staff to apply effort to inform and persuade respondents, and the evaluation of such inputs by respondents. In most such surveys, the field staff are under great pressure to produce completed interviews. Thus, as discussed in Kennickell [2004], they have an incentive to apply effort to cases that are most likely, in their view, to be completed with least effort. To the extent that interviewers’ perceptions are unbiased, such behavior would tend to amplify latent patterns of nonresponse. When the characteristics of respondents that affect the likelihood of participation are correlated with variables of analytical interest in the survey, bias results, unless a means can be found of discovering and adjusting for the underlying behavioral structures. But, absent constraints on the behavior of interviewers, the observed outcomes are contaminated by the endogeneity of effort, and only strong a priori assumptions could disentangle the interviewer effects from the respondent effects. To address the problem of endogenous effort, the 2004 Survey of Consumer Finances (SCF) introduced a phased plan of sample management intended to make effort more nearly exogenous through the first two of three phases of field work. As described in more detail later in this paper, interviewers were given a flexible protocol intended to ensure that all sample cases were exposed to a certain level of effort and that two specific points in the course of the application of effort were marked for cases that had not been completed earlier. Those points classify the field operations on the cases into at most three phases, the last of which is limited only by the close of field work. If the framework holds, then response within the first two phases may be taken as independent of behavioral variations in the level of effort applied. The first section of the paper provides an overview of the SCF and the approach to management of field resources developed for the 2004 survey and describes how this protocol worked in practice. The second section presents models of nonresponse in different phases of the field effort, conditional on both tract-level and case-level data. The final section summarizes the key findings and points toward the next step for the SCF.

Public Attitudes Toward the Police: Findings From a Dual-Frame Telephone Survey

Using telephone survey data collected in Houston, Texas, this study explores the feasibility of a dual-frame (landline and cell-phone-only household samples) design survey in police studies and compares the corresponding characteristics of survey respondents and multiple measures of citizens' attitudes toward the police. It was found that a cell-phone interview with the same scope and length of a landline interview is indeed operationally feasible, albeit at higher cost and lower response rate. Compared with their landline counterparts, respondents in our cell-phone-only sample are younger, males, members of ethnic minority, renters, mobile, and less educated. No appreciable attitudinal differences among the respondents are observed between the two samples. In addition, substantive outcomes from multivariate regression models do not seem to be impacted by the types of phones used. Implications for future studies are included.

Effects of using an Overlapping Dual-Frame Design on Estimates of Health Behaviors: A French General Population Telephone Survey

Effects of using an Overlapping Dual-Frame Design on Estimates of Health Behaviors: A French General Population Telephone Survey

Review of Estimation Methods for Landline and Cell Phone Surveys

The rapid proliferation of cell phone use and the accompanying decline in landline service in recent years have resulted in substantial potential for coverage bias in landline random-digit-dial telephone surveys, which has led to the implementation of dual-frame designs that incorporate both landline and cell phone samples. Consequently, researchers have developed methods to allocate samples and combine the data from the two frames. In this article, we review point and interval estimation methods of proportions that can be used to analyze overlapping dual-frame surveys. We use data from the survey of attitudes toward immigrants and immigration (Opinions and Attitudes of the Andalusian Population regarding Immigration survey), a dual-frame telephone survey conducted in Andalusia, Spain, to explore these different statistical adjustments for combining landline and cell phone samples. Our application obtains good results for calibration, fixed weight, pseudo-empirical likelihood, and single-frame procedures. We recommend that one of these internally consistent estimators be used in practice. The results of these methods of estimation show that the negative image toward immigration continues to spread.

Developing a weighting strategy to include mobile phone numbers into an ongoing population health survey using an overlapping dual-frame design with limited benchmark information.

BackgroundIn 2012 mobile phone numbers were included into the ongoing New South Wales Population Health Survey (NSWPHS) using an overlapping dual-frame design. Previously in the NSWPHS the sample was selected using random digit dialing (RDD) of landline phone numbers. The survey was undertaken using computer assisted telephone interviewing (CATI). The weighting strategy needed to be significantly expanded to manage the differing probabilities of selection by frame, including that of children of mobile-only phone users, and to adjust for the increased chance of selection of dual-phone users. This paper describes the development of the final weighting strategy to properly combine the data from two overlapping sample frames accounting for the fact that population benchmarks for the different sampling frames were not available at the state or regional level.MethodsEstimates of the number of phone numbers for the landline and mobile phone frames used to calculate the differing probabilities of selection by frame, for New South Wales (NSW) and by stratum, were obtained by apportioning Australian estimates as none were available for NSW. The weighting strategy was then developed by calculating person selection probabilities, selection weights, applying a constant composite factor to the dual-phone users sample weights, and benchmarking to the latest NSW population by age group, sex and stratum.ResultsData from the NSWPHS for the first quarter of 2012 was used to test the weighting strategy. This consisted of data on 3395 respondents with 2171 (64%) from the landline frame and 1224 (36%) from the mobile frame. However, in order to calculate the weights, data needed to be available for all core weighting variables and so 3378 respondents, 2933 adults and 445 children, had sufficient data to be included. Average person weights were 3.3 times higher for the mobile-only respondents, 1.3 times higher for the landline-only respondents and 1.7 times higher for dual-phone users in the mobile frame compared to the dual-phone users in the landline frame. The overall weight effect for the first quarter of 2012 was 1.93 and the coefficient of variation of the weights was 0.96. The weight effects for 2012 were similar to, and in many cases less than, the effects found in the corresponding quarter of the 2011 NSWPHS when only a landline based sample was used.ConclusionsThe inclusion of mobile phone numbers, through an overlapping dual-frame design, improved the coverage of the survey and an appropriate weighing procedure is feasible, although it added substantially to the complexity of the weighting strategy. Access to accurate Australian, State and Territory estimates of the number of landline and mobile phone numbers and type of phone use by at least age group and sex would greatly assist in the weighting of dual-frame surveys in Australia.

Inclusion of mobile telephone numbers into an ongoing population health survey in New South Wales, Australia, using an overlapping dual-frame design: impact on the time series.

BackgroundSince 1997, the NSW Population Health Survey (NSWPHS) had selected the sample using random digit dialing of landline telephone numbers. When the survey began coverage of the population by landline phone frames was high (96%). As landline coverage in Australia has declined and continues to do so, in 2012, a sample of mobile telephone numbers was added to the survey using an overlapping dual-frame design. Details of the methodology are published elsewhere. This paper discusses the impacts of the sampling frame change on the time series, and provides possible approaches to handling these impacts.MethodsPrevalence estimates were calculated for type of phone-use, and a range of health indicators. Prevalence ratios (PR) for each of the health indicators were also calculated using Poisson regression analysis with robust variance estimation by type of phone-use. Health estimates for 2012 were compared to 2011. The full time series was examined for selected health indicators.ResultsIt was estimated from the 2012 NSWPHS that 20.0% of the NSW population were mobile-only phone users. Looking at the full time series for overweight or obese and current smoking if the NSWPHS had continued to be undertaken only using a landline frame, overweight or obese would have been shown to continue to increase and current smoking would have been shown to continue to decrease. However, with the introduction of the overlapping dual-frame design in 2012, overweight or obese increased until 2011 and then decreased in 2012, and current smoking decreased until 2011, and then increased in 2012. Our examination of these time series showed that the changes were a consequence of the sampling frame change and were not real changes. Both the backcasting method and the minimal coverage method could adequately adjust for the design change and allow for the continuation of the time series.ConclusionsThe inclusion of the mobile telephone numbers, through an overlapping dual-frame design, did impact on the time series for some of the health indicators collected through the NSWPHS, but only in that it corrected the estimates that were being calculated from a sample frame that was progressively covering less of the population.

Just blowing smoke? Social desirability and reporting of intentions to quit smoking.

Do cigarette smokers really want to quit smoking or do they simply say they do in order to placate others and avoid criticism? In surveys of smokers, stated quit intentions and reports of quit attempts may be biased by social desirability concerns. This makes it difficult to interpret large-scale state and national surveys of smoking behavior that collect data through telephone and face-to-face interviews, methods that tend to evoke high levels of socially desirable responding. The 2007 Health Information National Trends Survey used a dual-frame design to query smokers' quit intentions and past quit attempts in 1 of 2 ways: A self-administered mail survey (low pressure for socially desirable responding; n = 563), or an interviewer-administered telephone survey (high pressure for socially desirable responding; n = 499). Estimates derived from the 2 formats were compared to test for social desirability effects. In both survey modes, approximately two thirds of smokers reported seriously considering quitting in the next 6 months (mail: 64.9%; telephone: 68.9%), and approximately half reported making a quit attempt in the past year (mail: 54.9%; telephone: 52.3%). Neither difference approached significance in logistic regressions controlling for demographics (ps > .24). It appears that a large proportion of smokers in the United States aspire to live smoke-free lives and are not simply responding in a socially desirable manner to deflect criticism in an antismoking social climate. Future research should (1) replicate this study with greater statistical power, (2) examine the possible effects of survey context (e.g., health survey vs. smoking pleasure survey), and (3) explore survey mode effects in specific subpopulations.

A dual-frame sampling methodology to address landline replacement in tobacco control research

ObjectivesWe assessed the comparability of self-reported smoking prevalence estimates from a dual-frame survey with those from two large-scale, national surveys.MethodsThe Social Climate Survey of Tobacco Control (SCS-TC) obtained self-reported current...

Inclusion of mobile phone numbers into an ongoing population health survey in New South Wales, Australia: design, methods, call outcomes, costs and sample representativeness.

BackgroundIn Australia telephone surveys have been the method of choice for ongoing jurisdictional population health surveys. Although it was estimated in 2011 that nearly 20% of the Australian population were mobile-only phone users, the inclusion of mobile phone numbers into these existing landline population health surveys has not occurred. This paper describes the methods used for the inclusion of mobile phone numbers into an existing ongoing landline random digit dialling (RDD) health survey in an Australian state, the New South Wales Population Health Survey (NSWPHS). This paper also compares the call outcomes, costs and the representativeness of the resultant sample to that of the previous landline sample.MethodsAfter examining several mobile phone pilot studies conducted in Australia and possible sample designs (screening dual-frame and overlapping dual-frame), mobile phone numbers were included into the NSWPHS using an overlapping dual-frame design. Data collection was consistent, where possible, with the previous years’ landline RDD phone surveys and between frames. Survey operational data for the frames were compared and combined. Demographic information from the interview data for mobile-only phone users, both, and total were compared to the landline frame using χ2 tests. Demographic information for each frame, landline and the mobile-only (equivalent to a screening dual frame design), and the frames combined (with appropriate overlap adjustment) were compared to the NSW demographic profile from the 2011 census using χ2 tests.ResultsIn the first quarter of 2012, 3395 interviews were completed with 2171 respondents (63.9%) from the landline frame (17.6% landline only) and 1224 (36.1%) from the mobile frame (25.8% mobile only). Overall combined response, contact and cooperation rates were 33.1%, 65.1% and 72.2% respectively. As expected from previous research, the demographic profile of the mobile-only phone respondents differed most (more that were young, males, Aboriginal and Torres Strait Islanders, overseas born and single) compared to the landline frame responders. The profile of respondents from the two frames combined, with overlap adjustment, was most similar to the latest New South Wales (NSW) population profile.ConclusionsThe inclusion of the mobile phone numbers, through an overlapping dual-frame design, did not impact negatively on response rates or data collection, and although costing more the design was still cost-effective because of the additional interviews that were conducted with young people, Aboriginal and Torres Strait Islanders and people who were born overseas resulting in a more representative overall sample.

Incentives for Cell Phone Only Users: What Difference Do They Make?

Providing incentives to potential respondents has been shown to be one of the more effective means of encouraging participation in surveys across modes of data collection. The increasing number of cell-phone only (CPO) households has sparked an interest in best practices for interviewing cell phone respondents. One aspect of interviewing cell phone respondents that has not been adequately addressed is the question of the need to provide some type of remuneration to those who complete the interview on a cell phone. In this research, we provide evidence that suggests that providing such remuneration does not have a significant impact on either response rates or results. Although the literature on this topic is not extensive, there have been several studies of this topic (Dillman 2007; Singer 2002; Trussell and Lavrakas 2004). For example, Brick and his colleagues (2007) found in a national survey of cell phone households that a $10 incentive significantly improved participation over a $5 incentive. Similar studies by Diop and his colleagues (2008a, 2008b) found that a $10 incentive produced a higher rate of production compared to no incentive or to a $5 incentive. These earlier studies were based on unscreened dual-frame designs that included all cell phone users – that is, those both with and without landlines – while our research focused on those in cell phone only households, which may account for some of the difference in the findings. In the early 2000s, most initial approaches to contact cell phone users presumed that some type of incentive or remuneration would be given. This was usually done as a reimbursement for possible charges incurred by the respondent, although in some cases they may have functioned as incentive payments. One of the conclusions from the Cell Phone Sampling Summit II was that “survey respondents reached on their cell phone should be properly reimbursed for their time on a research call” (Lavrakas and Shuttles 2005). As the cost structure of cell phone billing has changed and more users have moved to virtually “unlimited minutes” plans, does reimbursement of cell phone users remain a sine qua non?

Segmented or Overlapping Dual Frame Samples in Telephone Surveys

Cell phones have produced a dramatic decline in household coverage in telephone surveys using RDD landline sample. Dual frame (landline and cell phone) samples are increasingly used in telephone surveys to address this issue. However, there is no consensus on the best design for dual frame surveys: segmented designs which screen out respondents with landlines from the cell phone (wireless) sampling frame, or overlapping designs which include dual users from both wireless and landline samples. A national telephone survey of 2,501 adults was conducted in May 2009 using an overlapping dual frame design. The two designs can be compared by including (overlapping design) or excluding (segmented design) the dual users from the wireless sample. A comparison of the sample characteristics of this survey to Census population estimates finds the overlapping design generally yields results closer to population parameters than the segmented design. The study also finds significant differences in the characteristics of dual users from the two samples, explaining why the segmented design yields different and potentially less accurate population estimates.

Design and field procedures in the US National Comorbidity Survey Replication Adolescent Supplement (NCS‐A)

An overview is presented of the design and field procedures of the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A), a US face-to-face household survey of the prevalence and correlates of DSM-IV mental disorders. The survey was based on a dual-frame design that included 904 adolescent residents of the households that participated in the US National Comorbidity Survey Replication (85.9% response rate) and 9244 adolescent students selected from a nationally representative sample of 320 schools (74.7% response rate). After expositing the logic of dual-frame designs, comparisons are presented of sample and population distributions on Census socio-demographic variables and, in the school sample, school characteristics. These document only minor differences between the samples and the population. The results of statistical analysis of the bias-efficiency trade-off in weight trimming are then presented. These show that modest trimming meaningfully reduces mean squared error. Analysis of comparative sample efficiency shows that the household sample is more efficient than the school sample, leading to the household sample getting a higher weight relative to its size in the consolidated sample relative to the school sample. Taken together, these results show that the NCS-A is an efficient sample of the target population with good representativeness on a range of socio-demographic and geographic variables.

National Comorbidity Survey Replication Adolescent Supplement (NCS-A): II. Overview and Design

ObjectiveTo present an overview of the design and field procedures of the National Comorbidity Survey Replication Adolescent Supplement (NCS-A). MethodThe NCS-A is a nationally representative face-to-face household survey of the prevalence and correlates of DSM-IV mental disorders among U.S. adolescents (aged 13–17 years) that was performed between February 2001 and January 2004 by the Survey Research Center of the Institute for Social Research at the University of Michigan. The sample was based on a dual-frame design that included 904 adolescent residents of the households that participated in the National Comorbidity Survey Replication (response rate 85.9%) and 9,244 adolescent students selected from a representative sample of 320 schools in the same nationally representative sample of counties as the National Comorbidity Survey Replication (response rate 74.7%). ResultsComparisons of sample and population distributions on census sociodemographic variables and, in the school sample, school characteristics documented only minor differences that were corrected with poststratification weighting. Comparisons of DSM-IV disorder prevalence estimates among household versus school sample respondents in counties that differed in the use of replacement schools for originally selected schools that refused to participate showed that the use of replacement schools did not introduce bias into prevalence estimates. ConclusionsThe NCS-A is a rich nationally representative dataset that will substantially increase understanding of the mental health and well-being of adolescents in the United States.

Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling

Developing cost effective methods for reaching households which no longer have a landline but do have access to a cell phone, so called cell phone only households (CPOs), is a critical item on the agenda of most data collection organizations. Concerns about sample coverage and data biases resulting from the exclusion of CPOs have increased over the past few years as the penetration of CPO households continues to climb, exceeding 50% in some subgroups. To date, two methodologies have emerged as potential means for addressing this issue. The first involves sampling telephone numbers from known cell phone exchanges and calling these numbers or combining these with a sample of landline numbers in a dual frame design. This approach can be further refined by either interviewing all of those who answer the cell phone regardless of whether they have a landline or screening cell phone respondents to identify those living in CPO households. An alternative approach involves sampling of addresses rather than telephone numbers. Address based sampling (ABS) is a new technique built upon the relatively recent availability of large scale address databases. For example, residential address data from the U.S. Postal Service provides nearly 98% coverage of U.S. households. Further, these addresses can be reverse-matched to commercially available databases to identify a relatively large proportion of telephone numbers, facilitating the use of mixed-mode approaches. Here we delineate and compare the advantages and limitations of these two approaches, including discussion of sampling and weighting approaches, operational considerations, timeliness, and cost. We draw examples from several recently conducted studies employing each of these approaches. The findings from this research help bring into sharper focus the potential alternatives to traditional random digit dialed (RDD) telephone surveys for conducting cost effective data collection of the general public.