In reply

Abstract

I encourage readers of the articles1Cooper R.J. NHAMCS: does it hold up to scrutiny?.Ann Emerg Med. 2012; 60: 722-725Abstract Full Text Full Text PDF PubMed Scopus (67) Google Scholar, 2McCaig L.F. Burt C.W. Understanding and interpreting the National Hospital Ambulatory Medical Care Survey: key questions and answers.Ann Emerg Med. 2012; 60: 716-721Abstract Full Text Full Text PDF PubMed Scopus (164) Google Scholar and subsequent correspondence3Carlson J.N. Wang H.E. The challenge of analyzing and interpreting NHAMCS Letter to the editor re: NHAMCS: does it hold up to scrutiny?.Ann Emerg Med. 2013; 62: 99-100Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar, 4Green S.M. Cooper R.J. In reply: the challenge of analyzing and interpreting NHAMCS.Ann Emerg Med. 2013; 62: 100Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar, 5McCaig L.F. Burt C.W. Schappert S.M. et al.Letter to the editor regarding NHAMCS: does it hold up to scrutiny.Ann Emerg Med. 2013; 62: 548-549Abstract Full Text Full Text PDF PubMed Google Scholar, 6Pitts S.R. Medical records, chart reviews, and NHAMCS: becoming the lords of all that we survey.http://dx.doi.org/10.1016/j.annemergmed.2013.03.003Google Scholar about the quality of data abstraction in the National Hospital Ambulatory Medical Care Survey (NHAMCS) to think of the entire process as a quality improvement cycle rather than a question of who is correct and who is wrong. NHAMCS has been an invaluable source of information. Many estimates I am sure are accurate; however, there are a number of issues I have cited that raise questions about the data. This is not an indictment of NHAMCS staff or directors, or the hard-working abstractors. Science advances when we look at what we do and review our work with a critical eye and moves forward when we acknowledge our errors or possible errors and put in processes to correct them. We are all fallible: readers will note the corrections to my editorial. However, these issues do not change the concerns about NHAMCS data. McCaig et al5McCaig L.F. Burt C.W. Schappert S.M. et al.Letter to the editor regarding NHAMCS: does it hold up to scrutiny.Ann Emerg Med. 2013; 62: 548-549Abstract Full Text Full Text PDF PubMed Google Scholar indicate that authors of articles I cited appropriately addressed problems or concerns with NHAMCS data in the limitations. This is important; however, acknowledging that limitations exist is different from accounting for limitations when making conclusions or claims. Assuming that the limitation does not change the study results can be misleading. Addressing a limitation that could have been corrected in the study design, for example, by improving data collection (medically trained chart abstractors with specific training and orientation to the charting specifics of a given center, assessment of interrater reliability, and following other chart review standards), is the major issue. When there are divergent results from 2 studies, it is important to further investigate and determine whether the 2 studies are both correct but represent different populations, whether one is correct and one is not, or whether both are wrong. I suspect there are many instances in which the NHAMCS data are accurate but the investigators' interpretation of the findings is not. In other instances, it seems the NHAMCS data may not be accurate, and I maintain that for some studies authors have erroneously assumed that NHAMCS data must be correct. McCaig et al5McCaig L.F. Burt C.W. Schappert S.M. et al.Letter to the editor regarding NHAMCS: does it hold up to scrutiny.Ann Emerg Med. 2013; 62: 548-549Abstract Full Text Full Text PDF PubMed Google Scholar note that it is important to consider the methodology of each study, and I agree. The answers we obtain and conclusions we make can vary considerably if we collect data retrospectively from chart review as opposed to prospectively. The issue of possible oligoanalgesia is probably not best answered by retrospective data collection (NHAMCS or other studies based on chart review). Green7Green S.M. There is oligo-evidence for oligoanalgesia.Ann Emerg Med. 2012; 60: 212-214Abstract Full Text Full Text PDF PubMed Scopus (29) Google Scholar outlines many of the problems with studies of oligoanalgesia, which I will not repeat here. The field representatives collecting data and the statisticians and investigators at NHAMCS work incredibly hard to produce the NHAMCS data sets. The editorial by Pitts6Pitts S.R. Medical records, chart reviews, and NHAMCS: becoming the lords of all that we survey.http://dx.doi.org/10.1016/j.annemergmed.2013.03.003Google Scholar provides further insight and recognition of this hard work. His insights add credence to my previous statements that the data collection process is not ideal in adhering to best practices. He also correctly puts some of the blame on emergency physicians: a nonstandard chart and different abbreviations and definitions all can contribute to abstraction problems for non–medically trained chart reviewers. This is why prospective data should be collected to confirm findings whenever possible and why I suggest that NHAMCS data should usually be hypothesis generating. I can understand that some disagree with my questions about the validity of NHAMCS data and believe we do not have enough information to know the truth about which estimates or data are in error. But if what I have written leads to more discussion and if it encourages NHAMCS to improve their processes and authors to better account for the possibility of error in the NHAMCS data, then I have met my goals. NHAMCS: Does It Hold Up to Scrutiny?Annals of Emergency MedicineVol. 62Issue 5PreviewThe Centers for Disease Control and Prevention's (CDC's) National Center for Health Statistics (NCHS) is committed to collecting and disseminating high-quality data that meet the information needs of a wide range of users. We take issue with the unfavorable assessment of National Hospital Ambulatory Medical Care Survey (NHAMCS) data in a recent editorial by Cooper1 and appreciate the opportunity to respond to her comments. Full-Text PDF