The rocky exhilarating journey from data to wisdom

Abstract

Central MessageBoth clinical and administrative sources of data have an important role in quality improvement. The emerging field of data science will help to build the knowledge foundation of the future.This Invited Expert Opinion provides a perspective on the following papers: J Am Coll Surg. 2009;209(5):551-556. https://doi.org/10.1016/j.jamcollsurg.2009.08.008; J Am Coll Surg. 2016;223(4):551-557.e4. https://doi.org/10.1016/j.jamcollsurg.2016.06.393.See Commentaries on pages 1170 and 1171.Quality assessment in medicine is not a new phenomenon. Although somewhat anathema to modern medical ethics, the Code of Hammurabi over three and half millennia ago had very specific standards for physicians. The better-known guidelines of Hippocrates and Maimonides still resonate with physicians to this day. However, the modern era of quality assessment in health care stemmed from the efforts of Florence Nightingale to apply analytical standards to the care of soldiers during the Crimean War—based on, yes, data. Similar efforts by pioneering surgeon Ernest Codman gained him the invitation to leave the medical staff of the Massachusetts General Hospital but laid the foundation for what was later to become the American College of Surgeons and the Joint Commission. When disturbing reports of the lapses in quality from the Institute of Medicine converged with the necessity to better understand the value gained for the rapidly increasing costs of medical care, the need for reliable information to assess quality became paramount. In 1986, the Healthcare Finance Administration began releasing mortality reports based on administrative data that suggested a 5-fold variation in mortality for coronary artery bypass surgery in New York State. Concerned with the validity of the data as well as the findings, the state Department of Health decided to create a patient-level clinical database that could be used to assess institutional outcomes for coronary artery bypass surgery sensitive to differences in patient acuity.1Hannan E.L. Cozzens K. King S.B. Walford G. Shah N.R. History, contributions, limitations, and lessons for future efforts to assess and publicly report healthcare outcomes.J Am Coll Cardiol. 2012; 59: 2309-2316Crossref PubMed Scopus (126) Google Scholar Subsequent study documented not only the disparity between the administrative and clinical database approaches but found the registry to be more predictive of mortality.2Hannan E.L. Kilburn H. Lindsey M.L. Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter?.Med Care. 1992; 30: 892-907Crossref PubMed Scopus (215) Google ScholarPrecedent for the clinical approach to quality improvement had already been set by the Northern New England Cardiovascular Disease Study Group which, in 1987, began collecting clinical data on cardiac surgical outcomes in a voluntary effort to identify areas for quality improvement.3O'Connor G.T. Plume S.K. Olmstead E.M. A regional prospective study of in-hospital mortality associated with coronary artery bypass grafting.JAMA. 1991; 266: 803-809Crossref PubMed Scopus (346) Google Scholar Similarly, as early as 1984 surgeons from the Society of Thoracic Surgeons (STS) began exploring the feasibility of creating a tool to facilitate comparison of outcomes. Spurred by the release of Healthcare Finance Administration data, development of a national database was approved, and by 1990 the first iteration of the STS National database was released with 50 participants.Despite some initial challenges, the database has grown in scope, sophistication, and penetration, with more than 1000 national and international participating sites, with more than 6 million records that represent upward of 90% of the nongovernmental sites performing cardiac surgery in the country, and more than 95% of the patients.4D'Agostino R.S. Jacobs J.P. Badhwar V. Paone G. Wormuth D.W. Shahian D.M. The Society of Thoracic Surgeons adult cardiac surgery database: 2019 update on outcomes and quality.Ann Thorac Surg. 2019; 107: 24-32Abstract Full Text Full Text PDF PubMed Scopus (133) Google Scholar,5Jacobs J.P. Shahian D.M. He X. O'Brien S.M. Badhwar V. Cleveland Jr., J.C. et al.Penetration, completeness, and representativeness of the Society of Thoracic Surgeons adult cardiac surgery database.Ann Thorac Surg. 2016; 101: 33-41Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar Careful data analytics have been applied to generate well-validated risk models for mortality and morbidity of the most commonly performed cardiac surgical operations.6Shahian D.M. Jacobs J.P. Badhwar V. Kurlansky P.A. Furnary A.P. Cleveland Jr., J.C. et al.The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 1-background, design considerations, and model development.Ann Thorac Surg. 2018; 105: 1411-1418Abstract Full Text Full Text PDF PubMed Scopus (169) Google Scholar,7O'Brien S.M. Feng L. He X. Xian Y. Jacobs J.P. Badhwar V. et al.The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 2-statistical methods and results.Ann Thorac Surg. 2018; 105: 1419-1428Abstract Full Text Full Text PDF PubMed Scopus (205) Google Scholar Although voluntary, the database is regularly audited and data related to risk models have been found to have >95% accuracy.4D'Agostino R.S. Jacobs J.P. Badhwar V. Paone G. Wormuth D.W. Shahian D.M. The Society of Thoracic Surgeons adult cardiac surgery database: 2019 update on outcomes and quality.Ann Thorac Surg. 2019; 107: 24-32Abstract Full Text Full Text PDF PubMed Scopus (133) Google Scholar Modeling has further permitted composite ratings for both programs and surgeons, and has facilitated voluntary public reporting. Simultaneously, several states—most notably New York, New Jersey, Massachusetts, Pennsylvania, and California—maintain compulsory independent cardiac surgery reporting systems, with varying degrees of public reporting, some of which rely on STS data, some of which maintain independent data systems, and some of which combine administrative with select clinical data. In other states—Virginia, Michigan, and Washington—surgeons have formed quality collaboratives through which they share their STS data in a collective effort to drive quality improvement.So, the issue has been settled—history has validated the superiority of the clinical database. Not so fast! Administrative data abound, and their impact are profound. Over the past decade, the National Quality Forum has endorsed more than 700 measures, many, if not most, based on administrative data. The Centers for Medicare and Medicaid Services maintains several claims databases that are not only used for reimbursement purposes but serve the basis for quality benchmarking and reimbursement decisions, such as the recently instituted and somewhat-controversial metrics for readmissions.8Wadhera R.K. Maddox K.E.J. Wasfy J.H. Haneurse S. Shen C. Yeh R.W. Association of the hospital readmission reduction program with mortality among Medicare beneficiaries hospitalized for heart failure, acute myocardial infarction and pneumonia.JAMA. 2018; 320: 2542-2552Crossref PubMed Scopus (206) Google Scholar It is not surprising that administrative data are widely used—they are readily available, transmissible electronically, relatively uniform across health plans, payers, and medical institutions, and can be obtained for entire populations. Depending on the source, such data can combine the inpatient and outpatient experience, track medication usage, record costs, and link with state or national death indices to study both short- and long-term outcomes. Indeed, various national health plan databases, such those in Taiwan,9Wu C.K. Juang J.M.J. Chiang J.Y. Li Y.H. Tsai C.T. Chiang F.T. The Taiwan heart registries: its influence on cardiovascular patient care.J Am Coll Cardiol. 2018; 71: 1273-1283Crossref PubMed Scopus (24) Google Scholar Denmark,10Christiansen M.N. Køber L. Weeke P. Vasan R.S. Jeppesen J.L. Smith J.G. et al.Age-specific trends in incidence, mortality, and comorbidities of heart failure in Denmark, 1995 to 2012.Circulation. 2017; 135: 1214-1223Crossref PubMed Scopus (123) Google Scholar Canada,11Pu A. Ding L. Shin J. Price J. Skarsgard P. Wong D.R. et al.Long-term outcomes of multiple arterial coronary artery bypass grafting: a population-based study of patients in British Columbia, Canada.JAMA Cardiol. 2017; 2: 1187-1196Crossref PubMed Scopus (35) Google Scholar and elsewhere, have produced clinical insights that would not have otherwise been readily obtainable.So why the “love affair” with clinical registries? They can be labor-intensive, costly, and potentially burdensome for institutions to maintain. Indeed, the STS adult cardiac surgery database has swollen to several hundred fields, each of which requires a specific trained data manager entry. It is no surprise that many of the fields not currently incorporated into the risk models lie blank. Potentially meaningful information such as MELD (Model for End-stage Liver Disease) score of liver function and 5-meter walk test, a validated measure of frailty, are largely left blank and therefore unavailable for risk adjustment. With such a heavy burden of data entry, imagine if every medical specialty required this degree of applied resources—what hospital or health care system could sustain such an expense? And yet, especially in an area such as cardiac surgery, for which quality bears life and death consequences, and for which so much institutional resource and commitment are required to appropriately support high-level performance, one might ask what institution could afford NOT to have the ready availability of high-level clinical data.Administrative data may simply not be up to the task. First of all, even when aggregated on a national level specifically for the purpose of studying health care, such as the Agency for Healthcare Research and Quality has done with the Healthcare Cost and Utilization Project, the data are not readily usable. Analysis has shown that 85% of published studies using the Healthcare Cost and Utilization Project's National Inpatient Sample, including those in high-impact journals, did not adhere to at least 1 or more of required methodologic practices.12Khera R.J. Angraal S. Souch T. Welsh J.W. Nallamothu B.K. Girotra S. et al.Adherence to methodological standards in research using the national inpatient sample.JAMA. 2017; 318: 2011-2018Crossref PubMed Scopus (363) Google Scholar On a more granular level, statewide,13Shahian D.M. Silverstein T. Lovett A.F. Wolf R.E. Normand S.L.T. Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards.Circulation. 2007; 115: 1518-1527Crossref PubMed Scopus (156) Google Scholar regional,14Mack M.J. Herbert M. Prince S. Dewey T.M. Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes?.J Thorac Cardiovasc Surg. 2005; 129: 1309-1317Abstract Full Text Full Text PDF PubMed Scopus (85) Google Scholar and individual institutional studies15Prasad A. Helder M.R. Brown D.A. Schaff H.V. Understanding differences in administrative and audited patient data in cardiac surgery: comparison of the University HealthSystem Consortium and Society of Thoracic Surgeons databases.J Am Coll Surg. 2016; 223: 551-557.e4Abstract Full Text Full Text PDF PubMed Scopus (16) Google Scholar have all documented various degrees of inadequacy in institutional data. The finding of discrepancies in volume are particularly unsettling, since one might justifiably expect data amassed for billing purposes would, at very least, provide accurate information regarding the procedures performed, the date of surgery, and the hospital mortality. More careful analysis uncovered various plausible explanations—different ways of categorizing operations (isolated vs various combinations of procedures), different time frames (date of surgery vs date of discharge), different ways of categorizing patients (Medicare vs Medicare >65 vs Medicare Advantage, etc), and different patient groupings (adult cardiac surgery vs inclusion of adult congenital surgery). More challenging is the absence of potentially important clinical variables. Even the well-validated Charlson16Charlson M. Szatrowski T.P. Peterson J. Gold J. Validation of a combined comorbidity index.J Clin Epidemiol. 1994; 47: 1245-1251Abstract Full Text PDF PubMed Scopus (4407) Google Scholar and Elixhauser17Elixhauser A. Steiner C. Harris D.R. Coffey R.M. Comorbidity measures for use with administrative data.Med Care. 1998; 36: 8-27Crossref PubMed Scopus (6472) Google Scholar comorbidity indices cannot capture such potentially important variables as left ventricular ejection fraction, urgency of surgery, extent of left main other coronary artery disease, degree of mitral or other valvular regurgitation, etc. Moreover, it is frequently difficult to differentiate between comorbidities that were present before surgery versus those that accrued in the postoperative period. “Present on admission” notation, where available, as well as revised codes to more clearly delineate complications have been helpful but lack the definition and distinction of well-organized clinical databases.18Fry D.E. Pine M. Jordan H. Elixhauser A. Hoaglin D.C. Jones B. et al.Combining administrative and clinical data to stratify surgical risk.Ann Surg. 2007; 246: 875-885Crossref PubMed Scopus (66) Google Scholar Further confusion emerges from this paradigm when administrative databases such as the University HealthSystem Consortium (now Vizient), which shares data among more than 300 hospitals and represents more than 90% of the nation's nonprofit academic medical centers, develops diagnosis-related group–based risk models that achieve high discriminatory power by inclusion of postoperative events in the model—a potentially useful method for modeling cost but clearly inadequate for assessing performance across the spectrum of patient risk.19Kozower B.D. Invited commentary: administrative vs clinical data: the struggle continues.J Am Coll Surg. 2016; 223: 557-558Abstract Full Text Full Text PDF PubMed Scopus (1) Google ScholarDespite their more robust clinical perspective, even the most comprehensive databases such the STS are frequently not adequate for determining many subtle surgeon or hospital performance decisions that may impact major outcomes of interest (eg, what specifically drives the variability in the rate of stroke across centers or surgeons?). Moreover, most clinical databases are severely limited by their focus on perioperative events. The mere extension to the 30-day period may expose gaps in reporting when rectified with administrative data.20Hannan E.L. Samadashvili Z. Cozzens K. Chikwe J. Adams D.H. Sundt III, T.M. et al.Out-of-hospital 30-day deaths following cardiac surgery are often under-reported.Ann Thorac Surg. 2020; 110: 183-188Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar,21Edgerton J.R. Herbert M.A. Hamman B.L. Ring W.S. Can use of an administrative database improve accuracy of hospital-reported readmission rates?.J Thorac Cardiovasc surg. 2018; 155: 2043-2047Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar Even though perioperative outcomes are critical for assessing quality of surgical care, more long-term data are essential to perform the vital function of determining quality in surgical decision-making: what is the impact of multiple arterial grafting? Should operative treatment of atrial fibrillation at the time of surgery be considered a metric of quality? Successful linkage of clinical and administrative data such as the STS has performed with ASCERT (ACCF and STS Database Collaboration on the Comparative Effectiveness of Revascularization Strategies) trial of coronary revascularization, as well as several other studies, are certainly a step in the right direction.22Weintraub W.S. Grau-Sepulveda M.V. Weiss J.M. O'Brien S.M. Peterson E.D. Kolm P. et al.Comparative effectiveness of revascularization strategies.N Engl J Med. 2012; 366: 1467-1476Crossref PubMed Scopus (456) Google Scholar Current initiatives to more formally link the STS with National Death Index would be a most welcome advance. Similar linkage might provide important information regarding cost, better addressing the core issue of value that is driving much of this initiative. Even with such linkage, we still will need to find ways to integrate patient perceptions of care, an emerging field that in and of itself is evolving more adept ways of capturing meaningful data.Limitations of our current clinical databases have not gone unnoticed by the professional societies.23Blackstone EH. A Comprehensive AATS Quality Program for the Twenty-first Century. Presented at: 99th Annual Meeting of The American Association for Thoracic Surgery, Toronto, Ontario, Canada, May 4-7, 2019.Google Scholar Access to real-time information from the electronic medical record is an appealing prospect but will require much attention to issues of lack of electronic medical record interoperability, lack of accuracy promulgated by ubiquitous use of “copy/paste” functionality, limitations of semantic logic to translate raw and/or unstructured information into discrete clinical variables, and ever-increasing issues of data security and privacy as data are transferred across electronic networks. Despite these challenges, the emerging field of data science is providing methodologies—machine learning, neural networks, and other forms of artificial intelligence, in all of their many evolving forms to provide data-driven insights from enormous repositories of data. Already huge data warehouses such as Optum have merged administrative, clinical, and linkage streams that might not only provide the information necessary to address specific clinical questions24Yao X. Gersh B.J. Holmes D.R. Melduni R.M. Johnsrud D.O. Sangarlingham L.R. et al.Association of surgical left atrial appendage occlusion with subsequent stroke and mortality among patients undergoing cardiac surgery.JAMA. 2018; 319: 2116-2126Crossref PubMed Scopus (82) Google Scholar but use “big-data” approaches to provide previously unanticipated data-driven insights. Such approaches have already yielded exciting insights not apparent from a hypothesis-driven approach—with the increasing demands of precision medicine, access to and linkage of such large population resources will become essential for addressing the major issues on the horizon.25Krumholz H.M. Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system.Health Aff (Millwood). 2014; 33: 1163-1170Crossref PubMed Scopus (322) Google ScholarIn short: clinical or administrative? Yes. And much more. Now let's get to work!Conflict of Interest StatementThe author reported no conflicts of interest.The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. Both clinical and administrative sources of data have an important role in quality improvement. The emerging field of data science will help to build the knowledge foundation of the future. Both clinical and administrative sources of data have an important role in quality improvement. The emerging field of data science will help to build the knowledge foundation of the future. This Invited Expert Opinion provides a perspective on the following papers: J Am Coll Surg. 2009;209(5):551-556. https://doi.org/10.1016/j.jamcollsurg.2009.08.008; J Am Coll Surg. 2016;223(4):551-557.e4. https://doi.org/10.1016/j.jamcollsurg.2016.06.393. This Invited Expert Opinion provides a perspective on the following papers: J Am Coll Surg. 2009;209(5):551-556. https://doi.org/10.1016/j.jamcollsurg.2009.08.008; J Am Coll Surg. 2016;223(4):551-557.e4. https://doi.org/10.1016/j.jamcollsurg.2016.06.393. See Commentaries on pages 1170 and 1171. See Commentaries on pages 1170 and 1171. Quality assessment in medicine is not a new phenomenon. Although somewhat anathema to modern medical ethics, the Code of Hammurabi over three and half millennia ago had very specific standards for physicians. The better-known guidelines of Hippocrates and Maimonides still resonate with physicians to this day. However, the modern era of quality assessment in health care stemmed from the efforts of Florence Nightingale to apply analytical standards to the care of soldiers during the Crimean War—based on, yes, data. Similar efforts by pioneering surgeon Ernest Codman gained him the invitation to leave the medical staff of the Massachusetts General Hospital but laid the foundation for what was later to become the American College of Surgeons and the Joint Commission. When disturbing reports of the lapses in quality from the Institute of Medicine converged with the necessity to better understand the value gained for the rapidly increasing costs of medical care, the need for reliable information to assess quality became paramount. In 1986, the Healthcare Finance Administration began releasing mortality reports based on administrative data that suggested a 5-fold variation in mortality for coronary artery bypass surgery in New York State. Concerned with the validity of the data as well as the findings, the state Department of Health decided to create a patient-level clinical database that could be used to assess institutional outcomes for coronary artery bypass surgery sensitive to differences in patient acuity.1Hannan E.L. Cozzens K. King S.B. Walford G. Shah N.R. History, contributions, limitations, and lessons for future efforts to assess and publicly report healthcare outcomes.J Am Coll Cardiol. 2012; 59: 2309-2316Crossref PubMed Scopus (126) Google Scholar Subsequent study documented not only the disparity between the administrative and clinical database approaches but found the registry to be more predictive of mortality.2Hannan E.L. Kilburn H. Lindsey M.L. Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter?.Med Care. 1992; 30: 892-907Crossref PubMed Scopus (215) Google Scholar Precedent for the clinical approach to quality improvement had already been set by the Northern New England Cardiovascular Disease Study Group which, in 1987, began collecting clinical data on cardiac surgical outcomes in a voluntary effort to identify areas for quality improvement.3O'Connor G.T. Plume S.K. Olmstead E.M. A regional prospective study of in-hospital mortality associated with coronary artery bypass grafting.JAMA. 1991; 266: 803-809Crossref PubMed Scopus (346) Google Scholar Similarly, as early as 1984 surgeons from the Society of Thoracic Surgeons (STS) began exploring the feasibility of creating a tool to facilitate comparison of outcomes. Spurred by the release of Healthcare Finance Administration data, development of a national database was approved, and by 1990 the first iteration of the STS National database was released with 50 participants. Despite some initial challenges, the database has grown in scope, sophistication, and penetration, with more than 1000 national and international participating sites, with more than 6 million records that represent upward of 90% of the nongovernmental sites performing cardiac surgery in the country, and more than 95% of the patients.4D'Agostino R.S. Jacobs J.P. Badhwar V. Paone G. Wormuth D.W. Shahian D.M. The Society of Thoracic Surgeons adult cardiac surgery database: 2019 update on outcomes and quality.Ann Thorac Surg. 2019; 107: 24-32Abstract Full Text Full Text PDF PubMed Scopus (133) Google Scholar,5Jacobs J.P. Shahian D.M. He X. O'Brien S.M. Badhwar V. Cleveland Jr., J.C. et al.Penetration, completeness, and representativeness of the Society of Thoracic Surgeons adult cardiac surgery database.Ann Thorac Surg. 2016; 101: 33-41Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar Careful data analytics have been applied to generate well-validated risk models for mortality and morbidity of the most commonly performed cardiac surgical operations.6Shahian D.M. Jacobs J.P. Badhwar V. Kurlansky P.A. Furnary A.P. Cleveland Jr., J.C. et al.The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 1-background, design considerations, and model development.Ann Thorac Surg. 2018; 105: 1411-1418Abstract Full Text Full Text PDF PubMed Scopus (169) Google Scholar,7O'Brien S.M. Feng L. He X. Xian Y. Jacobs J.P. Badhwar V. et al.The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 2-statistical methods and results.Ann Thorac Surg. 2018; 105: 1419-1428Abstract Full Text Full Text PDF PubMed Scopus (205) Google Scholar Although voluntary, the database is regularly audited and data related to risk models have been found to have >95% accuracy.4D'Agostino R.S. Jacobs J.P. Badhwar V. Paone G. Wormuth D.W. Shahian D.M. The Society of Thoracic Surgeons adult cardiac surgery database: 2019 update on outcomes and quality.Ann Thorac Surg. 2019; 107: 24-32Abstract Full Text Full Text PDF PubMed Scopus (133) Google Scholar Modeling has further permitted composite ratings for both programs and surgeons, and has facilitated voluntary public reporting. Simultaneously, several states—most notably New York, New Jersey, Massachusetts, Pennsylvania, and California—maintain compulsory independent cardiac surgery reporting systems, with varying degrees of public reporting, some of which rely on STS data, some of which maintain independent data systems, and some of which combine administrative with select clinical data. In other states—Virginia, Michigan, and Washington—surgeons have formed quality collaboratives through which they share their STS data in a collective effort to drive quality improvement. So, the issue has been settled—history has validated the superiority of the clinical database. Not so fast! Administrative data abound, and their impact are profound. Over the past decade, the National Quality Forum has endorsed more than 700 measures, many, if not most, based on administrative data. The Centers for Medicare and Medicaid Services maintains several claims databases that are not only used for reimbursement purposes but serve the basis for quality benchmarking and reimbursement decisions, such as the recently instituted and somewhat-controversial metrics for readmissions.8Wadhera R.K. Maddox K.E.J. Wasfy J.H. Haneurse S. Shen C. Yeh R.W. Association of the hospital readmission reduction program with mortality among Medicare beneficiaries hospitalized for heart failure, acute myocardial infarction and pneumonia.JAMA. 2018; 320: 2542-2552Crossref PubMed Scopus (206) Google Scholar It is not surprising that administrative data are widely used—they are readily available, transmissible electronically, relatively uniform across health plans, payers, and medical institutions, and can be obtained for entire populations. Depending on the source, such data can combine the inpatient and outpatient experience, track medication usage, record costs, and link with state or national death indices to study both short- and long-term outcomes. Indeed, various national health plan databases, such those in Taiwan,9Wu C.K. Juang J.M.J. Chiang J.Y. Li Y.H. Tsai C.T. Chiang F.T. The Taiwan heart registries: its influence on cardiovascular patient care.J Am Coll Cardiol. 2018; 71: 1273-1283Crossref PubMed Scopus (24) Google Scholar Denmark,10Christiansen M.N. Køber L. Weeke P. Vasan R.S. Jeppesen J.L. Smith J.G. et al.Age-specific trends in incidence, mortality, and comorbidities of heart failure in Denmark, 1995 to 2012.Circulation. 2017; 135: 1214-1223Crossref PubMed Scopus (123) Google Scholar Canada,11Pu A. Ding L. Shin J. Price J. Skarsgard P. Wong D.R. et al.Long-term outcomes of multiple arterial coronary artery bypass grafting: a population-based study of patients in British Columbia, Canada.JAMA Cardiol. 2017; 2: 1187-1196Crossref PubMed Scopus (35) Google Scholar and elsewhere, have produced clinical insights that would not have otherwise been readily obtainable. So why the “love affair” with clinical registries? They can be labor-intensive, costly, and potentially burdensome for institutions to maintain. Indeed, the STS adult cardiac surgery database has swollen to several hundred fields, each of which requires a specific trained data manager entry. It is no surprise that many of the fields not currently incorporated into the risk models lie blank. Potentially meaningful information such as MELD (Model for End-stage Liver Disease) score of liver function and 5-meter walk test, a validated measure of frailty, are largely left blank and therefore unavailable for risk adjustment. With such a heavy burden of data entry, imagine if every medical specialty required this degree of applied resources—what hospital or health care system could sustain such an expense? And yet, especially in an area such as cardiac surgery, for which quality bears life and death consequences, and for which so much institutional resource and commitment are required to appropriately support high-level performance, one might ask what institution could afford NOT to have the ready availability of high-level clinical data. Administrative data may simply not be up to the task. First of all, even when aggregated on a national level specifically for the purpose of studying health care, such as the Agency for Healthcare Research and Quality has done with the Healthcare Cost and Utilization Project, the data are not readily usable. Analysis has shown that 85% of published studies using the Healthcare Cost and Utilization Project's National Inpatient Sample, including those in high-impact journals, did not adhere to at least 1 or more of required methodologic practices.12Khera R.J. Angraal S. Souch T. Welsh J.W. Nallamothu B.K. Girotra S. et al.Adherence to methodological standards in research using the national inpatient sample.JAMA. 2017; 318: 2011-2018Crossref PubMed Scopus (363) Google Scholar On a more granular level, statewide,13Shahian D.M. Silverstein T. Lovett A.F. Wolf R.E. Normand S.L.T. Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards.Circulation. 2007; 115: 1518-1527Crossref PubMed Scopus (156) Google Scholar regional,14Mack M.J. Herbert M. Prince S. Dewey T.M. Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes?.J Thorac Cardiovasc Surg. 2005; 129: 1309-1317Abstract Full Text Full Text PDF PubMed Scopus (85) Google Scholar and individual institutional studies15Prasad A. Helder M.R. Brown D.A. Schaff H.V. Understanding differences in administrative and audited patient data in cardiac surgery: comparison of the University HealthSystem Consortium and Society of Thoracic Surgeons databases.J Am Coll Surg. 2016; 223: 551-557.e4Abstract Full Text Full Text PDF PubMed Scopus (16) Google Scholar have all documented various degrees of inadequacy in institutional data. The finding of discrepancies in volume are particularly unsettling, since one might justifiably expect data amassed for billing purposes would, at very least, provide accurate information regarding the procedures performed, the date of surgery, and the hospital mortality. More careful analysis uncovered various plausible explanations—different ways of categorizing operations (isolated vs various combinations of procedures), different time frames (date of surgery vs date of discharge), different ways of categorizing patients (Medicare vs Medicare >65 vs Medicare Advantage, etc), and different patient groupings (adult cardiac surgery vs inclusion of adult congenital surgery). More challenging is the absence of potentially important clinical variables. Even the well-validated Charlson16Charlson M. Szatrowski T.P. Peterson J. Gold J. Validation of a combined comorbidity index.J Clin Epidemiol. 1994; 47: 1245-1251Abstract Full Text PDF PubMed Scopus (4407) Google Scholar and Elixhauser17Elixhauser A. Steiner C. Harris D.R. Coffey R.M. Comorbidity measures for use with administrative data.Med Care. 1998; 36: 8-27Crossref PubMed Scopus (6472) Google Scholar comorbidity indices cannot capture such potentially important variables as left ventricular ejection fraction, urgency of surgery, extent of left main other coronary artery disease, degree of mitral or other valvular regurgitation, etc. Moreover, it is frequently difficult to differentiate between comorbidities that were present before surgery versus those that accrued in the postoperative period. “Present on admission” notation, where available, as well as revised codes to more clearly delineate complications have been helpful but lack the definition and distinction of well-organized clinical databases.18Fry D.E. Pine M. Jordan H. Elixhauser A. Hoaglin D.C. Jones B. et al.Combining administrative and clinical data to stratify surgical risk.Ann Surg. 2007; 246: 875-885Crossref PubMed Scopus (66) Google Scholar Further confusion emerges from this paradigm when administrative databases such as the University HealthSystem Consortium (now Vizient), which shares data among more than 300 hospitals and represents more than 90% of the nation's nonprofit academic medical centers, develops diagnosis-related group–based risk models that achieve high discriminatory power by inclusion of postoperative events in the model—a potentially useful method for modeling cost but clearly inadequate for assessing performance across the spectrum of patient risk.19Kozower B.D. Invited commentary: administrative vs clinical data: the struggle continues.J Am Coll Surg. 2016; 223: 557-558Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar Despite their more robust clinical perspective, even the most comprehensive databases such the STS are frequently not adequate for determining many subtle surgeon or hospital performance decisions that may impact major outcomes of interest (eg, what specifically drives the variability in the rate of stroke across centers or surgeons?). Moreover, most clinical databases are severely limited by their focus on perioperative events. The mere extension to the 30-day period may expose gaps in reporting when rectified with administrative data.20Hannan E.L. Samadashvili Z. Cozzens K. Chikwe J. Adams D.H. Sundt III, T.M. et al.Out-of-hospital 30-day deaths following cardiac surgery are often under-reported.Ann Thorac Surg. 2020; 110: 183-188Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar,21Edgerton J.R. Herbert M.A. Hamman B.L. Ring W.S. Can use of an administrative database improve accuracy of hospital-reported readmission rates?.J Thorac Cardiovasc surg. 2018; 155: 2043-2047Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar Even though perioperative outcomes are critical for assessing quality of surgical care, more long-term data are essential to perform the vital function of determining quality in surgical decision-making: what is the impact of multiple arterial grafting? Should operative treatment of atrial fibrillation at the time of surgery be considered a metric of quality? Successful linkage of clinical and administrative data such as the STS has performed with ASCERT (ACCF and STS Database Collaboration on the Comparative Effectiveness of Revascularization Strategies) trial of coronary revascularization, as well as several other studies, are certainly a step in the right direction.22Weintraub W.S. Grau-Sepulveda M.V. Weiss J.M. O'Brien S.M. Peterson E.D. Kolm P. et al.Comparative effectiveness of revascularization strategies.N Engl J Med. 2012; 366: 1467-1476Crossref PubMed Scopus (456) Google Scholar Current initiatives to more formally link the STS with National Death Index would be a most welcome advance. Similar linkage might provide important information regarding cost, better addressing the core issue of value that is driving much of this initiative. Even with such linkage, we still will need to find ways to integrate patient perceptions of care, an emerging field that in and of itself is evolving more adept ways of capturing meaningful data. Limitations of our current clinical databases have not gone unnoticed by the professional societies.23Blackstone EH. A Comprehensive AATS Quality Program for the Twenty-first Century. Presented at: 99th Annual Meeting of The American Association for Thoracic Surgery, Toronto, Ontario, Canada, May 4-7, 2019.Google Scholar Access to real-time information from the electronic medical record is an appealing prospect but will require much attention to issues of lack of electronic medical record interoperability, lack of accuracy promulgated by ubiquitous use of “copy/paste” functionality, limitations of semantic logic to translate raw and/or unstructured information into discrete clinical variables, and ever-increasing issues of data security and privacy as data are transferred across electronic networks. Despite these challenges, the emerging field of data science is providing methodologies—machine learning, neural networks, and other forms of artificial intelligence, in all of their many evolving forms to provide data-driven insights from enormous repositories of data. Already huge data warehouses such as Optum have merged administrative, clinical, and linkage streams that might not only provide the information necessary to address specific clinical questions24Yao X. Gersh B.J. Holmes D.R. Melduni R.M. Johnsrud D.O. Sangarlingham L.R. et al.Association of surgical left atrial appendage occlusion with subsequent stroke and mortality among patients undergoing cardiac surgery.JAMA. 2018; 319: 2116-2126Crossref PubMed Scopus (82) Google Scholar but use “big-data” approaches to provide previously unanticipated data-driven insights. Such approaches have already yielded exciting insights not apparent from a hypothesis-driven approach—with the increasing demands of precision medicine, access to and linkage of such large population resources will become essential for addressing the major issues on the horizon.25Krumholz H.M. Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system.Health Aff (Millwood). 2014; 33: 1163-1170Crossref PubMed Scopus (322) Google Scholar In short: clinical or administrative? Yes. And much more. Now let's get to work! Conflict of Interest StatementThe author reported no conflicts of interest.The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. Conflict of Interest StatementThe author reported no conflicts of interest.The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. The author reported no conflicts of interest. The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest. Commentary: A statistical revolution: Channeling frustration to integrationThe Journal of Thoracic and Cardiovascular SurgeryVol. 162Issue 4PreviewBanks were initially created for individuals to receive loans, make deposits to keep their monies safe, and create savings accounts that earn interest. As societies developed and modernized, the system developed multifaceted functions of financing, bankruptcies, investments, credit cards, currency exchanges, and so on. Now, imagine this complex banking system if someone had to connect 2 separate structures to see all of his or her debt in 1 place and all of his or her financial assets in another. Full-Text PDF Introduction to Expert Opinions on appropriate use of databases in cardiothoracic research: Pounding nails with a screwdriverThe Journal of Thoracic and Cardiovascular SurgeryVol. 162Issue 4PreviewLauer and Blackstone2 reviewed various types and purposes of databases in cardiology and cardiac surgery that are germane to the series of expert opinions in this issue of the Journal on the use of these databases for published research.1,3-5 No database contains all the information about clinical encounters. Rather, each type of database contains a different fraction of the information, with different granularity and specificity, on the longitudinal health care of individuals, and each is constructed for a different purpose, although they at times overlap. Full-Text PDF Commentary: The end of one journey is the beginning of the nextThe Journal of Thoracic and Cardiovascular SurgeryVol. 162Issue 4PreviewData function as the new oil in the digital world. Health care may be one of the last industries to fully digitize, but we have long used data as part of the scientific method to make decisions for our patients. Now that we are consistently collecting data on all of our patients and analyzing them on a population level, do we have the ability to answer all clinical questions with the power of large databases? Full-Text PDF