Pediatric patients with sepsis are identified using related but distinct criteria for clinical, research, and administrative purposes. The overlap between these criteria will affect the validity of extrapolating data across settings. We sought to quantify the extent of agreement among different criteria for pediatric severe sepsis/septic shock and to detect systematic differences between these cohorts. Observational cohort study. Forty-two bed pediatric intensive care unit at an academic medical center. A total of 1,729 patients ≤ 18 yrs-old. None. All patients were screened for severe sepsis or septic shock using consensus guidelines (research criteria), diagnosis by healthcare professionals (clinical criteria), and International Classification of Diseases, Ninth Revision, Clinical Modification codes (administrative criteria). Cohen's κ determined the level of agreement among criteria, and patient characteristics were compared between cohorts. Ninety (5.2%) patients were identified by research, 96 (5.6%) by clinical, and 103 (6.0%) by administrative criteria. The κ ± standard error for pair-wise comparisons was 0.67 ± 0.04 for research-clinical, 0.52 ± 0.05 for research-administrative, and 0.55 ± 0.04 for clinical-administrative. Of the patients in the clinical cohort, 67% met research and 58% met administrative criteria. The research cohort exhibited a higher Pediatric Index of Mortality-2 score (median, interquartile range 5.2, 1.6-13.3) than the clinical (3.6, 1.1-6.2) and administrative (3.9, 1.0-6.0) cohorts (p = .005), an increased requirement for vasoactive infusions (74%, 57%, and 45%, p < .001), and a potential bias toward an increased proportion with respiratory dysfunction compared with clinical practice. Although research, clinical, and administrative criteria yielded a similar incidence (5%-6%) for pediatric severe sepsis/septic shock, there was only a moderate level of agreement in the patients identified by different criteria. One third of patients diagnosed clinically with sepsis would not have been included in studies based on consensus guidelines or International Classification of Diseases, Ninth Revision, Clinical Modification codes. Differences in patient selection need to be considered when extrapolating data across settings.