Comments on

Abstract

We would like to respond to the article entitled ‘‘Validity and reliability of a systematic database search strategy to identify publications resulting from pharmacy residency research projects.’’ We agree with the authors’ novel approach to confirm publication via a survey and to identify resident publications via a systematic approach to validate the utility of the database. The authors suggest that this approach can be used to compare publication success between residency programs. Additionally, this information can be used for marketing purposes for the individual residency program. One limitation noted by the authors was the use of only PubMed and EMBASE as the search engines to identify resident publications in the study and suggest other search engines, such as International Pharmaceutical Abstracts (IPA) and Google Scholar, may be used as alternatives. We aim to describe the methods and results of our project in which we assessed the utility of 5 different search engines to identify resident publications. Briefly, we identified the publication rate of abstracts presented at the 2003 (n 1⁄4 171), 2005 (n 1⁄4 221), and 2007 (n 1⁄4 273) Great Lakes Pharmacy Resident Conference (GLPRC) was 11.4% (n 1⁄4 76). We used a similar search strategy in Kwak et al to identify resident publication using the search engines (1) Scopus, (2) IPA, and (3) MEDLINE (PubMed). For this project, we used information from the abstracts that were previously confirmed for publications (n 1⁄4 76) to identify the proportion of these publications found in each of the following search engines: (1) Scopus, (2) IPA, (3) MEDLINE (PubMed), (4) EMBASE, and (5) Google Scholar. Two investigators (SMV and PMS) identified published research projects, and any disagreement between investigators was reviewed for consensus. A kappa statistic was calculated to determine interrater reliability for each search engine. The proportion of the published research was compared for each of the 5 search engines. Additionally, all pairs of search engines were assessed to determine the proportion published using combinations of searches. The proportions of published works were identified in the following search engines: Scopus (90.8%), IPA (75%), MEDLINE (PubMed; 80.3%), EMBASE (88.2%), and Google Scholar (90.8%; Table 1). Additionally, a higher proportion of nonpharmacy journals were identified using the search strategy compared to pharmacy journals (Table 1) in each individual search engine. After assessing various combinations, only Scopus plus IPA or Google Scholar plus IPA identified all 76 publications (Table 2). Based on our results, no single search engine was able to identify all publications. All 9 potential pairs were assessed. Only the combination use of either Scopus and IPA or Google Scholar and IPA resulted in identifying all 76 publications. The use of MEDLINE (PubMed) and EMBASE resulted in the poorest combination of search engines, as it only identified 88.2% (n 1⁄4 67) of publications. Kwak et al used the combination of MEDLINE (PubMed) and EMBASE with great success; however, the utility of this combination may differ in a general population of residency programs. Residents of certain geographic areas may submit to journals indexed to different search engines which may have contributed to the differences between our study and Kwak et al, as we assessed abstracts from the Great Lakes region. Additionally, confirmed publications in our previous search strategy used Scopus, IPA, and MEDLINE (PubMed) and not a validated measure which may have influenced our results. In conclusion, we affirm that a validated measure approach, as used by Kwak et al, is the gold standard for confirming the utility of various search engines; however, it may be difficult to replicate in a larger, generalized population of residents. The use of a single search engine is likely not adequate to identify all published residents’ studies. Regional differences may exist in the use of search engines, as the combination of MEDLINE (PubMed) and EMBASE resulted in a high sensitivity in Kwak