Methodological Search Filters Research Articles

Search strategies (filters) to identify systematic reviews in MEDLINE and Embase.

Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed. To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase. We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science). We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both. Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist. We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation. Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.

Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE

BackgroundSystematic reviews (SRs) are valuable resources as they address specific clinical questions by summarizing all existing relevant studies. However, finding all information to include in systematic reviews can be challenging. Methodological search filters have been developed to find articles related to specific clinical questions. To our knowledge, no filter exists for finding studies on the role of prognostic factor (PF). We aimed to develop and evaluate a search filter to identify PF studies in Ovid MEDLINE that has maximum sensitivity.MethodsWe followed current recommendations for the development of a search filter by first identifying a reference set of PF studies included in relevant systematic reviews on the topic, and by selecting search terms using a word frequency analysis complemented with an expert panel discussion. We evaluated filter performance using the relative recall methodology.ResultsWe constructed a reference set of 73 studies included in six systematic reviews from a larger sample. After completing a word frequency analysis using the reference set studies, we compiled a list of 80 of the frequent methodological terms. This list of terms was evaluated by the Delphi panel for inclusion in the filter, resulting in a final set of 8 appropriate terms. The consecutive connection of these terms with the Boolean operator OR produced the filter. We then evaluated the filter using the relative recall method against the reference set, comparing the references included in the SRs with our new search using the filter. The overall sensitivity of the filter was calculated to be 95%, while the overall specificity was 41%. The precision of the filter varied considerably, ranging from 0.36 to 17%. The NNR (number needed to read) value varied largely from 6 to 278. The time saved by using the filter ranged from 13–70%.ConclusionsWe developed a search filter for OVID-Medline with acceptable performance that could be used in systematic reviews of PF studies. Using this filter could save as much as 40% of the title and abstract screening task. The specificity of the filter could be improved by defining additional terms to be included, although it is important to evaluate any modification to guarantee the filter is still highly sensitive.

Clinical Instruments for Measuring Unsupported Sitting Balance in Subjects with Spinal Cord Injury: A Systematic Review.

Background: Evidence of intervention effectiveness depends on, among other things, the use of a common set of valid and reliable measures that are responsive to change and reflect clinically important outcomes. Objective: To identify clinical assessment instruments with properties for measuring unsupported sitting balance in subjects with spinal cord injury (SCI). Methods: Bibliographic databases (PubMed, Science Direct, CINAHL, and Central) were searched for articles with the key words "spinal cord injury," "unsupported sitting," and "outcome assessment" in combination with a specific methodological search filter for each database. Studies describing the application of any assessment instrument for measuring unsupported sitting balance in subjects with SCI, which had the evaluation of any measurement property, were included in the review. Publication details, measure's name, setting, summary statistics, measurement properties (reliability, validity, responsiveness), and statistical significance (p values) were extracted. Results: Eight hundred forty publications were identified; 8 articles were included in the systematic review. Twelve instruments were identified and analyzed, showing limited and incomplete measurement properties. Among them, 10 addressed activity, 1 addressed structures/body functions, and 1 addressed both activity and structures/body functions domains of the International Classification of Functioning, Disability and Health (ICF). Conclusion: Based mainly on the measurement properties and the development of the instruments analyzed in this review, the Sitting Balance Measure, the Trunk Control Test, and the Set of Assessment Tools for Measuring Unsupported Sitting seem to be the most appropriate and recommended measures to assess unsupported sitting in subjects with SCI.

Measurement properties of comorbidity indices in maternal health research: a systematic review

BackgroundMaternal critical illness occurs in 1.2 to 4.7 of every 1000 live births in the United States and approximately 1 in 100 women who become critically ill will die. Patient characteristics and comorbid conditions are commonly summarized as an index or score for the purpose of predicting the likelihood of dying; however, most such indices have arisen from non-pregnant patient populations. We sought to systematically review comorbidity indices used in health administrative datasets of pregnant women, in order to critically appraise their measurement properties and recommend optimal tools for clinicians and maternal health researchers.MethodsWe conducted a systematic search of MEDLINE and EMBASE to identify studies published from 1946 and 1947, respectively, to May 2017 that describe predictive validity of comorbidity indices using health administrative datasets in the field of maternal health research. We applied a methodological PubMed search filter to identify all studies of measurement properties for each index.ResultsOur initial search retrieved 8944 citations. The full text of 61 articles were identified and assessed for final eligibility. Finally, two eligible articles, describing three comorbidity indices appropriate for health administrative data remained: The Maternal comorbidity index, the Charlson comorbidity index and the Elixhauser Comorbidity Index. These studies of identified indices had a low risk of bias. The lack of an established consensus-building methodology in generating each index resulted in marginal sensibility for all indices. Only the Maternal Comorbidity Index was derived and validated specifically from a cohort of pregnant and postpartum women, using an administrative dataset, and had an associated c-statistic of 0.675 (95% Confidence Interval 0.647–0.666) in predicting mortality.ConclusionsOnly the Maternal Comorbidity Index directly evaluated measurement properties relevant to pregnant women in health administrative datasets; however, it has only modest predictive ability for mortality among development and validation studies. Further research to investigate the feasibility of applying this index in clinical research, and its reliability across a variety of health administrative datasets would be incrementally helpful. Evolution of this and other tools for risk prediction and risk adjustment in pregnant and post-partum patients is an important area for ongoing study.

Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study.

Effective study identification is essential for conducting health research, developing clinical guidance and health policy and supporting health-care decision-making. Methodological search filters (combinations of search terms to capture a specific study design) can assist in searching to achieve this. This project investigated the methods used to assess the performance of methodological search filters, the information that searchers require when choosing search filters and how that information could be better provided. Five literature reviews were undertaken in 2010/11: search filter development and testing; comparison of search filters; decision-making in choosing search filters; diagnostic test accuracy (DTA) study methods; and decision-making in choosing diagnostic tests. We conducted interviews and a questionnaire with experienced searchers to learn what information assists in the choice of search filters and how filters are used. These investigations informed the development of various approaches to gathering and reporting search filter performance data. We acknowledge that there has been a regrettable delay between carrying out the project, including the searches, and the publication of this report, because of serious illness of the principal investigator. The development of filters most frequently involved using a reference standard derived from hand-searching journals. Most filters were validated internally only. Reporting of methods was generally poor. Sensitivity, precision and specificity were the most commonly reported performance measures and were presented in tables. Aspects of DTA study methods are applicable to search filters, particularly in the development of the reference standard. There is limited evidence on how clinicians choose between diagnostic tests. No published literature was found on how searchers select filters. Interviewing and questioning searchers via a questionnaire found that filters were not appropriate for all tasks but were predominantly used to reduce large numbers of retrieved records and to introduce focus. The Inter Technology Appraisal Support Collaboration (InterTASC) Information Specialists' Sub-Group (ISSG) Search Filters Resource was most frequently mentioned by both groups as the resource consulted to select a filter. Randomised controlled trial (RCT) and systematic review filters, in particular the Cochrane RCT and the McMaster Hedges filters, were most frequently mentioned. The majority indicated that they used different filters depending on the requirement for sensitivity or precision. Over half of the respondents used the filters available in databases. Interviewees used various approaches when using and adapting search filters. Respondents suggested that the main factors that would make choosing a filter easier were the availability of critical appraisals and more detailed performance information. Provenance and having the filter available in a central storage location were also important. The questionnaire could have been shorter and could have included more multiple choice questions, and the reviews of filter performance focused on only four study designs. Search filter studies should use a representative reference standard and explicitly report methods and results. Performance measures should be presented systematically and clearly. Searchers find filters useful in certain circumstances but expressed a need for more user-friendly performance information to aid filter choice. We suggest approaches to use, adapt and report search filter performance. Future work could include research around search filters and performance measures for study designs not addressed here, exploration of alternative methods of displaying performance results and numerical synthesis of performance comparison results. The National Institute for Health Research (NIHR) Health Technology Assessment programme and Medical Research Council-NIHR Methodology Research Programme (grant number G0901496).

Protocol: a systematic review of studies developing and/or evaluating search strategies to identify prognosis studies

BackgroundPrognosis research is on the rise, its importance recognised because chronic health conditions and diseases are increasingly common and costly. Prognosis systematic reviews are needed to collate and synthesise these research findings, especially to help inform effective clinical decision-making and healthcare policy.A detailed, comprehensive search strategy is central to any systematic review. However, within prognosis research, this is challenging due to poor reporting and inconsistent use of available indexing terms in electronic databases. Whilst many published search filters exist for finding clinical trials, this is not the case for prognosis studies.This systematic review aims to identify and compare existing methodological filters developed and evaluated to identify prognosis studies of any of the three main types: overall prognosis, prognostic factors, and prognostic [risk prediction] models.MethodsPrimary studies reporting the development and/or evaluation of methodological search filters to retrieve any type of prognosis study will be included in this systematic review. Multiple electronic bibliographic databases will be searched, grey literature will be sought from relevant organisations and websites, experts will be contacted, and citation tracking of key papers and reference list checking of all included papers will be undertaken. Titles will be screened by one person, and abstracts and full articles will be reviewed for inclusion independently by two reviewers. Data extraction and quality assessment will also be undertaken independently by two reviewers with disagreements resolved by discussion or by a third reviewer if necessary.Filters’ characteristics and performance metrics reported in the included studies will be extracted and tabulated. To enable comparisons, filters will be grouped according to database, platform, type of prognosis study, and type of filter for which it was intended.DiscussionThis systematic review will identify all existing validated prognosis search filters and synthesise evidence about their applicability and performance. These findings will identify if current filters provide a proficient means of searching electronic bibliographic databases or if further prognosis filters are needed and can feasibly be developed for systematic searches of prognosis studies.

Measurement Properties of the Quebec Back Pain Disability Scale in Patients With Nonspecific Low Back Pain: Systematic Review.

The Quebec Back Pain Disability Scale (QBPDS) has been translated into different languages, and several studies on its measurement properties have been done. The purpose of this review was to critically appraise and compare the measurement properties, when possible, of all language versions of the QBPDS by systematically reviewing the methodological quality and results of the available studies. Bibliographic databases (PubMed, Embase, CINAHL, and PsycINFO) were searched for articles with the key words "Quebec," "back," "pain," and "disability" in combination with a methodological search filter for finding studies on measurement properties concerning the development or evaluation of the measurement properties of the QBPDS in patients with nonspecific low back pain. Assessment of the methodological quality was carried out by the reviewers using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist for both the original language version of the QBPDS in English and French and all translated versions. The results of the measurement properties were rated based on criteria proposed by Terwee et al. The search strategy resulted in identification of 1,436 publications, and 27 articles were included in the systematic review. There was limited-to-moderate evidence of good reliability, validity, and responsiveness of the QBPDS for the different language versions, but for no language version was evidence available for all measurement properties. For research and clinical practice, caution is advised when using the QBPDS to measure disability in patients with nonspecific low back pain. Strong evidence is lacking on all measurement properties for each language version of the QBPDS.

Reporting methodological search filter performance comparisons: a literature review.

Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons. Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations. Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter. Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

Systematic review to identify and appraise outcome measures used to evaluate childhood obesity treatment interventions (CoOR): evidence of purpose, application, validity, reliability and sensitivity.

Lack of uniformity in outcome measures used in evaluations of childhood obesity treatment interventions can impede the ability to assess effectiveness and limits comparisons across trials. To identify and appraise outcome measures to produce a framework of recommended measures for use in evaluations of childhood obesity treatment interventions. Eleven electronic databases were searched between August and December 2011, including MEDLINE; MEDLINE In-Process and Other Non-Indexed Citations; EMBASE; PsycINFO; Health Management Information Consortium (HMIC); Allied and Complementary Medicine Database (AMED); Global Health, Maternity and Infant Care (all Ovid); Cumulative Index to Nursing and Allied Health Literature (CINAHL) (EBSCOhost); Science Citation Index (SCI) [Web of Science (WoS)]; and The Cochrane Library (Wiley) - from the date of inception, with no language restrictions. This was supported by review of relevant grey literature and trial databases. Two searches were conducted to identify (1) outcome measures and corresponding citations used in published childhood obesity treatment evaluations and (2) manuscripts describing the development and/or evaluation of the outcome measures used in the childhood intervention obesity evaluations. Search 1 search strategy (review of trials) was modelled on elements of a review by Luttikhuis et al. (Oude Luttikhuis H, Baur L, Jansen H, Shrewsbury VA, O'Malley C, Stolk RP, et al. Interventions for treating obesity in children. Cochrane Database Syst Rev 2009;1:CD001872). Search 2 strategy (methodology papers) was built on Terwee et al.'s search filter (Terwee CB, Jansma EP, Riphagen II, de Vet HCW. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res 2009;18:1115-23). Eligible papers were appraised for quality initially by the internal project team. This was followed by an external appraisal by expert collaborators in order to agree which outcome measures should be recommended for the Childhood obesity Outcomes Review (CoOR) outcome measures framework. Three hundred and seventy-nine manuscripts describing 180 outcome measures met eligibility criteria. Appraisal of these resulted in the recommendation of 36 measures for the CoOR outcome measures framework. Recommended primary outcome measures were body mass index (BMI) and dual-energy X-ray absorptiometry (DXA). Experts did not advocate any self-reported measures where objective measurement was possible (e.g. physical activity). Physiological outcomes hold potential to be primary outcomes, as they are indicators of cardiovascular health, but without evidence of what constitutes a minimally importance difference they have remained as secondary outcomes (although the corresponding lack of evidence for BMI and DXA is acknowledged). No preference-based quality-of-life measures were identified that would enable economic evaluation via calculation of quality-adjusted life-years. Few measures reported evaluating responsiveness. Proposed recommended measures are fit for use as outcome measures within studies that evaluate childhood obesity treatment evaluations specifically. These may or may not be suitable for other study designs, and some excluded measures may be more suitable in other study designs. The CoOR outcome measures framework provides clear guidance of recommended primary and secondary outcome measures. This will enhance comparability between treatment evaluations and ensure that appropriate measures are being used. Where possible, future work should focus on modification and evaluation of existing measures rather than development of tools de nova. In addition, it is recommended that a similar outcome measures framework is produced to support evaluation of adult obesity programmes. The National Institute for Health Research Health Technology Assessment programme.

Examination of the Clinical Queries and Systematic Review “hedges” in EMBASE and MEDLINE

Introduction – This investigation sought to determine whether the methodological search filters in place as Clinical Queries limits in OvidSP EMBASE and OvidSP MEDLINE had been modified from those written by Haynes et al. and whether the translations of these in PubMed and EBSCO MEDLINE were reliable. The translated National Library of Medicine (NLM) Systematic Reviews hedges in place in OvidSP MEDLINE and EBSCO MEDLINE were also examined. Methods – Search queries were run using the Clinical Queries and Systematic Reviews hedges incorporated into OvidSP EMBASE, OvidSP MEDLINE, PubMed, and EBSCO MEDLINE to determine the reliability of these limits in comparison with the published hedge search strings. Results – Five of the OvidSP EMBASE Clinical Queries hedges produced results that were different from the published search strings. Three of the EBSCO MEDLINE and five of the PubMed translated Clinical Queries hedges yielded markedly different results (>10% difference) than those obtained using the OvidSP MEDLINE hedge counterparts. The OvidSP MEDLINE Systematic Reviews subject subset hedge was found to have a major error, which has been corrected. Discussion – Translations of hedges to appropriate syntax for other database platforms may result in significantly different search results. The platform searched should ideally be the one for which the hedges were written and tested. Regardless, the hedges in place may not be the same as the published hedge search strings. Quality control testing is needed to ensure that the hedges in place as limits are the same as those that have been published.

Choosing and using methodological search filters: searchers' views.

Search filters or hedges are search strategies developed to assist information specialists and librarians to retrieve different types of evidence from bibliographic databases. The objectives of this project were to learn about searchers' filter use, how searchers choose search filters and what information they would like to receive to inform their choices. Interviews with information specialists working in, or for, the National Institute for Health and Care Excellence (NICE) were conducted. An online questionnaire survey was also conducted and advertised via a range of email lists. Sixteen interviews were undertaken and 90 completed questionnaires were received. The use of search filters tends to be linked to reducing a large amount of literature, introducing focus and assisting with searches that are based on a single study type. Respondents use numerous ways to identify search filters and can find choosing between different filters problematic because of knowledge gaps and lack of time. Search filters are used mainly for reducing large result sets (introducing focus) and assisting with searches focused on a single study type. Features that would help with choosing filters include making information about filters less technical, offering ratings and providing more detail about filter validation strategies and filter provenance.

An Example of Systematic Searching for Guidelines to Prevent Catheter-associated Urinary Tract Infections - Part I: Using the PubMed Database

Purpose: Effective literature searching is essential to support evidence-based nursing. The aim of this study was to present our recent systematic search experience to identify guidelines in PubMed for prevention of catheter-associated urinary tract infections. Methods: Five approaches to the literature search via PubMed were employed. The searches were restricted to items published from 1980 to 2010, for patients admitted to hospital, and in the English language. The search results were compared using the number of records and relevant items, and the sensitivity and precision of each search strategy. Results: The individual approaches retrieved 19-141 of records and 3-6 of relevant items. Sensitivity ranged from 37.5% to 75.0% with the highest values for simple searches and a search combining MeSH terms and free textwords with a methodological search filter. Precision varied from 4.3% to 21.7% and the highest precision was found for MeSH terms with limits feature. Conclusion: The simple search in PubMed is an appropriate way for nurses in a busy clinical practice to search the literature for evidence. However, several approaches using MeSH terms, free textwords, limits feature or methodological search filters are also required to have more efficient and better informed search results.

Retrieval of diagnostic and treatment studies for clinical use through PubMed and PubMed's Clinical Queries filters

Clinical Queries filters were developed to improve the retrieval of high-quality studies in searches on clinical matters. The study objective was to determine the yield of relevant citations and physician satisfaction while searching for diagnostic and treatment studies using the Clinical Queries page of PubMed compared with searching PubMed without these filters. Forty practicing physicians, presented with standardized treatment and diagnosis questions and one question of their choosing, entered search terms which were processed in a random, blinded fashion through PubMed alone and PubMed Clinical Queries. Participants rated search retrievals for applicability to the question at hand and satisfaction. For treatment, the primary outcome of retrieval of relevant articles was not significantly different between the groups, but a higher proportion of articles from the Clinical Queries searches met methodologic criteria (p=0.049), and more articles were published in core internal medicine journals (p=0.056). For diagnosis, the filtered results returned more relevant articles (p=0.031) and fewer irrelevant articles (overall retrieval less, p=0.023); participants needed to screen fewer articles before arriving at the first relevant citation (p<0.05). Relevance was also influenced by content terms used by participants in searching. Participants varied greatly in their search performance. Clinical Queries filtered searches returned more high-quality studies, though the retrieval of relevant articles was only statistically different between the groups for diagnosis questions. Retrieving clinically important research studies from Medline is a challenging task for physicians. Methodological search filters can improve search retrieval.

Translation of Hedges in Medical Databases to Other Platforms’ Syntax May Cause Significantly Different Search Results

Translation of Hedges in Medical Databases to Other Platforms’ Syntax May Cause Significantly Different Search Results

Using Methodological Search Filters to Facilitate Evidence-Based Social Work Practice

The process of Evidence-Based Practice (EBP) requires clinical social workers to conduct systematic searches of academic databases in order to ascertain current best evidence and integrate this with client preferences/values and clinical state/circumstances. Yet social workers are often pressed for time, and searches for evidence disregarded as too time-consuming to conduct. There is hope. Searches of the literature can be more easily and quickly facilitated through the use of methodological search filters. This study introduces a new methodological search filter created especially for social care and evaluates the extent to which this and four other filters accurately and efficiently identify known social care effectiveness studies in two major scholarly databases (Psycinfo and Medline). Sensitivity, specificity, and a new metric for establishing efficiency (the AVALANCHE INDEX) are reported.

Comparison of the efficacy of three PubMed search filters in finding randomized controlled trials to answer clinical questions

The aim of this study was to compare the performance of three search methods in the retrieval of relevant clinical trials from PubMed to answer specific clinical questions. Included studies of a sample of 100 Cochrane reviews which recorded in PubMed were considered as the reference standard. The search queries were formulated based on the systematic review titles. Precision, recall and number of retrieved records for limiting the results to clinical trial publication type, and using sensitive and specific clinical queries filters were compared. The number of keywords, presence of specific names of intervention and syndrome in the search keywords were used in a model to predict the recalls and precisions. The Clinical queries-sensitive search strategy retrieved the largest number of records (33) and had the highest recall (41.6%) and lowest precision (4.8%). The presence of specific intervention name was the only significant predictor of all recalls and precisions (P = 0.016). The recall and precision of combination of simple clinical search queries and methodological search filters to find clinical trials in various subjects were considerably low. The limit field strategy yielded in higher precision and fewer retrieved records and approximately similar recall, compared with the clinical queries-sensitive strategy. Presence of specific intervention name in the search keywords increased both recall and precision.

S40– How well do methodological search filters perform?

S40– How well do methodological search filters perform?

Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments

ObjectivesFor the measurement of patient-reported outcomes, such as (health-related) quality of life, often many measurement instruments exist that intend to measure the same construct. To facilitate instrument selection, our aim was to develop a highly sensitive search filter for finding studies on measurement properties of measurement instruments in PubMed and a more precise search filter that needs less abstracts to be screened, but at a higher risk of missing relevant studies.MethodsA random sample of 10,000 PubMed records (01-01-1990 to 31-12-2006) was used as a gold standard. Studies on measurement properties were identified using an exclusion filter and hand searching. Search terms were selected from the relevant records in the gold standard as well as from 100 systematic reviews of measurement properties and combined based on sensitivity and precision. The performance of the filters was tested in the gold standard as well as in two validation sets, by calculating sensitivity, precision, specificity, and number needed to read.ResultsWe identified 116 studies on measurement properties in the gold standard. The sensitive search filter was able to retrieve 113 of these 116 studies (sensitivity 97.4%, precision 4.4%). The precise search filter had a sensitivity of 93.1% and a precision of 9.4%. Both filters performed very well in the validation sets.ConclusionThe use of these search filters will contribute to evidence-based selection of measurement instruments in all medical fields.

Information Resources Group (IRG) Workshop: Pushing the Frontiers of HTA Information Management

Make the IRG pre-conference workshop part of your annual HTAi (Health Technology Assessment International) meeting plans! Join us in Barcelona on Sunday, June 17th for a full day of training and lively discussions. Our workshop includes a mixture of presentations and interactive sessions highlighting important and emerging developments in HTA information management.&#x0D; &#x0D; Julie Glanville, IS Manager and Associate Director at the Centre for Reviews and Dissemination in York, will outline recent research into search filters to identify diagnostic tests and assessments of which databases and other resources should be searched for diagnostic test studies. &#x0D; &#x0D; Greg Bak, an information specialist from the Canadian Agency for Drugs and Technologies in Health, will discuss a CADTH-adapted critical appraisal instrument that can be used to rank and assess the quality of methodological search filters. His presentation will provide participants with the skills needed to make informed decisions about when and which methodological search filters to use.&#x0D; &#x0D; Sari Ormstad, a research librarian at the Norwegian Knowledge Centre for the Health Services, Oslo, will present on a new database that offers local decision-makers and health personnel an overview of all current international assessments of new health technologies and summarises those that are most relevant for them. &#x0D; &#x0D; Miquel Mayer, Director of Web Mèdica Acreditada, an international accreditation program of medical websites, will discuss the quality and use of health-related websites, different strategies to select and improve their quality, and finally how metadata and semantic web technologies could help users find the best health information.&#x0D; &#x0D; Elaine Alligood, an information specialist from Veterans Affairs Technology Assessment Program, will update participants on current search engines and how they can be used in HTA searches to complement traditional database searching.&#x0D; &#x0D; Andrew Booth, Director of Information Resources at the School of Health &amp; Related Research (ScHARR), and Amanda Burls, Senior Clinical Lecturer and Director of West Midlands Health Technology Assessment Collaboration, will deliver a half-day bilingual (English and Spanish) critical appraisal workshop in which they provide a "tapas" of critical appraisal skills and techniques. The workshop will demonstrate that skills in critical appraisal can be modeled and facilitated by library, information and other health staff working locally. &#x0D; Full details regarding conference registration are available at: http://www.htai.org/barcelona-2007/.&#x0D; &#x0D; For more information, contact Becky Skidmore, IRG Chair: bskidmore@sogc.com or Malene Fabricius Jensen, IRG Chair-Elect: mfj@sst.dk.

An alternative to the hand searching gold standard: validating methodological search filters using relative recall

BackgroundSearch filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard" – a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching.MethodsWe identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews.ResultsThe systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall.ConclusionAn approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.