Methods report on the development of the European S3 guidelines for the treatment of acne

Abstract

This methods report provides a comprehensive description of the development process behind the European S3 Guidelines for the Treatment of Acne. The Guidelines itself was developed according to the standard operating procedures (see Appendix A) of the European Dermatology Forum (EDF), and the underlying methodology incorporated the quality criteria contained within the Appraisal of Guidelines Research & Evaluation (AGREE) Instrument, as well as the recommendations of the Cochrane Collaboration, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group, and the German Association of Scientific Medical Societies (AWMF). The European S3 Guidelines for the Treatment of Acne was created using a structured development process comprising a systematic search of the literature within the relevant databases, a systematic evaluation of the search results, and a consensus conference based on formal consensus methodology (nominal group technique). Individuals were nominated to the expert panel – referred to henceforth as the EU Guidelines Group – by the guidelines group of the EDF or by the European Academy of Dermatology and Venereology (EADV). To be nominated as an expert, an individual had to satisfy at least one of the following criteria: Extensive clinical experience in the treatment of acne. Relevant publications in the field of acne. Relevant experience in evidence-based medicine. Emphasis was placed on selecting a representative panel of experts from throughout Europe. Representatives from various interest groups were involved in an external review of the Guidelines. Although extensive efforts were made to find a patient representative, these were ultimately unsuccessful due to the current lack of patient organizations in this field. Patients were, however, invited to participate in the external review. For a detailed overview of participating experts, see Appendix C. As evidence-based guidelines are strongly limited by issues of feasibility, the key questions to be addressed must be chosen carefully. In its initiation meeting, the EU Guidelines Group discussed which interventions and questions should be considered and subsequently reached a consensus regarding the main focus of the Guidelines. The EU Guidelines Group decided that suitable treatment options should be presented in a clinical treatment algorithm, taking into account the type of acne and the severity of disease. Interventions were selected according to the following criteria: Clinical relevance. Intervention had to be available/licensed in Europe either as a monotherapy or a fixed dose combination. Used for the treatment of acute acne. Treatment options consisting of more than two topical components were not included because of the likelihood of reduced patient adherence and/or because of a limitation in the feasibility of discussing all possible combinations and sequences. Fixed dose combinations were considered as long as they were licensed in a European country. In future versions of the Guidelines, aspects such as maintenance treatment (step 2) and the treatment of acne scarring, as well as skin care (step 3) should be considered. Using existing systematic reviews or guidelines as a basis can greatly facilitate the work of a guidelines group. We thus performed a search for these sources in MEDLINE, EMBASE, the Cochrane Database of Systematic Reviews, and the Guidelines International Network (G-I-N) Database. Standard Google searches were also performed. We refrained from conducting a systematic search at this point and stopped searching for certain interventions as soon as we had found a high quality source of information. Existing systematic reviews were assessed using the Methodology Checklist 1: Systematic Reviews and Meta-analyses published by the Scottish Intercollegiate Guidelines Network (SIGN) (see Appendix E). Existing guidelines were checked to determine whether they addressed the relevant questions agreed upon during the initiation meeting. All information retrieved from these reviews and guidelines was verified against the original publications and a grade of evidence was assigned to each trial. To find relevant trials, we performed systematic searches of the databases MEDLINE and EMBASE (for search strategies see Appendix B). For topical and systemic treatments, our search covered the period from 1 January 1999 through 10 March 2010, whereas for laser and light treatments it covered the period from 1 January 2007 through 13 April 2010. The beginning dates of our searches were based on the search periods used by the various systematic reviews considered in our Guidelines, whereas the end dates are the days upon which our own searches were conducted. All search results were verified by two independent assessors (AS and SR). In case of disagreement, a third assessor (AN) was involved and the conflict resolved through discussion. Reasons for excluding a study based on its abstract were as follows: No original data. No human data. No clinical data. Not dealing with the management of acne. Not in English (exception: selected German-language publications identified by hand search). After the search results were screened and agreed upon by the two (or, in some cases, three) assessors, the bibliographical information was transferred to an EndNote database and the full texts were obtained if available. Identified literature was evaluated by the two assessors (AS, SR) using a standardized Literature Evaluation Form (see Appendix D) comprising three parts (A, B, C). Both evaluators had been trained in using the Literature Evaluation Form for critical literature evaluation. The data generated by the two assessors were compared with each other, and any discrepancies were reviewed by a third assessor (AN) and resolved through discussion. Part A of the Literature Evaluation Form lists the exclusion criteria for clinical trials of interventions that the EU Guidelines Group had chosen for consideration in the Guidelines (see Table 1: Interventions included in the guidelines). The exclusion criteria were based on those used by Lehmann et al. (2001):1 Does not address management of active acne. More than 20% of the patients have chloracne, acne venenata, acne fulminans, acne necroticans, acne agminata, or rosacea. Patients differed at baseline (for example, occupational acne). Surrogate outcome measures only (for example, sebum production, propionibacterium acnes colony counts). No original data. Article not in English. Fewer than 10 patients per study arm. Not a controlled, prospective trial. No objective efficacy data (nota bene: added by the EU Guidelines Group). Part B of the Literature Evaluation Form was designed to assist the assessors in evaluating the methodological quality of each trial and to assign each trial a grade of evidence. If methodological weaknesses, such as the use of co-medication or the presence of baseline differences, were identified during part B, it was still possible to exclude the trial in question. Part C of the Literature Evaluation Form was designed to allow the assessors to extract outcome data on efficacy and safety from the included trials. The extracted data were summarized in evidence tables using MS Excel spreadsheets according to the methodology described in Part C. To address the demands of clinical practice, we categorized the included studies according to acne type. Because there is no widely accepted method for classifying acne severity or type, we chose to differentiate between comedonal acne; mild to moderate papulopustular acne; severe papulopustular acne/moderate nodular acne; and severe nodular acne/conglobate acne. Whenever possible, the different severity scales and definitions of acne types used in the included studies were harmonized to make the populations more comparable and provide valid evidence with a narrower focus on distinct clinical questions (for example, ‘What is the best drug for comedonal acne?’). If disease severity or acne type was not reported in a given study, we nevertheless classified and included it for later data extraction as long as lesion types and lesion counts at baseline were provided. In such instances, we classified >15 inflammatory lesions (IL) on the face in papulopustular acne and >6 nodules on the face in conglobate acne as severe disease. To provide a good overview of the available evidence on different types of acne, we generated separate evidence tables for comedonal acne, papulopustular acne and conglobate acne. Because very few trials to date have focused on comedonal acne or on conglobate acne alone, most of the trials that examined these (alongside other) acne types can also be found in the table for papulopustular acne. In short, trials could be categorized as having examined more than one acne type (and could thus be included in more than one table) if they included patients with different types of acne. To provide valid evidence for the different acne types, we generally favoured using a percentage reduction in the respective lesion count (see below) as an outcome measure. Only if such information was unavailable did we take other outcome measures into account, such as the Leeds Score/Burke-Cunliffe score, or mean acne severity (MAS) score. Comedonal acne: A trial on comedonal acne was categorized as such by our group if: (a) it had been designated in this manner by its authors and/or (b) this designation could be confirmed by patient baseline data provided in the study (comedone count, few or no inflammatory lesions). Because only very few studies have focused solely on comedonal acne to date, indirect evidence was generated by means of looking at the percentage reduction in non-inflammatory lesions (NIL). If a given study was categorized by methodologists as a trial on comedonal acne and no reduction in NIL was reported, other outcome measures, such as the MAS score, could be taken into account. Papulopustular acne: A trial on papulopustular acne was categorized as such by our group if: (a) it had been designated in this manner by its authors and/or (b) this designation could be confirmed by patient baseline data provided in the study. For papulopustular acne, the EU Guidelines Group agreed that the percentage reduction in IL was the outcome measure that would provide the best evidence. If this information was unavailable, the second and third choice outcome measures were the percentage reduction in papules (PA) and the percentage reduction in pustules (PU), respectively. If none of this information was available, the assessors relied on the total lesion count (i.e. the NIL count plus the IL count) or, similar to the approach taken with comedonal acne, an alternative outcome measure, such as the Leeds score. Nodular/conglobate acne: A trial on nodular/conglobate acne was categorized as such by methodologists if: (a) it had been designated in this manner by its authors and/or (b) it reported the nodule or cyst count over time, as well as relevant numbers of these lesions at the beginning of the trial. As was the case with comedonal acne, the assessors found comparatively few studies that focused solely on conglobate acne. The percentage reduction in nodules (NO) or cysts (CY) served as an outcome measure. Alternative outcome measures were the same as those described above for comedonal acne and papulopustular acne. Generally speaking, pooling information by means of evidence tables jeopardizes the integrity of results unless some elementary rules are applied to the columns of the tables (selection commented on in the following): Author: gives the name of the first author and year of publication. In cases where data obtained from the same patient sample were analysed and the results presented in two publications, the first authors of each publication are given. The number in parentheses refers to the bibliography in the end of the full guidelines text’. N = number: Gives the number of patients after randomization. S = severity: Gives a grade of severity (1, 2 or 3) according to the description of disease severity in the study or, if no such description was available, according to our own calculation using the lesion count at baseline or the scale/score given in the study for the baseline population (for example, Burke-Cunliffe/Leeds score). D = duration: Gives the duration of the study in weeks. In cases where a study lasted substantially longer than 12 weeks, the assessors generally attempted to extract data on outcomes at 12 weeks (or as close to this point as possible). In such cases, two numbers are shown in this column: the first number gives the overall length of the study in weeks, whereas the second number gives the point in time at which outcome data were extracted. Summary of efficacy: NIL, IL, NO, CY or their components (papules, pustules, open or closed comedones) were taken into account. We defined a difference as a ≥10% reduction in lesion count compared with baseline. Summary of safety: To provide meaningful data on safety, we followed a pragmatic approach by looking, first, at the three most common related adverse events/local side effects; second, at differences in drop-out rates due to adverse events; and, third, at the conclusions about safety drawn by the authors of each publication [conclusion of author (coa)]. If no such data were available, it was impossible to give a comparative statement on safety. It was also difficult to provide such statements in cases where the number of patients with adverse advents was low. A written summary of efficacy and safety was provided for sequential treatments or combinations with more than two drugs, for comparisons of verum vs. vehicle, or for comparisons of different concentrations of the same drug. Drop-outs: Gives the number of drop-outs due to adverse events if available. Grade of evidence: Each trial included in the Guidelines was evaluated with regard to its methodological quality and assigned a grade of evidence according to the grading system used in previous guidelines.2, 3 Randomized, double-blind clinical trial of high quality (for example, sample-size calculation, flow chart of patient inclusion, intention-to-treat (ITT) analysis, sufficient sample size) Randomized clinical trial of lesser quality (for example, only single-blind, limited sample size: at least 15 patients per study arm) Comparative trial with severe methodological limitations (for example, not blinded, very small sample size, no randomization) In addition to assigning a grade of evidence to individual trials, the assessors assigned levels of evidence to the various treatment options. The levels of evidence, which can be regarded as an overall rating of the available efficacy data for each treatment option, were defined as follows: Further research is very unlikely to change our confidence in the estimate of effect. At least two trials are available that were assigned a grade of evidence A and the results are predominantly consistent with the results of additional grade B or C studies. Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. At least three trials are available that were assigned a grade of evidence B and the results are predominantly consistent with respect to additional grade C trials. Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Conflicting evidence or limited amount of trials, mostly with a grade of evidence of B or C. Any estimate of effect is very uncertain. Little or no systematic empirical evidence; included trials are extremely limited in number and/or quality. One trial with a grade of evidence of A equals two trials with a grade of evidence of B. In all cases, it was possible for the EU Guidelines Group to modify the level of evidence assigned to a particular treatment option based on individual clinical assessments of a trial’s findings or in the event of conflicting findings among several studies (see, for example, comedonal acne: 3.3.1, 7.2.5, 5.6.1, 10.1; papulopustular acne: 1.4, 3.3.1, 4.2.3). The consensus conference took place in Dubrovnik, Croatia, on 11–12 May 2010. The conference was chaired by Prof. Berthold Rzany, who is an AWMF-certified moderator of consensus conferences. Prior to each consensus vote on a recommendation, the existing evidence was presented to the group and discussed with regard to efficacy, safety, patient preference, practicability and additional factors, such as antibiotic resistance and the results of pathophysiological reasoning. To avoid any potential confusion, standardized phrases were used to express the strength of a recommendation throughout the Guidelines. These were as follows: is strongly recommendedGood efficacy data, reasonable safety profile (especially regarding the benefit/harm ratio), good patient acceptance and a high level of evidence with direct evidence. can be recommendedGood efficacy data, reasonable safety profile (especially regarding the benefit/harm ratio), good patient acceptance, limitations pertaining to the level and directness of evidence. can be consideredEfficacy is lower than that of interventions that received a greater strength of recommendation; or: directness of evidence is insufficient or lacking; or: there were specific pathophysiological factors that led to a downgrade. is not recommendedInsufficient efficacy or less favourable benefit/harm ratio. may not be used under any circumstancesHarmful intervention with very unfavourable benefit/harm ratio. a recommendation for or against treatment X cannot be made at the present timeDue to a lack of evidence, it is impossible to make a recommendation for or against treatment X at the present time. Insufficient data from clinical trials; promising case reports or expert opinions may exist. The European S3 Guidelines for the Treatment of Acne underwent an extensive external review. From 13 May through 30 June 2011 the guidelines was available online for comments and amendments. This period of online availability was announced using the following mailing lists: EDF Board, EDF Guidelines Committee, EDF Members and the UMES Board. To reach more patients, the external review was also announced on two forums (http://www.acne.org, http://www.acneassasin.com). In addition, every participant was encouraged to invite all potentially interested parties to review and comment on the guidelines by participating in the external review process. The experts were nominated by the EDF and EADV. For a detailed overview of the participating experts, see Appendix C. The Division of Evidence Based Medicine at Charité– Universitätsmedizin Berlin was chosen as a methodological centre because of its experience in guidelines development in dermatology, such as the European S3 Guidelines for the Treatment of Psoriasis. In an initiation meeting in Paris on 19 March 2009, the EU Guidelines Group decided which interventions would be considered in the European S3 Guidelines for the Treatment of Acne (Table 1: Interventions included in the guidelines). Later in the course of the project, after the final makeup of the Group had been established, all members confirmed the choice of interventions. The Guidelines is based on a clinical algorithm that focuses in particular on the various treatment options for different types of acne. Efficacy, safety and patient preference were taken into account for each of the included treatment options, and these aspects were assessed based on suitable publications. In addition, aspects such as pathophysiology and antibiotic resistance were taken into account. It would have gone beyond the scope of this Guidelines to consider the pricing and reimbursement regimes in every European country. The difference in these are too large, as are those in patients’ willingness and ability to pay for medication, and in the availability of generics. This and other European guidelines are therefore always meant to be treated as a source for national and local adaption, and pharmacoeconomic considerations should be taken into account at these levels. As described in detail in section 2.3, a non-systematic search was performed to identify existing guidelines, consensus papers and systematic reviews. The following consensus paper and guidelines were identified and provided to the EU Guidelines Group as a basis for discussion: Consensus paper Gollnick et al. Global acne alliance.4 German acne guidelines.5 French acne guidelines.6 US acne guidelines.7 None of these guidelines assesses the available evidence with respect to the different acne types (S3). Because of this, the EU Guidelines Group used them for orientation purposes only and did not adapt them for use in the European S3 Guidelines on the Treatment of Acne. However, the clinical algorithm from the consensus paper from 2003 from the Global Acne Alliance was adapted as the basis for the algorithm used in the Guidelines. The following systematic reviews were identified: Arowojolu AO, Gallo MF, Lopez LM, Grimes DA, Garner SE. Combined oral contraceptive pills for treatment of acne. Cochrane Database Syst Rev 2009, Issue 3. Art. no: CD004425. DOI: 10.1002/14651858.CD004425.pub4. Garner SE, Eady A, Popescu CM, Newton J, Li Wan Po A. Minocycline for acne vulgaris: efficacy and safety. Cochrane Database Syst Rev 2003; (1): CD002086. Haedersdal M, Togsverd-Bo K, Wulf HC. Evidence-based review of lasers, light sources and photodynamic therapy in the treatment of acne vulgaris. J Eur Acad Dermatol Venereol 2008 Mar; 22: 267–78. Hamilton FL, Car J, Lyons C, Car M, Layton A, Majeed A. Laser and light therapies for the treatment of acne vulgaris: systematic review. Br J Dermatol 2009; 160: 1273–85. Lehmann HP, Andrews JS, Robinson KA, Holloway VL, Goodman SN. Management of Acne. Rockville, MD: Agency for Healthcare Research and Quality 2001. (Evidence Report/Technology Assessment, 17; AHRQ publication no 01-E019). Simonart T, Dramaix M. Treatment of acne with topical antibiotics: lessons from clinical studies. Br J Dermatol 2005; 153: 395–403. Simonart T, Dramaix M, De Maertelaer V. Efficacy of tetracyclines in the treatment of acne vulgaris: a review. Br J Dermatol 2008; 158: 208–16. Thielitz A, Abdel-Naser MB, Fluhr JW, Zouboulis CC, Gollnick H. Topical retinoids in acne – an evidence-based overview. J Dtsch Dermatol Ges 2008 Dec; 6: 1023–31. Williams H, Bigby M, Diepan T, Herxheimer A, Naldi L, Rzany B. Evidence-based dermatology, 2nd edn, Apr 2008; Web tables. URL: http://www.blackwellpublishing.com/medicine/bmj/dermatology/pdfs/web_tables_15.doc. (last accessed: 17 May 2010). Worret WI, Fluhr JW. Acne therapy with topical benzoyl peroxide, antibiotics and azelaic acid. J Dtsch Dermatol Ges 2006 April; 4: 293–300. After the quality of these systematic reviews had been evaluated (see SIGN checklists in Appendix E), the reviews listed below were chosen to be used as a basis for the present Guidelines (Table 2: Overview of available sources of evidence). Using the Literature Evaluation Form, the methodologists evaluated a total of 256 studies (without duplicates) that had been included in the systemic reviews. Of these studies, 184 trials were ultimately included and 72 were excluded (among which were four articles that could not be obtained). For more details, see Fig. 1 Overview of included and excluded articles and the systematic reviews in which they were cited. Overview of included and excluded articles and the systematic reviews in which they were cited. We performed a systematic literature search for each included intervention (see Appendix B for search strategies). As mentioned in section 2.4 above, the beginning dates of our searches were based on the search periods used by the various systematic reviews considered in our Guidelines. For topical and systemic treatments, our search covered the period from 1 January 1999 through 10 March 2010, whereas for laser and light therapies it covered the period from 1 January 2007 through 13 April 2010. Our search generated 1576 hits, including 161 on laser and light treatments. After screening all abstracts, the two independent assessors determined that 259 publications were eligible for a more detailed evaluation. Some of these 259 publications were included by hand search. The publications retrieved in the literature search were evaluated systematically using parts A–C of the Literature Evaluation Form. A total of 515 articles were evaluated using part A of the Literature Evaluation Form (five articles could not be obtained). Of these articles, 283 clinical trials fulfilled our inclusion criteria, whereas 232 did not and were excluded. The remaining 283 studies were further evaluated using parts B and C of the Literature Evaluation Form and the relevant information was transferred to MS Excel tables, which are available online (http://www.acne-guidelines.com). Data from the systematic reviews were transferred directly into the tables and later checked for errors. A level of evidence was assigned to each clinical question based on the available trials. The following levels of evidence for relevant key questions were applied; low levels of evidence indicate a need for more high-quality research in these areas. The distribution of studies for each grade of evidence is displayed in Table 3: Distribution of grades of evidence. Defining the clinical superiority of one treatment option over another was particularly challenging. Using indirect evidence from quality-of-life data, we chose to define superiority as a ≥10% reduction in the number of lesions.17 After a detailed presentation of the available data on efficacy, safety and patient preference, the recommendations were discussed during the consensus conference. The detailed voting results are available at the dEBM. Of the 41 recommendations put up for a vote, 59% passed with absolute consensus (i.e. 100% agreement), 39% passed with a strong consensus (>75%) and only 2% passed with a majority between 50% and 75% partial vote. The European S3 Guidelines for the Treatment of Acne was made available online for commenting for 7 weeks. All national societies were invited to comment and to pass on the invitation to their members. In addition, the mailing list of the EDF was used. The EU Guidelines Group received and evaluated 65 comments (see http://www.acne-guidelines.com), of which 37% led to changes in the text. Individuals or organizations that posted comments that were ultimately rejected were provided with an explanation of why their comments had not been considered further. Review by the UEMS (Union Européenne des Médicins Spécialistes) According to the standard operating procedures of the EDF (see Appendix A Standard operating procedures), the European S3 Guidelines for the Treatment of Acne was also sent to the UEMS for approval. Participation of patient representatives Although extensive efforts were made to find a patient representative, these were ultimately unsuccessful due to the current lack of patient organizations in this area. Patients were, however, invited to join the external review. Participation of further relevant disciplines A psychologist participated in the development process. She took part in the consensus conference, the internal review, and also in writing the texts. Depending on further funding, an implementation project may be conducted to increase awareness and acceptance of the European S3 Guidelines for the Treatment of Acne. Because many countries are involved, each with its own health system, traditions and languages, a project like this would face a range of challenges. Because no further funding for this Guidelines is available, no formal evaluation programme has been planed so far. The European S3 Guidelines for the Treatment of Acne was funded by the EDF. In accordance with the standard operating procedures of the EDF, the European S3 Guidelines for the Treatment of Acne will need to be updated in 3 years. Because new interventions may be licensed or relevant changes in information (for example, on adverse events) may become available before this point, the subcommittee on acne of the EDF will evaluate the need for an earlier update of particular (or all) interventions at regular intervals. Future updates to the Guidelines may also include maintenance therapy. All authors completed the ‘Form for Disclosure of Potential Conflicts of Interest’ of the International Committee of Medical Journal Editors (ICMJE), which is available at the dEBM and online (http://www.acne-guidelines.com). The European S3 Guidelines for the Treatment of Acne is the first in the European Union to focus on this disease. Developing evidence-based guidelines is demanding and time-consuming for all participants, especially when the process takes place at the European level. Coordinated structures and processes are essential in this context. In the present method report, we describe in detail this structured process, which was coordinated by the Division of Evidence Based Medicine at Charité– Universitätsmedizin Berlin, Germany. Two possibilities for updating the Guidelines are imaginable: (i) after a certain period (for example, 3 years); or (ii) at shorter intervals and based on continuous surveillance of the literature, which might involve updating only certain parts of the Guidelines. This latter option, which could be described as creating ‘living guidelines’, would ensure a high degree of timeliness and relevance in addition to high quality findings, but would clearly require sufficient and continuous funding. Search strategy for topical and systemic treatments: Databases: Ovid MEDLINE®, Ovid MEDLINE® In-Process, EMBASE Search strategies for laser and light treatments: Database: Ovid MEDLINE® Database: Ovid MEDLINE® In-Process Database: EMBASE Each member of the EU Guidelines Group has specific responsibilities. At all stages of the guidelines process, these responsibilities need to be defined. I. ACNE ABSTRACT REVIEW FORM (INCLUSION/EXCLUSION?) *to be specified in Part III Comments on reasons for exclusion or inclusion if these could not be explained above: ……………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………… II. ARTICLE REVIEW FORM (QUALITY OF ARTICLE) III. EFFICACY Comments on quality of study/additional information you think a reader of the Guideline would like to know about this study: ……………………………………………………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………