Peer Review Program Research Articles

A New Publishing Model forThe Canadian Journal of Infectious Diseases & Medical Microbiology

Twenty six years ago, we had a vision to publish Canadian medical research and open the window to what Canada contributed to medical research globally. We turned this dream into reality and succeeded beyond our expectations. We began by publishing The Canadian Journal of Cardiology and grew to publish 11 peer-reviewed medical journals, including The Canadian Journal of Infectious Diseases & Medical Microbiology, which has been published for over two decades now. The economic environment for publishing has changed significantly over the years and with declining advertising revenues, a general aversion of Canadian health care professionals to paying for subscriptions to Canadian publications and a dearth of support by federal funding agencies, new models must be sought. Many successful journals have moved toward an ‘open-access’ model with the authorship supporting the publication of their articles, which are then accessible for all to read, and to use (or reuse) to various extents. It is estimated that about 20% of the peer-reviewed literature is freely accessible at this time, but the demand is growing and the readership appreciates the increased reach, speed of access and availability of scholarly communications. At Pulsus, we are embracing this movement and The Canadian Journal of Infectious Diseases & Microbiology will be our first venture into open-access publishing. We know how important your scholarly work is and what it means for the Journal, and we acknowledge that the Journal needs our continued commitment. But we also need your support as we move toward an open-access format. The new open-access format will include a submission fee for the new online submission and peer-review program (launched September 20, 2010) and, as with all open-access formats, will also include a per published page fee for authors (submission fee will be removed from this fee for accepted papers). We share this watershed moment in our publishing history with you having confidence that we are doing the right thing, publishing your research for the benefit of Canadians and the global community. Pulsus will continue to work diligently on maintaining a robust and expedient publishing environment but we need your support….it is your Journal and we must do this together.

A Reference Standard-Based Quality Assurance Program for Radiology

The authors have developed a comprehensive radiology quality assurance (QA) program that evaluates radiology interpretations and procedures by comparing them with reference standards. Performance metrics are calculated and then compared with benchmarks or goals on the basis of published multicenter data and meta-analyses. Additional workload for physicians is kept to a minimum by having trained allied health staff members perform the comparisons of radiology reports with the reference standards. The performance metrics tracked by the QA program include the accuracy of CT colonography for detecting polyps, the false-negative rate for mammographic detection of breast cancer, the accuracy of CT angiography detection of coronary artery stenosis, the accuracy of meniscal tear detection on MRI, the accuracy of carotid artery stenosis detection on MR angiography, the accuracy of parathyroid adenoma detection by parathyroid scintigraphy, the success rate for obtaining cortical tissue on ultrasound-guided core biopsies of pelvic renal transplants, and the technical success rate for peripheral arterial angioplasty procedures. In contrast with peer-review programs, this reference standard-based QA program minimizes the possibilities of reviewer bias and erroneous second reviewer interpretations. The more objective assessment of performance afforded by the QA program will provide data that can easily be used for education and management conferences, research projects, and multicenter evaluations. Additionally, such performance data could be used by radiology departments to demonstrate their value over nonradiology competitors to referring clinicians, hospitals, patients, and third-party payers.

Declining mortality following acute myocardial infarction in the Department of Veterans Affairs Health Care System

BackgroundMortality from acute myocardial infarction (AMI) is declining worldwide. We sought to determine if mortality in the Veterans Health Administration (VHA) has also been declining.MethodsWe calculated 30-day mortality rates between 2004 and 2006 using data from the VHA External Peer Review Program (EPRP), which entails detailed abstraction of records of all patients with AMI. To compare trends within VHA with other systems of care, we estimated relative mortality rates between 2000 and 2005 for all males 65 years and older with a primary diagnosis of AMI using administrative data from the VHA Patient Treatment File and the Medicare Provider Analysis and Review (MedPAR) files.ResultsUsing EPRP data on 11,609 patients, we observed a statistically significant decline in adjusted 30-day mortality following AMI in VHA from 16.3% in 2004 to 13.9% in 2006, a relative decrease of 15% and a decrease in the odds of dying of 10% per year (p = .011). Similar declines were found for in-hospital and 90-day mortality.Based on administrative data on 27,494 VHA patients age 65 years and older and 789,400 Medicare patients, 30-day mortality following AMI declined from 16.0% during 2000-2001 to 15.7% during 2004-June 2005 in VHA and from 16.7% to 15.5% in private sector hospitals. After adjusting for patient characteristics and hospital effects, the overall relative odds of death were similar for VHA and Medicare (odds ratio 1.02, 95% C.I. 0.96-1.08).ConclusionMortality following AMI within VHA has declined significantly since 2003 at a rate that parallels that in Medicare-funded hospitals.

Implementing Peer Review

The performance appraisal process is a key component of professional nursing practice. The peer review process is referred to frequently in Magnet Nursing Standards as a key element in professional nursing. The peer review process allows professionals from common practice areas to systematically assess, monitor, make judgments, and provide feedback to peers by comparing actual practice to established standards. Peer review can engage a multigenerational workforce and lead to more satisfied, engaged employees. As a component of the annual performance appraisal, peer review can create positive relationships, foster a better work environment, and allow peers to increase individual and group accountability. Peer review has many common elements that can be individualized to fit any type of unit or work culture. This article describes how to design, implement, and evaluate a unit-based peer review program. The content and steps outlined are intended to support nurse managers in implementing unit-specific peer review programs by focusing on existing expertise and best practices. The implementation steps are divided into 5 phases describing the introduction of concepts and getting nurses engaged, implementation guidelines, piloting the process, staff education, and ongoing evaluation. Staff involvement is the key to a successful unit-based peer review process.

Quality Improvement Initiatives Improve Hypertension Care Among Veterans

We implemented a quality improvement initiative to improve hypertension care at Veterans Affairs-Tennessee Valley Healthcare System. We implemented multiple interventions among 2 teaching hospitals, 5 community-based outpatient clinics, and 4 contract clinic sites. Goals of the program were to (1) improve measurement and documentation of blood pressure (BP), (2) initiate outpatient patient education, (3) emphasize VA/Department of Defense hypertension treatment algorithms to providers, (4) emphasize external peer review program performance goals, and (5) initiate feedback of each clinic's performance. The primary outcome was the proportion of patients seen each week with a diagnosis of hypertension who had their last available BP in control (< or =140/90 mm Hg). Observation time was 40 weeks (14 weeks preintervention, 8 weeks intervention implementation, and 18 weeks postintervention), during which there were 55 586 unique clinic visits for hypertension. After intervention deployment, there was an absolute improvement of 4.2% in BP control (preintervention 61.5% [12 245/19 908] versus postintervention 65.7% [15 809/24 059], P<0.0001). Teaching hospital A had an absolute improvement of 1.4% (63.4% [3544/5591] versus 64.8% [4581/7073], P=0.108). Teaching hospital B showed a 0.8% absolute improvement in BP control (59.7% [2577/4315] versus 60.5% [3416/5650], P=0.456). The community-based outpatient clinics had a combined absolute improvement of 8.6% (60.2% [5252/8728] versus 68.8% [6895/10025], P<0.0001). The contract clinics had a combined improvement of 1.5% (68.4% [872/1274] versus 69.9% [917/1311], P=0.409). Results were sustained 1 year after intervention. After implementing small, focused, and inexpensive interventions, BP control improved 4.2%, thereby improving the quality of hypertension care.

Validation of a global self-assessment and peer-review quality programme dedicated to comprehensive cancer centres: A method to assess, improve and promote quality of integrated care, research, and education in cancer centers

e17565 Background: Oncology is a specialty requiring a combination of multidisciplinary expertises, novel technology, integration of innovation into care and research efforts in order to improve the quality of life and survival of patients. Provision of high-quality care and improvement of disease outcome should be promoted in cancer centers. Methods: The Organization of European Cancer Institutes developed an Accreditation Programme combining: 1-a quantitative questionnaire assessing the human, technical and financial resources and activities in care, research and education dedicated to oncology; 2- a set of standards and criteria for high-quality cancer management integrated in an electronic tool with a scoring system of compliance; 3- a peer-review process for review of the self-assessment outcome and on-site visits; 4- a report and improvement plan allowing to designate comprehensive cancer centers complying with required criteria for high-quality comprehensive cancer patient management. All components of the Accreditation Programme were tested in 8 different voluntary cancer centers in Europe. The objectives were to assess the feasibility, reproducibility and acceptance of the Accreditation Programme; to improve the standards and criteria in terms of understanding and consensus; to validate the peer review methodology and establish a method for reporting and implementing an improvement plan; to provide with a preliminary system of designation of comprehensive cancer centers based on criteria of high-quality integrated research, care and education. Results: The 2 consecutive pilot tests allowed to validate 1- relevant items for the quantitative questionnaire; 2-criteria and standards for high-quality integrated multidisciplinary cancer patient's management. 3-The impact of the Accreditation Programme on improvement of patients management and professionals involvement. Conclusions: The OECI Accreditation Programme is ready for dissemination. Participation to the Programme allows development of improvement plans for innovative and integrated comprehensive cancer patients management in cancer centers. No significant financial relationships to disclose.

Teaching Ethical, Experimental, and Quantitative (TEEQ) Biology through Problem‐Based Writing with Peer Review

Teaching Ethical, Experimental, and Quantitative (TEEQ) Biology is a project to enhance undergraduate students' understanding of bioscience research and thereby foster development necessary for entrance into dynamic careers. The target for this project is Biology 131, a large‐enrollment lecture course on organismal biology that serves biology majors. TEEQ engages students in critical review of scientific research by using the Calibrated Peer Review (CPR) program to structure writing assignments about experimental methods and quantitative approaches in biology. CPR delivers reading background information and a writing assignment that is followed by anonymous peer‐ and self‐ review of the assignments. Student gains were measured both qualitatively and quantitatively. The students completed a Participant Perception Inventory (PPI) to survey their knowledge, experience, and confidence about ethical, experimental, and quantitative aspects of research. Results pointed the way for improving students' confidence in their abilities to understand a historical continuum of changing theories, views, and methodologies in biology. The project also opened opportunities for faculty to interact with students in a way that should lead to better retention of students in the major and to students who are better prepared to make decisions about bioscience research and careers.

Analysis of Peer Review Reports: A Focus on Deficiencies of the Top 20 Triennially Inspected Firms

SUMMARY: This descriptive study analyzes deficiency data disclosed in peer review reports of the AICPA’s Center for Public Company Audit Firms Peer Review Program (CPCAF PRP). We analyze the reports of the largest 20 triennially inspected firms that have both a PCAOB inspection report and a CPCAF peer review report. The CPCAF focuses on audits of non-public companies, while the PCAOB focuses on public company audits. Our analysis identifies a number of interesting points. First, about 60 percent of the reports issued by the CPCAF and PCAOB identify at least one deficiency. Second, both CPCAF and PCAOB reports highlight the pervasiveness of insufficient documentation. Third, other common deficiencies in the CPCAF reports relate to incomplete or inaccurate management representation letters, incomplete financial statements and inaccurate audit reports, incomplete performance of analytical procedures, and insufficient review procedures.

Guideline-concordant hepatitis C virus testing and notification among patients with and without mental disorders

Objective We compared screening for hepatitis C virus (HCV), positive HCV test rates and test result notification among Veterans Affairs (VA) patients diagnosed with psychosis (e.g., schizophrenia, bipolar disorder) or depression or those with no psychiatric diagnosis. Method We identified all patients in the VA's External Peer Review Program national random sample of chart reviews regarding HCV care in fiscal year 2004. We then identified individuals diagnosed with psychosis or depression using data from the VA's National Psychosis and Depression registries. Using multivariable logistic regression and generalized estimating equation analyses, we evaluated associations between mental disorder diagnosis and receipt of HCV screening and notification of results. Results Out of 19,397 patients, 94% received HCV tests. Of those tested, 6.9% were HCV positive, and of those who were positive, 48.2% were notified of their result within 60 days. After adjusting for demographic and clinical factors (including illicit drug use and comorbidities), individuals with psychosis or depression were no less likely to have a positive test or be notified of their result ≤60 days compared to nonpsychiatric groups. Those with drug use disorder diagnoses (OR=4.64; P<.001) and Blacks (OR=1.96; P<.001) were more likely to have a positive test. Conclusions Study limitations included inability to fully control for sociodemographic confounders (e.g., homelessness, incarceration). Nevertheless, we found that although the vast majority of patients were tested, only half were notified of test results. VA HCV prevention and follow-up strategies focus on enhanced notification overall and tailoring HCV risk reduction programs to minorities and those with drug use disorders.

Cause of death in Washington state veterans hospitalized with acute coronary syndromes in the veterans health administration

BackgroundIn the United States, relatively little is known about cause of death in individuals who die prior to or after hospital discharge for acute coronary syndromes (ACS). The purpose of this report was to compare baseline patient characteristics according to whether the underlying cause of death was cardiac or non-cardiac.MethodsWe linked cause of death information from Washington State death records to the Department of Veterans Affairs (VA) External Peer Review Program ACS registry. From 524 individuals who were hospitalized for ACS in veterans hospitals located in Washington State or Oregon, we identified 136 individuals who according to VA death records died during the years 2003 to 2005. Of these, 117 (86%) were found in Washington State death records. Sociodemographic variables, as well as underlying and secondary causes of death, were obtained from Washington State death records provided by the Washington State Department of Health. Clinical variables, including medical histories, presentation on admission, and in-hospital death were extracted from the VA ACS registry.ResultsSomewhat surprisingly, only 52% of veterans died of cardiac causes when only the underlying cause of death was used. However, when secondary causes of death were added to the definition, the proportion that died of cardiac causes increased to 81%. Patient characteristics were similar in the two groups, although small numbers limited the ability to detect statistically significant differences.ConclusionThese preliminary findings suggest that it is important to consider secondary causes as well as the underlying one when classifying deaths as cardiac or non-cardiac.

The Relationship Between Measured Performance and Satisfaction with Care Among Clinically Complex Patients

Recent work has shown that clinically complex patients are more likely to receive recommended care, but it is unknown whether higher achievement on individual performance goals results in improved care for complex patients or detracts from other important but unmeasured aspects of care, resulting in unmet needs and lower satisfaction with care. To examine the relationship between measured performance and satisfaction with care among clinically complex patients An observational analysis of a national sample of 35,927 veterans included in the External Peer Review Program in fiscal years 2003 and 2004. First, compliance with individual performance measures (breast cancer screening with mammography, colorectal cancer screening, influenza vaccination, pneumococcal vaccination, lipid monitoring, use of ACE inhibitor in heart failure, and diabetic eye examination), as well as overall receipt of recommended care, was estimated as a function of each patient's clinical complexity. Second, global satisfaction with care was estimated as a function of clinical complexity and compliance with performance measures. Higher clinical complexity was predictive of slightly higher overall performance (OR 1.13, 95% CI 1.09 to 1.18) and higher performance on most individual performance measures, an effect that was mediated by increased visit frequency. High measured performance was associated with higher satisfaction with care among patients with high clinical complexity. In fact, as complexity increased, the effect of achieving high performance on the odds of being satisfied with care also increased Not only was measured performance higher in clinically complex patients, but satisfaction with care was also higher among clinically complex patients with high measured performance, suggesting that compliance with performance measures in clinically complex patients does not crowd out unmeasured care.

Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence

Problem-based learning (PBL) is an educational strategy designed to enhance self-assessment, self-directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice. This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools. We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross-tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression. We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not. There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.

Trends in Diabetes Mellitus Indicators in Veterans with Spinal Cord Injury

Persons with spinal cord injury (SCI) are at increased risk for developing diabetes mellitus (DM). However, published data on the care provided to this population are minimal. The purpose of this study was to examine a set of measures of quality of DM care in veterans with SCI. Retrospective analysis of the External Peer Review Program data for veterans with SCI and DM from 2002 to 2004 in the Veterans Healthcare System. Trends in DM measures were examined using generalized estimation equation models. The percentage of veterans who received testing for lipids, retinal, and renal exams significantly increased during this period. This was accompanied by significant improvements in intermediate outcomes, glycemic, lipid, and blood pressure (BP) control. The percentage of veterans with glycosylated hemoglobin (HbA1c) (levels < or = 9%; P < 0.001) and those with poorly controlled levels (HbA1c > 9.5%; P = 0.022) improved. BP (140/90) rates increased from 59% in fiscal year (FY) 2002 to nearly 70% in 2004 (P < 0.001). The percentage of veterans who received renal screening (anatomical tests, physiologic test, and urine microalbumin) increased significantly (P < 0.001). Retinopathy exam rates also increased from 55.1% in FY 2002 to 70.8% in FY 2004 (P < 0.001). Significant improvements were made in a set of DM measures used to evaluate care provided to veterans with SCI. The positive trends in DM care seen in the general veteran population were also evident in the SCI population.

Association of Timely Administration of Prophylactic Antibiotics for Major Surgical Procedures and Surgical Site Infection

Prophylactic antibiotic (PA) administration 1 to 2 hours before surgical incision (SIP-1) is a publicly reported process measure proposed for performance pay. We performed an analysis of patients undergoing major surgical operations to determine if SIP-1 was associated with surgical site infection (SSI) rates in Department of Veterans Affairs (VA) hospitals. Patients with External Peer Review Program Surgical Care Improvement Project (SCIP)-1 data with matched National Surgical Quality Improvement Program data were included in the study. Patient and facility level analyses comparing SCIP-1 and SSI were performed. We adjusted for clustering effects within hospitals, validated SSI risk score, and procedure type (percentage of colon, vascular, orthopaedic) using generalized estimating equations and linear modeling. The study population included 9,195 elective procedures (5,981 orthopaedic, 1,966 colon, and 1,248 vascular) performed in 95 VA hospitals. Timely PA occurred in 86.4% of patients. Untimely PA was associated with a rate of SSI of 5.8%, compared with 4.6% in the timely group (odds ratio = 1.29, 95% CI 0.99, 1.67) in bivariable unadjusted analysis. Patient level risk-adjusted multivariable generalized estimating equation modeling found the SSI risk score was predictive of SSI (p < 0.001); SIP-1 was not associated with SSI. Hospital level multivariable generalized linear modeling found procedure mix (p < 0.0001), but not SIP-1 rate or facility volume, to be associated with facility SSI rate. The study had 80% power to detect a 1.75% difference for patient level SSI rates. Timely PA did not markedly contribute to overall patient or facility SSI rates. These data are important for the ongoing discourse on how to measure and pay for quality of surgical care.

Lung Cancer Peer Review Survey

The National Cancer Peer Review Programme is undertaken by peer reviewers and user reviewers resulting in assessments on the quality of cancer services for NHS patients in England.1 2 Throughout...

Suboptimal depression screening following cancer diagnosis

Objective To describe the proportion of veterans with cancer screened for depression as compared to the general population (GenPop) of veterans. Methods Data were abstracted from electronic medical records (2000–2004) at a Midwestern Veterans Health Administration (VHA) facility and from the VHA External Peer Review Program (EPRP). Depression screening was assessed in the 12-month period following cancer diagnosis or in the 12-month period prior to EPRP medical record abstraction. Statistical analysis included multivariate logistic regression. Results Annual depression screening among veterans with cancer improved from 42% in 2000 to 81% in 2003. Screening was 9–31% lower and 11–50% lower among veterans with cancer at the Midwestern facility, as compared to the GenPop of veterans nationwide and at the Midwestern facility, respectively. Of subjects with cancer at the Midwestern facility, 19% screened positive. Advanced disease [odds ratio (OR)=0.51; 95% confidence interval (CI 95): 0.38–0.68] and respiratory cancers (OR=0.55; CI 95: 0.38–0.80) were associated with lower odds of screening receipt. Conclusions Screening for depression among veterans with cancer improved 39% but is considerably lower than the proportion of GenPop veterans screened nationally and at the local facility. Targeted interventions to improve screening in patients with cancer are required based on evidence that screening translates into increased provider recognition and treatment of depression.

Peer Review of Teaching

This article provides an overview and description of peer review of teaching for faculty members and administrators who would like to implement a peer review program. This may include classroom and clinical settings. A brief overview, procedure, and a teaching competence evaluation rubric are provided.

How Objective is Peer Review? Evidence from Self-Regulation of the Accounting Profession

I test whether peer reviews performed under the AICPA's Peer Review Program were objective indicators of audit quality. Using PCAOB inspection opinions of audit firms as a neutral, independent benchmark of audit quality, I find that firms that themselves review other firms consistently receive more favourable opinions from peer reviews than from PCAOB reviews. Firms with peer reviewers who are less likely to be independent (or more likely to be friendly), such as smaller, non-competing firms and firms likely to be `peer review specialists', also receive significantly more favourable opinions from peer review than from the PCAOB. Also, while positive peer reviews from relatively independent reviewers are negatively related to the likelihood of Accounting and Auditing Enforcement Releases, positive peer reviews from more friendly reviewers are not. These findings suggest that not all peer reviews were objective or rigorous, and that some firms were able to control peer review outcomes by choosing friendly reviewers. However, it is unclear whether audit committees impounded the independence of peer review opinions when using these opinions in decisions to hire or fire auditors.

Distributed Accountability: Creating District Systems to Ensure Teaching Quality

Districts play a key and relatively unexamined role in distributed leadership research. This article explores how leadership was distributed through a district structure designed to improve the quality of teaching by improving the quality of teacher evaluation. It examines peer assistance and review, a policy designed to address the key problems of traditional teacher evaluation by allowing administrators and teacher leaders to share accountability for evaluation processes and decisions. The article presents data from a peer assistance and review program in one urban district, detailing how the program distributed accountability for teacher quality across the district organization. The article extends previous work on distributed leadership by showing how the design of shared tasks can effectively distribute accountability. The article also extends previous work on distributed leadership by elucidating the democratic effects of that distribution. As such, the article addresses questions of instrumentality (i.e., how can district leaders design and implement a better teacher evaluation system?) and agency (i.e., what are the political implications of distributing formal authority for teacher evaluation out of the hands of administrators and into the hands of teachers?).

Depression Screening Disparities Among Veterans With Diabetes Compared With the General Veteran Population

We sought to describe the proportion of veterans with diabetes screened for depression compared with the general population of veterans. Electronic medical records (fiscal years 2001-2004) from a Midwestern Veterans Health Administration (VHA) facility and VHA External Peer Review Program (EPRP) data were used for the study. Facility-level data included inpatient and outpatient encounters, which included depression screen results. EPRP data were facility-level summary data, which detailed the proportion of general population veterans nationwide and patients at the Midwestern facility who were screened for depression. Logistic regression tested for associations between depression screen receipt and screening positive and demographic/clinical characteristics among patients with diabetes. Depression screening among those with diabetes improved from 62% in fiscal year 2001 to 83% in 2004. Screening was 9-23% lower and 11-22% lower in patients with diabetes compared with the general population of veterans nationwide and patients at the Midwestern facility, respectively. Seventeen percent of subjects with diabetes screened positive, which is two times higher than in the general population. Women (odds ratio 0.45 [95% CI 0.35-0.60]) and subjects with unknown A1C (0.40 [0.34-0.46]) were less likely to be screened for depression. A >or=50% service-connected disability rating was inversely associated with screening (0.84 [0.72-0.99]) but positively associated with screening positive for depression (1.56 [1.33-1.82]). Screening for depression among veterans with diabetes improved 21% but is considerably lower than the proportion of general population veterans screened nationally and at the facility of interest. Targeted interventions to improve screening in patients with diabetes are required based on evidence that screening translates into increased provider recognition and treatment of depression.