Abstract A37: Measuring internal consistency of community engagement principle items and calculation of community engagement scores

Abstract

Abstract Background: Comprehensive cancer centers have been mandated to engage communities in their work; thus measurement of the level of community engagement in projects is a priority area. Siteman Cancer Center's Program for the Elimination of Cancer Disparities (PECaD) works with communities to reduce health disparities through outreach, education, and training. PECaD projects seek to align with 11 engagement principles (EP) that have been previously developed in the literature. The PECaD evaluation team developed items that are designed to assess how well PECaD projects adhere to the 11 engagement principles. While the importance of community engagement in research has been previously established, there are few evidence-based approaches for measuring the level of community engagement in research projects. Methods: 58 community members newly engaged in community-academic research partnerships completed a community engagement survey. The community engagement survey contained 96 items on community engagement pertaining to 11 Engagement Principles (EP) and evaluates how well the project aligns with the PECaD engagement principles. The questions had Likert scale response options; 48 questions measuring quality (how well) and 48 questions measuring quantity (how often). Each Engagement Principle is measured by 3-5 quality items and 3-5 corresponding quantity items. The scores developed assess the level of community engagement with higher scores corresponding to higher quality/frequency of engagement. Cronbach's alpha is used to measure the internal consistency of items to measure a single EP. Quality scores are created using 48 quality items and quantity scores are created using the corresponding 48 quantity items. In order to be included in score calculations of the average sum score on each scale (quality/quantity), each respondent had to have at least 3 answered items for EPs that contained 4-5 items and at least 2 answered items for EPs that contained 3 items. Overall average sum scores for each principle were calculated by summing the quantity average and quality average scores for each engagement principal. The lowest possible EP-specific score was 2, while the highest possible score was 10. Results: Every EP item group had a Cronbach's alpha >.85, which indicates strong internal consistency for all question groups across both the quality and quantity scales. Average EP scores ranged from 6.87 to 8.06; distribution of scores was approximately normal. Survey respondents reported adherence to the 11 PECaD engagement principles between sometime and most of the time on the quantity scale and between good and very good on the quality scale. Conclusion: It's necessary to examine internal consistency in order to develop measures that accurately determine how well PECaD projects align with the engagement principles and develop appropriate measures (scores) for assessing community engagement. The strong results of this analysis suggest that question items are internally consistent with each engagement principle. Engagement scores although not extremely high; are quite good and better than expected considering there has been less than 10 months of engagement. Cronbach's alpha calculations are sufficiently precise for n=30, when at least 2 items are analyzed and have a mean intercorrelation of at least 0.70; which is true for all EP item groups. Despite its limitations, this makes this pilot study valuable even with its overall small sample size. Citation Format: Melody S. Goodman, Renee Gennarelli, Bettina F. Drake, Vetta Thompson, Deborah Bowen. Measuring internal consistency of community engagement principle items and calculation of community engagement scores. [abstract]. In: Proceedings of the Sixth AACR Conference: The Science of Cancer Health Disparities; Dec 6–9, 2013; Atlanta, GA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2014;23(11 Suppl):Abstract nr A37. doi:10.1158/1538-7755.DISP13-A37