Subjective Outcome Evaluation Findings Research Articles

Evaluation of Service-Learning in Project WeCan under COVID-19 in a Chinese Context.

Service-learning is a widely adopted educational pedagogy and philosophy. With the support from the Wharf (Holdings) Limited (Group), service-learning was conducted in the “Project WeCan” in Hong Kong. Prior to COVID-19, traditional service-learning was implemented with students learning in the classroom and applying their knowledge and skills to the community through providing direct face-to-face service. With the COVID-19 outbreak in the 2019–20 academic year, school lockdown measures appeared. Students had to learn online and to design and implement service offsite. As the impacts of this rapid shift in paradigm remain unknown, this study examined changes in university students using a pretest–posttest design (n = 124) and perceptions of service-learning experience via the subjective outcome evaluation design (n = 192) under COVID-19. The authors also investigated service recipients’ (n = 56) satisfaction with service activities they participated in during the pandemic. Both objective outcome evaluation and subjective outcome evaluation findings revealed that service providers (university students) and recipients (secondary school students) experienced benefits from the Project. Findings support the benefits of online service-learning in “Project WeCan” even during unprecedented times such as COVID-19.

Evaluation of the Project P.A.T.H.S. in Mainland China: Findings Based on Student Diaries

Objectives: Based on 859 student diaries, the present study evaluated a positive youth development program entitled “Tin Ka Ping (TKP) Positive Adolescent Training through Holistic Social programs (P.A.T.H.S.) ” project implemented in mainland China during the 2015–2016 academic year. Method: To understand the perceived effectiveness of the students, the study analyzed quantitative as well as qualitative data derived from the student diaries. Results: The students held very positive views toward the program and the implementers. The narratives of the students also showed that they perceived improvement in the intrapersonal, interpersonal, familial, and societal domains after joining the program. Conclusion: In conjunction with the subjective outcome evaluation findings, the present findings suggest that the “TKP P.A.T.H.S.” project is able to promote holistic development of secondary school students in mainland China.

Perceived effectiveness and satisfaction of a community-based positive youth development program: findings based on high school students

AbstractThis study examined participants’ perceptions of a community-based positive youth development (PYD) program (the Project P.A.T.H.S.) based on the responses of 16,420 junior secondary students who joined the program in 2015. Subjective outcome evaluation approach was adopted to examine the students’ views of program content, program instructors, and program effectiveness. Consistent with previous studies, results showed that students generally perceived the program positively, and positive relationships were found amongst the three domains of evaluation. Multiple regression analyses showed that perceived program content and instructor qualities were significant predictors and could explain 35% of the variance in program effectiveness perceived by the participants. The present findings are basically consistent with previous subjective outcome evaluation findings derived from the school-based and community-based programs of the Project P.A.T.H.S. in Hong Kong.

Is Subjective Outcome Evaluation Related to Objective Outcome Evaluation? Insights from a Longitudinal Study in Hong Kong

Study ObjectivesEmpirical studies investigating the inter-relationships among subjective outcomes and objective outcomes are sparse in the scientific literature. This study examined the relationship between these 2 forms of outcome evaluation data in Chinese adolescents, utilizing longitudinal data. DesignIn Wave 3 and Wave 4 of a randomized group trial, 2,784 Chinese secondary school students responded to the Chinese Positive Youth Development Scale (CPYDS) assessing objective outcome at pretest (Wave 3) and posttest (Wave 4). Students in the experimental schools also responded to a measure of client satisfaction assessing the subjective outcome of a positive youth development program at posttest (Wave 4). ResultsSubjective outcome evaluation measures (eg, process, program, and effectiveness) at Wave 3 were significantly correlated with CPYDS scores at Wave 4 and difference scores. Subjective outcome evaluation measures also predicted changes in the objective outcome evaluation measures scores across time. ConclusionsThe present findings replicated the previous findings that subjective outcome and objective outcome evaluation findings were intimately related, and subjective outcome measures predicted posttest measures and their changes over time. The present study suggests that subjective outcome evaluation findings may yield useful information regarding program effectiveness in youth work settings.

The relationship between subjective outcome evaluation and objective outcome evaluation findings: evidence from China

AbstractThe relationships between subjective outcome evaluation and objective subjective evaluation measures were examined in this study. In a positive youth development program (Tin Ka Ping P.A.T.H.S. Project), 1083 high school students responded to objective outcome measures, including the Chinese Positive Youth Development Scale (CPYDS) and measures of thriving and life satisfaction at posttest. The respondents also responded to the Chinese Subjective Outcome Scale (CSOS) at posttest. Results showed that different measures derived from the CSOS (lessons, subject, sharing and benefits) had significant relationships with different measures of the CPYDS, thriving and life satisfaction. The different dimensions of the CSOS scores also predicted different objective outcome measures. The present findings provide additional evidence supporting the linkages between objective outcome and subjective outcome measures.

Post-lecture subjective outcome evaluation of a university subject on leadership and positive youth development in Hong Kong

AbstractPost-lecture subjective outcome evaluation findings of a university subject promoting the leadership qualities and positive youth development of students at The Hong Kong Polytechnic University are reported. Students were invited to respond to a 16-item post-lecture questionnaire gauging their perceptions toward the subject, lecture, and the teachers after each of the 13 lectures. Factor analysis demonstrated that three factors were abstracted from the scale, including subject attributes, lecture attributes, and teacher attributes. Normative findings showed that students had positive perceptions on each of the dimensions throughout the lectures. Multiple regression analyses revealed that subject attributes, lecture attributes, and teacher attributes predicted global evaluation of the lecture and teacher. Based on the subjective outcome evaluation findings, the present study provides further support for the effectiveness of this subject in promoting the holistic development of the students.

Service leadership education for university students in Hong Kong: qualitative evaluation

AbstractThis study examined the perceptions of students taking a subject entitled “Service Leadership” at The Hong Kong Polytechnic University which was developed using the model proposed by the Hong Kong Institute of Service Leadership and Management (HKI-SLAM). After taking this subject, students were invited to participate in a qualitative study where they used three descriptors and a metaphor to describe their experiences about the subject. Based on the reflections of 50 students, results showed that 96.1% of the descriptors and 90% of the metaphors used by the students were positive. In conjunction with the objective outcome evaluation, subjective outcome evaluation and process evaluation findings, the present study suggests that “Service Leadership” is a subject that can promote holistic development in Chinese university students in Hong Kong.

Evaluation of the project P.A.T.H.S. (extension phase) based on the perspective of the program participants

Subjective outcome evaluation data were collected from 236 schools (n=87,943 students) after students had participated in the Tier 1 Program in the second year of the Extension Phase of the Project P.A.T.H.S. in Hong Kong. Using schools as the units of analysis, results showed that the program and implementers were perceived in a positive manner and approximately four-fifths of the participants regarded the program as helpful to them. There were some significant grade differences in the subjective outcome evaluation findings with small effect size. Multiple regression analyses showed that whereas perceived qualities of the program positively predicted perceived effectiveness of the program, perceived qualities of implementers negatively predicted program effectiveness. The present study suggests that irrespective of cohorts at different times, junior secondary school students perceived the program to be beneficial to them. The theoretical and practical implications of the present findings are discussed.

Helping adolescents with greater psychosocial needs: the extension phase of the project P.A.T.H.S. in Hong Kong

Subjective outcome evaluation data were collected from 24,198 students of 236 schools after completion of the Tier 2 Program in the second year of the Extension Phase of the Project P.A.T.H.S. in Hong Kong. Using schools as units of analysis, results showed that the program, implementers, and effectiveness were perceived in a positive manner. Although subjective outcome evaluation findings differed across participant types, grades, and program types did not show significant differences on the outcome measures. Consistent with previous reports, perceived effectiveness of the program was significantly predicted by student perceptions about the program and program instructor. These findings provide support for the claim that the Tier 2 Program is effective in promoting positive development among adolescents with greater psychosocial needs in the extension phase of the Project P.A.T.H.S. in Hong Kong.

Program implementers’ evaluation of Project P.A.T.H.S. in Hong Kong: a study based on different cohorts

A total of 216 schools participated in the Tier 1 Program (Secondary 1, 2 and 3 levels) of Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social Programmes) in the 2008/09 school year. Based on the subjective outcome evaluation findings collected from students and instructors in each grade, the program implementers wrote down five conclusions in their reports. Utilizing secondary data analysis, the conclusions in the reports were further analyzed. Results showed that most of the conclusions concerning perceptions of the program, instructors and effectiveness of the program were positive in nature. There were also conclusions indicating difficulties encountered and recommendations for improvement. The result of the present study is consistent with previous studies using the same method as well as using other evaluation means for the same grades. The findings suggest that the Tier 1 Program is well received by the stakeholders and the program is effective in promoting the holistic development of adolescents.

Subjective outcome evaluation of the project P.A.T.H.S. In different cohorts of students.

The project P.A.T.H.S. is an indigenously developed positive youth development program in Hong Kong. In the extension phase (2009/2010 school year), subjective outcome evaluation data were collected from 231 schools involving 89,068 participants after completion of the curricula-based Tier 1 Program. With schools as the units of analysis, results showed that participants generally had positive perceptions of the program content and implementers, with over four-fifth of the participants regarded the program as helpful to them. There were some significant grade differences in the subjective outcome evaluation findings, although the related effect size was not strong. Multiple regression analyses revealed that program content and program implementers predicted perceived effectiveness of the program. The present study suggests that irrespective of cohorts, students in the junior secondary years perceived the program to be beneficial to them.

Participants’ Evaluation of the Project P.A.T.H.S.: Are Findings Based on Different Datasets Consistent?

Subjective outcome evaluation findings based on the perspective of the participants of the Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social Programmes) in nine datasets collected from 2005 to 2009 (n = 206, 313 program participants) were examined in this paper. Based on the consolidated data with schools as units, results showed that the participants generally had positive perceptions of the program, implementers, and benefits of the program. More than four-fifths of the participants regarded the program as beneficial to their holistic development. Multiple regression analysis revealed that the perceived qualities of the program and the program implementers predicted perceived effectiveness of the program. Based on the subjective outcome evaluation findings, the present study provides support for the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. in Hong Kong.

Evaluation of the Training Program of the Project P.A.T.H.S.: Findings Based on the Perspective of the Participants from Different Cohorts

Subjective outcome evaluation findings based on the perspective of the participants participating in a 3-day training program of the Project P.A.T.H.S. are reported in this paper. The findings were based on the data collected from the training workshops conducted between 2005 and 2009 (N = 4.167). Results showed that the respondents had good and positive perceptions of the training program and found it very valuable, particularly with respect to training instructors and familiarization with the project. Besides, the training participants were able to acquire attitude, knowledge and skills that are conducive to the successful implementation of the program. Based on the subjective outcome evaluation findings, it is concluded that the training program was effective in helping the participants to acquire the necessary knowledge, attitudes and skills in implementing the program.

Training evaluation of the Secondary 3 Training Program of the Project P.A.T.H.S. in Hong Kong

Abstract A total of 235 participants from 48 schools participated in a 3-day training program designed to train teachers and social workers to implement the Secondary 3 Program of the Project P.A.T.H.S. in Hong Kong. At the end of the training program, participants were invited to respond to a structured subjective outcome evaluation questionnaire containing 31 items. The findings revealed that most of the respondents had very positive perceptions about the training program and the instructors. The present evaluation findings are consistent with the subjective outcome evaluation findings in Secondary 1 and Secondary 2 Training Programs, which revealed the training program of the Project P.A.T.H.S. generated positive reactions, enhanced learning, and desired behavioral changes in the program participants.

Predictors of subjective outcome evaluation findings in a positive youth development program in Hong Kong

This study attempted to examine correlates of subjective outcome evaluation findings amongst Chinese junior secondary students from 216 schools who joined in a positive youth development program in Hong Kong. With individual students as the unit of analysis, results revealed that three factors (perceived program quality, perceived instructor quality, and perceived effectiveness) were extracted from a 36-item client satisfaction scale. Significant grade differences in the subjective outcome evaluation findings were found, although the effect size was small. Multiple regression analyses revealed that perceived qualities of the program and the program implementers positively predicted perceived effectiveness of the program. The findings suggest that the use of schools vs. individuals as units of analyses would yield different results on the determinants of subjective outcome evaluation.

Subjective Outcome Evaluation Findings: Factors Related to the Perceived Effectiveness of the Tier 2 Program of the Project P.A.T.H.S.

After completion of the Tier 2 Program of the Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social Programmes), 8,489 participants in 196 schools responded to the Subjective Outcome Evaluation Form (Form C) to assess their views of the program, program workers, and perceived effectiveness of the program. Four major program elements were identified, including programs based on the adventure-based counseling approach (n = 48), programs concentrated on volunteer training and services (n = 44), programs with both the adventure-based counseling approach and volunteer training activities (n = 63), and other programs with different foci (n = 41). Descriptive statistics showed that the respondents had positive perceptions of the program, workers, and benefits of the program. Perceived qualities of the program and the program workers were positively associated with perceived effectiveness of the program. Multiple regression analysis revealed that perceived qualities of the program, but not the program workers, predicted perceived effectiveness of the program. The theoretical and practical implications of the findings are discussed.

Secondary Data Analyses of Subjective Outcome Evaluation Findings of the Project P.A.T.H.S. in Hong Kong

The Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social Programmes) is a positive youth development program in Hong Kong. After completion of the program, program implementers were required to draw five conclusions based on the subjective outcome evaluation findings collected from the program participants and implementers as reported in the evaluation report. Secondary analyses of the data collected from 48 schools that had joined the Secondary 3 program showed that most of the conclusions concerning perceptions of the program, instructors, and effectiveness of the program were positive in nature. There were also conclusions indicating strengths and possible improvement of the program. The present findings are consistent with the previous findings that suggest that the Project P.A.T.H.S. is well received by the stakeholders and the program is beneficial to the development of Chinese adolescents in Hong Kong.

Subjective Outcome and Objective Outcome Evaluation Findings: Insights From a Chinese Context

Objective: As there are few studies examining the relationship between subjective and objective outcome evaluation findings, this study investigated the linkage between these two types of outcomes in the Chinese culture. Method: In an experimental study, 3,298 Chinese secondary school students responded to the Chinese Positive Youth Development Scale assessing objective outcome (CPYDS) at pretest and posttest. They also responded to the Chinese Subjective Outcome Scale (CSOS) assessing subjective outcome at posttest. Results: The CSOS total and subscale scores were significantly correlated with posttest CPYDS scores and difference scores; they also predicted changes in CPYDS scores across time. Conclusions: The present findings demystify the common belief that findings based on the client satisfaction approach are unrelated to objective outcome evaluation findings.

Evaluation of Project P.A.T.H.S. (Secondary 2 Program) by the Program Implementers: Findings Based on the Experimental Implementation Phase

A total of 49 schools (N = 8,167 students) participated in the Secondary 2 Program of the Experimental Implementation Phase of the Project P.A.T.H.S. (Positive Adolescent Training through Holistic Social Programmes). After completion of the Tier 1 Program, 270 instructors completed the Subjective Outcome Evaluation Form (Form B) to assess their views of the program, their own performance, and perceived effectiveness of the program. Based on the consolidated reports submitted by the schools to the funding body, the research team aggregated the consolidated data to form a “reconstructed” overall profile on the perceptions of the program implementers. Results showed that high proportions of the instructors had positive perceptions of the program and their own performance, and over 90% of the instructors regarded the program as helpful to the program participants. These findings are consistent with the subjective outcome evaluation findings based on the perspective of the program participants.

Evaluation of Project P.A.T.H.S. (Secondary 1 Program) by the Program Implementers: Findings Based on the Full Implementation Phase

A total of 207 schools (N = 35,735 students) participated in the Secondary 1 Program of Project P.A.T.H.S. in the full implementation phase (2006/07 school year). After completion of the Tier 1 Program, 1,250 instructors completed a subjective outcome evaluation form (Form B) to assess their views of the program, instructors, and perceived effectiveness of the program. Utilizing the consolidated reports submitted to the funding body, the research team aggregated the consolidated data to form an overall profile of the perceptions of the program participants. Results showed that high proportions of the respondents had positive perceptions of the program and the instructors, and roughly four-fifths of the respondents regarded the program as helpful to the program participants and the workers. These findings complement the subjective outcome evaluation findings based on the perspective of the program participants.