Faculty Teaching Effectiveness Research Articles

Harmonizing Perceptions: Exploring Alignment of Numerical and Textual Evaluation Through Fine-Grained Sentiment Analysis

The assessment of faculty teaching effectiveness plays a pivotal role in shaping educational practices and ensuring academic excellence. While numerical evaluations provide quantifiable measures of teaching effectiveness, textual comments offer valuable qualitative insights into the faculty's instructional methods and interactions. However, discrepancies between these evaluation formats may hinder the accurate assessment of faculty performance. This study aims to determine the alignment between the numerical and textual evaluations of faculty teaching effectiveness using fine-grained sentiment analysis and determine the relationship between numerical ratings and textual comments to identify patterns of consistency or divergence. A large dataset of anonymized BSCS students’ feedback is analyzed through sentiment analysis to extract sentiments expressed in the textual comments. The study results reveal a misalignment between students’ subjective perceptions expressed in the textual comments and the numerical ratings. This misalignment arises due to several factors, such as student communication style variations, individual interpretations of rating scales, or subjective biases. This study advances faculty evaluation methodologies and offers useful recommendations for organizations looking to improve assessment procedures as well as educational institutions that want to improve the precision and efficacy of their faculty evaluation systems for better faculty development and decision-making.

Assessment of nursing-related teaching quality based on fuzzy comprehensive evaluation

Abstract This paper designs a scheme for a nursing professional teaching quality evaluation system, including constructing an evaluation index system and a nursing teaching quality evaluation model based on a fuzzy comprehensive evaluation. Among them, the establishment of a fuzzy comprehensive evaluation model involves the determination of fuzzy operators and the processing of evaluation indexes. Finally, the constructed method was used to conduct a study on evaluating nursing teaching quality in College H for students of different academic years. The results showed that freshman, sophomore, and junior students were inconsistent in all aspects of faculty teaching effectiveness and overall evaluations. The total rating of freshmen was 18.34±0.17. The total rating of sophomores was 20.50±0.15. The total rating of juniors was 23.46±0.12. And 66.7% of the faculty members thought that the quality of nursing teaching at the university was not up to standard. Therefore, this paper provides a valid assessment method for nursing education.

Limitations of Student Evaluations for Assessment of Faculty Teaching Effectiveness

Limitations of Student Evaluations for Assessment of Faculty Teaching Effectiveness

Influence of teacher’s motivation on their effectiveness in the faculty in state institution of higher learning in the province of Iloilo

This survey-correlational research determined the level faculty work motivation and teaching effectiveness as well as the significance of the differences and relationships. The antecedent variables were length of teaching experience, academic rank, educational qualification and college affiliation. They were further categorized as 10 yers and below, 11-20, or above 20 years for the length of teaching experience; instructor, assistant professor, associate professor, or professor for academic rank; baccalaureate, master’s or doctorate degree foe educational qualification; and management, teacher education, information technology, agriculture, or maritime for college affiliation. The independent variables was work motivation and teaching effectiveness was the dependent variable. The study was conducted among the 139 randomly selected faculty in a state institution of higher learning in the province of Iloilo. Two adopted, validated and pilot tested data gathering instruments were used – the Work Motivation Questionnaire (WMQ) by Steers and Braunstein (1976, in Siason 2008); and Teacher Effectiveness Questionnaire (Mishra, 2012). Statistical tools were means, standard deviations, one-way ANOVA, and the Pearson’s r. Significance level for all inferential tests was set at .05 alpha. Study results revealed that the faculty had high motivation to work and were effective as teachers. Significant differences were noted in work motivation and teaching effectiveness among the faculty classified according to educational qualification. A significant relationships existed among the faculty work motivation and teaching effectiveness. Human resource management officer shall update their faculty development plan in the area of motivation and effective teaching.

The Effect of Salary Compensation for Time Spent Teaching in an Orthopaedic Residency Program: An Analysis of Teaching Performance Reviews.

Although there has been a recent emphasis on standardized resident assessments within Accrediation Council for Graduate Medical Education programs, assessments of faculty teaching performance and effectiveness are less frequent. Our purpose was to compare the teaching performance of orthopaedic surgery faculty receiving compensation for time spent teaching with faculty without compensation. For this prospective investigation, we collected anonymous resident reviews of 23 orthopaedic faculty within a rural, academic orthopaedic residency program over 2 academic years. Performance reviews of the faculty used a validated assessment of clinical teaching effectiveness with nine domains (faculty knowledge, organization, enthusiasm, rapport, involvement in learning experiences, feedback, clinical skill, accessibility, and overall effectiveness). A composite teaching effectiveness score was determined by adding each of the scores from the individual domains. We compared reviews for faculty members with and without compensation for time spent teaching. A total of 202 performance reviews for 23 orthopaedic faculty were analyzed. Most of the faculty were male (91%), and 61% received compensation for teaching. No demographic differences were observed between the two faculty groups. Notable differences between the groups were noted in three domains: enthusiasm, ability to establish rapport as well as direction, and feedback. Faculty compensated for teaching demonstrated a markedly higher composite teaching effectiveness score than those without compensation. These data suggest that orthopaedic faculty compensated for teaching responsibilities provide a better educational experience for resident trainees compared with faculty without compensation for teaching. Future studies should aim to assess varying compensation models for teaching responsibilities across different departments.

Structured simulations improves students' knowledge acquisition and perceptions of teaching effectiveness: A quasi-experimental study.

To determine the effect of adding structured simulation to a traditional classroom lecture format on students' knowledge acquisition and their evaluation of faculty teaching effectiveness, compared to traditional classroom lectures alone. There is a critical lack of high-quality clinical placements/experiences in nursing education, particularly in clinical specialty populations, such as paediatrics. Simulation has been shown to help students practice in a safe environment. However, less is known about the outcomes of embedding simulation in didactic class sessions or classroom lectures. Additionally, data on the impact of simulation as a teaching pedagogy on faculty teaching effectiveness is limited. Post-test, nonequivalent control group quasi-experimental study. Students in two prelicensure nursing programs participated in a 10-week term paediatric nursing course taught by the same faculty member on two different campuses. The students at one campus, designated as the control group (n=43), received four hours of traditional, lecture-only, instructions. The group at a second campus, the intervention group (n=44), received the same lectures with added structured simulations. Students' knowledge acquisition and their evaluation of faculty teaching effectiveness were measured in both groups using the same standardised assessment measures. The CONSORT checklist was followed. Students' knowledge acquisition scores and their evaluation scores of faculty teaching effectiveness were significantly higher in the intervention (simulation) group compared to the control group. A significantly higher number of students in the simulation group scored above the national average Nursing Care of Children score as compared to students in the control group. Integrating simulation with didactic instruction enhanced students' knowledge acquisition and improved their opinion of faculty teaching effectiveness. Augmenting lecture with simulation may provide students with learning experiences that they may not have during clinical rotation due to a lack of paediatric clinical placement sites and differences between sites.

Faculty Work Engagement and Teaching Effectiveness in a State Higher Education Institution

This survey-correlational research determined the faculty work engagement and teaching effectiveness. The antecedent variables were length of teaching experience, academic rank, educational qualification and college affiliation. The independent variable was work engagement and teaching effectiveness was the dependent variable. The study was conducted among the 139 randomly selected faculty in a state institution of higher learning in the province of Iloilo. Two adopted, validated and pilot tested data gathering instruments were used - the Employee Engagement Survey Question (Margolis, 2018) and Teacher Effectiveness Questionnaire (Mishra, 2012). Statistical tools were means, standard deviations, one-way ANOVA, and the Pearson’s r. Significance level for all inferential tests was set at .05 alpha. Study results revealed the faculty were engaged in their work and were effective as teachers. The faculty did not differ significantly in their work engagement when they were classified according to length of teaching experience, academic rank, educational qualification and college affiliation. A significant difference was noted in teaching effectiveness among the faculty classified according to educational qualification but no significant differences when they were classified according to length of teaching experience, academic rank and and college affiliation. A significant relationships existed among the faculty work engagement, and teaching effectiveness.

Teaching Effectiveness and Morale of Criminology Faculty of Laguna State Polytechnic University, Philippines

Improving the quality of the faculty supposed to be a main concern in improving the quality of education. Therefore, this study was conducted to assess the teaching effectiveness and morale of criminology faculty of Laguna State Polytechnic University, Philippines using descriptive-correlational method. The study employed the complete enumeration of the faculty of Criminology of the mentioned University for the First Semester Academic Year 2018-2019 which was thirty-four (34). Descriptive statistics such as frequency and percentage, weighted mean, independent t-test, one-way analysis of variance (ANOVA), and pearson correlation were employed using Statistical Package for the Social Sciences (SPSS) data base software. Results revealed that no significant relationship was found between the profiles and morale of criminology faculty. Oppositely, there was significant relationship between the civil status and status of appointment-profile and teaching effectiveness. This signifies that the aforesaid profile-variables are important in rating the teaching effectiveness of the faculty. This information may be useful in the faculty training program and serves as a guide in higher education classrooms, particularly in criminology education. Although no significant influence was found between teaching effectiveness and morale, it still provides an important perspective as to what is necessary to further develop and foster a working environment that will aid the faculty to be the best possible resource of the University. With this, faculty of Criminology will perform responsibilities with initiatives to continually improve the quality of education delivered to their clientele, particularly students, and contribute to the realization of the University's vision, mission and quality policy.

Are all student evaluation scores created equally? Identifying factors that hospitality administrators should consider when assessing student feedback

Abstract: Hospitality program administrators must evaluate faculty on their perceived level of that faculty's teaching effectiveness. Interpreting student evaluation ratings is the most frequently used method of assessing faculty performance. This study examines factors that potentially should be taken into consideration while assessing student evaluation scores. Evaluation scores (n = 4893) were analyzed for one academic year. The findings show a relationship between class size, grade distribution, and teaching experience with student ratings of faculty. The results also discovered that student ratings were significantly higher for courses that required a laboratory component and near significantly higher for elective vs. required courses.

Measuring faculty teaching effectiveness using conditional fixed effects

Using a dataset of 48 faculty members and 88 courses over 26 semesters, the authors estimate Student Evaluation of Teaching (SET) ratings that are conditional on a multitude of course, faculty, and student attributes. They find that ratings are lower for required courses and those where students report a lower prior level of interest. Controlling for these variables substantially alters the SET ratings for many instructors. The average absolute value of the difference between the faculty ratings controlling just for time effects and fully conditional ratings is nearly one-half of a standard deviation in the students’ rating of how much they learned. This difference produces a change in quartile rank for over half the sample across two summary course evaluation measures.

Correlates of Teaching Effectiveness of the General Education Faculty in a State University of the Philippines

Universally recognized and accepted as a direct and promising strategy to improve education outcomes is improving the teacher quality and measuring teaching effectiveness. This study investigated variables that influence the students’ evaluation of the teaching effectiveness of the general education (GE) faculty of a state university in the Philippines for the Second Semester, School Year 2014-2015. It looked into factors pertaining to the professional profile of the GEfaculty members that affect the students’ evaluation of the faculty teaching with the use of correlational research design. Findings revealed that of the variables considered, length of service in the teaching profession has a significant inverserelationship with the students’ evaluation of the faculty members’ teaching effectiveness particularly regarding their commitment. The findings suggest that e GE faculty should continue to live up to the precept that teaching is a vocation. ger. The faculty members whether in the early years of teaching or the experienced faculty must be likewise provided with faculty development programs. The university should consider a shift to online student evaluation of faculty teaching effectiveness.

The scholarly productivity and work environments of academic pharmacists.

Productive faculty are key to generating new knowledge and advancing pharmacy practice. The work environments of academic pharmacists are critical to their vitality, commitment, and longevity. To (1) identify correlates of faculty scholarly productivity and teaching effectiveness, considering personal and environmental characteristics; (2) determine the relationship between a faculty's perception of organizational citizenship behaviors they witness with the organizational culture of their employing college/school of pharmacy; and (3) describe the relationship between organizational climate, job satisfaction, and commitment of academic pharmacists. A self-administered survey was disseminated to a random sample of U.S. academic pharmacists acquired from AACP list-servs. The survey measured perceptions of their organization's culture, the organizational citizenship behaviors they witness at their institution, their job satisfaction, teaching load and productivity, and scholarly productivity based upon peer-reviewed scholarly papers accepted. Both bivariate and multivariate (regression) procedures were employed to identify factors most responsible for explaining academic pharmacist's work environment. Responses were received from 177 of 600 survey recipients. Faculty reported having had accepted 10.9±13.6 papers in peer-reviewed journals during the previous 5 years, with most of those in journals with relatively low Impact Factor scores. Faculty productivity was related to type of academic institution employed, teaching effectiveness, job satisfaction, and other factors. Organizational citizenship behaviors and organizational culture was seen similarly by faculty of varied ranks and experience levels. Commitment to remain at the current college/school of pharmacy was highly associated with culture, climate, and job satisfaction conditions. The results provided evidence for a strong connection or nexus between teaching and research effectiveness. Organizational culture of academic pharmacy programs is highly important for faculty vitality and commitment. The findings should be helpful for academic leaders in devising programs for mentoring, development, and retention of faculty.

Moderation of Organisational Politics on Job Satisfaction and Teaching effectiveness among Engineering Faculties

The faculty teaching effectiveness is assessed based on the academic performance such as teaching ability, professional competence, student evaluation, interpersonal relationship and personality trait. The opinion of 410 faculties of various private engineering colleges in Rayalaseema region of Andhra Pradesh are been collected for analysis. The study follows descriptive research design. The study follows convenience sampling method in selection of faculty for collecting primary data. The results revealed: *The study found moderate perception levels of organizational politics, Job Satisfaction and Teaching Effectiveness among faculty. *There is significant relationship between teaching effectiveness, organisational politics and Job satisfaction. *The study finds negative relationship of organisational politics with Teaching Effectiveness. *The study finds insignificant negative moderation effect of organisational politics on Job Satisfaction and Teaching Effectiveness. There is insignificant moderation effect of organisational politics. However organisational politics is a significant variable in influencing Teaching effectiveness.

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.

The Numbers Tell It All: Students Don't Like Numbers!

Undergraduate Students' interest in taking quantitative vs. non quantitative courses has received limited attention even though it has important consequences for higher education. Previous studies have collected course interest ratings at the end of the courses as part of student evaluation of teaching (SET) ratings, which may confound prior interest in taking these courses with students' actual experience in taking them. This study is the first to examine undergraduate students' interest in quantitative vs. non quantitative courses in their first year of studies before they have taken any quantitative courses. Three hundred and forty students were presented with descriptions of 44 psychology courses and asked to rate their interest in taking each course. Student interest in taking quantitative vs non quantitative courses was very low; the mean interest in statistics courses was nearly 6 SDs below the mean interest in non quantitative courses. Moreover, women were less interested in taking quantitative courses than men. Our findings have several far-reaching implications. First, evaluating professors teaching quantitative vs. non quantitative courses against the same SET standard may be inappropriate. Second, if the same SET standard is used for the evaluation of faculty teaching quantitative vs. non quantitative courses, faculty are likely to teach to SETs rather than focus on student learning. Third, universities interested primarily in student satisfaction may want to expunge quantitative courses from their curricula. In contrast, universities interested in student learning may want to abandon SETs as a primary measure of faculty teaching effectiveness. Fourth, undergraduate students who are not interested in taking quantitative courses are unlikely to pursue graduate studies in quantitative psychology and unlikely to be able to competently analyze data independently.

Do student evaluations influence the teaching skills of clerkship clinical faculty?

Web-based student evaluations of clinical faculty were collected over an 8-year period. There were 19,881 medical student evaluations over the 8-year period for all clinical clerkships, representing a total of 952 faculty. Students used a 5-point Likert scale to rate the teaching effectiveness of faculty. Criterion-based methods and standard deviation option methods were used to group faculty as “optimal” and “sub-optimal” teachers. In more than one year, 794 faculty were evaluated allowing analysis of trends in student evaluations of their teaching skills. In contrast to clinical faculty who received optimal ratings for their teaching skills, those faculty who received sub-optimal student ratings tended not to improve their teaching skills.

17: MEASURING STUDENT LEARNING TO DOCUMENT FACULTY TEACHING EFFECTIVENESS

Recent research has questioned the validity of student ratings as proxy measures for how much students learn, and this learning is a commonly accepted meaning of faculty teaching effectiveness. Student ratings capture student satisfaction more than anything else. Moreover, the overriding assessment criterion in accreditation and accountability—that applied to programs, schools, and institutions—is student learning, so it only makes sense to evaluate faculty by the same standard. This chapter explains and evaluates course-level measures of student learning based on data that are easy for faculty to collect and administrators to use.

Determinants, Associations, and Psychometric Properties of Resident Assessments of Anesthesiologist Operating Room Supervision

A study by de Oliveira Filho et al. reported a validated set of 9 questions by which Brazilian anesthesia residents assessed faculty supervision in the operating room. The aim of this study was to use this question set to determine whether faculty operating room supervision scores were associated with residents' year of clinical anesthesia training and/or number of specific resident-faculty interactions. We also characterized associations between faculty operating room supervision scores and resident assessments of: (1) faculty supervision in settings other than operating rooms, (2) faculty clinical ability (family choice), and (3) faculty teaching effectiveness. Finally, we characterized the psychometric properties of the de Oliveira Filho etal. question set in an United States anesthesia residency program. All 39 residents in the Department of Anesthesia of the University of Iowa in their first (n = 14), second (n = 13), or third (n = 12) year of clinical anesthesia training evaluated the supervision provided by all anesthesia faculty who staffed in at least 1 of 3 clinical settings (operating room [n = 49], surgical intensive care unit [n = 10], pain clinic [n = 6]). For all resident-faculty pairs, departmental billing data were used to quantitate the number of resident-faculty interactions and the interval between the last interaction and the assessment. A generalizability study was performed to determine the minimum number of resident evaluations needed for high reliability and dependability. There were no significant associations between faculty mean operating room supervision scores and: (1) resident-faculty patient encounters (Kendall τb = 0.01; 95% confidence interval [CI], -0.02 to +0.04; P = 0.71), (2) resident-faculty days of interaction (τb = -0.01; 95% CI, -0.05 to +0.02; P = 0.46), and (3) days since last resident-faculty interaction (τb = 0.01; 95% CI, -0.02 to 0.05; P = 0.49). Supervision scores for the operating room and surgical intensive care unit were highly correlated (τb = 0.71; 95% CI, 0.63 to 0.78; P < 0.0001). Supervision scores for the operating room also were highly correlated with family choice scores (τb = 0.77; 95% CI, 0.70 to 0.84; P < 0.0001) and teaching scores (τb = 0.87; 95% CI, 0.82 to 0.92; P < 0.0001). High reliability and dependability (both G- and ϕ-coefficients > 0.80) occurred when individual faculty anesthesiologists received assessments from 15 or more different residents. Supervision scores provided by all residents can be given equal weight when calculating an individual faculty anesthesiologist's mean supervision score. Assessments of supervision, teaching, and quality of clinical care are highly correlated. When the de Oliveira Filho et al. question set is used in a United States anesthesia residency program, supervision scores are highly reliable and dependable when at least 15 residents assess each faculty.

17: Measuring Student Learning to Document Faculty Teaching Effectiveness

17: Measuring Student Learning to Document Faculty Teaching Effectiveness

Evaluation Research on Assessment of Clinical Nursing Teaching Quality Based on Fuzzy Comprehensive Evaluation Method

The paper aims to study the fuzzy comprehensive evaluation algorithm in the assessment system of the clinical nursing teaching quality. In terms of research methods, the paper objectively evaluates teaching effectiveness of clinical nursing faculty, and proposes a method named the fuzzy evaluation of teaching quality, puts forward basic principles and implementation methods, develops the index system, uses Gaussian function to determine the membership degree of index, and apply fuzzy decision-making quantitative calculation in the evaluation results. The research results can be summarized as follows: in the four evaluation sets of excellent, good, medium, and poor, the evaluation of the clinical nursing teaching quality has the following membership degree respectively: 82.3%, 7.4%, 10%, 0.2% and 0. The teachers are favored in the conclusions of teaching quality evaluation. Thus, we come to the following conclusion: the fuzzy comprehensive evaluation has the advantage of good application in the assessment of teaching quality and it has great significance on urging the school to change the education system, assessing teaching achievements and guiding students to the learning in practice.