Problem-based Learning Assessment Research Articles

Development of Instrument Soft Skill Assessment in Problem-Based Learning to Improve Students' Collaboration Skills, Communication Skills, and Social Skills

The research aims to produce the instrument soft skill assessment in problem-based learning that are valid, practical, and effective in assessing students' collaboration, communication, and social skills. This research uses the 4-D development model consisting of four stages: define, design, develop, and disseminate. At the define stage, a preliminary study is carried out in the form of a needs analysis and literature study, then the formulation of objectives. At the design stage, product determination is carried out, product design is created and the instrument soft skills assessment is designed. At the development stage, initial field tests are carried out in the form of expert validation, product revisions, main field tests to obtain data on the practicality and effectiveness of the product, then the final product is obtained. At the disseminate stage, product distribution was carried out on a large scale, but this research only reached the development stage due to limited research time. The research sample consists of tenth-grade students at SMA N 1 Padang Cermin, Lampung. Data collection techniques include methods such as self assessment questionnaires and documentation. Data analysis employs qualitative analysis with descriptive percentage for expert practitioner validation, effectiveness, and quantitative analysis using SPSS for effectiveness. The results of this study indicate that the instrument soft skill assessment in problem-based learning to improve students' collaboration, communication, and social skills in high schools are developed with theoretical, practical, and effective feasibility.

Learning enhancement through deploying personal learning environment, network and problem-based learning assessment

Intended to foster students’ self-directed learning and research ability at the postgraduate level, this educational research evaluated the implications of personal learning environment & network (PLE&N) and problem-based learning (PBL) assessment as student-centric and student-driven pedagogic approaches. Compared to conventional instructional paradigms, the PLE&N platform presents an agile and interactive platform that empowers students to partake in collaborative discourse while orchestrating their learning journey through the judicious utilization of communal resources. The vitality of the PLE&N model is substantiated by the enduring engagement of peers, teachers, professionals, and external experts in the field, fostering a milieu of enriched knowledge exchange, synergistic collaboration, and personalized curation of learning approaches. In contrast, the PBL assessment was designed by incorporating intricate scenarios similar to the real world, thereby affording an avenue for cultivating students’ cogitation and adroit problem-solving skills within a guided learning environment. By integrating constructivist and connectivist pedagogies, the PBL assessment facilitated teamwork while engaging students in independent research and empowered their knowledge construction on selected topics. Students’ self-evaluation through the survey indicated their enhancement in self-directed learning behavior and competence building for complex problem-solving tasks. Moreover, peer feedback and suggestions received through both PLE&N and PBL were highly beneficial for students to analyze information and learning resources from diverse perspectives, which further helped to expand their learning horizon. This study underscored the significance and advantages of modern student-centric approaches such as PLE&N and PBL. These pedagogical paradigms stand as seminal strategies for nurturing nascent post-graduate students, thereby endowing them with indispensable proficiencies in self-directed learning and robust competencies in scholarly inquiry. Keywords: PLE&N, PBL, Constructivism, Connectivism, Pedagogy, Self-directed learning.

A Multi-Project Evaluation of Engineering Students’ Performance for Online PBL: Taking the Sustainable Decision Analysis Course as an Example

In order to meet the growing demand for engineering professionals who can incorporate sustainable solutions into their work, sustainability courses have been launched in online problem-based learning (PBL) environments through various real-life projects. Nonetheless, the conventional one-off grading approach may fail to capture the intricate variations in students’ performance across different projects. To address this problem, a multi-project evaluation framework utilizing the probability exceedance method (PEM) is proposed, which can fuse linguistic evaluation data presented in probability distributions without the need to obtain weights of criteria. In the case study, a comprehensive evaluation of the performance of students majoring in engineering management is conducted within a study group over an online PBL course on sustainable decision analysis. The sensitivity analysis demonstrates that consistent scores can be achieved after assigning different values of fuzzy measures to each criterion. This study enables teachers to holistically evaluate students without being bound by rigid numerical standards or strict weighting schemes, thus allowing them to focus on other educational tasks while ensuring effective and reliable results. Moreover, it contributes to educational innovation by introducing a modern and comprehensive approach for engineering student assessment in online PBL, aligning with the evolving needs of educational sustainability in higher education.

Assessment Of Problem-Based Learning In Dental Education And Exploring Barriers To Its Implementation: Theory To Reality.

There are 59 dental colleges in Pakistan out of which 17 are in the public sector and 42 in private. However, only a few use problem-based learning methods, though it is a popular strategy in dental education all around the world. This study aims to assess problem-based learning model in dental education and explore the barriers of its implementation in a private dental college of Karachi. Qualitative case-study approach was employed. The philosophical stance used was critical realism. Qualitative data was collected by participant observation, video recorded observation and video elicited semi-structured in-depth interviews of five faculty members and 15 students. Results showed that students were more interested in interactive sessions while faculty members were in favour of problem-based learning sessions. Thematic analysis was done to generate themes. This research applies reproduction method to explain the necessary and contingent relations and causal powers. Lack of motivation among students and lack of faculty dedication are causal mechanisms of barriers in the implementation of problem-based learning.

Medical Students' Perspective on Assessment Mechanism During Problem-Based Learning at Debre Tabor University: An Explanatory Mixed Study.

Assessment in problem-based learning should aim to improve students' active learning. In due course, significant student involvement in any assessment process may aid them in meeting the curriculum's objectives. The primary goal of this study is to assess medical students' attitudes towards the assessment method used during PBL tutorials at Debre Tabor College of Health Sciences. A mixed explanatory study design was used conducted at Debre Tabor university. For quantitative and qualitative data, cross-sectional survey and phenomenological study designs were used, respectively. A self-administered questionnaire with a 5-point Likert scale was used to collect quantitative data, while Focused Group Discussions (FGDs) were used to collect qualitative data. The current study included 192 out of 195 medical students. 40%, 57.2%, and 43.2% of study participants felt the tutor did not provide constructive feedback, facilitate self-assessment/self-reflection, or encourage peer assessment, respectively. On the role of tutors in facilitating self and peer assessment, a statistically significant mean difference in agreement is observed. Their PBL assessment did not take into account punctuality or contribution to the discussion. It is perceived as biassed due to tutors' bias towards various factors such as first impressions and student academic rank. They also stated that they did not receive sufficient information about the assessment in PBL. According to the findings of this study, medical students believed they were not fairly assessed during their PBL tutorial. Due to the uncertainty of the evaluation process, a neutral perspective on comprehension skills was appreciated. The students also perceived that the tutors' ability to assess students, poor feedback experience, and limited information about the assessment mechanism influenced their PBL assessment.

Gender transformative advertising pedagogy: promoting gender justice through marketing education

ABSTRACT We introduce a feminist pedagogical approach to marketing education through designing gender transformative advertising pedagogy, to promote diversity, ethics and gender justice in the classroom towards developing more inclusive industry practice and cultures. We detail how this was implemented through feminist praxis and a problem-based learning assessment task. Our analysis highlights that the feminist classroom raises consciousness of sexist advertising which can awaken students to unconscious gender biases. Further, we demonstrate how gender transformative advertising pedagogy encourages students to confront representational conventions of gender, with students aiming to portray diverse subjectivities yet struggling to transform femininity and relying on more ‘palatable’ forms of masculinity. Finally, we make the case for the marketing classroom as a site for gender justice, where through incremental action students come to link gendered structures with marketing practices and become equipped to act as agents of feminist change in and beyond the classroom.

Assessment of Problem-Based Learning (PBL): Poster Presentation as a Tool of Assessment

Assessment is considered the single most significant determinant of learning that influences student learning behavior broadly. However, assessment in the context of PBL requires a constructive alignment between PBL objectives and assessment, which can be achieved through an integrative assessment approach. This study aimed to assess the performance of groups of students participating in PBL for poster preparation and presentation on criteria-referenced. This type of research is descriptive research. The methods used in collecting data are observation and questionnaires. The instrument used in this research is a questionnaire. The techniques used to analyze the data are descriptive, qualitative, and quantitative. The study results were that half of the students scored less than 50% in designing posters on the "subject content of each poster of the courses covered in PBL. Two groups scored more than 9 (out of ten), 6 groups more than 8 (out of ten), and 2 groups of 7 (out of ten). Seven groups scored 4.25 or more (out of 5), and three groups were between 3 and 4 (out of 5) for verbal poster presentations. The substandard student group performance on poster design was excellent on "subject content" and outstanding for verbal poster presentations.

Assessment Modalities That Provoke Test Anxiety Among Medical Students

The prevalence of distress among medical students continues to rise. Studies have shown that stressrelated to examinations – test anxiety (TA) – is the most frequently reported source of distress. Research on the relationship between TA and assessment modalities is thus critical for determining the potential ramifications of this problem. This study aimed to explore assessment modalities that aggravate TA among medical students in a Malaysian medical school. A cross-sectional study was conducted among medical students at the School of Medical Sciences (SMS) of Universiti Sains Malaysia (USM). Students rated TA for each assessment modality used in the school. Each modality was scored from 1 to 10, with 1 indicating no TA and 10 representing extreme TA. Forty-five students participated in the study. The group was divided almost equally in terms of sex. The assessment modalities that provoked the most TA were the objective structured clinical examination (OSCE), the short case, the short essay question (SEQ) and the long case, with mean scores of 7.9, 7.8, 7.7 and 7.7, respectively. The case write-up, the problem-based learning (PBL) assessment, the multiple true-false (MTF) questions and the Simplified Thematic Engagement of Professionalism Scale (STEPS) were the assessment methods that induced the least TA, with mean scores of 5.1, 5.0, 4.4 and 4.0, respectively. This study found that the worst assessment modalities in terms of TA were the OSCE, the short and long cases, and the short essay question, while the case write-up, the PBL assessment, the MTF questions, and the STEPS induced the least TA. Most students reported that memorisation difficulties and facing examiners were the most common causes of TA. Remedial measures include examiner training on how to deal with examinees during assessments, evaluating the distribution of marks according to assessment modality and student training focused on study skills and exam preparation.

Problem-based learning as an assessment: Enhancing students’ connective learning and constructive learning

For the sake of enhancing students’ broad knowledge, problem-solving skills, and professional competences, this educational research project implemented problem-based learning (PBL) as a pedagogic tool in a large-class setting with 154 students involved at The Hong Kong Polytechnic University. The students participated in a problem-solving task that accounted for 25% of their course assessment (summative), which was designed with complex scenarios similar to real-life circumstances. The PBL assessment was structured in a way that required students' efforts to explore and connect relevant ideas supplemented with verifiable study materials, and a guided learning environment was provided by the course instructor and tutors. Learning activities were tailored to foster students’ active engagement and self-directed learning attitude towards resolving the PBL assessment. To evaluate the performance of the PBL approach, pre-and post-test questionnaire surveys were conducted to understand the improvement of students’ confidence regarding their learning ability, knowledge building, critical thinking, skill transfer, etc. The findings of these surveys revealed that the PBL assessment and related learning activities successfully stimulated students’ self-directed learning behaviour, enhanced their collaboration among peers, and expanded their capacity to develop critical skills for complex problem-solving tasks. Despite a challenging online learning environment compared to traditional classroom settings owing to the pandemic situation, the students considered this PBL assessment with guidance by the course instructor and tutors as creative, exciting, and very effective for life-long skill development. This research highlighted great prospects regarding implementing PBL approaches, even as an assessment component, in higher education institutions, which could be highly beneficial for nurturing competent graduates with essential practice-oriented skills and self-directed learning behaviour. Keywords: Higher Education, Scholarship of Teaching and Learning (SoTL), Learning Theories, Teaching Pedagogy, Constructivism, Connectivism.

Development of chemic media (chemistry comic) based on problem based learning on chemical bond materials for class x students

This research is a development research that aims to determine the feasibility of chemic media based on problem based learning on chemical bonding material for class X students based on BSNP standards. In this research, a modified ADDIE development model was used where only the analysis, design, development and evaluation stages were carried out. This research will be conducted at the Department of Chemistry State University of Medan, SMAN 1 Secanggang and SMAS Bintang Langkat. The feasibility of the media is seen through a feasibility questionnaire based on the BSNP standard with a Likert scale of 1 to 5. The validity of the chemic media is obtained from the results of the validation of lecturers and teachers, while the respondents' assessment of chemic media is obtained from the results of the assessments of teachers, students, and students. The results obtained indicate that the chemical media developed is valid or feasible to use. This can be seen from the validation data in terms of content, presentation feasibility, language feasibility, graphic feasibility and problem based learning assessment, respectively, obtaining a percentage of 80.83%; 82.50%; 94.38%; 82.19% and 82.14% so that an average percentage of 82.00% is obtained which is included in the very high category. The results of respondents' assessments of chemic media were reviewed based on aspects of appearance, material and benefits which respectively obtained a percentage of 88.96%; 86.73%; and 86.67% with an average percentage of 87.6% which is included in the very high category.

Assessment in the context of problem-based learning

Arguably, constructive alignment has been the major challenge for assessment in the context of problem-based learning (PBL). PBL focuses on promoting abilities such as clinical reasoning, team skills and metacognition. PBL also aims to foster self-directed learning and deep learning as opposed to rote learning. This has incentivized researchers in assessment to find possible solutions. Originally, these solutions were sought in developing the right instruments to measure these PBL-related skills. The search for these instruments has been accelerated by the emergence of competency-based education. With competency-based education assessment moved away from purely standardized testing, relying more heavily on professional judgment of complex skills. Valuable lessons have been learned that are directly relevant for assessment in PBL. Later, solutions were sought in the development of new assessment strategies, initially again with individual instruments such as progress testing, but later through a more holistic approach to the assessment program as a whole. Programmatic assessment is such an integral approach to assessment. It focuses on optimizing learning through assessment, while at the same gathering rich information that can be used for rigorous decision-making about learner progression. Programmatic assessment comes very close to achieving the desired constructive alignment with PBL, but its wide adoption—just like PBL—will take many years ahead of us.

The Role of Using Formative Assessments in Problem-based Learning: A Health Sciences Education Perspective

Practitioners in the field of pharmacy are often confronted with ill-structured problems. Specifically, pharmacists are tasked with making patient-specific recommendations that are both safe and effective, which requires combining knowledge from the biomedical, behavioral, and pharmaceutical sciences. Given the dynamic nature of pharmacy as a profession, the field has begun to explore learning strategies that go beyond mere content coverage to strategies that better support higher-order learning outcomes. One of these approaches is problem-based learning (PBL). While studies have focused on how to support PBL to improve learning outcomes, the role of assessment is often overlooked. Further exploration is thus needed since assessment plays a pivotal role in teaching and learning. This Voices paper will explore this idea within a larger context; we will also share the experience of how a subject matter expert (SME) worked with a team of instructional designers (IDs) to revise an existing course to more explicitly employ PBL and thus adopt an inquiry-based mindset needed for complex clinical decision making. Given the inherent challenges of assessment in PBL, further discussion will be focused on how to (a) design ill-structured problems, (b) align assessments to the PBL curriculum, and (c) how to hold students accountable in cases where a traditional grade is not attached.

Tutor assessment of PBL process: does tutor variability affect objectivity and reliability?

BackgroundEnsuring objectivity and maintaining reliability are necessary in order to consider any form of assessment valid. Evaluation of students in Problem-Based Learning (PBL) tutorials by the tutors has drawn the attention of critiques citing many challenges and limitations. The aim of this study was to determine the extent of tutor variability in assessing the PBL process in the Faculty of Medical Sciences, The University of the West Indies, St Augustine Campus, Trinidad and Tobago.MethodAll 181 students of year 3 MBBS were assigned randomly to 14 PBL groups. Out of 18 tutors, 12 had an opportunity to assess three groups: one assessed 2 groups and 4 tutors assessed one group each; at the end each group had been assessed three times by different tutors. The tutors used a PBL assessment rating scale of 12 different criteria on a six-point scale to assess each PBL Group. To test the stated hypotheses, independent t-test, one-way ANOVA followed by post-hoc Bonferroni test, Intra Class Correlation, and Pearson product moment correlations were performed.ResultThe analysis revealed significant differences between the highest- and lowest-rated groups (t-ratio = 12.64; p < 0.05) and between the most lenient and most stringent raters (t-ratio = 27.96; p < 0.05). ANOVA and post-hoc analysis for highest and lowest rated groups revealed that lenient- and stringent-raters significantly contribute (p < 0.01) in diluting the score in their respective category. The intra class correlations (ICC) among rating of different tutors for different groups showed low agreement among various ratings except three groups (Groups 6, 8 and 13) (r = 0.40). The correlation between tutors’ PBL experiences and their mean ratings was found to be moderately significant (r = 0.52; p > 0.05).ConclusionLeniency and stringency factors amongst raters affect objectivity and reliability to a great extent as is evident from the present study. Thus, more rigorous training in the areas of principles of assessment for the tutors are recommended. Moreover, putting that knowledge into practice to overcome the leniency and stringency factors is essential.

Aprendizaje basado en problemas: percepción del proceso enseñanza aprendizaje de las ciencias preclínicas por estudiantes de Kinesiología

ObjectiveTo determine the opinions of the Kinesiology student on problem-based learning (PBL) methodology in the Department of Preclinical Science of the University of La Frontera (Chile). Material and methodA qualitative research using an intrinsic case study. The sample consisted of 11 second-year Kinesiology students, who formed a focus group after giving informed consent. The data analysis was made using the Glaser Strauss constant comparative method and following the Huberman-Miles steps: data reduction, data display, and conclusion drawing/verification using the Atlas.ti software as a computer resource. ResultsA total of 1,097 significance units were identified and grouped under 40 categories, with 11 meta-categories emerging, and four qualitative domains were generated, called: ‘assessment of problem-based learning’, ‘work in mentoring’, ‘clinical case’ and ‘tutor role’. ConclusionStudents perceived PBL methodology as a facilitating and motivating learning strategy, helping them to build and integrate knowledge using r the clinical case study. The students also developed generic skills not achieved with the traditional methodology, such as teamwork, critical thinking, and learn to learn. For them feeling like a protagonist and managers of their own educational process is fundamental for meaningful learning. Furthermore, they pointed out conditions that can influence the development of classwork such as: the mentor, classmates, and the environment generated in the tutorial groups.

Changes to an Active Learning Curriculum in Osteopathic Medical Education: Effects on Exam Outcomes and Board Scores.

With the introduction of McMaster University's problem-based, self-directed learning and cognitive integration in the medical school curriculum, learning in small groups has been gaining popularity with medical schools worldwide. Problem-based learning (PBL) places emphasis on the value of basic medical sciences as the basis of learning medicine using clinical problems. For a successful outcome, a PBL curriculum needs to have a student-centered learning environment, problem-based design and facilitation, and assessment of learning in PBL domains. We describe a PBL program that has been used for undergraduate medical education, including changes made to learning resources and assessment. The changes required input from both faculty educators and students, and success depended on buy-in into the process. One of the changes included implementing the use of standard textbooks, which students use as the primary source of information during self-directed learning. Another change was the use of several reliable, valid, and cost-effective high-stakes written exams from internal and external sources, to promote spaced retrieval of biomedical facts and clinical contexts. By making these and other changes, we have been able to achieve pass rates and board scores which are consistently above the national average for 12years. We conclude that in order to ensure sustainable successful outcomes, it is important to keep our program dynamic by making improvements in the PBL domains and assessment methods, taking into consideration students' course evaluations of the learning environment.

Understanding the prospect of success in professional training: an ethnography into the assessment of problem-based learning

ABSTRACTOne of the most fundamental innovations in higher education is the introduction of the method known as problem-based learning (PBL). While literature has largely focused on its learning goals and the transition from lectures to tutorials, little research has problematised why this is a successful methodology and what we consider as students’ success. Drawing upon various ethnographic techniques, the authors analyse PBL as a field of expectations in light of Expectation-Value Theory. Beyond merely showing that PBL is a culturally constructed practice, the article elaborates on how students’ expectations inform practices, performance and evaluation, which is important for assessing the successfulness of the method. By discussing students’ expectations, dynamics and power relations, the present article is a contribution to the research addressing what has come to be known as the ‘black box’ of PBL.

Follow-up assessment of problem-based learning in dental alveolar surgery education: a pilot trial

Limited information exists on the effects of an oral and maxillofacial surgery integrated problem-based learning (PBL) course for dental undergraduates and the performance of these students after graduation. Therefore, we designed a PBL-implemented course with integrated dental alveolar surgery to evaluate its effects on the preparedness of graduates for clinical practice, their lifelong learning habits and their ability to collaborate, compared with the results of traditional courses. A randomised controlled trial was performed with 90 undergraduate students. The undergraduates were asked to perform a theoretical examination and to complete a clinical case analysis at the end of the course. Three years later, a follow-up survey was administered via a telephone interview and a questionnaire that measured self-perceived and supervisor-rated preparedness for clinical practice related to professional knowledge of dental alveolar surgery, lifelong learning habits, attitude and collaboration ability. All results were analysed using one-way analysis of variance (ANOVA) and a t-test. At the 3-year post-graduation time point, both PBL graduates and their supervisors rated their preparation for clinical practice as higher than those who received traditional lecture-based courses. In addition, the respondents in the PBL group believed that they were more likely than their counterparts in the traditional group to consult with professionals and other health-care staff members. No significant between-group differences were observed in the graduates' habits or lifelong learning attitudes. The PBL mode of teaching integrated dental alveolar surgery may improve preparedness for clinical practice and help undergraduates to develop the desire to collaborate.

Comparative Assessment of Problem-Based Learning and Traditional Teaching to Acquire Knowledge on Ventilator Associated Pneumonia

This research was conducted to find out the effect of problem-based learning (PBL) versus traditional teaching (TT) on knowledge regarding ventilator-associated pneumonia (VAP) among nursing students in selected Colleges of Nursing, West Bengal, India. Non-equivalent pre-test post-test design was adopted, and 83 students were selected by total enumeration technique from two colleges. After taking a pre-test, the lecture was conducted to TT group (42), and PBL was introduced to PBL group (41). Post-test was taken on the 8th day. The result of the study revealed that PBL was more effective to enhance knowledge rather than TT whereas TT was also effective. PBL was effective to enhance post-test knowledge score (μ=24.10) than TT group (μ=16.33). There is a significant association between pre-test knowledge score and exposure to learning resources in case of both PBL and TT group.

Construction of integrated organ-system oriented curriculum system of basic medical circulatory system in the practice of excellent doctor education training program

In order to carry out the training objectives of excellent doctor education training program implemented by the Ministry of Education and the Ministry of Health, an integrated organ-system oriented curriculum system of circulatory system was applied in five year clinical medicine education. The teaching contents of circulatory system from traditional Histology and Embryology, Physiology, Pathology, Pathophysiology, and Pharmacology were integrated and optimized to compile the integrated circulatory system textbook. problem-based learning(PBL)and comprehensive morphology experimental teaching were implemented into the integrated circulatory system curriculum. Diversified examination methods including continuous assessment of PBL were used to evaluate the learning effect of students. As a result, a new curriculum model of circulatory system teaching was constructed. And on the base of this curriculum model, an innovation course research experiment was performed, a digital network teaching platform with 4A(anybody, anything, anywhere, anytime)was set up, and diversified teaching methods were applied. This new developing curriculum system is helpful for students to relate what they have learned in the basic sciences to clinical medicine, gain clinical thinking competence, lifelong skills of self-directed learning, innovative thinking, and ability of scientific research and practice. Key words: Circulatory system; Organ-system oriented; Integrated curriculum; Curriculum system; Problem-based learning(PBL)

Concept mapping as a Tool for Learning and Assessment in Problem-based Learning

The knowledge structure of medical experts and successful learners is characterized by comprehensive integrated frameworks of related concepts. The development of this knowledge structure can facilitate clinical problem solving and other higher cognitive activities. Concept mapping is a learning tool which can depict the representation of knowledge structure of learners. Concept maps are built on learning theory concepts which complement problem-based learning (PBL). Students studying in PBL programs can construct concept maps centered on PBL cases and integrating the basic sciences and community concepts related to the case they study. Furthermore, concept maps can be used in PBL tutorial sessions for enhancing discussion, identifying learning gaps, generation of learning objectives, promoting learning integration and assessment of learning. Other uses of concept mapping in PBL could be in large group sessions in Seminars or team-based learning sessions to facilitate interactions and relevance of different concepts to the PBL case. However, challenges to concept mapping use in medical education include the development of tools with reliability and validity-evidence.