Progressive Test Research Articles

Exploration of medical students’ approach to progress test preparation

Introduction: Medical students are expected to develop competencies in applying clinical knowledge. Progress testing is an assessment for learning and is intended to assist in knowledge acquisition and to promote ongoing recall. This study explored student preparation for progress tests (PTs), relationships between approach and performance, and patterns that could assist with targeting learning support. Methods: A cross-sectional survey exploring study approach and learning context, comprising multi-choice and open-ended responses, was administered to students during their clinical years (n = 297). Quantitative data was analysed using descriptive statistics and comparative analysis, and qualitative data was categorised. Results: One hundred twenty-nine students responded (43.4%), with most demonstrating a stable PT performance over time. Twenty-two students had dynamic changes in their aggregate grades. Poor early PT performance was associated with developing an improvement strategy (χ2 6.954, p = 0.008). Students never falling below satisfactory were less likely to have a strategy (χ2 10.084, p = 0.001). All poorly performing students were using practice question banks, but this was not true for students who scored satisfactorily. The need for pastoral care was associated with poorer performance (χ2 4.701, p = 0.030). Conclusions: Student approaches to PT preparation are diverse, however there are correlations between preparation factors and performance outcomes. Students with poor early performance demonstrated improved results over time that may be attributed to targeted support and preparation strategies. Widespread use of practice questions may not be sufficient to impact results without additional strategies. External impacts to performance were common, and barriers to accessing pastoral care were evident. The feedback dashboard was underutilised, suggesting a need to improve feedback literacy and ensure this is fit for purpose.

Progress test as an assessment for learning approach in an Infectious Diseases Residency Program: a case study

Assessment is an essential component for all educational programs and must check competence acquirement while foster and promote learning. Progress Test (PT) is well recognized to assess cognitive knowledge, clinical reasoning and decision making in the clinical context, offering important information about the individual performance and program quality. It is widely used in Brazilian and international medical schools; however, it still has little role in assessing medical residents in Brazil. We present the experience of a PT pilot implementation in an Infectious Diseases residency program over two years. The first, second and third-year residents did four serial exams with 40 multiple choice questions (item)/each. Preceptors were trained on best practices on item writing. All the items were reviewed by a panel of experts and, after approval, included in the item bank. All participants answered a survey on their perceptions about the experience. The final score was higher for the third-year residents in all exam applications. The level of satisfaction was high among the participants, who mentioned the learning opportunity with the exam and the feedback. PT can improve residents’ assessment along the training period and residents’ performance should guide review and improvement of the programs.

Understanding students' feedback use in medical progress testing: A qualitative interview study.

Active engagement with feedback is crucial for feedback to be effective and improve students' learning and achievement. Medical students are provided feedback on their development in the progress test (PT), which has been implemented in various medical curricula, although its format, integration and feedback differ across institutions. Existing research on engagement with feedback in the context of PT is not sufficient to make a definitive judgement on what works and which barriers exist. Therefore, we conducted an interview study to explore students' feedback use in medical progress testing. All Dutch medical students participate in a national, curriculum-independent PT four times a year. This mandatory test, composed of multiple-choice questions, provides students with written feedback on their scores. Furthermore, an answer key is available to review their answers. Semi-structured interviews were conducted with 21 preclinical and clinical medical students who participated in the PT. Template analysis was performed on the qualitative data using a priori themes based on previous research on feedback use. Template analysis revealed that students faced challenges in crucial internal psychological processes that impact feedback use, including 'awareness', 'cognizance', 'agency' and 'volition'. Factors such as stakes, available time, feedback timing and feedback presentation contributed to these difficulties, ultimately hindering feedback use. Notably, feedback engagement was higher during clinical rotations, and students were interested in the feedback when seeking insights into their performance level and career perspectives. Our study enhanced the understanding of students' feedback utilisation in medical progress testing by identifying key processes and factors that impact feedback use. By recognising and addressing barriers in feedback use, we can improve both student and teacher feedback literacy, thereby transforming the PT into a more valuable learning tool.

Computer Adaptive vs. Non-adaptive Medical Progress Testing: Feasibility, Test Performance, and Student Experiences.

Computerized adaptive testing tailors test items to students' abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students' performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing. Therefore, we assessed the correlation between CA-PT and conventional PT test performance and explored the feasibility and student experiences of CA-PT in a large medical cohort. In this cross-over study medical students (n = 1432) of three Dutch medical schools participated in both a conventional PT and CA-PT. They were stratified to start with either a conventional PT or CA-PT to determine test performance. Student motivation, engagement and experiences were assessed by questionnaires in students from seven Dutch medical schools. Parallel-forms reliability was assessed using the Pearson correlation coefficient. A strong correlation was found (0.834) between conventional PT and CA-PT test performance. The CA-PT was administered without system performance issues and was completed in a median time of 83 minutes (67-102 minutes). Questionnaire response rate was 31.7% (526/1658). Despite a higher experienced difficulty, most students reported persistence, adequate task management and good focus during the CA-PT. CA-PT provides a reliable estimation of students' ability level in less time than a conventional non-adaptive PT and is feasible in students throughout the entire medical curriculum. Despite the strong correlation between PT scores, students found the CA-PT more challenging.

A Comparison of the Academic Achievement at the End of the Medicine Undergraduate Degree Program Between Students Who Only Used the University Admission Test and Those Who Used the University Admission Test Plus Marks from the High School National Exam (ENEM) at a Single Brazilian Center.

The role of marks in the University Admission Test (UAT) plus the marks from pre-university academic records in predicting academic achievement at the end of the Medicine undergraduate degree program is not completely known. This study was undertaken to compare the performance of marks in the UAT alone with those of the UAT plus marks from the National High School Exam (ENEM in Brazil) regarding students' outcomes at the end of the Medicine undergraduate degree program. Fifty-one (51) students from the last semester (12th) of our Medicine undergraduate degree program were included in the study. They were divided into a group of those who used the marks obtained in the UAT plus the marks obtained in the ENEM (ENEM group, n=9), and those who only used the marks in the UAT (non-ENEM group, n=42). We compared the academic achievement of the non-ENEM group with that of the ENEM group regarding the mean marks obtained in the clerkship, in the Progress Test (PT), and in the Objective Structured Clinical Examination (OSCE). The mean scores obtained in the disciplines of the clerkship were higher in the non-ENEM group compared to the ENEM group (7.32 ± 0.41 vs 6.98 ± 0.31, p= 0.01). Both groups obtained similar mean marks in the OSCE and in the PT. A moderate correlation was observed between the marks in the clerkship with those of the UAT from the non-ENEM group (p=0.00006; r=0.45). Marks of the UAT alone appear to be associated with a higher academic achievement in the clerkship than marks of the UAT plus scores obtained from the ENEM at the end of the Medicine undergraduate degree program.

Progress is impossible without change: implementing automatic item generation in medical knowledge progress testing

Progress tests (PT) are a popular type of longitudinal assessment used for evaluating clinical knowledge retention and long-life learning in health professions education. Most PTs consist of multiple-choice questions (MCQs) whose development is costly and time-consuming. Automatic Item Generation (AIG) generates test items through algorithms, promising to ease this burden. However, it remains unclear how AIG-items behave in formative assessment (FA) modalities such as PTs compared to manually written items. The purpose of this study was to compare the quality and validity of AIG-items versus manually written items. Responses to 126 (23 automatically generated) dichotomously scored single best-answer five-option MCQs retrieved from the 2021 University of Minho PT of medicine were analyzed. Procedures based on item response theory (IRT), dimensionality testing, item fit, reliability, differential item functioning (DIF) and distractor analysis were used. Qualitative assessment was conducted through expert review. Validity evidence of AIG-items was assessed by using hierarchical linear modeling (HLM). The PT proved to be a viable tool for assessing medical students cognitive competencies. AIG-items were parallel to manually written-items, presenting similar indices of difficulty and information. The proportion of functional distractors for both AIG and manually written items was similar. Evidence of validity for AIG-items was found while showing higher levels of item quality. AIG-items functioned as intended and were appropriate for evaluating medical students at various levels of the knowledge spectrum.

Relationship between medical programme progress test performance and surgical clinical attachment timing and performance

Purpose Progress tests (PTs) assess applied knowledge, promote knowledge integration, and facilitate retention. Clinical attachments catalyse learning through an appropriate learning context. The relationship between PT results and clinical attachment sequence and performance are under-explored. Aims: (1) Determine the effect of Year 4 general surgical attachment (GSA) completion and sequence on overall PT performance, and for surgically coded items; (2) Determine the association between PT results in the first 2 years and GSA assessment outcomes. Materials and methods All students enrolled in the medical programme, who started Year 2 between January 2013 and January 2016, were included; with follow up until December 2018. A linear mixed model was applied to study the effect of undertaking a GSA on subsequent PT results. Logistic regressions were used to explore the effect of past PT performance on the likelihood of a student receiving a distinction grade in the GSA. Results 965 students were included, representing 2191 PT items (363 surgical items). Sequenced exposure to the GSA in Year 4 was associated with increased performance on surgically coded PT items, but not overall performance on the PT, with the difference decreasing over the year. PT performance in Years 2–3 was associated with an increased likelihood of being awarded a GSA distinction grade (OR 1.62, p < 0.001), with overall PT performance a better predictor than performance on surgically coded items. Conclusions Exposure to a surgical attachment improves PT results in surgically coded PT items, although with a diminishing effect over time, implying clinical exposure may accelerate subject specific learning. Timing of the GSA did not influence end of year performance in the PT. There is some evidence that students who perform well on PTs in preclinical years are more likely to receive a distinction grade in a surgical attachment than those with lower PT scores.

Reflexões sobre a utilização do Teste de Progresso na avaliação programática do estudante

Abstract: Introduction: The Progress Test (PT) is a well-established and mostly successful modality of student knowledge assessment in the health professions, mainly those in the medical area, with the potential to contribute substantially to the formative and informative purposes (quality control and indication of improvement in the teaching-learning processes). Additionally, the PT has characteristics that are adequate for its inclusion in institutional evaluation systems that facilitate the formative purpose, such as programmatic assessment (PA), but that also meet the summative purpose. In schools that have defined actions aimed at introducing PA in their undergraduate courses, it is necessary to reflect on the strengths and limitations of using PT in the evaluation system. Development: based on the considerations of a working group representative of the entire institution, tasked with proposing means of introducing PA in a new curriculum for the medical course, with international advice with experience in both PT and PA, we generated a reflection on this topic, based on the authors’ experience and data from the literature. It is proposed that, within the longitudinal perspective of the PA, the PT constitutes one of the pillars in the assessment of knowledge. The PT can be used as a basis for monitoring the students, in the context of their class (cohort), and its results should be discussed with the mentors who accompanies and supports them. The PT must also play a central role in management, as a source of information for eventual review and qualification of the curriculum and its teaching-learning activities. It is predictable that the use of the PT in PA will bring different challenges and barriers, which will be more easily overcome if the institution has already consolidated experiences in the application of institutional exams and in faculty development for the production of good quality objective questions. Conclusion: the effectiveness of the PT within the institutional PA system will depend on measures aimed at increasing its effectiveness in the assessment and that encourage the student’s active participation, reflecting on their performance in the PT, with the support of their mentor, aiming to engage in actions that encourage learning self-regulation.

Assessment of medical students' Surgery knowledge based on Progress Test.

Progress Testing (PT) is an assessment tool whose use has grown throughout Brazil in the last decade. PT makes it possible to assess the students' knowledge gain throughout the undergraduate course and, for their interpretations to be valid, their items (questions) must have adequate quality from the point of view of content validity and reliability of results. In this study, we analyzed the psychometric characteristics of the items and the performance of students in the content area of surgery from 2017 to 2023. For the analyses, we used the assumptions of Classical Test Theory, Bloom's taxonomy and Cronbach's alpha reliability coefficient. The items were easy (average difficulty index between 0.3-0.4), with fair to good discrimination (discrimination index between 0.3-0.4) and with a predominance of medium to high taxonomy. Reliability remained substantial over the years (>0.6). Students' knowledge gain in surgery was found to be progressive and more important from the 3rd year of the undergraduate course, reaching approximately 70-75% in the 6th year. This measurements framework can be replicated in other contexts for a better understanding of student learning and for qualification of evaluation processes.

Effects of COVID-19 Pandemic on Progress Test Performance in German-Speaking Countries

Background. The COVID-19 pandemic has been the source of many challenges for medical students worldwide. The authors examined short-term effects on the knowledge gain of medical students in German-speaking countries. Methods. The development of the knowledge gain of medical students during the pandemic was measured by comparing the outcomes of shared questions within Berlin Progress Test (PT) pairs. The PT is a formative test of 200 multiple choice questions at the graduate level, which provides feedback to students on knowledge and knowledge gain during their course of study. It is provided to about 11,000 students in Germany and Austria around the beginning of each semester. We analyzed three successive test pairs: PT36-PT41 (both conducted before the pandemic), PT37-PT42 (PT37 took place before the pandemic; PT42 was conducted from April 2020 onwards), and PT38-PT43 (PT38 was administered before the pandemic; PT43 started in November 2020). The authors used mixed-effect regression models and compared the absolute variations in the percentage of correct answers per subject. Results. The most recent test of each PT pair showed a higher mean score compared to the previous test in the same pair (PT36-PT41 : 2.53 (95% CI: 1.31–3.75), PT37-PT42 : 3.72 (2.57–4.88), and PT38-PT43 : 5.66 (4.63–6.69)). Analogously, an increase in the share of correct answers was observed for most medical disciplines, with Epidemiology showing the most remarkable upsurge. Conclusions. Overall, PT performance improved during the pandemic, which we take as an indication that the sudden shift to online learning did not have a negative effect on the knowledge gain of students. We consider that these results may be helpful in advancing innovative approaches to medical education.

10 years of experience in adopting, implementing and evaluating progress testing for Saudi medical students

ObjectivesThe progress test (PT) is a comprehensive examination that is designed to match the knowledge acquisition necessary at graduation and monitors progress during the entire period of an undergraduate program. Qassim College of Medicine (QCM) began using the multi-institutional PT in the Kingdom of Saudi Arabia (KSA). This study aimed to determine if the PT can be utilized to assess the progress of medical students at different Saudi medical colleges with different educational approaches, as well as whether this testing modality could be accepted by other colleges. MethodsBeside the establishment of a PT committee, comprehensive blueprinting was crafted to sample 200 A-type multiple choice questions (MCQs) from different disciplines. The PT is a paper-and-pencil model and is answered in a 4-h period. All PT items followed a uniform design. ResultsIn total, 13 rounds of the progress test have been conducted. The number of participating colleges increased from three (with 285 students) in the first test (May 2012) to more than 20 (with >6000 students) in the ninth round (February 2017). The average % scores for first-year students ranged from 3.0% to 7.9% while the average scores for fifth-year students ranged from 34.0% to 43.0%. ConclusionThe conduction of this meticulously crafted test to evaluate knowledge achievement at medical graduation is a fruitful tool and helps to provide constructive feedback for test-takers and other stakeholders relating to their relative positions among other fellows at the national level.

Teste de Progresso: ausência de alunos de Medicina é sugestiva de mau desempenho acadêmico

Abstract: Introduction: It is not known whether the absence of medical students at the Progress Test (PT) is random event or if it due to some systematic characteristic of the students, which could influence the representativeness of the results obtained by the participants. Objectives: 1) to compare the academic performance indexes, in UFSC Medical School, of students who were present and absent from the PT in 2019; 2) to propose a way to estimate, based on these indexes, what the absentee’s grades would be if they had participated in the PT; 3) to identify factors associated with absence from the PT. Method: The averages of academic performance indexes, overall and in the different phases (semesters) in the groups of students who were present and absent from the PT, were compared using Student’s t test for independent samples. Using a linear regression technique, the probable PT scores were assigned to the group of absent students. Results: The global averages of the three academic indicators were significantly lower in students absent from the PT (p ranging from &lt; 0.03 to &lt; 0.0001); in 10 of the 11 analyzed course phases (semesters), the academic indicators of absentees were worse than those present at the test. The attribution of PT grades to the absentees allowed us to verify that there is a correlation (R=0.62) between the percentage of these students and the difference in grades between the groups that took and those that did not take the PT. Among male students, 25.8% did not attend the PT, while among female students the number of absentees was 16.6% (difference with p &lt;0.01). Conclusions: The absence of students at the PT does not occur randomly. Among the absentees, there is a systematic tendency to have students with worse academic performance. The use of multiple imputation of data demonstrate a correlation between the percentage of absentees and the difference in the average of grades in the PT of this group, compared to the average of the participants’ grades. The proportion of male students who missed the PT was significantly higher than that of female students.

University Admission Test Associates with Academic Performance at the End of Medical Course in a PBL Medical Hybrid Curriculum.

PurposeMost studies assessing the value of the university admissions test (UAT) to predict academic performance at the end of a medical course were carried out on lecture-based medical courses. However, the association between performance in the UAT with academic achievement at the end of medical course in a problem-based learning (PBL) medical hybrid curriculum remains controversial. The aim of this study was to correlate marks in the UAT with those obtained in the Organized Structured Clinical Examination (OSCE), in the progress testing (PT), and in the final marks of the clerkship (FMC).MethodsWe used data from 48 medical students. A single and a multiple dependency studies were performed to assess bivariate and multiple correlation between the UAT or the essay scores (dependent variables) and the OSCE, PT, and FMC (independent variables). Pearson test, multiple linear regression, and ANOVA tests were used and a p-value < 0.05 was considered significant.ResultsIn the bivariate analysis, only the UAT and FMC marks were correlated (r=0.34; p=0.02). However, the multiple dependency study showed a moderate correlation among UAT, OSCE, PT, and FMC marks (r=0.46; p=0.01). No correlation was found between the essay scores and PT, FMC, and OSCE scores.ConclusionOur study shows that UAT marks, but not essay scores, can predict academic achievement, particularly in terms of clinical competence (FMC) at the end of a medical course in a PBL hybrid curriculum.

Progress Testing and Module Final Scoring as part of Curriculum Evaluation in Faculty of Medicine Syarif Hidayatullah State Islamic University Jakarta, Indonesia

Introduction: Progress testing (PT) reflects students’ knowledge development and is a valuable indicator for curriculum evaluation. Since 2009, Faculty of Medicine Syarif Hidayatullah State Islamic University Jakarta (FMSH) has been conducting PT every year as a formative assessment. In 2012, curriculum reform was addressed to revise the 2005 curriculum; until then PT and curriculum evaluation were not conducted concurrently. This study aims to evaluate PT and assess whether there is a relationship between PT performance and final scores in modules, as part of curriculum evaluation.Method: It reviews PT for two cohorts: 571 students in 2015 and 562 students in 2016. 120 systembased topics were addressed in the PT. In this study the final scores for the old (2015) and new (2016) curriculum neuropsychiatry modules are reviewed, since their scores were lower than for other modules. Comparisons were made using ANOVA. Pearson correlations were calculated to examine the relationship between PT and final module scores.Results: This study revealed that PT scores between each grade (p &lt; 0.001) from 2015 to 2016 improved significantly (54.49 ± 7.43 and 55.07 ± 8.32; p &lt; 0.001). The mean of the final score of the new neuropsychiatry module was 69.36 ± 3.78 while the old one was 70.92 ± 3.99. Furthermore, Pearson correlation showed a weak correlation between final scores for the neuropsychiatry module and PT scores in 2015 (ρ = 0.191, p = 0.011).Discussion: PT scores increased significantly. Despite the final score of the new neuropsychiatry module being lower than the old one, there was heterogeneity in scores within the old neuropsychiatry module. The small number of neuropsychiatry items in the PT explains why the correlation between PT and final scores was weak. The weak correlation between final scores for the neuropsychiatry module and the PT scores in 2015; PT and final module scores seem reliable as indicators of curriculum evaluation. Further study is needed to analyze more cohort PT scores and modules.International Journal of Human and Health Sciences Vol. 05 No. 01 January’21 Page: 62-68

Developing an Item Bank for Progress Tests and Application of Computerized Adaptive Testing by Simulation in Medical Education

Progress Test (PT) is a form of assessment that simultaneously measures ability levels of all students in a certain educational program and their progress over time by providing them with same questions and repeating the process at regular intervals with parallel tests. Our objective was to generate an item bank for the PT and to examine the possible fit of CAT for PT application. This study is a descriptive study. 1206 medical students participated. During the analysis of the psychometric properties of PT item bank, “the Rasch model for dichotomous items was used”. Several CAT simulations were performed by applying various stopping rules of different standard errors. CAT simulation estimates were compared with the estimates generated from the original calibration of the Rasch model where all items were included. After Rasch analysis, a unidimensional PT item bank consisting of 103 items was obtained. The item bank reliability was calculated as 0.77 with Person Separation Index (PSI) and Kuder-Richardson Formula 20 (KR-20). A high correlation between θ estimations obtained from paper-and-pencil (θRM) and CAT applications (θCAT) was detected for simulation conditions ([N(0,1)] and [N(0,3)]) at the end of our analysis. In CAT, estimation can be made with an average of 14 questions (reduced 86,4%) and 17 questions (reduced 83,4%) [for N(0,1) and [N(0,3) respectively] with reliability of 0,75. This study reveals that it is possible to develop an appropriate item bank for the PT, and the difficulty of administering large number of items in PT can be scaled down by incorporating CAT application.

Formative assessment scores in tutorial sessions correlates with OSCE and progress testing scores in a PBL medical curriculum

ABSTRACTBackground: Effective assessments programs are a challenge in problem-based learning (PBL). One of the main principles of this educational setting is the Formative Assessment (FA). We hypothesized that students’ performance assessed by FA in tutorial sessions in a PBL curriculum is related to other summative assessments.Objective: To investigate the correlation among FA in tutorial sessions with grades obtained in Objective Structured Clinical Evaluation (OSCE) and Progress Testing (PT) to better understand the assessment process in PBL medical teaching approach and to predict student’s future performance.Design: An observational cross-sectional study was conducted comparing FA, OSCE and PT scores from 4th to 8th semester medical students. Correlation analyses were performed using pooled and separate data from the 4th and 8th semesters.Results: From the 5th to 8th semester, OSCE scores were smaller compared to the FA, while PT scores were lower in all stages. In the pooled data, the correlation analysis showed a significant positive relationship between grades on FA and OSCE, FA and PT and OSCE and PT. A significant correlation among the three assessments strategies was also detected in the 8th semester, but not in the 4th semester.Conclusions: Assessment strategies in PBL approach, including FA, OSCE and PT, have positive correlations, which increases as the medical course becomes more complex.

Toward a better judgment of item relevance in progress testing

BackgroundItems must be relevant to ensure item quality and test validity. Since “item relevance” has not been operationalized yet, we developed a rubric to define it. This study explores the influence of this rubric on the assessment of item relevance and on inter-rater agreement.MethodsMembers of the item review committee (RC) and students, teachers, and alumni (STA) reassessed the relevance of 50 previously used progress test (PT) items and decided about their inclusion using a 5-criteria rubric. Data were analyzed at item level using paired samples t-tests, Intraclass Correlation Coefficients (ICC), and linear regression analysis, and at rater level in a generalizability analysis per group.ResultsThe proportion of items that the RC judged relevant enough to be included decreased substantially from 1.00 to 0.72 (p < 0.001). Agreement between the RC and STA was high, with an ICC of >0.7 across items. The relation between inclusion and relevance was strong (correlation = 0.89, p < 0.001), and did not differ between RC and STA. To achieve an acceptable inter-rater reliability for relevance and inclusion, 6 members must serve on the RC.ConclusionsUse of the rubric results in a stricter evaluation of items’ appropriateness for inclusion in the PT and facilitates agreement between the RC and other stakeholders. Hence, it may help increase the acceptability and validity of the PT.

Applicability of Progress Testing in Veterinary Medical Education.

A substantial part of graduate education in veterinary medicine is spent in clinical practice. During the clinical experiential phase, it is difficult to monitor students' actual knowledge development: they build individual records of experiences based on the cases they have to deal with, while mainly focusing on knowledge that is of direct, clinical relevance to them. As a result, students' knowledge bases may differ to such a degree that a single test alone may not be able to provide an adequate reflection of progress made. In these circumstances, progress testing, which is a method of longitudinal assessment independent of the curricular structure, may offer a viable solution. The purpose of this study, therefore, was to determine the extent to which progress tests (PT) can be used to monitor progress in knowledge development at a graduate level in veterinary medical education. With a 6-month interval, we administered two tests to students based on the Maastricht Progress Test format that covered a large variety of veterinary topics. Consequently, we analyzed students' progress in knowledge development. Based on a substantive appraisal of the questions and analysis of the test results, we concluded that the tests met the measurement criteria. They appeared sensitive enough to gauge the progress made and were appreciated by the students. Hence, in spite of the differences within the whole graduate group, the PT format can be used to monitor students' knowledge development.

English

Progress Test (PrT) is a longitudinal assessment strategy in which tests composed of contents from all the curriculum are periodically applied to all students of a course. Such strategy allows measurement of deep, long-lasting meaningful learning as well as early detection and remediation of underperforming students. It was introduced more than forty years ago, and it has been used in several health schools in the world, especially medicine. Assessment of students’ knowledge gain and its application over the course is a challenge. PrT has become a relevant method to monitor the development of the student through graduation. There is no culture of using longitudinal assessments in Brazil's pharmacyschools. This scenario is an opportunity for use of PrT. The objective of this work is to make a literature review about how progress test has been used in context of education. The authors also discuss the possibilities of PrT’s application in Pharmacy undergraduate courses in Brazil. PrT has a long history of use by various institutions in the world. Most user experiences come from medical schools, but there are articles showing the application of PrT in dentistry and psychology schools. PrT has been shown to be an effective assessment tool in a problem-based (PBL) and traditional curricula. PrT is recommended as a tool to longitudinal assesses growth in knowledge. Phamacy schools may develop their own framework for PrT collaborations, which could optimize the educational utility of student’s assessment instruments as tools to enhance learning. Key words: Progress test, assessment, knowledge.

Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment

Background: Progress tests (PT) are used to assess students on topics from all medical disciplines. Progress testing is usually one of the assessment methods of the cognitive domain. There is limited knowledge on how positioning of the PT in a program of assessment (PoA) influences students’ PT scores, use of PT feedback and perceived learning value.Methods: We compared PT total scores and use of a PT test feedback (ProF) system in two medical courses, where the PT is either used as a summative assessment or embedded in a comprehensive PoA and used formatively. In addition, an interview study was used to explore the students’ perception on use of PT feedback and perceived learning value.Results: PT total scores were higher, with considerable effect sizes (ESs) and students made more use of ProF when the PT was embedded in a comprehensive PoA. Analysis of feedback in the portfolio stimulated students to look for patterns in PT results, link the PT to other assessment results, follow-up on learning objectives, and integrate the PT in their learning for the entire PoA.Conclusions: Embedding the PT in an assessment program designed according to the principles of programmatic assessment positively affects PT total scores, use of PT feedback, and perceived learning value.