Reason Tier Research Articles

Development and Validation of a Four-Tier Test for the Assessment of Secondary School Students' Conceptual Understanding of Amino Acids, Proteins, and Enzymes.

Previous studies noted the scantiness of diagnostic instruments for the assessment of students’ understanding of fundamental biochemistry concepts. Consequently, within this study, a four-tier test for the examination of secondary school students’ conceptual understanding of amino acids, proteins, and enzymes has been developed. Items in the test consist of the answer tier, the reason tier, and a confidence rating for each of these tiers. Following the three-phase development process, the final version of the test, comprising eight items, was administered to 123 students in the main study. The internal consistency of both cognitive scores and confidence ratings proved to be adequate (Cronbach’s alpha values were 0.76 and 0.87, respectively) and the test–retest reliability was also satisfactory (Pearson’s r value was 0.74 for the cognitive scores and 0.88 for the confidence ratings). If the correct responses to both the answer and the reason tier of items are considered, the mean score on the test was 3 out of 8 (38%). The students were more confident when producing the correct compared to the wrong answers but, overall, their confidence was not high (3.65 out of 6). Furthermore, the confidence in the incorrect answers that were provided (3.52 out of 6) indicates the presence of alternative conceptions. Such results show that the content regarding amino acids, proteins, and enzymes is conceptually challenging for the students. Therefore, the test can help secondary school teachers to uncover the students’ conceptual difficulties related to this content and develop effective strategies for their remediation and prevention.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11165-022-10075-5.

Investigating the differences between confidence ratings in the answer and reason tiers in fourth graders via online four-tier test

In this study, the author enrolled 982 fourth graders in Taiwan to investigate the differences (a) in confidence ratings between the answer and reason tier and (b) between students who provided correct and incorrect responses. This study administered a four-tier number sense test comprising an answer, confidence, reason, and confidence tier (the first, second, third, and fourth tier, respectively). The students had significantly higher confidence ratings on the reason tier than on the answer tier both for each number sense component and on the entire test. Approximately 87 % of the students selected equal confidence ratings in both tiers, and 8% had higher confidence ratings in the reason tier than in the answer tier. The results indicate that the students had significant misconceptions in 4 out of 32 questions in the answer tier and 11 out of 32 questions in the reason tier. These results have considerable implications for researchers and educators.

An investigation of students’ conceptual understanding levels on fluid dynamics using four-tier test

Student repeatedly have difficulties in understanding complex physics concepts, such as fluid dynamic concept. It is occurred because the students’ conceptual understanding is low, thus indicating a low student mental model. The study aims to describe the students’ conceptual understanding levels on fluid dynamic using four-tier diagnostic test. A four-tier diagnostic test is a multiple-choice diagnostic test that have four tiers consist of answer tier, confidence ratings for the answer tier, and reason tier. The instrument test utilized to describe students’ conceptual understanding level regarding to the mental models. This research uses a descriptive-quantitative method. The data were analyzed using the four-tier diagnostic test rubrics that were used in related literature to analysis students’ conceptual understanding levels. The subject of this study is 24 students in the eleventh grade (16-17 age in average) at high school in Bandung. The results show that students’ conceptual understanding levels mostly in sound understanding/SU (35,27%), partial understanding/PU (42,91%), alternative understanding/AC (17,82%) and no understanding/NU (4%), Based on the result, four-tier test is able to use for investigating conceptual understanding levels.

Development and Validation of a Four-Tier Diagnostic Instrument for Chemical Kinetics (FTDICK)

The present study outlines the development and validation of a four-tier diagnostic instrument to explore first year undergraduate students’ understanding of chemical kinetics (FTDICK). The four-tier instrument is a recent format and has been applied only sparsely in a limited number of subject areas, not including chemical kinetics. This study confirms the importance of a four-tier approach in fully investigating students’ poor knowledge and understanding. The FTDICK described here involves 20 questions with a confidence level linked to both the question tier and the reason tier. The development of the instrument followed the procedure used for the two-tier instrument developed by Treagust and involves (1) testing & interviewing, (2) identifying & collecting students’ unscientific ideas, (3) developing the prototype FTDICK, (4) validating the prototype FTDICK, and (5) developing the final FTDICK. The initial steps revealed a number of areas of misconceptions to be explored in the final instrument. The instrument has been developed and the prototype tested using international cohorts of students from the University of Reading, UK and two Indonesian Universities. Comprehensive item analysis on the results showed the instrument to be valid and reliable and suitable for identifying students’ understanding of chemical kinetics. This study confirms the importance of a four-tier approach for investigating students’ prior knowledge and understanding.

Development of a three-tier number sense test for fifth-grade students

To assess the strength of conceptual understanding of number sense, a three-tier number sense test (TTNST) for fifth-grade students was developed and validated in this study. The three-tier test includes a content tier that assesses content knowledge of number sense, a reason tier that assesses a reason given for selecting a first-tier response, and a confidence tier that assesses how confident fifth-grade students are in their responses in the first two tiers. A total of 819 fifth-grade students from elementary schools in Taiwan participated in this study. Collected data showed that the test had good reliability and validity. The results revealed that many of the sample students performed poorly on number sense but maintained extremely high confidence; this indicated that many of the students held severe misconceptions and some were lacking in number sense. In addition, this study confirmed that the inclusion of a third tier (with confidence ratings) in the number sense test can mitigate the weaknesses of a previously used two-tier test. The educational implications of the findings are discussed in this paper.

Performance of sixth graders in Hong Kong on a number sense three-tier test

ABSTRACT This study examined the performance of sixth graders in Hong Kong on a number sense three-tier test (NSTTT) and identified their possible misconceptions about number sense. The NSTTT comprises a content tier, which assesses the content knowledge of number sense; a reason tier, which assesses the reasons for the first-tier responses; and a confidence tier, which assesses students’ confidence in their responses to questions in the first two tiers. The study participants were 942 sixth graders in Hong Kong. The results indicated that the percentage of correct answers (and confidence indices) of the first two tiers (i.e. the content tier and reason tier) of the sixth graders was 53.96% (4.27). In addition, the sixth graders showed the highest performance on “ability to recognise number size” and the lowest on “ability to judge the reasonableness of computational results”. Moreover, the results revealed that some students showed a low performance on number sense but demonstrated extremely high confidence levels, indicating that they may have significant misconceptions regarding number sense and may lack of number sense. Furthermore, this paper discusses the educational implications of the findings.

Development and Evaluation of a Three-Tier Diagnostic Test to Assess Undergraduate Primary Teachers’ Understanding of Ecological Footprint

This study focused on the development and validation of a three-tier multiple-choice diagnostic instrument about the ecological footprint. Each question in the three-tier test comprised by; (a) the content tier, assessing content knowledge; (b) the reason tier, assessing explanatory knowledge; and (c) the confidence tier that differentiates lack of knowledge from misconception through the use of a certainty response index. Based on the literature, the propositional knowledge statements and the identified misconceptions of 97 student-teachers, a first version of the test was developed and subsequently administered to another group of 219 student-teachers from Primary and Early Childhood Education Departments. Due to the complexity of the ecological footprint concept, and that it is a newly introduced concept, unknown to the public, both groups have been previously exposed to relevant instruction. Experts in the field established face and content validity. The reliability, in terms of Cronbach’s alpha, was found adequate (α = 0.839), and the test-retest reliability, as indicated by Pearson r, was also satisfactory (0.554). The mean performance of the students was 56.24% in total score, 59.75% in content tiers and 48.05% in reason tiers. A variety of concepts about the ecological footprint were also observed. The test can help educators to understand the alternative views that students hold about the ecological footprint concept and assist them in developing the concept through appropriately designed teaching methods and materials.

Development and Evaluation of the Diagnostic Power for a Computer-Based Two-Tier Assessment

This study adopted a quasi-experimental design with follow-up interview to develop a computer-based two-tier assessment (CBA) regarding the science topic of electric circuits and to evaluate the diagnostic power of the assessment. Three assessment formats (i.e., paper-and-pencil, static computer-based, and dynamic computer-based tests) using two-tier items were conducted on Grade 4 (n = 90) and Grade 5 (n = 86) students, respectively. One-way ANCOVA was conducted to investigate whether the different assessment formats affected these students’ posttest scores on both the phenomenon and reason tiers, and confidence rating for an answer was assessed to diagnose the nature of students’ responses (i.e., scientific answer, guessing, alternative conceptions, or knowledge deficiency). Follow-up interview was adopted to explore whether and how the various CBA representations influenced both graders’ responses. Results showed that the CBA, in particular the dynamic representation format, allowed students who lacked prior knowledge (Grade 4) to easily understand the question stems. The various CBA representations also potentially encouraged students who already had learning experience (Grade 5) to enhance the metacognitive judgment of their responses. Therefore, CBA could reduce students’ use of test-taking strategies and provide better diagnostic power for a two-tier instrument than the traditional paper-based version.

Assessing 10- to 11-year-old children’s performance and misconceptions in number sense using a four-tier diagnostic test

Background: Number sense is a key topic in mathematics education, and the identification of children’s misconceptions about number is, therefore, important. Information about students’ serious misconceptions can be quite significant for teachers, allowing them to change their teaching plans to help children overcome these misconceptions. In science education, interest in children’s alternative conceptions has led to the development of three- and four-tier tests that not only assess children’s understandings and misconceptions, but also examine children’s confidence in their responses. However, there are few such tests related to mathematical content, especially in studies of number sense.Purpose: The purpose of this study was to investigate children’s performance and misconceptions with respect to number sense via a four-tier diagnostic test (Answer Tier → Confidence rating for Answer Tier → Reason Tier → Confidence rating for Reason Tier).Design and method: A total of 195 fifth graders (10–11 years old) from Taiwan participated in this study. The four-tier test was web-based and contained 40 items across five components of number sense.Findings: The results show that (1) students’ mean confidence rating for the answer tier was significantly higher than for the reason tier; (2) an average of 68% of students tended to have equal confidence ratings in both answer and reason tiers; (3) students who chose correct answers or reasons had higher mean confidence ratings in most items (36 out of 40) than those who did not; and (4) 16 misconceptions were identified and most of them were at a strong level.Conclusion: The four-tier test was able to identify several misconceptions in both the answer and reason tier and provide information about the confidence levels. By using such information, teachers may be better positioned to understand the nature of learners’ misconceptions about number sense and therefore support their pupils’ progress in mathematics.

Exploring students’ understanding of electrochemical cells using an enhanced two-tier diagnostic instrument

Background:The development of two-tier multiple-choice questions has permitted the diagnosis of students’ understanding on various topics in the sciences as well as helped to ascertain the alternative conceptions they have. A limitation of two-tier diagnostic instruments that has been noted in the literature, but which has not been systematically explored, is that some questions can have a diagonal response. This can allow students to select a second-tier response based on cues from a particular response in the first tier, and it can limit the analytical utility of the diagnostic instrument to some extent.Purpose:The study explores an enhancement of the traditional two-tier diagnostic instrument to address diagonal response. In the format proposed, there are dual sections for both the answer and reason tiers. The answer tier is in dichotomous form and for each of the dichotomous responses, there are separate polytomous responses in the reason tier.Sample:There were 99 students of grade 10 level who participated in this study.Design and Method:The research design follows the usual approach adopted for the development of diagnostic instruments but with some modifications. The topic of electrochemical cells was chosen to evaluate the effectiveness of the enhanced format of the two-tier instrument.Results:It was found that with the enhanced format, the issue of diagonal response has been satisfactorily addressed. Also, a situation where students get the answer wrong but the reason correct, which is sometimes encountered in two-tier questions, does not arise with this format. The instrument has also been able to shed some light on the students’ understanding of electrochemical cells and their alternative conceptions.Conclusions:The enhanced format of the two-tier instrument proposed in this study can also be used for the study of alternative conceptions in science education.

Exploring Undergraduates’ Understanding of Transition Metals Chemistry with the use of Cognitive and Confidence Measures

Compared to studies on school students’ understanding of various topics in the sciences, studies involving university students have received relatively less attention in the science education literature. In this study, we investigated university students’ understanding of transition metals chemistry, a topic in inorganic chemistry, which has been only scarcely explored in the science education literature. A four-tier diagnostic instrument was used. The instrument comprises 25 questions, and each question has an answer tier, a confidence rating for this tier, a reason tier and a confidence rating for this tier. Versions of the instrument were refined iteratively during the preliminary and pilot phases of the study. This study reports on the results obtained from the main phase of the study, using a sample of 140 students. Overall, the diagnostic test was difficult for the students. The students had a mean score of 38 %, based on correct responses for both answer and reason tiers for the questions. It was accompanied by a mean confidence of only 3.49 out of 6 (that is, 58.2 %) for the whole test. The results indicate that transition metals chemistry is a difficult topic for the students. Twenty-four alternative conceptions have been identified in this study, including some indication of their strengths. Some implications of the study are discussed.

University Students’ Understanding of Chemical Thermodynamics

This study explored undergraduate students’ understanding of the chemistry topic of thermodynamics using a 4-tier diagnostic instrument, comprising 30 questions, and follow-up interviews. An additional objective of the study was to assess the utility of the 4-tier instrument for use in studies on alternative conceptions (ACs) as there has been no study done on it since its introduction in the literature in the year 2010. A total of 296 students majoring in Chemistry at a university in Singapore participated in this study—88 students in the preliminary study, 102 students in the pilot study and 106 students in the main study. This article reports on the results obtained with students in the main study; their age ranges from 20 to 22 years. Comprising answer and reason tiers plus associated confidence ratings, the 4-tier diagnostic instrument enabled the eliciting of 34 ACs harbored by the undergraduates as well as the strengths of these ACs. Of concern to note is that even for questions which were answered correctly, the mean confidence was not very high. The results of this study reiterate the point that thermodynamics is a topic fraught with conceptual difficulties and ACs. Based on the results from this study, the potential of the 4-tier test for AC studies is further underscored. Some implications of the study are discussed.

Development and Application of a Three‐Tier Diagnostic Test to Assess Secondary Students’ Understanding of Waves

This study focused on the development and application of a three‐tier multiple‐choice diagnostic test (or three‐tier test) on the nature and propagation of waves. A question in a three‐tier test comprises the content tier, which measures content knowledge; the reason tier, which measures explanatory knowledge; and the confidence tier, which measures the strength of conceptual understanding of the respondents. This paper presents results based on the responses of 243 Grade 10 students after they were formally instructed on the topic. The vast majority of the respondents showed an inadequate grasp of concepts about waves. Eleven alternative conceptions (ACs), which were expressed with confidence by more than 10% of the students, were identified; four of these ACs were expressed with high confidence.

Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students’ content knowledge and explanatory knowledge, respectively. The two additional tiers measure the level of confidence of students in the correctness of their chosen options for the answer and reason tiers respectively. The 4TMC diagnostic test focused on the properties and propagation of mechanical waves. It was administered to 598 upper secondary students after they were formally instructed on the foregoing topics. The vast majority of the respondents were found to have an inadequate grasp of the topics tested. Mean scores and mean confidence associated with the answer tier was higher than those associated with the reason tier. The students tended to be poorly discriminating between what they know and what they do not know. Familiarity with the topic tested was associated with greater percentage of students giving correct answers, higher confidence, and better discrimination quotient. Nine genuine alternative conceptions (which were expressed with moderate levels of confidence by students) were identified.