Two-tier Multiple-choice Items Research Articles

UPPER SECONDARY SCHOOL STUDENTS’ CONCEPTIONS OF CHEMICAL EQUILIBRIUM IN AQUEOUS SOLUTIONS: DEVELOPMENT AND VALIDATION OF A TWO-TIER DIAGNOSTIC INSTRUMENT

Student understanding of chemical equilibrium in aqueous solutions (CEAS) plays a vital role in their upper secondary school chemistry learning and everyday life. Diagnosis of students’ alternative conceptions (ACs) of the CEAS will provide teachers with valuable information to make instructional decisions on student learning. This study aims to develop and validate an instrument to diagnose students’ ACs about the CEAS, including ionization equilibrium, water self-ionization equilibrium, the equilibrium of salt hydrolysis, and precipitation and dissolution equilibrium. Using Treagust (1998)’s development framework, we have developed 25 two-tier multiple-choice items for the CEAS diagnostic test. After completing the corresponding courses, 750 Grade 11 students from five public schools responded to the CEAS diagnostic test. Rasch modeling approach was employed to provide psychometric properties of the CEAS diagnostic test consisting of one-dimensionality, reliability, and validity. This study identified 15 ACs toward the CEAS. This study found that most students performed better on concept tiers rather than reasoning tiers. In addition, students have difficulties in connecting acidity, solubility, ionization, and chemical reaction and in using mathematical thinking to do transformation between concentration, equilibrium constant, and pH value. Keywords: chemical equilibrium in aqueous solutions, alternative conceptions, two-tier multiple-choice items, Rasch modeling

Senior high school students’ understanding of Newton’s laws in outer space: Identification of misconceptions

One of the most frequently observed explanations in misconceptions is conceptual understanding. This research aimed to determine the senior high school students’ conceptual understanding of Newton’s laws in outer space. The researchers assumed that the research results would be rich in variation by elaborating the item questions based on the precise step of Digital Story Conceptual Change-Oriented as a learning medium. The researchers researched State Senior High School 5 Yogyakarta. The participants of this study consisted of 91 students. The applied instruments were fifteen two-tier multiple-choice items. The researchers also used a non-test instrument, namely an interview sheet. The researchers found related misconceptions in Newton’s law in outer space during the research. It was found that 30% of students experienced misconceptions. The results are important for the learning process that through identification, teachers could use strategy to explain the materials.

High School and Preservice Chemistry Teacher Education Students’ Understanding of Voltaic and Electrolytic Cell Concepts: Evidence of Consistent Learning Difficulties Across Years

High school students learn the basic voltaic and electrolytic cell concepts during their last year prior to entering an undergraduate teacher-education science degree. During the 4 years of university, students complete a sequence of topics designed to build on conceptual understanding presented in previous years. At the end of their degrees, graduating students are expected to have developed a comprehensive understanding of the subject that they are required to teach. In this research, we designed and developed a 12-item diagnostic instrument which addressed 10 propositional content knowledge statements based on the grade 12 chemistry curriculum that will be taught. In this cross-section study, 50 grade 12 high school students and 216 preservice chemistry teacher education undergraduates in years 1–4 responded to the Electrochemistry Conceptual Test (ECT) consisting of 12 two-tier multiple-choice items. The instrument was content validated by authors and peers prior to administration and when implemented had a Cronbach alpha reliability coefficient of 0.64. Overall, the students across years possessed basic knowledge about electrochemical cells but frequently were unable to explain their knowledge. While the grand mean trend in understanding electrochemistry concepts from high school through university study did show some improvement, the mean scores remained relatively low, and the year group means per item showed no such trend exacerbated by items having varying levels of difficulty. Based on this research, the lack in understanding about electrochemical concepts suggests that instruction in high school and ongoing university chemistry education faces ongoing challenges.

Analysis of Students’ Misconception and Strategic Thinking Skill on Parabolic Motion Material Using Two-Tier Multiple Choice Items Instrument

This study aims to develop an instrument test in the form of two-tier multiple-choice items that can be used to analyze students’ misconceptions and measure students’ strategic thinking skills. This research uses the research and development method with the ADDIE model. The resulting test instrument has ten questions that refer to indicators of strategic thinking skills. The results of the validation of the average score of 89.20% with very good criteria. The results of the Rasch Model analysis report that the test instrument that has been developed is declared feasible with acceptable criteria, all items that have been developed have reliability in the good category, and all items that have been developed are valid. The results of the Rasch Model analysis are also informed that in the implementation of the test, it is implausible that cheating, carelessness, and lucky phenomena will occur when doing the test. The results of the Rasch Model analysis also group students into three, namely high, medium, and low abilities, which convey information on students’ strategic thinking skills. The analysis results showed that 13.64% of students understood the overall concept of parabolic motion, 69.32% of students had misconceptions about the parabolic motion material, and 11.36% of students did not understand the parabolic motion material at all. The remaining 5.68% cannot be predicted with certainty whether the student understands, has misconceptions, or does not understand at all. This study concludes that the two-tier multiple choice items instrument developed is valid, reliable, and feasible to be used in analyzing students’ misconceptions and measuring students’ strategic thinking skills on parabolic motion material.

Generative Learning Strategy Assisted by Flash Animation to Remediate Students’ Misconceptions on Newton’s Law of Gravity

The generative learning strategy assisted by flash animation had not been applied to overcome learners’ misconceptions. This research aims to find out the effectiveness of generative learning strategy assisted by flash animation to remediate learners’ misconceptions toward the Newton’s law about gravity and to determine the way to decrease learners’ misconceptions after the intervention. This research applied the quasi-experimental method with pretest-posttest control group design. This research was conducted at Public Senior High School 5 Yogyakarta. The population consisted all ten graders, 216 participants. The sample was taken with purposive sampling from X-science-mathematics-2 learners as the control group and X-science-mathematics-4 learners as the experimental group. There were seventy-two learners. The applied instruments were two-tier multiple choice items and the learner-interview sheet, as the non-test instrument. From the research, the obtained N-Gain was 0.476. The hypothesis test showed that the t-test obtained the asymp.sig 2(tailed)of 0.000 and the probability score was 0.05. The asymp.sig (2-tailed) was lesser than the probability score and its effectiveness was 0.988. The decreased misconception after the intervention found in experimental group was 20% while the control group was 2.30%. It meant generative learning strategy assisted with flash animation was effective to remediate the tenth-graders’ misconceptions of Public Senior High School 5 Yogyakarta on Newton’s law about gravity

A Guided Inquiry laboratory to enhance students’ understanding of the Law of Mechanical Energy Conservation

This study investigated tenth grade Bhutanese students’ understanding of the law of mechanical energy
 conservation. A low-cost hands-on model developed using locally available materials designed as an integral part
 of a guided inquiry laboratory was used as intervention. A single-group pretest-posttest research design was
 employed. The instruments used to explore the students’ conceptual understanding and views and attitudes were
 13 two-tier multiple-choice items and 20 close-ended Likert-based items, respectively. The study also examined
 the students’ views and attitudes toward a guided inquiry laboratory and the effectiveness of a hands-on model
 through a semi-structured interview protocol. The data was analysed by calculating the mean, standard deviation,
 and t-test. The paired-sample t-test indicated a significant enhancement of students’ conceptual understanding
 due to the intervention of the guided inquiry laboratory from pretest (M=12.59, SD=3.28) to posttest [M=21.39,
 SD=3.07), t(99) = – 52.14, p

Impact of 21st-Century Collaboration on Pre-Service Science Teachers’ Understandings of Acid-Base Concepts in Selected Science Colleges of Education in Ghana

This study investigated the impact of 21st-century collaboration on pre-service teachers’ comprehension of acid-base concepts in selected science colleges of education in the Volta Region of Ghana. A pre-test/post-test quasi-experimental design with a non-equivalent control group was used in this study. The Acids-Bases Chemistry Achievement Test (ABCAT) was adapted from Damanhuri, Treagust, Won, and Chandrasegaran, (2016) to evaluate the extent to which pre-service science teachers in Ghanaian science colleges of education achieved the intended curriculum on acid-base concepts, specifically, concentration in mol/dm3 or g/dm3, properties of acids and bases, pH, pOH, neutralisation reaction and titrimetry. A sample of 52 second year pre-service science teachers from two intact science colleges of education was conveniently selected to participate in the study. An ABCAT comprising 19 items made up of 10 multiple-choice items and nine two-tier multiple-choice items was administered to pre-service science teachers as pre-test and post-test. Students taught with the 21st-century collaboration performed better than those taught using the Lecture method (LM) concerning acid-base conception when one-way between-group analysis of covariance (ANCOVA) and post hoc analysis with a Bonferroni adjustment was conducted on ABCAT. The results suggest that 21st-century collaboration facilitated the conceptual understandings of pre-service teachers in the experimental group.

High School Students’ Understanding of Acid-Base Concepts: An Ongoing Challenge for Teachers

Using a quantitative case study design, the Acids-Bases Chemistry Achievement Test (ABCAT) was developed to evaluate the extent to which students in Malaysian secondary schools achieved the intended curriculum on acid-base concepts. Responses were obtained from 260 Form 5 (Grade 11) students from five schools to initially create the two-tier multiple-choice items. After pilot testing, the final version of the ABCAT consisting of 19 items, 10 multiple-choice items and nine two-tier multiple-choice items, was administered to 304 students in Form 4 (Grade 10) from seven secondary schools when 12 alternative conceptions were identified by at least 10% of the students. Of these alternative conceptions, three were displayed by less than 15% of students. The two-tier multiple-choice items had a slightly higher internal consistency reliability (Cronbach’s alpha) of 0.54 than the multiple-choices items with a value of 0.42. The data from the study suggest that the ABCAT has shown the extent to which the teaching has reduced the incidence of students’ scientifically inappropriate understandings; for example, in nine of the 19 items, no alternative conceptions were displayed by the students.

Investigating High School Students’ Understanding of Chemical Equilibrium Concepts

This study investigated the year 12 students’ (N = 56) understanding of chemical equilibrium concepts after instruction using two conceptual tests, the Chemical Equilibrium Conceptual Test 1(CECT-1) consisting of nine two-tier multiple-choice items and the Chemical Equilibrium Conceptual Test 2(CECT-2) consisting of four structured questions. Both these tests were administered before and after the intervention. Students’ responses to the items in both the instruments indicated limited understanding of the various concepts related to chemical equilibrium. Less than 50% of the students provided correct responses to four of the nine items in the CECT-1. The total scores in the CECT-1 ranged from 0 to 8 with a mean score of 4.14 (out of a maximum of 9). In the CECT-2 the total scores ranged from 7 to 17 with a mean score of 11.0 out of a maximum score of 22. Almost half the number of students (44.6%) scored less than 50% of the total marks in the CECT-2; only 0% to 42.9% of students scored the maximum possible marks for each of the four items while achievement in all four items of the CECT-2 was below 50%. The findings will be valuable and assist teachers in planning their instruction on chemical equilibrium by taking into consideration students’ preconceptions about the topic.

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. But are the knowledge and reasoning involved in these tiers really distinguishable? Are the tiers equally challenging for students? The answers to these questions influence how we use and interpret TTMC instruments. We apply the Rasch measurement model on TTMC items to see if the items are distinguishable according to different traits (represented by the tier), or according to different content sub-topics within the instrument, or to both content and tier. Two TTMC data sets are analyzed: data from Singapore and Korea on the Light Propagation Diagnostic Instrument (LPDI), data from the United States on the Classroom Test of Scientific Reasoning (CTSR). Findings for LPDI show that tier-2 reasoning items are more difficult than tier-1 knowledge items, across content sub-topics. Findings for CTSR do not show a consistent pattern by tier or by content sub-topic. We conclude that TTMC items cannot be assumed to have a consistent pattern of difficulty by tier—and that assessment developers and users need to consider how the tiers operate when administering TTMC items and interpreting results. Researchers must check the tiers’ difficulties empirically during validation and use. Though findings from data in Asian contexts were more consistent, further study is needed to rule out differences between the LPDI and CTSR instruments.

Diagnostic assessment of student misconceptions about the particulate nature of matter from ontological perspective

Student conceptions related with matter and the particulate nature of matter (PNM), are vital for advanced understanding in chemistry, and have been a research area of significant attention. Lacking in the literature are studies addressing chemical misconceptions from an ontological point of view. The purpose of the current study was to develop a diagnostic instrument by utilizing ontological categories as theoretical lens and to evaluate student understanding on the topic PNM through assessment based on the relationship between ontology and misconceptions. Aligned with content in the middle school curricula, an assessment instrument of 25 distractor-driven, multiple-choice items, 15 of which were two-tier, was constructed. Subsequent to content validity work, to utilize student feedback for the improvement of the items and the validity of the inferences, we pilot tested our instrument with 178 students. Cross-sectional survey methodology was employed for the larger assessment work across Grades 6–11. Data were collected from a randomly selected sample of 696 students attending primary and secondary schools. The overall trend emerged to be improved student knowledge with increasing grade level. The students most often tended to attribute the properties of macroscopic matter to its microscopic particles and had most difficulty in explaining dissolution at the particle level, confirming previous national and international findings. An understanding of misconceptions from an ontology perspective may further assist educators to enhance a radical conceptual change. Distractor-driven, two-tier multiple-choice items are envisioned to aid stakeholders in curriculum development and implementation by illuminating student conceptions and ways of thinking.

Development and application of a two-tier multiple-choice diagnostic test for high school students’ understanding of cell division and reproduction

This study involved the development and application of a two-tier diagnostic test for measuring students’ understanding of cell division and reproduction. The instrument development procedure had three general steps: defining the content boundaries of the test, collecting information on students’ misconceptions, and instrument development. Misconception data were collected from interviews and multiple-choice questions with open response. The data were used to develop fourteen two-tier multiple-choice items. The conceptual knowledge examined was preliminary functions of cell division, basic processes in cell division, mitotic cell cycle and asexual reproduction, the role of meiosis in sexual life cycles, as well as cell types and cell division. The diagnostic instrument was administered to 403 high-school students. The reliability coefficient of the test was 0.81. Difficulty indices ranged from 0.31 to 0.72, and discrimination indices ranged from 0.23 to 0.61. Results of the cell division and reproduction diagnostic test suggested that students did not acquire a satisfactory understanding of cell division and reproduction concepts. The obtained misconceptions and the reasons for them were discussed in the analysis of the items which could inform biology instruction and resource.

Evaluation of an intervention instructional program to facilitate understanding of basic particle concepts among students enrolled in several levels of study

The efficacy of an intervention instructional program was evaluated to facilitate understanding of particle theory concepts among students (N = 190) using a diagnostic instrument consisting of eleven two-tier multiple-choice items in a pre-test – post-test design. The students involved were high school students, undergraduates and postgraduates from six educational levels. The eleven items evaluated understanding in three key conceptual categories: (1) intermolecular spacing in matter, (2) the influence of intermolecular forces on changes of state and (3) diffusion in liquids and gases. The intervention program proved effective in improving students' overall understanding of particle theory concepts, with statistically significant improvement in overall scores among students in five of the six groups. However, only 25-46% of the students displayed consistent understanding of the concepts in the three conceptual categories.

Development and Application of a Two-Tier Diagnostic Test for High School Students’ Understanding of Flowering Plant Growth and Development

This study involved the development and application of a two-tier diagnostic test measuring students’ understanding of flowering plant growth and development. The instrument development procedure had three general steps: defining the content boundaries of the test, collecting information on students’ misconceptions, and instrument development. Misconception data were collected from interviews and multiple-choice questions with open response answers. The data were used to develop 13 two-tier multiple-choice items. The conceptual knowledge examined was flowering plant life cycles, reproduction, precondition of germination, plant nutrition, and mechanism for growth and development. The diagnostic instrument was administered to 477 high school students. The correlation coefficient of test-retest was 0.75. Difficulty indices ranged from 0.24 to 0.82, and discrimination indices ranged from 0.32 to 0.65. Results of the Flowering Plant Growth and Development Diagnostic Test suggested that students did not acquire a satisfactory understanding of plant growth and development concepts. Nineteen misconceptions were identified through analysis of the items that could inform biology instruction and resource.