Test Items Research Articles

Training to Analyze Learning Result Test Items for High School Teachers

Training to Analyze Learning Result Test Items for High School Teachers

Analyses of Pre-Service Teachers’ Errors in Solving Algebraic Tasks in Ghana.

This study analyzed prospective basic education teachers’ errors in solving algebraic tasks using Newman’s Error Analysis (NEA). The study employed an interpretivist philosophical paradigm with a qualitative case study design. Data were collected from 250 prospective mathematics teachers randomly selected from four colleges of education in the eastern region of Ghana using super item tests and unstructured interview protocols. Thematic analysis was employed to analyze the qualitative data. The study findings revealed that transformation and encoding errors among prospective mathematics teachers are the most prevalent errors. Other error types encountered included comprehension errors and process skills errors. In order to reduce aspiring mathematics teachers’ errors, the study emphasizes the necessity of focused interventions and professional development programs to address transformation and comprehension errors in algebraic concepts.

12481 Evaluating The Device Handling And Preference Assessment Questionnaire For Growth Hormone Deficiency: Results From Content Validation Interviews

Abstract Disclosure: J. Neergaard: Employee; Self; Novo Nordisk. S. Akhtar: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. B. Berg: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. S. Hamilton: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. E. Hildebrand: Consulting Fee; Self; Novo Nordisk. J. Medina: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. G. Ter-Borch: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. N. Rasmussen: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. Background: Patient experiences with injection devices may affect treatment adherence, and hence it is important to ensure patient-reported outcomes (PRO) instruments used to assess experience are supported by evidence of validity. As part of a study that compared patient preference and ease of use between the devices used to administer Somapacitan (Sogroya®, Novo Nordisk A/S) and Somatrogon (Ngenla®, Pfizer), we assessed relevance, comprehensiveness and comprehension of a PRO instrument, the Device Handling and Preference Assessment Questionnaire (DHPAQ), in adolescent patients with growth hormone deficiency (GHD) and their caregivers. Methods: We recruited 33 adolescent patients with GHD and 37 caregivers across five sites in the US for an open label, randomized, crossover study in which participants simulated an injection with each device using an injection pad or mannequin. The DHPAQ was used to assess patients’ device preference. Two-sided statistical tests with 5% significance levels were performed using DHPAQ responses. After completing the DHPAQ, a random subgroup of 10 adolescents and 10 caregivers were offered additional compensation to participate in a cognitive interview (CI). The CIs were conducted in accordance with the US Food and Drug Administration 2022 Principles for PRO Instruments and 2009 PRO Guidance for Industry. Trained interviewers used a structured guide to evaluate questions, statements, tasks and response options in the DHPAQ. Participants’ interview responses for each question were coded and added to summary tables. These summary tables were evaluated by the study team to determine the relevance, comprehensiveness and participant comprehension of instructions, item stems and response options. Results: When asked if any questions in the DHPAQ did not seem relevant to their device experience, all 20 CI participants answered “no”. Questions on “learning how to use the device”, “selecting the correct dose” and “injecting the dose” were ranked as most relevant. Participants responses from the CIs supported perceived comprehensiveness of all DHPAQ items. All CI participants reported comprehension of all test items in the DHPAQ, and only three items did not receive a “no difficulty” response from all 20 participants. In the main study, the number of participants who preferred the somapacitan device was significantly greater than those who preferred the somatrogon device (84.3% vs 12.9%, p<0.0001). More participants found the somapacitan device easy or very easy to use compared with the somatrogon device (98.6% vs 74.3%). Conclusions: The results from the CIs indicated that the instructions, item stems and response options of the DHPAQ were relevant, comprehensive and fully comprehended by all participants. In the main study, patients with GHD and their caregivers were found to prefer the somapacitan device to the somatrogon device. Presentation: 6/2/2024

Effect of reverberation chamber volume on result of qualification procedure for loudspeaker and loudspeaker positions for airborne sound insulation measurements

Measurement of sound insulation from airborne sound is carried out by determining the level of sound power radiated by the test item. In laboratory practice, this translates into measuring the sound level in the transmitting and receiving chambers and measuring the reverberation time in the receiving chamber. When performing these measurements, significant fluctuations in the sound level for low frequencies can be observed, which is related to the occurrence of standing waves inside the chambers. To ensure proper level averaging, as well as repeatability and reproducibility of measurements ISO 10140-5 describes a qualification procedure for sources and source positions. This article presents the effect of increasing the volume of the source chamber on the results of the qualification procedure for sources and source positions for floor airborne sound insulation measurements.

Information accumulation on the item versus source test of source monitoring: Insights from diffusion modeling.

Source monitoring involves attributing previous experiences (e.g., studied words as items) to their origins (e.g., screen positions as sources). The present study aimed toward a better understanding of temporal aspects of item and source processing. Participants made source decisions for recognized items either in succession (i.e., the standard format) or in separate test blocks providing independent measures of item and source decision speed. Comparable speeds of item and source decision across the test formats would suggest a full separation between item and source processing, whereas different speeds would imply their (partial) temporal overlap. To test these alternatives, we used the drift rate parameter of the diffusion model (Ratcliff, Psychological Review, 85, 59-108, 1978). We examined whether the drift rates, together with the other parameters, assessed separately for the item and source decision varied as a function of the test format. Threshold separation and nondecision time differed between the test formats, but item and source decision speeds represented by drift rates did not change significantly. Thus, despite facilitation on the source decision when the item decision was immediately followed by a test for source memory than when item and source were tested in separate blocks, findings did not suggest that source information already begins accumulating in the item test in the standard format. We discuss the temporal sequence of item and source processing in light of different assumptions about the contribution of familiarity and recollection.

Field-Testing Multiple-Choice Questions With AI Examinees: English Grammar Items.

Field-testing is an essential yet often resource-intensive step in the development of high-quality educational assessments. I introduce an innovative method for field-testing newly written exam items by substituting human examinees with artificially intelligent (AI) examinees. The proposed approach is demonstrated using 466 four-option multiple-choice English grammar questions. Pre-trained transformer language models are fine-tuned based on the 2-parameter logistic (2PL) item response model to respond like human test-takers. Each AI examinee is associated with a latent ability θ, and the item text is used to predict response selection probabilities for each of the four response options. For the best modeling approach identified, the overall correlation between the true and predicted 2PL correct response probabilities was .82 (bias = 0.00, root mean squared error = 0.18). The study results were promising, showing that item response data generated from AI can be used to calculate item proportion correct, item discrimination, conduct item calibration with anchors, distractor analysis, dimensionality analysis, and latent trait scoring. However, the proposed approach did not achieve the level of accuracy obtainable with human examinee response data. If further refined, potential resource savings in transitioning from human to AI field-testing could be enormous. AI could shorten the field-testing timeline, prevent examinees from seeing low-quality field-test items in real exams, shorten test lengths, eliminate test security, item exposure, and sample size concerns, reduce overall cost, and help expand the item bank. Example Python code from this study is available on Github: https://github.com/hotakamaeda/ai_field_testing1.

Physical capacity and performance correlation in sub-elite Indonesian archery athletes

The aim of this research is to analyse the physical components that can support the proper archery performance of young Indonesian athletes. This research employed a quantitative descriptive design involving 18 athletes from the East Java and West Java Archery Training Centre as the research subjects. Each athlete carried out a physical test consisting of 9 physical test items (1' incline sit up, Bosy Right, Bosy Left, MFT, Hover, Superman Back Hold, Pinch Grip, standing balance test, standing archery hold), which the trainer has prepared according to the needs of archery. After that, archery performance assessment was done using the recurve division's individual qualification system, i.e., arrow athletes performed 72 arrow shots. The data were then analysed using descriptive tests, silhouette method tests, and K-means clustering analysis to classify physical components and simplify the data collection process. These processes increased the accuracy of changes needed in the training process to improve athlete performance. A correlation test was also conducted using a Spearman's rho with a significance p<0.05. The study's results revealed that the silhouette method divided the variables into six clusters. The K-means cluster analysis results showed that exercises focused more on the arms, while trunk and lower extremity exercises received less attention. The research also found two physical aspects that had a positive and significant correlation with archery performance, namely, left pinch grip and right pinch grip. In addition, this research showed that the archers’ physical aspects of pinch grip strength (left and right) were closely related to performance achievements or scores in archery. Keywords: Physical capacity, sub-elite athletes, archery, archery performance.

Establishment and discussion of autoverification rules for transfusion compatibility testing.

To develop an automated verification workflow for transfusion compatibility testing (TCT) based on the AUTO10-A guidelines and blood group serology characteristics and to conduct a simulated validation of the test and subtest results by assessing the appropriateness of the autoverification rules. The accuracy of TCT results is a fundamental prerequisite for ensuring the safety of blood transfusions. However, the verification of these results still requires manual intervention. Five autoverification rules and their standards were determined: agglutination intensity, normal results, logical relationships, delta checks and interlaboratory test comparisons. The established categories and standards for the five rules were retrospectively validated using 13 506 samples (requests) that had been manually verified in our laboratory from January 2020 to June 2023. A total of 66 638 test items were involved in the autoverification, with 3844 items violating the verification rules, resulting in a pass rate of 96.10%. Considering individual test items, four tests had a pass rate of more than 90% in both the test item result table and the test subitem result table. However, there were significant differences in the pass rates between different tests. The same conclusion can be drawn when the unit is requests. The different standards set for the agglutination intensity and the delta check in the ABO typing testing subitems showed significant differences in pass rates. The incorporation of manually verified results into the automated verification simulation indicated that the five rules established in this study have good applicability, and appropriate standards can lead to reasonable pass rates.

Evaluating the Quality of Multiple-Choice Test Questions in the Postlicensure Environment.

Multiple-choice test questions are among the main measures of knowledge used by educators in the postlicensure environment; however, these tests are often constructed in the absence of guidelines or the means to evaluate examination quality after administration. Previously, guidance was provided on constructing quality test items. Here we present instruction for professional development specialists to use postadministration test data for an item analysis, providing insight on test flaws and opportunities for iterative examination improvement. The topics of item difficulty, index of discrimination, and distractor analysis are covered for independent analysis, and topics such as reliability are addressed for those who have access to a formal program. Three levels of strategies are described: using a learning management system for item analysis, using free open-source software, and using a minimal standards method of evaluating test items. [J Contin Educ Nurs. 2024;55(10):487-492.].

ONLINE PRACTICE TEST BANK SOLUTION: A WEB-BASED REVIEW PLATFORM FOR LIBRARY AND INFORMATION SCIENCE PRACTITIONERS

Test construction and test administration in most institutions still adopt the traditional pen-and-paper method of examination, which is very tedious (Kotwal et al., 2016). Its lengthy process can limit the time instructors in devoting to their proactive instruction and assessment. The general purpose of the study is to develop an Online Practice Test bank for Library and Information Science Practitioners. To actualize the Online Practice Test bank, the researcher followed the three phases of the modified rapid application model: Planning or Analyze Phase, Design and Development or Prototyping Phase, and Testing. The result of the study revealed that the developed system was able to execute its different operations. Specifically, by providing an avenue to the contributors/review masters and the reviewees to communicate and interact regarding the different reviewers provided. The reviewees can also track their progress through the learning analytics feature of the system. The contributors/review masters can monitor if the review materials they have provided has efficient test items by accessing the test item analysis. The system also has undergone functionality testing through conducting test cases, which have also gained positive remarks. Based on the results of the developed Online Practice Test bank, it is concluded that the system attained its three objectives in having integrated features that would be a great help, especially to the future licensed professional librarians as they will have a platform where they do their review and assess their knowledge on the six core subjects of library science. It is therefore recommended that mobile or android-based user modules must be developed for it to be more accessible to the users. And also an automated intervention or scaffolding for coping with the results from the performance of the review. And it must have an option or platform for contributors outside the institution who want to share their review materials relevant to the Library and Information Science profession.

Simulation study: Seeking fairness in a high-stakes placement test

The primary objective of Cognitive Diagnostic Assessment (CDA) lies in problem solving processes to systematically categorize test takers in mastery/non-mastery groups. What is of concern is using some attributes in reading comprehension test item options which are suspected to bias, and is simulating unbiased test items in a reversed engineering attempt. To this end, 4200 PhD candidates who sat for the examination were randomly selected. A Q-matrix was constructed, and raw data were fed into the R-studio software. Two groups of female and male differences were considered for the Differential Item Functioning (DIF) detection based on the Deterministic Inputs, Noisy “and” Gate (DINA) model. Results of the simulation study revealed that in 100 times of item generations, items were non-significant in favor of gender bias. In conclusion, this study was an attempt to determine whether an item identified with DIF is a correct detection or failed to be identified in fine-grained details.

Pengaruh Metode Quantum Teaching terhadap Pemahaman Materi Keberagaman di Kelas 4 SDN No 67/1 Sengkati Baru

This study aims to evaluate the impact of the Quantum Teaching method on the understanding of KEBERAGAMAN material in grade 4 at SDN No 67/1Sengkati Baru. Quantum Teaching is an instructional approach that focuses on actively engaging students through various learning styles, including visual, auditory, and kinesthetic. This research employs a quantitative experimental design to compare learning outcomes between students taught using the Quantum Teaching method and those taught using conventional methods. The sample consists of 25 students divided into two groups: one group using Quantum Teaching and the other using conventional methods. Data was collected through end-of-term tests to measure understanding of the material and analyzed using R Table validity test, ANOVA hypothesis testing, and frequency distribution of learning outcomes. The results indicate that all test items were valid, and there was a significant difference in learning outcomes between the two groups, with the Quantum Teaching group performing significantly better than the conventional method group. Additionally, the frequency distribution of learning outcomes shows that most students scored above average after implementing Quantum Teaching. The study also found that 80% of students preferred the Quantum Teaching method over the conventional method, indicating that this approach is more effective in enhancing student motivation and engagement in the learning process. These findings provide a strong basis for considering the implementation of Quantum Teaching as a superior alternative in primary school education.

Teachers’ computational thinking content knowledge: Development of a measurement instrument

Computational thinking has become an integral component of curricula worldwide, necessitating teachers to develop this competence in their students. To effectively meet these curricular requirements, teachers themselves need a solid foundation of computational thinking content knowledge, which refers to the understanding and skills they possess in this area. However, despite widespread recognition of this need, few studies have rigorously examined teachers' content knowledge in this domain. Addressing this gap requires the development of high-quality measurement tools. This study details the development of an instrument, created as part of the International Computer and Information Literacy Study (ICILS) 2023 in Flanders, to measure lower secondary school teachers' computational thinking content knowledge in a valid and reliable way. The article first outlines the construction process of the instrument, which involved close collaboration with experts in the field and drew upon the framework of Fraillon et al. (2023). Following this, the instrument's psychometric properties are presented, which include both item-level and overall instrument characteristics. These properties were evaluated using data from a sample of 352 participants, applying both Classical Test Theory and Item Response Theory. The final tool consists of sixteen multiple-choice and short constructed response questions. The results indicate favorable item and overall instrument characteristics, thereby affirming its potential to measure the intended construct in a valid and accurate way.

Adaptation of a Chinese Version of the Relational Needs Satisfaction Scale Based on Item Response Theory Among Chinese Adults Aged 18-30.

The satisfaction of relational needs is an important indicator of the quality of interpersonal relationships, but few studies on this topic have been conducted in China. The Relational Needs Satisfaction Scale (RNSS) was developed and used in three countries. In this study, the scale was standardized. A total of 2136 Chinese adults aged 18-30 years were recruited to complete the RNSS-Chinese (RNSS-C) and the Satisfaction with Life Scale (SWLS) online, and classical test theory and item response theory (IRT) were used to analyze the psychometric characteristics of the RNSS-C. The measurement invariance of the Chinese and Czech RNSSs was analyzed. (1) The RNSS still has good reliability (Cronbach's alpha = 0.935) and validity (CFI=0.948, TLI=0.940, RMSEA=0.049, and SRMR=0.032) in China. (2) The RNSS-C revealed differences in the perceptions of relational needs construct between the two groups (ΔCFI=0.014>0.01). These findings contribute to a deeper understanding of the cultural factors that shape interpersonal relationships and satisfaction in China.

A portable test system for the radar

Abstract In order to adapt to the increasingly fierce battlefield confrontation scenarios, a portable, fast deployment and folding test system for the radar is proposed. The corresponding key test items of radar such as spectrum, waveform, lobe, target tracking and data analysis can be completed on the test system. After the Built-in-Test (BIT) information and test system results are integrated, the corresponding evaluation report and implementation recommendations can be provided to the maintenance personnel. This paper expounds the design of each subsystem step by step. The Main Control Unit (MCU) is the task scheduling and information integration center for the system. The corresponding high frequency components, processing components, storage components have been implemented systematic and miniaturized design. In the end, the improvement direction and prospect of the test system are given.

Research on automated verification method for temperature-related tests of electrical energy meters

Abstract Program of pattern evaluation of fixed electrical energy meters-alternating current and GB/T 17215 series of national standards for the examination of energy meter performance by the impact of temperature change, set up a variety of temperature-related test items. Systematic research is possible to conclude that temperature-related tests can be divided into two categories according to the test procedure: Hold at test temperature for a certain period of time, return to the reference temperature and then measure the relevant parameters; measurement of the relevant parameters immediately after a certain period of time at the test temperature. Relevant parameters include errors at different load points and daily timing error of the internal clock of the energy meter. Data processing is undertaken after the test mainly consists of deriving error shift and mean temperature coefficient. This automated calibration method allows you to freely set the test temperature, test duration, reference temperature, stabilization duration, detection parameters, test points and data processing according to the test requirements. The paper explains in detail the application of the automated calibration method to realize typical temperature dependence, test of time-keeping accuracy of the energy meter with temperature and climatic tests (high temperature test, low temperature test and durability test of accuracy) The method breaks the shackles of the traditional calibration method of fixed test temperature, fixed test points and fixed test duration. It can adapt to the development needs of different levels of pattern evaluation of energy meter and national standards, for the laboratory to realize automated testing to provide the basis for industry standards and product testing commissioned inspection.

Investigating pre-tertiary students’ mistakes in solving algebraic word problems: Insights from Asutifi North District, Ghana

Algebra word problems inculcate in learners the skills and ability to think critically and look for answers to problems in society and the world. Using an explanatory sequential mixed design, the study investigated the errors made by pre-tertiary students in algebraic word problems in a selected school in the Asutifi North District of the Ahafo Region of Ghana. Whereas the Newman Error Analysis Model was used to find the types of errors students make when translating and solving algebraic word problems, protocols from topics relating to the study were carefully selected and used as interview guides alongside inspiration from the Newman Error Analysis Model. Quota and simple random sampling techniques were used to select ninety-eight (98) respondents. The Algebraic Word Problem Achievement Test (AWPAT) was given to the ninety-eight students who had studied several topics on word problems, followed by a structured interview to elicit more information from the respondents. The scores gained from the marked test items illustrated that students made more transformation errors, which happened indirectly from a lack of understanding of the concept. A careful analysis of the written responses of the five (5) students interviewed also revealed that, lack of comprehension directly translated into students’ inability to transform algebraic word problems into mathematics equations. In the same vein, the study showed that the main cause of students’ failure to translate and solve algebraic word problems was students’ lack of understanding of the concept of “word problem” among others. The study recommends that students be motivated by encouragement and praise to arouse their interest in translating and solving algebraic word problems.

Assessing of Biology Test Question Quality Using Anates Software

This research uses a quantitative descriptive approach. This research aims to determine the quality and appropriateness of Class XI Biology question items consisting of validity, reliability, level of difficulty, distinguishing power and distractor quality. The subjects are SMA N 1 Driyorejo Gresik as many as 30 students. Data collection in this research was carried out through student answer sheets, then the data was processed using Anates Ver 4.0.9. The test results show that the questions are appropriate and of good quality with a correlation of 0.62 and a reality coefficient of 0.76. This resulted in the question criteria being declared as having high validity, with four levels of difficulty and a small number of the quality of the distractor has poor quality test items.

Fagerstrom Test for Nicotine Dependence among a Sample of Smokers from Secondary School Students in Al-Najaf City, Iraq

Background: Tobacco addiction is a global social issue, leading to one death every six seconds worldwide. The Fagerstrom Test for Nicotine Dependence (FTND) is a commonly used tool to assessing physical nicotine dependence. The FTND scores range from 0 to 10 and consist of six questions, each with its score reflecting the smoker's level of nicotine dependence. This study aimed to assess the nicotine dependence of secondary school students using the Fagerstrom test. Methodology: A cross-sectional study was conducted from October 1, 2023, to April 10, 2024, in AL-Najaf City. The study sample included 283 smoking students, selected from a total of 829 participants from eight secondary schools, selected by using a multistage simple random sampling technique. The study instrument was a questionnaire with two parts: the first part collected some demographic characteristics, and the second part consisted of the Fagerstrom Test for Nicotine Dependence. Result: Out of 283 smokers, 234 were males, and 49 were females. According to Fagerstrom Test items, the results show 43.1% smoke the first cigarette 60 minutes after waking up, 68.9% do not find it difficult to refrain from smoking in any forbidden places, 58% cannot give up smoking cigarettes after eating, 61.8% for ≤ 10 smoked cigarettes per day, 62.9% for never smoke more frequently during the first hours after waking and 65.7% for not smoke when so ill. Conclusion: - 47% of smokers were in low dependence, 22.3% for low to moderate, 29.7% for moderate and 1.1 % for high dependence, with a significant difference between the Fagerstrom test and socio-economic status.

EXAMINING NUMBER SENSE SKILLS OF KINDERGARTEN CHILDREN

The aim of this study is to determine the number sense skills of preschool (Kindergarten) children. In the study, survey design, which is one of the quantitative research methods, was used. The study population of the research consists of a total of 114 children attending all of kindergartens (5 kindergartens) in the city center of Tunceli in the 2020-2021 academic year. The data collection tool used in the study is the "Preschool Number Sense Test" developed by the researchers. In the analysis of the data, the items in the number sense test; The correct answer obtained by using number sense skills, the correct answer obtained by using rule and operation-based methods, the wrong answer obtained by using number sense skills, the wrong answer obtained by using rule-operation-based solutions, and the blank answer were coded. As a result of the analysis made, it was determined that preschool students used rule- and operation-based solutions more in solving questions, and it was seen that they resorted to number-sense-based solutions less frequently.