Test Items Research Articles

Preoperative prediction of early mortality after surgery for spinal metastases.

The objective of this study was to provide a convenient preoperative prediction of the risk of early postoperative mortality. This retrospective study included patients who underwent surgery for spinal metastasis at our hospital between 2009 and 2021. Preoperative blood test data of all patients were collected, and the survival time was calculated by dividing the blood data. A multivariate analysis was conducted using a Cox proportional hazards model to identify prognostic factors. The study population included 83 patients (average: 64.5years), 22 of whom died within 3 months. The most common lesion was the thoracic spine, and incomplete paralysis was observed in 57 patients. The surgical methods included posterior implant fixation (n=17), posterior decompression (n=31), and posterior decompression with fixation (n=35). In the univariate analysis, the presence of abnormal values was significantly associated with postoperative survival in six preoperative blood collection items (hemoglobin, C-reactive protein, albumin, white blood cell, gamma-glutamyl transpeptidase, and lactate dehydrogenase). In a multivariate analysis, four test items (hemoglobin, C-reactive protein, white blood cell, and lactate dehydrogenase) were identified as independent prognostic factors.Comparing cases with ≥3 abnormal values among the above four items (high-risk group; n=23) and those with ≤2 (low-risk group; n=60), there was a significant difference in survival time. In addition, it was possible to predict cases of early death within 3months after surgery with 73% sensitivity and 89% specificity. The study showed that four preoperative blood test abnormalities (hemoglobin, C-reactive protein white blood cell, and lactate dehydrogenase) indicated the possibility of early death within 3months after surgery.

LITMUS Turkish sentence repetition test: The best items, effect of scoring and diagnostic accuracy

Purpose This study aimed to examine LITMUS Turkish Sentence Repetition Test’s (LITMUS-TR) diagnostic accuracy, as well as the best scoring method and most distinguishing test items. We also sought to ascertain whether age has an impact on the sensitivity and specificity. Method Two hundred and fifty children with typical language development (TD) between the ages of 4 and 7, as well as 44 children with developmental language disorder (DLD), took part in the study. Data was collected using TODİL, LITMUS-TR, and the pediatric family interview form. LITMUS-TR was graded using four different methods. Results The performance of children with DLD in each score type was lower and the number of errors higher than those with TD. All items have excellent or acceptable item difficulty and discrimination values for binary scoring and total number of errors. LITMUS-TR's most distinctive items were complex structures with dependencies, such as syntactic movement and embedding. LITMUS-TR had high diagnostic accuracy for the whole test (0.887) and each scoring method. A separate analysis of each age group showed sensitivity and specificity above 0.80. Conclusions When employed as a supportive objective measure, LITMUS-TR was proven to be an effective diagnostic tool for DLD, with age influencing the diagnostic accuracy outcomes.

Different estimation techniques and data analysis for constant-partially accelerated life tests for power half-logistic distribution

Partial accelerated life tests (PALTs) are employed when the results of accelerated life testing cannot be extended to usage circumstances. This work discusses the challenge of different estimating strategies in constant PALT with complete data. The lifetime distribution of the test item is assumed to follow the power half-logistic distribution. Several classical and Bayesian estimation techniques are presented to estimate the distribution parameters and the acceleration factor of the power half-logistic distribution. These techniques include Anderson–Darling, maximum likelihood, Cramér von-Mises, ordinary least squares, weighted least squares, maximum product of spacing and Bayesian. Additionally, the Bayesian credible intervals and approximate confidence intervals are constructed. A simulation study is provided to compare the outcomes of various estimation methods that have been provided based on mean squared error, absolute average bias, length of intervals, and coverage probabilities. This study shows that the maximum product of spacing estimation is the most effective strategy among the options in most circumstances when adopting the minimum values for MSE and average bias. In the majority of situations, Bayesian method outperforms other methods when taking into account both MSE and average bias values. When comparing approximation confidence intervals to Bayesian credible intervals, the latter have a higher coverage probability and smaller average length. Two authentic data sets are examined for illustrative purposes. Examining the two real data sets shows that the value methods are workable and applicable to certain engineering-related problems.

Large language multimodal models for new-onset type 2 diabetes prediction using five-year cohort electronic health records

Type 2 diabetes mellitus (T2DM) is a prevalent health challenge faced by countries worldwide. In this study, we propose a novel large language multimodal models (LLMMs) framework incorporating multimodal data from clinical notes and laboratory results for diabetes risk prediction. We collected five years of electronic health records (EHRs) dating from 2017 to 2021 from a Taiwan hospital database. This dataset included 1,420,596 clinical notes, 387,392 laboratory results, and more than 1505 laboratory test items. Our method combined a text embedding encoder and multi-head attention layer to learn laboratory values, and utilized a deep neural network (DNN) module to merge blood features with chronic disease semantics into a latent space. In our experiments, we observed that integrating clinical notes with predictions based on textual laboratory values significantly enhanced the predictive capability of the unimodal model in the early detection of T2DM. Moreover, we achieved an area greater than 0.70 under the receiver operating characteristic curve (AUC) for new-onset T2DM prediction, demonstrating the effectiveness of leveraging textual laboratory data for training and inference in LLMs and improving the accuracy of new-onset diabetes prediction.

Using Performance Improvement Methods to Evaluate Processes for Writing Multiple-Choice Test Questions in the Postlicensure Clinical Environment: A Case Study.

This article is the last of a four-part series to guide educators on the construction and evaluation of multiple-choice test items in the post-licensure environment. Previous articles in this series described the problem and the mechanics of test item construction and evaluation. A replicable strategy for evaluating the organizational process for constructing multiple-choice test questions is provided. Steps taken to create change are described; work tools are provided. Guidance and training are needed to create multiple-choice test questions. Many educators have not had training in item construction. Educators welcomed training. Personalized mentorship resulted in improvement. Asynchronous learning alone was helpful and well received and improved self-perceived knowledge, yet fell short of achieving competence. Voluntary training may not be adequate to assure enculturation of best practices without accountability standards and monitoring. Future research is indicated to assess the situation and provide national standards for adoption within health care organizations. [J Contin Educ Nurs. 2024;55(11):535-542.].

Fl-IRT-ing with Psychometrics to Improve NLP Bias Measurement

To prevent ordinary people from being harmed by natural language processing (NLP) technology, finding ways to measure the extent to which a language model is biased (e.g., regarding gender) has become an active area of research. One popular class of NLP bias measures are bias benchmark datasets—collections of test items that are meant to assess a language model’s preference for stereotypical versus non-stereotypical language. In this paper, we argue that such bias benchmarks should be assessed with models from the psychometric framework of item response theory (IRT). Specifically, we tie an introduction to basic IRT concepts and models with a discussion of how they could be relevant to the evaluation, interpretation and improvement of bias benchmark datasets. Regarding evaluation, IRT provides us with methodological tools for assessing the quality of both individual test items (e.g., the extent to which an item can differentiate highly biased from less biased language models) as well as benchmarks as a whole (e.g., the extent to which the benchmark allows us to assess not only severe but also subtle levels of model bias). Through such diagnostic tools, the quality of benchmark datasets could be improved, for example by deleting or reworking poorly performing items. Finally, in regards to interpretation, we argue that IRT models’ estimates for language model bias are conceptually superior to traditional accuracy-based evaluation metrics, as the former take into account more information than just whether or not a language model provided a biased response.

Research on enhancing the efficiency of food safety sampling inspections in China based on Pareto's law.

Food safety is pivotal for public welfare and directly impacts consumer health. Food safety sampling inspections (FSSIs) are essential in detecting unqualified food products and non-compliant manufacturers, which form an integral part of government regulatory frameworks. However, given the constraints on budgetary resources, improving the efficiency of food safety sampling inspections (EFSSIs) remains a considerable challenge in China's food quality and safety supervision. This study aims to apply Pareto's law, starting from the examination of food sample testing items and major hazard types, to theoretically analyze methods for improving the EFSSIs. Following the theoretical analysis, the research employs provincial food sampling data from China in 2022 to empirically validate the proposed improvement strategies. The research findings indicate that applying Pareto's law significantly reduces the number of items that should be tested for each food subcategory, effectively lowering testing costs for each batch of food samples. Theoretically, employing Pareto's law in sampling inspections can increase the EFSSIs to 2.78 times the current observed level. Furthermore, empirical validation using food sampling data confirms that EFSSIs can be improved to 2.12 times the existing level, consistent with theoretical predictions. Implementing Pareto's law in FSSIs facilitates the detection of more unqualified food products and non-compliant manufacturers without additional financial burden, significantly enhancing the EFSSIs. This approach provides an innovative strategy for government to bolster their food safety management efforts. © 2024 Society of Chemical Industry.

Adaptations in Vocational Guidance: Experience from the Colombian Amazon

As we move into late postmodernity, the instruments and processes proposed require research and development processes that allow them to remain in force within current technological mediations and trends. In this context, the instruments that facilitate vocational guidance processes are included. Therefore, this innovation experience arises as a response to the needs present in the Colombian context, based on two objectives: To reduce the number of items in the CHASIDE vocational orientation test, 2) To validate the resulting test. A longitudinal study with a quantitative approach was carried out, with a convenience sampling in three Colombian municipalities during 2019-2023. Quantitative statistical analysis techniques, data were analyzed including statistics of mean, variance, Pearson's correlation coefficient, hypothetical difference of means, degrees of freedom, t-statistic, P(t) and critical value of t (one tail and two tails), and others. Additionally, internal consistency was checked using Cronbach's Alpha. Managing to reduce the test from 98 items to 35 items, preserving its two dimensions (Aptitudes and Interests), with a simpler structure, which responds to a Multifactorial Profile (PM), its validation confirmed the new structure, that established 7 factors for the dimension of aptitudes, constituted by 27 items and 3 factors for the Interests dimension with 8 items. In conclusion, social changes are reflected in factor structures and their groupings within professions, leaving the unifactorial proposed by Holland in the RIASEC, to give way to interactions within aptitudes and interests. Aptitudes being more stable than interests over time.

Korean assessment of adult basic literacy: Instrument development

This study describes the development of a Korean assessment instrument measuring adults’ basic literacy ability. Instrument items were generated from a literature review, inputs from experts through the Delphi method, and focus group interviews. Items and measurement procedures were tested and confirmed through three steps of pilot implementations. As a result, three sets of test items measuring up to level 4 of the Korean basic literacy ability for adults were successfully developed. This assessment is simple, fast, and inexpensive and reflects the distinctive characteristics of the Korean language and Korean contexts. The instrument’s practicality will contribute to its further utilization in identifying and supporting adults in need of basic literacy education.

Item Analysis of Tool to Examine the Effect of E-Module on the Academic Achievement of Chemistry Students at Secondary Level

This study was conducted to analyze test items by measuring quantitative characteristics (difficulty level and discrimination index). There were 25 MCQ type test items designed by using first two chapters of Chemistry Class 10th. This test was used as a pretest to examine the “Effect of e-module on the Academic Achievement of Chemistry Students at Secondary Level”. Researcher selected 150 students through multistage random sampling from district Gujrat for item analysis. A total of 150 students were used to measure the difficulty index. Researcher arranged the students in descending order and selected 33% high achievers and 33% low achiever by taking a total of 100 students (50 from HEs and 50 from LEs). In this way, 50 top students from high achievers and 50 bottom students from low achievers were selected for Item discrimination. Data were analyzed through descriptive statistics. Findings revealed that 18 items out of 25 were valid. Seven test items (2, 12, 15, 16, 19, 20 and 25) were rejected from the analysis of difficulty index. Out of these seven, difficulty index value for six items was very high (>70). Only one item (19) was difficult and rejected due to the very low value of the difficulty index (< 29). 18 items were retained after necessary revision. Discrimination index of 25 items showed that 18 items were discriminant after the revision of marginal items. It was concluded that 18 items out of 25 were valid based on the Difficulty index and Discrimination index used as a pretest. It was recommended that item analysis is a useful method to ensure the validity and reliability of test items.

Psychometric Properties and Factor Structure of the Disgust Scale-Revised (DS-R) for the Romanian population

Research on measuring the intensity of disgust has required the development of instruments to measure this construct. Given that, a more culturally appropriate scale is needed to ensure a valid and reliable measurement of disgust. The concept of “disgust” does not have an appropriate literature in Romanian pathology, and Disgust Scale-Revised (DS-R) is quite rarely used in the public health context. Therefore, this study focuses on the psychometric properties of the Romanian adaptation of the DS-R, one of the preferred instruments for measuring disgust. More specifically, the research investigated different forms of reliability and validity of the DS-R, using both Classical Test Theory (CTT) and Item Response Theory (IRT) analysis approaches. In general, the results showed good fidelity, fit of the single- factor model of the DS-R, good convergent and divergent validity with respect to the relationships of this scale with other variables. The study also highlights the important role of disgust and anxiety in predicting obsessive- compulsive symptoms. In conclusion, further refinement of the scale could be useful for a more comprehensive development of this instrument in the clinical setting.

A Retrospective Analysis of Four STI Pathogens in Obstetrics and Gynecological Patients in Hangzhou, China.

Among sexually transmitted infection (STI) pathogens, UU, CT, NG, and MG are very commonly detected. A retrospective study was conducted to analyze the clinical situations and laboratory data of patients in-fected with the four pathogens in Hangzhou city, Zhejiang Province, China. For this study, 4,716 female outpatients and inpatients of the Department of Obstetrics and Gynecology of Hangzhou Third People´s Hospital were randomly selected from January 2019 to December 2023. Multiple types of specimens were collected and four STI pathogens were detected. Data collection was taken from the hospital's electronic medical records and statistical analysis was processed with SPSS 25.0 software. In the past five years, year by year, the positive rate presented an increasing trend. Out of 4,716 patients, 2,931 were positive, and the positive rate was 62.15%. The positive rate of single infection was significantly higher than of co-infection (p < 0.05). The highest positive rate of all STI patients was found in the  20 age group, and the rate was significantly different between each age group (p < 0.05). Single infection with UU occurred significantly more often than with the other three pathogens (p < 0.05), and co-infection with UU and CT and UU, CT, and MG were significantly dominating (p < 0.05). Only three patients had an infection with all four pathogens, and it was extremely rare. The 21 - 30 and 31 - 40 age groups had high incidences of infection, with a significant difference compared with other age groups (p < 0.05). The positive rate shows an increasing trend in Hangzhou in the past five years. More single infection than co-infection and more younger patients were detected. Single infection with UU was dominant in all positive patients. The publicity and more screening testing items of STI for younger people should be strengthened in Hangzhou, and the people of Hangzhou should be made more aware that information technology is helpful in aiding the control of STI diseases.

Investigating level descriptors and their impact on validity

This article presents a validation study which investigates the relationship between mastery level descriptors and item difficulty in the National Tests of English (NTE) in Norway. The aim is to establish the extent to which the descriptors indicate item difficulty and thus support the argument that the mastery levels are a reflection of the framework of the NTE. This argument strength has direct implications for validity and for the defensibility of moving from criterion-based descriptors to norm-based national data. The study involved a panel of 10 raters assigning level descriptors from 7 content categories to 80 test items, giving a total of 5,600 individual judgements. These judgements are compared to the real test scores of around 46,000 pupils to establish if the level descriptors assigned by raters can predict pupil performance on the tests. The results show a strong correlation between rater judgements and real test scores, meaning that the descriptors offer an indication of item difficulty. However, some individual descriptor categories contain deviations from the expected order. We conclude that, while the level descriptors reflect the test’s framework, and thus support the validity argument, the argument could be strengthened with the revision of some individual descriptors.

Age prediction from COVID-19 blood test for ensuring robust artificial intelligence

With the advancement of artificial intelligence (AI) nowadays, the world is experiencing conveniences in automating some complex and tedious tasks, such as analysing large data and predicting the future by mimicking human expertise. AI has also shown promise for mitigating future crisis, such as pandemic. Since the beginning of the COVID-19, several AI models have been published by the researchers to help the healthcare to fight in this situation. However, before deploying the model, one needs to ensure that the model is robust and safe to learn from the real environment, especially in medical domain, where the uncertainty and incomplete information are not unusual. In the effort of providing robust AI, we proposed to use patient age as one of the feasible feature for ensuring vigorous AI models from electronic health record. We conducted several experiment with 28 blood test items and radiologist report from 1,000 COVID-19 patients. Our result shows that with the predicted age as an additional feature in mortality classification task, the model is significantly improved when compared to adding the actual age. We also reported our findings regarding the predicted age in the dataset.

Evaluation of sports organizational systems and performance of floorball athletes the Indonesian Floorball Association

The coaching program with outstanding achievements has indeed been implemented based on the objectives of the coaching itself and the critical factors contained in a program. Factors from an organizational system that is not good can also affect achievement results. With appropriate stages and fulfilling these factors, it is hoped that achievement will be more easily achieved. Looking at the introduction above, it is clear that Indonesian floorball still has many obstacles in its sports management system. Still needs to be developed in all regions of Indonesia. Besides that, Indonesian floorball athletes' performance is considered less able to match Asian countries. Hence, this study aims to evaluate the sports organization system and the implementation of floorball athletes in the Indonesian Floorball Association. The objects of this research include the Chairperson of Floorball at the central and regional levels, the Secretary, the Treasurer, the Coach, and 96 athletes who competed in the big eight rounds of the national floorball championship in Jakarta. Data was collected using the mixed method by administering a CIPP questionnaire and giving physical test items to floorball athletes. The output of this research is an evaluation of the sports organization system in which policymakers can improve and create a good formula so that the future organization can be better so that floorball athletes' achievements can increase.

Empowering Chemistry Education: Leveraging Technology to Elevate Academic Achievement at Kajaji Senior High School, Sene District, Bono East Region

This research work compares alternative methodologies; educational technology (projector media) and the traditional method of delivery, to improve upon students’ academic performance in chemistry among students of science students of Kajaji senior high school. Research questions and null hypotheses were formulated to serve as guide for this study. Form two science students were used for this study by convenience sampling and were placed into experimental and control groups. The study made use of standardized achievement test items from some selected topics in chemistry. Quasi-experimental non-randomised design involving pre-test and post-test was employed for the purpose of this study. The test scores obtained were analysed using t-test. The result obtained in this study showed that one of the null hypotheses formulated and tested at 95% level of significance was rejected. The findings revealed that the use of projector media in teaching was a more effective way of improving student’s performance in the teaching and learning of chemistry as compared to the conventional method.

Evidence for Safety of The Dietary Ingredient Agmatine Sulfate as Assessed by Mutagenicity and Genotoxicity Studies

Agmatine, 1-Amino-4-guanidinobutane, is a ubiquitous naturally occurring molecule present in low amounts in a wide variety of foodstuff. Clinical trials have demonstrated the safety of oral agmatine sulfate and have led to its development as an effective dietary ingredient for promoting resilient nerve functions. Although clearly required, the mutagenic and genotoxic effects of agmatine have not been previously reported. The present study, therefore, undertook to assess the safety profile of agmatine using currently accepted in vitro and in vivo mutagenicity and genotoxicity tests. The test item was G-Agmatine®, a proprietary brand of agmatine sulfate. Using the bacterial reverse mutation assay (Ames test), the study found that G-Agmatine® has no mutagenic effects. It had no clastogenic effects as observed by the in vitro chromosomal aberration test using Chinese Hamster lung cells. And it lacked genotoxic effects as evidenced by the lack of increased frequency of micronucleated polychromatic immature erythrocytes following oral administration in the mouse micronucleus test. Taken together with previously published data, results of the present study further support the safety of agmatine sulfate as a dietary ingredient.

Developing and validating measures for AI literacy tests: From self-reported to objective measures

The majority of AI literacy studies have designed and developed self-reported questionnaires to assess AI learning and understanding. These studies assessed students' perceived AI capability rather than AI literacy because self-perceptions are seldom an accurate account of true measures. International assessment programs that use objective measures to assess science, mathematical, digital, and computational literacy back up this argument. Furthermore, because AI education research is still in its infancy, the current definition of AI literacy in the literature may not meet the needs of young students. Therefore, this study aims to develop and validate an AI literacy test for school students within the interdisciplinary project known as AI4future. Engineering and education researchers created and selected 25 multiple-choice questions to accomplish this goal, and school teachers validated them while developing an AI curriculum for middle schools. 2390 students in grades 7 to 9 took the test. We used a Rasch model to investigate the discrimination, reliability, and validity of the items. The results showed that the model met the unidimensionality assumption and demonstrated a set of reliable and valid items. They indicate the quality of the test items. The test enables AI education researchers and practitioners to appropriately evaluate their AI-related education interventions.

Sparse item testing of clinical scales in neurology trials to alleviate burden to patients.

Neurology trials typically rely on composite scales for measuring symptom severity. Completing all items in a long scale can be burdensome for patients, caregivers, and trial personnel. To test the hypothesis that sparse item testing, aided by item-response modelling, can preserve the power for detecting treatment effect in a controlled trial. UPDRS (Unified Parkinson's Disease Rating Scale) Part III (motor examinations) data from a placebo-controlled trial (N = 391) of ropinirole were analysed with a longitudinal item-response model. Symptom severity was estimated directly from item scores as a latent variable, without needing the total score. This enabled sparse item testing. With the symptom severity as a clinical endpoint, the potential power loss for detecting treatment effect due to the sparse testing was assessed by simulation. When each patient took 18 of all 27 tests in UPDRS Part III at each study visit, there was no appreciable power loss. Reducing four visits to three also had negligible effects on power. A threefold reduction of the total tests that each patient needed to do throughout the trial, from 108 to 27, only compromised power slightly, e.g., from 92 to 87% at N = 160. These findings show that using the symptom severity derived from item scores as the endpoint allows sparse testing to drastically reduce trial burden without incurring major power loss. This benefit would multiply for indications like Alzheimer's disease where modern trials often require patients to be tested on multiple scales at several times.

The Effect of PDEODE and SWOM Strategies in Developing 10th Grade Literary Students’ Historical Thinking Skills

The current study which is an extract from a PhD dissertation adopted a mixed- method, experimental-design to find out the effect of PDEODE and SWOM strategies in developing tenth- grade literary students’ historical thinking skills. The sample of the study consists of sixty tenth grade students from the literary branch at Tafan Preparatory School for Girls. After equalizing them, they are divided in this way: Group A regarded as the first experimental group taught by PDEODE strategy, Group B as the control group taught conventionally and group C taught by SWOM strategy as the second experimental group. For each of these groups, a twenty- four daily teaching plan was used and a 30- item test was prepared by the researcher to know the effect of these strategies in developing historical thinking skills. After analyzing the data through the statistical method ONE- WAY ANNOVA, it was displayed that there are statistically significant differences among the groups in a way the first experimental group is better than the second experimental and control groups. It was also concluded that PDEODE and SWOM as two new teaching strategies can provide appropriate platform for improving teaching and learning processes as they focus on students’ intellectual, psychological and personal dimensions, making them the center of the process which enables them to apply their knowledge in different contexts.