Proficiency Testing Research Articles

The role of the EXAVER for English B.A. students

This research focuses on finding out the reason(s) why students from the B.A. in English language take the EXAVER proficiency test, more precisely to discover why EXAVER is useful for them, as well as their opinions regarding the EXAVER. The participants of this study are students from the B.A. in English language at the Universidad Veracruzana. The nature of this research is a mixed method approach, also the data were collected through the survey and interview methods and contrasted to the literature review to finally analyze them through triangulation. The highlighted findings are how the EXAVER test is used within the UV, and so the participants’ reasons to use the EXAVER certificate. The analysis also showed that the EXAVER is considered a test that evaluates English language skills accurately since the participants would take it again and some of them have taken it more than once.

Assessment of the Impact of Teachers’ Pedagogical Content Knowledge and Self-efficacy on Senior High School Learners’ Mathematics Proficiency

Aim: The purpose of this study was to provide insights into the specific ways in which teachers’ pedagogical content knowledge and self-efficacy impact the mathematics proficiency of senior high school students. Methods: The study employed positivism quantitative research approach and a descriptive design. The study sample size was 178 comprising 152 Home Economics students and 26 mathematics teachers. The sample members were chosen for the study using purposive sampling technique. A questionnaire and a test of mathematical proficiency were used to gather data. In this work, a basic linear regression model and descriptive statistics were employed as data analysis tools. Results: The study's findings indicate that teachers have a relatively high level of pedagogical content knowledge, with mean scores ranging from 2.88 to 3.69; 2). The study also discovered that teachers have a relatively high level of self-efficacy, with mean scores ranging from 2.88 to 3.69; 3). Additionally, the findings of the simple linear regression analysis did not show a significant correlation between the pedagogical content knowledge, self-efficacy, and students mathematics test performance. Conclusion: The study concluded that teachers’ pedagogical content knowledge and self-efficacy have no significant impact on senior high school learners’ mathematics proficiency. This therefore implies that other elements, such as student background variables, school resources, or educational practices, may have a greater impact on students' mathematical success. Recommendation: The study recommended that future studies use more complete measures of teacher dedication and other factors that may affect student outcomes to examine the association between teacher traits and student accomplishment in mathematics education.

Analysis of Punctuation and Spelling Errors Associated with Academic Writing among Iraqi EFL Learners

The present study aims to analyse and find punctuation and spelling errors EFL learners face in their academic writing as part of English proficiency tests. A quantitative method was adopted to implement the research objective. 117 written samples were collected and analysed. The participants were 82 females and 35 males. The average age is 23 to 37 years. The results showed different kinds of errors and challenges. The present study concerns limited errors and challenges: punctuation and spelling errors. The findings show a large prevalence of punctuation and spelling errors among Iraqi EFL learners. The most common errors using punctuation marks were using capital letters (54.72%), commas (25. 71%), and periods (19.58%). In terms of spelling errors, omission recorded the highest percentage of errors (40.9%), followed by substitution (30.5%), transposition (15.0), and insertion (13.6%).

Factors Affect In IELTS Test For Students of STIKOM Uyelindo Kupang

IELTS (International English Language Testing System) is a standardized English language proficiency test which is a prerequisite for continuing study or work. There are still many people who experience difficulties in carrying out the IELTS test to achieve a score above 6. The aim of the research is to know what are the factors that affect the result of students IELTS test. This research wants to find out what factors influence students' mastery of IELTS material. The method used in this research is descriptive qualitative. Qualitative descriptive research uses a series of methods to specify and describe data findings that occur in the field without providing experimental treatment. There are four IELTS materials given during the test, including: reading, writing, speaking, and listening. These four materials require strategies in their implementation and each has their own difficulties in mastering them. This research will be conducted at STIKOM Uyelindo Kupang with the research object being students of the Informatics and Information Systems Engineering Study Program Semester 2. It is hoped that this research will provide readers with an understanding regarding the success factors in carrying out the IELTS test.

Effect of Gamification Combined with Mobile Learning on Chinese Language Speaking Skills of Second-Year Foreign Students at Guangdong University of Foreign Studies

Background and Aims: In the era of the mobile internet, transforming language learning content into interesting game tasks could stimulate students' interest in learning and improve engagement and learning outcomes. The study aimed to evaluate the impact of gamification combined with a mobile learning approach on students' Chinese language speaking skills and satisfaction. The significance of the study was to innovate teaching methods and improve oral expression. Students were guided to participate in interactive learning by designing gamified learning tasks with clear objectives. Methodology: This study utilized a pre-experimental design featuring a one-group pretest-posttest model. The sample of this study was 30 second-year foreign students studying at Guangdong University of Foreign Studies derived from a cluster random sampling method. The research tools were (1) Course lesson plan, (2) Chinese language speaking skill test, and (3) Student satisfaction questionnaire. Conduct pre-test and post-test on the sample using the Chinese speaking proficiency test, and analyze the mean, standard deviation, paired sample t-test, and one sample t-test of the data using statistics. Result: After using gamification combined with mobile learning, students' Chinese language speaking skills improved by 13.75%, surpassing the predetermined criteria of 70% and the difference was statistically significant at 0.05. Based on the scores from the statistical student satisfaction questionnaire, the overall results of Gamification combined with mobile learning by students are at a high level with (M=4.02, S.D. = 0.77) Conclusion: The study confirms that learning management using gamification combined with mobile learning significantly enhances Chinese language speaking skills it stimulates students' interest in learning, develops their independent learning ability, boosts their motivation and initiative, and caters to their individual learning needs. This is a new trend in future education.

International Proficiency Test Targeting a Large Panel of Botulinum Neurotoxin Sero- and Subtypes in Different Matrices

Detection of botulinum neurotoxins (BoNTs) involves a combination of technical challenges that call for the execution of inter-laboratory proficiency tests (PTs) to define the performance and ease of implementation of existing diagnostic methods regarding representative BoNT toxin-types spiked in clinical, food, or environmental matrices. In the framework of the EU project EuroBioTox, we organized an international proficiency test for the detection and quantification of the clinically relevant BoNT/A, B, E, and F sero- and subtypes including concentrations as low as 0.5 ng/mL. BoNTs were spiked in serum, milk, and soil matrices. Here, we evaluate the results of 18 laboratories participating in this PT. Participants have implemented a wide array of detection methods based on functional, immunological, and mass spectrometric principles. Methods implemented in this proficiency test notably included endopeptidase assays either coupled to mass spectrometry (Endopep-MS) or enzyme-linked immunosorbent assays (Endopep-ELISA). This interlaboratory exercise pinpoints the most effective and complementary methods shared by the greatest number of participants, also highlighting the importance of combining the training of selected methods and of distributing toxin reference material to reduce the variability of quantitative data.

Consistency between inter-institutional panels using a three-level Angoff-standard setting in licensure tests of foreign-trained dentists in Sweden: A cohort study.

Licensure exams play a crucial role in ensuring the competence of individuals entering a profession, thereby safeguarding the public and maintaining the quality and integrity of the profession. In Sweden, dentists educated outside the European Union seeking to practise dentistry must undergoa re-certification process. The re-certification process includes a theoretical examination where pass marks are set using a three-level Angoff method. This study aimed to determine the consistency of the Angoff ratings using independent panels at two Swedish universities. Two cohorts of panellists were included in the study: one reference and one external. The reference panel was responsible for rating the upcoming theoretical examinations in the proficiency test, which were used to set the pass mark. The external panel, recruited from a dental school at a university in another region in Sweden, provided ratings after the examinations. Three examinations during 2019-2020 were included in this study (267 items in total). There was a strong correlation (ρ ≥ 0.70, p < .001) between the ratings of the two independent panels, with no significant differences in item ratings across the full exams, dental disciplines, and professional qualifications analysed. This suggests that the three-level Angoff method reliably produces similar standards for assessing the competence of the minimally qualified dentist across different institutions. The expectations of the minimally qualified but still acceptable dentist were comparable between the two independent panels across the three theoretical examinations explored. The alignment between the panels indicates valid, reliable standards across institutions, despite the independent syllabi of the two study programmes. However, while there is an alignment, differences in ratings remain. Consequently, involving multiple institutions in future standard-setting processes could help ensure that the standards reflect a broader range of educational practices, supporting the credibility of licensure examinations.

Optimizing the Automated Analysis of Inorganic Gunshot Residue Particles by SEM-EDX: From Synthetic Particle Standards to More Time-Efficient Settings for Daily Casework

Gunshot residues deposited on all surfaces in the nearest vicinity of the shooting incident, when revealed, can contribute to the explanation of various aspects of such an incident for forensic purposes. Examinations of gunshot residue, mainly inorganic particles, at forensic laboratories are expected to be reliable and fast. This primarily depends on the performance of the used scanning electron microscope integrated with an energy dispersive X-ray spectrometer and the automatic program searching for particles of defined characteristics. Among the milestones on the pathway towards quality assurance in examinations of gunshot particles, the invention of the synthetic gunshot residue specimen ought to be named. Such a specimen with particles of known chemical content, size, and location is now used for proficiency testing, which is a condition for a forensic laboratory to obtain accreditation in this subject matter. In this publication, the need for optimization of the procedure for the examination of a synthetic specimen, in alignment with the necessary modifications for real gunshot particles, has been addressed. The presented process of validation resulted in two procedures. The first demonstrates the full capacity of the instrument for detecting all particles present in the synthetic specimen, including the 0.5 micrometer particle at the magnification of 250×. The other procedure is the modification of the first, however aiming at 1-micrometer particles or bigger (at the magnification of 120×) and allowing the necessary backscattered signal threshold changes depending on the actual composition of gunshot residue as well as the abundance of light element debris in the case of real gunshot particles.

A generative AI-driven interactive listening assessment task

IntroductionAssessments of interactional competence have traditionally been limited in large-scale language assessments. The listening portion suffers from construct underrepresentation, whereas the speaking portion suffers from limited task formats such as in-person interviews or role plays. Human-delivered tasks are challenging to administer at large scales, while automated assessments are typically very narrow in their assessment of the construct because they have carried over the limitations of traditional paper-based tasks to digital formats. However, computer-based assessments do allow for more interactive, automatically administered tasks, but come with increased complexity in task creation. Large language models present new opportunities for enhanced automated item generation (AIG) processes that can create complex content types and tasks at scale that support richer assessments.MethodsThis paper describes the use of such methods to generate content at scale for an interactive listening measure of interactional competence for the Duolingo English Test (DET), a large-scale, high-stakes test of English proficiency. The Interactive Listening task assesses test takers’ ability to participate in a full conversation, resulting in a more authentic assessment of interactive listening ability than prior automated assessments by positing comprehension and interaction as purposes of listening.Results and discussionThe results of a pilot of 713 tasks with hundreds of responses per task, along with the results of human review, demonstrate the feasibility of a human-in-the-loop, generative AI-driven approach for automatic creation of complex educational assessments at scale.

Clinical skills examination as part of the Swedish proficiency test of dentists educated outside of the EU/EEA.

The increase in the migration of dentists educated outside the EU/EEA calls for the sharing of information and evaluation of recognition processes within countries in the EU. In 2017, the Swedish National Board of Health and Welfare implemented the Proficiency test, a recognition process for dentists who have completed an education programme outside the EU/EEA. The Proficiency test consists of a theoretical and an integrated clinical skills examination, followed by a 6-month clinical practice. The clinical skills examination is a two-part examination that includes an OSCE and an operative test on a dental manikin. This paper presents data from proficiency tests between 2018 and 2022, and explores factors related to grade fail, that is, demographics, theoretical exam scores and language comprehension. In a cohort study, demographics and factors associated with grade fail were explored using test results from theoretical and clinical skills examinations (n = 181) from 2018 to 2022. Pearson correlation coefficient and linear regression analysis were used for studying correlations and associations between exam results. Univariable linear and logistic regression models were used for background variable associations with clinical skills exam outcomes. Higher age was a significant risk factor for failing the clinical skills examination and the OSCE. Higher scores in the theoretical exam reduced the odds of failing the OSCE but were not associated with results in the operative test or the overall results of the clinical skills examination. Regarding the OSCE there was a statistically significant difference within all professional qualifications explored between participants who passed and participants who failed the OSCE. Four years of data collection reveal that age and previous theoretical exam results influence the odds of failing the clinical examination. The study results also highlight the necessity of multiple assessment formats to assess clinical and communication skills of foreign-trained dentists.

Incoherent coherence? Using systemic functional linguistics to improve oral language assessment literacy

AbstractThis paper discusses the challenges of defining coherence in the context of oral language assessment literacy and proposes that better understanding of the construct can be achieved through a systemic-functional linguistic lens. Coherence is taken to be a foundational quality of written and spoken discourse and is a standard feature in the assessment rubrics of most large-scale English-language oral proficiency tests such as IELTS, TOEFL, and Cambridge. However, the precise meaning of coherence, and how it can be identified in language, is difficult to grasp. This has implications for the makers and raters of tests as well as the candidates who take them. The literature suggests coherence is the most troublesome category to rate reliably and validly. The systemic-functional linguistic (SFL) model of language provides a rich description of coherence, but has been under-utilised by the language testing industry to date. In this paper we examine three aspects of coherence: (1) how it is currently conceptualised in oral language proficiency testing; (2) how it is conceptualised in linguistics in general and SFL in particular; and (3) how insights from SFL could inform the evaluation of coherence in oral language proficiency examinations.

Clinical Need for Point-of-Care Testing for Diabetes in Clinical and Laboratory Improvement Amendments–Waived Settings

Point-of-care testing (POCT) has been used in multiple care settings for acute disease and, to a lesser extent, chronic disease testing. All POCT is regulated under the Clinical and Laboratory Improvement Amendments of 1988 (CLIA). CLIA-waived POCT requires no proficiency testing and can be carried out by nonlaboratory personnel. This review describes the benefits and limitations of POCT for cardiometabolic diseases and related conditions. It also explores the current U.S. regulatory environment for CLIA-waived POCT, highlighting the need for increased access.

Revitalization Of Learning Loss in Moji level Intermediate of Japanese Language Education Program Brawijaya University

This study explores the learning loss experienced at the Moji Intermediate level within the Japanese Language Education Study Program during the COVID-19 pandemic. Over a period of two years, traditional face-to-face learning was replaced by online instruction, which affected the implementation of certain learning stages, such as kanji writing exercises conducted in the classroom. This research aims to identify the extent of learning loss during online learning, specifically with kanji writing skills at the Moji Intermediate level. A descriptive qualitative method was used, comparing course grades from both online and offline learning periods. According to Rajesh, Hasibuan, Susanto, Solihat et al. (2022), while learning loss has been widely studied with various approaches, few have addressed its impact on specific courses, particularly in Japanese language education. The findings reveal that online learning significantly reduced students' kanji writing abilities. As a result, teachers must adopt new strategies to compensate for this gap and ensure that students perform well on the Japanese Language Proficiency Test (JLPT), which is essential for graduation from the Japanese Language Education Study Program.

Ensuring Quality and Consistency in OPC-53 Cement Testing: A Comprehensive Proficiency Testing Evaluation

The proficiency testing (PT) program is a critical part of quality assurance in laboratories, especially within industries like construction where cement testing is essential. This study provides a comprehensive evaluation of the fifth PT scheme for Ordinary Portland Cement (OPC) grade 53, conducted by the Nodal Laboratory. The key parameters assessed in this program were standard consistency, compressive strength, and soundness using the Le-Chatelier method. Twenty-four laboratories participated, and their performance was analyzed using advanced statistical methods, adhering to ISO 17043:2010 and ISO 13528:2015 standards. The results showed that most laboratories performed satisfactorily, with only one outlier identified in the soundness test. This paper covers all aspects of the PT program, from sample preparation to the distribution of materials, as well as the homogeneity and stability tests conducted on the samples. Statistical evaluation was carried out using Z-scores, which allowed the laboratories to be ranked according to their performance in each parameter. This method highlights discrepancies and provides a clear framework for participating laboratories to improve their testing procedures. Proficiency testing is essential for maintaining high standards in laboratory performance. It helps standardize testing methods across multiple laboratories, ensuring that OPC-53 cement meets strict quality and performance specifications. The program enables laboratories to identify performance gaps and take corrective actions, ensuring the reliability and accuracy of results. Participation in PT programs is a step toward continuous improvement, keeping laboratories aligned with international standards, ultimately benefiting the broader construction industry by ensuring the structural integrity of buildings and infrastructure. This study underscores the importance of regular PT participation. It not only provides a benchmark for laboratory performance but also ensures compliance with ISO standards. This fosters the consistent quality of cement used in large-scale construction, reducing the risk of structural failures and contributing to long-term safety and durability. The findings demonstrate the positive impact of PT programs on overall testing accuracy and laboratory reliability, emphasizing their role in advancing best practices in the construction industry.

To be with artificial intelligence in oral test or not to be: a probe into the traces of success in speaking skill, psychological well-being, autonomy, and academic buoyancy

There has been a surge in employing artificial intelligence (AI) in all areas of language pedagogy, not the least among them language testing and assessment. This study investigated the effects of AI-powered tools on English as a Foreign Language (EFL) learners’ speaking skills, psychological well-being, autonomy, and academic buoyancy. Using a concurrent mixed-methods design, the study included 28 upper-intermediate EFL students from an Ethiopian university. We gave the Michigan Language Proficiency Test to evaluate degrees of proficiency before the TOEFL iBT speaking section, which used ChatGPT for scoring and feedback. Speaking abilities were assessed using pretests, immediate posttests, and delayed posttests. Furthermore, we evaluated the impacts on psychological well-being, autonomy, and academic buoyancy using narrative frames. We used one-way repeated measurements to examine the quantitative data and thematically evaluated the qualitative data. According to the results, speaking abilities, psychological well-being, learner autonomy, and academic buoyancy showed notable increases. The results suggest that by improving skill development, offering individualized feedback, and meeting students' emotional and psychological needs, AI systems like ChatGPT have the capacity to transform language assessment and pedagogy. Encouraging the incorporation of AI technologies to enhance educational outcomes and provide a more flexible and adaptable learning environment, the study presents important implications for various stakeholders.

Chinese Folk Songs Can Facilitate Chinese Language Learning - A Pilot Study.

While previous research has demonstrated the beneficial role of pop music in foreign language learning, there is a lack of studies exploring the potential impact of Chinese folk songs on Mandarin language acquisition. This study aimed to investigate whether a curriculum based on Chinese folk songs enhances the outcomes of Mandarin Chinese learning in foreign speakers. International students in a university in Beijing who were attending regular Mandarin courses were allocated into two groups: the group receiving an additional Chinese folk song-based curriculum (intervention) and the group not (control). Mandarin proficiency after one and two semesters between the two groups was assessed using the Chinese Proficiency Test (i.e., Hanyu Shuiping Kaoshi, HSK) by the Generalized Estimating Equations (GEE) analyses. 16 international students were analysed (intervention: 8; control: 8). After adjusting for time, the intervention group showed a significantly higher HSK score in listening section (adjusted β = 7.86, p = 0.015) than that of the control group. In conclusions, Chinese folk song-based curriculum has the potential to enhance Mandarin listening among foreign speakers.

Proficiency test of SARS-CoV-2 Omicron variant detection in diagnostics samples by veterinary diagnostic laboratories

AbstractVeterinary diagnostic laboratories (VDLs) play a critical role in screening both human and animal samples for SARS-CoV-2. To evaluate the SARS-CoV-2 detection methods used by VDLs, a proficiency test was performed by the US Food and Drug Administration’s Veterinary Laboratory and Investigation and Response Network in collaboration with two other organizations. Thirty-two sets of 12 blind-coded samples were prepared by fortifying Molecular Transport Medium (MTM) or feline feces with SARS-CoV-2 Omicron variant or non-SARS-CoV-2 equine coronavirus RNA at various concentrations and shipped to 32 participants for blinded (unbiased) analysis. Results were analyzed according to the principles of International Organization for Standardization 16140-2:2016 using two approaches such as establishing the rate of detection (ROD) and the success rate by applying the analysis of binary outcome by logit approach. ROD provided the overall assessment of laboratories performance, whereas the novel logit approach provided an insight to more specific analysis based on the complexity of each sample. The ROD was 83% and 98% for MTM samples at 200 and 20000 genome copies per 100 µL, respectively. Fecal samples were classified as challenging exploratory, and results were not included in the assessment of performance but discussion purposes only. Fecal samples exhibited matrix interference impacting the performance. The ROD was 44% and 89% for fecal samples at 2000 and 20000 genome copies per 100 µL, respectively. The non-COVID coronavirus RNA, which was used to address the specificity, did not interfere with methods used. Establishing the success rate by evaluating the qualitative results (detected/not detected) applying a logit approach revealed that, out of thirty-two participants, twenty-eight had satisfactory results, one participant had unsatisfactory results, and three participants had questionable results for MTM samples. For fecal samples, three participants out of thirty-two did not meet the expectations at higher concentrations. Lower concentrations of fecal samples were excluded from this analysis. Again, the fecal samples were considered as challenge samples and the results were provided to assist participants in their continuous efforts to improve their performance and not to evaluate their performance.

Decreased variability in the site-specific results during participation in the External Quality Assurance Program Oversight Laboratory (EQAPOL) proficiency program for IFN-gamma enzyme-linked immunospot (IFN-γ ELISpot) assay

The NIAID DAIDS-sponsored External Quality Assurance Program Oversight Laboratory (EQAPOL) manages an interferon-gamma (IFN-γ) enzyme-linked immunospot (ELISpot) external proficiency program. The ELISpot program evaluates the accuracy and variability of results across laboratories. The variability in the program is quantified via the dispersion, which is the ratio of the variance over the mean of the background-corrected spot-forming cells (SFC) replicates obtained under stimulation with different peptide pools (CMV, CEF). This report includes the longitudinal analysis of the ELISpot program cohort composed of 22 laboratories from 2011 to 2022 to assess whether the within-lab variability has improved over time. Random intercept models of the dispersion over time showed a significant decrease in overall dispersion from an average of approximately 1.8 in 2011 to approximately 1.25 in 2022. Out of the 21 sites, 16 sites (4 being statistically significant) had a negative trend for dispersion over time. Our finding of a reduction of overall within-lab variability demonstrates the need for and benefit of proficiency testing programs.

Simultaneous determination of macrolide and quinolone antibiotics in animal tissues for human consumption validated by Regulation (EU) 2021/808

In this work, an analytical method for the simultaneous detection and quantification of 11 macrolides (erythromycin A, gamithromycin, lincomycin, neospiramycin, pirlimycin, spiramycin, tildipirosin, tilmicosin, tulathromycin A and its metabolite CP-60300, and tylosin A) and 10 quinolones (ciprofloxacin, danofloxacin, difloxacin, enrofloxacin, flumequine, marbofloxacin, nalidixic acid, norfloxacin, oxolinic acid, and sarafloxacin) in muscle (bovine, ovine, porcine, equine, rabbit, poultry including laying hens, and fish) and kidney (bovine, ovine, porcine, equine) for human consumption was developed and validated according to Commission Implementing Regulation (EU) 2021/808. Sample preparation consisted of liquid extraction using a citrate–phosphate/EDTA buffer solution (McIlvaine buffer) followed by solid phase extraction for cleanup and UHPLC-MS/MS analysis. The developed method was comprehensively validated with regard to relative matrix effects (CVMF ≤ 20 %), linearity (R2 ≥ 0.98), accuracy (84–119 %), trueness (−16 to + 19 %), repeatability (2.1–20.0 %), reproducibility (3.0–25.0 %), selectivity, and robustness. The method limit of quantification values ranged from 2 to 40 µg kg−1, established as 0.1 × the lowest maximum residue limit (MRL) by species for the authorized antibiotics. Method applicability was demonstrated in six interlaboratory proficiency tests, and in the official control analyses of 1200 field samples of muscle and kidney.

Readability Of Higher Education Institutions Exam Basic Proficiency Test Turkish Questions

Readability is a measurable quantitative feature of texts. Readability gives information about how easy or difficult a text is to read. This information can help decide appropriate texts for courses and exams. There are 3 formulas, called Ateşman, Çetinkaya-Uzun, and Bezirci-Yılmaz, which are widely used in determining the readability of Turkish texts. The purpose of this study is to determine the readability of the Turkish questions in the Higher Education Institutions Exam, the Basic Proficiency Test conducted between 2019-2023 with Ateşman and Bezirci-Yılmaz formulas. The dataset consists of a total of 200 questions, 40 from each year, asked between 2019 and 2023. According to the findings, it was revealed that both formulas used in the study were consistent and the readability levels of the formulas were similar. According to the used formulas, it was observed that the questions in 2019 were the easiest and the test questions in 2021 were the most difficult to read. It has been determined that the number of questions in question types varies over the years, phonics questions are the question type with the easiest, and questions related to sentence meaning are the most difficult readability level. It was concluded that the questions were mainly at the difficult level according to the Ateşman formula, and at the academic level according to the Bezirci-Yılmaz formula.