Difficulty Of Questions Research Articles

Comparing the performance of artificial intelligence learning models to medical students in solving histology and embryology multiple choice questions

IntroductionThe appearance of artificial intelligence language models (AI LMs) in the form of chatbots has gained a lot of popularity worldwide, potentially interfering with different aspects of education, including medical education as well. The present study aims to assess the accuracy and consistency of different AI LMs regarding the histology and embryology knowledge obtained during the 1st year of medical studies. MethodsFive different chatbots (ChatGPT, Bing AI, Bard AI, Perplexity AI, and ChatSonic) were given two sets of multiple-choice questions (MCQs). AI LMs test results were compared to the same test results obtained from 1st year medical students. Chatbots were instructed to use revised Bloom’s taxonomy when classifying questions depending on hierarchical cognitive domains. Simultaneously, two histology teachers independently rated the questions applying the same criteria, followed by the comparison between chatbots’ and teachers’ question classification. The consistency of chatbots’ answers was explored by giving the chatbots the same tests two months apart. ResultsAI LMs successfully and correctly solved MCQs regarding histology and embryology material. All five chatbots showed better results than the 1st year medical students on both histology and embryology tests. Chatbots showed poor results when asked to classify the questions according to revised Bloom’s cognitive taxonomy compared to teachers. There was an inverse correlation between the difficulty of questions and their correct classification by the chatbots. Retesting the chatbots after two months showed a lack of consistency concerning both MCQs answers and question classification according to revised Bloom’s taxonomy learning stage. ConclusionDespite the ability of certain chatbots to provide correct answers to the majority of diverse and heterogeneous questions, a lack of consistency in answers over time warrants their careful use as a medical education tool.

Difficulty-controllable question generation over knowledge graphs: A counterfactual reasoning approach

Difficulty-controllable question generation (DCQG) over knowledge graphs aims to generate questions with a given subgraph and a difficulty label, such as “easy” or “hard.” However, three significant challenges currently confront DCQG: (1) limited modes for modeling difficulty, (2) the inability to ensure causality between difficulty labels and generated outcomes, and (3) lack of difficulty-annotated datasets. To overcome these challenges, we present DiffQG, a DCQG model that uses soft templates and counterfactual reasoning. DiffQG utilizes a mixture of experts as soft template selectors to enhance the diversity of difficulty representation. Soft templates can efficiently capture the similarity among questions of different difficulties, avoiding the need for constructing explicit templates. A disentanglement module is introduced to separate triple representations in the input subgraph that are pertinent and extraneous to the current question’s difficulty. Disentanglement minimizes the interference of irrelevant information on the generated output in neural networks due to entanglement. More importantly, disentangled representations enable the model to create training samples for counterfactual reasoning, strengthening causality between inputs and outputs. Additionally, we propose a question difficulty estimation method that simultaneously considers the input subgraph, question, and answering process. Extensive experiments reveal that our model can successfully generate questions at desired difficulty levels, surpassing the baselines by at least 8% in terms of difficulty control. Furthermore, DiffQG exhibits superior generalizability and interpretability.

The efficacy of artificial intelligence in urology: a detailed analysis of kidney stone-related queries

PurposeThe study aimed to assess the efficacy of OpenAI's advanced AI model, ChatGPT, in diagnosing urological conditions, focusing on kidney stones.Materials and methodsA set of 90 structured questions, compliant with EAU Guidelines 2023, was curated by seasoned urologists for this investigation. We evaluated ChatGPT's performance based on the accuracy and completeness of its responses to two types of questions [binary (true/false) and descriptive (multiple-choice)], stratified into difficulty levels: easy, moderate, and complex. Furthermore, we analyzed the model's learning and adaptability capacity by reassessing the initially incorrect responses after a 2 week interval.ResultsThe model demonstrated commendable accuracy, correctly answering 80% of binary questions (n:45) and 93.3% of descriptive questions (n:45). The model's performance showed no significant variation across different question difficulty levels, with p-values of 0.548 for accuracy and 0.417 for completeness, respectively. Upon reassessment of initially 12 incorrect responses (9 binary to 3 descriptive) after two weeks, ChatGPT's accuracy showed substantial improvement. The mean accuracy score significantly increased from 1.58 ± 0.51 to 2.83 ± 0.93 (p = 0.004), underlining the model's ability to learn and adapt over time.ConclusionThese findings highlight the potential of ChatGPT in urological diagnostics, but also underscore areas requiring enhancement, especially in the completeness of responses to complex queries. The study endorses AI's incorporation into healthcare, while advocating for prudence and professional supervision in its application.

Implementation of the Problem-Based Learning Model in Teaching of Mathematics about Adding Fractions with Different Denominators in Elementary Schools

Research has been carried out to find the level of student ability and the level of difficulty of essay questions on fractions, through polytomous data analysis by applying the Rasch model. The research was conducted in the In-Service Professional Teacher Education program, specifically in the Elementary School Teacher Education. The research was conducted at Belang 1 State Elementary School, Southeast Minahasa Regency, North Sulawesi, Indonesia, in grade 6 with a total of 27 students. Experimental research method with one group pretest-posttest design. The experimental treatment is a Problem-Based Learning (PBL) learning model. The test instrument is in the form of an essay with material on adding fractions with different denominators. The instrument was content validated through a video conference forum attended by experts, tutors, and subject teachers. Data was collected through formative pre-tests and post-tests. Scores are processed as polytomy data. Data analysis used the Rasch model with the Ministep 4.3.2 software application. The results of the analysis, as a class group, the pre-test average was 7.0 and the post-test was 77.0. This group's data shows that the use of the Problem-Based Learning (PBL) model in mathematics learning is very effective. Another conclusion is that the Rasch Model can accurately analyze polytomous data and find 1 student who has a high ability exceeding the highest level of test item difficulty in completing the essay test on adding fractions with different denominators. Then the Rasch Model can analyze the level of difficulty of the 5 items of the essay test instrument, it was found that 2 test items had a low level of difficulty. Using the Rasch model, it is consistently detected that adding fractions with different denominators containing the number 7 is categorized as having a high level of difficulty.

Can ChatGPT pass the MRCP (UK) written examinations? Analysis of performance and errors using a clinical decision-reasoning framework

ObjectiveLarge language models (LLMs) such as ChatGPT are being developed for use in research, medical education and clinical decision systems. However, as their usage increases, LLMs face ongoing regulatory concerns....

Advancing Interview Preparation: An AI-driven Approach

This research paper presents a transformative approach to interview preparation through the integration of cutting-edge artificial intelligence and personalized learning techniques. In today's competitive job market, traditional methods of interview readiness often lack the immediacy and tailored support needed for success. Addressing this gap, our platform utilizes advanced AI algorithms to analyze user responses, offer real-time feedback, and dynamically adjust question difficulty based on performance. By prioritizing user-centric design and ethical considerations, our research explores the development, implementation, and evaluation of this innovative solution. Insights from user feedback and performance metrics underscore the platform's effectiveness in enhancing interview skills and fostering confidence among job seekers. This paper contributes to the broader discourse on AI-driven learning technologies and their potential to empower individuals in navigating the complexities of professional advancement. Key Words: Interview Preparation, Artificial Intelligence, Natural Language Processing, Personalized Learning, Skill Development.

Pre-training Question Embeddings for Improving Knowledge Tracing with Self-supervised Bi-graph Co-contrastive Learning

Learning high-quality vector representations (aka. embeddings) of educational questions lies at the core of knowledge tracing (KT), which defines a task of estimating students’ knowledge states by predicting the probability that they correctly answer questions. Although existing KT efforts have leveraged question information to achieve remarkable improvements, most of them learn question embeddings by following the supervised learning paradigm. In this article, we propose a novel question embedding pre-training method for improving knowledge tracing with self-supervised Bi -graph Co -contrastive learning ( BiCo ). Technically, on the basis of self-supervised learning paradigm, we first select two similar but distinct views (i.e., representing objective and subjective semantic perspectives) as the semantic source of question embeddings. Then, we design a primary task (structure recovery) together with two auxiliary tasks (question difficulty recovery and contrastive learning) to further enhance the representativeness of questions. Finally, extensive experiments conducted on two real-world datasets show BiCo has a higher expressive power that enables KT methods to effectively predict students’ performances.

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis.

The United States Medical Licensing Examination (USMLE) has been critical in medical education since 1992, testing various aspects of a medical student's knowledge and skills through different steps, based on their training level. Artificial intelligence (AI) tools, including chatbots like ChatGPT, are emerging technologies with potential applications in medicine. However, comprehensive studies analyzing ChatGPT's performance on USMLE Step 3 in large-scale scenarios and comparing different versions of ChatGPT are limited. This paper aimed to analyze ChatGPT's performance on USMLE Step 3 practice test questions to better elucidate the strengths and weaknesses of AI use in medical education and deduce evidence-based strategies to counteract AI cheating. A total of 2069 USMLE Step 3 practice questions were extracted from the AMBOSS study platform. After including 229 image-based questions, a total of 1840 text-based questions were further categorized and entered into ChatGPT 3.5, while a subset of 229 questions were entered into ChatGPT 4. Responses were recorded, and the accuracy of ChatGPT answers as well as its performance in different test question categories and for different difficulty levels were compared between both versions. Overall, ChatGPT 4 demonstrated a statistically significant superior performance compared to ChatGPT 3.5, achieving an accuracy of 84.7% (194/229) and 56.9% (1047/1840), respectively. A noteworthy correlation was observed between the length of test questions and the performance of ChatGPT 3.5 (ρ=-0.069; P=.003), which was absent in ChatGPT 4 (P=.87). Additionally, the difficulty of test questions, as categorized by AMBOSS hammer ratings, showed a statistically significant correlation with performance for both ChatGPT versions, with ρ=-0.289 for ChatGPT 3.5 and ρ=-0.344 for ChatGPT 4. ChatGPT 4 surpassed ChatGPT 3.5 in all levels of test question difficulty, except for the 2 highest difficulty tiers (4 and 5 hammers), where statistical significance was not reached. In this study, ChatGPT 4 demonstrated remarkable proficiency in taking the USMLE Step 3, with an accuracy rate of 84.7% (194/229), outshining ChatGPT 3.5 with an accuracy rate of 56.9% (1047/1840). Although ChatGPT 4 performed exceptionally, it encountered difficulties in questions requiring the application of theoretical concepts, particularly in cardiology and neurology. These insights are pivotal for the development of examination strategies that are resilient to AI and underline the promising role of AI in the realm of medical education and diagnostics.

Pentingnya Meningkatkan Pengaruh Kemampuan Komunikasi Matematis dan Faktor yang Mempengaruhi Kemampuan Komunikasi Matematis Siswa

Students’ ability to communicate their mathematical ideas in various ways is not as expected. It is difficult for teachers to improve students’ mathematical communication skills by improving learning styles, improving the learning atmosphere, and using appropriate learning models. Therefore, it is important to know the factors that influence students’ mathematical communication abilities. The aim of this research is to describe the factors that influence students’ mathematical communication skills. The subjects of this research were four children aged approximately 11 years who live in Hamparan Perak District. The data collection techniques used were observation and interviews. Data were analyzed using a qualitative approach. The results of the research show that the factors that influence students’ oral communication skills are study partners, self-confidence, shyness, nervousness, group learning atmosphere, learning time, and gender. Factors that influence students’ written communication skills include the inability to write down known information and be asked about it, lack of concentration in studying and inability to understand the content, as well as question difficulty, which includes the level of composure and composure in solving questions.

Exploring Multiple-Objective Optimization for Efficient and Effective Test Paper Design with Dynamic Programming Guided Genetic Algorithm.

Automatic test paper design is critical in education to reduce workloads for educators and facilitate an efficient teaching process. However, current designs fail to satisfy the realistic teaching requirements of educators, including the consideration of both test quality and efficiency. This is the main reason why teachers still manually construct tests in most teaching environments. In this paper, the quality of tests is quantitatively defined while considering multiple objectives, including a flexible coverage of knowledge points, cognitive levels, and question difficulty. Then, a model based on the technique of linear programming is delicately designed to explore the optimal results for this newly defined problem. However, this technique is not efficient enough, which cannot obtain results in polynomial time. With the consideration of both test quality and generation efficiency, this paper proposes a genetic algorithm (GA) based method, named dynamic programming guided genetic algorithm with adaptive selection (DPGA-AS). In this method, a dynamic programming method is proposed in the population initialization part to improve the efficiency of the genetic algorithm. An adaptive selection method for the GA is designed to avoid prematurely falling into the local optimal for better test quality. The question bank used in our experiments is assembled based on college-level calculus questions from well-known textbooks. The experimental results show that the proposed techniques can construct test papers with both high effectiveness and efficiency. The computation time of the test assembly problem is reduced from 3 hours to 2 seconds for a 5000-size question bank as compared to a linear programming model with similar test quality. The test quality of the proposed method is better than the other baselines.

Research on personalized recommendation of teaching resources based on joint probability matrix decomposition model and CNN improvement algorithm

Abstract This study proposes a teaching resource recommendation method (TRRDLMF) based on deep learning and probabilistic matrix decomposition, aiming to improve teaching resource recommendation’s accuracy and personalization level. The study combines hybrid neural network feature extraction of teaching resources and user feature extraction based on extended noise-reducing self-encoder and extended probabilistic matrix decomposition-based teaching resource recommendation technique. The study validates the model’s effectiveness by conducting empirical analyses on four online learning platform datasets of different sizes. The recommendation system can effectively track learners’ knowledge mastery status and learning preferences, and provide corresponding teaching resources. Test question similarity and difficulty analysis results show that the model can accurately capture the correlation between test questions and provide complementary data in education. The personalized recommendation analysis reveals the learners’ knowledge preference states that change with learning, demonstrating the model’s ability to give explanations at the level of learners’ knowledge preferences. The analysis of the impact of data size on recommendation results shows that the recommender system can achieve up to 73.12% accuracy and 97.89% recall for different TOP-K recommendation lists, with the best F-value at TOP-6. The personalized recommendation system for teaching resources based on the joint probability matrix decomposition model and CNN improvement algorithm proposed by the research demonstrates significant effects in improving the accuracy and personalization level of teaching resources recommendation, which helps learners to acquire suitable learning resources more efficiently, and has high practical value and promotion potential.

Performance of ChatGPT incorporated chain-of-thought method in bilingual nuclear medicine physician board examinations.

This research explores the performance of ChatGPT, compared to human doctors, in bilingual, Mandarin Chinese and English, medical specialty exam in Nuclear Medicine in Taiwan. The study employed generative pre-trained transformer (GPT-4) and integrated chain-of-thoughts (COT) method to enhance performance by triggering and explaining the thinking process to answer the question in a coherent and logical manner. Questions from the Taiwanese Nuclear Medicine Specialty Exam served as the basis for testing. The research analyzed the correctness of AI responses in different sections of the exam and explored the influence of question length and language proportion on accuracy. AI, especially ChatGPT with COT, exhibited exceptional capabilities in theoretical knowledge, clinical medicine, and handling integrated questions, often surpassing, or matching human doctor performance. However, AI struggled with questions related to medical regulations. The analysis of question length showed that questions within the 109-163 words range yielded the highest accuracy. Moreover, an increase in the proportion of English words in questions improved both AI and human accuracy. This research highlights the potential and challenges of AI in the medical field. ChatGPT demonstrates significant competence in various aspects of medical knowledge. However, areas like medical regulations require improvement. The study also suggests that AI may help in evaluating exam question difficulty and maintaining fairness in examinations. These findings shed light on AI role in the medical field, with potential applications in healthcare education, exam preparation, and multilingual environments. Ongoing AI advancements are expected to further enhance AI utility in the medical domain.

Racking Analysis Instrument Mastery Test Concepts in Learning Science Using the RADEC Model in Elementary School Students

The study's objective was to assess the content's validity, reliability, and degree of difficulty for third-grade students in one of the East Jakarta elementary schools utilizing the RADEC learning model by analyzing the pre-test and post-test items using the Rasch Racking model of analysis. The results of the Radec learning model showed that students with low and high abilities experienced a change in the concept of student effectiveness for the better. This research is a quantitative study, namely the evaluation of learning outcomes using descriptive quantitative methods with data collection techniques using pre-test and post-test questions. In early 2023, The learning activity was conducted using data from 32 student respondents. The result indicated the validity of the suitability level of the item, with ten questions found to be valid. The difficulty level of the questions, in general, is in the medium category, meaning that they are excellent because they are relatively straightforward. The conclusion is that the analysis of the items using the Rasch model in the learning evaluation activities through the pre-test and post-test is valid with excellent categories of 50% and good 50%. The question's difficulty level is excellent

Progress in International Reading Literacy Study-Based Reading Competency Assessment Instrument with Culturally Educative Texts for Primary School Students

The objectives of this research and development are (1) to produce a product prototype of the PIRLS equivalent reading literacy competency assessment instrument with educative cultural content text for grade 6 elementary school students, (2) to test the PIRLS equivalent reading literacy competency assessment instrument with educative cultural content text for grade 6 elementary school students from the aspects of content validity, construct validity, Esay question reliability test, Objective question reliability test, difficulty level, and differentiability. This type of research and development uses the Borg & Gall model which consists of 10 stages. The data collected is qualitative and quantitative data. The data is related to content validity, construct validity, readability, and practicality of the assessment instrument as well as the practicality of the user manual. The collected data were analyzed with content validity (Gregory), Product Moment, and Cronbach's Alpha. Based on the results of the analysis and discussion, the results of the correlation person analysis for the overall r value of 30 items > r table for subject 28 the r table value is 0.361 with the acquisition of the smallest value of 0.383 and the largest of 0.724. In addition to the results of the limited subject test, the results of the analysis of the broad subject test using SPSS for windows version 26 software also showed the results that the person correlation value for the reading literacy competency assessment instrument from a total of 30 items obtained a calculated r value> r table for subjects 141 people the r table value of 0.361 with the acquisition of the smallest value of 0.439 and the largest value of 0.896. While the acquisition of the teacher response score is 0.882. So, based on these results it can be said that the product of the reading literacy competency assessment instrument equivalent to the Progress in International Reading Literacy Study with educational cultural content text is suitable for direct use for every grade 6 elementary school student.

OpenAI's GPT-4 performs to a high degree on board-style dermatology questions.

Artificial intelligence tools such as OpenAI's GPT-4 have shown promise in medical education, but their potential in dermatology remains unexplored. To assess GPT-4's performance on dermatology board-style questions and determine its value as a supplementary educational tool for trainees and educators. This cross-sectional study evaluated GPT-4's performance on 250 random dermatology board-style questions sampled from the American Academy of Dermatology's Board Prep Plus resource. Questions were divided into five subspecialties and various difficulty levels. GPT-4 responses were compared to the correct answers and evaluated by two physicians. GPT-4 achieved an overall accuracy of 75% on the 250 questions, with no significant variation based on subspecialty or question difficulty. The most common errors were factual and misunderstanding inaccuracies. Responses scored high in clarity, accuracy, and relevance but frequently lacked depth and completeness. GPT-4 performed to a high degree and demonstrated promising performance as an educational adjunct in dermatology. Improvements in response depth and completeness are needed before its use as an unsupervised learning tool is established.

ChatGPT's performance in German OB/GYN exams - paving the way for AI-enhanced medical education and clinical practice.

Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial learning and large language model tool developed by OpenAI in 2022. It utilizes deep learning algorithms to process natural language and generate responses, which renders it suitable for conversational interfaces. ChatGPT's potential to transform medical education and clinical practice is currently being explored, but its capabilities and limitations in this domain remain incompletely investigated. The present study aimed to assess ChatGPT's performance in medical knowledge competency for problem assessment in obstetrics and gynecology (OB/GYN). Two datasets were established for analysis: questions (1) from OB/GYN course exams at a German university hospital and (2) from the German medical state licensing exams. In order to assess ChatGPT's performance, questions were entered into the chat interface, and responses were documented. A quantitative analysis compared ChatGPT's accuracy with that of medical students for different levels of difficulty and types of questions. Additionally, a qualitative analysis assessed the quality of ChatGPT's responses regarding ease of understanding, conciseness, accuracy, completeness, and relevance. Non-obvious insights generated by ChatGPT were evaluated, and a density index of insights was established in order to quantify the tool's ability to provide students with relevant and concise medical knowledge. ChatGPT demonstrated consistent and comparable performance across both datasets. It provided correct responses at a rate comparable with that of medical students, thereby indicating its ability to handle a diverse spectrum of questions ranging from general knowledge to complex clinical case presentations. The tool's accuracy was partly affected by question difficulty in the medical state exam dataset. Our qualitative assessment revealed that ChatGPT provided mostly accurate, complete, and relevant answers. ChatGPT additionally provided many non-obvious insights, especially in correctly answered questions, which indicates its potential for enhancing autonomous medical learning. ChatGPT has promise as a supplementary tool in medical education and clinical practice. Its ability to provide accurate and insightful responses showcases its adaptability to complex clinical scenarios. As AI technologies continue to evolve, ChatGPT and similar tools may contribute to more efficient and personalized learning experiences and assistance for health care providers.

Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study.

The reliability of GPT-4, a state-of-the-art expansive language model specializing in clinical reasoning and medical knowledge, remains largely unverified across non-English languages. This study aims to compare fundamental clinical competencies between Japanese residents and GPT-4 by using the General Medicine In-Training Examination (GM-ITE). We used the GPT-4 model provided by OpenAI and the GM-ITE examination questions for the years 2020, 2021, and 2022 to conduct a comparative analysis. This analysis focused on evaluating the performance of individuals who were concluding their second year of residency in comparison to that of GPT-4. Given the current abilities of GPT-4, our study included only single-choice exam questions, excluding those involving audio, video, or image data. The assessment included 4 categories: general theory (professionalism and medical interviewing), symptomatology and clinical reasoning, physical examinations and clinical procedures, and specific diseases. Additionally, we categorized the questions into 7 specialty fields and 3 levels of difficulty, which were determined based on residents' correct response rates. Upon examination of 137 GM-ITE questions in Japanese, GPT-4 scores were significantly higher than the mean scores of residents (residents: 55.8%, GPT-4: 70.1%; P<.001). In terms of specific disciplines, GPT-4 scored 23.5 points higher in the "specific diseases," 30.9 points higher in "obstetrics and gynecology," and 26.1 points higher in "internal medicine." In contrast, GPT-4 scores in "medical interviewing and professionalism," "general practice," and "psychiatry" were lower than those of the residents, although this discrepancy was not statistically significant. Upon analyzing scores based on question difficulty, GPT-4 scores were 17.2 points lower for easy problems (P=.007) but were 25.4 and 24.4 points higher for normal and difficult problems, respectively (P<.001). In year-on-year comparisons, GPT-4 scores were 21.7 and 21.5 points higher in the 2020 (P=.01) and 2022 (P=.003) examinations, respectively, but only 3.5 points higher in the 2021 examinations (no significant difference). In the Japanese language, GPT-4 also outperformed the average medical residents in the GM-ITE test, originally designed for them. Specifically, GPT-4 demonstrated a tendency to score higher on difficult questions with low resident correct response rates and those demanding a more comprehensive understanding of diseases. However, GPT-4 scored comparatively lower on questions that residents could readily answer, such as those testing attitudes toward patients and professionalism, as well as those necessitating an understanding of context and communication. These findings highlight the strengths and limitations of artificial intelligence applications in medical education and practice.

Analisis Butir Soal Ulangan Harian Semester Gasal Menggunakan Google Formulir di SMA Ipiems Surabaya

This research aims to (1) determine the quality of the daily test items for the even semester of class The sample for this research was students of class X Merdeka 5 at SMA Ipiems Surabaya with 33 students as respondents. This research instrument includes multiple choice questions and answer keys resulting from students' work, then tabulated analysis, validity, reliability, difficulty of questions, and differentiability of question items using the help of Microsoft Excel software. The results of this research show that of the 20 multiple choice questions, 21 students completed the multiple choice questions with a reliability value of r11 0.611, rtable level 1% 0.424, 1% reliable category, and the item difficulty index obtained was a value of 11 items in the medium category, 9 questions in the easy category, and 0 questions in the difficult category.

Multiple Choice Question Difficulty Level Classification with Multi Class Confusion Matrix in the Online Question Bank of Education Gallery

Multiple Choice Question Difficulty Level Classification with Multi Class Confusion Matrix in the Online Question Bank of Education Gallery

Arabic Listening Skill Test of 'Al-Arabiya Baina Yadaik Volume 1 and 2 Through Bloom's Taxonomy

This research aims to explore and explain the evaluation of Arabic language listening skills tests based on Bloom's Taxonomy, primarily focusing on utilizing the 'al-Arabiya Baina Yadaik' books in the first two volumes. This study utilizes a qualitative approach, specifically a descriptive analytical method, when the data is derived from literary sources. The primary focus of the data is 'Al-Arabiya Baina Yadaik Volumes 1 and 2, the books that serve as the focal point of analysis to understand content related to listening skills in Arabic. The data was obtained through an in-depth study of the content of 'Al-Arabiya Baina Yadaik books. The data collection method involves a comprehensive analysis of the text to identify question structures and difficulty levels according to Bloom's taxonomy. The analysis results indicate that listening skills tests based on Bloom's taxonomy, 'al-Arabiya Baina Yadaik' Volumes 1 and 2, suit beginner learners. In Volume 1, the remembering level (C1) has five types of test questions (33%), while the understanding level (C2) has 11 types (67%). In Volume 2, Remembering (C1) has one question (10%), Understanding (C2) has five questions (50%), Analysis (C4) has two questions (20%), and Evaluation (C6) has two questions (20%). This research contributes significantly to developing Arabic language listening skills tests and offers valuable guidance for educators in selecting appropriate teaching materials.