ALIGNING HIGH SCHOOL ENTRANCE EXAMS WITH CURRICULUM GOALS: A STUDY ON THE DISTRIBUTION AND COGNITIVE LEVELS OF SCIENCE QUESTIONS

This study aims to evaluate exam questions set by science teachers for eighth grade students and science questions from a central high school entrance exam (HSEE) according to the Revised Bloom’s Taxonomy (RBT). In this study, document analysis technique was employed, as one of the recognized methods of qualitative research. The HSEE science questions and the teacher-prepared exam questions were evaluated separately in the dimensions of knowledge and cognitive process, and the frequency and percentage distribution of the questions were examined according to the RBT. The science teachers’ exam questions were found to be the most suitable for factual knowledge in the RBT knowledge dimension, and the most appropriate for the remembering and understanding levels of the cognitive process dimension. It was determined that the HSEE science questions were the most suitable for conceptual and procedural types of knowledge in the RBT knowledge dimension, and for the understanding and analyzing levels in the cognitive process dimension. Both the questions prepared by teachers and the HSEE science questions were not homogeneously distributed in terms of the RBT. It was determined that while the science teachers’ exam questions were at the lower level of the cognitive process, the HSEE science questions were at a level higher than those prepared by the teachers.

Bu çalışmanın amacı, 2015- 2020 yılları arasında ulusal düzeyde gerçekleştirilmiş olan ortaöğretime geçiş sınavlarındaki fen bilimleri dersi sorularının MEB kazanımlarına, PISA yeterlik seviyelerine ve Yenilenmiş Bloom taksonomisine (YBT) göre analizini ortaya koymaktır. Bu amaç eşliğinde, nitel araştırma metotlarından olan doküman analizi ile veriler toplanmıştır. Elde edilen veriler, önceden süregelen kategori veya boyutlar gözetilerek özetlenmesi ve yorumlanmasına dayalı betimsel analizi ile çözümlenmiştir. Araştırmada 2015-2020 yılları arasında yapılan ortaöğretim geçiş sınavlarında sorulan fen bilimleri soruları, 2013 ile 2018 senelerindeki fen bilimleri dersi öğretim programı doküman olarak kullanılmıştır. Araştırmaya dahil edilen TEOG sınavları kapsamında 80, LGS sınavları kapsamında 60 fen bilimleri dersi sınav sorusu bulunmaktadır. 2013 senesi Fen Bilimleri Öğretim Programında, 8. sınıfta ise 78, 2018 yılı Fen Bilimleri Dersi Öğretim Programında ise 8. sınıfta 61 tane kazanımın olduğu belirlenmiştir. Araştırma kapsamında 140 fen bilimleri sorusu ve bu sorulara karşılık gelen 128 kazanım betimsel analiz kullanılarak çözümlenmiştir. Araştırmanın sonuçları 2015-2020 yılları arasında ulusal çapta düzenlenen ortaöğretime geçiş sınavlarında sorulan MEB kazanımları ile uyumlu olduğuna işaret etmektedir. Bunun yanı sıra soruların PISA yeterlik düzeyleri bağlamında 2. ve 3. Düzeylerde yoğunlaştığı tespit edilmiştir. 2015-2020 yılları arasında sorulan fen bilimleri dersi sorularının Yenilenmiş Bloom taksonomisinin kavrama bilgi boyutu, anlama ve uygulama biliş düzeyinde yoğunlaştığı saptanmıştır. Araştırmanın sonuçları bağlamında; ulusal sınavlarda YBT’ye göre işlemsel ve metabilişsel bilgi düzeyinde, uygulama ve çözümleme biliş düzeyinde soruların sayılarının arttırılması önerilmektedir. Yine ulusal sınavlarda PISA yeterlik seviyelerine göre üst düzey soruların temsilinin arttırılması önerilmektedir.

In this study, the relationship between the levels of 5th grade science course exam questions and the 5th class learning outcomes of the science curriculum in the revised Bloom taxonomy was examined. The research was carried out using document analysis method. Since the revised Bloom taxonomy categories were used for the analysis, the data obtained were analyzed with the descriptive analysis technique. The study included 967 science questions and 40 learning outcomes in the 2017-2018 academic year. These questions and learning outcomes were analyzed. At the end of analysis, the relationship between the learning outcomes and exam questions was determined. The inter-rater reliability computing has been made in the analysis of questions-learning outcomes. The reliability co-efficient was calculated .81 for learning outcomes and .77 for questions, indicating an acceptable reliability. According to the results of the analysis, it was determined that the most learning outcomes were in the conceptual knowledge dimension and the most questions were included in the factual knowledge dimension. In the cognitive processes dimension, it was determined that most learning outcomes are at the level of understanding, and the most questions are at the level of remembering. It is understood that 37% of the exam questions are at the level of learning outcomes. In addition, it was determined that there were no questions about some learning outcomes (24%).

The aim of this study was to determine the relationship between the mathematics questions asked in the High School Entrance Exam (LGS) and the 8th grade objectives of the Primary Education Mathematics Course Curriculum, and to reveal at which level cognitive processes are measured through these questions according to the revised Bloom taxonomy. Within the scope of the study, 20 mathematics questions in the 2021 LGS and 52 8th grade objectives based on the formation of these questions were analyzed by document analysis, one of the qualitative research methods. The questions and achievements were classified independently by the researcher teacher and a field expert, and descriptive statistics were used in the analysis of the data. The data obtained showed that the 2021 LGS mathematics questions were operational and metacognitive according to the knowledge dimension of the revised Bloom taxonomy, while the 8th grade mathematics objectives based on the formation of these questions were factual, conceptual and operational. In addition, it was concluded that most of the 2021 LGS mathematics questions measured the cognitive processes in the application and analysis steps in the cognitive dimension of the revised Bloom taxonomy, while the 8th grade mathematics objectives focused on the understanding and application steps. It is recommended that the mathematics questions in the high school entrance exams and the mathematics course objectives should be reviewed according to the revised Bloom's taxonomy metacognitive dimensions

Exam questions as a test instrument to measure educational success must have good quality. This quality can be measured by the level of cognition expected of students. The level of cognition reflects the mastery of learning materials as a form of evaluation of the teaching and learning process outlined in the curriculum. For this reason, the exam questions need to be mapped into the cognitive process dimensions, namely based on the categories in the Revised Bloom's Taxonomy (RBT). The mapping method used is Natural Language Processing (NLP) as one of the fields of Artificial Intelligence development. The stages in this mapping are pre-processing, including tokenization, stemming, stop-word removal, and feature extraction using POS Tagging. The output of this mapping process is in the form of categories of test items into RBT: remembering (C1), understanding (C2), applying (C3), analyzing (C4), evaluating (C5), and creating (C6). The classification results obtained information that the exam questions prepared were still dominated at the C2 cognitive level, which was indicated by the use of operational verbs in the understanding category. The results of testing the method used produce an accuracy of 82.22%. Thus, the NLP method can classify test items into Revised Bloom's Taxonomy to determine the dimensions of students' cognitive processes.

Bu araştırma, Kamu Personeli Seçme Sınavı (KPSS) Öğretmenlik Alan Bilgisi Testlerindeki (ÖABT) Analitik Kimyayla ilişkili soruları Yenilenmiş Bloom Taksonomisine (YBT) göre analiz etmek amacıyla yapılmıştır. Bu kapsam-da 2013-2018 KPSS/ÖABT’ deki toplam 510 soru dokü-man inceleme tekniği ile incelenmiş ve Analitik Kimyayla ilişkili 83 soru analiz edilmiştir. Soruların 71’i Kimya, 12’si Fen Bilimleri/Fen ve Teknoloji ÖABT’ den alınmıştır. Soruların analizlerinde araştırmacı tarafından hazırlanan şablon kullanılarak tek tek analiz edilmiş ve iki boyutlu taksonomi tablosunda uygun kutucuğa yerleştirilmiştir. Yerleştirmede aynı anda birden fazla bilişsel süreç boyu-tunda olabileceği düşünülen sorularda daha üst düzeydeki boyutlar alınmıştır. Boyutların belirlenmesinde sorunun ad ve fiil kısımları dikkate alınmış olup ad kısmı bilgi boyutu-na, fiil kısmı bilişsel süreç boyutuna karşılık gelecek şekilde analizleri yapılmıştır. Elde edilen bulgulara göre soruların bilişsel süreç boyutları sırasıyla %8,4“Hatırlama”, %27,7 “Anlama”, %49,4” Uygulama”, %14,5 “Çözümleme” olduğu” “Değerlendirme” ve “Yaratma” boyutlarında soru bulunmadığı belirlenmiştir. Bilgi boyutları sırasıyla %7,2’i “Olgusal”, %31,3’ü “Kavramsal” ve %61,5’u “İşlemsel” boyutlarında olduğu, “Üstbilişsel” bilgi boyu-tunda soru bulunmadığı tespit edilmiştir. Soruların yıllara göre dağılımında “Üstbilişsel” boyutta en fazla soru 2015’de, alt bilişsel boyutta 2016’da olduğu belirlenmiştir. Sınav soruları hazırlanırken öncelikle hedeflerin belirlen-mesi, hedeflere rehberlik edecek taksonomilerin oluşturul-ması ve bunun için YBT tablosunun dikkate alınması öne-rilmektedir.

Students who are getting education in our country are subject to various exams in order to be placed in a higher education. Since 2013-2014 academic year, Transition from Primary Education to Secondary Education (TEOG) exam has started to be applied for secondary education. TEOG exam questions are prepared in such a way to include teaching program achievements/learning outcomes published by the Ministry of National Education. In this study, the relations between the levels of the TEOG exam science and technology course questions and the 8th class achievements of the Science and Technology course curriculum in the renewed Bloom taxonomy were examined. In the research, data were subjected to descriptive analysis using the document analysis method. A total of 120 Science and Technology questions of the TEOG exams held between 2013 and 2016 were included in the study but 118 of the questions were analyzed since two of the questions were canceled. At the end of the research, it was determined that 58 questions from the Science and Technology course of Semester I included in the TEOG exam were the questions related to the 31 learning outcomes of the total of 34 outcomes included in the examination program. It was observed that 60 questions from the Science and Technology course of Semester II were related to the 42 learning outcomes of the total of 96 outcomes included in the program. In addition, it was also determined that there was no question in TEOG related to some of the course units.

Abstract: Nine food science and technology (FST) subjects were assessed for alignment between the learning outcomes and assessment using revised Bloom's taxonomy (RBT) of cognitive knowledge. Conjoint analysis was used to estimate the utilities of the levels of cognitive, knowledge, and the attribute importance (cognitive process and knowledge dimension) for learning outcomes and assessments. Lecturers for these subjects produced learning outcomes for ability of students to Understand (4.935) Procedural (3.316) as well as Apply (4.491) Conceptual (3.083) knowledge. Lecturers’ expected students’ to move beyond mere recall and recognition of knowledge to higher order cognitive knowledge of apply, analyze, evaluate, and create. However, the assessments tested students’ ability to Understand (4.791) Conceptual (4.168) as well as Remember (3.217) Procedural (0.581) knowledge resulting in a misaligned teaching and learning exercise. For all the subjects, emphasis was more (52.9% to 72.9%) on the cognitive dimension than on forms of knowledge in formulating the learning outcomes, whereas emphasis placed on the cognitive (33.3% to 62.5%) dimension and the knowledge (37.5% to 66.7%) forms in the questions differed from subject to subject. The cognitive weight in the assessment was more for Understand (1.781)/Remember (0.787) Conceptual (1.416) knowledge. RBT provides an assessment framework that can be used to assist instructors in going beyond factual knowledge and comprehension to include academic skills such as application, analysis, evaluation, and creation.

The application of the revised Bloom's taxonomy in the development of chemistry problems in the domain of knowledge is still difficult for some chemistry teachers. This paper is a practical study of the preparation of learning objectives and the development of chemical items by applying the revised Bloom's taxonomy which includes three domains, namely knowledge (cognitive), attitude (affective), and skills (psychomotor). This paper is focused on the preparation of learning objectives and the development of chemistry items for the assessment of the domain of knowledge which includes the levels of remembering, understanding, applying, analyzing, evaluating, and creating. The six cognitive processes are sequential from the lowest cognitive processes to the highest cognitive processes called C1-C6 cognition processes. The methodology in this paper consists of three steps, namely examining the meaning of each level of cognition in the revised Bloom taxonomy, developing learning objectives that are in accordance with the cognitive level being studied and continuing with the arrangement of the problem items in chemistry subjects. The strengths of this study are very helpful for chemistry teachers in developing learning objectives and chemistry questions in the domain of knowledge based on the revised Bloom taxonomy, because this study is equipped with descriptions of each cognitive process, examples of learning objectives and appropriate question items. Examples of questions given in this paper are multiple choice types and essay tests, because each type of question has its advantages. The result of this study is that the application of the revised Bloom's taxonomy to the development of chemical problems requires an understanding of each level of the cognitive level in the revised Bloom's taxonomy, as the basis for developing learning objectives and the appropriate problem questions.

Abstract:This study aims to describe the distribution of cognitive levels of questions in the 2018 Revised Class XII Mathematics Textbook for Congruence and Similarity Based on Bloom's Taxonomy. Bloom's Taxonomy used is a revised Bloom's Taxonomy with cognitive process dimensions consisting of: remembering(C1), understanding(C2), applying(C3), analyzing(C4), evaluating(C5), and creating(C6). This type of research was descriptive research with a qualitative approach. Sources of data in this study were questions about the material congruence, similarity and competency test questions contained in the mathematics textbooks of class XII students. The research was carried out with the method of collecting documentation data and using research instruments in the form of analysis sheets and analysis conformity sheets. The results showed that the distribution of the cognitive level of questions on the congruence and similarity material contained the cognitive level of understanding(C2) as much as 18.2%, applying(C3) as much as 50% and analyzing(C4) as much as 31.8%, with the dominant cognitive level being cognitive level of applying(C3) and does not include cognitive levels of remembering(C1), evaluating(C2) and creating(C6). So it was found that the distribution of the cognitive level of the questions on the material of congruence and similarity topic did not vary or was not proportional. with the dominant cognitive level being cognitive level of applying(C3) and does not include cognitive levels of remembering(C1), evaluating(C2) and creating(C6). So it was found that the distribution of the cognitive level of the questions on the material of congruence and similarity topic did not vary or was not proportional. with the dominant cognitive level being cognitive level of applying(C3) and does not include cognitive levels of remembering(C1), evaluating(C2) and creating(C6). So it was found that the distribution of the cognitive level of the questions on the material of congruence and similarity topic did not vary or was not proportional.
 Keywords: Mathematics Textbook, Congruence, Similarity, Question, Revised Bloom's Taxonomy, Cognitive Level.

<p style="text-align:justify">The aim of this study is to compare 2018 Science Course Curriculum (SCC), 2015 Trends in International Mathematics and Science Study (TIMSS) and 2018 High School Entrance Examination (HSE) in terms of content domains, cognitive domains and learning objectives. Qualitative research method, was used in this study. Data were analyzed using document review matrices to determine the similarities and differences between the objectives of SCC, TIMSS and HSE. SCC outcomes and HSE science questions were also classified according to TIMSS cognitive domains. Results show that the learning objectives of the fields of Physics, Biology and Earth Sciences of TIMSS are compatible with those of all grade levels of SCC and that the objectives of Chemistry are compatible with those of the seventh and eighth grades. Most of HSE questions are compatible with the objectives of SCC, however, the latest revision in the curriculum has introduced some eighth grade objectives to other grade levels. HSE science questions measure higher-level skills than TIMSS science questions. The subject domain of the “Organisms and Life” of SCC has the most learning objectives in the levels of “knowing” and “reasoning” while the subject domain of the “Physical Events” has the most learning objectives in the levels of “applying.” Besides, the seventh-, fifth- and eighth-graders have the most objectives in the levels of “knowing,” “applying,” and “reasoning,” respectively. It is hoped that the results will contribute the literature in improvement of science curricula and interpretation of national and international exams.</p>

ABSTRACTIn this paper, the knowledge dimension for Revised Bloom's taxonomy (RBT) is unpacked for integral calculus. As part of this work, the 11 subtypes of the knowledge dimension are introduced, and through document analysis of chapter 4 of the RBT handbook, these subtypes are defined. Then, by consulting materials frequently used for teaching integral calculus, each subtype is exemplified. The developed dimension may enable or enhance opportunities for dialogue between lecturers, teachers, and researchers about how to develop and align educational objectives, teaching activities, and assessments in integral calculus, or how metacognition and metacognitive knowledge could be used to support teaching and learning.

This study evaluates the learning outcomes in secondary education English Curriculums regarding knowledge and cognitive process in Revised Bloom Taxonomy. Document analysis was adopted as a qualitative research method in data analysis. A total of 245 learning outcomes was studied, 52 of which were in grade 5, 60 in grade 6, 63 in grade 7 and 70 in Grade 8. As a result of research it was seen that most of the outcomes, belonging to 5, 6, 7, and 8th grade English teaching curriculum, were at the dimension of “conceptual knowledge” of Revised Bloom Taxonomy’s knowledge dimension. According to the conceptual knowledge dimension, the most outcomes were seen at the “applying”. An analysis of the results suggested that knowledge and cognitive process were not evenly distributed in English courses. The results pointed out that most of the learning outcomes accounted for “conceptual knowledge” in grade 5, 6, 7 and 8 level English courses. In terms of the “cognitive process”, the most frequent one is was the “applying” section. Applying increased as a learning outcome in grade 5, 6 and 7, whereas analysis and creation are were included in learning outcomes for grade 8 and advanced thinking. However, the cognitive process failed to increase in the learning outcomes from grade 5 to 7. Hence, it is possible to suggest that the distribution of learning outcomes be planned carefully based on the students’ levels

The purpose of this study is to describe the cognitive level of the questions in the eighth grade mathematics textbook on the material of the Pythagorean theorem, circles and tangents to circles according to the revised bloom taxonomy. In this study, the research subject was a competency test item contained in the Mathematics book for SMP / Mts Class VIII Semester 2 written by M Cholik Adinawan and published by Erlangga as many as 101 questions divided into 33 questions of competency 1, competency test 2 of 39 and competency test 3 as many as 29 questions. This research was conducted by analyzing the cognitive level of the questions based on the cognitive process in solving the questions. Based on the cognitive indicators of the revised bloom taxonomy, each cognitive level of questions was classified into six cognitive levels. The result of this research is the cognitive level in each competency test. The results of each cognitive level of the material pythagorean theorem C1 3.03%, C2 6.06%, C3 60.6%, C4 24.24%, C5 6.06%, and C6 0%. The results of the cognitive process level questions on the subject circle C1 2.56%, C2 25.64%, C3 51.28%, C4 20.51%, C5 and C6 0%. And the results of the cognitive process level questions on the subject tangent to circle C2 21.42%, C3 62.06%, C4 17.24% and C1, C5 and C6 0%. These results meet the achievement criteria for Basic Competence at C3 and C4 as much as 40%, do not meet the achievement for C1 and C2 as much as 30% and C5 and C6 as much as 30%. From the above results it can be concluded that the percentage of question analysis has not fulfilled the distribution of questions that support the achievement of basic competencies

Bloom's taxonomy is a classification of learning objectives originally developed for general educational purposes. The taxonomy was subsequently revised to expand beyond cognitive processes and to include an additional knowledge dimension. The revision was prompted by psychometric measurements indicating inconsistencies in the original taxonomy, and the revised taxonomy has been adapted for use in many disciplines. However, biology education researchers and practitioners continue to use the original taxonomy despite the concerns of validity. To facilitate the use of the revised Bloom's taxonomy in biology, we developed a discipline‐specific rubric from an analysis of assessment items from diverse biology disciplines.Implicit learning objectives of 1,432 assessment items were coded by two researchers. Items were collected from AP Biology, MCAT, and both introductory and advanced undergraduate biology courses. A biology‐specific rubric was generated using the revised taxonomy as a model. Problems were coded independently by two raters in batches of 50–100, followed by a consensus process. After coding 831 items, a revised rubric was developed and tested on the remaining 601 items. Inter‐rater reliability was 0.76 and 0.70 (Cohen's κ) for the cognitive‐process and knowledge dimensions respectively.Using thematic analysis as the methodology over two versions of our rubric, we identified distinctive features among biology assessment items that define the six cognitive processes and four types of knowledge in the revised Bloom's taxonomy. Quantitative analysis indicates that our data set contains problems mostly in the remember or understand categories (43.9% and 37.3% respectively) but rarely in the apply, analyze, evaluate, or create categories (7.0%, 5.2%, 4.7%, 2.0% respectively). These results are aligned with existing biology education research literature using the original Bloom's taxonomy. In the knowledge dimension, factual, conceptual, and procedural knowledge are over‐represented (38.4%, 48.7%, 12.8% respectively), with essentially no metacognitive knowledge (0.2%). This suggests a potential area for change in biology education, as metacognition is critical to how people learn.We performed statistical analyses on the 601 items coded with our revised rubric to examine the Bloom's taxonomy framework. Contingency analysis indicates that the cognitive processes and knowledge dimensions are not independent (χ2 = 249.05, df = 10, p < 0.0001). Correspondence analysis identifies two components that account for 100% of the inertia or variation in how the two dimensions are related, with three distinct clusters of knowledge and cognitive process: factual knowledge with the remember cognitive process, procedural knowledge with the apply cognitive process, and conceptual knowledge with the remaining four cognitive processes. Together, these results provide a two‐dimensional, non‐hierarchical framework of Bloom's taxonomy for use of in biology.Support or Funding InformationThis project was supported in part by an institutional award for undergraduate education from the Howard Hughes Medical Institute and the Office of Undergraduate Research at Northwestern University.