National Question Research Articles

THEMATIC ANALYSIS OF SELECTED NOVELS OF RAJA RAO: AN OVERVIEW

Literature is the mirror to society. It reflects the people, the lifestyle, culture, religion, traditions, trends, and practices of the contemporary world. Raja Rao, a doyen of Indian English literature, is viewed as a philosopher who emphasizes the spirit of India with the modernity of the world. The paper will carry out a thematic analysis of some select novels of Raja Rao with a view to exploring how Raja Rao's works reflect India's socio-cultural milieu, spiritual traditions, and the human quest for meaning. In such novels as Kanthapura and The Serpent and the Rope, Rao unravels the synthesis of Indian metaphysical thought with its adverse swing towards modernity. Kanthapura evokes the intricate joining of India's struggle for independence with Gandhian ideals of communal unity, non-violence, and the strength of rural India, while The Serpent and the Rope covered the gamut on questions of identity, spirituality, and the search for self-realization that draws heavily from Advaita philosophy. This paper also tries to understand how the works of Raja Rao have brought out the tension between tradition and modernity, showing the cultural dilemma of post-colonial India.His depiction of the character's inner struggle, his spiritual pursuit, and relationship with Indian traditions reflect his intense involvement with India's philosophical traditions. Besides, the employment of myth, folklore, and oral traditions in his narratives gives added dimensions to his engagement with questions of identity, nationalism, and existentialism. The paper thematically explicates these novels with the view of establishing Raja Rao's contribution to Indian English literature and his role in shaping modern Indian thought, especially in the post-colonial era. His novels stand as a testimony to the continuing dialogue between the past of India and the variously emerging present of the country.

Identification of HOTS-Based Islamic Education Tests in Junior High Schools : an Approach Bloom's Taxonomy and Rasch Model

Background: Higher-Level Thinking Skills (HOTS) are thinking abilities that need not only the ability to remember, but also the ability to think creatively and critically. Research Objectives: This study identifies and analyzes the level of questions on the Islamic education Examination and analyzes the standardization of question items using the Rasch model technique. Methods: Mixed method for analyzing document data with 160 multiple-choice questions. A quantitative approach was utilized to assess the quality of standard questions based on 31 students' accessible answer data. The document data was examined logically utilizing Bloom's Taxonomy Theory. Meanwhile, the data from students' answers was evaluated using the Item Response Theory (IRT) 1-parameter logistic model, also known as the Rasch model Conclusions: The result shows that many PAI subject questions are still at the LOTS level. Meanwhile, in terms of the quality of the item test using the Rasch model approach, it was found that many items were still not fit, even though in general the reliability of the items was categorized as quite good.

BB-GeoGPT: A framework for learning a large language model for geographic information science

BB-GeoGPT: A framework for learning a large language model for geographic information science

Long-term Outcomes of Multiligament Knee Injuries in American Football Players

Background: Long-term outcomes for isolated anterior cruciate ligament (ACL) reconstructions in competitive American football athletes are well reported in the literature, but little data currently exist regarding multiligament knee injury (MLKI) reconstruction outcomes. Purpose: To examine patient-reported and return-to-sport outcomes of competitive American football athletes who underwent primary, single-staged, multiligament knee reconstruction. Study Design: Case series; Level of evidence, 4 Methods: We identified patients from our institution’s prospectively collected data repository between 2001 and 2020 who underwent single-staged surgical reconstruction of an MLKI sustained during competitive participation in American football. We assessed patient-reported outcomes at a minimum of 2 years after surgery using the International Knee Documentation Committee (IKDC) Subjective Knee Form and questions regarding surgical satisfaction and return to sport. Successful return to sport was defined as a return to preinjury level of competition. We summarized all outcome data and compared outcomes between 2-ligament and >2-ligament groups and between ACL-only MLKI injury and bicruciate MLKI injury groups using independent t test for IKDC scores and chi-square test for return to sport. Additionally, we evaluated predictors of postoperative IKDC scores using linear regression and predictors of return to sport using logistic regression. Results: Outcome data were successfully collected for 53 of 73 total eligible patients (73%; mean follow-up time, 7.7 ± 4.0 years; all male; mean age at surgery, 18.1 ± 2.7 years). The mean postoperative IKDC score was 84 ± 16. The most common level of preinjury competition was high school (n = 36; 68%), followed by college (n = 10; 19%). Seven patients did not return to sport competition at any level due to limitations from their knee surgery, and 82% of patients that attempted to return to preinjury level of sport were able to do so. A total of 50 patients (94%) were satisfied or very satisfied with their surgical outcome. The 2-ligament (n = 39) and >2-ligament (n = 14) groups did not significantly differ in IKDC scores (P = .96) or proportions with successful return to sport (P = .77). Similarly, the ACL-MLKI injury (n = 39) and bicruciate MLKI injury (n = 14) groups did not significantly differ in IKDC scores (P = .89) or proportions with successful return to sport (P = .77). Age and body mass index were not significantly associated with IKDC scores or successful return to sport at follow-up (all P > .05). Conclusion: This study may represent the largest cohort of competitive American football athletes evaluated for longitudinal outcomes after multiligament knee reconstruction. Despite the severity of these injuries, we found good knee-related function and that the large majority of athletes who attempted to return to sport were successful. The majority of athletes (94%) were satisfied with their operative treatment.

“This Is Our Revolution Too”: LGBTQ Rights Activism and the National Question in Northern Ireland

Existing research frames ethnonationalism as homophobic, in which the queer is positioned outside of the boundaries of the ethnic nation. Less well understood is how LGBTQ activism engages with ethnonationalism, particularly in relation to questions of national self-determination. The limited research on this issue frames LGBTQ activist movements as groups that reject divisive ethnonationalisms by forging alternative cosmopolitan and transnational expressions of sexual identity. By exploring a single case-study, Northern Ireland, this article draws attention to how LGBTQ movements can be sites of conflict over questions of national self-determination, particularly when sexual liberation is wedded to anti-imperialist struggles. The paper concludes by reaffirming Horowitz’s model of ethnic seepage, in which the dominant ethnonational cleavages infuse all issues in divided societies. The article uses a wide range of primary data, including interviews and archival documents, to analyze LGBTQ activism in Northern Ireland during the period of ethnonationalist conflict in the 1970s and 1980s.

Efter det kritiske subjekt

The article revisits the question of the critical subject through a reading of Jacques Rancière and Fred Moten. It starts out by giving an account of a longer historical trajectory where the notions of critique and revolution have run into the ground. David Scott’s nuanced analysis of the shift from an anticolonial romantic revolutionary perspective to a tragic postcolonial stance is both affirmed and challenged. Affirmed in so far as Scott’s analysis is embedded in a broader historical context, challenged by the presentation of two attempts to think the notion of a critical subject after the disappearance of a previous notion of an antisystemic subject. The second half of the article is a discussion of Rancière’s notion of a political subjectivation that is not based on pre-existing identities but is a break with the logic of a functionalist society. Followed by a discussion of Moten’s concept of Blackness that follows the Afro-Pessimist reading of the radical dispossession of Blacks but turn it into a possibility for a different kind of community.

SUBJECTIVE ANSWERS EVALUATION USING ML & NLP

Every year, educational boards and universities conduct exams in offline mode, where a large number of students take subjective type exams. Evaluating such a vast number of papers manually requires significant effort. The quality of evaluation can also vary depending on the evaluator's mood, making the process inconsistent. Moreover, the manual evaluation is both lengthy and time-consuming. On the other hand, competitive and entrance exams typically consist of objective or multiple-choice questions (MCQs). These exams are evaluated using machines, making the process straightforward. This method also conserves resources and minimizes human interaction, resulting in error-free evaluations. While multiple systems are available for evaluating objective (MCQ) type questions, there is a lack of provision for assessing subjective (descriptive) type questions. Automating the evaluation process for descriptive answers would be extremely beneficial for educational institutions, as it would enable efficient and accurate assessment of students' exam answer sheets. Key Words: Subjective type exams, manual evaluation, evaluation consistency, time-consuming, objective type questions, multiple-choice questions (MCQs), automated evaluation, educational institutions, efficient assessment, accurate assessment, error-free evaluations, evaluation systems, descriptive answers, resource conservation, human interaction minimization

Ecological Subjectivity in the Philosophy of Posthumanism

The changes in perceptions of subjectivity in posthumanism are analyzed. Considering the differences between a “strong” and a “weak” subject, the author shows that environmental subjectivity in posthumanism implements the project of a weak philosophy. The question of subjectivity in this case is determined not in connection with consciousness, but in connection with the transformation of corporeality. Posthumanism suggests the transformation of a person through hybridization, that is, various manipulations of the body (for example, by adding plant and animal genes to a person). The attitude towards oneself is understood here as a mirror peering into one’s otherness, which is now connected not with the figure of other, but with an animal. The basic opposition of “man-animal” is called into question, since qualitative differences between man and animal disappear. It is shown that the rationale of this posthumanistic idea is based on Aristotle and postmodern philosophy. Similarly, posthumanism redefines the boundary between man and technology, opening up wide opportunities for human modification and transformation into a cyborg. The essence of the philosophical foundations of this position is revealed and it is concluded that procedural philosophy allows not only to eliminate the differences between man and nature, but also to identify the artificial and the natural. Thus, subjectivity is understood as belonging to existence, and a more environmentally friendly subjectivity of a person means the need to incorporate other beings into his world. Eco-friendly subjectivity in this case boils down to ecological physicality, that is, to an attempt to concentrate different entities in the body.

Creating and validating the Fine-Grained Question Subjectivity Dataset (FQSD): A new benchmark for enhanced automatic subjective question answering systems.

In the domain of question subjectivity classification, there exists a need for detailed datasets that can foster advancements in Automatic Subjective Question Answering (ASQA) systems. Addressing the prevailing research gaps, this paper introduces the Fine-Grained Question Subjectivity Dataset (FQSD), which comprises 10,000 questions. The dataset distinguishes between subjective and objective questions and offers additional categorizations such as Subjective-types (Target, Attitude, Reason, Yes/No, None) and Comparison-form (Single, Comparative). Annotation reliability was confirmed via robust evaluation techniques, yielding a Fleiss's Kappa score of 0.76 and Pearson correlation values up to 0.80 among three annotators. We benchmarked FQSD against existing datasets such as (Yu, Zha, and Chua 2012), SubjQA (Bjerva 2020), and ConvEx-DS (Hernandez-Bocanegra 2021). Our dataset excelled in scale, linguistic diversity, and syntactic complexity, establishing a new standard for future research. We employed visual methodologies to provide a nuanced understanding of the dataset and its classes. Utilizing transformer-based models like BERT, XLNET, and RoBERTa for validation, RoBERTa achieved an outstanding F1-score of 97%, confirming the dataset's efficacy for the advanced subjectivity classification task. Furthermore, we utilized Local Interpretable Model-agnostic Explanations (LIME) to elucidate model decision-making, ensuring transparent and reliable model predictions in subjectivity classification tasks.

ON SOCIO-CULTURAL ASPECTS OF THE USE OF POSTCARDS IN THE GERMAN-CZECH CONFLICT AT THE TURN OF THE 19TH CENTURY

Postcards as a reflection of German-Czech contradictions in the period of the Badeni crisis in the late 19th and early 20th centuries have more than once been the subject of study. The research novelty of this article lies in the fact that the problem is examined not through postcard images themselves, but from the point of view of the primary functionality of postcards as a means of communication and a commercial product. The author focuses on commercial, postal-technical, consumer practices, institutional and legal aspects, as well as the reception of postcard messages, both individual and public. Various national associations and private entrepreneurs, main producers of postcards, oft en deliberately radicalized their content in their political or commercial interests. The specificity of the postcard as a means of postal communication and its mass appeal served as the basis for unique practices in mobilizing supporters and criticizing opponents by “cartographical attacks”. Even the design of postcard forms was a means of national struggle and expression of personal position. Publishers of the respective orientations designed them in the “German” or “Czech” spirit, while the official position was to refuse any name for this postal product. At the same time, the numerous variants of handwritten edits concerning all the constituent parts of the postcard (technical design of the form, address, and stamp) allowed users to state their position on the national question. Th e legal side of postcards’ production, functioning and existence was determined, on the one hand, by conditions which were common to other types of printed matter, in particular, by the liberal press law of 1868, and, on the other hand, by postal regulations that periodically withdrew certain postcards from circulation. All this became the subject of litigations and debates in parliament and thus provided a way for postcards to enter the public sphere and, accordingly, served to attract further public attention to the national conflict and its aggravation.

The Marxist concept of national question and the analysis of Ethiopian reality during the Derg regime (1974 to 1991)

The Marxist concept of national question and the analysis of Ethiopian reality during the Derg regime (1974 to 1991)

Peaceful Strategy to Black National Question in Ralph Ellison’s Juneteenth

This article analyzes the theme of Black national liberation as it appears in Ralph Ellison's novel Juneteenth. The study has relevance to probe into the remedies of Black liberation movement in America. The article addresses on the research problems concerning to the black protagonist's inability to identify and deal with an appropriate path that leads the freedom of blacks in American society. The study analyzes the issues through the research approach (methodology) of the Marxist concept of dialectics. According to this theory, the conflict between society's opposing forces is permanent, while any resolution to it is conditional and only temporary. This idea maintains that the key to the liberation of the oppressed nationality and class is the battle against the oppressor nationality and class. One of the novel's two main protagonists, Reverend Hickman, belongs to the oppressed black nationality and class. Hickman seeks to free the downtrodden Blacks from its constraints, but he chooses the incorrect road by making peace with the country's ruling white class. Hickman believes that the battles of Afro-Americans alone cannot end black national oppression; instead, he looks to some heavenly figure from the white race, such as Abraham Lincoln, to grant Afro-Americans their independence, justice, and equality. Bliss was nurtured by Hickman in the hopes that he would become the American equivalent of Abraham Lincoln, but Bliss betrayed Hickman by becoming into the racial baiting white senator Adam Sunraider. Despite Bliss's betrayal, Hickman continues to have faith in Sunraider. The study reveals that this Hickman's message of peaceful approach while dealing with ruling whites keeps the Afro-Americans weak, far from liberating the oppressed black nationality.

AN UNFINISHED POLEMICS “AMONG OURSELVES”

The paper examines the polemical dialogue on the national question between two Ukrainian literary classics, Ivan Franko and Lesia Ukrainka, which took place in the late 19th and early 20th century, as well as its impact on other writers, social, and political figures. There was a noticeable divergence in the positions of two prominent writers, with Ivan Franko criticizing the trans-Dnieper Ukrainian intellectuals for distancing themselves from the people, while Lesia Ukrainka justified their participation in the statewide all-Russian movement. This is primarily explained by the asynchrony of social development in the conditions of the Russian autocratic empire and the Austro-Hungarian constitutional state, but there was still another factor. The views of both Ivan Franko and Lesia Ukrainka were formed under the influence of the social-democratic ideas of Mykhailo Drahomanov. However, by that time, Ivan Franko had already shifted towards national positions, while Lesia Ukrainka continued to share Drahomanov’s principles. The evolution of Lesia Ukrainka’s views suggests that it was rather a temporary misunderstanding than an ideological confrontation with Ivan Franko. During the Soviet period, this opposition articulated by Ivan Franko as the one “among ourselves” and temporary “polemical controversy” (Lesіa Ukrainka) was intentionally ignored, as the communist regime in the USSR made efforts to silence the presence of national movements. Interest in this polemic appeared already in the first post-revolutionary years in Ukraine (Mykhailo Drai-Khmara) and remained vivid within the Ukrainian emigration (Dmytro Dontsov, Yurii Lavrinenko). Subsequently, it was revived in literary works during the period of Ukrainian state independence, with contributions from Ivan Denysiuk, Mykhailo Nechytaliuk, Vasyl Horyn, Halyna Levchenko, Mykhailo Hnatiuk, and Bohdan Tykholoz. The polemic between Ivan Franko and Lesia Ukrainka proves its relevance in current circumstances as well.

Jewish Text in Howard Fast’s Novel “Torquemada”

The present study aims to identify the specifics of the Jewish text in Howard Fast’s iconic novel Torquemada. The study is based on the semiotic definition of the text. The Jewish theme in the oeuvre of Fast, a writer of Jewish origin, occupies a significant place. The specific construction of the Jewish theme allows one to highlight the boundaries of a certain Jewish text in the author’s work, which was clearly formed during the period of ideological and national quest. The novel Torquemada occupies a special place in the Jewish text of Fast. The Jewish text of the novel Torquemada is represented by a multi-level model. At the anthological level, the Jewish text of the novel can be correlated with the ideological quests of the author, Howard Fast. However, at the same time, it is an allegoric narrative about the fate of the Jews of the 20th century and the tragedy of the Holocaust. The events of the 15th century are the allegory of the events of the 20th century, and the times of the Spanish Inquisition are associated with the tragedy of the Jews of Europe.

Assessment of oral manifestations of systemic health related knowledge of medical students pursuing internship programme at a tertiary health care centre

Clinicians across the entire medical profession are responsible for detection, prevention and imparting importance of oral health to the patients. However, many studies conducted so far show lack of knowledge of medical students regarding dental conditions. The present observational study was undertaken to evaluate the knowledge of oral manifestations of systemic diseases amongst medical students undergoing Internship program at a tertiary centre. A self-structured Questionnaire comprising of 20 questions was used as a survey tool to assess the understanding of 100 medical students undergoing internship program. They were explained about the purpose of the study. The questionnaire consisted of an objective part Yes/No (7 questions) and subjective part designed on likert scale(13 questions). There were questions on their view on current curriculum for dentistry in medical undergraduate training and possible enhancement in teaching modalities. The subjective part had five options to choose from, Strongly agree, Agree, Neutral, Disagree And Strongly Disagree. The responses of objective questions was recorded and percentage calculated for both options. The total score was obtained for each participant based on their reponses to the subjective questions based on the Likert scale. Student t-test was applied to calculate means between male and female students. For all objective questions, percentage was calculated for affirmative and negative responses. The responses were calculated on scale5-1 for the subjective questions. Reverse scoring was done for questions 10,11,12 i. e scale 1-5. The maximum achievable score in this category was 65 which was further divided into three category as Excellent score range 55-65, Good score range 45-54, Unsatisfactory score range 35-44. The number of students in excellent category were 11, Good 66, unsatisfactory 23. The average score obtained was 48.175. The mean scores for males was 48.0161 and for females it was 48.142. The scores were slightly better for females but the difference was statistically insignificant as (p-value=0.455056). The present study showed good level of Knowledge on oral manifestations of systemic diseases among medical students as compared to other studies which forms an integral part of medical curriculum. The students responses on enhancing dental curriculum and teaching aids for improved education tools can help the faculty provide a better teaching learning experience.

The Eighteenth Constitutional Amendment and the Baloch National Question in Pakistan

The passage of the eighteenth amendment bill in 2010 represented a critical milestone in Pakistan’s constitutional history. It sought to better manage the perennial federal-provincial tensions by restructuring the country’s constitutional framework to assuage the grievances of the ethno-regional movements opposed to the centralised state system, especially in Balochistan, where Pakistan’s territorial integrity faces its toughest challenge. Islamabad’s inflexibility in accommodating ethnic dissidence through various institutional measures has exacerbated ethnic tensions and hardened nationalist sentiments. The eighteenth amendment aimed to reverse deformations introduced during dictatorships and promote a more federal polity. More than a decade has passed since the introduction of the bill, yet the question lingers: Did the amendment achieve its intended goal of mitigating the ethnic conflict in Balochistan? This article scrutinises the current situation in Balochistan and offers a critically oriented analysis of the amendment’s impact on the province’s security scenario. It contends that the amendment fell short of its objectives because it remained silent on issues of critical import to the Baloch and failed to address the underlying issues fuelling the conflict.

Freak out! At the acousmonium: François Bayle and the dynamics of the subject

French electroacoustic music in the post-war decades has typically been associated with a high modernism that spurned popular or mass culture. This article situates electroacoustic music in the late 1960s and 1970s in relation to the transformation of the cultural field in this period, which unsettled clear divisions between high and low cultures. Attending in particular to the question of subjectivity as it appears in the theoretical writings of the composers Pierre Schaeffer and François Bayle, the article aligns Bayle's thinking with that of contemporary writers including Julia Kristeva and Roland Barthes, who launched a critique of the Enlightenment subject. This theoretical nexus suggests that Bayle's L’Expérience acoustique (1969–1972) posits a form of subjectivity closer to the psychedelic counterculture than to the sober listening of Schaeffer's writings. In unsettling the Schaefferian listening subject, the piece opens itself to a heterogeneous listening, the social consequences of which remain significant today.

“Decolonizationˮ and “Deimperializationˮ: Modern Western Policy Towards Fragmentation of Russia

Introduction. Multinational Russia at various stages of its existence faces the playing out of the national question by opposing geopolitical centers. The discourse of “decolonization” and “deimperialization” of Russia has acquired a new level of relevance in the international arena after February 24, 2022. The purpose of the article is to identify and investigate the features of the destructive form of ethno-political mobilization in the regions of the Russian Federation through the construction and promotion of the discourse of “decolonization” and “deimperialization” of Russia by Western states. Materials and Methods. The materials used in the study include U.S. legal documents, resolutions of the European Parliament, publications and speeches of representatives of the Western political elite and scientific and expert community, materials of the United States Commission on Security and Cooperation in Europe, content of various organizations, foreign mass media and media foreign agents. The methodological basis of the study is represented by systemic, geopolitical and instrumentalist approaches, as well as such methods as monitoring, discourse analysis and content analysis. Results. The narrative of “deimperialization” and “decolonization” of Russia in the publications of the Western scientific and expert community and media materials is studied, and the political technologies of Western states oriented at the actualization of ethnic nationalism and, as a consequence, separatist tendencies in the regions of the Russian Federation are revealed. Western projects of practical policy oriented at actualization of national contradictions in the Russian Federation are implemented mainly through the following initiatives: holding events dedicated to “decolonization” and “deimperialization” of Russia; using representatives of Western-oriented national elites to actualize separatist tendencies in the regions of the Russian Federation; adopting declarations on “liberation” of the peoples of Russia from “colonial” dependence; popularization of information about the need for “decolonization”; and the use of the “decolonization” of Russia. Discussion and Conclusion. “Decolonization” and “deimperialization” of Russia are focused on the implementation of a destructive form of ethno-political mobilization and fragmentation of the Russian Federation along national lines of settlement. The authors identify five technologies for the formation of ethnic nationalism in the regions of the Russian Federation, which are used by Western states in the process of modern confrontation with Russia. The results of the study will be useful for public authorities at both regional and federal levels to develop a set of measures aimed at countering the playing of the “national card” on the territory of Russia by competing geopolitical centers.

ETHNOCULTURAL COMPLEXITY OF THE POPULATION OF CHUVASHIA ACCORDING TO THE 2020 CENSUS

The relevance of the study is determined by new approaches in the official accounting of the nationality of Russian citizens. A decreasing number of representatives of the main ethnic groups both in Russia and in its individual regions increases the attention paid to the problems of ethnocultural development and interethnic relations. The publication of the 2020 census materials caused a mixed reaction to its organization and individual results. The purpose of the article is to analyze the census materials on the national composition of the population, an attempt to count citizens who have declared their difficult ethno–cultural choice. The reasons for the noticeable difference between the materials of sociological surveys and population census data are considered. Materials and methods. The article analyzes the published data of the 2020 population census, in particular, the data contained in the fifth volume of the national census 2020 results “National composition and language proficiency” for Russia in general and for the Chuvash Republic in particular. In addition, the “Methodological explanations” to the census materials were studied, as well as a number of regulatory documents. According to the table “Population’s answers to the question “Your nationality”, information was calculated on the number of people who chose a difficult answer to the question of nationality. Study results. The census materials showed that the number of individual ethnic groups of Chuvashia, representing the old-age population (except the Russians), decreased during the inter-census period. This process was determined not by the natural and mechanical movement of the population, but by the change of ethnic identity. The increase in the number and proportion of persons who did not indicate their ethnicity occurred both due to refusal to answer the question and due to a negative answer about specific nationality. In addition, in Chuvashia, the proportion of citizens whose information was obtained from administrative documents (gender and age) has increased significantly. The number of citizens who declared a complex ethnic identity was about 0.2% of those who named their ethnicity. This indicator lags far behind the data obtained in the course of sociological surveys. Conclusions. It follows from the above materials that the residents of Chuvashia were more loyal to the issue of their nationality compared to the citizens of Russia. The choice of a complex variant of ethnicity was not massive, but it also differed from a similar process in Russia as a whole. For example, in the republic, more often than in the country, the Russians, the Tatars, the Mordvins and the Ukrainians chose a difficult variant of nationality. The opposite picture was typical for the Chuvash; the Tatars living in Russia and Chuvashia had the same indicators of ethnocultural complexity. For the majority of the population, the definition of their ethnic identity did not cause any particular difficulties, but at the same time, both difficulties and a desire to distance themselves from such self-determination were noted in this process.

SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research

Recently, there has been growing interest in using Large Language Models (LLMs) for scientific research. Numerous benchmarks have been proposed to evaluate the ability of LLMs for scientific research. However, current benchmarks are mostly based on pre-collected objective questions. This design suffers from data leakage problem and lacks the evaluation of subjective Q/A ability. In this paper, we propose SciEval, a comprehensive and multi-disciplinary evaluation benchmark to address these issues. Based on Bloom's taxonomy, SciEval covers four dimensions to systematically evaluate scientific research ability. In particular, we design a "dynamic" subset based on scientific principles to prevent evaluation from potential data leakage. Both objective and subjective questions are included in SciEval. These characteristics make SciEval a more effective benchmark for scientific research ability evaluation of LLMs. Comprehensive experiments on most advanced LLMs show that, although GPT-4 achieves SOTA performance compared to other LLMs, there is still substantial room for improvement, especially for dynamic questions. The codes and data are publicly available on https://github.com/OpenDFM/SciEval.