Quantitative Educational Research Research Articles

Why it works: Using qualitative methods in economic education research

The authors of this article present arguments for why and how qualitative research should be used in economic education. These arguments recognize the nature of economics as a discipline and economics educators’ current expertise and preferences. The authors have five goals: (i) clarifying how and why the use of qualitative research in economic education fits in with economics as a discipline; (ii) identifying practical limitations of quantitative research in economic education; (iii) reviewing the limited literature on qualitative research in economic education; (iv) proposing how qualitative and quantitative methods can be combined in a mixed methods approach that recognizes the methodological assumptions of economics and the expertise of economists, and finally, (v) suggesting characteristics of systematic and rigorous qualitative research in economic education.

Leveraging evaluation of quality on medical education research with ChatGPT

What is the educational challenge? The Medical Education Research Study Quality Instrument (MERSQI) is widely used to evaluate the quality of quantitative research in medical education. It has strong evidence of validity and is endorsed by guidelines. However, the manual appraisal process is time-consuming and resource-intensive, highlighting the need for more efficient methods. What are the proposed solutions? We propose to use ChatGPT to evaluate the quality of medical education research with the MERSQI and compare its scoring with those of human evaluators. What are the potential benefits to a broader global audience? Using ChatGPT to evaluate medical education research with the MERSQI can decrease the resources required for quality appraisal. This allows faster summaries of evidence, reducing the workload of researchers, editors, and educators. Furthermore, ChatGPTs’ capability to extract supporting excerpts provides transparency and may have the potential for data extraction and training new medical education researchers. What are the next steps? We plan to continue evaluating medical education research with ChatGPT using the MERSQI and other instruments to determine its feasibility in this realm. Moreover, we plan to investigate which types of studies ChatGPT performs best in.

Trends and Motivations in Critical Quantitative Educational Research: A Multimethod Examination Across Higher Education Scholarship and Author Perspectives

To challenge “objective” conventions in quantitative methodology, higher education scholars have increasingly employed critical lenses (e.g., quantitative criticalism, QuantCrit). Yet, specific approaches remain opaque. We use a multimethod design to examine researchers’ use of critical approaches and explore how authors discussed embedding strategies to disrupt dominant quantitative thinking. We draw data from a systematic scoping review of critical quantitative higher education research between 2007 and 2021 (N = 34) and semi-structured interviews with 18 manuscript authors. Findings illuminate (in)consistencies across scholars’ incorporation of critical approaches, including within study motivations, theoretical framing, and methodological choices. Additionally, interview data reveal complex layers to authors’ decision-making processes, indicating that decisions about embracing critical quantitative approaches must be asset-based and intentional. Lastly, we discuss findings in the context of their guiding frameworks (e.g., quantitative criticalism, QuantCrit) and offer implications for employing and conducting research about critical quantitative research.

Interrogating Side Effects in Critical and Quantitative Research in Education

Quantitative education research is often perceived to be “objective” or “neutral.” However, quantitative research has been and continues to be used to perpetuate inequities; these inequities arise as both intended effects and unintended side effects of traditional quantitative research. In this review of the literature, we synthesize how quantitative researchers have attempted to use critical paradigms to address questions of equity in education research published over the past 15 years. We identify and describe three main tensions that critical and quantitative researchers navigated: (a) creating and analyzing social group categories, (b) trying to describe commonalities within group experiences without erasing heterogeneity of experience within the group, and (c) determining what is a “significant” result when conducting critical and quantitative research.

Critical Quantitative Literacy: An Educational Foundation for Critical Quantitative Research

Education research has recently seen the emergence of two distinct frameworks guiding the application of quantitative methods through a more critical and equity-oriented lens. These two frameworks are critical quantitative (CritQuant) studies and quantitative critical race theory (QuantCrit). Although different in their intellectual traditions, they both acknowledge the oppressive history of quantitative methods and the need to improve the criticality of quantitative research in education. For applied quantitative research in education to become more critical, it is imperative that learners of quantitative methodology be made aware of its historical and modern misuses. This directive calls for an important change in the way quantitative methodology is taught in educational classrooms. Critical quantitative literacy (CQL) is introduced in this manuscript as a paradigm for teaching, learning, understanding, and applying quantitative methods in a way that supports the application of CritQuant and QuantCrit frameworks in educational research.

Educational Disparities in Primary School Students in Austria

Abstract: In this study, we analyze data from the Austrian assessments of educational standards in German and Mathematics, applying an intersectional perspective. We simultaneously consider the effects of gender, socioeconomic status (SES), and immigrant background on fourth-graders’ competencies in both subjects. Besides the expected main effects – girls perform better in German, boys better in mathematics, lower SES and immigrant background predict lower performance in both subjects – we found the following interaction effects: Second-generation immigrant students showed better achievement in German than their peers who had migrated to Austria later in life, except when their country of origin was Turkey. In students without an immigrant background, SES was more strongly related to achievement in mathematics in boys, while among students originating from former Yugoslavia, SES was more strongly related to achievement in German in girls. The study demonstrates that intersectional perspectives in quantitative educational research can help to identify groups with increased support needs.

Transforming the future of quantitative educational research: a systematic review of enacting quantCrit

ABSTRACT Quantitative Critical Race Theory (QuantCrit) is a burgeoning field of study seeking to challenge and improve the use of statistical data in social research. It pulls lessons and insights from Critical Race Theory and applies them to understanding social challenges. In this paper, we aim to improve the quality of quantitative research produced by showing examples of how pioneers in this field are effectively enacting QuantCrit. We conducted a systematic review of the literature to include all empirical education studies published since 2010 through 2022. Twenty-seven studies fit the criteria. Our data shows there is room for innovation, experimentation, and exploration. However, the study highlights exemplars of authors who embody QuantCrit principles through their professional and personal positionality statements, cognizance of community, robust racial/ethnic categories, intentionality on not centering whiteness, use of atypical methods, new measurement tools centering Black and Brown students, and innovative interpretations of findings.

A Critical Realist Critique of “Measurement in Quantitative Educational Research: An Instrumental Mistake?” by Sigve Høgheim

Both Sigve Høgheim and I advance a similar critique of representationalism, operationalism and construct validity as found in psychology. Our respective critiques hinge on the way that these concepts do not adequately consider the ontological question about the reality of our constructs. Høgheim suggests that we can circumvent the problem by returning to the classic definition of measurement which avoids “coding,” but I think that avoiding coding (representation, modelling, theorising) is impossible. Instead of trying to avoid it, I suggest that we embrace it, using a number of critical realist concepts such as emergence, layered reality, retroduction, judgemental rationalism and the semiotic triangle. I also argue that the current approach to operationalism reflects a deep contradiction, known as the epistemic fallacy, in which scientists reduce questions of ontology to questions of epistemology. Nevertheless, despite their contradictory version of science, research scientists in the fields of psychology and education still manage to discover things about the world, but in order to do so, they need to break their own rules. Høgheim wants such scientists to avoid the contradiction – the breaking of the rules of science – by remaining faithful to Euclid, Newton, and Descartes. On the contrary, I suggest that the contradiction can be avoided by changing the theory of science to a more adequate, critical realist version that better reflects the scientific practice of psychologists and educators.

“Effectiveness of Structural Teaching Programme on Knowledge and Practice Regarding Breast Self-Examination among Female Students – A Quasi Experimental Study”

Background of the study: Breast cancer is a major public health issue in both developed and developing countries and it is the leading cause of illness and mortality in women. Every year, about 1.15 million new instances of breast cancer are diagnosed worldwide. The high death rate among women owing to breast cancer is exacerbated by late-stage diagnosis. Aim: To evaluate the impact of a teaching program on female students' knowledge of breast self-examination. Methodology: The quantitative educational and evaluative research approach was deemed the most suitable for this investigation. The research design was "A Quasi Experimental Study to Evaluate the Effectiveness of a Structured Teaching Program on Breast Self-Examination Knowledge and Practice among Female College Students in a Selected College in Panipat." Stratified random sampling was utilized to determine the sample. The data collection instruments were self-administered. Two-part multiple-choice questionnaire, with the first section containing demographic data and the second assessing knowledge of breast self-examination. The data were analyzed and hypotheses tested using descriptive statistics (frequency, percentage, mean, and standard deviation) and inferential statistics (Chi square, paired "t" test). Results: At the 0.05 level, a comparison of the pre-test mean of 8.35 and the post-test mean of 24.18 for breast self-examination knowledge was found to be statistically significant. This study demonstrates that an educational program is effective at increasing breast self-examination awareness. Conclusion: Overall, the results of this study showed that female students' understanding and practice of BSE were greatly improved because to the structured training program. Therefore, an effective health education campaign should be developed to raise awareness and encourage regular self-breast examination among college-aged women in order to detect breast cancer at an earlier stage.

Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements

The COVID-19 pandemic meant that, in 2020, students in England were unable to sit their examinations and instead received predicted grades, or “centre assessment grades” (CAGs), from their teachers to allow them to progress. Using the Grading and Admissions Data for England (GRADE) dataset for students from 2018 to 2020, this study treats the use of CAGs as a natural experiment for causally understanding how teacher judgements of academic ability may be biased according to the demographic and socio-economic characteristics of their students. A variety of machine learning models were trained on the 2018–19 data and then used to generate predictions for what the 2020 students were likely to have received had their examinations taken place as usual. The differences between these predictions and the CAGs that students received were calculated and then averaged across students’ different characteristics, revealing what the treatment effects of the use of CAGs were likely to have been for different types of students. No evidence of absolute negative bias against students of any demographic or socio-economic characteristic was found, with all groups of students having received higher CAGs than the grades they were likely to have received had they sat their examinations. Some evidence for relative bias was found, with consistent, but insubstantial differences being observed in the treatment effects of certain groups. However, when higher-order interactions of student characteristics were considered, these differences became more substantial. Intersectional perspectives which emphasise interactions and sub-group differences should be used more widely within quantitative educational equalities research.

Teachers’ professional development: active approach and meaningful learning in the instruction in english as a foreign language

This study aims to contribute to the professional teacher’s development proposing the use of active and meaningful learning to improve English Language instruction. The research team used qualitative and quantitative educational research approaches to collect information. The participants were 76 people among high school students, trainee teachers, and teachers in service in Manta, Ecuador. The Likert scale questionnaire, survey, and open-ended interview questions were used to gather information. Results allowed the research team to determine that the traditional methodology used for the instruction limits the students’ engagement and motivation for learning English as a foreign language. It concluded that an active approach and meaningful learning could improve student-teacher feedback creating a positive synergy and better educational environments. EFL classes should be more active and exciting for students to answer according to their local context and interests. In the same way, lessons should be supported with the scaffolding process for building more meaningful lessons to remember.

Qualitative research in education

Since it entered education in the mid-20th century, qualitative research has been a well-established methodology among educational researchers. Its strength lies in its nature, scope, and methodological flexibility to understand education's cultural, social, political and moral phenomena. Between the 1970s and 1980s, the relevance and superiority of quantitative vs. qualitative research in education were debated in academic forums. The paradigm debate was so intense that it reached political discussions. In 2002, the National Research Council (NRC) of the United States of America recommended the Federal Department of Education the exclusion of qualitative research from federal funds because it was not considered scientific research. A similar situation was observed in England and Australia. Since then, qualitative research has continued to develop as a model of educational research with the support of private funds. This paper revisits qualitative education research and examines its developments to reposition itself as a legitimate model of scholarly research in a political-scientific era.

Estimating the replicability of technology education research

Technology education research is a growing field, with the rate of growth increasing over the last 2 decades. As the field grows, it is paramount that credibility is maintained in published findings. To date there is no evidence to suggest a lack trust is warranted, however in the midst of the replication crisis there is need to ensure continued rigour. This article presents a z-curve analysis of the replicability of quantitative research in technology education since 1983 using statcheck for automated data extraction. The results indicate that authors often mis-report p-values, typically due to rounding errors, with a small percentage (1.59%) of inconsistently reported p-values leading to decision errors in terms of statistical inference. With respect to replicability, overall it is estimated that 55.7% of reported quantitative results in technology education would replicate, however since 2020 this estimate appears to be increasing. These results do not indicate specific findings which are likely or unlikely to replicate, but do suggest a need to invest effort in identifying studies which would have a high value in being replicated, particularly in the timeframe of work published from 2010 to 2020.

Replicable quantitative psychological and educational research: Possibility or pipe dream?

ABSTRACT Since the advent of the twenty-first century, science has experienced a crisis pertaining to the replicability of quantitative research findings, which has become known as the ‘replication crisis’. The replication crisis has particularly afflicted research in the behavioural sciences, and psychology in particular. Given the relevance of psychology to education, it is unsurprising that the replication crisis also presents an issue for quantitative educational research, thus potentially compromising its practical usefulness. This article outlines the replication crisis in psychology, and highlights its significance for quantitative educational research. Greater methodological rigour, aimed at addressing methodological deficiencies in the conduct of scientific research, has been suggested as a response to the replication crisis. Following a review of various calls for greater methodological rigour, the current article argues that, whilst methodological deficiencies may be a contributory factor, there is potentially a more fundamental reason for the replication crisis. The case is made that the non-separability of a measured attribute from the measurement instrument, and the irreducible uncertainty in unmeasured attributes, may be the principal reasons for the replication crisis in psychology and education. Implications of such an explanation for the crisis are articulated in relation to the evidence-based policy agenda in education.

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice

BackgroundAs a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators.MethodsA bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes.ResultsThere were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies.ConclusionsOverall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.

Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary.

When seeking to explain the eventual outcomes of a higher education experience, do the personal attributes and background factors students bring to college matter more than what the college is able to contribute to the development of the student through education or other institutional factors? Most education studies tend to simply ignore cognitive aptitudes and other student characteristics—in particular the long history of research on this topic—since the focus is on trying to assess the impact of education. Thus, the role of student characteristics has in many ways been underappreciated in even highly sophisticated quantitative education research. Conversely, educational and institutional factors are not as prominent in studies focused on cognitive aptitudes, as these fields focus first on reasoning capacity, and secondarily on other factors. We examine the variance in student outcomes due to student (e.g., cognitive aptitudes) versus institutional characteristics (e.g., teachers, schools). At the level of universities, two contemporary U.S. datasets are used to examine the proportion of variance accounted for in various university rankings and long-run salary by student cognitive characteristics and institutional factors. We find that depending upon the ways the variables are entered into regression models, the findings are somewhat different. We suggest some fruitful paths forward which might integrate the methods and findings showing that teachers and schools matter, along with the broader developmental bounds within which these effects take place.

Pedagogical Approaches for Sustainable Development in Building in Higher Education

Education for sustainable development (ESD) is one of the great challenges that university faculties have to face. Therefore, a multidisciplinary team from the faculty of Engineering of Gipuzkoa (EIG) at the University of the Basque Country (UPV/EHU) has developed pedagogical approaches to apply in construction degrees, namely Civil Engineering and Technical Architecture. Pedagogical tools, such as problem-based learning (PBL) or research-based learning (RBL), and environmental tools, such as the life cycle assessment (LCA) and computational thinking (CT), have been used; in doing so, they acquire a sustainable approach to work “soft-skills” competencies into sustainability. For example, research-based tools have helped to revalorize waste both outside and inside the university; they have contributed to more sustainable industrial processes, collaborative research projects, and participation in conferences and scientific publications. Based on academic results, the designed tools are appropriate for teaching in Technical Architecture and Civil Engineering degrees; however, to demonstrate their potential in terms of sustainable education, holistic rubrics based on in-depth quantitative educational research are required. Thus, to analyze the ability of the students to incorporate sustainability principles in their work, the multidisciplinary team presenting this paper plans to collaborate with psychologists and sociologists within the framework of the Bizia-Lab program of the UPV/EHU.

The Benefits of Longitudinal Data and Multilevel Modeling to Measure Change in Adventure Education Research

Background: A common critique of adventure education research methodology is the overreliance on pre-/post-study designs to measure change. Purpose: This paper compares and contrasts two methods of data analysis on the same adventure education data set to show how these distinct approaches provide starkly different results and interpretation. Methodology/Approach: Using secondary data analysis, we employ a longitudinal data set of the social skill development of urban middle school students who participated in an adventure education program over the course of three academic years. First, change was assessed using a pre-/post-design by a traditional analysis of variance (ANOVA). Next, change was assessed with a six-wave, longitudinal design using multilevel modeling. Findings/Conclusions: Results show that the multilevel modeling approach revealed nonlinear change in the social skill development of middle school students, resulting in more accurate, nuanced estimation of change in social skill development than the ANOVA approach. Implications: The use of longitudinal data and multilevel modeling can be a useful methodological approach and statistical tool for adventure education researchers to not only address the criticisms of quantitative research in adventure education, but more importantly, provide a more thorough understanding of the impact of adventure education on human development.

Making a case for more feminist approaches in quantitative research: How commonly used quantitative approaches in adult education research marginalise and oversimplify diverse and intersectional populations

In contrast to qualitative and theoretical approaches, the mainstream of quantitative research often still finds it difficult to incorporate modern concepts of diversity and intersectionality into its work. This article aims to highlight various aspects in which large studies and their evaluations marginalise or ignore certain parts of the population. In surveying data, large-scale surveys like the Programme for the International Assessment of Adult Competencies (PIAAC) often not only operate on a binary gender concept but also do not differentiate between a person gender identity and their social gender. In addition, commonly used methods keep unequal distributions invisible. Non-binary people are virtually invisible, unequal benefits for women remain hidden and the intersectional diversity inside the broad gender categories poses challenges to the mainstream of quantitative research in adult education. Therefore, there is a need for a feminist approach to statistics and quantitative research.

23 Questions to Guide the Writing of a Quantitative Medical Education Research Proposal

23 Questions to Guide the Writing of a Quantitative Medical Education Research Proposal