Research Methods Textbooks Research Articles

Coverage and treatment of qualitative inquiry in general research methods textbooks.

Coverage and treatment of qualitative inquiry in general research methods textbooks.

The Challenges of Conducting Research in Diverse Classrooms: Reflections on a Pragmatics Teaching Experiment

For researchers, the typical way of determining whether a pedagogical innovation works is by conducting an experiment. In migrant settings, however, experiments are more challenging to carry out due to the diversity of the learner population. Unfortunately, how to deal with these challenges is not addressed in a practical way in research methods textbooks, which typically provide a normative view of the research process. This paper aims to draw attention to the realities of classroom research carried out in the Language Instruction for Newcomers to Canada (LINC) setting. These classes consist of adult immigrants and refugees from a wide range of cultural, linguistic and educational backgrounds. We illustrate how this diversity along with other characteristics of LINC programs impact the decision-making of the researcher with respect to a pedagogical experiment focused on pragmatics. The study compared a formula-enhanced approach to teaching speech acts to the more mainstream approach aimed at raising learners’ meta-pragmatic awareness about speech act behaviour. The pre-post-delayed-post-test gains appear to favour the Formula group, but the interpretability of these results is compromised by the fact that the composition of the two classes was very different. Discussion of the limitations of this case study feeds into a broader consideration of the implications for classroom research of linguistic and cultural diversity typical of L2 educational contexts like LINC.

Successful Recruitment to Qualitative Research: A Critical Reflection

Recruitment to qualitative research is an important methodological consideration. However, the process of recruitment is under-communicated in qualitative research articles and methods textbooks. A robust recruitment plan enhances trustworthiness and overall research success. Although recruitment has recently received increased attention in the qualitative methodology literature, a more nuanced understanding is required. We realized successful recruitment to our focused ethnographic inquiry. Numerous nurse educators, researchers, and administrators volunteered within three months of study initiation. Using Gibbs’ Reflective Cycle, we conducted a critical reflection on the recruitment log and participant interview data to surface factors contributing to our success. This article offers our insights into the facilitators of successful recruitment. Our reflection revealed four themes contributing to successful enrollment: (a) laying the groundwork, (b) recruitment plan, (c) building rapport, and (d) participant motivations. Two new recruitment strategies accounted for over 60% of our sample. Reporting on successful strategies for recruiting participants to qualitative research and specifying participants’ motivations to volunteer, from their perspective, make important contributions to the recruitment literature. Our article offers guidance to qualitative researchers pursuing successful recruitment. Additional research is required to evaluate the relative influence of various recruitment strategies.

What’s the Harm? The Coverage of Ethics and Harm Avoidance in Research Methods Textbooks

Methods textbooks play a role in socializing a new generation of researchers about ethical research. How do undergraduate social research methods textbooks portray harm, its prevalence, and ways to mitigate harm to participants? We conducted a content analysis of ethics chapters in the 18 highest-selling undergraduate textbooks used in sociology research methods courses in the United States and Canada in 2013. We found that experiments are portrayed as the research design most likely to harm participants. Textbooks overwhelmingly referred to high-profile, well-known examples of harmful research. Chapters primarily characterize participants as at risk for psychological and physical harm. Textbooks engage in detailed discussions of how to avoid harm; informed consent figures prominently as an essential way to mitigate risk of harm. We conclude that textbooks promote a procedural rather than nuanced approach to ethics and that content in ethics chapters is out of step with scholarly research in research ethics.

Frightened by an Old Scarecrow: The Remarkable Resilience of Demand Characteristics

More than 50 years ago, the idea of demand characteristics was introduced by Martin Orne in a widely cited American Psychologist article. Through the 1960s and the mid-1970s, numerous studies were conducted investigating the role of demand characteristics in a variety of research areas. Demand characteristics faded from researchers’ attention in the late 1970s, relegated to brief descriptions in research methods textbooks. The present article traces the origins of and battles fought over demand characteristics during its heyday. Evidence is provided that suggests demand characteristics experienced a rebirth in the 1980s and it remains a widely referenced idea up to today. Demand characteristics reflect perennial concerns about the difficulties of and limitations to doing research with humans, concerns that often surface in the periodic crises that confront psychology. The types of problems that animated the crisis of confidence associated with demand characteristics in the 1970s form one dimension of the current replication crisis. Reinterpretation of this current replication crisis and a new direction for experimental research with human subjects are derived from this review of demand characteristics.

“I Have Ten Peer Reviewed Articles. Now What?” How Political Science Research Methods Textbooks Teach Students About Scholarly Context

This article examines the treatment of one aspect of information literacy—establishing and understanding the scholarly context of research—in political science research methods textbooks. This important information literacy skill speaks to the ability of students to synthesize and engage with information. We analyzed how the most commonly assigned research methods textbooks addressed how to understand and use scholarly research and how these discussions compare to national standards and performance indicators for information literacy. We find that there are substantial discussions of how to establish and understand the scholarly context, at least in some of the textbooks. These discussions tend to focus on teaching students how to write a literature review for their research projects, stressing the importance, one hand, of reading and understanding the scholarly literature, and, on the other, understanding the key debates and concepts in the literature. Only a handful of textbooks provide concrete advice to undergraduates about how to go about this process. We conclude with some suggestions about how instructors and librarians can work to do more to integrate the lessons from these textbooks into assignments and also course and curriculum design.

Trends in Kinesiology Research Quality over Three Decades

The quantity of published research in kinesiology has increased exponentially over the past three decades. During this period, journal editors have published recommendations for improving the quality of research studies and of research reporting. However, limited evidence has been collated to systematically investigate longitudinal trends in the quality of kinesiology research. PURPOSE: To examine trends in the quality of published kinesiology research over the past three decades. METHODS: The journal Research Quarterly for Exercise and Sport was selected as representative of a broad range of disciplines within kinesiology. All articles in the 1993 (n = 62), 2003 (n = 56), and 2013 (n = 62) volumes were evaluated using quality indicators recommended in editorials and research methods textbooks. Trends were analysed using either oneway ANOVA or KWANOVA for numeric variables or chi-square for categorical variables. RESULTS: After initial examination, 117 of the 180 articles (65%) were determined to include primary quantitative data collection. The percent of these studies using experimental or quasi-experimental designs did not change from 1993 (36.8%) to 2003 (44.2%) or to 2013 (47.2%). Sample size did not change from 1993 (Mdn=64) to 2003 (Mdn=38) or to 2013 (Mdn=42). Mean (±SD) number of authors did not change from 1993 (3.1±1.7) to 2003 (3.1±1.5) but increased (p < 0.05) in 2013 (4.3±2.4). The percent of articles reporting a clear purpose statement in the abstract did not change from 1993 (31.6%) to 2003 (30.2%) but increased (p < 0.05) in 2013 (97.2%). Within the body text, a high percent of studies included a clear purpose statement in all three volumes (68.4% in 1993; 83.7% in 2003; 86.1% in 2013). Among experimental studies, the percent that reported effect sizes increased (p < 0.05) from 1993 (21.4%) to 2003 (52.6%) and to 2013 (76.5%). CONCLUSIONS: Across three decades, no discernible changes were noted in the use of stronger (experimental) designs, or in sample size. Improvements in reporting of effect sizes may be due to increased focus on this area in editorial commentaries. The increase in clear reporting of the study purpose in the abstract was likely due to a change in editorial policy in 2012. Improvements in research quality may require more stringent editorial and reviewing decisions.

An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples

Sampling strategy has critical implications for the validity of a researcher's conclusions. Despite this, sampling is frequently neglected in research methods textbooks, during the research design process, and in the reporting of our journals. The lack of guidance on this issue often leads reviewers and journal editors to rely on simple rules of thumb, myth, and tradition for judgments about sampling, which promotes the unnecessary and counterproductive characterization of sampling strategies as universally “good” or “bad.” Such oversimplification, especially by journal editors and reviewers, slows the progress of the social sciences by considering legitimate data sources to be categorically unacceptable. Instead, we argue that sampling is better understood in methodological terms of range restriction and omitted variables bias. This considered approach has far-reaching implications because in industrial–organizational (I-O) psychology, as in most social sciences, virtually all of the samples are convenience samples. Organizational samples are not gold standard research sources; instead, they are merely a specific type of convenience sample with their own positive and negative implications for validity. This fact does not condemn the science of I-O psychology but does highlight the need for more careful consideration of how and when a finding may generalize based on the particular mix of validity-related affordances provided by each sample source that might be used to investigate a particular research question. We call for researchers to explore such considerations cautiously and explicitly both in the publication and in the review of research.

Foucauldian Discourse Analysis in Psychology: Reflecting on a Hybrid Reading of Foucault When Researching “Ethical Subjects”

This article attempts to address a novel dilemma the author recently faced when undertaking qualitative psychological research into sustainable tourism. The article embraces notions of reflexivity to highlight how the research process was far removed from the sanitised version often presented in research methods textbooks. The article provides a reflexive account of the struggles of analysing Internet and interview data in relation to sustainable tourism via the dominant version of Foucauldian Discourse Analysis familiar to many qualitative/critical psychologists. Turning to an account of Foucault’s later work on ethics, this article presents an alternative approach to Foucauldian Discourse Analysis that adopts a hybrid reading of Foucault’s work on power, knowledge, and ethics. Drawing on Foucault’s four precepts helps us explore the ways individuals “cultivate the self as an ethical subject,” and interview data are presented to highlight the ways such an approach can enrich analysis. It is concluded that while presenting issues surrounding understandings of structure and agency, such an approach did offer a pragmatic solution to an ethical question and may indeed be useful in a range of other areas.

Principles for ethical research involving humans: ethical professional practice in impact assessment Part I

Drawing on various national statements on the ethical conduct of research, the codes of ethics of professional associations and international agencies, and ethical guidelines in social research methods textbooks, this paper identifies current principles for ethical research involving humans and discusses their implications for impact assessment practice generally and social impact assessment specifically. The identified ethical principles include: respect for participants, informed consent, specific permission required for audio or video recording, voluntary participation and no coercion, participant right to withdraw, full disclosure of funding sources, no harm to participants, avoidance of undue intrusion, no use of deception, the presumption and preservation of anonymity, participant right to check and modify a transcript, confidentiality of personal matters, data protection, enabling participation, ethical governance, provision of grievance procedures, appropriateness of research methodology, and full reporting of methods. Topics such as a conflict of interest, moral hazard and duty of care are also considered. Also discussed are Indigenous peoples’ rights and their implications for ethical impact assessment processes.

Writing history into management research

This essay addresses the apparent exclusion of historical methods from management research methods textbooks. We suggest that this alleged ostracism is in fact only partial, if the notion of history is defined more broadly to include historicizing research methodologies such as discourse analysis. We also note the reluctance of the community of historians to be clear or reflexive about the methods in use as a potential explanation for the separation historians feel. We conclude by arguing that the community of historians would be better served by a greater degree of openness and reflexivity, in order to construct a more emancipated methodological present and achieve more presence in methods textbooks.

Preparing Teachers of Second Language Reading

In preparing teachers of English language learners (ELLs), teacher educators face a formidable challenge in the area of reading. Reading is a complex skill that is critical to ELLs' academic achievement. Given the complexity and importance of reading and reading instruction, what topics should be addressed by teacher educators in methods courses? This article reports on a study designed to answer this question. The ESOL faculty in a small urban school district were interviewed about reading and observed in their classrooms over the course of 2 years. Six issues were identified as important to these teachers. They were (a) working with a range of learner proficiencies; (b) the use of materials; (c) instructional practices in the areas of decoding skills, vocabulary, writing, and thematic teaching; (d) developing students' love of reading; (e) coping with mainstream teachers and school demands; and (f) working with students who have limited proficiency or schooling in their first languages. The article compares issues raised by the practicing teachers in interviews and observations with the research literature and methods textbooks and outlines considerations for a methods curriculum.

The Status of Visual Data in Research Methods Textbooks for Sport and Leisure Programmes

This paper evaluates the status and role of visual data in research methods textbooks for programmes relating to sport and leisure. The paper highlights the scope for greater sophistication in the analysis of visual data, including mediated data. Key textbooks in the field of research methods for sport and leisure are reviewed; developing the argument that the growth of academic interest in visual and mediated forms of sporting and leisure experience is insufficiently represented. The claim is made that student research practice may benefit from the teaching of research methods that include an emphasis on the analysis of visual data, in particular semiotics.

Issues and dilemmas in teaching research methods courses in social and behavioural sciences: US perspective

This paper is an attempt to initiate a dialogue among the 'first generation' of those who teach mixed methods research courses in the social and behavioural sciences. It is argued that in the past three decades, research methodology, and the teaching of research, has faced strong pressures toward polarizing into a qualitative and quantitative dichotomy. Research courses are either qualitative or quantitative, and graduate students are often encouraged to choose a 'track' early on in their education. Following the structure of most introductory research methods textbooks, when both approaches (qualitative and quantitative) are taught in a research course, they are discussed separately, with little effort to link them together. On the other hand, strong pressures are currently being faced from applied researchers in the field to train 'research experts' who are able to incorporate both the qualitative and quantitative methods and approaches in their projects. It is argued that those who teach research methodology in the social and behavioural sciences have a responsibility to prepare their students for a professional world that is increasingly using mixed methods. Issues related to the teaching of mixed methods in each of five phases of a typical research project are presented. A sample course with a mixed methods approach is also presented in this article.

Research methods for criminal justice graduate students: Comparing textbook coverage and classroom instruction

Research methods are perhaps the most difficult subject matter to teach in the graduate criminal justice curricula. This is in part due to the mix of practitioners and aspiring researchers in most criminal justice graduate departments, leading many instructors to question whether their own coverage is in line with the needs of their graduate students as well as the coverage of other instructors. Also complicating matters is that research methods textbooks are not geared specifically for the graduate level, and thus may neglect topics that are necessary for today's criminal justice graduate students. The present study addresses these concerns by providing an analysis of 11 current criminal justice research methods textbooks, as well as a survey of 36 instructors of graduate criminal justice and criminology research methods classes. Both the texts and instructors are found to place a strong focus on quantitative methods, while the textbooks tend to place a greater emphasis on qualitative methods than the instructors. Moreover, both the texts and instructors neglect topics crucial to today's criminal justice graduate student, including grant writing, article writing and critiquing, and standards for collaborative research efforts.

The coverage of ethics in research methods textbooks*

Ethics is one of the most important topics in criminal justice and criminological research, yet it is also one of the most neglected. Indeed, it is common for professors to merely gloss over ethical issues at the end of their research methods classes, or to exclude them completely. The present study attempts to determine if such disregard for ethics carries over into the coverage found in criminal justice research methods textbooks. Eleven research methods textbooks that are commonly used in criminal justice and criminology were analyzed, and coverage of ethics was found to vary greatly across the books. Unfortunately, we also found that some important issues within research ethics are being omitted in the majority of the books, and we make recommendations toward expanding the coverage of research ethics.

Teaching Students about Graphs

Preparing graphs is an essential part of data analysis and representation. Although psychologists use graphs extensively, most introductory statistics and research methods textbooks provide a limited review of good techniques for designing graphs. In this article, I briefly review the research on graphical techmquesand references on graphing.

The Changing Nature of Sociological Inquiry: A Review of Research Methods Textbooks

The Changing Nature of Sociological Inquiry: A Review of Research Methods Textbooks