An Overview of Experimental and Quasi-Experimental Research in Technical Communication Journals (1992–2011)

Abstract

This study explores a comprehensive sample of experimental and quasi-experimental research within five leading technical communication journals over a 20-year period. Exploratory studies can overview how a method has evolved within a field, highlighting how it has advanced understanding of communication and identifying areas for further inquiry. Research questions: (1) How has experimental research in technical communication journals developed over the 20-year period? Specifically, how much is being published, which journals publish experiments, what topics are being explored, and what fields are informing this research? (2) What content characterizes experimental research in technical communication? Specifically, how explicit are the research questions/hypotheses, are the results of pilot studies reported, what are the sample sizes and populations used, and what measures do researchers use? (3) Who publishes experimental research in technical communication? Specifically, which authors and affiliates are most associated with experimental research, and how does the sample's gender and authorship distribution compare to existing research? Literature review: We first address how scholars have assessed research in technical communication and how these findings implicate experimental research. We then review features of other exploratory studies that inform this study's design. Methodology: We conducted a quantitative and qualitative analysis of 137 experiments, a comprehensive sample identified from a corpus of 2,118 refereed papers published from 1992 to 2011. We coded 14 variables related to the causal relationships that the experiments addressed and who produced the research. We subjected the data to multiple statistical measures, including contingency table analysis and correspondence analysis. Results and conclusions: Over the 20 years, the journals published experimental research at a consistent rate. This could indicate that these methods have a stable presence in the field, or a discouraging sign that output is not on the rise despite calls from leading scholars. IEEE Transactions on Professional Communication (TPC) emerged as a strong producer of experiments, publishing 45% of the sample. TPC was also associated with most recent experiments, assuming this role from Journal of Technical Writing and Communication, which was associated with early experiments. In addition, TPC, Journal of Business and Technical Communication, and Technical Communication correlated with experiments on collaboration, pedagogy, and intercultural communication, respectively. The results also revealed that recent experiments reported significantly more explicit research question/hypotheses and pilot studies, an encouraging sign for the quality of future experiments. Finally, Spyridakis published the most experiments over the past 20 years, and researchers at the University of Washington and the University of Twente were the top affiliates associated with output. The configuration of both of these institutions' programs, which seem to align with a traditional science model, might suggest how the evolution of technical communication programs impacts the type of research that its affiliates produce. Our results are limited by the small, though comprehensive, sample and the exploratory natures of measures like correspondence analysis. Future research could use the proposed framework to investigate the evolution of other research methods in technical communication, strengthening our body of knowledge.