Abstract
The success of education with technology research is in part because the field draws upon theories and methods from multiple disciplines. However, drawing upon multiple disciplines has drawbacks because sometimes the methodological expertise of each discipline is not applied when researchers conduct studies outside of their research training. The focus here is on research using methods drawn largely from psychology, for example, evaluating the impact of different systems on how students perform. The methodological concerns discussed are: low power; not using multilevel modeling; dichotomization; and inaccurate reporting of the numeric statistics. Examples are drawn from a recent set of proceedings. Recommendations, which are applicable throughout the social sciences, are made for each of these.
Highlights
ReadingThe report by the Open Science Collaboration (2015), while focusing on psychology research, discusses topics relevant to those applicable to the Education with Technology (EwT) studies considered. Cohen (1994) presents a good discussion of what null hypothesis significance testing is and is not.CONCERN #1: POWER ANALYSIS AND SMALL SAMPLESThe hypothesis testing framework explicitly recognizes the possibility of errantly rejecting the null hypothesis
While the theme of this paper is to look at how EwT researchers deal with some issues, there is a crisis within the sciences more broadly that requires discussion
If it is relatively inexpensive to recruit participants or if your PhD/job prospects require that you detect an effect if it is as large as the minimum effect size to detect (MED), it would be wise to set your power higher, for example 95%
Summary
The field of EwT was chosen for three reasons. First, it offers valuable potential for education, though the impact has failed to live up to the potential/hype (see Cuban, 2001; Reingold, 2015). The papers and posters are examined here: the doctoral papers often sought advice on how to conduct the planned research; the industry papers often described a product or were a case study using a product; and the tutorials gave accounts of what their audiences would learn. According to their website, only 36 of the 121 papers submitted for oral presentations were accepted as oral presentations. It is important to stress that while examples will be shown to illustrate concerns, some aspects of these studies were good and overall the conference papers are high-quality. At the end of each section specific readings are recommended
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
