Journal Of Experimental Education Research Articles

Promoting Deep Learning In First-Year Physiology Through Structured In-class Activities: Building a Bridge for Success

The transition from high school to university is often a diffcult journey for learners. From data we have collected (unpublished), many students report successful use of surface learning strategies in secondary school, such as highlighting and memorizing their notes or rote recall using flash cards. As such, most students continue using these same techniques in first-year STEM courses1, such as physiology. Furthermore, first-year learners are often unsure how to change their study habits, even after receiving a disappointing midterm grade, as most post-secondary students report that they have not been taught how to study2. This leads to the reinforcement of surface learning, creating a larger knowledge gap as students progress into upper-year courses without having achieved deeper learning in pre-requisite courses. Although many universities offer learning strategy workshops, it has been shown that teaching students these skills within the context of their course work is the most effective3. We hypothesize that modelling deep learning strategies through a variety of weekly hands-on activities will positively influence students’ self-reported study strategies, leading to the incorporation of these active learning techniques in their personal study time. Our first-year physiology course is a pre-requisite for both Kinesiology students and those in the Foods and Nutrition program. Approximately 420 students are enrolled and includes those in science and non-science majors. We have 12 tutorial sections of approximately 40 learners each and 21 weeks of tutorials over the academic year. These tutorials serve as active learning sessions that apply lecture content from the week earlier. Each week, we have designed activities such as concept mapping, case studies, card sorting, problem set worksheets, and Play-Doh modelling, to name a few. Students complete a learning attitude survey after each activity, where quantitative and qualitative data is collected, and two test-your-knowledge exit questions on the content covered by that activity. Students experience each activity in both semester 1 and semester 2, although for different course material. We will present examples of activities used during tutorials as well as preliminary data, which is currently being collected, on the learning attitudes submitted by our students for these activities. We anticipate that as deeper learning strategies are modelled in tutorials, students will be more likely to adapt these techniques as they study. We also anticipate that repeated exposure to the same learning activity in semester 2 will positively influence their attitude toward the usefulness of that tool. Physiology educators have embraced a shift away from didactic lectures to expand the use of problem-based learning in recent years. Our study contributes concrete examples that any educator could implement in their classroom and the associated learning attitudes regarding each activity from a diverse group of students taking their first physiology course. References cited: 1Cook, A. and Leckey, L. (1999) Do expectations meet reality? A survey of changes in first-year student opinion. Journal of Further and Higher Education. 23: 157-171. 2Kornell, K. and Bjork, RA. (2007) The promise and perils of self-regulated learning. Psychonomic Bulletin & Review. 14: 219-224. 3Hattie, J., Biggs, J., and Purdie, N. (1996) Effects of learning skills interventions on student learning, a meta-analysis. The Journal of Experimental Education. 66: 99-136. No support or funding. This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Envisioning the future of learning and teaching engineering in the artificial intelligence era: Opportunities and challenges

Envisioning the future of learning and teaching engineering in the artificial intelligence era: Opportunities and challenges

Modeling multiple dependent variables in meta-analysis of single-case experimental design using multilevel modeling.

Although meta-analyses of single-case experimental design (SCED) often include multiple types of dependent variables (DVs), multiple DVs are rarely considered within models in the analysis. Baek et al. (Journal of Experimental Education, 90(4), 934-961, 2022) identified several statistical issues that arise when researchers fail to model multiple DVs in meta-analyses of SCED data. However, the degree to which non-modeling of multiple DVs impacts the results of the meta-analysis of SCED has not been fully examined. In this simulation study, we have systematically investigated the impact of non-modeling of multiple DVs when analyzing meta SCED data using multilevel modeling. The result demonstrates that modeling multiple DVs has advantages over the non-modeling option for meta-analysis of SCED. Modeling multiple DVs enables the determination of precise effects from different DVs in addition to the unbiased and accurate average effect and accurate estimates and inferences for the error variances at the study level as well as the observation level. The current study also reveals potential factors (i.e., the number of DVs, degree of heterogeneity in the level-1 error variances and autocorrelation, and presence of the moderator effect) that impact the precision and accuracy of the variance parameters.

Does Two-Stage Testing Promote Long-Term Individual Learning?

Two-stage testing (TST) involves individual testing followed by taking the same test in teams. Previously, Vogler and Robinson (The Journal of Experimental Education, 84(4), 787–803, 2016) found that TST facilitated individual performance. The present study addressed methodological limitations in the Vogler and Robinson study in two replications along with isolating the components of TST in a laboratory study. In Experiments 1 and 2, where undergraduates remained in testing conditions (individual or TST) for the entire semester, no TST advantages were found. Experiment 3 controlled for the effects of initial test format [short answer (SA) versus multiple choice (MC)], mode (individual vs. group), and repetition (once vs. twice) on new and old (appeared on previous tests) MC and SA items. Two weeks after watching a lecture and being tested once or twice individually or in teams, students were tested once more individually. Results revealed effects for repetition (taking an initial test twice was better than once—confirming test-enhanced learning) and format (taking an initial SA test was better than MC—confirming the superiority of supply over recognition tests). However, taking an initial team test was no better than taking an individual test. TST alone may not lead to individual learning gains.

Framing and Studying Learning and Identity in Virtual Learning Environments: Introduction to the Special Issue

In a 21st century context of rapid technological advancement, virtual learning environments (VLEs) such as games have been increasingly designed, implemented, and researched for their capacity to offer immersive and interdisciplinary experiences that support student motivation and learning. As such, VLEs have acquired greater mainstream acceptance as tools for learning that is embodied, enactive (i.e. learning by doing), extended into the learners’ environment, and embedded in authentic and potentially valuable contexts. However, this domain is still in its infancy and requires research for developing theories of the learning and identity mechanisms occurring in VLEs, evidence-based measurement of these processes, and design principles for virtual learning environments and experiences that promote learners’ knowledge, identity processes, and career paths. As such, this special issue of The Journal of Experimental Education seeks to provide education scholars with insight into current theoretical and methodological approaches to conceptualize, facilitate, and empirically examine learning and identity in VLEs. It brings together empirical examples of conceptually, methodologically, and analytically well-grounded research to illustrate the learning and identity processes, characteristics, and outcomes that VLEs can facilitate. A guest commentary and an invited theoretical article provide rich takeaways for advancing research and practice in this burgeoning field.

Relationship between Socio-Economic Status, Interpersonal Communication, and School Climate with Parental Involvement in Early Childhood Education

Relationship between Socio-Economic Status, Interpersonal Communication, and School Climate with Parental Involvement in Early Childhood Education

Complex Systems Approaches to Educational Research: Introduction to the Special Issue

To explore the movement toward increased complex systems (CS) research in education, this special issue of the Journal of Experimental Education was developed to identify some under-examined places where CS approaches have advanced research. The articles provide empirical examples of research leveraging methods and analyses from complexity science. In this introduction article to the special issue, we discuss the authors’ contributions to defining and explicating concepts and methods that fall under the umbrella of CS perspectives and how these methods were used to investigate complex processes in their topics of study. The articles capture the goal of the special issue to demonstrate how the research questions framed by CS assumptions can change our expectations about the very nature of the processes under study in education. We then take a more global perspective and acknowledge the commonalities amongst the papers, including (a) the reliance on intensive data to answer research questions, and (b) the use of dynamic approaches that yield findings about the stability and change of these systems and phenomena.

EFEKTIVITAS IMPLEMENTASI LESSON STUDY LEARNING COMMUNITY DALAM MENINGKATKAN KUALITAS PEMBELAJARAN

Abstrak: Penelitian ini dilakukan guna mendeskripsikan persepsi dosen model dalam efektivitas menerapkan lesson study learning community (LSLC), serta proses monitoring dan evaluasi dalam pelaksanaan LSLC pada proses pembelajaran. Pendekatan yang digunakan dalam penelitian ini ialah pendekatan kualitatif. Informan ditentukan dengan teknik purposive dan snowball sampling. Proses pengumpulan data dilakukan dengan observasi, wawancara serta dokumentasi. Data penelitian yang terkumpul selanjutnya dianalisis menggunakan pola interaktif Miles, Huberman, dan Saldana, meliputi: pengumpulan data, kondensasi data, penyajian data, dan penarikan kesimpulan. Hasil penelitian menunjukkan bahwa implementasi LSLC sangat efektif dilakukan dalam meningkatkan kualitas pembelajaran. Hal ini dibuktikan dengan adanya kolaborasi antar dosen model dalam melaksanakan proses pembelajaran, perkembangan tingkat keprofesionalan dosen dalam melaksanakan pembelajaran mengalami peningkatan yang signifikan sehingga berdampak pada kualitas pembelajaran yang lebih optimal. Selain itu, proses monitoring dan evaluasi oleh tim pengembang LSLC pada tingkat fakultas juga menambah keefektifan LSLC dalam meningkatkan kualitas pembelajaran.
 Abstract: This research was conducted to describe the perception of model lecturers in the effectiveness of implementing the lesson study learning community (LSLC), as well as the monitoring and evaluation processes in implementing the LSLC in the learning process. This research using a qualitative approach. Informants were determined by purposive technique and snowball sampling. The data collection process was carried out by observation, interviews and documentation. The collected research data were then analyzed using the interactive patterns of Miles, Huberman, and Saldana, including: data collection, data condensation, data presentation, and drawing conclusions. The results showed that the implementation of LSLC was very effective in improving the quality of learning. This is evidenced by the collaboration between model lecturers in carrying out the learning process, the development of the professional level of the lecturers in implementing learning has increased significantly so that it has an impact on the quality of learning that is more optimal. In addition, the process of monitoring and evaluation by the LSLC development team at the faculty level also adds to the effectiveness of LSLC in improving the quality of learning.

Teachers’ self-efficacy and its association with their quality of relationships with pre and early adolescents: a hierarchical linear modelling approach

ABSTRACTThe present study was set out to investigate the association between teachers’ perception of the quality of their relationships with their students’ and teachers’ professional self-efficacy. Fifty public school teachers were asked: (a) to assess their perceived levels of efficacy and (b) to randomly select eight students from their classrooms (4 males and 4 females) for whom (n = 395) to evaluate the quality of their relationships. The two administered instruments were the Teachers’ Sense of Efficacy Scale [Tschannen-Moran and Hoy. 2001. “Teacher Efficacy: Capturing an Elusive Construct.” Teaching and Teacher Education 17 (7): 783–805] and the Teacher–Student Relationship Inventory (TSRI; Ang. 2005. “Development and Validation of the Teacher–Student Relationship Inventory Using Exploratory and Confirmatory Factor Analysis.” The Journal of Experimental Education 74 (1): 55–74). Initially, the psychometric properties of the TSRI were examined. Confirmatory factor analysis showed that the Greek version of TSRI has the same three-factor structure reported in other cultures. Multilevel analysis revealed that teachers’ beliefs were not related to any of the TSRI dimensions. Teachers’ and students’ gender seem to affect teachers’ perceptions of their relationships with their students.

Lost in Translation: Thoughts Regarding the Translation of Existing Psychological Measures Into Other Languages

Over the last decades, psychological science has become more and more international (Alonso-Arbiol & van de Vijver, 2010). In order to be able to compare research findings from different countries and in different languages, it is important to ensure the comparability of the assessment methods used. Thus, translations of standard measures have now appeared in a multitude of different languages. The increased availability of different language versions of the same measurement tools would seemingly create a need to internationally publish articles introducing these versions. An often-heard argument is that, in order for the translated measure to be useful, an English language publication has to test its psychometric properties. Let us scrutinize this argument more closely. An argument often used to support the need for an English-language publication concerning a translated instrument is that the international research community would not accept research findings based on translated instruments whose psychometric properties have not also been published in English. This is a dangerous thought, on the one hand, because it implies a certain mistrust: A peer-reviewed publication would otherwise suffice. On the other hand, besides this theoretical consideration, we will address some more practical issues below. When a new measurement tool is constructed, among the first questions to be answered are the following: 1. For what measurement purpose is the instrument designed (e.g., personnel selection, clinical assessment)? 2. What is the target population (e.g., adolescents, adults, patients)? 3. Who will employ the instrument (e.g., researchers, practitioners)? When translating an existing measurement tool into another language, clearly one must answer these questions as well. Obviously, the original authors have already defined the measurement purpose of the instrument. The same usually holds true for the target population and the instrument user. However, during translation of a measurement tool these aspects should be reconsidered and expressed specifically. Sometimes a translation goes hand in hand with a changed (or changing) target population or measurement purpose. Thus, measures are usually translated with a specific goal. Most translations probably aim for one of the following three goals: 1. to make a particular instrument available in a different language, 2. to provide a means for cross-cultural research, 3. to conduct research on the specific instrument itself. Not all of these translation goals have the same readership in mind. Consequently, depending on the translation goal, an English-language publication of the instrument’s psychometric properties may be more or less useful. When making a standard instrument available to a different language community, practitioners usually represent the targeted readership. This in itself is a valuable goal because practitioners probably make up the majority of test-users. However, it can be assumed that most practitioners have better access to journals published in their own language. Moreover, practitioners most likely find it easier to comprehend the ever-increasing complexity of data analyses if they can read it in their native language. Consequently, publishing findings in national journals should in fact positively affect the acceptance of the translated instrument. For this reason, this should be the preferred outlet of the article if the translation goal is to make the instrument available to local practitioners. This is especially true if testing the psychometric properties closely mirrors the original publication, so that the translated instrument and its inherent qualities are introduced to the practitioner at the same time. There are several examples where such a publication strategy was successful. Rammstedt and John (2005) as well as Lang, Ludtke, and Asendorpf (2001) published German versions of the Big Five Inventory (BFI), albeit differing in length. Both papers are cited 8–9 times per year on average since their appearance. Moreover, there are Germanas well as English-language papers that apply these translated BFI versions. Thus, the publications reached the intended audience and sparked new research that was accepted internationally.

Assessing Students' Understanding of Macroevolution: Concerns regarding the validity of the MUM

In a recent article, Nadelson and Southerland (2010. Development and preliminary evaluation of the Measure of Understanding of Macroevolution: Introducing the MUM. The Journal of Experimental Education, 78, 151–190) reported on their development of a multiple-choice concept inventory intended to assess college students' understanding of macroevolutionary concepts, the Measure of Understanding Macroevolution (MUM). Given that the only existing evolution inventories assess understanding of natural selection, a microevolutionary concept, a valid assessment of students' understanding of macroevolution would be a welcome and necessary addition to the field of science education. Although the conceptual framework underlying Nadelson and Southerland's test is promising, we believe the test has serious shortcomings with respect to validity evidence for the construct being tested. We argue and provide evidence that these problems are serious enough that the MUM should not be used in its current form to measure students' understanding of macroevolution.

Teacher Efficacy In the Context of Teaching Low Achieving Students

This paper focused on the efficacy of Singapore teachers who teach low achieving adolescent students. Three dimensions of self-reported teacher efficacy—instructional strategies, classroom management, and student engagement—were examined in relation to teacher attributes and the teacher–student relationship. Data were obtained from the Teacher Self-Efficacy Scale Tschannen-Moran and Woolfolk Hoy (Teaching and Teacher Education 17:783–805, 2001) and the Teacher–Student Relationship Scale Ang (The Journal of Experimental Education 74:55–73, 2005). Significant differences between novice teachers and experienced teachers emerged in teacher efficacy beliefs in relation to instructional strategies, classroom management, and student engagement. Conflict in teacher–student relationship was found to predict teacher efficacy for teachers of low achieving students. Implications for teachers’ professional development were discussed.

Robust testing of level changes in interrupted time-series analysis

Ramsey and Ramsey [Ramsey, P.P. and Ramsey, P.H., 2003, Comparing lease-squares lines for testing level changes in interrupted time-series analysis. Journal of Statistical Computation and Simulation, 73, 31–44.] have shown that a composite procedure (CP) can provide accurate tests of level changes in interrupted time-series analysis. A CP combines the Mann–Wald procedure [Gottman, J.M., 1981, Time-Series Analysis (New York: Cambridge University Press): Mann, H.B. and Wald, A., 1943, On the statistical treatment of linear stochastic difference equations. Econometrica, 11, 173–220.] with the Kromrey and Foster-Johnson [Kromrey, J.D. and Foster-Johnson, L., 1996, Determining the efficacy of intervention: the use of effect sizes for data analysis in single-subject research. The Journal of Experimental Education, 65, 73–93.] regression procedure allowing for testing changes in level, as well as changes in both trend and level in the presence of trend effects in time-series data. Monte Carlo simulation was used to demonstrate Type I error rates for sample sizes from 5 to 20. A variety of composite procedures were found to provide such control.

Organizational learning and climate as predictors of self-efficacy

This study broadens organizational contextual considerations by examining organizational learning, participation in organizational learning activities and organizational climate as possible predictors of self-efficacy. As schools have been studied as organizations (Handy, 1986, Understanding schools as organizations, Harmondsworth: Penguin; Ostroff, 1993, Organization Behavior and Human Decision Process, 56, 56–90; Ostroff & Schmitt, 1993, Acadamy of Management Journal, 36(6), 1345–1361), this study provides regression results from n = 679 teachers and indicated that organizational climate and organizational learning were significant predictors of teacher self efficacy, controlling for several individual level variables (i.e., participation in organizational learning, personal self-efficacy, and teaching efficacy). Differences between results from the sample of teachers and n = 734 non-teachers (e.g., principals, assistant principals, administrators, counselors, paraprofessional, custodial, food service, and transportation staff) were examined with regard to predictor variables for teaching efficacy. For teachers, both organizational learning and personal self-efficacy were significant predictors of teaching efficacy, whereas for the group of non-teachers, they were not. In both groups, however, participation in organizational learning was a significant predictor, while organizational climate was not (contrary to Taylor & Tashakkori, 1995, Journal of Experimental Education, 63(3), 217–230). Overall, findings provide evidence for extending the research beyond schools and including organizational learning as a potentially important variable in further studying individual outcomes and organizational effectiveness.

Assessing tenth-grade students’ problem solving ability online in the area of Earth sciences

This study examined tenth-grade students' (n=263) problem solving ability (PSA) online through assessing students' domain-specific knowledge (DSK) and reasoning skills (RS) in Earth sciences as well as their attitudes toward (AT) Earth sciences related topics in a secondary school of Taiwan. The students' PSA was evaluated based on a previous model (Chang, C. Y. (2004, November 26-27). Trends in assessing student earth science problem solving ability: the importance of domain-specific knowledge and reasoning skills in earth sciences. Paper presented at the Seoul Conference for International Earth Science Olympiad (IESO), Seoul, Korea; Chang, C. Y., & Barufaldi, J. P. (submitted). Does problem solving=prior knowledge+reasoning skills in science? An exploratory study. Journal of Experimental Education; Chang, C. Y., & Weng, Y. H. (2002). An exploratory study on students' problem-solving ability in earth science. International Journal of Science Education, 24(5), 441-452) which empirically established that students' PSA is a composite of DSK, RS and AT subscales. Major findings are as follows: (a) The correlation coefficient among students' DSK, RS and AT was relatively small, indicating that these subscales might have successfully represented different constructs of students' PSA; (b) a significantly positive correlation existed between students' PSA total scores and each subscale. It is, therefore, suggested that students' PSA may be potentially assessed online by measuring their essential components in the area of Earth sciences.

Research methods and data analysis procedures used by educational researchers

To assess the status and the trends of subject matters investigated and research methods/designs and data analysis procedures employed by educational researchers, this study surveyed articles published by the American Educational Research Journal (AERJ), Journal of Experimental Education (JEE) and Journal of Educational Research (JER) from 1971 to 1998. Results of the study show that almost three quarters of subject matters in each journal focused on psychology in education; teachers; teaching/instruction; measurement/assessment; and methods of inquiry. Frequently used research methods were experimental research; descriptive research; correlational study; comparative approach; and survey. Descriptive statistics, ANOVA/ANCOVA, correlation, regression, t‐test, and psychometric statistics were frequently used data analysis procedures. There was a continuous drop of percentages of experimental quantitative research since the mid 1980s with a relative increase of non‐experimental qualitative research in AERJ and JER. Though some new data analysis procedures were introduced into educational research during the last decade, the trends of frequently used data analysis procedures stayed relatively steady. These trends were discussed in light of the changing editorial policies of the journals. Implications for the training of educational researchers were also noted.

Women’s Involvement in Educational Psychology Journals from 1976 to 2004

Previously, Robinson, McKay, Katayama, and Fan (1998) examined women’s involvement in six educational psychology journals (American Educational Research Journal, Contemporary Educational Psychology, Educational Psychologist, Educational Psychology Review, Journal of Experimental Education, and Journal of Educational Psychology) from 1976 to 1996 and found that although women made gains as authors and editorial board members, they remained underrepresented as editors. In this Reflections on the Field article, we compare the Robinson et al. data with our more recent data. We found that from 1996 to 2004, the percentage of women members in education and psychology organizations has risen. Meanwhile, the percentage of women authors has remained stable; the percentage of women editorial board members has risen slightly; and the percentage of women editors has risen dramatically. We also found that the percentage of women authors and editorial board members lags slightly behind what would be expected based on organizational membership percentages for women. On the other hand, there are now more female editors than would be expected.

Is Educational Intervention Research on the Decline?

The authors examined intervention studies that appeared in 4 educational psychology journals (Cognition & Instruction, Contemporary Educational Psychology, Journal of Educational Psychology, Journal of Experimental Education) and the American Educational Research Journal (AERJ) in 1983 and from 1995 to 2004. The majority of studies included adults (age 18 and older) as participants, administered brief (less than 1 day) interventions, assessed intervention effects immediately following the intervention, and did not report treatment integrity. Most studies included multiple outcome measures and exhibited an increase in effect-size reporting from 4% in 1995 to 61% in 2004. The percentage of total articles based on randomized experiments decreased over the 21-year period in both the educational psychology journals (from 40% in 1983 to 34% in 1995 to 26% in 2004) and AERJ (from 33% to 17% to 4%). Limitations of the study and future research issues are discussed.

Guest editorial: Improving the interpretation and reporting of quantitative research

Guest editorial: Improving the interpretation and reporting of quantitative research

Statistical Significance Testing Practices in The Journal of Experimental Education

The APA Task Force on Statistical Inference (Shea, 1996), the new APA (1994) style manual, and a recent series of articles on statistical significance testing (cf. Kirk, 1996; Schmidt, 1996; Thompson, 1996, 1997) all have prompted closer scrutiny of contemporary analytic practices. It has been suggested that the field should move away from emphasizing statistical significance tests and toward emphasizing evaluations of (a) practical significance and (b) result replicability. In the present study, 3 aspects of recommended practice in all the quantitative reports in Volumes 63 and 64 (academic years 1994-1995 and 1995-1996) of The Journal of Experimental Education (JXE) were examined. Examples of both errors and desirable practices were noted. The new JXE guidelines for contributors should result in further improvements in the use and reporting of statistical significance tests.