Open-ended Learning Environments Research Articles

Fostering computational thinking through digital storytelling: a distinctive approach to promoting computational thinking skills of pre-service teachers

AbstractThe present study illustrates digital storytelling (DS) as an open-ended learning environment, which is one of the ways of preparing pre-service teachers (PST) to practice computational thinking (CT) in their future teaching. An 8-week teacher training program was organized and presented. The study aims to investigate the program’s impact on the PST's CT skills, CT-integrated DS design skills, and their perspectives. A study group of fifty-two second-year PSTs took part in the Instructional Technologies course. An embedded mixed-method design was used. The CT scale was used as a pre-test and post-test to examine the development of the PST’s CT skills for quantitative data collection. The qualitative data comprise the CT-integrated DS projects developed by the PST during the training and the reflection reports they wrote at the end of the training. The paired samples t-test was used in the analysis of quantitative data. The DS projects were evaluated using the rubrics with the story/storyboard, DS, and CT skills dimensions. Reflection reports were analyzed using thematic analysis. The findings revealed a significant improvement in the PST’s CT skills after the training program. The qualitative findings show the PST’s perspectives on CT definitions, the perceived benefits of CT and DS, and the use of DS to promote CT. The qualitative findings also present evidence for the CT-integrated DS design skills of the PST. Therefore, the study presents early findings showing that CT components could be integrated with DS and used in teacher education effectively without any pressure to use programming tools. The study also shows how DS is gradually integrated with the CT components as a suggestion for future studies.

Open-Ended Mathematics Learning

Mathematical learning has an important role and is often prioritized in education. In K-16 education, algebra is one of the most vital mathematical content domains: it represents one of the top barriers for students pursuing a postsecondary education. Game-based learning has been effective in fostering classroom math learning environments that are collaborative and focused on conceptual understanding. Sandbox games provide open-ended learning environments where players can set their own goals and level of effort. As part of the project “Math Snacks,” the team designed Agrinautica, a sandbox game to enable constructivist-informed early algebra learning. This article identifies design recommendations for creating meaningful sandbox games for learning, considering students' and teachers' needs. Researchers discuss the decisions to create a sandbox game and describe challenges inherent in math learning through sandbox-type gameplay. This study provides impact results from a large-scale study of users of the game, and shares recommendations for developing future sandbox learning games.

Using Minecraft to cultivate student interest in STEM

Due to the popularity and flexibility of Minecraft, educators have used this game to develop instructional materials and activities to cultivate student interests in science, technology, engineering and mathematics (STEM). One example of such an initiative is the What-If Hypothetical Implementations in Minecraft (WHIMC) project of the University of Illinois Urbana Champaign. The study reported in this paper describes a WHIMC deployment in the Philippines and the effects this deployment had on student STEM interest. The study used quantitative and qualitative methods to determine the effect of WHIMC on the STEM interest of Filipino students. We performed quantitative analysis of the pre- and post-STEM Interest Questionnaire (SIQ) ratings and Game Experience Questionnaire (GEQ) ratings of the high- and low-performers to determine the effect of using WHIMC in the students’ STEM interest and the difference between the game experience of high- and low-performers, respectively. Qualitative analysis of the answers to the open-ended questions about the attributes of the module was also conducted to determine the relationship between the module attributes and student performance. The analysis of the aggregated SIQ ratings before and after using the WHIMC-based modules revealed only a minimal effect on the STEM interests of the students. However, there was a significant effect in the Choice Actions construct, which implies that students recognize the importance of studying hard if they want to pursue STEM-related careers. Further, the analysis of the overall GEQ of high-performers and low-performers also revealed no significant difference. Although no significant difference was observed in the overall GEQ, high-performers had significantly higher GEQ ratings in the Immersion dimension. This result suggested that high-performers had a more positive, engaging, and enjoyable learning experience. Moreover, the findings on the favorite module attributes suggested that students perform better in the out-of-game assessments when they like all the module attributes. This implies that students must be engaged in the game and learning task aside from being interested in the learning topic to have better assessment scores. The study also showed that open-ended learning environments coupled with tasks that demand exploration, observation, and higher-ordered thinking are demanding even on high-performers.

Do instructional strategies considering activity emotions reduce students’ boredom in a computerized open-ended learning environment?

Providing students with efficient instruction tailored to their individual characteristics in the cognitive and affective domains is an important goal in research on computer-based learning. This is especially important when seeking to enhance students' learning experience, such as by counteracting boredom, a detrimental emotion for learning. However, studies comparing instructional strategies triggered by either cognitive or emotional characteristics are surprisingly scarce. In addition, little research has examined the impact of these types of instructional strategies on performance and boredom trajectories within a lesson. In the present study, we compared the effectiveness of an intelligent tutoring system that adapted variable levels of hint details to a combination of students' dynamic, self-reported emotions and task performance (i.e., the experimental condition) to a traditional hint delivery approach consisting of a progressive, incremental supply of details following students' failures (i.e., the control condition). Linear mixed models of time-related changes in task performance and the intensity of boredom over two 1-h sessions showed that students (N = 104) in the two conditions exhibited equivalent progression in task performance and similar trajectories in boredom intensity. However, a consideration of students' achievement levels in the analyses (i.e., their final performance on the task) revealed that higher achievers in the experimental condition showed a reduction in boredom during the first session, suggesting possible benefits of using emotional information to increase the contingency of the hint delivery strategy and improve students’ learning experience.

Designing augmented reality for makerspaces: Guidelines, lessons and mitigation strategies from 5+ years of AR educational projects

Makerspaces are a relatively recent type of open-ended environment where users learn through authentic problem solving and interactions with peers. Augmented reality (AR) technology can improve learning and collaboration in such spaces, but it is unclear how one might design AR applications suitable for the physical, social and pedagogical richness of these environments. In this paper we present 5 research projects that explored the use of AR in makerspaces, covering various learning topics, physical configurations, and collaborative activities. We discuss lessons learned and distill these into 14 design guidelines. We conclude with a discussion of tensions to consider when designing AR for open-ended learning environments.

Analysing adaptive scaffolds that help students develop self‐regulated learning behaviours

AbstractBackgroundProviding adaptive scaffolds to help learners develop effective self‐regulated learning (SRL) behaviours has been an important goal for intelligent learning environments. Adaptive scaffolding is especially important in open‐ended learning environments (OELE), where novice learners often face difficulties in completing their learning tasks.ObjectivesThis paper presents a systematic framework for adaptive scaffolding in Betty's Brain, a learning‐by‐teaching OELE for middle school science, where students construct a causal model to teach a virtual agent, generically named Betty. We evaluate the adaptive scaffolding framework and discuss its implications on the development of more effective scaffolds for SRL in OELEs.MethodsWe detect key cognitive/metacognitive inflection points, that is, moments where students' behaviours and performance change during learning, often suggesting an inability to apply effective learning strategies. At inflection points, Mr. Davis (a mentor agent in Betty's Brain) or Betty (the teachable agent) provides context‐specific conversational feedback, focusing on strategies to help the student become a more productive learner, or encouragement to support positive emotions. We conduct a classroom study with 98 middle schoolers to analyse the impact of adaptive scaffolds on students' learning behaviours and performance. We analyse how students with differential pre‐to‐post learning outcomes receive and use the scaffolds to support their subsequent learning process in Betty's Brain.Results and ConclusionsAdaptive scaffolding produced mixed results, with some scaffolds (viz., strategic hints that supported debugging and assessment of causal models) being generally more useful to students than others (viz., encouragement prompts). Additionally, there were differences in how students with high versus low learning outcomes responded to some hints, as suggested by the differences in their learning behaviours and performance in the intervals after scaffolding. Overall, our findings suggest how adaptive scaffolding in OELEs like Betty's Brain can be further improved to better support SRL behaviours and narrow the learning outcomes gap between high and low performing students.ImplicationsThis paper contributes to our understanding and impact of adaptive scaffolding in OELEs. The results of our study indicate that successful scaffolding has to combine context‐sensitive inflection points with conversational feedback that is tailored to the students' current proficiency levels and needs. Also, our conceptual framework can be used to design adaptive scaffolds that help students develop and apply SRL behaviours in other computer‐based learning environments.

A systematic review of empirical studies using log data from open‐ended learning environments to measure science and engineering practices

AbstractTechnology‐based, open‐ended learning environments (OELEs) can capture detailed information of students' interactions as they work through a task or solve a problem embedded in the environment. This information, in the form of log data, has the potential to provide important insights about the practices adopted by students for scientific inquiry and problem solving. How to parse and analyse the log data to reveal evidence of multifaceted constructs like inquiry and problem solving holds the key to making interactive learning environments useful for assessing students' higher‐order competencies. In this paper, we present a systematic review of studies that used log data generated in OELEs to describe, model and assess scientific inquiry and problem solving. We identify and analyse 70 conference proceedings and journal papers published between 2012 and 2021. Our results reveal large variations in OELE and task characteristics, approaches used to extract features from log data and interpretation models used to link features to target constructs. While the educational data mining and learning analytics communities have made progress in leveraging log data to model inquiry and problem solving, multiple barriers still exist to hamper the production of representative, reproducible and generalizable results. Based on the trends identified, we lay out a set of recommendations pertaining to key aspects of the workflow that we believe will help the field develop more systematic approaches to designing and using OELEs for studying how students engage in inquiry and problem‐solving practices. Practitioner notesWhat is already known about this topic Research has shown that technology‐based, open‐ended learning environments (OELEs) that collect users' interaction data are potentially useful tools for engaging students in practice‐based STEM learning. More work is needed to identify generalizable principles of how to design OELE tasks to support student learning and how to analyse the log data to assess student performance. What this paper adds We identified multiple barriers to the production of sufficiently generalizable and robust results to inform practice, with respect to: (1) the design characteristics of the OELE‐based tasks, (2) the target competencies measured, (3) the approaches and techniques used to extract features from log files and (4) the models used to link features to the competencies. Based on this analysis, we can provide a series of specific recommendations to inform future research and facilitate the generalizability and interpretability of results: Making the data available in open‐access repositories, similar to the PISA tasks, for easy access and sharing. Defining target practices more precisely to better align task design with target practices and to facilitate between‐study comparisons. More systematic evaluation of OELE and task designs to improve the psychometric properties of OELE‐based measurement tasks and analysis processes. Focusing more on internal and external validation of both feature generation processes and statistical models, for example with data from different samples or by systematically varying the analysis methods. Implications for practice and/or policy Using the framework of evidence‐centered assessment design, we have identified relevant criteria for organizing and evaluating the diverse body of empirical studies on the topic and that policy makers and practitioners can use for their own further examinations. This paper identifies promising research and development areas on the measurement and assessment of higher‐order constructs with process data from OELE‐based tasks that government agencies and foundations can support. Researchers, technologists and assessment designers might find useful the insights and recommendations for how OELEs can enhance science assessment through thoughtful integration of learning theories, task design and data mining techniques.

The evolution of metacognitive strategy use in an open-ended learning environment: Do prior domain knowledge and motivation play a role?

There is a growing interest in viewing self-regulated learning as events unfolding over time, especially when students perform learning tasks in computer-based environments. Metacognitive activities are critical events in self-regulated learning. This study investigated the evolution of metacognitive strategy use in an open-ended computer-based learning environment, Betty's Brain. The data were from 93 sixth graders who used Betty's Brain to learn about climate change for four days. We extracted indicators of metacognitive strategy use from action logs. A knowledge test and self-report questionnaire were administrated before students started using Betty's Brain to assess prior domain knowledge and motivation, respectively. Results showed that metacognitive strategy use increased from the first to the second day and remained stable from the second to the fourth day of the study. The evolution of these behaviors varied across students. Task value and prior domain knowledge partially explained the individual differences in this evolution. Task value and prior domain knowledge also predicted the use of metacognitive strategies. Self-efficacy did not influence metacognitive strategy use. These results suggest the need for further investigation into the role of motivation and prior domain knowledge in the temporal evolution of metacognitive events.

Toward capturing divergent collaboration in makerspaces using motion sensors

PurposeThis study aims to uncover divergent collaboration in makerspaces using social network analysis to examine ongoing social relations and sequential data pattern mining to invesitgate temporal changes in social activities.Design/methodology/approachWhile there is a significant body of qualitative work on makerspaces, there is a lack of quantitative research identifying productive interactions in open-ended learning environments. This study explores the use of high frequency sensor data to capture divergent collaboration in a semester-long makerspace course, where students support each other while working on different projects.FindingsThe main finding indicates that students who diversely mix with others performed better in a semester-long course. Additional results suggest that having a certain balance of working individually, collaborating with other students and interacting with instructors maximizes performance, provided that sufficient alone time is committed to develop individual technical skills.Research limitations/implicationsThese discoveries provide insight into how productive makerspace collaboration can occur within the framework of Divergent Collaboration Learning Mechanisms (Tissenbaum et al., 2017).Practical implicationsIdentifying the diversity and sequence of social interactions could also increase instructor awareness of struggling students and having this data in real-time opens new doors for identifying (un)productive behaviors.Originality/valueThe contribution of this study is to explore the use of a sensor-based, data-driven, longitudinal approach in an ecologically valid setting to understand divergent collaboration in makerspaces. Finally, this study discusses how this work represents an initial step toward quantifying and supporting productive interactions in project-based learning environments.

Temporal pathways to learning: How learning emerges in an open-ended collaborative activity.

The learning process depends on the nature of the learning environment, particularly in the case of open-ended learning environments, where the learning process is considered to be non-linear. In this paper, we report on the findings of employing a multimodal Hidden Markov Model (HMM) based methodology to investigate the temporal learning processes of two types of learners that have learning gains and a type that does not have learning gains in an open-ended collaborative learning activity. Considering log data, speech behavior, affective states and gaze patterns, we find that all learners start from a similar state of non-productivity, but once out of it they are unlikely to fall back into that state, especially in the case of the learners that have learning gains. Those who have learning gains shift between two problem solving strategies, each characterized by both exploratory and reflective actions, as well as demonstrate speech and gaze patterns associated with these strategies, that differ from those who don't have learning gains. Further, the teams that have learning gains also differ between themselves in the manner in which they employ the problem solving strategies over the interaction, as well as in the manner they express negative emotions while exhibiting a particular strategy. These outcomes contribute to understanding the multiple pathways of learning in an open-ended collaborative learning environment, and provide actionable insights for designing effective interventions.

Measuring and Analyzing Students’ Strategic Learning Behaviors in Open-Ended Learning Environments

Measuring and Analyzing Students’ Strategic Learning Behaviors in Open-Ended Learning Environments

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

The integration of computational modelling in science classrooms provides a unique opportunity to promote key 21st century skills including computational thinking (CT) and collaboration. The open-ended, problem-solving nature of the task requires groups to grapple with the combination of two domains (science and computing) as they collaboratively construct computational models. While this approach has produced significant learning gains for students in both science and CT in K–12 settings, the collaborative learning processes students use, including learner regulation, are not well understood. In this paper, we present a systematic analysis framework that combines natural language processing (NLP) of collaborative dialogue, log file analyses of students’ model-building actions, and final model scores. This analysis is used to better understand students’ regulation of collaborative problem solving (CPS) processes over a series of computational modelling tasks of varying complexity. The results suggest that the computational modelling challenges afford opportunities for students to a) explore resource-intensive processes, such as trial and error, to more systematic processes, such as debugging model errors by leveraging data tools, and b) learn from each other using socially shared regulation (SSR) and productive collaboration. The use of such SSR processes correlated positively with their model-building scores. Our paper aims to advance our understanding of collaborative, computational modelling in K–12 science to better inform classroom applications.

Time-driven modeling of student self-regulated learning in network-based tutors

ABSTRACT This study applies a time-driven approach to model self-regulated learning (SRL) on the basis of elapsed time metrics in the context of open-ended learning environments (OELEs), specifically, network-based tutors. In doing so, we examine how students allocated attentional resources to distinct phases of SRL as a measure of depth of information processing. Student teachers (N=68) were assigned to two different versions of nBrowser: a static version where the network did not converge on the basis of student interactions and a dynamic version where the network was continually updated by the system. Students designed a lesson plan and completed pre- and post-test self-report measures of knowledge gains. In both the experimental conditions, the results show four distinct SRL profiles that are relatively consistent and can be detected on the basis of behavioral patterns logged by the system across behaviors, namely, planning, requesting hints, studying examples, and monitoring. Although students who allocated more attentional resources to studying examples performed more poorly, their efforts to engage in planning, requesting hints, and monitoring were found to predict knowledge gains and design skills. Furthermore, students assigned to the dynamic version of the system outperformed those assigned to the static version in pedagogical knowledge gains.

Modeling and Analyzing Inquiry Strategies in Open-Ended Learning Environments

Modeling and predicting student learning in computer-based environments often relies solely on sequences of accuracy data. Previous research suggests that it does not only matter what we learn, but also how we learn. The detection and analysis of learning behavior becomes especially important, when dealing with open-ended exploration environments, which do not dictate prescribed learning sequences and skills. In this paper, we work with data collected from an inquiry-based environment. We demonstrate that 1) students’ inquiry strategies indeed influence the learning outcome, and 2) students’ inquiry strategies also seem to be predictive for their academic achievement. Furthermore, we identified a new positive inquiry strategy, which has not yet been described in the literature. We propose the use of a probabilistic model jointly representing student knowledge and strategies and show that the inclusion of learning behavior into the model significantly improves prediction of external posttest results compared to only using accuracy data, a result that we validated on a second data set. Furthermore, we cluster the children into different groups with similar learning strategies to get a complete picture of students’ inquiry behavior. The obtained clusters can be semantically interpreted and are not only correlated to learning success in the game, but also to students’ science grades and standardized math assessments. We also validated the cluster solution on a second data set. The inquiry-based environment together with the clustering solution has the potential to serve as an assessment tool for teachers and tutors.

C2STEM: a System for Synergistic Learning of Physics and Computational Thinking

Synergistic learning combining computational thinking (CT) and STEM has proven to be an effective method for advancing learning and understanding in a number of STEM domains and simultaneously helping students develop important CT concepts and practices. We adopt a design-based approach to develop, evaluate, and refine our Collaborative, Computational STEM (C2STEM) learning environment. The system adopts a novel paradigm that combines visual model building with a domain-specific modeling language (DSML) to scaffold learning of high school physics using a computational modeling approach. In this paper, we discuss the design principles that guided the development of our open-ended learning environment (OELE) using a learning-by-modeling and evidence-centered approach for curriculum and assessment design. Students learn by building models that describe the motion of objects, and their learning is supported by scaffolded tasks and embedded formative assessments that introduce them to physics and CT concepts. We have also developed preparation for future learning (PFL) assessments to study students’ abilities to generalize and apply CT and science concepts and practices across problem solving tasks and domains. We use mixed quantitative and qualitative analysis methods to analyze student learning during a semester-long study run in a high school physics classroom. We document some of the lessons learned from this study and discuss directions for future work.

Predicting student understanding by modeling interactive exploration of evidence during an online science investigation

ABSTRACT This study mined student interactions with visual representations as a means to automate assessment of learning in a complex, inquiry-based learning environment. Log trace data of 143 middle school students’ interactions with an interactive map in Research Quest (an inquiry-based, online learning environment) were analyzed. Students used the interactive map to make scientific observations for an evidence-based hypothesis. The examination of classification error using an artificial neural network, compared against the majority class for prediction, suggests that student performance on several metrics of critical thinking can be classified based on different patterns in interactions with visual representations. Two alternative methods are compared in this study for training and evaluating data-mined models of student performance. In accordance with the general consensus in the literature, the error estimates for models’ predictions were less variable using a student-level cross-validation. Implications of these findings for open-ended inquiry-based learning environments are discussed.

An online support vector machine for the open-ended environment

Support Vector Machine (SVM) has been widely applied in expert systems. Traditional SVM works as a closed system, which means the parameters of the SVM freeze after training is finished. This strategy leads to the SVM cannot adapt to the online learning environment and also hinders the application of the SVM in expert systems which need to be updated in an online way. Many online SVMs retrain SVM with the current support vectors and new incoming data. However, the retained system usually shows a declined performance due to losing much information about the data. To handle this problem, we propose an Online Support Vector Machine (OSVM). The OSVM includes a Representative Prototype Area (RPA), which retains the representative data of all historical data. In the RPA, each class is maintained by an Online Incremental Feature Map (OIM) which learns a suitable representative set from the stream data automatically, e.g., we do not need to predefine a compression ratio, the OIM will dynamically adapt to the current input data. Retraining is periodically conducted on the updated OIM, which can avoid a declined performance and significantly reduce the computational cost. Variety data sets from different fields are used to test our method and the experimental results demonstrate that the accuracy of OSVM is comparable to that of the state-of-the-art SVM and the speed of OSVM is much faster. Moreover, OSVM is more stable than the other methods in the open-ended learning environment, where stream data from novel classes occur. This implies the OSVM has a good application prospect in online learning expert systems.

Do Students' Learning Behaviors Differ when they Collaborate in Open-Ended Learning Environments?

Researchers have long recognized the importance of using technology to support students' collaboration in learning and problem solving tasks. Recently, there has been a lot of research in capturing and characterizing student discourse and how they regulate each other when they perform learning tasks in pairs or in small groups. In this paper, our goal is to dive a little deeper into how students collaborate, and the learning behaviors they exhibit when working in pairs on a learning by modeling task, while also teaching a virtual agent in the Betty's Brain system. We report the results of a quasi-experimental study, where students were divided into two groups: one group worked in pairs and the other group worked individually. The results illustrate that students in the collaborative group built more correct causal maps than those working individually, and their pre-post test results show significantly higher learning gains in the science content. A differential sequence mining algorithm applied to their action sequences captured in log files showed differences in the learning behaviors between the two groups. The differences imply that the collaborative groups were better at debugging their evolving causal maps than the students who worked individually.

Note-taking and science inquiry in an open-ended learning environment

Note-taking is important for academic success and has been thoroughly studied in traditional classroom contexts. Recent advancements of technology have led to more students taking notes on computers, and in different situations than are common in traditional instructional contexts. However, research on computer-based note-taking is still an emerging area, and findings from these studies are mixed. In this exploratory study, we conducted multilevel analysis to comprehensively investigate the relationship between note-taking measures and subsequent student success at science inquiry among middle school students, using two scenarios of an open-ended learning environment named Virtual Performance Assessments. Analysis revealed an advantage for content elaborative note-taking over content reproductive note-taking conditional on the source of notes taken, but other measures were less consistent between the two scenarios. Implications of the findings and limitations of this research are also discussed.

INTRODUCING STUDIO ORIENTED LEARNING ENVIRONMENT (SOLE) IN UPM SERDANG: ACCESSING STUDENT-CENTERED LEARNING (SCL) IN THE ARCHITECTURAL STUDIO

This article reports the initial results of the exploratory research related to student-centered learning (SCL) in final year Architecture studio education in Universiti Putra Malaysia (UPM). SCL is defined as an approach to empower students in their own learning. Although studies on the adaptability of this concept in education has increased, there are few studies conducted for the benefits of architectural education from studio design perspective. In this article, we define SCL as an approach to increase student autonomy in learning curve especially in making decisions related to design subjects using a summative and formative approach. The objective of this article is to experiment the SOLE model and how it can move forward. The SOLE (Studio Oriented Learning Environment) model encompasses lecturing, sharing and peer review that is inspired by self-regulated theory. However, several problems and difficulties were identified namely, a lack of tutor input and problems in discussion dynamic in addition to resistance to peer assessment. This article suggests a number of improvements for future recommendations. The study will benefit educators in the architectural field in contributing to helping students to build on unique background knowledge and at the same time let the students generate learning opportunities and reconstruct knowledge dynamically in an open-ended learning environment to implement SCL in the studio module.