Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity

Abstract

The Digital Competency Profiler (DCP) is an online application for surveying the technology preferences and abilities of students in higher education. To explore the DCP as a digital-learning-readiness tool, a mixed-methods research design was developed for relating self-reported digital competencies and online-learning activity. To this end, three authentic scenarios, comprised of six tasks mapped to self-report items, were constructed. Having submitted their survey data, each of 15 participants visited the EILAB to complete a randomly-assigned scenario with a tablet. Both the performance activity and post-activity interviews were recorded digitally using a unique activity-station setup, and task artefacts were gathered as performance outcomes. Analysis was conducted in three phases. In Phase 1, both the audio-video performance data and activity artefacts were coded, assessed and scored. Exploratory correlational analyses showed a pattern of positive relationships at the task and scenario levels for two scenario groups, suggesting some predictive value for the DCP in this context. For the third group, a positive correlation was found at the scenario level, but negative correlations were found at the task level. In Phase 2, detailed case-studies were conducted, incorporating self-report data, coded performance timelines, and post-activity interviews. Several situational influencers related to problem-solving strategy, device comfort, task difficulty and motivation, beyond the purview of the DCP, were identified. In Phase 3, the findings were interpreted to position the DCP as a tool for identifying segments of students with members who, without support, will likely struggle to engage fully in technology-rich learning environments.