Mobile-based Assessment Research Articles

A Novel Mobile-based Assessment Application for Evaluating Students' Creative Artworks in Higher Education

A Novel Mobile-based Assessment Application for Evaluating Students' Creative Artworks in Higher Education

A comprehensive model explaining teachers’ intentions to use mobile-based assessment

ABSTRACT This study was conducted to understand pre-service and in-service teachers’ intentions to use mobile-based assessment in classes by integrating theory of planned behaviour, technology acceptance model, and self-determination theory into one conceptual framework. Data were analysed using the structural equation model (SEM). Results of SEM from a sample of 522 pre-service teachers and 279 in-service teachers revealed that, compared to existing theories, the proposed model has a better acceptable level of explanatory power for intention to use mobile-based assessment. Results also showed that all the hypothesis relationship among constructs within the model was supported. In addition, the study identified the prominent mediating role of perceived usefulness, perceived ease of use, and attitude and moderating impact of gender on intention to use mobile-based assessment. All in all, the study provides useful implications for policymakers, teacher educators, and school administrators as well as suggestions for future researchers.

Non-English Department Students' Perspectives on the Use of Mobile Technology in English Language Learning Assessment

The use of technology is a common thing in the 21st century learning era. Educational practitioners also follow the rapid development of technology. Especially in learning English which is considered difficult to learn and master, there is one component that has a psychological effect that has the potential to suppress the optimality of student achievement. This is an assessment process carried out on the foreign language. Referring to the view that English is an important language, an idea emerged that assessment in English could be developed to be more attractive by integrating mobile technology. Therefore, this study aims to determine non-English department students' views toward mobile-based assessment in English courses. This research is descriptive qualitative research with categorization as the analysis process. By collecting, reducing and describing data, this study aims to reveal a general view of the implementation of mobile-based assessment in higher education. In general, students feel various improvements in terms of the learning process including excitement in reading and writing assessment. The use of Schoology as the mobile technology supporting learning and assessment seems to be useful as the result of this research

Students’ Attitude Towards Ruanguji Mobile-Based Assessment: An Explanatory Case Study in Tangerang, Indonesia

This qualitative study aims to unveil the junior high school students’ attitude towards Ruanguji mobile-based assessment. It uses an explanatory case study design engaging 225 students in Tangerang, Indonesia as the respondents. To analyze the data got by the questionnaire, the study uses the multi-component model of attitude and one-dimensional perspective. The study reveals that their attitude towards Ruanguji comprises the cognitive, affective, and behavioral components with different degree. For the students, Ruanguji is a supplementary media offering various tests and assignments to help them face the national examination. With the application, they feel convenient to prepare themselves for the examination. If they have chances, they will use it to explore testing materials and assignments. Their attitude is identified as a positive one that impedes the existence of the negative one. Its positivity degree is determined by internal factors, like their perception and emotion; and external factors, like the application’s user-friendliness. In addition, their positive attitude towards the application also contributes to their success in their school’s summative tests and national examination. The result of the study leads to the conclusion that a positive attitude towards the mobile assessment can be a determinant factor of their success in the study.

Transformation of Arabic Assessment in Indonesia: Conventional Assessment Toward Digital Assessment

Since 1960, digital assessment has been developed and implemented in foreign language assessments, but there are still few studies that discuss Arabic digital assessment because most of them follow a traditional paper and pencil assessment. The study investigates the transformation of Arabic assessment in Indonesia from conventional to digital. It further describes the factors influencing the transformation. This study employs a qualitative-case study involving Indonesian Islamic State Universities. Data collection techniques used are interviews and observations. Then the data is analyzed interactively with data reduction, data display and conclusion. The results show that the transformation of the Arabic language assessment has begun in 2016 at UIN Maliki Malang but it faced constraints regarding its TOAFL implementation. Implementing the digital Arabic assessment is comprehensively done during the online learning policy. Two factors influence the shift in Arabic assessment culture: the rapid development of the 4.0 Industrial Revolution and the COVID-19 pandemic. There are two models of digital assessment: web-based assessment and mobile-based assessment

CC Thinker: Mobile-based Assessment to Train Creative and Critical Thinking in Students

This research aims to combine assessment of creative and critical thinking in a mobile device context, to see its effect on the creative and critical thinking of students. Design-based research as the method was used to develop the assessment as learning. It is begun with the assessment model and android application development, also experiments in the end. Three assessment sessions are participated by 46 students. The analysis using Kruskal Wallis gains the result that there are significant differences in the creative thinking (Chi-square 18.245, Asymp. Sig 0.000) and the critical thinking (Chi-square 7.620. Asymp. Sig 0.022). A significant difference in creative thinking is seen since the second session, while the difference in critical thinking needs the time up to the third session.

Technology integration in learning management: A post-pandemic phenomenological study in elementary schools

The post-pandemic period in the education sector is marked by government policies to carry out face-to-face education processes in schools. This situation has the potential to show various changes in learning management in elementary schools, especially with the integration of technology. This study aims to analyse the integration of technology in learning management in elementary schools during the post-pandemic period. This study uses a qualitative approach to the type of phenomenology. The research subjects were elementary school teachers who had been selected through a convenience technique based on the availability of subjects in the field. Data were collected through interviews and documentation. Data were analysed using qualitative analysis through data condensation, data presentation, verification and drawing conclusions. The study’s findings show that the integration of technology in learning management tends to decrease during the post-pandemic period. Some habits of using technology in learning during the pandemic are no longer applied during the pandemic. However, some uses of technology are still maintained, such as the use of the WhatsApp group application in the delivery of learning information and the use of mobile-based assessment applications in learning assessment. 
 
 Keywords: Technology integration, learning management, post-pandemic.

Taking the pulse of a classroom with a gamified audience response system

Background and objectiveThis paper presents an empirical study of a gamified mobile-based assessment approach that can be used to engage students and improve their educational performance. MethodA gamified audience response system called G-SIDRA was employed. Three gamification elements were used to motivate students in classroom activities: badges for achievements to increase engagement, points to indicate progression and performance in the subject and ranking for promoting competitiveness. A total of 90 medical students in a General and Descriptive Anatomy of the Locomotor System course were taught using G-SIDRA in the academic year 2019/2020. Smart bracelets were configured to collect heart rate measurements from 30 students with the aim of evaluating the impact of the gamification elements. The control group consisted of a sample of 110 students enrolled on the same course in the academic year 2016/2017 using non-gamified SIDRA. ResultsStatistically significant differences were found between multiple choice questions (MCQ) scores obtained by using SIDRA and G-SIDRA in the four experiments (U = 1.621,50, p < 0,01 for Exp1; U = 1.950,00, p < 0,01 for Exp2; U = 955,00, p < 0,01 for Exp3; U = 2.335,00, p < 0,01 for Exp4). In the students’ final exam grades, statistically significant differences between students that used G-SIDRA as opposed to SIDRA (T(157) = 3.992; p = 0.044) were obtained. Concerning gamification elements, statistically significantly differences were found in comparing the pulse increases after and before the badge event in the four experiments (U = 2.484,00, p = 0,038 for Exp1; U = 2.109,50, p = 0,046 for Exp2; U = 1.790,50, p = 0,025 for Exp3; U = 1.557,0, p = 0,048 for Exp4). However, there are not statistically significant differences between the pulse increases after and before the ranking event in the four experiments. In a 5-point Likert-type scale, the students expressed satisfaction with G-SIDRA (M = 4.552) and thought the system helped to better understand both theoretical and practical concepts (M = 4.092). Their global assessment of the G-SIDRA platform was 4.471. ConclusionsOf the three gamification elements used in the study, only badge has an effect on heart rate. Better student responses and academic performance were achieved when using G-SIDRA. Nevertheless, more research is required to evaluate the impact of the gamification elements on the motivation, engagement and performance of students. Physiological measures are promising approaches for gamification elements evaluation.

AutoQual: task-oriented structural vibration sensing quality assessment leveraging co-located mobile sensing context

In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect sensing approach’s learning performance is prone to deployment variance when signals propagate through the environment. As a result, current systems heavily rely on expert knowledge and manual assessment to achieve effective deployments and high sensing task performance. In order to mitigate this expert effort, we propose to systematically study factors that reflect deployment environment characteristics and methods to measure them autonomously. We present AutoQual that measures a series of assessment factors (AFs) reflecting how the deployment environment impacts the system performance. AutoQual outputs a task-oriented sensing quality (TSQ) score by integrating measured AFs trained from known deployments as a prediction of untested system’s performance. In addition, AutoQual achieves this assessment without manual effort by leveraging co-located mobile sensing context to extract structural vibration signal for processing automatically. We evaluate AutoQual by using it to predict untested systems’ performance over multiple sensing tasks. We conduct real-world experiments and investigate 48 deployments in 11 environments. AutoQual achieves less than 0.10 average absolute error when auto-assessing multiple tasks at untested deployments, which shows a le 0.018 absolute error difference compared to the manual assessment approach.

A Systematic Review of Mobile-Based Assessment Acceptance Studies from 2009 To 2019

Despite many studies being conducted on mobile learning acceptance, few investigate mobile-based assessment acceptance. The objectives of this research are to provide valuable insights into current research on mobile-based assessment literature, and to identify the main gaps in the mobile-based assessment acceptance literature. Therefore, the present study systematically reviews 48 previous studies and eight articles related to mobile-based assessment acceptance to provide a comprehensive analysis of the articles published from 2009 to 2019. Findings indicate that majority of mobile-based assessment studies focused on evaluating the effectiveness and performance of mobile-based assessment system and conducted at the secondary school level. In addition, this study identified several gaps. Further research is needed to study the acceptance problem of a mobile-based assessment system. More investigation is required to predict which external factors that can enhance the acceptance and use of mobile-based assessment among students. The findings of this review study provide a valuable reference for researchers about the current trend of mobile-based assessment research as well as the research gaps that should be covered in future studies.

Students’ acceptance of mobile-based assessment

The effective development of a Mobile-Based Assessment (MBA) depends on students’ acceptance. The aim of this paper was to examine the determinant factor of students’ behavioral intention to use mobile-based assessment. Data were collected from 105 second grade students of a vocational high school through an online survey questionnaire. Partial Least Squares (PLS) was used to test the measurement and the structural model. Results showed Perceived Ease of Use as the strongest direct predictor of Behavioral Intention to Use, followed by Perceived Usefulness. Content and Mobile Self-Efficacy only has an indirect effect. These four variables explain 51.3 percent of the variance of Behavioral Intention to Use.

USING MOBILE-BASED FORMATIVE ASSESSMENT IN ESL/EFL SPEAKING

With the widespread application of smartphones in and outside the classroom, mobile-based teaching and learning is drawing much attention and hence being extensively practised nowadays across the globe. Recently, using smartphones for assessment purposes has been a new phenomenon and the researchers are still examining what processes the use of mobile-based assessment tools may include and what outcomes and challenges they can cause to teachers and students in terms of learning/teaching performance, motivation and attitudes. There have been a good number of research studies on the use of Mobile Assisted Language Learning (MALL) or Mobile Learning (ML) in EFL or ESL classroom but not much literature is known about the mobile-based language assessment, especially mobile-based formative assessment (MBFA). Hence, this study attempts to shed light on MBFA and review the recent literature available on it and its effective utilization in developing ESL/EFL speaking skill. This paper uses a qualitative research method that exclusively uses the relevant secondary references/works available on the topic. The literature revealed that MBFA practices in ESL/EFL speaking classes are effective to a certain extent and some tools and procedures seem to be more effective than others depending on the design principles and strategies used by teachers or app developers.

Leveraging Walking Performance to Understand Work Fatigue Among Young Adults: Mixed-Methods Study.

BackgroundWork fatigue negatively impacts personal health in the long term. Prior research has indicated the possibility of leveraging both walking parameters and perceptual measures to assess a person’s fatigue status. However, an effective and ubiquitous approach to assessing work fatigue in young adults remains unexplored.ObjectiveThe goals of this paper were to (1) explore how walking rhythms and multiple streams of data, including reaction time, self-reports, and an activity diary, reflect work-induced fatigue in the lab setting; (2) identify the relationship between objective performance and subjective perception in indicating fatigue status and fatigability; and (3) propose a mobile-based assessment for work-induced fatigue that uses multiple measurements.MethodsWe conducted a 2-day in-lab study to measure participants’ fatigue status using multiple measurements, including the stair climb test (SCT), the 6-minute walk test (6MWT), and the reaction time test. Both the SCT and the 6MWT were conducted at different points in time and under 2 conditions (measurement time, including prior to and after work, and pace, including normal and fast). Participants reported their fatigue perception through questionnaires completed before conducting walking tests and in an activity diary recorded over a week. Walking performance data were collected by a smartphone with a built-in 3-axis accelerometer. To examine the effect of fatigability on walking performance, we first clustered participants into 2 groups based on their reported mental fatigue level in the entry surveys and then compared their walking performance using a generalized linear model (GLM). The reaction time was examined using a 2-way repeated-measures GLM. We conducted semistructured interviews to understand participants’ fatigue perception after each day’s walking tests.ResultsAll participants (N=26; mean age 24.68 years) were divided into 2 groups—the fatigue-sensitive group (11/26, 42%) and the fatigue-nonsensitive group (15/26, 58%)—based on their mental subscores from 3 entry surveys: Fatigue Scale-14, Three-Dimensional Work Fatigue Inventory, and Fatigue Self-Assessment Scale (FSAS). The fatigue-sensitive group reported a significantly higher FSAS score in the before-work setting (t50=–3.361; P=.001). The fatigue-sensitive group covered fewer steps than the fatigue-nonsensitive group (β1=–0.099; SE 0.019; t1=–5.323; P<.001) and had a higher step-to-step time variability in the 6MWT (β1=9.61 × 10–4; t1=2.329; P=.02). No strong correlation between subjective and objective measurements was observed in the study.ConclusionsWalking parameters, including step counts and step-to-step time variability, and some selected scales (eg, FSAS) were found to reflect participants’ work-induced fatigue. Overall, our work suggests the opportunity of employing mobile-based walking measurements to indicate work fatigue among young adults.

The Effect of Electronic Educational Assessment Environment (Navigation and Content) on Students' Intention to use MOBILE BASED ASSESSMENT from Motivational Perspective View

The Effect of Electronic Educational Assessment Environment (Navigation and Content) on Students' Intention to use MOBILE BASED ASSESSMENT from Motivational Perspective View

Mobile-based dynamic assessment and the development of EFL students' oral fluency

This paper reports on a study investigating the potential effect of the impact of Mobile-based Dynamic Assessment (MbDA) on improving English as a Foreign Language (EFL) learners' oral fluency. To fulfil the purpose of the study, 120 pre-intermediate learners of English were randomly assigned to one of three groups (two experimental groups and one control group), each consisting of 40 participants. The participants in the experimental groups completed eight communicative tasks via dynamic assessment under two different contexts: MbDA via text-chat context and MbDA via voice-chat context. The speech rate, average length of pauses, the number of repetitions and the number of reformulation served as dependent variables. The study found that the participants in the experimental groups who received MbDA improved their speaking fluency significantly compared with their peers in the control group.

A comparative study between a computer-based and a mobile-based assessment

PurposeThe purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course.Design/methodology/approachThe study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale.FindingsAttractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions.Practical implicationsThe usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers.Originality/valueThe study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.

Mobile-based assessment: A literature review of publications in major referred journals from 2009 to 2018

Mobile devices not only provide a medium for delivering personalized and context-aware learning but also facilitate the delivery of assessment activities anytime and anywhere. With the growing adoption of the Bring Your Own Device (BYOD) practices in education, Mobile-Based Assessment (MBA) is an emerging field in the context of mobile learning research. Although a considerable number of literature reviews exists about mobile learning, there is no such review study to provide insight into mobile-based assessment. The current study is a review of forty-three (43) articles about mobile-based assessment published in seven major educational technology research journals from January 2009 to February 2018. Major findings include that most mobile-based assessment studies focused on formative assessments with elementary students and in STEM subjects. Most of the reviewed articles reported a significant positive impact on student learning performance, motivation and attitudes. Moreover, the study identified several gaps in the mobile assessment literature. More research is needed to investigate issues and concerns related to negative perceptions against mobile assessment, especially form the teachers’ point of view. Also, a stronger alignment needs to be developed between student motivation and different mobile-based assessment practices. The study can be a valuable reference for educators and researchers working in the field of mobile-based assessment.

Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation

Mobile learning (M-learning) is a new wave in the era of educational technology that provides informal, personal, voluntary and situated learning opportunities for both learners and educators. Mobile-based assessment is one of the emerging technologies that attract many scholars to investigate its effectiveness due to the wide spectrum of its features like portability, interactivity, flexibility, and ubiquity. Based on the surveyed literature, we noticed that there is a lack of studying the students' attitudes towards the utilization of mobile technologies in the context of e-Evaluation. The present study attempts to investigate the students' attitudes towards the utilization of mobile technologies in the e-Evaluation system of instructors. The data was collected through a questionnaire survey from Al Buraimi University College (BUC) in Oman. The total number of participants is 354 students. Findings indicated that 99% of the students own a mobile phone or tablet. Moreover, results indicated a statistically significant difference among the students’ attitudes in terms of their gender where the differences were in favor of male students. In contrast, results revealed no statistically significant differences among the students’ attitudes in terms of their age, degree, and department. Furthermore, other implications and future work are also reported in the study.

Mobile-based assessment: Investigating the factors that influence behavioral intention to use

Acceptance and intention to use mobile learning is a topic of growing interest in the field of education. Although there is a considerable amount of studies investigating mobile learning acceptance, little research exists that investigates the driving factors that influence students' intention to use mobile technologies for assessment purposes. The aim of this study is to provide empirical evidence on the acceptance of Mobile-Based Assessment (MBA), the assessment delivered through mobile devices and technologies. The proposed model, Mobile-Based Assessment Acceptance Model (MBAAM) is based on the Technology Acceptance Model (TAM). MBAAM extends TAM in the context of MBA by adding to the Perceived Ease of Use and Perceived Usefulness, the constructs of Facilitating Conditions, Social Influence, Mobile Device Anxiety, Personal Innovativeness, Mobile-Self-Efficacy, Perceived Trust, Content, Cognitive Feedback, User Interface and Perceived Ubiquity Value and investigates their impact on the Behavioral Intention to Use MBA. 145 students from a European senior-level secondary school experienced a series of mobile-based assessments for a three-week period. Structured equation modeling was used to analyze quantitative survey data. According to the results, MBAAM explains and predicts approximately 47% of the variance of Behavioral Intention to Use Mobile-Based Assessment. The study provides a better understanding towards developing mobile-based assessments that support learners, enhance learning experience and promote learning, taking advantage of the distinguished features that mobile devices may offer. Implications are discussed within the wider context of mobile learning acceptance research.

Mobile-Based Assessment: Integrating acceptance and motivational factors into a combined model of Self-Determination Theory and Technology Acceptance

Mobile-Based Assessment (MBA) is an alternative or complementary to paper- or computer-based assessment delivery mode. Its successful implementation depends on users' acceptance. However, no study exists exploring the factors that influence students' acceptance of mobile-based assessment. Furthermore, research that combines acceptance with motivational factors is limited. The current study builds on the theoretical framework of the Self-Determination Theory (SDT) of Motivation and the Technology Acceptance Model (TAM) and proposes the Mobile Based Assessment - Motivational and Acceptance Model (MBA-MAM), a combined model that explains and predicts Behavioral Intention to Use Mobile-based Assessment. One-hundred and forty students (N = 140) from a European senior-level secondary school participated in mobile-assisted assessment activities and self-reported their perceptions about ΜΒΑ afterwards. Structured equation modeling used to analyze quantitative survey data. The study confirmed the proposed model, explaining and predicting students’ intention to use MBA in terms of both acceptance and motivational (autonomy, competence and relatedness) factors. The study provides a better understanding towards the development of mobile-based assessments by relating acceptance and motivational factors into an integrated model. Implications are discussed within the wider context of mobile learning acceptance research.