Multi-Criteria Evaluation and Benchmarking for Young Learners’ English Language Mobile Applications in Terms of LSRW Skills

N K Ibrahim,M A Alsalem,Shahad Nidhal,E M Almahdi,Ali Najm Jasim,M J Baqer,R T Mohammed,B B Zaidan,N S Jalood,Musaab Alaa,O S Albahri,Hamsa Hammed,Ali H Shareef,A A Zaidan

doi:10.1109/access.2019.2941640

Abstract

This study proposes an evaluation and benchmarking decision matrix (DM) on the basis of multi-criteria decision making (MCDM) for young learners’ English mobile applications (E-apps) in terms of listening, speaking, reading and writing (LSRW) skills. Benchmarking E-apps for young learners is challenging due to (a) multiple criteria, (b) criteria importance and (c) data variation. The DM was constructed on the basis of the intersection amongst evaluation criteria in terms of LSRW and E-apps for young learners. The criteria were adopted from a preschool education curriculum standard. The DM data included six E-apps as alternatives and 17 skills as criteria. Thereafter, the six E-apps were evaluated by distributing a checklist form amongst six English learning experts. These apps were subsequently benchmarked by utilising MCDM methods, namely, best–worst method (BWM) and technique for order of preference by similarity to ideal solution (TOPSIS). BWM was used for criterion weighting, whereas TOPSIS was employed to benchmark and rank the apps. TOPSIS was utilised in two contexts, namely, individual and group. In the group context, internal and external aggregations are applied. Mean was computed to ensure that the E-apps undergo a systematic ranking for objective validation. This study provides scenarios and a benchmarking checklist to evaluate and compare the proposed work with six relative studies. Results indicated that (1) BWM is suitable for criteria weighting. (2) TOPSIS is suitable for benchmarking and ranking E-apps. Moreover, the internal and external TOPSIS group decision making exhibited similar findings, with the best app being ‘Montessori’ and the worst app being ‘FunWithFlupe.’ (3) For objective validation, remarkable differences were observed amongst the group scores, which indicate that the internal and external ranking results are identical. (4) In the evaluation, the proposed DM revealed advantages over the six relative studies by 40.00%, 53.33%, 40.00%, 46.67%, 46.67% and 46.67%.

Highlights

The world is considered a global village where individuals communicate with one another through a popularThe associate editor coordinating the review of this manuscript and approving it for publication was Xiao-Sheng Si .language, which is known as English, a common language in the international community [1]
This section presents the results of the proposed decision matrix (DM) for evaluating and benchmarking the English learning apps in terms of LSRW skills based on the 2016 KSPK standard
Three experts are asked for their preferences on the evaluation criteria of English learning apps via a best–worst method (BWM) comparison question form

Summary

Introduction

The world is considered a global village where individuals communicate with one another through a popularThe associate editor coordinating the review of this manuscript and approving it for publication was Xiao-Sheng Si .language, which is known as English, a common language in the international community [1]. Many criteria used in this research for evaluation the English learning mobile apps for childhood. These evaluation criteria were divided into three main groups, namely, (1) Listening and Speaking, (2) Reading, and (3) Writing; The Listening and Speaking group includes nine metrics of criteria, namely: (stimulus given, rimes, poems and rhymes, stories, favourite things and activities, oral texts, familiar activities and experiences, stories heard, and daily situations). The following figure 9 illustrates the levels: Section 2: comparison questions Comparison measurement scale The comparisons (relative importance) of each criterion are measured according to a numerical scale from 1 to 9 These relative scales (1 to 9), as shown, Please use this scale in comparison These relative scales (1 to 9), as shown in Table 12, Please use this scale in comparison

Methods

Results

Conclusion