Abstract
In this paper, we study how to assign weights to a set of evaluations obtained at the end of an international mobility experience in order to aggregate them into a composite indicator. The mobility experience was evaluated by three categories of actor: the participant; the school or company sending the participant; and the school or company hosting the participant. We estimated the weights starting from the assessors’ mutual evaluations of the beneficiaries of the mobility experiences. In particular, the aim of the paper was to compare two strategies for estimating the weights: (1) a weighted function of the univariate rank distribution of frequencies; and (2) the normalised elements of the first eigenvector of the dominance matrix computed by mediating the actors’ dominance matrices derived from the rankings of mobility beneficiaries. Variants of the two strategies were also introduced. Even though each strategy had different assumptions, the analyses produced several important findings. First, the optimum weighting model depends on the loss function used to evaluate the quality of the results. In particular a between-ranking variability function favours both univariate and unweighted multivariate models, while a bias-based function favours weighted multivariate models. Second, in both univariate and multivariate analyses, the application of rank-order-centroid and rank-reciprocal rules give more accurate results than both linear and exponential rules.
Highlights
ROI-MOB (Measuring return on investment from European Union (EU) VET mobility) is a European project that aims to represent the final outcomes of an Erasmus + mobility experience using a single, complex indicator (Fabbris and Boetti 2019)
We applied various weighting procedures that included at least three levels of information present in the collected rankings: (a) just the top recipient as stated by assessors; (b) the full distribution of frequencies derived from the rankings delivered by the assessors; and (c) the betweenalternative dominance relations that can be deduced from the assessed rankings
We showed that the more information we used in the scoring procedure, the more stable and accurate were the estimates
Summary
ROI-MOB (Measuring return on investment from EU VET mobility) is a European project that aims to represent the final outcomes of an Erasmus + mobility experience using a single, complex indicator (Fabbris and Boetti 2019). Education and Training (VET) experience, which consists of an internship undertaken by a student or apprentice at a workplace in another country. The surveys collected data from four groups of assessors: (i) the participants directly involved in VET international mobility, either residing or hosted in one of four countries (Germany, Italy, Portugal, and Spain); (ii) the schools and training centres facilitating the participation of students or apprentices, either by sending them abroad or hosting them; (iii) the companies and public bodies that sent or hosted the participants; and (iv) representatives of the national or European institutions promoting VET international mobility. For the sake of simplicity, only the data from the first three groups of assessors are processed in this paper
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
