Abstract
BackgroundThe 2010 reform of the Italian university system introduced the National Scientific Habilitation (ASN) as a requirement for applying to permanent professor positions. Since the CVs of the 59,149 candidates and the results of their assessments have been made publicly available, the ASN constitutes an opportunity to perform analyses about a nation-wide evaluation process.ObjectiveThe main goals of this paper are: (i) predicting the ASN results using the information contained in the candidates’ CVs; (ii) identifying a small set of quantitative indicators that can be used to perform accurate predictions.ApproachSemantic technologies are used to extract, systematize and enrich the information contained in the applicants’ CVs, and machine learning methods are used to predict the ASN results and to identify a subset of relevant predictors.ResultsFor predicting the success in the role of associate professor, our best models using all and the top 15 predictors make accurate predictions (F-measure values higher than 0.6) in 88% and 88.6% of the cases, respectively. Similar results have been achieved for the role of full professor.EvaluationThe proposed approach outperforms the other models developed to predict the results of researchers’ evaluation procedures.ConclusionsSuch results allow the development of an automated system for supporting both candidates and committees in the future ASN sessions and other scholars’ evaluation procedures.
Highlights
Quantitative indicators have been extensively used for evaluating scientific performances of a given research body
We focused on the 2012 session of the attained the National Scientific Habilitation (ASN) because: (i) it is a representative sample of the whole population asking for habilitation; (ii) since in 2016 different people were appointed in the committees, in this way we exclude biases and other problems introduced by changes in the evaluation committees
The aim of the analyses presented is to answer the two Research Questions (RQs) discussed in ‘‘Introduction’’
Summary
Quantitative indicators have been extensively used for evaluating scientific performances of a given research body. Predicting the results of evaluation procedures of academics. Since the CVs of the 59,149 candidates and the results of their assessments have been made publicly available, the ASN constitutes an opportunity to perform analyses about a nation-wide evaluation process. The main goals of this paper are: (i) predicting the ASN results using the information contained in the candidates’ CVs; (ii) identifying a small set of quantitative indicators that can be used to perform accurate predictions. The proposed approach outperforms the other models developed to predict the results of researchers’ evaluation procedures. Such results allow the development of an automated system for supporting both candidates and committees in the future ASN sessions and other scholars’ evaluation procedures
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have