Constructing the Evaluation Index System of Chinese-Portuguese Machine Translation using the Delphi and Analytic Hierarchy Process Methods

Abstract

Quality assessment is an important part of ensuring the quality and long-term development of machine translation. There is still much room for research into the evaluation of machine translation from Chinese to Portuguese, and no specific evaluation metrics have yet been developed. This study uses the Delphi method and Analytic Hierarchy Process (AHP) to construct an index system of Chinese-Portuguese Machine Translation and analyzes the relative index weight. The system includes four first-level indicators and nine second-level indicators. The weight of primary indicators, from high to low, is "Accuracy, fidelity" (52.5%), "Readability" (22.9%), "Fluency" (16.8%), and "Acceptability & Usability" (7.8%). The secondary indicators with the highest weight of each primary indicator are "Correction", "Lexicon", "Readability", and "Utility". AHP was used to transform the qualitative problems in evaluating Chinese-Portuguese Machine Translation into the quantitative problem in order to provide a theoretical basis and practical guidance for the future manual evaluation of Chinese-Portuguese Machine Translation, and for the eventual modification of automatic machine translation evaluation methods.