Abstract
In this article we report the initial results of experiments using HMEANT metric (semi-automatic evaluation metric used for scoring translation quality by matching semantic role fillers) on the Polish language. The metric is evaluated in the task of Machine Translation (MT) and in re-speaking quality assessment. GUI-based annotation interface was developed and with this tool (https://github.com/krzwolk/HMEANT-metric-for-Polish) evaluation was conducted practically by not IT-related personnel. Reliability, correlation with automatic metrics, language independence and time costs were analysed as well. Role labelling and alignment using GUI interface were done by two annotators with no related background (they were only instructed for about 10 min). The results of our experiments showed high inter-annotator agreement as far as role labelling was concerned and a good correlation of the HMEANT metric with human judgements based on re-speaking evaluation.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
