The purpose of this paper is to investigate linguistic factors affecting the evaluation of the argumentative essays in written tests taken by junior and senior students, aged 16 to 18, attending high schools in Greece. To achieve this, we analyzed textual characteristics and scoring of 265 juniors and seniors, graded by 15 different raters. To examine the contribution of linguistic parameters to the assessment, we developed an automated tool to record and evaluate students' lexical and syntactic features in the Greek language. The results revealed that the extensive use of nominal groups including an adjective and a noun and the utilization of both impersonal and passive syntax, as well as adverbs to a lesser extent, contribute the most to positive grading in language tests. Furthermore, we identified a correlation between language and the other criteria of the evaluation rubric, namely content and organization. The paper contributes to the discussion about objectivity in writing evaluation in the Greek setting and to the creation of a rubric that ensures a more effective assessment of writing tasks.