Students' ability in mathematics mainly relies on their performance in the assessment task such as tests, quizzes, assignments and final examinations. However, the grading process depends on the respective mathematics teacher who sets a marking scheme in assessing students' learning. How do these teachers assign grades to their students' problem solving work? What does it mean by five marks or ten marks for a mathematics problem? How does a teacher evaluate a student's mathematical knowledge and skills based on the grades? These questions address the vagueness of the grading process that gives no concrete evidence about a student's mathematical thinking. Hence, this paper aims to discover the effectiveness of using a marking scheme rubric to assess students' mathematical knowledge. The paper begins by reviewing different types of scoring rubrics in assessing mathematical problem solving tasks. A marking scheme rubric was proposed to assess samples of actual students' problem solving work in an applied algebra test. The rubric serves as an assessment instrument to gather information about students' achievement level in demonstrating both knowledge and skills in the test. Based on the findings, the score reflected the quality of the students’ work rather than just a numerical representation. It showed the students’ comprehension of adapting the mathematical concepts and problem solving strategies. In a nutshell, the implementation of rubric marking scheme has improved the consistency in grading and made the scoring points as a meaningful figure that describes the quality of a students' performance.