Performance Evaluation of LSA, NMF and ILSA in Electronic Assessment of Free Text Document

Abstract

Aims: To evaluate the performance of an Improved Latent Semantic Analysis (ILSA), Latent Semantic Analysis (LSA), Non-Negative Matrix Factorization (NMF) algorithms in an Electronic Assessment Application using metrics, Term Similarity, Precision, Recall and F-measure functions, Mean divergence, Assessment Accuracy and Adequacy in Semantic Representation.
 Methodology: The three algorithms were separately applied in developing an Electronic Assessment application. One hundred students’ responses to a test question in an introductory artificial intelligence course were used. Their performance was measured based on the following metrics, Term Similarity, Precision, Recall and F-measure functions, Mean divergence and Assessment Accuracy.
 Results: ILSA outperformed the LSA and NMF with an assessment accuracy of 96.64, mean divergence from manual score of 0.03, and recall, precision and f-measure value of 0.83, 0.85 and 0.87 respectively.
 Conclusion: The research observed the performance of an improved algorithm ILSA for electronic Assessment of free text document using Adequacy in Semantic Representation, Retrieval Quality and Assessment Accuracy as performance metrics. The results obtained from the experimental designs shows the adequacy of the improved algorithm in semantic representation, better retrieval quality and improved assessment accuracy.