Evaluating Semantic and Personalized Information Retrieval with Deep Learning Models: A Performance Metrics Analysis

Abstract

Listen

Translate article

Evaluation of the Information Retrieval (IR) model is an important, complex process that involves evaluating many aspects of performance using quantitative measurements called evaluation metrics. This work explores the efficacy of Deep learning (DL) models in personalized and semantic information retrieval (SIR), an area that has witnessed significant advancements in the last few years. A thorough analysis of several DL models for search engine index retrieval is provided. The particular task, relevant standards, and desirable qualities, among others, influence the use of evaluation metrics in IR. A comprehensive evaluation of the performance of an IR system uses several metrics. The paper emphasizes how DL models can produce IR results that are more accurate, with an emphasis on choosing important metrics for evaluation. Among these metrics, on a given input query, a recall of 1.0 indicates full coverage in detecting pertinent cases, while a precision of 0.5 indicates 50% accuracy in positive predictions on a given input query. At 0.6667, the F1-score strikes a compromise between recall and precision. The model's ability to rank and prioritize is assessed by Mean Average Precision (MAP) at 0.5426 and Normalized Discounted Cumulative Gain (NDCG) at 0.8896. Precision in the top 5 suggestions is evaluated by Precision@5, which has a value of 0.6. This is important in situations where user attention or recommendation space are restricted. These measurements offer priceless information on the efficacy of IR models.