Search strategy effectiveness and relevance of YouTube videos

Abstract

AbstractPrecision and recall are two widely accepted measures to Information Retrieval (IR) system performance. Different search strategies can yield different precisions and recalls for the same IR system, search engine or website. Therefore, precision and recall can also be used to measure search strategy effectiveness. YouTube® is a popular social media website which hosts the largest number of user‐generated videos on the web. The YouTube® search engine only provides text words search function and search results are retrieved through the system matching these search terms to video descriptions, tags, comments, etc. Therefore, precision and recall can also be used to measure search strategy effectiveness. This paper aims to test if a search strategy using multiple search terms is effective by analyzing the relevance of the retrieved YouTube® videos on Smokeless Tobacco Products (STP) and YouTube® provided video usage statistics and community engagement statistics.17 search terms were used to retrieve YouTube® videos on STP which were recorded in a master file. All duplicate videos from the retrieved results were excluded. Unique videos were then selected based on pre‐defined inclusion and exclusion criteria set by the research team. A sample of 440 unique videos was randomly selected for data analysis. SPSS 19.0 was used for data analysis. Descriptive analysis, precisions and recalls, logistic regression, and odds ratio test results are reported. The study found that only a few search terms out of the total 17 terms are more effective in terms of relevance of the retrieved videos. Also, YouTube® provided video usage statistics and community engagement statistics did not have significant association with the relevance of the retrieved videos. Recommendations for potential solutions on designing more effective search strategies are provided.