Re-rank Search Engine Results via Machine Learning

Abstract

Research on search engines has demonstrated their significant impact on information search and user behavior and highlighted the need for continued investigation of their effectiveness, efficiency, and ethical implications. Moreover, search engine optimization (SEO) has emerged as a crucial aspect of online content creation and marketing, as it involves optimizing websites and online materials to improve their visibility and ranking on search engine results pages, thereby increasing their reach and impact. SEO metrics provide valuable insight into a website’s search engine performance. However, the relative importance of these metrics remains unclear as search engine algorithms and keyword competitiveness can vary widely. Therefore, ongoing research is needed to better understand the relationship between SEO metrics and search engine rankings. Given the challenge of determining the relative importance of SEO metrics using traditional statistical methods, interested in applying machine learning techniques. By training models with custom datasets of three search engine results (Google, Bing, and DuckDuckGo) and their associated SEO metrics, we hope to gain a better understanding of the complex relationships between these variables. The first two pages of each search engine return the most relevant results. In this article, we propose two types of methods to do this. First, classify the page index of web search results. Second, learning ranking methods to rank them. LambdaRank has the best NDCG compared to other methods for Google and DuckDuckGo which are 0.86 and 0.93 respectively. RankNet is the best learning-to-rank method for Bing with 0.85 as the NDCG value. Logistic regression has the highest accuracy compared to other methods in page index classification for Bing and Google, which are 61.29% and 64.71%, respectively. K-nearest neighbor performs best for DuckDuckGo with an accuracy of 66.67%.