Auditing Source Diversity Bias in Video Search Results Using Virtual Agents

Mykola Makhortykh,Roberto Ulloa,Aleksandra Urman

doi:10.1145/3442442.3452306

Mykola Makhortykh, Roberto Ulloa + Show 1 more

Open Access

https://doi.org/10.1145/3442442.3452306

Copy DOI

Abstract

We audit the presence of domain-level source diversity bias in video search results. Using a virtual agent-based approach, we compare outputs of four Western and one non-Western search engines for English and Russian queries. Our findings highlight that source diversity varies substantially depending on the language with English queries returning more diverse outputs. We also find disproportionately high presence of a single platform, YouTube, in top search outputs for all Western search engines except Google. At the same time, we observe that Youtube's major competitors such as Vimeo or Dailymotion do not appear in the sampled Google's video search results. This finding suggests that Google might be downgrading the results from the main competitors of Google-owned Youtube and highlights the necessity for further studies focusing on the presence of own-content bias in Google's search results.

Highlights

The public tends to put high trust in the information retrieved via search engines [1]
One way of uncovering bias in web search outputs is to engage in algorithmic auditing [24]
We investigate the presence of source diversity bias - that is a systematic prevalence of specific information sources [11] - in video search outputs

Summary

Introduction

The public tends to put high trust in the information retrieved via search engines [1]. Search engine outputs are prone to different forms of bias [11, 21] which can result in distorted representation of subjects which users are searching for [12, 25]. One way of uncovering bias in web search outputs is to engage in algorithmic auditing [24]. Though a few studies have audited bias in text and image search (e.g., [21, 22, 26]), to the best of our knowledge no audits of video search results were conducted. The fact that video information can have powerful influence

Objectives

Methods

Results

Conclusion