Agent-based Testing Research Articles

Accessing the Impact of TikTok’s Algorithm on Regional Inequality in Health Information

ABSTRACT This study aims to audit the potential algorithmic bias in TikTok’s health-related video recommendation toward geographically diverse groups in China. We employed 120 cloud phones and conducted two agent-based testing experiments simulating users’ geographical locations and online behaviors. The results indicated significant regional inequality in video sources recommended by the TikTok algorithm, t(118) = 3.02, p = .003, with users from developed cities encountering a higher proportion of professional videos than those from underdeveloped cities. However, when users from both regions expressed a similar preference for the same type of information, an equal proportion of professional videos was recommended. Our findings suggest that widely used algorithms may covertly perpetuate social inequities and reinforce preexisting class-based inequalities, particularly affecting vulnerable population from low-income regions. This study also highlights the importance of enhancing eHealth literacy among disadvantaged users to mitigate problematic outcomes in the AI-based communication landscape.

It matters how you google it? Using agent-based testing to assess the impact of user choices in search queries and algorithmic personalization on political Google Search results

It matters how you google it? Using agent-based testing to assess the impact of user choices in search queries and algorithmic personalization on political Google Search results

New Digital Divide Shaped by Algorithm? Evidence from Agent-Based Testing on Douyin’s Health-Related Video Recommendation

While recommendation algorithms have significantly empowered human communication process, there is an emerging scholarly and societal concern regarding the potential discrimination inherent in algorithmic decision-making. The present study employs a novel agent-based testing approach to conduct an automated audit of the Douyin algorithm’s recommendations for health-related videos, aiming to investigate its relationship with information inequalities between individuals who are socioeconomically advantaged and disadvantaged. Our findings imply the possible existence of a new digital divide, wherein the algorithm tends to recommend a smaller proportion of authenticated health-related videos on cheap phone models, representing users in low socioeconomic group. Furthermore, this divide was mitigated when both groups demonstrated the same watching strategy for authenticated information. This study implies that recommendation algorithms actively perpetuate and exacerbate discriminatory social structures in reality. We also advocate for increased attention and education among the low socioeconomic group to enhance their health information literacy and understanding of algorithmic mechanisms.

Aligning agent-based testing (ABT) with the experimental research paradigm: a literature review and best practices

Aligning agent-based testing (ABT) with the experimental research paradigm: a literature review and best practices

Algorithms without frontiers? How language-based algorithmic information disparities for suicide crisis information sustain digital divides over time in 17 countries

ABSTRACT This study focuses on the changes in the global digital divide produced by language-based, algorithmic information disparities in relation to crisis-prevention resources for suicide available through the Google search engine. We used agent-based testing to emulate Google searches performed in 17 countries and in 16 different languages as a direct replication and extension of previous work. We compare data collected in 2017 with data collected in 2021. Our analyses revealed that Google searches in English from within the United States still have the highest likelihood of triggering the display of additional crisis-prevention information prominently shown in addition to the regular search results (i.e., Google’s suicide-prevention result). Searches in Spanish from within the United States are informationally disadvantaged. Display rates are only slightly lower in other English-speaking countries and when searches are performed in English. While information disparities and digital divides narrowed between 2017 and 2021, substantial differences in the display of crisis-prevention resources remain observable within multilingual countries, especially when other languages compete with English. In Bahrain, South Africa, and Sweden, the crisis-prevention information functionality seems unimplemented. Our findings suggest that the use of automated computational methods is both useful to continuously observe the implementation of new algorithmic functionalities and necessary to hold global media institutions accountable for their actions.

Curating Quality? How Twitter’s Timeline Algorithm Treats Different Types of News

This article explores how Twitter’s algorithmic timeline influences exposure to different types of external media. We use an agent-based testing method to compare chronological timelines and algorithmic timelines for a group of Twitter agents that emulated real-world archetypal users. We first find that algorithmic timelines exposed agents to external links at roughly half the rate of chronological timelines. Despite the reduced exposure, the proportional makeup of external links remained fairly stable in terms of source categories (major news brands, local news, new media, etc.). Notably, however, algorithmic timelines slightly increased the proportion of “junk news” websites in the external link exposures. While our descriptive evidence does not fully exonerate Twitter’s algorithm, it does characterize the algorithm as playing a fairly minor, supporting role in shifting media exposure for end users, especially considering upstream factors that create the algorithm’s input—factors such as human behavior, platform incentives, and content moderation. We conclude by contextualizing the algorithm within a complex system consisting of many factors that deserve future research attention.

How search engines may help reduce drug-related suicides

Background and aimsNumbers of drug-overdose deaths, both intentionally and unintentionally, have been increasing in the United States. Of interest, Google spotlights counselling services as helpful resources when users query for suicide-related search terms. However, the search engine does so at varying display rates, depending on terms used. Display rates in the drug-overdose deaths domain are unknown. MethodsWe emulated suicide-related potentially harmful searches at large scale across the U.S. to explore Google’s response to search queries including or excluding additional drug-related terms. Employing agent-based testing we conducted 215,999 search requests with varying combinations of search terms. ResultsCounseling services such as helpline telephone numbers were displayed at high rates after suicide-related potentially harmful search queries (e.g., “how to commit suicide”). While this is a desirable outcome, display rates were substantially lower when drug-related terms, indicative of users’ suicidal overdosing tendencies, were added (e.g., “how to commit suicide fentanyl”). Importantly, the addition of any drug-related search term to the suicide-related queries decreased the display frequency of helpful prevention-related resources substantially. ConclusionsSearch queries such as “easy way to commit suicide fentanyl” may indicate acute suicidal crises. Helpful resources should be displayed right in such search moments. Search engines should adjust their algorithms to increase these display rates to direct users to such resources. By doing this, search engines may contribute to the prevention of drug-related suicides.

Agent-based Testing: An Automated Approach toward Artificial Reactions to Human Behavior

ABSTRACTVast amounts of news are consumed through algorithmically curated media environments, such as search engines, social networking sites, or news aggregators. This renders algorithmic content curation with much societal relevance and highlights the urgent need for independent and resilient academic research. Therefore, a plethora of methodological approaches have been applied, such as case studies, expert interviews, observations, or agent-based approaches. The paper discusses the applicability of these methodological efforts for journalism studies, showing that all of these approaches face their limitations, especially with regard to external validity, recruitment difficulties, and data reliability. Thereby, agent-based testing represents one of the most promising approaches to overcome plenty of these methodological limitations. Agent-based testing is a systematic and experimental approach that emulates online human behavior to test algorithmically curated media environments under various conditions. For this to be achieved properly, this paper suggests a multitude of settings and requirements to adequately face the technological, legal, and ethical challenges, which come with the empirical investigation of algorithmic content curation. Ultimately, the paper presents both general considerations and practical instructions (using the “ScrapeBot”) to employ agent-based testing for journalism studies.

Investigating Google's suicide-prevention efforts in celebrity suicides using agent-based testing: A cross-national study in four European countries.

Google can act as a "gatekeeper" for individuals who seek suicide-related information online (e.g., "how to kill oneself"). The search engine displays a "suicide-prevention result" (SPR) at the very top of some suicide-related search results. This SPR comes as an info box and contains supposedly helpful crisis help information such as references to a telephone counseling service. It remains unknown, however, how Google has implemented the SPR in the especially dangerous context of celebrity suicide for which imitational copycat suicides in vulnerable individuals are most likely. Relying on agent-based testing, a computational social science method, we emulated a total of 137,937 Google searches in April 2019, using both general suicide-related and specific celebrity suicide-related search terms. Given the recently discovered language-based differences in SPR display rates, we held the language constant and focused on German-speaking populations in four European countries. The SPR was never shown in searches for celebrities who died by suicide in all four countries. Furthermore, analyses indicated a digital divide in access to suicide-prevention information with moderately high SPR display rates in Germany and Switzerland, yet with no display in Austria and Belgium. Higher SPR display rates could support global suicide-prevention efforts at virtually no cost by providing preventive information to vulnerable users precisely at the moment when it is apparently needed.

Googling Politics: Parties, Sources, and Issue Ownerships on Google in the 2017 German Federal Election Campaign

Democratic election campaigns require informed citizens. Yet, while the Internet allows for broader information through greater media choices, algorithmic filters, such as search engines, threaten to unobtrusively shape individual information repertoires. The purpose of this article is to analyze what search results people encounter when they employ various information orientations, and how these results reflect people’s attributions of issue ownership. A multimethod approach was applied during the 2017 German Federal Election campaign. First, human search behavior depicting various information orientations was simulated using agent-based testing to derive real search results from Google Search, which were then manually coded to identify information sources and ascribe issue ownerships. Second, a survey asked participants about which issues they attribute to which party. We find that search results originated mainly from established news outlets and reflected existing power relations between political parties. However, issue-ownership attributions of the survey participants were reflected poorly in the search results. In total, the results indicate that the fear of algorithmic constraints in the context of online search might be overrated. Instead, our findings (1) suggest that political actors still fail to claim their core issues among political search results, (2) highlight that news media (and thus existing media biases) feature heavily among search results, and (3) call for more media literacy among search engine users.

Equal access to online information? Google’s suicide-prevention disparities may amplify a global digital divide

Worldwide, people profit from equally accessible online health information via search engines. Therefore, equal access to health information is a global imperative. We studied one specific scenario, in which Google functions as a gatekeeper when people seek suicide-related information using both helpful and harmful suicide-related search terms. To help prevent suicides, Google implemented a “suicide-prevention result” (SPR) at the very top of such search results. While this effort deserves credit, the present investigation compiled evidence that the SPR is not equally displayed to all users. Using a virtual agent-based testing methodology, a set of 3 studies in 11 countries found that the presentation of the SPR varies depending on where people search for suicide-related information. Language is a key factor explaining these differences. Google’s algorithms thereby contribute to a global digital divide in online health-information access with possibly lethal consequences. Higher and globally balanced display frequencies are desirable.

The problem of land value betterment: a simplified agent-based test

In this paper, we employ behaviour-driven cellular automata as a simplified agent-based modelling approach to test the seminal Coase theorem, with a policy focus on the land value betterment effect of urban infrastructure provision. Four prototypical development regimes are identified from international practice: (1) the private developer-oriented Metroland model dating back to the suburbanisation of London in the nineteenth century England; (2) the private household-led special district model, which can be observed in many contemporary US suburbs; (3) the public planning-regulated model, as featured in most post-war European welfare state countries, including the present-day UK; (4) the public state-as-developer model, which characterises what has been taking place in China. A repeated analysis of variance (ANOVA) based on the results of a large number of cellular automata simulations suggests no significant difference between models I and II in terms of their welfare outcomes measured by aggregate utility. However, models III and IV are both found to generate significantly less welfare than models I and II, under strict assumptions of zero transaction costs, perfect information, perfect capital markets and perfect competition.

Agent-based unit testing environment for extreme programming

This paper presents a new approach of using multi-agent system (MAS) to build a software engineering environment to support unit testing process for extreme programming (XP). The MAS reference model by Foundation for Intelligent Physical Agents (FIPA) is extended in this paper. The extended components are: 1) agent roles, 2) storage components, and 3) ontology. The FIPA-complied Java Agent Development framework (JADE) agent platform is used in implementation. The Darpa agent mark-up language (DAML) is used as process data format, including ontology. And, JENA API by Hewlett-Packard (HP) is used to store DAML ontology. Further, examples are given to show feasibility of this approach. The expected benefits of this agent-based approach are to: 1) Improve environment flexibility and reliability, 2) Facilitate XP practices, and 3) Use DAML format that is suitable for agents.

Agent-based unit testing environment for extreme programming

This paper presents a new approach of using multi-agent system (MAS) to build a software engineering environment to support unit testing process for extreme programming (XP). The MAS reference mode...

Fuzzy complexity assessment model for resource negotiation and allocation in agent-based software testing framework

Although each paradigm has its own influence in the software engineering field on the support of its proficiencies, due to the exceptional growth of the software industry, researchers continue to strive for more efficient and powerful techniques. As new requirements arise, on the one hand, from the increasing complexity of modern software systems, and on the other hand, from distribute of today's information economies, it has been recognized that the modularity and reusability provided by existing techniques and approaches are in sufficient. Agents are being advocated as a next generation ad hoc model for engineering complex, distributed systems they facilitate the automated software testing by virtue of their high-level decomposition, independency and parallel activation. Here, we address a set of more specific characteristics of agent-based approach (modularity, independency and parallel activation) and its efficacy in software testing. In this paper, we do not just described the claims for agent-based approach in software testing, also developed an ad hoc distributed multi-agent system for software. The multi-agent system illustrated here is on the basis of few basic operational real-world testing techniques, as an attempt to describe how to practice Agent-Oriented Software Testing (AOST), which has not previously done.

Multi-agent-based integrated framework for intra-class testing of object-oriented software

The primary features of the object-oriented paradigm lead to develop a complex and compositional testing framework for object-oriented software. Agent-oriented approach has become a trend in software engineering. Agent technologies facilitate the software testing by virtue of their high-level independency with parallel activation and automation. This paper proposed an integrated framework that has been built on two existing testing techniques namely Mutation Testing and Capability Testing. In both the cases, testing is carried out at Autonomous Unit Level (AUL) and Inter-Procedural Level (IPL). Mutation-Based Testing-Agent and Capability Assessment Testing-Agent have been developed for performing AUL testing and Method Interaction Testing-Agent has been developed for performing IPL testing. This agent-based framework is an attempt for developing an automated software testing environment and among the several phases of Software Development Life Cycle (SDLC), this framework is recommended for unit testing in code complete phase and alpha phase. This methodology gives the basic approach to agent-based frameworks for testing and to optimization of agent-based testing schedules, subject to timing constraints. This adds ''interesting new opportunities in the object-oriented software testing phase'' to the existing literature that is concerned with software testing frameworks.

Deployed software component testing using dynamic validation agents

Software component run-time characteristics are highly dependent on their actual deployment situation. Validating that software components meet required functional and non-functional properties is time consuming and for some properties quite challenging. We describe the use of “validation agents” to automate the testing of deployed software components to verify that component functional and non-functional properties are met. Our validation agents utilise “component aspects” that describe functional and non-functional cross-cutting concerns impacting software components. Aspect information is queried by our validation agents and these construct and run automated tests on the deployed software components. The agents then determine if the deployed components meet their aspect-described requirements. Some agents deploy an existing performance test-bed generation tool to run realistic loading tests on these components. We describe the motivation for our work, how component aspects are designed and encoded, our automated agent-based testing process, the architecture and implementation or our validation agents, and our experiences using them.