User Agent Research Articles

Frankenstein's Monster in the Metaverse: User Interaction With Customized Virtual Agents.

Enabled by the latest achievements in artificial intelligence (AI), computer graphics as well as virtual, augmented, and mixed reality (VR/AR/MR), virtual agents are increasingly resembling humans in both their appearance and intelligent behavior. This results in enormous potential for agents to support users in their daily lives, for example in customer service, healthcare, education or the envisioned all-encompassing metaverse. Today's technology would allow users to customize their conversation partners in the metaverse - as opposed to reality - according to their preferences, potentially improving the user experience. On the other hand, there is little research on how reshaping the head of a communication partner might affect the immediate interaction with them. In this paper, we investigate the user requirements for and the effects of agent customization. In a two-stage user study ($N=30$), we collected both self-reported evaluations (e.g., intrinsic motivation) and interaction metrics (e.g., interaction duration and number of tried out items) for the process of agent customization itself as well as data on how users perceived the subsequent human-agent interaction in VR. Our results indicate that users only wish to have full customization for agents in their personal social circle, while for general services, a selection or even a definite assignment of pre-configured agents is sufficient. When customization is offered, attributes such as gender, clothing or hair are subjectively more relevant to users than facial features such as skin or eye color. Although the customization of human interaction partners is beyond our control, customization of virtual agents significantly increases perceived social presence as well as rapport and trust. Further findings on user motivation and agent diversity are discussed in the paper.

Learning from conceptual models – a study of the emergence of cooperation towards resource protection in a social–ecological system

Abstract. Engaging ecological resource users in intervention to protect resources is challenging for governments due to the self-interest of users and uncertainty about intervention consequences. Focusing on a case of forest insect infestations, we addressed questions of resource protection and environmentally responsible behavior promotion with a conceptual model. We coupled a forest infestation model with a social model in which a governing agent applies a mechanism for the recognition and promotion of environmentally responsible behavior among several user agents. We ran the coupled model in various scenarios with a reinforcement-learning algorithm for the governing agent as well as best-case, worst-case, and random baselines. Results showed that a proper recognition policy facilitates emergence of environmentally responsible behavior. However, ecosystem health may deteriorate due to temporal differences between the social and ecological systems. Our work shows it is possible to gain insight about complexities of social–ecological systems with conceptual models through scenario analysis.

Human-AI Teamwork Interface Design Using Patterns of Interactions

Increasingly capable Artificial Intelligence (AI) agents are reaching new performance levels and humans need to work effectively with them in a variety of collaborative team structures. Of particular interest is “shared control collaboration”, where actions taken by the team depend on the blended combination of inputs from all team entities. We conducted an experiment where participants played a game and maneuvered a spacecraft while working in concert with an AI agent and simultaneously performing a secondary task. The experiment results (i) show different patterns of interactions across user interface designs, agent capabilities, and participants’ game experience, (ii) underscore the importance of providing information about the shared control collaborative task to humans who are working with a less-capable agent, and (iii) imply the potential of real-time communication about agent’s state to achieve better teamwork. Our findings are expected to be transferable to other shared control collaboration settings, including AI-enabled autopilots.

Human-AI collaboration: Unraveling the effects of user proficiency and AI agent capability in intelligent decision support systems

Artificial intelligence (AI) agents are integral components of modern intelligent decision support systems (IDSS), providing their capability to assist in decision-making processes. Understanding the influence of AI involvement on human responses within these systems is crucial for fostering appropriate reliance on AI advice and ensuring a comfortable user experience. This study delves into the impact of AI agents on decision change, correct rate change, confidence rate change and pleasure rating among users. It explores the main and interactive effects of user proficiency and AI agent capability on user responses. A block-factorial experiment involving 45 participants was conducted. The results indicated that high-capability AI agents were associated with substantial decision change, correct rate change and confidence rate change. Participants with higher proficiency, both subjectively and objectively, exhibited reduced reliance on AI agents for task execution. Notably, the involvement of AI agents led to a slight decline in correct rates for participants of high competence (i.e., participants' independent correct rate exceeds AI agent capability), while still enhancing their confidence levels. Additionally, agents lacking human-like embodiment tended to enhance participants’ pleasure ratings. Individual differences in gender, collaboration experience with AI agent, and stereotype on AI agent capability also shaped the decision changes and pleasure ratings of participants. These findings underscore the presence of complementary competence interaction between user and AI agent within IDSS, and can be applied in designing capability of advisory AI agents with consideration of user proficiency and individual characteristics. Relevance to industryThe findings can be used as guidelines for designing AI agents and specifying their application strategies to integrate the assistance of AI agents effectively in intelligent decision support systems.

Client-side and Server-side Tracking on Meta: Effectiveness and Accuracy

Growing concern over digital privacy has led to the widespread use of tracking restriction tools, such as ad blockers, Virtual Private Networks (VPN), and privacy-focused web browsers. All major browser vendors have also deprecated, or plan to deprecate, third-party cookies to reduce tracking. Despite these efforts, advertising companies continuously innovate to overcome these restrictions. Recently, advertising platforms, like Meta, have been promoting server-side tracking solutions to bypass traditional browser-based tracking restrictions. This paper explores how server-side tracking technologies can link website visitors with their user accounts on Meta products. The goal is to assess the effectiveness and accuracy of employing this technology, as well as the effect of tracking restrictions on online tracking. Our methodology involves a series of experiments where we integrate Meta's client-side tracker (the Meta Pixel) and server-side technology (the Conversions API) on different web pages. We then drive traffic to these pages and evaluate the success rate of linking website visitors to their profiles on Meta products. Our findings show that Meta's server-side technology can match between 34% and 51% of website visitors to user profiles on Meta products using basic information like the visitor's IP address, user agent, and location data. This is comparable to Pixel-based user matching in optimal conditions (i.e., in the absence of tracking restrictions), which links between 42% and 61% of user profiles. Nevertheless, we see a considerable difference in accuracy: while the Pixel-based tracking achieves 100% accuracy, less than 65% of the profiles matched by server-side tracking are accurate.

Agent-based modelling of polarized news and opinion dynamics in social networks: a guidance-oriented approach

Abstract In the contemporary social network landscape, opinion polarization has emerged as a prominent challenge, sparking concerns about the effective guidance of news sentiment and mitigation of opposing opinions. This is particularly pertinent in the intricate web of social networks, where complexity reigns supreme. Addressing this pivotal issue, this article introduces a news opinion guidance approach grounded in motif recognition. To accurately mirror real-world social networks, we have crafted an agent-based model that simulates polarized news propagation. This model encompasses diverse media agents and user agents, meticulously replicating the news dissemination process within the network. In our quest to unveil the underlying structures of social networks, we have enhanced the Augmented Multiresolution Network approach, incorporating multi-dimensional node attributes for more nuanced clustering and network mapping. This refinement enables us to pinpoint potential motif regions with greater precision. Leveraging these insights, we introduce a triangular motif-based opinion guidance strategy aimed at shaping opinion distribution by bolstering the influence of nodes within these motifs. Once the pertinent motifs are identified, we undertake simulation experiments that reveal the remarkable efficacy of our motif recognition-driven guidance strategy. Notably, it reduces opinion polarization by a substantial 74% compared to scenarios without guidance strategies. This research offers a fresh perspective on crafting personalized and targeted news sentiment guidance strategies. It presents a versatile and potent computational framework for understanding and managing polarization phenomena in social networks, carrying profound theoretical and practical ramifications.

Long-term resilience in wastewater management: Optimizing treated wastewater allocation with a dynamic multi-agent approach

Water scarcity poses a significant challenge to sustainable development, necessitating innovative approaches to manage limited resources efficiently. Effective water resource management involves not just the conservation and distribution of freshwater supplies but also the strategic reuse of treated wastewater (TWW). This study proposes a novel approach for the optimal allocation of treated wastewater among three key sectors (user agents): agriculture, industry, and urban green space. Recognizing the intricate interplays among these sectors, System Dynamics (SD) and Agent-Based Modeling (ABM) were integrated in a Complex Adaptive System (CAS) to capture the interactions and feedback mechanisms inherent within treated wastewater allocation systems. The Non-dominated Sorting Genetic Algorithm II (NSGA-II) serves as the optimization tool, enabling the identification of optimal allocation strategies across various management scenarios over a 25-year simulation period. Our research navigates the complexities of long-term resource management, accounting for each sector's evolving its objectives and guidelines along the whole system objectives and strategies. The outcomes demonstrate how treated wastewater can be effectively distributed to support economic and social equity –as the system objectives-while supporting agricultural and industrial growth and enhancing efficiency and social well-being –reflecting individual agent objectives-within the CAS framework. The research explores four distinct management scenarios, each prioritizing different sectors to address water resource management challenges. Notably, all four scenarios align with the strategies required by the ruler (government), providing strategic guidance to water resource managers for decision-making. The simulation results reveal a scenario where all sectors' demands are met, with Scenario 4 emerging as the most effective. Scenario 4 aligned with the objectives and guidelines of each sector, demonstrating significant improvements in the CY (Agriculture agent index; increased from 0.2 to 0.68), IGI (Industry agent index; increased from 1 to 1.63), and GAI (Urban Green Space agent index; increased from 1 to 1.23) indices over the 25-year simulation period. By providing a strategic blueprint for policymakers and stakeholders, this study contributes significantly to the discourse on sustainable water resource management, presenting a replicable model for similar contexts globally, where judicious allocation of treated wastewater is paramount for achieving harmony between human activity and ecological preservation.

Effects of Demonstrating Consensus Between Robots to Change User’s Opinion

In recent years, the research of humanoid robots that can change users’ opinions has been conducted extensively. In particular, two robots have been found to be able to improve their persuasiveness by cooperating with each other in a sophisticated manner. Previous studies have evaluated the changes in opinions when robots showed consensus building. However, users did not participate in the conversations, and the optimal strategy may change depending on their prior opinions. Therefore, in this study, we developed a system that adaptively changes conversations between robots based on user opinions. We investigate the effect on the change in opinions when the discussion converges to the same position as the user and when it converges to a different position. We conducted two subject experiments in which a user and virtual robotic agents talked to each other using buttons in a crowded setting. The results showed that users with confidence in their opinions increased their confidence when the robot agents’ opinions converged to the same position and decreased their confidence when the robot agents’ opinions converged to a different position. This will significantly contribute to persuasion research using multiple robots and the development of advanced dialogue coordination between robots.

MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIs

As the number of selectable items increases, point-and-click interfaces rapidly become complex, leading to a decrease in usability. Adaptive user interfaces can reduce this complexity by automatically adjusting an interface to only display the most relevant items. A core challenge for developing adaptive interfaces is to infer user intent and chose adaptations accordingly. Current methods rely on tediously hand-crafted rules or carefully collected user data. Furthermore, heuristics need to be recrafted and data regathered for every new task and interface. To address this issue, we formulate interface adaptation as a multi-agent reinforcement learning problem. Our approach learns adaptation policies without relying on heuristics or real user data, facilitating the development of adaptive interfaces across various tasks with minimal adjustments needed. In our formulation, a user agent mimics a real user and learns to interact with an interface via point-and-click actions. Simultaneously, an interface agent learns interface adaptations, to maximize the user agent's efficiency, by observing the user agent's behavior. For our evaluation, we substituted the simulated user agent with actual users. Our study involved twelve participants and concentrated on automatic toolbar item assignment. The results show that the policies we developed in simulation effectively apply to real users. These users were able to complete tasks with fewer actions and in similar times compared to methods trained with real data. Additionally, we demonstrated our method's efficiency and generalizability across four different interfaces and tasks.

Remapping the Document Object Model using Geometric and Hierarchical Data Structures for Efficient Eye Control

The Web Content Accessibility Guidelines (WCAG) are there to ensure that websites are perceivable, operable, understandable and robust across different user agents and assistive technologies. However, people who rely on eye trackers (ETs) may find that even WCAG-compliant websites are hard to access, and this is further accentuated by designs that offer little to no affordances for ET interaction. Areas with a high density of interactive elements, along with hierarchical navigation menus, such as megamenus or fly-out menus, are just two examples where ET interaction can be problematic. This paper introduces two novel interaction patterns as part of a purpose-built gaze-native web browser (Cactus), namely (a) Quadtree-based Target Selection with Secondary Confirmation and (b) Hierarchical Re-rendering of Navigation Menus. We present results from a between-subject single-blind study with 30 participants and report on metrics such as performance, perceived workload and usability, with demonstrable improvements over the state of the art.

Low-carbon economic scheduling of multi-energy system based on price interactive response

—Regarding the increase of carbon dioxide emissions with growing energy demand, recent emerging technologies for multi-energy system have attracted much attention, as they can effectively couple and coordinate different energies to further control carbon dioxide emissions. This paper focuses on distribution-level multi-energy system and proposes an optimal scheduling method for hydrogen support based on dual-agent energy-carbon price interactive response. Firstly, a hydrogen-supported multi-energy closed-loop flow mode is developed, which provides an effective way to improve renewable energy consumption and brings additional economic revenue of solid carbon. Secondly, a dual-agent energy-carbon price interactive response scheme is designed based on the concept of energy-carbon participation ratio, which takes carbon dioxide emissions of gas load and various carbon revenues into account. This scheme can motivate users for energy demand adjustment and active carbon dioxide reduction by setting energy and carbon prices as two guiding signals. Finally, a bi-level optimal scheduling model is proposed for dual agents of energy service provider and users to maximize the benefit of each side. Case study illustrates that the proposed model can effectively balance benefits of dual agents and has economic advantage of carbon dioxide reduction, while promoting the consumption of high proportion renewable energy and system stability.

Cognitive model of collective behavior of distributed intelligent multi-agent systems

The goal of this study is to develop a model of collective behavior of intelligent agents based on neurocognitive multi-agent architectures. The consistency of agent behavior is based on a single semantic model that is based on the desire of all agents to maximize the energy that a given agent or a collective extract from the environment as a result of performing actions prescribed by a suboptimal plan for such behavior. The main method used to synthesize collective behavior is to create conditions for the growth and development of the controlling multi- agent neurocognitive architecture of an intelligent agent based on the data it receives in the process of interaction with the user, autonomous or robotic agents.

WEB CLOUD WEB-BASED CLOUD STORAGE FOR SECURE DATA SHARING ACROSS PLATFORMS

With more and more data moving to the cloud, privacy of user data have raised great concerns. Client-side encryption/decryption seems to be an attractive solution to protect data security, however, the existing solutions encountered three major challenges: low security due to encryption with low-entropy PIN, inconvenient data sharing with traditional encryption algorithms, and poor usability with dedicated software/plugins that require certain types of terminals. This work designs and implements WebCloud, a practical browser-side encryption solution, leveraging modern Web technologies. It solves all the above three problems while achieves several additional remarkable features: robust and immediate user revocation, fast data processing with offline encryption and outsourced decryption. Notably, our solution works on any device equipped with a Web user agent, including Web browsers, mobile and PC applications. We implement WebCloud based on ownCloud for basic file management utility, and utilize WebAssembly and Web Cryptography API for complex cryptographic operations integration. Finally, comprehensive experiments are conducted with many well-known browsers, Android and PC applications, which indicates that WebCloud is cross-platform and efficient. As an interesting by-product, the design of WebCloud naturally embodies a dedicated and practical ciphertext-policy attribute-based key encapsulation mechanism (CP-ABKEM) scheme, which can be useful in other applications.

MOVING FROM INFORMAL INTERACTION TO FOCUSED KNOWLEDGE SHARING AMONG USERS IN AN AGENT-BASED LEARNING ENVIRONMENT

Informal interaction among users may reveal significant information regarding their behaviors in smallcommunities. Although apps like WhatsApp are very popular for informal interaction activities, they, however, lack in providing the user with contextual information of other users present in the surroundings who might be helpful in addressing the queries thus enabling focused information exchange. Thus, there is a requirement of informal interactive environments that can be built on-the-fly, providing anytime-anywhere cost-effective interaction possibilities to the users. Mobile adhoc networks that can be built over wireless networks using wi-fi enabled smart phones, can provide a promising solution to the above problem, even in regions lacking adequate infrastructure. This paper proposes MAKSS, a mobile agent-based knowledge sharing system, comprising mobile user agents to facilitate information exchange among groups of users available in a close vicinity like a college campus. In this system, a user agent belongs to two views- a) a physical view formed by cellular frequency ranges of mobile devices b) a context view constituting a group of user agents having similar interests or contexts. These views help in dissemination of user queries and responses within a group (multicasting) or their routing to the intended persons (of same context as sender but belonging to different physical views). The simulated experiment highlights that the proposed system would work well in introducing users to the focused groups and also evaluates how user agents behave in the collaborative process of knowledge exchange on the parameters of inquisitiveness, responsiveness, reciprocity and cooperativeness.

Computation Offloading and Beamforming Optimization for Energy Minimization in Wireless-Powered IRS-Assisted MEC

Intelligent reflecting surface (IRS) has been recently exploited as a symbiotic radio (SR) technology to improve energy-and spectral-efficiencies in wireless systems. In this paper, we consider a symbiotic IRS-assisted mobile edge computing (MEC) system that allows edge users to first harvest RF power from a hybrid access point (HAP) and then offload its computational workload to the MEC server associated with the HAP. We aim to minimize the HAP’s energy consumption by jointly optimizing the users’ offloading schemes, the HAP’s active beamforming, and the IRS’s passive beamforming strategies. We propose an optimization-driven hierarchical deep deterministic policy gradient (OH-DDPG) framework to decompose the energy minimization problem into the optimization and the learning sub-problems, respectively. The outer-loop DDPG learning method adapts the IRS’s passive beamforming strategy, while the inner-loop optimization deals with the other control variables with reduced dimensionality. Moreover, to improve the learning efficiency, we extend OH-DDPG to the multi-agent scenario. In particular, the HAP first estimates the users’ offloading strategy by the inner-loop optimization and shares it with all user agents. Then, each user agent refines its offloading decision using the DDPG algorithm independently. This can avoid signaling overhead among users and improve the multi-user learning efficiency. Simulation results show that the proposed OH-DDPG and the multi-user extension can achieve significant performance gains compared to the conventional model-free learning algorithms.

Snoopy: A Webpage Fingerprinting Framework With Finite Query Model for Mass-Surveillance

Internet users are vulnerable to privacy attacks despite the use of encryption. Webpage fingerprinting, an attack that analyzes encrypted traffic, can identify the webpages visited by a user in a given website. Recent research works have been successful in demonstrating webpage fingerprinting attacks on individual users, but have been unsuccessful in extending their attack for mass-surveillance. The key challenges in performing mass-scale webpage fingerprinting arises from (i) the sheer number of combinations of user behavior and preferences to account for, and; (ii) the bound on the number of website queries imposed by the defense mechanisms (e.g., DDoS defense) deployed at the website. These constraints preclude the use of conventional data-intensive ML-based techniques. In this work, we propose Snoopy, a first-of-its-kind framework, that performs webpage fingerprinting for a large number of users visiting a website. Snoopy caters to the generalization requirements of mass-surveillance while complying with a bound on the number of website accesses (finite query model) for traffic sample collection. For this, Snoopy uses a feature (i.e., sequence of encrypted resource sizes) that is either unaffected or predictably affected by different browsing contexts (OS, browser, caching, cookie settings). Snoopy uses static analysis techniques to predict the variations caused by factors such as header sizes, MTU, and User Agent String that arise from the diversity in browsing contexts. We show that Snoopy achieves approximately 90% accuracy when evaluated on most websites, across various browsing contexts. A simple ensemble of Snoopy and an ML-based technique achieves approximately 97% accuracy while adhering to the finite query model, in cases when Snoopy alone does not perform well.

TECHNES: Penyedia Layanan Jasa Service Berbasis Aplikasi Android

This article describes how the waterfall method approach was used for the creation of an Android app by Technes, an app-based service provider. The waterfall method is a methodical and linear approach to developing software that starts with analyzing requirements and progresses through design, testing, and implementation. In this article, we will discuss how the waterfall method was applied to the development of the Technes app to ensure it was well-organized and efficient. The waterfall methodology's requirements analysis, design, implementation, testing, and implementation phases are all thoroughly explained. Developing a solid understanding of the technical user requirements and deciding on the functionality that the app should have are the tasks involved in the requirements analysis process. An intuitive and responsive user interface is then produced during the design phase to enhance the user experience. The main concern of this article is how the waterfall methodology can help solve challenging problems when developing complex Android applications. We discuss many phases of the waterfall method such as requirements analysis, design, implementation, testing, and planning, and we illustrate how each stage contributes to the final outcome of the Technes application. The findings of the discussion show that the waterfall method can help organize, track, and evaluate each stage of app development in a clear and organized way. We also recognize the limitations of the waterfall method with respect to changing user requirements and backend operations, but these issues can be resolved with effective communication. The end result is a Technes application that fully meets user needs, has a responsive user agent, and offers a consistent user experience.

Client-Side Encryption Based Secure Data Sharing Schemes for Cloud Platforms

Abstract: With more and more data moving to the cloud, privacy of user data have raised great concerns. Client-side encryption/decryption seems to be an attractive solution to protect data security, however, the existing solutions encountered three major challenges: low security due to encryption with low-entropy PIN, inconvenient data sharing with traditional encryption algorithms, and poor usability with dedicated software/plugins that require certain types of terminals. This work designs and implements Web Cloud, a practical browser-side encryption solution, leveraging modern Web technologies. It solves all the above three problems while achieves several additional remarkable features: robust and immediate user revocation, fast data processing with offline encryption and outsourced decryption. Notably, our solution works on any device equipped with a Web user agent, including Web browsers, mobile and PC applications. We implement Web Cloud based on own Cloud for basic file management utility, and utilize Web Assembly and Web Cryptography API for complex cryptographic operations integration. Finally, comprehensive experiments are conducted with many well-known browsers, Android and PC applications, which indicates that WebCloud is cross-platform and efficient. As an interesting by-product, the design of Web Cloud naturally embodies a dedicated and practical ciphertext- policy attribute-based key encapsulation mechanism (CP-AB-KEM) scheme, which can be useful in other applications

Distributed source-user estimation over directed graphs

This paper tackles an estimation problem in networked systems. A given time-varying signal is assumed to be measured or computed by only a subset of agents, named sources, while the other agents, the users, are required to estimate this signal in a distributed way. Each agent communicates only with a subset of neighbouring mates and the communication topology is described by a directed graph with relatively weak connectivity properties. The problem is solved for two class of signals with, respectively, null or bounded derivative of a given order, by resorting to a bank of distributed estimators of the signal and its derivatives run by each user agent. Convergence properties and noise rejection capabilities are investigated. Simulations are run to show the effectiveness of the approach and assess its performance.

Enhancing JWT Authentication and Authorization in Web Applications Based on User Behavior History

The rapid growth of the web has transformed our daily lives and the need for secure user authentication and authorization has become a crucial aspect of web-based services. JSON Web Tokens (JWT), based on RFC 7519, are widely used as a standard for user authentication and authorization. However, these tokens do not store information about the user’s behavior history. To address this issue, this paper presents a solution to enhance the trustworthiness of user authentication in web applications based on their behavior history. The solution considers factors such as the number of password attempts, IP address consistency, and user agent type and assigns a weight or percentage to each. These weights are summed up and stored in the user’s account, and updated after each transaction. The proposed approach was implemented using the .NET framework, C# programming language, and PostgreSQL database. The results show that the proposed solution effectively increases the level of trust in user authentication. The paper concludes by highlighting the strengths and limitations of the proposed solution.