Multi-object active search and tracking by multiple agents in untrusted, dynamically changing environments

Multi-object tracking has been a key research subject in many computer vision applications. We propose a novel approach based on multi-agent deep reinforcement learning (MADRL) for multi-object tracking to solve the problems in the existing tracking methods, such as a varying number of targets, non-causal, and non-realtime. At first, we choose YOLO V3 to detect the objects included in each frame. Unsuitable candidates were screened out and the rest of detection results are regarded as multiple agents and forming a multi-agent system. Independent Q-Learners (IQL) is used to learn the agents' policy, in which, each agent treats other agents as part of the environment. Then, we conducted offline learning in the training and online learning during the tracking. Our experiments demonstrate that the use of MADRL achieves better performance than the other state-of-art methods in precision, accuracy, and robustness.

Active search refers to the problem of efficiently locating targets in an unknown environment by actively making data-collection decisions. In this paper, we are focusing on multiple aerial robots (agents) detecting targets such as gas leaks, radiation sources or human survivors of disasters. One of the main challenges of active search with multiple agents in unknown environments is impracticality of central coordination due to the difficulties of connectivity maintenance. In this paper, we propose two distinct active search algorithms that allow for multiple robots to independently make data-collection decisions without a central coordinator. Throughout we consider that targets are sparsely located around the environment in keeping with compressive sensing assumptions and its applicability

Excitation emission spectroscopy (EES) has been used in the past to characterize many different types of tissue. This technique uses multiple excitation wavelengths and samples a complete optical spectrum for each, yielding an excitation-emission matrix (EEM). Upon study of the EEM, it is possible to determine the presence of multiple optical contrast agents since these dyes can have characteristic spectra that can be separated. Here, we demonstrate EES specifically designed for use in conjunction with MR. This EES is applied with an in-suite control setup that permits real-time navigation, utilizing active MR tracking catheters, and providing a platform for MR-guided tissue characterization. The EES system is used in a demonstration experiment to highlight MR imaging, MR guidance in conjunction with a catheter-based optical measurement.

The active search for objects of interest in an unknown environment has many robotics applications including search and rescue, detecting gas leaks or locating animal poachers. Existing algorithms often prioritize the location accuracy of objects of interest while other practical issues such as the reliability of object detection as a function of distance and lines of sight remain largely ignored. Additionally, in many active search scenarios, communication infrastructure may be unreliable or unestablished, making centralized control of multiple agents impractical. We present an algorithm called Noise-Aware Thompson Sampling (NATS) that addresses these issues for multiple ground-based robots performing active search considering two sources of sensory information from monocular optical imagery and depth maps. By utilizing Thompson Sampling, NATS allows for decentralized coordination among multiple agents. NATS also considers object detection uncertainty from depth as well as environmental occlusions and operates while remaining agnostic of the number of objects of interest. Using simulation results, we show that NATS significantly outperforms existing methods such as information-greedy policies or exhaustive search. We demonstrate the real-world viability of NATS using a pseudo-realistic environment created in the Unreal Engine 4 game development platform with the AirSim plugin.

A key objective of multi-agent surveillance systems
\nis to monitor a much larger region than the limited field-of-view
\n(FoV) of any individual agent by successfully exploiting
\ncooperation among multiple agents. Whenever either a centralized
\nor a distributed approach is pursued, this goal cannot
\nbe achieved unless an appropriately designed fusion strategy is
\nadopted. This paper presents a novel information fusion approach
\nby considering for each agent a known limited, and possibly
\ndifferent, FoV. The proposed method, named Bayesian-operation
\nInvaRiance on Difference-sets (BIRD) fusion, relies on Generalized
\nCovariance Intersection (GCI) and exploits a general and exact
\ndecomposition of each multi-object posterior by partitioning the
\nglobal FoV, i.e. the union of the FoVs of the fusing agents, into
\ncommon and exclusive FoVs. It is shown how BIRD fusion can
\nbe used to perform multi-object estimation based on random
\nfinite sets on both a centralized and a distributed peer-to-peer
\nsensor network. Simulation experiments on realistic multi-object
\ntracking scenarios demonstrate the effectiveness of BIRD fusion.

Information-driven control can be used to develop intelligent sensors that can optimize their measurement value based on environmental feedback. In object tracking applications, sensor actions are chosen based on the expected reduction in uncertainty also known as information gain. Random finite set (RFS) theory provides a formalism for quantifying and estimating information gain in multi-object tracking problems. However, estimating information gain in these applications remains computationally challenging. This paper presents a new tractable approximation of the RFS expected information gain applicable to sensor control for multi-object search and tracking. Unlike existing RFS approaches, the information gain approximation presented in this paper considers the contributions of non-ideal noisy measurements, missed detections, false alarms, and object appearance/disappearance. The effectiveness of the information-driven sensor control is demonstrated through two multi-vehicle search-while-tracking experiments using real video data from remote terrestrial and satellite sensors.

Through automatic control, intelligent sensors can be manipulated to obtain the most informative measurements about objects in their environment. In object tracking applications, sensor actions are chosen based on the predicted improvement in estimation accuracy, or information gain. Although random finite set theory provides a formalism for measuring information gain for multi-object tracking problems, predicting the information gain remains computationally challenging. This paper presents a new tractable approximation of the random finite set expected information gain applicable to multi-object search and tracking. The approximation presented in this paper accounts for noisy measurements, missed detections, false alarms, and object appearance/disappearance. The effectiveness of the approach is demonstrated through a ground vehicle tracking problem using real video data from a remote optical sensor.

This paper considers a multi-agent multi-objective path-finding problem that requires not only finding collision-free paths for multiple agents from their respective start locations to their respective goal locations but also optimizing multiple objectives simultaneously. In general, there is no single solution that optimizes all the objectives simultaneously, and the problem is thus to find the so-called Pareto-optimal frontier. To solve this problem, an algorithm called Multi-Objective Conflict-Based Search (MO-CBS) was recently developed and is guaranteed to find the exact Pareto-optimal frontier. However, MO-CBS does not scale well with the number of agents due to the large branching factor of the search, which leads to a lot of duplicated effort in agent-agent collision resolution. This paper therefore develops a new algorithm called Binary Branching MO-CBS (BB-MO-CBS) that reduces the branching factor as well as the duplicated collision resolution during the search, which expedites the search as a result. Our experimental results show that BB-MO-CBS reduces the number of conflicts by up to two orders of magnitude and often doubles or triples the success rates of MO-CBS on various maps given a runtime limit.

Conforming coalitions in Markov Stackelberg security games: Setting max cooperative defenders vs. non-cooperative attackers

Object Tracking is essential for many computer vision applications, such as autonomous navigation, surveillance, and robotics. Unlike Passive Object Tracking (POT), which relies on static camera viewpoints to detect and track objects across consecutive frames, Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios due to limited information gathering and processing capabilities, often resulting in suboptimal decision-making. Alleviating these limitations necessitates the development of a multi-agent system where different agents perform distinct roles and collaborate to enhance learning and robustness in dynamic and complex environments. Although some multi-agent approaches exist for AOT, they typically rely on external auxiliary agents, which require additional devices, making them costly. In contrast, we introduce the Collaborative System for Active Object Tracking (CSAOT), a method that leverages multi-agent deep reinforcement learning (MADRL) and a Mixture of Experts (MoE) framework to enable multiple agents to operate on a single device, thereby improving tracking performance and reducing costs. Our approach enhances robustness against occlusions and rapid motion while optimizing camera movements to extend tracking duration. We validated the effectiveness of CSAOT on various interactive maps with dynamic and stationary obstacles.

This paper presents a tool, namely Multi-Language Information Searching (MLIS), for a meaning-oriented search. MLIS allows users to access the right information and provides the search results with knowledge from different cultures and languages. MLIS takes advantage of agent technology as well as web services to enhance the quality of existing search engines in terms of accessibility, usability and flexibility. Their are several advantages in using MLIS, including: MLIS simultaneously performs in different languages translation and search activities as background processes hidden from users; it provides a friendly graphical user interface that instantly displays the search results in separated tabs, categorizing them according to languages chosen by users; it is proposed with a flexible architecture to automatically create multiple web-service agents based on users' preferences; it is service-independent and can integrate, for the same session, several web services from different service providers.

We introduce a novel active stereo vison-based object tracking system for a humanoid robot. The system tracks a moving object that is dynamically changing its appearance and scale. The system features an in-built learning process that simultaneously learns short term models for the object and potential distractors. These models evolve over time, rectifying the inaccuracies of the tracking in a cluttered scene and allowing the system to identify unusual events such as sudden displacement, hiding behind or being masked by an occluder, and sudden disappearance from the scene. The system deals with these through different response modes such as active search when the object is lost, intentional waiting for reappearance when the object is hidden, and reinitialization of the track when the object is suddenly displaced by the user. We demonstrate our system on the iCub robot in an indoor environment and evaluate its performance. Our experiments show a performance enhancement for long occlusions through the learning of distractor models.

Coordinate-aligned multi-camera collaboration for active multi-object tracking

This paper introduces a novel adaptive guidance law to address active target tracking of multiple agents system, where multiple followers construct a circular formation with time-varying formation radius. The relative ranges between multiple followers and a target, the follower-target relative velocity components along the line-of-sight and the follower-target relative velocity components normal to the line-of-sight can be regulated to early-designed ones by adaptive parameters of the guidance law, while all followers coordinate their motion to maintain a circular formation whose center pursues the active target. Numerical simulations with the comparisons demonstrate the effectiveness and the superiority of the proposed method.

Multi-agent multi-target tracking has a wide range of applications, including wildlife patrolling, security surveillance or environment monitoring. Such algorithms often make restrictive assumptions: the number of targets and/or their initial locations may be assumed known, or agents may be pre-assigned to monitor disjoint partitions of the environment, reducing the burden of exploration. This also limits applicability when there are fewer agents than targets, since agents are unable to continuously follow the targets in their fields of view. Multi-agent tracking algorithms additionally assume interagent synchronization of observations, or the presence of a central controller to coordinate joint actions. Instead, we focus on the setting of decentralized multi-agent, multi-target, simultaneous active search-and-tracking with asynchronous interagent communication. Our proposed algorithm DecSTER uses a sequential monte carlo implementation of the probability hypothesis density filter for posterior inference combined with Thompson sampling for decentralized multi-agent decision making. We compare different action selection policies, focusing on scenarios where targets outnumber agents. In simulation, we demonstrate that DecSTER is robust to unreliable inter-agent communication and outperforms information-greedy baselines in terms of the Optimal Sub-Pattern Assignment (OSPA) metric for different numbers of targets and varying teamsizes.