Distributed Vision System Research Articles

TempDiff: Feature Map-Level CNN Sparsity Enhancement at Near-Zero Memory Overhead via Temporal Difference

The demand for distributed vision systems at an unceasingly larger scale requires the availability of sensor nodes able to execute high-level visual tasks (e.g., object detection for visual monitoring). Such tasks routinely entail the high computational cost of Convolutional Neural Networks (CNN), which conflicts with the tight power budget and memory capacity of sensor nodes at the edge. This paper introduces an approach to reduce the computational cost of object detection in CNN accelerators available in edge devices. The proposed approach induces additional feature map-level sparsity (i.e., computation skipping) at inference time by exploiting temporal correlation among frames. The proposed approach has an uncommonly favorable computation-memory tradeoff, as significant computation reduction is achieved at the cost of very small additional memory for the storage of intermediate features. As further benefits, no architectural changes or retraining are required, allowing immediate deployment in existing vision frameworks and suppressing the need for storing multiple models. Results show that the proposed TempDiff method achieves up to 37% computation reduction with 1.1% accuracy drop based on the SSD(VGG16) object detection network, under both VIRAT and ImageNet-VID datasets. Similarly, 18.3% (35.8%) computation reduction at 3.3% (3.2%) memory overhead, and 3.8% (6.8%) accuracy drop in YOLOv1 (VGG16) (SSD (VGG16)) is achieved under the CAMEL dataset. Furthermore up to 58% computation reduction with 2% accuracy drop and 3.7% memory overhead were achieved for YOLOv3-Tiny network under the ImageNet-VID dataset.

Energy-Quality Scalable Memory-Frugal Feature Extraction for Always-On Deep Sub-mW Distributed Vision

In this work, an energy-quality (EQ) scalable and memory-frugal architecture for video feature extraction is introduced to reduce circuit complexity, power and silicon area. Leveraging on the inherent resiliency of vision against noise and inaccuracies, the proposed approach introduces properly selected EQ tuning knobs to reduce the energy of feature extraction at graceful quality degradation. As opposed to prior art, the proposed architecture enables the adjustment of such knobs, and adapts its cycle-level timing to reduce the amount of computation per frame at lower quality targets. As further benefit, the approach adds opportunities for energy reduction via aggressive voltage scaling. The proposed architecture mitigates the traditionally dominant area/energy of the on-chip memory by reducing the number of pixels stored on chip, introducing memory access reuse and on-the-fly computation. At the same time, EQ tuning preserves the ability to conventionally operate at maximum quality, when required by the task or the visual context. A 0.55 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> testchip in 40nm exhibits power down to 82μW at 5fps frame rate (i.e., 33X lower than prior art), while assuring successful object detection at VGA resolution. To the best of the authors' knowledge, this is the first feature extractor with sub-mW operation and sub-mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> area, making the proposed approach well suited for tightly power-constrained and low-cost distributed vision systems (e.g., video sensor nodes).

A Multi-Task Distributed Vision System Embedded on a Hex-Rotorcraft UAV

In this paper, are presented the general architecture and implementation of a multi-task distributed vision system designed and embedded onboard a Hex-Rotorcraft UAV. The system uses multiple cheap heterogeneous cameras in order to perform various tasks such as: ground target pedestrian detection, tracking, creating panoramic images, video stabilization and streaming multiple data/video feeds over a wireless secure channel. In what follows, are discussed this multi-agent architecture designed to provide our UAV with an embedded intelligent vision system using autonomous agents entrusted with managing the previously listed functionalities. In addition to the cheap set of USB and module cameras, the presented vision system is composed of a Local Data Processing Module connected to each camera and a Central Module used to control the overall system, process the regrouped data and streams it to the ground station. The overall vision system has been tested in real flights and is still under improvements.

Command Fusion Based Mobile Robot Tracking by Distributed Vision System of Robotic Space

The Robotic Space is able of obtaining information from the observed world where many intelligent sensing and tracking devices, such as computers and multi sensors, are distributed. According to the cooperation of many intelligent devices, the environment, it is very important that the system knows the location information to offer the useful services. In order to achieve these goals, we suggest a method for representing, tracking and human following by fusing distributed multiple vision systems in Robotic Space. And the article presents the integration of color distributions into SOM(Self Organizing Map) based particle filtering. We propose to track the moving objects by generating hypotheses not in the image plan but on the top-view reconstruction of the scene. Comparative results on real video sequences show the advantage of our method for multi-motion tracking. Experiments are carried out to evaluate the proposed performance.

Embedded distributed vision system for humanoid soccer robot

Computer vision is one of the most challenging applications in sensor systems since the signal is complex from spatial and logical point of view. Due to these characteristics vision applications require high computing resources, which makes them especially difficult to use in embedded systems, like mobile robots with reduced amount memory and computing power. In this work a distributed architecture for humanoid visual control is presented using specific nodes for vision processing cooperating with the main CPU to coordinate the movements of the exploring behaviours. This architecture provides additional computing resources in a reduced area, without disturbing tasks related with low level control (mainly kinematics) with the ones involving vision processing algorithms. The information is exchanged allowing to linking control loops between both nodes.

Flexible Vision

This article presented a mobile agent-based distributed vision fusion architecture that provides a flexible vision fusion solution to increase power efficiency by reducing excessive communication and enhance sensor fusion capabilities with migratory in situ on demand algorithms for vision data processing and analysis. The IEEE FIPA standard-compliant mobile agent system, Mobile-C, implemented as a C library, is used as the foundation for the mobile agent-based distributed vision fusion architecture. Mobile agents dynamically migrate from one sensor node to another to fully combine all necessary sensor data in a desired manner specific to the system requesting the data. Dispatching mobile agents to target vision systems on the network is done on demand, reducing network congestion and the required communication bandwidth. The use of mobile agents in a distributed vision system allows for the encapsulation of specific fusion techniques. The differences between monolithic and mobile agent-based approaches along with future considerations were discussed. The validity of the architecture was proven through two separate case studies. The first case study involves the localization of a part in a real experimental setup with a retrofitted robotic workcell composed of a Puma 560, IBM 7575, conveyor system, and vision system. The second case study vertically and horizontally integrates multiple systems as a tier-scalable planetary reconnaissance experimental system involving two vision systems: a Puma 560 manipulator and a K-Team Khepera III mobile robot. All source code including Mobile-C, the mobile agents, and the mobile agent code presented in the article are available at the project Web site.

環境を認識し行動するためのロボットの視覚 -Active, Omnidirectional and Distributed Vision Systems-

環境を認識し行動するためのロボットの視覚 -Active, Omnidirectional and Distributed Vision Systems-

Robot localisation enhanced by a distributed vision system in dynamic environments

PurposeImage‐based localisation has been widely investigated in mobile robotics. However, traditional image‐based localisation approaches do not work when the environment appearance changes. The purpose of this paper is to propose a new system for image‐based localisation, which enables the approach to work also in highly dynamic environments.Design/methodology/approachThe proposed technique is based on the use of a distributed vision system (DVS) composed of a set of cameras installed in the environment and of a camera mounted on a mobile robot. The localisation of the robot is achieved by comparing the current image grabbed by the robot with the images grabbed, at the same time, by the DVS. Finding the DVS's image, most similar to the robot's image, gives a topological localisation of the robot.FindingsExperiments reported in the paper proved the system to be effective, even exploiting a pre‐existent DVS not designed for this application.Originality/valueWhilst, aware that DVSs, as the one used in this work, are not diffuse nowadays, this work is significant because a novel idea is proposed for dealing with dynamic environments in the image‐based localisation approach and the idea is validated with experiments. Camera Sensor networks currently are an emerging technology and they may be introduced in several daily environments in the future.

A Hybrid Distributed Vision System for Robot Localization

A Hybrid Distributed Vision System for Robot Localization

실시간 다중이동물체 추적에 의한 이동로봇의 위치개선

본 논문은 실내외 공간에서 인간을 포한함 이동물체의 영상정보를 이용하여 이동로봇의 자기위치를 인식하기 위한 방법을 제시하고 있다. 제시한 방법은 로봇자체의 DR센서 정보와 카메라에서 얻은 영상정보로부터 로봇의 위치추정방법을 결합한 것이다. 그리고 이동물체의 이전 위치정보와 관측 카메라의 모델을 사용하여 이동물체에 대한 영상프레임 좌표와 추정된 로봇위치간의 관계를 표현할 수 있는 식을 제시하고 있다. 또한 이동하는 인간과 로봇의 위치와 방향을 추정하기 위한 제어방법을 제시하고 이동로봇의 위치를 추정하기 위해서 칼만필터 방법을 적용하였다. 그리고 시뮬레이션 및 실험을 통하여 제시한 방법을 검증하였다. The Intelligent Space(ISpace) provides challenging research fields for surveillance, human-computer interfacing, networked camera conferencing, industrial monitoring or service and training applications. ISpace is the space where many intelligent devices, such as computers and sensors, are distributed. According to the cooperation of many intelligent devices, the environment, it is very important that the system knows the location information to offer the useful services. In order to achieve these goals, we present a method for representing, tracking and human Jollowing by fusing distributed multiple vision systems in ISpace, with application to pedestrian tracking in a crowd. This paper describes appearance based unknown object tracking with the distributed vision system in intelligent space. First, we discuss how object color information is obtained and how the color appearance based model is constructed from this data. Then, we discuss the global color model based on the local color information. The process of learning within global model and the experimental results are also presented.

歩行経路観測に基づく重要点抽出による移動ロボットナビゲーション

In this paper, we investigate a mobile robot navigation system which can localize the mobile robot correctly and navigate based on observation of human walking in order to operate in a space populated by humans with minimal disturbance to them. To realize this system, we utilize multiple intelligent devices embedded in the environment. The human walking paths are obtained from a distributed vision system. We extract important points from observation and build a topological map of nodes which are important points and of edges which are added based on the presence of paths between important points. In addition, the human walking paths between two important points are averaged to get frequently used paths. The averaged paths are utilized as paths of the mobile robots. The mobile robot navigation based on observation of human is also performed with the support of the system. The position and orientation of the mobile robot are estimated from wheel encoder and distributed vision system measurement data using an extended Kalman filter. The system navigates the mobile robot along the frequently used paths by tracking control.

Knowledge propagation in a distributed omnidirectional vision system

In this paper an omnidirectional Distributed Vision System (DVS) is presented. The presented DVS is able to learn to navigate a mobile robot in its working environment without any prior knowledge a...

A temporal fusion strategy for cross-camera data association

The purpose of this work is the design of a distributed vision system for vehicles tracking over wide areas. The tracking is performed by the re-identification of objects perceived by distant non-overlapping cameras. The data association is controlled by a temporal reasoning scheme. Decisions combine temporal and visual information. The visual information is composed by the 3D dimension and the normalised colour histogram of detected objects. Temporal constraints based on an acceleration model between nodes, are continuously updated with respect to the observed traffic behaviour. These constraints maintain a dynamic lifespan for all tracked objects. The management of the uncertainty represents an important component of the system. Statistical measurements are exploited at the sensor level information and a possibilistic approach permits to manage the ambiguities of the data association stage.

Cooperation Issues and Distributed Sensing for Multirobot Systems

This paper considers the properties a multirobot system should exhibit to perform an assigned task cooperatively. Our experiments regard specifically the domain of RoboCup middle-size league (MSL) competitions. But the illustrated techniques can be usefully applied also to other service robotics fields like, for example, videosurveillance. Two issues are addressed in the paper. The former refers to the problem of dynamic role assignment in a team of robots. The latter concerns the problem of sharing the sensory information to cooperatively track moving objects. Both these problems have been extensively investigated over the past years by the MSL robot teams. In our paper, each individual robot has been designed to become reactively aware of the environment configuration. In addition, a dynamic role assignment policy among teammates is activated, based on the knowledge about the best behavior that the team is able to acquire through the shared sensorial information. We present the successful performance of the Artisti Veneti robot team at the MSL Challenge competitions of RoboCup-2003 to show the effectiveness of our proposed hybrid architecture, as well as some tests run in laboratory to validate the omnidirectional distributed vision system which allows us to share the information gathered by the omnidirectional cameras of our robots

Human Tracking using Multiple-Camera-Based Global Color Model in Intelligent Space

We propose an global color model based method for tracking motions of multiple human using a networked multiple-camera system in intelligent space as a human-robot coexistent system. An intelligent space is a space where many intelligent devices, such as computers and sensors(color CCD cameras for example), are distributed. Human beings can be a part of intelligent space as well. One of the main goals of intelligent space is to assist humans and to do different services for them. In order to be capable of doing that, intelligent space must be able to do different human related tasks. One of them is to identify and track multiple objects seamlessly. In the environment where many camera modules are distributed on network, it is important to identify object in order to track it, because different cameras may be needed as object moves throughout the space and intelligent space should determine the appropriate one. This paper describes appearance based unknown object tracking with the distributed vision system in intelligent space. First, we discuss how object color information is obtained and how the color appearance based model is constructed from this data. Then, we discuss the global color model based on the local color information. The process of learning within global model and the experimental results are also presented.

Cooperative Localization and Tracking in Distributed Robot-Sensor Networks

This paper addresses the problem of real-time position and orientation estimation of networked mobile robots in two-dimensional Euclidean space with simultaneous tracking of a rigid unknown object based on exteroceptive sensory information extracted from distributed vision systems. The sufficient and necessary conditions for team localization are proposed. A localization and object tracking approach based on statistical operators and graph searching algorithms is presented for a team of robots localized with heterogeneous sensors. The approach was implemented in an experimental platform consisting of car-like mobile robots equipped with omnidirectional video cameras and IEEE 802.11b wireless networking. The experimental results validate the approach.

Appearance Based Object Identification for Mobile Robot Localization in Intelligent Space with Distributed Vision Sensors

Robots will be able to coexist with humans and support humans effectively in near future. One of the most important aspects in the development of human-friendly robots is to cooperation between humans and robots. In this paper, we proposed a method for multi-object identification in order to achieve such human-centered system and robot localization in intelligent space. The intelligent space is the space where many intelligent devices, such as computers and sensors, are distributed. The Intelligent Space achieves the human centered services by accelerating the physical and psychological interaction between humans and intelligent devices. As an intelligent device of the Intelligent Space, a color CCD camera module, which includes processing and networking part, has been chosen. The Intelligent Space requires functions of identifying and tracking the multiple objects to realize appropriate services to users under the multi-camera environments. In order to achieve seamless tracking and location estimation many camera modules are distributed. They causes some errors about object identification among different camera modules. This paper describes appearance based object representation for the distributed vision system in Intelligent Space to achieve consistent labeling of all objects. Then, we discuss how to learn the object color appearance model and how to achieve the multi-object tracking under occlusions.

Real-time multitarget tracking by a cooperative distributed vision system

Target detection and tracking is one of the most important and fundamental technologies to develop real-world computer vision systems such as security and traffic monitoring systems. This paper first categorizes target tracking systems based on characteristics of scenes, tasks, and system architectures. Then we present a real-time cooperative multitarget tracking system. The system consists of a group of active vision agents (AVAs), where an AVA is a logical model of a network-connected computer with an active camera. All AVAs cooperatively track their target objects by dynamically exchanging object information with each other With this cooperative tracking capability, the system as a whole can track multiple moving objects persistently even under complicated dynamic environments in the real world. In this paper we address the technologies employed in the system and demonstrate their effectiveness.

Mobile robot navigation by a Distributed Vision System

This paper proposes aDistributed Vision System (DVS), which is an intelligent infrastructure for mobile robots consisting of manyvision agents (VAs) embedded in an environment. The system monitors the environment from various viewing points with the VAs, maintains the dynamic environment models, and provides various information to robots. Based on this concept, we have developed a prototype of the DVS which consists of sixteen vision agents and simultaneously navigates two robots in a model town.

分散視覚システムによる移動ロボットの誘導

This paper proposes a general infrastructure for robot navigation in an outdoor environment. The infrastructure, called a Distributed Vision System, consists of vision agents connected with a computer network, monitors the environment, maintains the environment models, and provides various information for robots by organizing communication between the vision agents. As the first step of our research, we have developed a prototype of the distributed vision system for navigating mobile robots. The experimental results show that the system has an ability to navigate the robots in a complex environment.