Mobile Vision Systems Research Articles

Analysis of the Potential Use of Unmanned Aerial Vehicles and Image Processing Methods to Support Road and Parking Space Management in Urban Transport

Progressive urban density affects city centers especially and results in growing congestion, lack of parking spaces, and increasing environmental costs of transportation, causing increased air pollutant emissions and noise. These phenomena reduce the attractiveness of the city and result in a degradation of the quality of life for its residents. In light of these phenomena, there is a clear need for intelligent management of urban space using new technologies that would be complementary to existing intelligent transportation systems. Expanding information resources obtained from mobile cameras will have a positive impact on increasing the efficiency of transportation management and use of limited space in city centers. It will also have an impact on reducing external transport costs and increasing the quality of logistics services provided in the city. The main aim of the paper is to develop a concept of a transport management system in cities using mobile vision systems mounted on unmanned aerial vehicles. The model will concern the cases of lane occupation by freight vehicles and the analysis of parking spaces in the city in order to improve their management. The results of the developed model will contribute to the automation of the parking space management process and increase the efficiency of the use of city parking space resources.

An online semantic mapping system for extending and enhancing visual SLAM

We present a real-time semantic mapping approach for mobile vision systems with a 2D to 3D object detection pipeline and rapid data association for generated landmarks. Besides the semantic map enrichment the associated detections are further introduced as semantic constraints into a simultaneous localization and mapping (SLAM) system for pose correction purposes. This way, we are able generate additional meaningful information that allows to achieve higher-level tasks, while simultaneously leveraging the view-invariance of object detections to improve the accuracy and the robustness of the odometry estimation. We propose tracklets of locally associated object observations to handle ambiguous and false predictions and an uncertainty-based greedy association scheme for an accelerated processing time. Our system reaches real-time capabilities with an average iteration duration of 65 ms and is able to improve the pose estimation of a state-of-the-art SLAM by up to 68% on a public dataset. Additionally, we implemented our approach as a modular ROS package that makes it straightforward for integration in arbitrary graph-based SLAM methods.

CONTOUR METHOD FOR POSITIONING OBJECTS IN MOBILE COMPUTER VISION SYSTEMS

The paper considers the possibility of solving the problem of improving the quality of technical vision using the contour method, which is used to position objects in mobile computer vision systems. The hardware part of the object positioning system includes two video cameras, a Raspberry Pi 3 microcomputer, a depth contour map screen, and a motor control unit. The codes of programs based on the OpenCV library, the algorithm of the system and examples of the implementation of the contour method are given. The algorithm of the developed positioning technique includes the selection of the contours of objects on the frames of a stereopair, removal of all open contours, calculation of the moment (center of mass) of each closed contour, determination of the displacement along the x-axis of the moments of the corresponding contours, filling each closed contour with points with a brightness inversely proportional to the displacement of the moments. The presence of two video cameras, a Raspberry Pi 3 microcomputer, a contour depth map screen provides stereoscopic and panoramic "vision", that is, the ability to determine the presence of objects and their distance, as well as to get an overall picture in the "field of view" of the system. The engine control unit allows mobile devices to avoid obstacles. Based on the analysis of the research results, it was found that the proposed system provides an increase in the quality of positioning of objects and a decrease in the required computing resource, which gives a significant decrease in power consumption and ensures the autonomy of the system.

VIDEO DATA QUALITY IMPROVEMENT METHODS AND TOOLS DEVELOPMENT FOR MOBILE VISION SYSTEMS

Subject of study. The article proposes new methods of input and preliminary processing of video data for web- and specialized pi-cameras in monocular and stereo vision systems based on Raspberry Pi microcomputers to improve the quality of work of modern mobile vision systems. This approach is always relevant, since the design of modern vision systems constantly requires new non-trivial hardware, algorithmic and software solutions. Objectives. The goals are to compare quality indicators of the known methods of input and preliminary processing of video data in vision systems and to develop new methods and algorithms providing better speed, the necessary frame resolution and the independence of the frame brightness from changes in scene illumination while reading video data. Methods used. The paper formulates a comprehensive criterion for improving the quality of the Raspberry Pi microcomputer video input and preprocessing video data. Based on the accepted quality indicators (video input speed, resolution, and stability indicators of average brightness of the current frames of the received video stream), video input and preprocessing algorithms that satisfy the specified requirements are synthesized. This allows to find the optimal method for processing video data and to overcome the contradiction of reducing the input speed due to the need of increasing the resolution of video frames for each project. The created universal program for input and preliminary processing of these data allowed to obtain quantitative estimates of the effectiveness of the developed algorithms and formulate recommendations for their further use. All this allows you to significantly increase the efficiency of using Raspberry Pi microcomputers in modern mobile vision systems. The results obtained are the basis for the creation of a universal software product for high-speed input (in real time) and preliminary processing of video data for face detection and recognition systems, as well as stereo vision systems. Conclusions. The conducted experimental studies confirmed the efficiency and effectiveness of the proposed methods and algorithms for high-speed input of video data with different values of the resolution of the frame and the ability to adaptively adjust its brightness. Based on the created methods and algorithms, various options for its software implementation are proposed. This allows us to recommend the results for practical use. Prospects for further research include the expansion of the vector of criteria for assessing quality and features for optimizing video data, as well as the creation of new algorithms and various versions of programs based on them.

Dual-Resolution Dual-Path Convolutional Neural Networks for Fast Object Detection.

Downsampling input images is a simple trick to speed up visual object-detection algorithms, especially on robotic vision and applied mobile vision systems. However, this trick comes with a significant decline in accuracy. In this paper, dual-resolution dual-path Convolutional Neural Networks (CNNs), named DualNets, are proposed to bump up the accuracy of those detection applications. In contrast to previous methods that simply downsample the input images, DualNets explicitly take dual inputs in different resolutions and extract complementary visual features from these using dual CNN paths. The two paths in a DualNet are a backbone path and an auxiliary path that accepts larger inputs and then rapidly downsamples them to relatively small feature maps. With the help of the carefully designed auxiliary CNN paths in DualNets, auxiliary features are extracted from the larger input with controllable computation. Auxiliary features are then fused with the backbone features using a proposed progressive residual fusion strategy to enrich feature representation.This architecture, as the feature extractor, is further integrated with the Single Shot Detector (SSD) to accomplish latency-sensitive visual object-detection tasks. We evaluate the resulting detection pipeline on Pascal VOC and MS COCO benchmarks. Results show that the proposed DualNets can raise the accuracy of those CNN detection applications that are sensitive to computation payloads.

A Survey of Computational Photography in the Small: Creating intelligent cameras for the next wave of miniature devices

The sheer ubiquity of smartphones and other mobile vision systems has begun to transform the way that humans and machines interact with each other and the way that they interact with the world. Even so, a new wave of widespread computing is on the horizon, with devices that are even smaller. These are micro and nano platforms, with feature sizes less than one millimeter. These types of platforms are quickly maturing out of research labs. These devices can potentially induce futuristic applications; for example, swarms of robotic flapping insects could have applications in agriculture and security, while medical devices such as those described in and would enable body area networks and minimally invasive procedures. Devices such as those described in are commercially available and could allow the creation of far-flung sensor networks.

Low-Power, Real-Time Object-Recognition Processors for Mobile Vision Systems

A new low-power object-recognition processor achieves real-time robust recognition, satisfying modern mobile vision systems' requirements. The authors introduce an attention-based object-recognition algorithm for energy efficiency, a heterogeneous multicore architecture for data- and thread-level parallelism, and a network on a chip for high on-chip bandwidth. The fabricated chip achieves 30 frames/second throughput and an average 320 mW power consumption on test 720p video sequences, yielding 640 GOPS/W and 10.5 NJ/pixel energy efficiency.

Mobile Media in Action: Remote Target Localization and Tracking

Smartphones have come a long way over the past 10 years—from containing a simple database of calendar entries and contacts to including a digital camera, digital compass, GPS, and thousands of apps available with the touch of a button on the screen. People like smartphones for many reasons, particularly because of their mobility, friendly user interface, large number of useful applications, interactive games, and higher bandwidth with the help of 3G networks. From a technical point of view, the fundamental reason behind the popularity of smartphones is that they are not only a communications device, but also provide a powerful sensing and computing platform. A growing number of computing tasks previously executed on PCs are now being performed on smartphones. Because of all this, smartphones are increasingly attracting attention from researchers, with research efforts in mobile media streaming, video retargeting, location-based systems, and mobile vision systems, among others. PositionIt, a mobile media-based remote target-localization and tracking system, is one part of a comprehensive system that uses networked smartphones to localize a remote target of interest by fusing data sensed by microphones, cameras, and other embedded sensors.

Optimized hardware architecture for object recognition and tracking

The recent research is focused on development of mobile vision systems and algorithms suitable for very large-scale integration implementation. These systems can be used in various applications. We propose a novel field-programmable gate array (FPGA)-based architecture for early vision. The central idea is to take into account the perceptual aspects of visual tasks inspired by biological vision systems: shape and color. For this reason, we propose an original approach based on a system implemented in an FPGA connected to a CMOS imager. The proposed algorithm implementation analysis and optimization methodology under resource constraints enable one to implement the algorithm on only one FPGA chip. To prove the proposed concept the system was implemented and tested on an autonomous mobile platform. The implementation framework enables direct algorithm implementation in application-specific integrated circuit.

Inter-Frame Motion Estimation for Mobile Vision Systems

One important issue when dealing with mobile vision systems is that small rotations of the camera can seriously complicate the interpretation of the observations. In this paper we present an approach that estimates these rotations in practice. The motion of our vehicle is predominantly translational. However, irregularities in the road and small steering actions cause additional roll, pitch and yaw components to this egomotion. We estimate these rotational components from the observed optical flow field of background points, using advanced least squares estimation techniques. In this paper we illustrate the performance of our method in practice. The performance of different orthogonal least squares techniques is compared. The results indicate the robustness and high accuracy of the approach in practice.

Charge-Coupled Computing For Focal Plane Image Preprocessing

A new class of charge-coupled devices called charge-coupled-computing devices is described. These analog circuits perform arithmetic functions such as addition, subtraction, and magnitude comparison in the charge domain. The circuits are compact and are designed to be insensitive to rail voltages, simplifying their utilization. These devices, in conjunction with input, output, and analog memory circuits, can be combined to form a simple but general-purpose and fully programmable charge-coupled computer. A prototype charge-coupled computer has been fabricated and tested. Prospects for forming a large array of computers (e.g., 1000 to 10,000) on a single chip for spatially parallel image preprocessing are discussed. Such image plane preprocessing of data would find use in real-time mobile robot vision systems, in which low power, lightweight computing is critical for economical viability.