Real-time Object Recognition Research Articles

SHORT: Segmented histogram technique for robust real-time object recognition

Object recognition is a broad area that covers several topics including face recognition, gesture recognition, human gait recognition, traffic road signs recognition, among many others. Object recognition plays a vital role in several real-time applications such as video surveillance, traffic analysis, security systems, and image retrieval. This work introduces a novel, real-time object recognition approach, namely “SHORT”: segmented histogram object recognition technique. “SHORT” implements segmentation technique applied on the histogram of selected vectors of an image to identify similar image(s) in a database. The proposed technique performance was evaluated by means of two different image databases, namely the Yale Faces and Traffic Road Signs. The robustness was also assessed by applying different levels of distortion on both databases using Gaussian noise and blur, and testing distortion impact on recognition rates. Additionally, the efficiency was evaluated by comparing the recognition execution time of the proposed technique with another well-known recognition algorithm called “Eigenfaces”. The experimental results revealed successful recognition on clear and distorted objects. Moreover, “SHORT” performed 4.5X faster than the “Eigenfaces” algorithm under the same conditions. Furthermore, the “SHORT” algorithm was implemented on FPGA hardware by exploiting data parallelism to improve the execution performance. The results showed that the FPGA hardware version is 28X faster than the “Eigenfaces” algorithm, which makes “SHORT” a robust and practical solution for real-time applications.

Folo: Latency and Quality Optimized Task Allocation in Vehicular Fog Computing

With the emerging vehicular applications, such as real-time situational awareness and cooperative lane change, there exist huge demands for sufficient computing resources at the edge to conduct time-critical and data-intensive tasks. This paper proposes Folo, a novel solution for latency and quality optimized task allocation in vehicular fog computing (VFC). Folo is designed to support the mobility of vehicles, including vehicles that generate tasks and the others that serve as fog nodes. Considering constraints on service latency, quality loss, and fog capacity, the process of task allocation across stationary and mobile fog nodes is formulated into a joint optimization problem. This task allocation in VFC is known as a nondeterministic polynomial-time hard problem. In this paper, we present the task allocation to fog nodes as a bi-objective minimization problem, where a tradeoff is maintained between the service latency and quality loss. Specifically, we propose an event-triggered dynamic task allocation framework using linear programming-based optimization and binary particle swarm optimization. To assess the effectiveness of Folo, we simulated the mobility of fog nodes at different times of a day based on real-world taxi traces and implemented two representative tasks, including video streaming and real-time object recognition. Simulation results show that the task allocation provided by Folo can be adjusted according to actual requirements of the service latency and quality, and achieves higher performance compared with naive and random fog node selection. To be more specific, Folo shortens the average service latency by up to 27% while reducing the quality loss by up to 56%.

A semi-automatic image-based object recognition system for constructing as-is IFC BIM objects based on fuzzy-MAUT

Building information modelling (BIM) could support different activities throughout the life cycle of a building and has been widely applied in design and construction phases nowadays. However, BIM has not been widely implemented in the operation and maintenance (O&M) phase. As-is information for the majority of existing buildings is not complete and even outdated or incorrect. Lack of accurate and complete as-is information is still one of the key reasons leading to the low-level efficiency in O&M. BIM performs as an intelligent platform and a database that stores, links, extracts and exchanges information in construction projects. It has shown promising opportunities and advantages in BIM applications for the improvement in O&M. Hence, an effective and convenient approach to record as-is conditions of the existing buildings and create as-is BIM objects would be the essential step for improving efficiency and effectiveness of O&M, and furthermore possibly refurbishment of the building. Many researchers have paid attention to different systems and approaches for automated and real-time object recognition in past decades. This paper summarizes state-of-the-art statistical matching-based object recognition methods and then presents the image-based Industry Foundation Classes (IFC) BIM object creation application, which extracts object information by simply conducting point-and-click operations. Furthermore, the object recognition research system is introduced, including recognizing structure object types and their corresponding materials. This paper combines the multi-attribute utility theory (MAUT) with the fuzzy set theory to be Fuzzy-MAUT, since the MAUT allows complex and powerful combinations of various criteria and fuzzy set theory assists improving the performance of this system. With the goal of creating an effective method for as-is IFC BIM objects construction, this image-based object recognition system and its recognition process are further validated and tested. Key challenges and promising opportunities are also addressed.

Real-time concealed object detection and recognition in passive imaging at 250 GHz

The presented study concerns detection and recognition of hidden objects covered with various types of clothing by using passive imagers operating in a terahertz (THz) range at 1.2mm (250GHz). The aim of this study is to propose a detection and classification algorithm operating robustly at a high processing speed. The paper briefly describes properties of the THz spectrum, theoretical limitations, performance of the imager, and physical properties of fabrics in a wide range of frequencies. Two methods have been presented, trained, and tested using a dataset with various configurations in sessions each lasting 30min. During experiments, different clothes and hidden objects have been combined. The paper presents a comparison of robust detection and recognition methods for concealed objects using a multiframe single-shot detector and region-based fully convolutional networks. The comparison of the original results of various experiments is presented.

3D object recognition method with multiple feature extraction from LiDAR point clouds

During autonomous driving, fast and accurate object recognition supports environment perception for local path planning of unmanned ground vehicles. Feature extraction and object recognition from large-scale 3D point clouds incur massive computational and time costs. To implement fast environment perception, this paper proposes a 3D recognition system with multiple feature extraction from light detection and ranging point clouds modified by parallel computing. Effective object feature extraction is a necessary step prior to executing an object recognition procedure. In the proposed system, multiple geometry features of a point cloud that resides in corresponding voxels are computed concurrently. In addition, a scale filter is employed to convert feature vectors from uncertain count voxels to a normalized object feature matrix, which is convenient for object-recognizing classifiers. After generating the object feature matrices of all voxels, an initialized multilayer neural network (NN) model is trained offline through a large number of iterations. Using the trained NN model, real-time object recognition is realized using parallel computing technology to accelerate computation.

Real Time Object Recognition and Classification using Deep Learning

Navigation in indoor environments is highly challenging for visually impaired person, particularly in spaces visited for the first time. Various solutions have been proposed to deal with this challenge. In this project consider as the real time object Recognition and classification using deep learning algorithms. Object detection mainly deals with identification of real time objects such as people, animals, and objects. Object detection algorithm uses a wide range of image processing applications for extracting the object's desired portion. This enables one to identify the objects and calculate the accuracy of the object and deliver through voice. Using this information, the system determines the user's trajectory and can locate possible obstacles in that route.

Improved embedding product quantization

Real-time object matching and recognition is a challenging task in computer vision probably due to the extensively computational overload posed by large and high dimensional data space. Indexing approaches can help achieving thousands of times in speedups when comparing to sequential search. In this work, we propose a novel usage of product quantization and hierarchical clustering so that search speedups can be even improved further. To validate the proposed indexing algorithm, a number of experiments have been carried out on different datasets. Experimental results demonstrate that the proposed method works very efficiently and far outperforms many other state-of-the-art techniques.

Threshold-Guided Design and Optimization for Harris Corner Detector Architecture

High-speed corner detection is an essential step in many real-time computer vision applications, e.g., object recognition, motion analysis, and stereo matching. Hardware implementation of corner detection algorithms, such as the Harris corner detector (HCD) has become a viable solution for meeting real-time requirements of the applications. A major challenge lies in the design of power, energy and area efficient architectures that can be deployed in tightly constrained embedded systems while still meeting real-time requirements. In this paper, we proposed a bit-width optimization strategy for designing hardware-efficient HCD that exploits the thresholding step in the algorithm to determine interest points from the corner responses. The proposed strategy relies on the threshold as a guide to truncate the bit-widths of the operators at various stages of the HCD pipeline with only marginal loss of accuracy. Synthesis results based on 65-nm CMOS technology show that the proposed strategy leads to power-delay reduction of 35.2%, and area reduction of 35.4% over the baseline implementation. In addition, through careful retiming, the proposed implementation achieves over 2.2 times increase in maximum frequency while achieving an area reduction of 35.1% and power-delay reduction of 35.7% over the baseline implementation. Finally, we performed repeatability tests to show that the optimized HCD architecture achieves comparable accuracy with the baseline implementation (average decrease of repeatability is less than 0.6%).

Object registration in semi-cluttered and partial-occluded scenes for augmented reality

This paper proposes a stable and accurate object registration pipeline for markerless augmented reality applications. We present two novel algorithms for object recognition and matching to improve the registration accuracy from model to scene transformation via point cloud fusion. Whilst the first algorithm effectively deals with simple scenes with few object occlusions, the second algorithm handles cluttered scenes with partial occlusions for robust real-time object recognition and matching. The computational framework includes a locally supported Gaussian weight function to enable repeatable detection of 3D descriptors. We apply a bilateral filtering and outlier removal to preserve edges of point cloud and remove some interference points in order to increase matching accuracy. Extensive experiments have been carried to compare the proposed algorithms with four most used methods. Results show improved performance of the algorithms in terms of computational speed, camera tracking and object matching errors in semi-cluttered and partial-occluded scenes.

Real-time video object detection and classification using hybrid texture feature extraction

ABSTRACT In video processing, feature extraction and classification are necessary steps to classify the video frames. To improve accuracy, an efficient texture-based feature extraction is required. Also, before improving the feature extraction, the background subtraction step is almost equal to the ground truth level. An efficient real-time video object recognition and classification utilizing hybrid texture feature extraction are proposed. A stationary wavelet transform-based joint bilateral filtering is used to remove the noise. An effective background subtraction is employed before extracting the features. In feature extraction, hybrid texture feature extraction is proposed where gray level co-occurrence matrix (GLCM) and discrete wavelet transform (DWT) features are combined and applied to the categorization process. The GLCM is used to extract the second-order statistical texture features for the estimation of motion in the videos. And, the DWT algorithm reduces the storage space in real-time video processing. The main objectives are to implement the real-time video object detection and classification using the hybrid texture feature extraction method called the GLCM-DWT technique, and investigate the performance evaluation of the proposed methodology and compare with the existing technique. The parameters, namely accuracy, sensitivity, specificity, and execution time are to be evaluated for the proposed algorithm in the MATLAB software.

Real-Time Recognition Method for 0.8 cm Darning Needles and KR22 Bearings Based on Convolution Neural Networks and Data Increase

The complexity of the background and the similarities between different types of precision parts, especially in the high-speed movement of conveyor belts in complex industrial scenes, pose immense challenges to the object recognition of precision parts due to diversity in illumination. This study presents a real-time object recognition method for 0.8 cm darning needles and KR22 bearing machine parts under a complex industrial background. First, we propose an image data increase algorithm based on directional flip, and we establish two types of dataset, namely, real data and increased data. We focus on increasing recognition accuracy and reducing computation time, and we design a multilayer feature fusion network to obtain feature information. Subsequently, we propose an accurate method for classifying precision parts on the basis of non-maximal suppression, and then form an improved You Only Look Once (YOLO) V3 network. We implement this method and compare it with models in our real-time industrial object detection experimental platform. Finally, experiments on real and increased datasets show that the proposed method outperforms the YOLO V3 algorithm in terms of recognition accuracy and robustness.

Research on Substation Real-Time Object Recognition Algorithm Based on Deep Learning

The use of deep learning algorithms to intelligently identify objects from video has a wide range of applications. The more advanced system based on the tensorflow framework is proposed for deep neural network recognition of objects in this paper. Our work is mainly the following: (1) Using the vgg deep convolutional network architecture as the infrastructure. (2) Real-time recognition of objects in video. (3) The system framework can be used in substations to realize real-time identification of objects with security threats.

A comprehensive review of recent advances on deep vision systems

Real-time video objects detection, tracking, and recognition are challenging issues due to the real-time processing requirements of the machine learning algorithms. In recent years, video processing is performed by deep learning (DL) based techniques that achieve higher accuracy but require higher computations cost. This paper presents a recent survey of the state-of-the-art DL platforms and architectures used for deep vision systems. It highlights the contributions and challenges from over numerous research studies. In particular, this paper first describes the architecture of various DL models such as AutoEncoders, deep Boltzmann machines, convolution neural networks, recurrent neural networks and deep residual learning. Next, deep real-time video objects detection, tracking and recognition studies are highlighted to illustrate the key trends in terms of cost of computation, number of layers and the accuracy of results. Finally, the paper discusses the challenges of applying DL for real-time video processing and draw some directions for the future of DL algorithms.

A novel robust algorithm for position and orientation detection based on cascaded deep neural network

Abstract Estimating position and orientation of the object by using machine vision is essential in industrial automation. The traditional canny operator and Hough transform edge detection algorithm is widely used, but its accuracy and real-time object recognition in complex backgrounds are very limited. Other algorithms such as SVM and BP network are usually inaccurate for regression issues. In this paper, the method of a cascade of convolution networks is proposed which results in high precision pose estimates. SSD is utilized to obtain the bounding box of the object to narrow down the recognition range. Convolution neural network is utilized to detect the orientation of the object. This method can extract weak features of the sample image. In generally, the proposed method possess a greatly improved accuracy and recognition rate compared with the traditional algorithm.

Real-time Recognition and Pursuit in Robots Based on 3D Depth Data

In this work, we address the problem of robot pursuit based on a real-time object recognition system with 3D depth sensors. Compared with traditional RGBD data based recognition approaches, we propose a novel global online descriptor designed for object recognition from solely depth data. Proposed descriptor, which we name as Differential Histogram of Normal Vectors (DHONV), is designed to extract the geometric characteristics of the captured 3D surfaces of the objects presented in depth images. In order to obtain a brief description of the visible 3D surfaces of each object, we quantize the differential angles of the surface’s normal vectors into a 1D histogram. The object recognition experiments on a self-collected dataset and a benchmark RGB-D object dataset show that our proposed descriptor outperforms other depth data based descriptors. Moreover, we conducted real-time experiments with RoboCars. Our experiments with RoboCars validate our proposed method capability to perform a real-time recognition and pursuit tasks within indoor environment based solely on depth data.

Toward real-time 3D object recognition: A lightweight volumetric CNN framework using multitask learning

3D data are becoming increasingly popular and easier to access, making 3D information increasingly important for object recognition. Although volumetric convolutional neural networks (CNNs) have been exploited to recognize 3D objects and have achieved notable progress, their computational cost is too high for real-time applications. In this paper, we propose a lightweight volumetric CNN architecture (namely, LightNet) to address the real-time 3D object recognition problem leveraging on multitask learning. We use LightNet to simultaneously predict class and orientation labels from complete and partial shapes. In contrast to the earlier version of this method presented at 3DOR 2017, this extended version introduces batch normalization and better training strategies to improve the recognition accuracy, and also includes more experiments on the newly released large-scale ShapeNet Core55 dataset. Our model has been evaluated on three publicly available benchmarks of complete 3D CAD shapes and incomplete point clouds. Experimental results show that our model achieves the state-of-the-art 3D object recognition performance among shallow volumetric CNNs with the smallest number of training parameters. It is also demonstrated that our method can perform accurate object recognition in real time (less than 6 ms).

3D gesture based real-time object selection and recognition

Among other naturally performed gestures by users, pointing gesture is one of the most intuitive interfaces for selection. Many systems have been proposed on pointing gesture for HCI systems. The detection and selection of pointed object can be helpful in many applications. In this paper, we approximate the pointing location of the user by localizing the 3D position of upper human body skeletal joints and tracking the skeleton. This was achieved with real time constraints, using Microsoft Kinect sensor, which has the ability to estimate human joint location invariant to pose, clothing, body shape etc. The pointing direction is based on the line of sight connecting the shoulder joint to hand joint location. The developed system senses intentional arm pointing gesture. Pointing of the user in the earth ordinal direction is also approximated with respect to the user location in 3D space. The target pointed to is extracted, localized, zoomed and recognized. The recognition of object is performed by Hue-Saturation Histogram matching. The developed method allows selecting the object using both hands, either separately or simultaneously.

Transferring deep knowledge for object recognition in Low-quality underwater videos

In recent years, underwater video technologies allow us to explore the ocean in scientific and noninvasive ways, such as environmental monitoring, marine ecology studies, and fisheries management. However the low-light and high-noise scenarios pose great challenges for the underwater image and video analysis. We here propose a CNN knowledge transfer framework for underwater object recognition and tackle the problem of extracting discriminative features from relatively low contrast images. Even with the insufficient training set, the transfer framework can well learn a recognition model for the special underwater object recognition task together with the help of data augmentation. For better identifying objects from an underwater video, a weighted probabilities decision mechanism is introduced to identify the object from a series of frames. The proposed framework can be implemented for real-time underwater object recognition on autonomous underwater vehicles and video monitoring systems. To verify the effectiveness of our method, experiments on a public dataset are carried out. The results show that the proposed method achieves promising results for underwater object recognition on both test image datasets and underwater videos.

A Real Time Object Recognition and Counting System for Smart Industrial Camera Sensor

In the industry 4.0, factories around the world grow automated and intelligent, and where smart camera plays an important role. Smart camera is equipped with processor, memory, communication interface, and operating system, so it can process large amounts of data in advance to assist follow-up automatic inspection and judgment. Additionally, since smart camera is an independent system, it will not affect the original system of factories, which is an immense advantage in troubleshooting. Besides, thanks to technology breakthroughs in recent years, using Graphics Processing Unit (GPU) to implementing tons of parallel computing helps to significantly boost the overall efficiency. Therefore, when a rising number of factories consider improving production capacity of production lines, how to use GPU to assist the improvement is an important issue. Based on this scenario, this paper used NVidia Tegra TX1 platform with 256 GPU CUDA cores and Quad-core ARM Cortex A57 processor and Basler USB 3.0 industrial camera to simulate a smart industrial camera, which has GPU and can perform a myriad of complex computations. This paper designed how to recognize and count objects in a real time manner in a high-speed industrial inspection environment with large volumes of data, so as to verify the concept (smart camera with GPU cores) we proposed. The experimental results proved our ideas, and the software design architecture provided in this paper is a simple and efficient design. In the future application in the Internet of Things or the Internet of Everything, this structure can be a valuable reference.

A hardware-oriented histogram of oriented gradients algorithm and its VLSI implementation

A challenging and important issue for object recognition is feature extraction on embedded systems. We report a hardware implementation of the histogram of oriented gradients (HOG) algorithm for real-time object recognition, which is known to provide high efficiency and accuracy. The developed hardware-oriented algorithm exploits the cell-based scan strategy which enables image-sensor synchronization and extraction-speed acceleration. Furthermore, buffers for image frames or integral images are avoided. An image-size scalable hardware architecture with an effective bin-decoder and a parallelized voting element (PVE) is developed and used to verify the hardware-oriented HOG implementation with the application of human detection. The fabricated test chip in 180 nm CMOS technology achieves fast processing speed and large flexibility for different image resolutions with substantially reduced hardware cost and energy consumption.