Vision-based Object Recognition Research Articles

StereoYOLO: A Stereo Vision-Based Method for Maritime Object Recognition and Localization

Image recognition is vital for intelligent ships’ autonomous navigation. However, traditional methods often fail to accurately identify maritime objects’ spatial positions, especially under electromagnetic silence. We introduce the StereoYOLO method, an enhanced stereo vision-based object recognition and localization approach that serves autonomous vessels using only image sensors. It is specifically refined for maritime object recognition and localization scenarios through the integration of convolutional and coordinated attention modules. The method uses stereo cameras to identify and locate maritime objects in images and calculate their relative positions using stereo vision algorithms. Experimental results indicate that the StereoYOLO algorithm boosts the mean Average Precision at IoU threshold of 0.5 (mAP50) in object recognition by 5.23%. Furthermore, the variation in range measurement due to target angle changes is reduced by 6.12%. Additionally, upon measuring the distance to targets at varying ranges, the algorithm achieves an average positioning error of 5.73%, meeting the accuracy and robustness criteria for maritime object collision avoidance on experimental platform ships.

Remote Perception Attacks against Camera-based Object Recognition Systems and Countermeasures

In vision-based object recognition systems imaging sensors perceive the environment and then objects are detected and classified for decision-making purposes; e.g., to maneuver an automated vehicle around an obstacle or to raise alarms for intruders in surveillance settings. In this work we demonstrate how camera-based perception can be unobtrusively manipulated to enable an attacker to create spurious objects or alter an existing object, by remotely projecting adversarial patterns into cameras, exploiting two common effects in optical imaging systems, viz., lens flare/ghost effects and auto-exposure control. To improve the robustness of the attack, we generate optimal patterns by integrating adversarial machine learning techniques with a trained end-to-end channel model. We experimentally demonstrate our attacks using a low-cost projector on three different cameras, and under different environments. Results show that, depending on the attack distance, attack success rates can reach as high as 100%, including under targeted conditions. We develop a countermeasure that reduces the problem of detecting ghost-based attacks into verifying whether there is a ghost overlapping with a detected object. We leverage spatiotemporal consistency to eliminate false positives. Evaluation on experimental data provides a worst-case equal error rate of 5%.

Improved Discriminative Object Localization Algorithm for Safety Management of Indoor Construction.

Object localization is a sub-field of computer vision-based object recognition technology that identifies object classes and locations. Studies on safety management are still in their infancy, particularly those aimed at lowering occupational fatalities and accidents at indoor construction sites. In comparison to manual procedures, this study suggests an improved discriminative object localization (IDOL) algorithm to aid safety managers with visualization to improve indoor construction site safety management. The IDOL algorithm employs Grad-CAM visualization images from the EfficientNet-B7 classification network to automatically identify internal characteristics pertinent to the set of classes evaluated by the network model without the need for further annotation. To evaluate the performance of the presented algorithm in the study, localization accuracy in 2D coordinates and localization error in 3D coordinates of the IDOL algorithm and YOLOv5 object detection model, a leading object detection method in the current research area, are compared. The comparison findings demonstrate that the IDOL algorithm provides a higher localization accuracy with more precise coordinates than the YOLOv5 model over both 2D images and 3D point cloud coordinates. The results of the study indicate that the IDOL algorithm achieved improved localization performance over the existing YOLOv5 object detection model and, thus, is able to assist with visualization of indoor construction sites in order to enhance safety management.

A FRAMEWORK FOR DESIGNING AN AUGMENTED REALITY APPLICATION FOCUSING ON OBJECT FUNCTION FOR CHILDREN WITH AUTISM

Augmented Reality (AR) is the evolution of the concept of Virtual Reality (VR). Its goal is to enhance a person's perception of the surrounding world. Augmented reality techniques are often applied to facilitate understanding and create attractive educational and health tools. As such, augmented reality is deemed suitable to be implemented as one of the potential intervention methods as the treatment of autism in a fun environment. Hence, this study is aimed to develop a conceptual framework to design augmented reality applications based on object function to help in the communication of children with autism. The study framework will be developed based on vision-based object recognition. Object recognition has been used in many applications, especially in bio-imaging, industrial inspection, and robotic vision. The findings of this study will benefit autistic children in visual communication and indirectly help them to effectively link objects with their functions. This framework will then help designers to develop augmented reality applications suited to be an intervention tool that fits the need of autistic children.

Integration of Industrially-Oriented Human-Robot Speech Communication and Vision-Based Object Recognition

This paper presents a novel method for integration of industrially-oriented human-robot speech communication and vision-based object recognition. Such integration is necessary to provide context for task-oriented voice commands. Context-based speech communication is easier, the commands are shorter, hence their recognition rate is higher. In recent years, significant research was devoted to integration of speech and gesture recognition. However, little attention was paid to vision-based identification of objects in industrial environment (like workpieces or tools) represented by general terms used in voice commands. There are no reports on any methods facilitating the abovementioned integration. Image and speech recognition systems usually operate on different data structures, describing reality on different levels of abstraction, hence development of context-based voice control systems is a laborious and time-consuming task. The aim of our research was to solve this problem. The core of our method is extension of Voice Command Description (VCD) format describing syntax and semantics of task-oriented commands, as well as its integration with Flexible Editable Contour Templates (FECT) used for classification of contours derived from image recognition systems. To the best of our knowledge, it is the first solution that facilitates development of customized vision-based voice control applications for industrial robots.

Stereo Vision-Based Object Recognition and Manipulation by Regions with Convolutional Neural Network

This paper develops a hybrid algorithm of adaptive network-based fuzzy inference system (ANFIS) and regions with convolutional neural network (R-CNN) for stereo vision-based object recognition and manipulation. The stereo camera at an eye-to-hand configuration firstly captures the image of the target object. Then, the shape, features, and centroid of the object are estimated. Similar pixels are segmented by the image segmentation method, and similar regions are merged through selective search. The eye-to-hand calibration is based on ANFIS to reduce computing burden. A six-degree-of-freedom (6-DOF) robot arm with a gripper will conduct experiments to demonstrate the effectiveness of the proposed system.

Vision-Based Object Recognition and Precise Localization for Space Body Control

The space motion control is an important issue on space robot, rendezvous and docking, small satellite formation, and some on-orbit services. The motion control needs robust object detection and high-precision object localization. Among many sensing systems such as laser radar, inertia sensors, and GPS navigation, vision-based navigation is more adaptive to noncontact applications in the close distance and in high-dynamic environment. In this work, a vision-based system serving for a free-floating robot inside the spacecraft is introduced, and the method to measure space body 6-DOF position-attitude is presented. At first, the deep-learning method is applied for robust object detection in the complex background, and after the object is navigated at the close distance, the reference marker is used for more precise matching and edge detection. After the accurate coordinates are gotten in the image sequence, the object space position and attitude are calculated by the geometry method and used for fine control. The experimental results show that the recognition method based on deep-learning at a distance and marker matching in close range effectively eliminates the false target recognition and improves the precision of positioning at the same time. The testing result shows the recognition accuracy rate is 99.8% and the localization precision is far less than 1% in 1.5 meters. The high-speed camera and embedded electronic platform driven by GPU are applied for accelerating the image processing speed so that the system works at best by 70 frames per second. The contribution of this work is to introduce the deep-learning method for precision motion control and in the meanwhile ensure both the robustness and real time of the system. It aims at making such vision-based system more practicable in the real-space applications.

A comprehensive survey on computer vision based approaches for automatic identification of products in retail store

The ability to recognize a product on the shelf of a retail store is an ordinary human skill. The same recognition problem presents an exceptional challenge for machine vision systems. Automatic detection of products on the shelf of a retail store provides enhanced value-added consumer experience and commercial benefits to retailers. Compared to machine vision based object recognition system, automatic detection of retail products in a store setting has lesser number of successful attempts. In this paper, we present a survey of machine vision based retail product recognition system and define a new taxonomy for this field. We also describe the intrinsic challenges associated with the problem. In this comprehensive survey of published papers, we analyze features used in state-of-the-art attempts. The performances of these approaches are compared. The details of publicly available datasets are presented. The paper concludes pointing to possible directions of research in related fields.

Combination of high-frequency SSVEP-based BCI and computer vision for controlling a robotic arm

Objective. Recent attempts in developing brain–computer interface (BCI)-controlled robots have shown the potential of this area in the field of assistive robots. However, implementing the process of picking and placing objects using a BCI-controlled robotic arm still remains challenging. BCI performance, system portability, and user comfort need to be further improved. Approach. In this study, a novel control approach, which combines high-frequency steady-state visual evoked potential (SSVEP)-based BCI and computer vision-based object recognition, is proposed to control a robotic arm for performing pick and place tasks that require control with multiple degrees of freedom. The computer vision can identify objects in the workspace and locate their positions, while the BCI allows the user to select one of these objects to be acted upon by the robotic arm. The robotic arm was programmed to be able to autonomously pick up and place the selected target object without moment-by-moment supervision by the user. Main results. Online results obtained from ten healthy subjects indicated that a BCI command for the proposed system could be selected from four possible choices in 6.5 s (i.e. 2.25 s for visual stimulation and 4.25 s for gaze shifting) with 97.75% accuracy. All subjects could successfully complete the pick and place tasks using the proposed system. Significance. These results demonstrated the feasibility and efficiency of combining high-frequency SSVEP-based BCI and computer vision-based object recognition to control robotic arms. The control strategy presented here could be extended to control robotic arms to perform other complicated tasks.

An Efficient Color Space for Deep-Learning Based Traffic Light Recognition

Traffic light recognition is an essential task for an advanced driving assistance system (ADAS) as well as for autonomous vehicles. Recently, deep-learning has become increasingly popular in vision-based object recognition owing to its high performance of classification. In this study, we investigate how to design a deep-learning based high-performance traffic light detection system. Two main components of the recognition system are investigated: the color space of the input video and the network model of deep learning. We apply six color spaces (RGB, normalized RGB, Ruta’s RYG, YCbCr, HSV, and CIE Lab) and three types of network models (based on the Faster R-CNN and R-FCN models). All combinations of color spaces and network models are implemented and tested on a traffic light dataset with 1280×720 resolution. Our simulations show that the best performance is achieved with the combination of RGB color space and Faster R-CNN model. These results can provide a comprehensive guideline for designing a traffic light detection system.

Baseline Fusion for Image and Pattern Recognition—What Not to Do (and How to Do Better)

The ever-increasing demand for a reliable inference capable of handling unpredictable challenges of practical application in the real world has made research on information fusion of major importance; indeed, this challenge is pervasive in a whole range of image understanding tasks. In the development of the most common type—score-level fusion algorithms—it is virtually universally desirable to have as a reference starting point a simple and universally sound baseline benchmark which newly developed approaches can be compared to. One of the most pervasively used methods is that of weighted linear fusion. It has cemented itself as the default off-the-shelf baseline owing to its simplicity of implementation, interpretability, and surprisingly competitive performance across a widest range of application domains and information source types. In this paper I argue that despite this track record, weighted linear fusion is not a good baseline on the grounds that there is an equally simple and interpretable alternative—namely quadratic mean-based fusion—which is theoretically more principled and which is more successful in practice. I argue the former from first principles and demonstrate the latter using a series of experiments on a diverse set of fusion problems: classification using synthetically generated data, computer vision-based object recognition, arrhythmia detection, and fatality prediction in motor vehicle accidents. On all of the aforementioned problems and in all instances, the proposed fusion approach exhibits superior performance over linear fusion, often increasing class separation by several orders of magnitude.

Stacked convolutional auto-encoders for surface recognition based on 3d point cloud data

This paper addresses the problem of feature extraction for 3d point cloud data using a deep-structured auto-encoder. As one of the most focused research areas in human---robot interaction (HRI), the vision-based object recognition is very important. To recognize object using the most common geometry feature, surface condition that can be obtained from 3d point cloud data could decrease the error during the HRI. In this research, the surface normal vectors are used to convert 3D point cloud data to a surface-condition-feature map, and a sub-route stacked convolution auto-encoder (sCAE) is designed to classify the difference between the surfaces. The result of the trained filters and the classification of sCAE shows the surface-condition-feature and the specified sCAE are very effective in the variation of surface condition.

Study on vision-based object recognition to improve performance of industrial manipulator

Study on vision-based object recognition to improve performance of industrial manipulator

2足ヒューマノイドロボットの研究

The KBHR (Kanagawa Biped Humanoid Robot) that is composed of 42 DOFs have been developed. This paper describes a trajectory generation method for 7-DOF manipulators of the KBHR and a robot vision-based object recognition method. The trajectory generation method is based on numerical analysis using Jacobian pseudo-inverse and transpose matrix. In order to recognize objects, an OpenCV image processing library is used. Through an experiment recognizing an object and handing over it to a person, the effectiveness of the trajectory generation algorithm and the object recognition methods is confirmed.

Computer vision-based object recognition for the visually impaired in an indoors environment: a survey

Though several electronic assistive devices have been developed for the visually impaired in the past few decades, however, relatively few solutions have been devised to aid them in recognizing generic objects in their environment, particularly indoors. Nevertheless, research in this area is gaining momentum. Among the various technologies being utilized for this purpose, computer vision based solutions are emerging as one of the most promising options mainly due to their affordability and accessibility. This paper provides an overview of the various technologies that have been developed in recent years to assist the visually impaired in recognizing generic objects in an indoors environment with a focus on approaches based on computer vision. It aims to introduce researchers to the latest trends in this area as well as to serve as a resource for developers who wish to incorporate such solutions into their own work.

Vision based Object Recognition and Localization by a Network Connected Distributed Robotic Systems

Object recognition and localisation are important processes in computer vision and robotics. Advances in computer vision have resulted in many object recognition techniques, but most of them are computationally very intensive and require robots with powerful processing systems. For small robots, these techniques are not applicable because of the constraints of execution time. In this study, an optimised implementation of SURF based recognition technique is presented. Suitable image pre-processing techniques were developed which reduced the recognition time on small robots with limited processing resources. The recognition time was reduced from 39 seconds to 780 milliseconds. This recognition technique was adopted by a team of small robots which were given prior training to search for objects of interest in the environment. For the localisation of the robots and objects a new template, designed for passive markers based tracking, was introduced. These markers were placed on the top of each robot and they were tracked by the two ceiling mounted cameras. The information from both sources, that is ceiling mounted cameras and team of robots, was used collectively to localise the objects in the environment. The objects were localised with an error ranging from 2.8cm to 5.2cm from their actual positions in the test arena which has the dimensions of 150x163cm.

Object Recognition Based on Sift Features and A Novel Feature Matching Algorithm

Vision based object recognition is the task of finding object in an image or video sequence by comparing them with an image of that object. Images can be taken from different viewpoints, different scales or even when they are translated or rotated. In this paper, a new algorithm is proposed for object recognition. First the feature points are extracted using scale invariant feature transform (SIFT), then an initial matching is established using photometrical features of the points. A novel geometric based method is used for optimizing matching points. After removing false matches, by using proposed algorithm, remaining key points can be categorized and each distinctive group is assigned to a separated object. The group with maximum likelihood is considered as desirable object. Important innovation in our method is the use of geometrical constraints of feature points between two matching images to find correct matching.

Monocular vision based 6D object localization for service robot’s intelligent grasping

Intelligent grasping is still a hard problem for home service robots. There are two major issues in the intelligent grasping, i.e. the object recognition and the pose estimation. To grasp casually placed objects, the robot needs the object’s full 6 degrees of freedom pose data. To deal with the challenges such as illumination changes, cluttered background, occlusion, etc., we propose a monocular vision based object recognition and 6D pose estimation method. The SIFT feature point matching and brute-force search algorithm is used to do a tentative object recognition. The object recognition result is then verified with the homography constraint. After passing the verification, the 6D pose estimation is obtained through the decomposition of the homography matrix and the result is refined using the Levenberg–Marquardt algorithm. We embed our pose estimation method in a tracking by detection framework to keep computing and refining the pose during the whole approaching procedure. To test our method, a robot arm of seven degrees of freedom was utilized for a group of grasping experiments. The experimental results showed that our approach successfully recognized and grasped a variety of household objects with decent accuracy.

Fast buffering for FPGA implementation of vision-based object recognition systems

Real-time frame rate is an important factor for practical deployment of computer vision systems. Field programmable gate array (FPGA) technology has been considered for many applications due to its parallel computing capability. FPGA implementations of computer vision algorithms normally involve buffering data on external memory devices, which could slow down the whole system. This paper proposes a buffering scheme suitable for implementing real-time vision-based systems on an FPGA that does not require external memory to buffer data. A stop sign detection system implemented on an FPGA employing the proposed buffering scheme is presented as an example system. This system is capable of processing over 200 fps at the frame size of 480 × 752 pixels.

주행로봇을 위한 GPU 기반의 고속 인공표식 인식

주행 로봇 환경에서 비전 기반의 물체 인식은 물체의 주변 요소와 동적인 환경에 대한 다양한 영상처리 문제를 포함한다. SURF(Speeded Up Robust Features)는 영상의 크기와 회전변화에 강인하게 물체를 인식하는 알고리즘으로 많은 연구자에 의해 사용되고 있다. 하지만 SURF 기반의 영상처리 방법은 고차원의 벡터 성분을 사용하기 때문에 연산 과정에서 많은 시간을 소비하며, 그로 인해 실시간 시스템에서 수행의 어려움을 가지고 있다. 본 연구에서는 이러한 문제점을 해결하기 위해서, 연산량이 많은 SURF 처리 과정을 GPU(Graphics Processing Unit)에서 수행하도록 하여, 보다 빠른 영상 인식을 구현하고자 한다. NVIDIA의 CUDA 라이브러리를 이용하여 GPU 상의 수행 프로그램을 구현하고, 실험을 통해 이동 로봇의 속도와 영상의 크기변화에 따른 표식의 인식률 및 수행시간에 대해서 CPU와 성능을 비교한다. Vision based object recognition in mobile robots has many issues for image analysis problems with neighboring elements in dynamic environments. SURF(Speeded Up Robust Features) is the local feature extraction method of the image and its performance is constant even if disturbances, such as lighting, scale change and rotation, exist. However, it has a difficulty of real-time processing caused by representation of high dimensional vectors. To solve th problem, execution of SURF in GPU(Graphics Processing Unit) is proposed and implemented using CUDA of NVIDIA. Comparisons of recognition rates and processing time for SURF between CPU and GPU by variation of robot velocity and image sizes is experimented.