Object Feature Extraction Research Articles

Small Object Detection via Pixel Level Balancing With Applications to Blood Cell Detection.

Object detection technology has been widely used in medical field, such as detecting the images of blood cell to count the changes and distribution for assisting the diagnosis of diseases. However, detecting small objects is one of the most challenging and important problems especially in medical scenarios. Most of the objects in medical images are very small but influential. Improving the detection performance of small objects is a very meaningful topic for medical detection. Current researches mainly focus on the extraction of small object features and data augmentation for small object samples, all of these researches focus on extracting the feature space of small objects better. However, in the training process of a detection model, objects of different sizes are mixed together, which may interfere with each other and affect the performance of small object detection. In this paper, we propose a method called pixel level balancing (PLB), which takes into account the number of pixels contained in the detection box as an impact factor to characterize the size of the inspected objects, and uses this as an impact factor. The training loss of each object of different size is adjusted by a weight dynamically, so as to improve the accuracy of small object detection. Finally, through experiments, we demonstrate that the size of objects in object detection interfere with each other. So that we can improve the accuracy of small object detection through PLB operation. This method can perform well with blood cell detection in our experiments.

Texture-Less Shiny Objects Grasping in a Single RGB Image Using Synthetic Training Data

In the industrial domain, estimating the pose of texture-less shiny parts is challenging but worthwhile. In this study, it is impractical to utilize texture information to obtain the pose because the features are likely to be affected by the surrounding objects. In addition, the colors of the metal parts are similar, making object segmentation challenging. This study proposes dividing the entire process into three steps: object detection, feature extraction, and pose estimation. We use the Mask-RCNN to detect objects and HRNet to extract the corresponding features. For metal parts of different shapes, different keypoints were chosen accordingly. Conventional contour-based methods are inapplicable to parts containing planar surfaces because the objects occlude each other in clustered environments. In this case, we used dense discrete points along the edges as semantic keypoints for metal parts containing planar elements. We chose skeleton points as semantic keypoints for parts containing cylindrical components. Subsequently, we combined the localization of semantic keypoints and the corresponding CAD model information to estimate the 6D pose of an individual object in sight. The implementation of deep learning approaches requires massive training datasets and intensive labeling. Thus, we propose a method to generate training datasets and automatically label them. Experiments show that the algorithm based on synthetic data performs well in a natural environment, despite not utilizing real scenario images for training.

AUTO-ASSISTANCE SYSTEM FOR VISUALLY IMPAIRED PEOPLE USING DEEP LEARNING

World Health Organization (WHO) outlined that there are two eighty-five million visually disabled worldwide. Among them thirty-nine million people are completely blind. One of the difficult activities that could be conducted by visually impaired is object detection which could be implemented using Machine Learning. It is an approach towards Artificial Intelligence that provides system the capacity for natural learning and development from experience without specifically programmed. It provides computer vision to the system which make decisions based on training algorithms. The chief goal of this research paper is to develop an object detection system to assist totally blind individual to manage their activities independently. Paper also compares different object detection algorithms like Haar Cascade and Convolutional Neural Network with yolov5(CNN). Haar Cascade classifier is a basic face detection algorithm which could also be trained to detect different objects whereas convolutional neural network falls under deep learning approach which could be employed for object recognition. The custom dataset is created with 2300 images consisting of 3 different classes. This comparison is being executed to find the yolov5 as a suitable algorithm for this system from the aspect of accuracy for real time scenario. Keywords- Object Detection, Computer Vision, Deep learning, Feature Extraction and Recognition , Convolution neural network.

Assessment of the levels of damage caused by Fusarium head blight in wheat using an improved YoloV5 method

Object segmentation in deep learning has been recently used for the detection of Fusarium head blight (FHB), a worldwide disease in wheat. Such method, however, cannot detect the disease with high accuracy and is difficult to be used in labelling annotation. However, object detection network can solve the above problem. The object detection network has high detection accuracy and easy for labeling. Yolov5 is an advanced object detection network, but it can’t detect the neighboring wheat ears well. So in this study, a novel method was developed based on object detection network, feature extraction and classifier to overcome these disadvantages. We combined Yolov5 object detection network with distance intersection over union non maximum suppression (DIOU-NMS) to form an improved Yolov5 object detection network. The improved YoloV5 object detection network was employed to detect and record wheat ears in images collected from field plots at two locations over 2 years. Pre-segmentation was conducted for single individual wheat ear images using threshold segmentation; HSV and CMYK color spaces were used as the baseline in each wheat ear image for extracting comprehensive color feature (CCF). The Res-Net network was used for extracting each wheat ear’s high dimension feature (HDF). CCF and HDF were then merged as the comprehensive feature (CF) of each single wheat image. The random forest was used to classify wheat ear images into healthy wheat ears and diseased wheat ears by CF and then calculate the ratio of diseased wheat ears to total wheat ears as the level of damage caused by FHB. The results of performance evaluation of the proposed method in two different locations and years demonstrate its strong robustness in both time and spatial domains to effectively detect the levels of damage caused by FHB under the complex field background conditions. The average detection accuracy and detection time were 90.67% and 0.73 ms, respectively. The average accuracies of counting total wheat ears and diseased wheat ears were 96.16% and 81.66%, respectively. The improved YoloV5 method developed from this study can be used as a quick, efficient, and convenient tool for assessment of the levels of damage caused by FHB in wheat under field conditions.

A novel deep learning‐based single shot multibox detector model for object detection in optical remote sensing images

AbstractRemote sensing image object detection is widely used in civil and military fields. The important task is to detect objects such as ships, planes, airports, harbours and so on, and then it can obtain object category and position information. It is of great significance to use remote sensing images to observe the densely arranged and directional targets such as cars and ships parked in parking lots and harbours. The object detection task mainly includes object localization and classification. Remote sensing images contain large number of small objects and dense scenes due to the long shooting distance and wide coverage. Small objects occupy few pixels in the image, and they are easily miss‐detected. In dense scenes, the overlapping part of each object is large, so it is easy to detect objects repeatedly. The traditional small object detection methods deliver low accuracy and take long time. Therefore, object detection is very challenging. We put forward a novel deep learning‐based single shot multibox detector (SSD) model for object detection. First, we propose an improved inception network to optimize SSD to strengthen the small object feature extraction ability (FEA) in the shallow network. Second, the feature pyramid network is modified to enhance the fusion effect. Third, the deep feature fusion module is designed to improve the FEA of the deep network. Finally, the extracted image features are matched with candidate boxes with different aspect ratios to perform object detection and location with different scales. Experiments on DOTA show that the proposed algorithm can adapt to the remote sensing object detection in different backgrounds, and effectively improve the detection effect of remote sensing objects in complex scenes.

Tiny Vehicle Detection for Mid-to-High Altitude UAV Images Based on Visual Attention and Spatial-Temporal Information.

Mid-to-high altitude Unmanned Aerial Vehicle (UAV) imagery can provide important remote sensing information between satellite and low altitude platforms, and vehicle detection in mid-to-high altitude UAV images plays a crucial role in land monitoring and disaster relief. However, the high background complexity of images and limited pixels of objects challenge the performance of tiny vehicle detection. Traditional methods suffer from poor adaptation ability to complex backgrounds, while deep neural networks (DNNs) have inherent defects in feature extraction of tiny objects with finite pixels. To address the issue above, this paper puts forward a vehicle detection method combining the DNNs-based and traditional methods for mid-to-high altitude UAV images. We first employ the deep segmentation network to exploit the co-occurrence of the road and vehicles, then detect tiny vehicles based on visual attention mechanism with spatial-temporal constraint information. Experimental results show that the proposed method achieves effective detection of tiny vehicles in complex backgrounds. In addition, ablation experiments are performed to inspect the effectiveness of each component, and comparative experiments on tinier objects are carried out to prove the superior generalization performance of our method in detecting vehicles with a limited size of 5 × 5 pixels or less.

Object Pose Estimation and Feature Extraction Based on PVNet

In traditional industrial fields, a robot arm is usually used for high-precision or highly repetitive movements, but now, with the development of three-dimensional (3D) stereo machine vision in smart manufacturing, the smart factory has moved toward the development of a robot arm combined with the image recognition technology. Currently, in the manufacturing industry, most of the images for computing are obtained using two-dimensional (2D) machine vision; here, the 2D advantage is that the camera lens can obtain the simulation of the plane color pixel, but the disadvantage is that it cannot obtain the real space depth distance information, resulting in a more accurate analysis of the workpiece position and features. Therefore, in this study, a pixel-wise voting network (PVNet)-based object pose estimation and feature extraction was developed to perform more diverse object testing for the considered network model. Unlike other workpiece picking systems for smart manufacturing, most of the systems today still framed the workpiece in two dimensions only, but the approach proposed in this paper framed the workpiece pose in three dimensions. Thus, the network could successfully predict the pose even when the workpiece was obscured or the image was not fully captured. The images were input into the neural network by means of supervised learning, and training was performed using transformation matrices between multi-angle images of the artifacts and feature points extracted from the 3D models. The results of this study revealed the pose estimation results of various objects at different viewing angles and proposed the feature gripping strategy for the robot arm to follow this process in the future.

Small object detection in remote sensing images based on super-resolution

Accurate objects detection in remote sensing images is very important, because security, transportation, and rescue applications in military and civilian fields require fully analyzing and using these images. To address the problem that many small-sized objects in remote sensing images are difficult to detect, this paper proposes an improved S2ANET-SR model based on S2A-NET network. In this paper, the original and reduced image are fed to the detection network at the same time, and then a super-resolution enhancement module for the reduced image is designed to enhance the feature extraction of small objects, after that, the perceptual loss and texture matching loss is proposed as supervision. Extensional experiments are conducted to evaluate the performance on the general remote sensing dataset DOTA, and the results show that our proposed method can achieve 74.47% mAP, which is 0.79% better than the accuracy of S2A-NET.

Infrared Small Object Tracking Based on Att-Siam Network

Aiming at the practical problems such as background clutter interference and occlusion during the tracking of small air targets by infrared imaging guided missiles, we propose an improved SiamFC infrared small target tracking algorithm. By mapping the original spatial data to a high-dimensional Hilbert space, the nonlinear mapping is implicit in the linear learner. This method has the advantages of learnability, efficient calculation, linearizability, and good generalization performance. The target tracking problem provides a new and effective approach with the actual needs of the application. We propose a Siamese web tracker (Att-Siam). Att-Siam fuses convolutional channel attention mechanism, stacked channel attention mechanism and spatial attention mechanism to improve tracking performance. Feature extraction of target objects is enhanced by fusing the channel attention mechanism and two convolution blocks at low convolutional layers. Compared with the benchmark algorithm SiamFC, the improved algorithm is tested on the infrared air weak and small target data set, and the tracking success rate and accuracy are increased by 32.3% and 20.9% respectively. The experimental results show that the proposed algorithm can adapt to complex and diverse infrared air scenes, and achieve effective and stable real-time tracking of small infrared air targets.

Enhancement of light field disparity maps by reducing the silhouette effect and plane noise

During the last decade, there has been an increasing number of applications dealing with multidimensional visual information, either for 3D object representation or feature extraction purposes. In this context, recent advances in light field technology, have been driving research efforts in disparity estimation methods. Among the existing ones, those based on the structure tensor have emerged as very promising to estimate disparity maps from Epipolar Plane Images. However, this approach is known to have two intrinsic limitations: (i) silhouette enlargement and (ii) irregularity of surface normal maps as computed from the estimated disparity. To address these problems, this work proposes a new method for improving disparity maps obtained from the structure-tensor approach by enhancing the silhouette and reducing the noise of planar surfaces in light fields. An edge-based approach is initially used for silhouette improvement through refinement of the estimated disparity values around object edges. Then, a plane detection algorithm, based on a seed growth strategy, is used to estimate planar regions, which in turn are used to guide correction of erroneous disparity values detected in object boundaries. The proposed algorithm shows an average improvement of 98.3% in terms of median angle error for plane surfaces, when compared to regular structure-tensor-based methods, outperforming state-of-the-art methods. The proposed framework also presents very competitive results, in terms of mean square error between disparity maps and their ground truth, when compared with their counterparts.

Real-Time Multiple Pedestrian Tracking With Joint Detection and Embedding Deep Learning Model for Embedded Systems

This paper proposes an improvement to the multi-object tracking system framework based on the image inputs. By analyzing the role and performance of each block in the original multi-objects tracking system, the blocks of the original system are reconstructed to enhance the efficiency and yield a faster processing speed suiting the real-time applications. In the proposed method, the first two parts of the multi-object tracking system are merged into a single neural network designed for object detection and feature extraction. A new object association judgment method and JDE inspired prediction head are included in order to achieve a better and an outstanding association effect resulting in the overall improvement of the original system by 45.2%. The enhanced method is aimed at the application of smart roadside units and uses fixed-viewpoint image input to achieve multi-object tracking on embedded platforms. The proposed method is implemented on the NVIDIA Jetson AGX Xavier embedded platform. The NVIDIA TensorRT software development kit is used to accelerate the neural network. The overall performance of the proposed system yields better efficiency compared to that of the original SDE design and the overall computing performance achieve up to 14-26 images per second, making it ideal for the real-time smart roadside unit applications.

A Confounder-Free Fusion Network for Aerial Image Scene Feature Representation

The increasing number and complex content of aerial images have made some recent methods based on deep learning not fit well with different aerial image processing tasks. The coarse-grained feature representation proposed by these methods is not discriminative enough. Besides, the confounding factors in the datasets and long-tailed distribution of the training data will lead to biased and spurious associations among the objects of aerial images. This study proposes a confounder-free fusion network (CFF-NET) to address the challenges. Global and local feature extraction branches are designed to capture comprehensive and fine-grained deep features from the whole image. Specifically, to extract the discriminative local feature and explore the contextual information across different regions, the models based on gated recurrent units (GRUs) are constructed to extract features of the image region and output the important weight of each region. Further, the confounder-free object feature extraction branch is proposed to generate reasonable visual attention and provide more multi-grained image information. It also eliminates the spurious and biased visual relationships of the image on the object-level. Finally, the output of the three branches is combined to obtain the fusion feature representation. Extensive experiments are conducted on the three popular aerial image processing tasks: image classification, image retrieval, and image captioning. It is found that the proposed CFF-NET achieve reasonable and state-of-the-art results, including high-level task such as aerial image captioning.

Image-Based Visual Servoing System for Components Alignment Using Point and Line Features

The design of visual servoing system for robotic high-precision assembly is of great challenge. For the difficulty of insufficient accuracy of target object feature extraction, a deep neural network combined with feature pyramid network (FPN) structure is proposed. This lightweight network requires only a small amount of labeled data to achieve significant segmentation results. The control laws of translation and orientation for component alignment are separately designed. The translation is controlled based on the interaction matrix of point features. The orientation is controlled based on the interaction matrix of line features. The relations between the cameras’ motion and the end-effector of the manipulator are calibrated via the manipulator’s active movements, which are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3 \times $ </tex-math></inline-formula> 3 transformation matrices. The depth information of feature points is integrated into the transformation matrix. The alignment pose error estimation is realized with the interaction matrices, transformation matrices, and point and line features. A robotic assembly system is developed to assemble two aviation circular connectors with six degree-of-freedoms (DOFs) in high precision in 3-D space. The experimental results verify the effectiveness of the proposed method.

Cusp Pixel Labelling Model for Objects Outline Using R-CNN

In order to accomplish machine visualization, object detection algorithms strive to recognise all target items in the image and derive the categories and object location information. Numerous ways to solve this problem have been presented where most of them are inspired by computer vision and deep learning methodologies. Existing techniques, on the other hand, consistently fail to recognise small, dense objects and even fail to detect objects with random geometric modifications. Object skeletons can help in object representation and detection. Skeleton is a natural object’s inherent visual description that contains comprehensive shape semantics. They supplement the object outline by providing additional information, such as how the object scale in thickness that varies between object components. However, extracting object skeletons from natural photos is difficult because the extractor must be able to capture both local and non-local visual contexts in order to calculate the scale of each skeleton pixel. To solve this issue, a Cusp Pixel Labelled Model with Precise Tuned Outline using Machine Learning (CPLM-PTOML) is proposed in this paper that accurately detects the cusp points of the object in the image by extracting the skeleton of the object to recognize the exact object resided in the image. A multi scale associated outputs to each stage of the image by monitoring the relationship between the receptive field sizes of the different layers in the image and the skeleton scales they can capture. The model is trained via multi-task learning, with one job being skeleton localization, which determines whether or not a pixel is a skeleton pixel, and the other being skeleton scale prediction, which predicts the scale of each skeleton pixel. At various phases, supervision is imposed by directing the scale-associated side outputs toward the ground-truth skeletons at the appropriate scales. The proposed model is compared with the traditional model in terms Data Training Samples, Object Feature Extraction Time Levels, Feature Extraction Accuracy, Cusp Pixel Labelling Time Levels, Cusp Pixel Identification Accuracy, Cusp Point Linking Accuracy, Image Considered and Cusp Point Recognition levels and the results exhibit that the proposed model exhibits better outcome.

An efficient deep learning‐based video captioning framework using multi‐modal features

AbstractVisual understanding has become more significant in gathering information in many real‐life applications. For a human, it is a trivial task to understand the content in a visual, however the same is a challenging task for a machine. Generating captions for images and videos for better understanding the situation is gaining more importance as they have wide application in assistive technologies, automatic video captioning, video summarizing, subtitling, blind navigation, and so on. The visual understanding framework will analyse the content present in the video to generate semantically accurate caption for the visual. Apart from the visual understanding of the situation, the gained semantics must be represented in a natural language like English, for which we require a language model. Hence, the semantics and grammar of the sentences being generated in English is yet another challenge. The captured description of the video is supposed to collect information of not just the objects contained in the scene, but it should also express how these objects are related to each other through the activity described in the scene, thus making the entire process a complex task for a machine. This work is an attempt to peep into the various methods for video captioning using deep learning methodologies, datasets that are widely used for these tasks and various evaluation metrics that are used for the performance comparison. The insights that we gained from our premiere work and the extensive literature review made us capable of proposing a practical, efficient video captioning architecture using deep learning which that will utilize the audio clues, external knowledge and attention context to improve the captioning process. Quantum deep learning architectures can bring about extraordinary results in object recognition tasks and feature extraction using convolutions.

Object Detection in Autonomous Driving Scenarios Based on an Improved Faster-RCNN

Object detection is one of the key algorithms in automatic driving systems. Aiming at addressing the problem of false detection and the missed detection of both small and occluded objects in automatic driving scenarios, an improved Faster-RCNN object detection algorithm is proposed. First, deformable convolution and a spatial attention mechanism are used to improve the ResNet-50 backbone network to enhance the feature extraction of small objects; then, an improved feature pyramid structure is introduced to reduce the loss of features in the fusion process. Three cascade detectors are introduced to solve the problem of IOU (Intersection-Over-Union) threshold mismatch, and side-aware boundary localization is applied for frame regression. Finally, Soft-NMS (Soft Non-maximum Suppression) is used to remove bounding boxes to obtain the best results. The experimental results show that the improved Faster-RCNN can better detect small objects and occluded objects, and its accuracy is 7.7% and 4.1% respectively higher than that of the baseline in the eight categories selected from the COCO2017 and BDD100k data sets.

Survey of Machine Learning Applications of Convolutional Neural Networks to Medical Image Analysis

Abstract: Recently, a machine learning (ML) area called deep learning emerged in the computer-vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in many fields, including medical image analysis, have started actively participating in the explosively growing field of deep learning. In this paper, deep learning techniques and their applications to medical image analysis are surveyed. This survey overviewed 1) standard ML techniques in the computer-vision field, 2) what has changed in ML before and after the introduction of deep learning, 3) ML models in deep learning, and 4) applications of deep learning to medical image analysis. The comparisons between MLs before and after deep learning revealed that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is learning image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The survey of deep learningalso revealed that there is a long history of deep-learning techniques in the class of ML with image input, except a new term, “deep learning”. “Deep learning” even before the term existed, namely, the class of ML with image input was applied to various problems in medical image analysis including classification between lesions and nonlesions, classification between lesion types, segmentation of lesions or organs, and detection of lesions. ML with image input including deep learning is a verypowerful, versatile technology with higher performance, which can bring the current state-ofthe-art performance level of medical image analysis to the next level, and it is expected that deep learning will be the mainstream technology in medical image analysis in the next few decades. “Deep learning”, or ML with image input, in medical image analysis is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical image analysis in the next few decades. Keywords: Deep learning, Convolutional neural network, Massive-training artificial neural network, Computer-aided diagnosis, Medical image analysis, Classification (key words)

Deep Neural Networks to Detect Weeds from Crops in Agricultural Environments in Real-Time: A Review

Automation, including machine learning technologies, are becoming increasingly crucial in agriculture to increase productivity. Machine vision is one of the most popular parts of machine learning and has been widely used where advanced automation and control have been required. The trend has shifted from classical image processing and machine learning techniques to modern artificial intelligence (AI) and deep learning (DL) methods. Based on large training datasets and pre-trained models, DL-based methods have proven to be more accurate than previous traditional techniques. Machine vision has wide applications in agriculture, including the detection of weeds and pests in crops. Variation in lighting conditions, failures to transfer learning, and object occlusion constitute key challenges in this domain. Recently, DL has gained much attention due to its advantages in object detection, classification, and feature extraction. DL algorithms can automatically extract information from large amounts of data used to model complex problems and is, therefore, suitable for detecting and classifying weeds and crops. We present a systematic review of AI-based systems to detect weeds, emphasizing recent trends in DL. Various DL methods are discussed to clarify their overall potential, usefulness, and performance. This study indicates that several limitations obstruct the widespread adoption of AI/DL in commercial applications. Recommendations for overcoming these challenges are summarized.

A Comprehensive Review of Image Segmentation Techniques

Image segmentation is a wide research topic; a huge amount of research has been performed in this context. Image segmentation is a crucial procedure for most object detection, image recognition, feature extraction, and classification tasks depend on the quality of the segmentation process. Image segmentation is the dividing of a specific image into a numeral of homogeneous segments; therefore, the representation of an image into simple and easy forms increases the effectiveness of pattern recognition. The effectiveness of approaches varies according to the conditions of objects arrangement, lighting, shadow, and other factors. However, there is no generic approach for successfully segmenting all images, where some approaches have been proven to be more effective than others. The major goal of this study is to provide summarize of the disadvantages and the advantages of each of the reviewed approaches of image segmentation.

A new MFL imaging and quantitative nondestructive evaluation method in wire rope defect detection

As a reliable and flexible loading tool, wire rope has been widely applied in various of industries and practical applications. Nondestructive testing and evaluation for wire rope defect inspection have become necessary methods in safety guaranteeing. However, traditional one-dimensional (1D) and two-dimensional (2D) magnetic flux leakage (MFL) signal processing methods for wire rope nondestructive evaluation (NDE) may lead to miss and false detection, especially under complex operation conditions with strong noise interference. Thus, a new MFL imaging and quantitative defect recognition method is proposed. Based on the reshaped sine function, wavelet transformation and grid entropy matrix reconstruction, five different MFL imaging algorithms are presented and compared. Besides, three MFL image processing methods through the Gabor filtering, morphological filtering, and small object feature extraction as well as the quantitative centroid distance calculation are also investigated. Finally, the quantitative evaluation model for wire rope defect inspection is built and tested in the case study and performance comparison, where six groups of wire rope testing signal recognition results not only verify the feasibility and validity of the proposed quantitative methods, but also demonstrate its great application promise in wire rope defect evaluation under sophisticated conditions. Additionally, advantages and disadvantages of the proposed method as well as the future wok are discussed.