Object Extraction Research Articles

Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking.

Bin-picking of small parcels and other textureless planar-faced objects is a common task at warehouses. A general color image–based vision-guided robot picking system requires feature extraction and goal image preparation of various objects. However, feature extraction for goal image matching is difficult for textureless objects. Further, prior preparation of huge numbers of goal images is impractical at a warehouse. In this paper, we propose a novel depth image–based vision-guided robot bin-picking system for textureless planar-faced objects. Our method uses a deep convolutional neural network (DCNN) model that is trained on 15,000 annotated depth images synthetically generated in a physics simulator to directly predict grasp points without object segmentation. Unlike previous studies that predicted grasp points for a robot suction hand with only one vacuum cup, our DCNN also predicts optimal grasp patterns for a hand with two vacuum cups (left cup on, right cup on, or both cups on). Further, we propose a surface feature descriptor to extract surface features (center position and normal) and refine the predicted grasp point position, removing the need for texture features for vision-guided robot control and sim-to-real modification for DCNN model training. Experimental results demonstrate the efficiency of our system, namely that a robot with 7 degrees of freedom can pick randomly posed textureless boxes in a cluttered environment with a 97.5% success rate at speeds exceeding 1000 pieces per hour.

Minutea Object Extraction in Fingerprint Image Using Morphological Methods and Gabor Filters

Minutiae is part of the fingerprint, which is the point where the fingerprint line stops or branches, which can be observed by scanning at a resolution of 500 pp. a fingerprint has minutiae that range from 50-100 pieces scattered throughout the surface of the fingerprint. To clarify the fingerprint can be done by extracting the minutiae contained in the fingerprint. With this extraction process, fingerprint images can be clarified, so identification of a fingerprint will be easy to do. This research extracts minutiae objects in the fingerprint image, so that the fingerprint line object can be seen clearly. The first stage in this research is object detection and edge detection using morphological methods. The next step is the extraction of minutiae objects with the gabor filter and minutiae extraction . The results obtained can display the fingerprint line of the fingerprint image clearly. From the results of testing 10 fingerprint images proved that the minutiae object in the image can be extracted, so that the fingerprint line of the image is clearer than the original image

Minutea Object Extraction in Fingerprint Image Using Morphological Methods and Gabor Filters (Ekstraksi Objek Minutea Pada Citra Sidik Jari Dengan Metode Morfologi dan Gabor Filter)

Minutiae is part of the fingerprint, which is the point where the fingerprint line stops or branches, which can be observed by scanning at a resolution of 500 pp. a fingerprint has minutiae that range from 50-100 pieces scattered throughout the surface of the fingerprint. To clarify the fingerprint can be done by extracting the minutiae contained in the fingerprint. With this extraction process, fingerprint images can be clarified, so identification of a fingerprint will be easy to do. This research extracts minutiae objects in the fingerprint image, so that the fingerprint line object can be seen clearly. The first stage in this research is object detection and edge detection using morphological methods. The next step is the extraction of minutiae objects with the gabor filter and minutiae extraction . The results obtained can display the fingerprint line of the fingerprint image clearly. From the results of testing 10 fingerprint images proved that the minutiae object in the image can be extracted, so that the fingerprint line of the image is clearer than the original image.

Deep learning for geometric and semantic tasks in photogrammetry and remote sensing

ABSTRACTDuring the last few years, artificial intelligence based on deep learning, and particularly based on convolutional neural networks, has acted as a game changer in just about all tasks related to photogrammetry and remote sensing. Results have shown partly significant improvements in many projects all across the photogrammetric processing chain from image orientation to surface reconstruction, scene classification as well as change detection, object extraction and object tracking and recognition in image sequences. This paper summarizes the foundations of deep learning for photogrammetry and remote sensing before illustrating, by way of example, different projects being carried out at the Institute of Photogrammetry and GeoInformation, Leibniz University Hannover, in this exciting and fast moving field of research and development.

Extending contour level set model for multi-class image segmentation with Application to Breast Thermography Images

Extraction of target objects with the desired intensity value in multi-region images is a challenging task. This paper presents a new modification to the region-based level set method called Extending Contour Level Set (ECLS) model. In classical level set methods, the intensity of each pixel is compared with both average intensities inside and outside the contour; therefore, image domain is segmented into two regions with the most difference between average intensities. In multi-region images, there is no reason the selected region belongs to the desired one. In our approach, with the assumption that the initial contour is located inside the target object, outside-contour pixels are allocated to the inside-contour region if they have an intensity value close to the inside-contour average intensity. The ECLS model is compared with well-known level set methods such as the Chan-Vese model, the local binary fitting energy model, the geodesic level set model, the local statistical active contour model, and the local and global fitting image model. Simulation results indicate the remarkable advantages of the proposed method in terms of accuracy and computational complexity. To show the ability of the ECLS model in practice, a two-step approach is proposed to segment breast thermography images. In the first step, the ECLS model is exploited to produce a semi-final region. Then, in the second step, a new Controlled Chan-Vese (CCV) model is presented leading to accurate results. The CCV model controls the distance between the average intensities inside and outside the contour based on the image statistics. Comparison with other segmentation methods shows the superiority of the proposed approach. In addition, statistical measures are calculated to demonstrate the significant capabilities of the proposed method.

Image Object Extraction Based on Semantic Segmentation and Label Loss

Object extraction refers to the operation of obtaining an object area from an image based on a small amount of mark information given by users, which is a key step in image processing. In order to obtain a complete object profile, current methods usually require a large number of manual annotations, especially for objects with irregular contours. Since traditional algorithms rely on low-level pixel features without semantic information, and are based on obvious mathematical assumptions (ie, strong inductive bias), it is difficult to completely identify objects. At present, in order to improve the integrity of object extraction, semantic segmentation-based methods increase the complexity and latancy by adding more pre-processing and post-processing steps. In this paper, we propose a novel model named IOEBSS, which includes a fast binary plane pre-processing, an improved Deeplab v3+ semantic segmentation model, and an auxiliary loss function named Label Loss. Through the fast binary plane pre-processing, the model can accelerate the transformation of interactive inputs. The improved semantic segmentation model makes the extracted results more semantically complete, and Label Loss is more conducive to gradient flow and accelerates training convergence. For the above reasons, IOEBSS can accurately and quickly identify objects with complex contours and colors. On Pascal VOC and COCO datasets, compared to current methods, IOEBSS has a significant improvement in accuracy, inference speed, and convergence speed.

Image Object Extraction Based on Semantic Detection and Improved K-Means Algorithm

Object extraction is an important tool in many applications within the image processing and computer vision communities. You Only Look Once version 3 (YOLOv3) has been extensively applied to many fields as a state-of-the-art technique for object semantic detection. Despite its numerous characteristics, YOLOv3 has to be combined with appropriate image segmentation technologies to achieve effective 2D object extraction in real-time monitoring, robot navigation, and target search. In this article, the K-means algorithm is applied to the segmentation of depth images. Considering the inherent sensitivity to the randomness of the initial cluster center and the uncertainty of cluster number K in the initialization phase of the K-means algorithm, this article proposes a new method that combines the semantic image information with the image depth information. Specifically, this method proposed to pre-classify the center depth of the object to determine the appropriate value of K required in the K-means algorithm. At the same time, the proposed algorithm improves the selection of the initial center via the maximin method. This article introduces a multi-parameter extraction method to enable to correctly identify the object of interest after image segmentation. The technique considers three parameters to achieve this: i) the elements of size, ii) the connected domain, and iii) the diagonal detection. Experiments using open-source datasets demonstrate that the average processing time and the segmentation accuracy of the improved K-means algorithm are 20.36% faster and 3.12% higher than the conventional K-means algorithm, respectively. The extraction accuracy of the proposed method is 6.69% higher than that of the SuperCut extraction method.

Multispectral image denoising methods: A literature review

The Multispectral images appear to give a more true representation of real-world scenes to improve the efficiency of many tasks involving computer vision and object extraction, identification and quantification, tagging operations and imagery segmentation, compared to standard RGB or grayscale images. The use of visible multi-spectral spectrum images as opposed to standard RGB systems enables high colour reproduction characteristics due to the achievement of minimal data from RGB images. Due to the restriction of facilities, restricted bandwidth and radiant energy losses, MSIs are impaired during capture by many noises. The formulation of a new mathematical definition of the deep study model for denoising is a complex investigation problem and several researchers have defined various MSI denoising algorithms or methods. Many researchers suggested its use as a sparse coding of noisy patches in the application of the neural network. These additionally allow different methods to change themselves with an algorithm for a mission. In reality, however, multiple noises often come across a multi-spectral image. We presented in this study the previous techniques for the MSI influenced by noise. The literature survey explains the description and advantages of previous approaches relative to each other.

Video Synopsis Based on Attention Mechanism and Local Transparent Processing

The increased number of video cameras makes an explosive growth in the amount of captured video, especially the increase of millions of surveillance cameras that operate 24 hours a day. Since video browsing and retrieval is time consuming, while video synopsis is one of the most effective ways for browsing and indexing such video that enables the review of hours of video in just minutes. How to generate the video synopsis and preserve the essential activities in the original video is still a costly and labor-intensive and time-intensive work. This paper proposes an approach to generating video synopsis with complete foreground and clearer trajectory of moving objects. Firstly, the one-stage CNN-based object detecting has been employed in object extraction and classification. Then, combining integrating the attention-RetinaNet with Local Transparency-Handling Collision (LTHC) algorithm is given out which results in the trajectory combination optimization and makes the trajectory of the moving object more clearly. Finally, the experiments show that the useful video information is fully retained in the result video, the detection accuracy is improved by 4.87% and the compression ratio reaches 4.94, but the reduction of detection time is not obvious.

An Optimized Point Cloud Classification and Object Extraction Method Using Graph Cuts

The requirement for 3D scene classification and understanding has dramatically increased with the widespread use of airborne Light Detection And Ranging (LiDAR). This paper focuses on precise classification and object extraction based on point cloud data in complex scenes. There are usually gross errors in the initial classification based on locally independent classifiers, due to the over- and under-clustering in the feature extraction. We introduce a graph-cut based method to improve the classification precision and eliminate errors by using contextual information. The intuition behind our algorithm is based on the fact that nearby points have a high probability of belonging to the same object, while the distances of points belonging to different objects will be large. Based on this assumption, objects of interest can be precisely extracted, and we can thus optimize the classification results. The experiments undertaken in this study proved that the classification method we propose can be effectively used for point cloud classification in complex scenes.

OEBR-GAN: Object Extraction and Background Recovery Generative Adversarial Networks

Generative adversarial networks (GAN) have been widely used in the field of image-to-image translation. In this paper, we have proposed a novel object extraction and background recovery (OEBR-GAN) model, which can extract objects from an image and then complete the image by inpainting the background of the image. This model has been developed for a solar panel installation project, where the user would like to input an original colored image of the roof, and as output, the user requires an edge detected roof image. However, the condition in user requirement is that any object that is hiding the roof edges should be removed first and the background of that part of the roof image should be recovered so that the user can obtain a complete connected edge detected image of the roof. Therefore, the model also completes the image by connecting the hidden edges of the roof. We could achieve the user objective by building a GAN model with a dual generator and dual discriminator network. The generators have been built using an encoder-decoder network with and without skip connections and the discriminators have been built using deep convolutional neural networks and encoder architecture. Quantitative comparisons in the result section shows that OEBR-GAN performs much better than other adversarial models on our collected dataset.

Geometric Primitives in LiDAR Point Clouds: A Review

To the best of our knowledge, the most recent light detection and ranging (lidar)-based surveys have been focused only on specific applications such as reconstruction and segmentation, as well as data processing techniques based on a specific platform, e.g., mobile laser. However, in this article, lidar point clouds are understood from a new and universal perspective, i.e., geometric primitives embedded in versatile objects in the physical world. In lidar point clouds, the basic unit is the point coordinate. Geometric primitives that consist of a group of discrete points may be viewed as one kind of abstraction and representation of lidar data at the entity level. We categorize geometric primitives into two classes: shape primitives, e.g., lines, surfaces, and volumetric shapes, and structure primitives, represented by skeletons and edges. In recent years, many efforts from different communities, such as photogrammetry, computer vision, and computer graphics, have been made to finalize geometric primitive detection, regularization, and in-depth applications. Interpretations of geometric primitives from multiple disciplines try to convey the significance of geometric primitives, the latest processing techniques regarding geometric primitives, and their potential possibilities in the context of lidar point clouds. To this end, primitive-based applications are reviewed with an emphasis on object extraction and reconstruction to clearly show the significances of this article. Next, we survey and compare methods for geometric primitive extraction and then review primitive regularization methods that add real-world constrains to initial primitives. Finally, we summarize the challenges, expected applications, and describe possible future for primitive extraction methods that can achieve globally optimal results efficiently, even with disorganized, uneven, noisy, incomplete, and large-scale lidar point clouds.

基于前背景图割的点云分类优化及地物提取

基于前背景图割的点云分类优化及地物提取

Hidden Markov Model-based Extraction of Target Objects in X-ray Image Sequence for Lung Radiation Therapy

Hidden Markov Model-based Extraction of Target Objects in X-ray Image Sequence for Lung Radiation Therapy

Text non-text classification based on area occupancy of equidistant pixels

Text detection and localization from text embedded natural images is still considered as a challenging problem in complex imagery environment. Foreground object segmentation followed by classification is a popular approach for this task. Component level object classification in clutter environment is therefore an important sub-problem. Appropriate extraction of foreground objects leads to effective classification that may certainly increase the performance of text detection. In this paper, a novel feature vector is developed based on area occupancy profile of equidistant pixels is reported for text/non-text classification. The generated feature descriptors are script invariant and much effective in practical scenario. This proposed feature set is evaluated on our dataset using five different pattern classifiers and experimental result shows that the said feature set yields more than 86% classification accuracy irrespective of scripts.

ANALYSIS OF DUAL-ENERGY X-RAY ABSORPTIOMETRY IMAGES USING COMPUTER VISION METHODS

PURPOSE: Dual-energy x-ray absorptiometry (DXA) is the “golden standard” for diagnosing osteoporosis. Its analyzing algorithm (software) makes it possible to distinguish the bone from the soft tissue. Until now there are only attempts to process and acquire images using automatic segmentation with convolutional neural networks (CNN). Machine reconstruction and precise specific models of anatomic structures from medical images could be accomplished using computer vision. The objective of the current work is to introduce the potential of the two computer methods and their application in the diagnostic DXA analysis. METHODS: DXA generates a report in the DICOM format which includes patient data (age, gender, height, weight, bone mineral density, T-score and Z-score) and an image of the scanned spine as well as the region of interest (ROI). The CNN methods are based mainly on intermediate analysis. The learning of the segmentation of CNN by generating segmentation labels using simple heuristic is done using computer vision. The functions of the loss and the architecture of the CNN is then determined. In that manner the right analysis of the existing medical image is made possible. RESULTS: The computer library OpenCV is the way to realize a model for the assessment of a DXA analysis. The library is available for Python programming language. The library has functions for the extraction of colour objects, image smoothing, Canny’s edge detector, Hough transform and methods for work with contours. CONCLUSIONS: The detection and extraction of images is fundamental for the analysis of DXA which is a step forward in the precision of the in-vivo diagnostic of the bone.

An Efficient Image Segmentation Approach using Superpixels with Colorization

Abstract Object-based image analysis is an important task in image processing. For the extraction of an accurate object, segmentation plays a vital role. The major problem raised with image segmentation is over segmentation, which can be corrected using superpixel based image segmentation. Superpixel generation is affected by the divergent distribution of pixel intensity values while acquiring the image. To overcome this problem, a novel colorization method is proposed as a preprocessing technique to enhance superpixel generation. Colorization transforms image into unique intensity values with particular color. Quantitative evaluation of experimental results yields promising accuracy, for the proposed approach.

Depth extraction in computational integral imaging based on bilinear interpolation

We proposed a method using a merit function to determine the depth of objects in computational integral imaging by analyzing the existing methods for depth extraction of target objects. To improve the resolution of reconstructed slice images, we use a digital camera moving in horizontal and vertical direction with the set interval to get elemental images with high resolution and bilinear interpolation algorithm to increase the number of pixels in slice image which improves the resolution obviously. To show the feasibility of the proposed method, we carried out our experiment and presented the results. We also compared it with other merit functions. The results show that merit function SMD2 to determine the depth of objects is more accurate and suitable for real-time application.

Martial Arts Routine Difficulty Action Technology VR Image Target Real-Time Extraction Simulation

With the gradual increase in the difficulty of competitive martial arts, athletes must complete fine, stable, high-quality and difficult movements in order to achieve excellent performance. The real-time extraction of martial arts movements is a topic that many martial arts enthusiasts care about. This study mainly discusses the real-time extraction and simulation of VR image target in the difficult movement technology of martial arts routine. Considering the complex characteristics of martial arts movements, this article will analyze the preprocessing content of existing images. This includes image enhancement and image filtering, and uses median filtering methods to enhance the characteristics of the collected images. In this way, the visual effect of the original image can be improved, and the processed image will contribute to the subsequent segmentation. A new image segmentation method is proposed for the color model of the image. According to the H component of the HSV model representing the characteristics of chromaticity, the color image is transformed into the HSV model, and the H component is extracted. The histogram concept applies to H components. Based on the histogram of the H component, the segmentation threshold is determined, and the cropping target in the image is detected. Because the model space is very sensitive to color, VR technology is used to automatically determine the segmentation target. Combined with the above division methods, the automatic extraction of objects in the image is completed. The method of using VR technology for image extraction processing has high precision, and the error value is 3.92%<5%. The research results show that the method has good segmentation results, and is suitable for image segmentation under complex background and automatic image extraction under complex background.

Position and Posture Estimation of Primitive Shapes Using Deep Learning-Based Object Extraction and Point Cloud-Based Face Extraction

Position and Posture Estimation of Primitive Shapes Using Deep Learning-Based Object Extraction and Point Cloud-Based Face Extraction