Generate Candidate Regions Research Articles

Object Detection Algorithms Based on Convolutional Neural Networks

Object detection is a fundamental and challenging task in computer vision, and it has attracted much attention from researchers worldwide. In recent years, deep learning technology has made remarkable progress and enabled new possibilities for object detection. Convolutional neural networks (CNNs), which are powerful tools for feature extraction and representation learning, have become the dominant approach for object detection, surpassing the traditional methods. This article reviews the development history of CNNs and their applications to object detection. It also introduces and compares two main branches of CNN-based object detection algorithms: region-based methods, which use a two-stage pipeline to first generate candidate regions and then classify them, and regression-based methods, which directly predict the bounding boxes and labels of objects in a single stage. Finally, it summarizes the current state-of-the-art and discusses the future directions of object detection research.

TD-Net: Trans-Deformer network for automatic pancreas segmentation

Accurate and efficient pancreas segmentation is the basis for subsequent diagnosis and qualitative treatment of pancreatic cancer. Segmenting the pancreas from abdominal CT images is a challenging task because the morphology of the pancreas varies greatly among different individuals and may be affected by problems such as the unbalanced category and blurred boundaries. This paper proposes a two-stage Trans-Deformer network to solve these problems of pancreas segmentation. To be specific, we first use 2D Unet for coarse segmentation to generate candidate regions of the pancreas. In the fine segmentation stage, we propose to integrate deformable convolution into Vision Transformer (VIT) for solving the deformation problem of the pancreas. For the problem of blurred boundaries caused by low contrast in the pancreas, a multi-input module based on wavelet decomposition is proposed to make our network pay more attention to high-frequency texture information. In addition, we propose using the Scale Inter-active Fusion (SIF) module to merge local features and global features. Our method was evaluated on the public NIH dataset including 82 abdominal contrast-enhanced CT volumes and the public MSD dataset including 281 abdominal contrast-enhanced CT volumes via fourfold cross-validation. We have achieved the average Dice Similarity Coefficient (DSC) values of 89.89 ± 1.82 % on the NIH dataset, and 91.22 ± 1.37 % on the MSD dataset, outperforming other exiting state-of-the-art pancreas segmentation methods.

Fabric defect detection based on transfer learning and improved Faster R-CNN

Fabric defect detection is an important quality inspection process in the textile industry. A fabric defect detection system based on transfer learning and an improved Faster R-CNN is proposed to solve the problems of low detection accuracy, general convergence ability, and poor detection effect for small target defects in existing fabric defect detection algorithms. The pre-trained weights on the big dataset Imagenet are first extracted for transfer learning. Images are then input into the improved Faster R-CNN network, while the ResNet50 and ROI Align are used to replace the original VGG16 feature extraction network structure and a region of interest (ROI) pooling layer to avoid the problems of region mismatch caused by two quantizations from ROI pooling. The region proposal network (RPN) is combined with the multi-scale feature pyramid FPN to generate candidate regions with richer semantic information and project them onto the feature map to obtain the corresponding feature matrix. Cascaded modules are integrated and different IoU thresholds are used for each level to distinguish positive and negative samples. Finally, the softmax classifier is adopted to identify the image and obtain the predictions. The experimental results show that the detection accuracy and convergence ability of the improved Faster R-CNN are greatly enhanced compared with the current mainstream models, which provides a reference for future fabric defect detection methods.

Seeding Crop Detection Framework Using Prototypical Network Method in UAV Images

With the development of unmanned aerial vehicle (UAV), obtaining high-resolution aerial images has become easier. Identifying and locating specific crops from aerial images is a valuable task. The location and quantity of crops are important for agricultural insurance businesses. In this paper, the problem of locating chili seedling crops in large-field UAV images is processed. Two problems are encountered in the location process: a small number of samples and objects in UAV images are similar on a small scale, which increases the location difficulty. A detection framework based on a prototypical network to detect crops in UAV aerial images is proposed. In particular, a method of subcategory slicing is applied to solve the problem, in which objects in aerial images have similarities at a smaller scale. The detection framework is divided into two parts: training and detection. In the training process, crop images are sliced into subcategories, and then these subcategory patch images and background category images are used to train the prototype network. In the detection process, a simple linear iterative clustering superpixel segmentation method is used to generate candidate regions in the UAV image. The location method uses a prototypical network to recognize nine patch images extracted simultaneously. To train and evaluate the proposed method, we construct an evaluation dataset by collecting the images of chilies in a seedling stage by an UAV. We achieve a location accuracy of 96.46%. This study proposes a seedling crop detection framework based on few-shot learning that does not require the use of labeled boxes. It reduces the workload of manual annotation and meets the location needs of seedling crops.

Automatic recognition of tunnel lining elements from GPR images using deep convolutional networks with data augmentation

Tunnel lining inspection using ground penetrating radar (GPR) is a routine procedure to ensure construction quality. Yet, the interpretation of GPR data relies heavily on manual experience that may lead to low efficiency and recognition error when a large volume of data is involved. We introduced a deep learning-based automatic recognition method to identify tunnel lining elements, including steel ribs, voids, and initial linings from GPR images. Based on the mask region-based convolutional neural network (Mask R-CNN), this approach uses the 101-layer deep residual network (ResNet101) with the feature pyramid network (FPN) to extract features, the region proposal network (RPN) to generate candidate regions, a group of fully connected layers to detect the presence and locations of steel ribs and voids, and a fully convolutional network (FCN) to segment the area of the initial lining. To improve the recognition performance of the network, the finite-difference time-domain (FDTD) method and deep convolutional generative adversarial network (DCGAN) are employed to create synthetic GPR images for data augmentation. The test results on a synthetic example show that the mean absolute errors for steel rib, void, and initial lining thickness recognition are 1.2, 2.2, and 4.2mm, respectively, demonstrating the feasibility of the recognition network. In a field GPR survey experiment, the recognition accuracies achieved 96.02%, 91.17%, and 95.45% for the three targets. With the optimal proportions of synthetic images added to the training dataset, the accuracies were further improved to 98.86%, 94.53%, and 99.27%, respectively.

A real-time approach of diagnosing rice leaf disease using deep learning-based faster R-CNN framework.

The rice leaves related diseases often pose threats to the sustainable production of rice affecting many farmers around the world. Early diagnosis and appropriate remedy of the rice leaf infection is crucial in facilitating healthy growth of the rice plants to ensure adequate supply and food security to the rapidly increasing population. Therefore, machine-driven disease diagnosis systems could mitigate the limitations of the conventional methods for leaf disease diagnosis techniques that is often time-consuming, inaccurate, and expensive. Nowadays, computer-assisted rice leaf disease diagnosis systems are becoming very popular. However, several limitations ranging from strong image backgrounds, vague symptoms’ edge, dissimilarity in the image capturing weather, lack of real field rice leaf image data, variation in symptoms from the same infection, multiple infections producing similar symptoms, and lack of efficient real-time system mar the efficacy of the system and its usage. To mitigate the aforesaid problems, a faster region-based convolutional neural network (Faster R-CNN) was employed for the real-time detection of rice leaf diseases in the present research. The Faster R-CNN algorithm introduces advanced RPN architecture that addresses the object location very precisely to generate candidate regions. The robustness of the Faster R-CNN model is enhanced by training the model with publicly available online and own real-field rice leaf datasets. The proposed deep-learning-based approach was observed to be effective in the automatic diagnosis of three discriminative rice leaf diseases including rice blast, brown spot, and hispa with an accuracy of 98.09%, 98.85%, and 99.17% respectively. Moreover, the model was able to identify a healthy rice leaf with an accuracy of 99.25%. The results obtained herein demonstrated that the Faster R-CNN model offers a high-performing rice leaf infection identification system that could diagnose the most common rice diseases more precisely in real-time.

Predictive Analysis of Class Attention Based on CNN Model

As the current classroom teaching reform has been integrated into every class of higher vocational colleges, modern artificial intelligence, deep learning, big data and other high-end technologies are integrated into the classroom, and classroom teaching has become more and more diverse. But the attention of modern vocational and technical students to the classroom has also received widespread attention from the society. This article uses artificial intelligence and big data technology to analyze class attention, and then uses the CNN model to predict the overall class attention. This article discusses the core literacy strategies for cultivating the efficiency of vocational and technical students in the classroom, and proposes methods such as setting suspense to guide independent inquiry, organizing learning to discover learning rules, and in-depth exploration of students’ independent learning ability, so as to increase the attention of the classroom. Based on the current popular CNN target detection algorithms can be divided into two categories. The first category is a two-stage detection algorithm, which divides the detection problem into two stages. First, generate candidate regions, and then classify the candidate regions. The typical representative of these algorithms is the R-CNN series algorithm based on region proposal. The other type is the 1-stage detection algorithm, which makes predictions for every part of the image, that is, it does not need to generate candidate regions. On this basis, a target prediction system based on the CNN model is designed and implemented, which predicts and analyzes the class attention of higher vocational colleges, uses a deep learning framework, and uses PyQt5 as an interface development framework combined with python. Experimental research results show that the designed CNN-based classroom attention model in higher vocational colleges can effectively predict and analyze students, and the effect of the practical experiment is good.

Structured Object-Level Relational Reasoning CNN-Based Target Detection Algorithm in a Remote Sensing Image

Deep learning technology has been extensively explored by existing methods to improve the performance of target detection in remote sensing images, due to its powerful feature extraction and representation abilities. However, these methods usually focus on the interior features of the target, but ignore the exterior semantic information around the target, especially the object-level relationship. Consequently, these methods fail to detect and recognize targets in the complex background where multiple objects crowd together. To handle this problem, a diversified context information fusion framework based on convolutional neural network (DCIFF-CNN) is proposed in this paper, which employs the structured object-level relationship to improve the target detection and recognition in complex backgrounds. The DCIFF-CNN is composed of two successive sub-networks, i.e., a multi-scale local context region proposal network (MLC-RPN) and an object-level relationship context target detection network (ORC-TDN). The MLC-RPN relies on the fine-grained details of objects to generate candidate regions in the remote sensing image. Then, the ORC-TDN utilizes the spatial context information of objects to detect and recognize targets by integrating an attentional message integrated module (AMIM) and an object relational structured graph (ORSG). The AMIM is integrated into the feed-forward CNN to highlight the useful object-level context information, while the ORSG builds the relations between a set of objects by processing their appearance features and geometric features. Finally, the target detection method based on DCIFF-CNN effectively represents the interior and exterior information of the target by exploiting both the multiscale local context information and the object-level relationships. Extensive experiments are conducted, and experimental results demonstrate that the proposed DCIFF-CNN method improves the target detection and recognition accuracy in complex backgrounds, showing superiority to other state-of-the-art methods.

Deep learning framework based on integration of S-Mask R-CNN and Inception-v3 for ultrasound image-aided diagnosis of prostate cancer

The computer-aided diagnosis of prostate ultrasound images can aid in the detection and treatment of prostate cancer. However, the ultrasound images of the prostate sometimes come with serious speckle noise, low signal-to-noise ratio, and poor detection accuracy. To overcome this shortcoming, we proposed a deep learning model that integrates S-Mask R-CNN and Inception-v3 in the ultrasound image-aided diagnosis of prostate cancer in this paper. The improved S-Mask R-CNN was used to realize the accurate segmentation of prostate ultrasound images and generate candidate regions. The region of interest align algorithm was used to realize the pixel-level feature point positioning. The corresponding binary mask of prostate images was generated by the convolution network to segment the prostate region and the background. Then, the background information was shielded, and a data set of segmented ultrasound images of the prostate was constructed for the Inception-v3 network for lesion detection. A new network model was added to replace the original classification module, which is composed of forward and back propagation. Forward propagation mainly transfers the characteristics extracted from the convolution layer pooling layer below the pool_3 layer through the transfer learning strategy to the input layer and then calculates the loss value between the classified and label values to identify the ultrasound lesion of the prostate. The experimental results showed that the proposed method can accurately detect the ultrasound image of the prostate and segment prostate information at the pixel-level simultaneously. The proposed method has higher accuracy than that of the doctor’s manual diagnosis and other detection methods. Our simple and effective approach will serve as a solid baseline and help ease future research in the computer-aided diagnosis of prostate ultrasound images. Furthermore, this work will promote the development of prostate cancer ultrasound diagnostic technology.

UAV tracking based on saliency detection

This paper presents a novel unmanned aerial vehicle tracking framework. First, hierarchical convolutional neural network features are used to track the object independently in a correlation filter tracking framework. Second, a stability criterion is proposed, which is based on the variance of tracking results of each layer. Next, tracking result is adaptively fused via the variance. Meanwhile, the criterion can be used to measure the quality of tracking results. A saliency detection method is utilized to generate candidate regions when tracking failure occurs. By virtue of this method, our tracking algorithm can robustly cope with appearance changes and prevent drifting issues. Experimental results show that our proposed tracking algorithm performs favorably against state-of-the-art methods on two benchmark datasets.

Adaptive Anchor Networks for Multi-Scale Object Detection in Remote Sensing Images

Accurate and effective object detection in remote sensing images plays an extremely important role in marine transport, environmental monitoring and military operations. Due to the powerful ability of feature representation, region-based convolutional neural networks (RCNNs) have been widely used in this field, which firstly generate candidate regions through extracted feature maps and then classify and locate objects. However, most of existing methods generally use traditional backbone networks to extract feature maps with a decreased spatial resolution because of the continuous down-sampling, which will weaken the information detected from small objects. Besides, sliding windows strategy is employed in these methods to generate fixed anchors with a preset scale on feature maps, which is inappropriate for multi-scale object detection in remote sensing images. To solve the above problems, a novel and effective object detection framework named DetNet-FPN (Feature Pyramid Network) is proposed in this paper, in which a feature pyramid with strong feature representation is created by combining feature maps of different spatial resolution, at the same time, the resolution of feature maps is maintained by involving dilation convolutions. Furthermore, to match the proposed backbone, the GA (Guided Anchoring)-RPN strategy is improved for adaptive anchor generation, this strategy simultaneously predicts the locations where the center of objects are likely to exist as well as the scales and aspect ratios at different locations. Extensive experiments and comprehensive evaluations demonstrate the effectiveness of the proposed framework on DOTA and NWPU VHR-10 datasets.

Detecting Maneuvering Target Accurately Based on a Two-Phase Approach From Remote Sensing Imagery

Maneuvering target detection in satellite images is difficult due to their small sizes, blurred appearances under various illuminations and shadows, and occlusion by trees and buildings. Recently, a fully convolutional regression network (FCRN) was proposed and achieved by the state-of-the-art performance in the Munich vehicle database. However, such a one-phase approach often makes mistakes at difficult places because of its swift glance and rejecting any second check. In this letter, a new object spatial density building net (SDBN) was designed, and a two-phase detection approach was proposed. It used the first SDBN to generate candidate regions and the second SDBN to proceed with a meticulous check on the object categories. Experiments on four maneuvering target databases, the Munich vehicle database, the Open Vehicle Database of San-Francisco (OVDS), the Overhead Imagery Research Data Set (OIRDS), and the Open Aircraft Database (OAD) show that the proposed method outperforms FCRN by an obvious margin. In addition, the accurate geometrical parameters (positions, orientations, and lengths) of all the objects were computed based on the spatial density maps, and the published experimental result of FCRN in OIRDS was pointed out and the corrected result was given. All source codes and databases are available at http://www.github.com/cxy177/SDBN .

Geospatial Object Detection on High Resolution Remote Sensing Imagery Based on Double Multi-Scale Feature Pyramid Network

Object detection on very-high-resolution (VHR) remote sensing imagery has attracted a lot of attention in the field of image automatic interpretation. Region-based convolutional neural networks (CNNs) have been vastly promoted in this domain, which first generate candidate regions and then accurately classify and locate the objects existing in these regions. However, the overlarge images, the complex image backgrounds and the uneven size and quantity distribution of training samples make the detection tasks more challenging, especially for small and dense objects. To solve these problems, an effective region-based VHR remote sensing imagery object detection framework named Double Multi-scale Feature Pyramid Network (DM-FPN) was proposed in this paper, which utilizes inherent multi-scale pyramidal features and combines the strong-semantic, low-resolution features and the weak-semantic, high-resolution features simultaneously. DM-FPN consists of a multi-scale region proposal network and a multi-scale object detection network, these two modules share convolutional layers and can be trained end-to-end. We proposed several multi-scale training strategies to increase the diversity of training data and overcome the size restrictions of the input images. We also proposed multi-scale inference and adaptive categorical non-maximum suppression (ACNMS) strategies to promote detection performance, especially for small and dense objects. Extensive experiments and comprehensive evaluations on large-scale DOTA dataset demonstrate the effectiveness of the proposed framework, which achieves mean average precision (mAP) value of 0.7927 on validation dataset and the best mAP value of 0.793 on testing dataset.

Mobile Multi-Food Recognition Using Deep Learning

In this article, we propose a mobile food recognition system that uses the picture of the food, taken by the user’s mobile device, to recognize multiple food items in the same meal, such as steak and potatoes on the same plate, to estimate the calorie and nutrition of the meal. To speed up and make the process more accurate, the user is asked to quickly identify the general area of the food by drawing a bounding circle on the food picture by touching the screen. The system then uses image processing and computational intelligence for food item recognition. The advantage of recognizing items, instead of the whole meal, is that the system can be trained with only single item food images. At the training stage, we first use region proposal algorithms to generate candidate regions and extract the convolutional neural network (CNN) features of all regions. Second, we perform region mining to select positive regions for each food category using maximum cover by our proposed submodular optimization method. At the testing stage, we first generate a set of candidate regions. For each region, a classification score is computed based on its extracted CNN features and predicted food names of the selected regions. Since fast response is one of the important parameters for the user who wants to eat the meal, certain heavy computational parts of the application are offloaded to the cloud. Hence, the processes of food recognition and calorie estimation are performed in cloud server. Our experiments, conducted with the FooDD dataset, show an average recall rate of 90.98%, precision rate of 93.05%, and accuracy of 94.11% compared to 50.8% to 88% accuracy of other existing food recognition systems.

ListNet-based object proposals ranking

In object detection, object proposal methods have been widely used to generate candidate regions which may contain objects. Object proposal based on superpixel merging is one kind of object proposal methods, and the merging strategies of superpixels have been extensively explored. However, the ranking of generated candidate proposals still remains to be further studied. In this paper, we formulate the ranking of object proposals as a learning to rank problem, and propose a novel object proposals ranking method based on ListNet. In the proposed method, Selective Search, which is one of the state-of-the-art object proposal methods based on superpixel merging, is adopted to generate the candidate proposals. During the superpixel merging process, five discriminative objectness features are extracted from superpixel sets and the corresponding bounding boxes. Then, to weight each feature, a linear neural network is learned based on ListNet. Consequently, objectness scores can be computed for final candidate proposals ranking. Extensive experiments demonstrate the effectiveness and robustness of the proposed method.

Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks

In this paper, we focus on tackling the problem of automatic accurate localization of detected objects in high-resolution remote sensing images. The two major problems for object localization in remote sensing images caused by the complex context information such images contain are achieving generalizability of the features used to describe objects and achieving accurate object locations. To address these challenges, we propose a new object localization framework, which can be divided into three processes: region proposal, classification, and accurate object localization process. First, a region proposal method is used to generate candidate regions with the aim of detecting all objects of interest within these images. Then, generic image features from a local image corresponding to each region proposal are extracted by a combination model of 2-D reduction convolutional neural networks (CNNs). Finally, to improve the location accuracy, we propose an unsupervised score-based bounding box regression (USB-BBR) algorithm, combined with a nonmaximum suppression algorithm to optimize the bounding boxes of regions that detected as objects. Experiments show that the dimension-reduction model performs better than the retrained and fine-tuned models and the detection precision of the combined CNN model is much higher than that of any single model. Also our proposed USB-BBR algorithm can more accurately locate objects within an image. Compared with traditional features extraction methods, such as elliptic Fourier transform-based histogram of oriented gradients and local binary pattern histogram Fourier, our proposed localization framework shows robustness when dealing with different complex backgrounds.

Connected Component Clustering Based Text Detection with Structure Based Partition and Grouping

Extraction of text from natural scene images is a challenging problem because of its complex backgrounds and large variations of text patterns. In this paper, we presents an innovative scene text detection algorithm with the help of two machine learning classifiers: one for candidate generation and the other which filters out non text ones. And the enhancement technique followed by a text string detection with arbitrary orientations based on structure- based partition and grouping. To extracted the connected components (CCs) in images an algorithm us used ,popularly known as maximally stable extremal region algorithm. These extracted CCs are partitioned into clusters and generate candidate regions. However, the methods use AdaBoost classifier to training the samples and improve the text detection accuracy. The scale, skew, and color of each candidate can be estimated from CCs, and filtered the non text from the normalized images. To find the text string consists of two steps: A) Image partition to detect the text character candidates by using gradient magnitude of character components and B)Text character grouping to detect text strings by using structural analysis of text characters and then merges them into text string for example size of character,distance between neighboring characters.To improve efficiency and accuracy, our algorithms are carried out in multi-scales. The proposed system yield very high identification accuracy and take less time for detection as compared to the existing system.

Scene Text Detection via Connected Component Clustering and Nontext Filtering

In this paper, we present a new scene text detection algorithm based on two machine learning classifiers: one allows us to generate candidate word regions and the other filters out nontext ones. To be precise, we extract connected components (CCs) in images by using the maximally stable extremal region algorithm. These extracted CCs are partitioned into clusters so that we can generate candidate regions. Unlike conventional methods relying on heuristic rules in clustering, we train an AdaBoost classifier that determines the adjacency relationship and cluster CCs by using their pairwise relations. Then we normalize candidate word regions and determine whether each region contains text or not. Since the scale, skew, and color of each candidate can be estimated from CCs, we develop a text/nontext classifier for normalized images. This classifier is based on multilayer perceptrons and we can control recall and precision rates with a single free parameter. Finally, we extend our approach to exploit multichannel information. Experimental results on ICDAR 2005 and 2011 robust reading competition datasets show that our method yields the state-of-the-art performance both in speed and accuracy.