Hard Examples Research Articles

Face Manipulation Detection Based on Supervised Multi-Feature Fusion Attention Network.

Nowadays, faces in videos can be easily replaced with the development of deep learning, and these manipulated videos are realistic and cannot be distinguished by human eyes. Some people maliciously use the technology to attack others, especially celebrities and politicians, causing destructive social impacts. Therefore, it is imperative to design an accurate method for detecting face manipulation. However, most of the existing methods adopt single convolutional neural network as the feature extraction module, causing the extracted features to be inconsistent with the human visual mechanism. Moreover, the rich details and semantic information cannot be reflected with single feature, limiting the detection performance. Therefore, this paper tackles the above problems by proposing a novel face manipulation detection method based on a supervised multi-feature fusion attention network (SMFAN). Specifically, the capsule network is used for face manipulation detection, and the SMFAN is added to the original capsule network to extract details of the fake face image. Further, the focal loss is used to realize hard example mining. Finally, the experimental results on the public dataset FaceForensics++ show that the proposed method has better performance.

Regularized Hardmining loss for face recognition

For face recognition using deep learning architectures, loss functions have become a topic of research these days. This is because of the fact that when the discriminative ability of the loss function increases, then the face recognition accuracy increases. Hardmining loss is one such generic loss function that can be used with any basic loss function and has the ability to enhance the face recognition accuracy of the given basic loss function. Hardmining loss achieves the improvement by introducing greater penalty for hard examples. However, the problem with Hardmining loss is that the easy examples are allocated loss value near to zero. This limits the contribution of the easy examples specially in the later training stages. We therefore propose an improved Hardmining loss called the Regularized Hardmining loss. The Regularized Hardmining loss allocates a reasonable loss value to the easier examples as well. It thus helps easy examples maintain their contribution in the later training stage while still giving relatively greater penalty for hard examples thus preserving the property of the Hardmining loss. The Regularized Hardmining loss fine-tunes the behavior of Hardmining loss for better performance. The results of Regularized Hardmining loss when applied with Cross Entropy loss show an increased accuracy of face recognition from 93.78% to 95.55% on LFW dataset.

A Hard Example Mining Approach for Concealed Multi-Object Detection of Active Terahertz Image

Concealed objects detection in terahertz imaging is an urgent need for public security and counter-terrorism. So far, there is no public terahertz imaging dataset for the evaluation of objects detection algorithms. This paper provides a public dataset for evaluating multi-object detection algorithms in active terahertz imaging. Due to high sample similarity and poor imaging quality, object detection on this dataset is much more difficult than on those commonly used public object detection datasets in the computer vision field. Since the traditional hard example mining approach is designed based on the two-stage detector and cannot be directly applied to the one-stage detector, this paper designs an image-based Hard Example Mining (HEM) scheme based on RetinaNet. Several state-of-the-art detectors, including YOLOv3, YOLOv4, FRCN-OHEM, and RetinaNet, are evaluated on this dataset. Experimental results show that the RetinaNet achieves the best mAP and HEM further enhances the performance of the model. The parameters affecting the detection metrics of individual images are summarized and analyzed in the experiments.

Fabric defect detection with an attention mechanism based on hard sample training

In view of the various types of fabric defects, and the problems of confusion, density unevenness and small target defects, which are difficult to detect, this paper builds a deep learning defect detection network incorporating an attention mechanism. The data augmentation strategy is used to enrich the number of samples of each defective type, and the enriched samples were extracted by the feature extraction network integrated with the attention mechanism, which can improve the feature extraction ability of confusable defect types and small defect types. Region proposal generation generates a proposal box for extracted features, and adds an online hard example mining strategy to re-learn hard examples to accelerate network convergence. Region feature aggregation maps the proposal box to the feature map to obtain the region of interest. Finally, the defect features are classified and the bounding boxes are regressed. The results show that this algorithm can effectively detect 39 categories of fabric defects with a detection speed of 0.085 s and a detection accuracy of 0.9338.

Contrastive Learning for Mitochondria Segmentation.

Mitochondria segmentation in electron microscopy images is essential in neuroscience. However, due to the image degradation during the imaging process, the large variety of mitochondrial structures, as well as the presence of noise, artifacts and other sub-cellular structures, mitochondria segmentation is very challenging. In this paper, we propose a novel and effective contrastive learning framework to learn a better feature representation from hard examples to improve segmentation. Specifically, we adopt a point sampling strategy to pick out representative pixels from hard examples in the training phase. Based on these sampled pixels, we introduce a pixel-wise label-based contrastive loss which consists of a similarity loss term and a consistency loss term. The similarity term can increase the similarity of pixels from the same class and the separability of pixels from different classes in feature space, while the consistency term is able to enhance the sensitivity of the 3D model to changes in image content from frame to frame. We demonstrate the effectiveness of our method on MitoEM dataset as well as FIB-SEM dataset and show better or on par with state-of-the-art results.

Study on an infrared multi-target detection method based on the pseudo-two-stage model

One-stage or two-stage deep learning based methods have problems when performing infrared multi-target detection, namely low accuracy and low running speed. Inspired by regression ideas of two-stage anchor-based model from coarse to fine, we propose a high performance pseudo-two-stage model that is specific to infrared images, in order to make a trade-off. The model retains the fast speed of the one-stage detection model through the introduction of cascade regression. We designed a dual-pass fusion module (DFM) and adaptive channel enhancement module (ACEM) to implement infrared image key feature fusion and calibration. To further optimize the model, we exploited the cascade regression and hard example mining by analyzing the shortcomings of the current one-stage detection approach. We conducted comparative experiments on the FLIR ADAS dataset and our method obtained 75.57% mAP, which is about 5% and 13% higher than the Faster R-CNN two-stage model and the SSD one-stage model, respectively. We ran the model at 21.4 FPS on Geforce RTX 2080 Ti, which made it 3FPS slower than SSD. The promising results show that the proposed approach is effective and practical.

MSB R-CNN: A Multi-Stage Balanced Defect Detection Network

Deep learning networks are applied for defect detection, among which Cascade R-CNN is a multi-stage object detection network and is state of the art in terms of accuracy and efficiency. However, it is still a challenge for Cascade R-CNN to deal with complex and diverse defects, as the widely varied shapes of defects lead to inefficiency for the traditional convolution filter to extract features. Additionally, the imbalance in features, losses and samples cause lower accuracy. To address the above challenges, this paper proposes a multi-stage balanced R-CNN (MSB R-CNN) for defect detection based on Cascade R-CNN. Firstly, deformable convolution is adopted in different stages of the backbone network to improve its adaptability to the varying shapes of the defect. Then, the features obtained by the backbone network are refined and enhanced by the balanced feature pyramid. To overcome the imbalance of classification and regression loss, the balanced L1 loss is applied at different stages to correct it. Finally, for the sample selection, the interaction of union (IoU) balanced sampler and the online hard example mining (OHEM) sampler are combined at different stages to make the sampling more reasonable, which can bring a better accuracy and convergence effect to the model. The results of our experiments on the DAGM2007 dataset has shown that our network (MSB R-CNN) can achieve a mean average precision (mAP) of 67.5%, an increase of 1.5% mAP, compared to Cascade R-CNN.

Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms

Diatom test is one of the commonly used diagnostic methods for drowning in forensic pathology, which provides supportive evidence for drowning. However, in forensic practice, it is time-consuming and laborious for forensic experts to classify and count diatoms, whereas artificial intelligence (AI) is superior to human experts in processing data and carrying out classification tasks. Some AI techniques have focused on searching diatoms and classifying diatoms. But, they either could not classify diatoms correctly or were time-consuming. Conventional detection deep network has been used to overcome these problems but failed to detect the occluded diatoms and the diatoms similar to the background heavily, which could lead to false positives or false negatives. In order to figure out the problems above, an improved region-based full convolutional network (R-FCN) with online hard example mining and the shape prior of diatoms was proposed. The online hard example mining (OHEM) was coupled with the R-FCN to boost the capacity of detecting the occluded diatoms and the diatoms similar to the background heavily and the priors of the shape of the common diatoms were explored and introduced to the anchor generation strategy of the region proposal network in the R-FCN to locate the diatoms precisely. The results showed that the proposed approach significantly outperforms several state-of-the-art methods and could detect the diatom precisely without missing the occluded diatoms and the diatoms similar to the background heavily. From the study, we could conclude that (1) the proposed model can locate the position and identify the genera of common diatoms more accurately; (2) this method can reduce the false positives or false negatives in forensic practice; and (3) it is a time-saving method and can be introduced.

Word Representation Learning Based on Bidirectional GRUs With Drop Loss for Sentiment Classification

Sentiment classification is a fundamental task in many natural language processing applications. Neural networks have achieved great successes on the sentiment classification task in recent years, since recurrent neural networks and long-short-term memory networks have the ability to deal with sequences of different lengths and to capture contextual semantic information. However, the effectiveness of these methods is limited when used to extract contextual information from relatively long texts. Therefore, in our model, we apply bidirectional gated recurrent units to capture contextual information as far as possible when learning word representations, which may effectively reduce the noise compared to other methods. We also propose a novel loss function namely drop loss (DL) which makes the model focus on the hard examples - examples which are easily classified incorrectly - in order to improve the accuracy of the model. We experiment on four commonly used datasets, and the results show that the proposed method has a good performance on four datasets, and needs fewer parameters compared with recent benchmarks, such as CoVe, ULMFiT, embeddings from language models, and bidirectional encoder representations from transformers. Furthermore, we demonstrate that the classification performance of existing shallow network models can be significantly improved by using DL. In particular, the accuracy of the CNN+LSTM model improves 9% on the IMDB-10 dataset.

Fully convolutional siamese networks based change detection for optical aerial images with focal contrastive loss

This paper proposes a change detection algorithm based on Fully Convolutional Siamese Networks for optical aerial images, which is trained by using an improved Contrastive Loss-Focal Contrastive Loss (FCL). The proposed framework equipped with contrastive loss can extract features directly from image pairs and measure changes by using a distance metric. In other words, this method encourages reducing intra-class variance and enlarging inter-class difference, so that the binarized change map can be obtained by a simple threshold. In change detection task, a critical problem is how to overcome example imbalance (i.e. unchanged examples are much more than changed examples). To address this challenge, a novel focal contrastive loss is proposed to further improve the performance of the model. FCL can reduce the impact of example imbalance and make the model focus learning on hard examples. Extensive experiments demonstrate that the proposed approach is more abstract as well as robust. Compared with other baseline methods, the presented method achieves better results on SAZDA, TISZADOB, CDD, and WHU-CD data set. It achieves state-of-the-art performance in terms of the F1 measure.

A high resolution representation network with multi-path scale for retinal vessel segmentation

Background and objectives: Automatic retinal vessel segmentation (RVS) in fundus images is expected to be a vital step in the early image diagnosis of ophthalmologic diseases. However, it is a challenging task to detect the retinal vessel accurately mainly due to the vascular intricacies, lesion areas and optic disc edges in retinal fundus images. Methods: In this paper, we propose a high resolution representation network with multi-path scale (MPS-Net) for RVS aiming to improve the performance of extracting the retinal blood vessels. In the MPS-Net, there exist one high resolution main road and two lower resolution branch roads where the proposed multi-path scale modules are embedded to enhance the representation ability of network. Besides, in order to guide the network focus on learning the features of hard examples in retinal images, we design a hard-focused cross-entropy loss function. Results: We evaluate our network structure on DRIVE, STARE, CHASE and synthetic images and the quantitative comparisons with respect to the existing methods are presented. The experimental results show that our approach is superior to most methods in terms of F1-score, sensitivity, G-mean and Matthews correlation coefficient. Conclusions: The promising segmentation performances reveal that our method has potential in real-world applications and can be exploited for other medical images with further analysis.

Robust Vehicle Detection in High-Resolution Aerial Images With Imbalanced Data

Vehicle detection in images from unmanned aerial vehicles (UAVs) plays an important role in traffic surveillance and urban planning due to the popularity of UAVs. However, the class imbalance problem is an important factor that restricts the performance of vehicle detectors. There are two types of class imbalance in UAV images, i.e., foreground-background imbalance and foreground–foreground imbalance. For anchor-based single stage detector, as many ground truths cannot be assigned to corresponding anchors because of low intersection over union, it makes the foreground-background imbalance problem more severe. Therefore, we propose a novel bag-based single-stage detector, which treats each position on the feature map as a bag. A simple and adaptive definition of bags is proposed along with the positive sample definition method, which is utilized to ensure more ground truths can be assigned to proper bags. In addition, we utilize online hard example mining method to control the proportion of positive and negative samples during the training process. To address the foreground–foreground imbalance, we propose a novel data augmentation algorithm, which allows us to create appropriate visual context for under-represented class. Extensive experiments demonstrate the superiority of the proposed algorithm, compared with other state-of-the-art solutions. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement—Recently, unmanned aerial vehicles (UAVs) are widely used in intelligent transportation due to their low price and high flexibility, which makes vehicle detection in UAV images important for automatically gathering of traffic information. However, the class imbalance problem, which is common in object detection where some classes have far fewer frequencies in the dataset, has an adverse effect on the performance of vehicle detectors. The data augmentation method and deep learning based vehicle detector proposed in this article are able to reduce the negative impact and improve detection performance by at least 1.27% in mean average precision index. In addition, compared with algorithms with similar detection performance, our method is at least 15 ms faster. The proposed method can benefit users in a wide variety of applications including UAV transportation, traffic surveillance, and urban planning.

Bone Metastasis Detection in the Chest and Pelvis from a Whole-Body Bone Scan Using Deep Learning and a Small Dataset

The aim of this study was to establish an early diagnostic system for the identification of the bone metastasis of prostate cancer in whole-body bone scan images by using a deep convolutional neural network (D-CNN). The developed system exhibited satisfactory performance for a small dataset containing 205 cases, 100 of which were of bone metastasis. The sensitivity and precision for bone metastasis detection and classification in the chest were 0.82 ± 0.08 and 0.70 ± 0.11, respectively. The sensitivity and specificity for bone metastasis classification in the pelvis were 0.87 ± 0.12 and 0.81 ± 0.11, respectively. We propose the use of hard example mining for increasing the sensitivity and precision of the chest D-CNN. The developed system has the potential to provide a prediagnostic report for physicians’ final decisions.

Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks

Adversarial Training is proved to be an efficient method to defend against adversarial examples, being one of the few defenses that withstand strong attacks. However, traditional defense mechanisms assume a uniform attack over the examples according to the underlying data distribution, which is apparently unrealistic as the attacker could choose to focus on more vulnerable examples. We present a weighted minimax risk optimization that defends against non-uniform attacks, achieving robustness against adversarial examples under perturbed test data distributions. Our modified risk considers importance weights of different adversarial examples and focuses adaptively on harder examples that are wrongly classified or at higher risk of being classified incorrectly. The designed risk allows the training process to learn a strong defense through optimizing the importance weights. The experiments show that our model significantly improves state-of-the-art adversarial accuracy under non-uniform attacks without a significant drop under uniform attacks.

Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.

Esophageal cancer detection based on classification of gastrointestinal CT images using improved Faster RCNN

PurposeEsophageal cancer is a common malignant tumor in life, which seriously affects human health. In order to reduce the work intensity of doctors and improve detection accuracy, we proposed esophageal cancer detection using deep learning. The characteristics of deep learning: association and structure, activity and experience, essence and variation, migration and application, value and evaluation. MethodThe improved Faster RCNN esophageal cancer detection in this paper introduces the online hard example mining (OHEM) mechanism into the system, and the experiment used 1520 gastrointestinal CT images from 421 patients. Then, we compare the overall performance of Inception-v2, Faster RCNN, and improved Faster RCNN through F-1 measure, mean average precision (mAP), and detection time. ResultsThe experiment shows that the overall performance of the improved Faster RCNN is higher than the other two networks. The F-1 measure of our method reaches 95.71%, the mAP reaches 92.15%, and the detection time per CT is only 5.3s. ConclusionThrough comparative analysis on the esophageal cancer image data set, the experimental results show that the introduction of online hard example mining mechanism in the Faster RCNN algorithm can improve the detection accuracy to a certain extent.

DGANet: Dynamic Gradient Adjustment Anchor-Free Object Detection in Optical Remote Sensing Images

Remote sensing image object detection has been studied by many researchers in recent years using deep neural networks. However, optical remote sensing images contain many scenes with small and dense objects, resulting in a high rate of misrecognition. Firstly, in this work we selected a deep layer aggregation network with updated deformable convolution layers as the backbone to extract object features. The detection and classification of objects was based on the center-point network without non-maximum suppression. Secondly, the dynamic gradient adjustment embedded into the classification loss function was put forward to harmonize the quantity imbalance between easy and hard examples, as well as between positive and negative examples. Furthermore, the complete intersection over union (CIoU) loss function was selected as the objective function of bounding box regression, which achieves better convergence speed and accuracy. Finally, in order to validate the effectiveness and precision of the dynamic gradient adjustment network (DGANet), we conducted a series of experiments in remote sensing public datasets UCAS-AOD and LEVIR. The comparison experiments demonstrate that the DGANet achieves a more accurate detection result in optical remote sensing images.

Object Identification and Localization using Convolution Neural Network

Improving object identification against impediment, obscure and clamor image is a basic advance to deploy detector in real time applications. Since it is preposterous to expect to debilitate all picture abandons through information assortment, numerous specialists look to produce hard examples in preparing. The produced hard examples are either pictures or highlight maps with coarse patches exited in the spatial measurements. Huge overheads are needed in preparing the extra hard examples and additionally assessing drop-out patches utilizing additional organization branches. In this paper we proposed GRAD CAM++ with Mask Regional Convolution Neural Network (Mask RCNN) based item limitation and identification. The significant advantages of utilizing Mask R-CNN is that they beat all the partner techniques in the space and can likewise be utilized in unaided environments. The proposed identifier dependent on GRAD CAM++ with Mask R-CNN gives a vigorous and plausible capacity on recognizing and grouping objects exist and their shapes progressively on location. It is discovered that the proposed strategy can perform exceptionally successful and productive in a wide scope of pictures and gives higher goal visual portrayal.

An Improved Object Detection Method for Automatic Electrical Equipment Defect Detection

Employing computer vision and machine learning on equipment defect detection has become an important trend of electric power inspection. This paper presents a new electrical equipment default detection method based on improved YOLOv3. By combining abundant geometric measures, Complete IOU(CIOU) make the bounding box regression during NMS more accurate. Focal loss function, which focus on differentiate between easy and hard examples, is employed to deal with the class imbalance problem. Experiment results show that the proposed approach obtains competitive performance compared with state-of-the-art deep learning object detection methods.

Transformer Oil Leakage Detection Based on Loop Training Method

<p indent=0mm>Inspecting whether the transformer has oil leakage problem is of great value in maintaining the safety and stability of the power grid. The oil stain on the ground is an important basis for judging whether the transformer leaks oil. The different shapes of the oil stain areas, complex background and the influence of shadow have brought challenges to the oil stain detection. Shadow is a ubiquitous physical phenomenon in nature, and the impact on oil stain detection is inevitable. In order to eliminate the influence of shadow, we propose a loop training method. The histogram equalization is adopted to enhance the contrast of the hard example between the oil stain area and the shadow, and the enhanced images are iteratively trained to reduce the interference of the shadow to improve the recall. At the same time, we introduce negative examples to alleviate the false detection of oil stain to improve the precision. The data are collected in the real environment of the substation. To validate the effectiveness of the proposed method, 8 schemes are designed for comparison experiments. Experimental results show that the models using the proposed method can effectively eliminate the influence of shadow on oil stain detection, and significantly improve the accuracy of oil stain detection.