Image Recognition Performance Research Articles

DataPoste: a mobile application based on Understandable Deep Learning to characterise electrical equipment in secondary distribution substation

Gathering electrical reference information of equipment is a time-consuming task done by technicians on site. To make the task easier, we develop DataPoste, a mobile application performing image recognition with deep learning, to provide this information from a taken picture. Due to its black-box nature, we wanted to ensure the model performance in real-world condition and provide trust and acceptance for the domain experts. To do so, we investigate explainable artificial intelligence methods to better understand the model decision-making logic. Accordingly, we identified misbehaviour from the model and corrected it. Providing these results also help the domain experts to trust the model.

Textual Backdoor Attack for the Text Classification System

Deep neural networks provide good performance for image recognition, speech recognition, text recognition, and pattern recognition. However, such networks are vulnerable to backdoor attacks. In a backdoor attack, normal data that do not include a specific trigger are correctly classified by the target model, but backdoor data that include the trigger are incorrectly classified by the target model. One advantage of a backdoor attack is that the attacker can use a specific trigger to attack at a desired time. In this study, we propose a backdoor attack targeting the BERT model, which is a classification system designed for use in the text domain. Under the proposed method, the model is additionally trained on a backdoor sentence that includes a specific trigger, and afterward, if the trigger is attached before or after an original sentence, it will be misclassified by the model. In our experimental evaluation, we used two movie review datasets (MR and IMDB). The results show that using the trigger word “ATTACK” at the beginning of an original sentence, the proposed backdoor method had a 100% attack success rate when approximately 1.0% and 0.9% of the training data consisted of backdoor samples, and it allowed the model to maintain an accuracy of 86.88% and 90.80% on the original samples in the MR and IMDB datasets, respectively.

Integrated Transfer Learning Method for Image Recognition Based on Neural Network

As recognition technology advances in various fields, the problem of requiring a large number of correct image recognition is also accompanied. So as to promote the correctness of image recognition, we adopted two Convolutional Neural Network (CNN) integrated migration learning network models VGG-16 CNN and LeNet-5 CNN. Experimental results prove that the performance of image recognition in various models are improved. Also, the migration learning and convolutional neural network are used to construct a K-selection migration learning algorithm and a migration framework model whose parameters are migrated to the image data set. The use of a label data set with a large sample size, the selection of a pre-trained convolutional neural network model with a deep layer of complex network structure and the selection of an appropriate network fine-tuning depth can optimize the migration adaptation effect of the migration network. The image recognition precision of the algorithm is improved by about 3%, which further improves the migration adaptation effect.

Research on fan vibration fault diagnosis based on image recognition

The conventional methods for vibration fault detection and diagnosis relies on feature extraction from the waveforms of the vibration signals. This article exploits the scope of image recognition application for the detection and diagnosis of fan vibration faults. In this paper, a novel image recognition technique is proposed for vibration-based fault diagnosis using the spectrum images of the vibration signals. 1D vibration signal spectrum is initially computed using Fast Fourier Transform (FFT) and the FFT frequencies are adjusted such that it captures a vibration spectrum diagram as 2D image representation. FFT based vibration analysis is done and the image recognition concept is utilized for feature extraction and a machine learning classification module is used for fault analysis and diagnosis. Effective feature generation is done using Principal Component Analysis (PCA) by removing the redundancy from the feature vectors and machine learning classifiers are used to obtain improved image recognition and classification performance. Artificial Neural Network (ANN) classifier yields better performance in terms of various performance parameters and percentage improvement in terms of accuracy for ANN classification methods over Support Vector Machine (SVM), k-Nearest Neighbours (kNN) and Random Forest Ensemble (RFE) methods are 10.01 %, 4.51 % and 2.01 % respectively. Comparative scenarios are considered in this work for fan vibration fault detection as well as diagnosis based on the image features for various realistic vibration fault conditions. Effectiveness of the proposed image recognition-based technique is compared with the state-of-the-art methods, justifying its outperformance for fan fault detection and diagnosis using the combination of spectrum adjustment, PCA and ANN classification method.

Few-shot contrastive learning for image classification and its application to insulator identification.

This paper presents a novel discriminative Few-shot learning architecture based on batch compact loss. Currently, Convolutional Neural Network (CNN) has achieved reasonably good performance in image recognition. Most existing CNN methods facilitate classifiers to learn discriminating patterns to identify existing categories trained with large samples. However, learning to recognize novel categories from a few examples is a challenging task. To address this, we propose the Residual Compact Network to train a deep neural network to learn hierarchical nonlinear transformations to project image pairs into the same latent feature space, under which the distance of each positive pair is reduced. To better use the commonality of class-level features for category recognition, we develop a batch compact loss to form robust feature representations relevant to a category. The proposed methods are evaluated on several datasets. Experimental evaluations show that our proposed method achieves acceptable results in Few-shot learning.

Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on neural decoding and encoding analyses where DNN unit activations and human brain activity are predicted from each other. We find that BH scores for 29 pre-trained DNNs with various architectures are negatively correlated with image recognition performance, thus indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that single-path sequential feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method may provide new ways to design DNNs in light of their representational homology to the brain.

Scene Recognition Using Deep Softpool Capsule Network Based on Residual Diverse Branch Block.

With the improvement of the quality and resolution of remote sensing (RS) images, scene recognition tasks have played an important role in the RS community. However, due to the special bird’s eye view image acquisition mode of imaging sensors, it is still challenging to construct a discriminate representation of diverse and complex scenes to improve RS image recognition performance. Capsule networks that can learn the spatial relationship between the features in an image has a good image classification performance. However, the original capsule network is not suitable for images with a complex background. To address the above issues, this paper proposes a novel end-to-end capsule network termed DS-CapsNet, in which a new multi-scale feature enhancement module and a new Caps-SoftPool method are advanced by aggregating the advantageous attributes of the residual convolution architecture, Diverse Branch Block (DBB), Squeeze and Excitation (SE) block, and the Caps-SoftPool method. By using the residual DBB, multiscale features can be extracted and fused to recover a semantic strong feature representation. By adopting SE, the informative features are emphasized, and the less salient features are weakened. The new Caps-SoftPool method can reduce the number of parameters that are needed in order to prevent an over-fitting problem. The novel DS-CapsNet achieves a competitive and promising performance for RS image recognition by using high-quality and robust capsule representation. The extensive experiments on two challenging datasets, AID and NWPU-RESISC45, demonstrate the robustness and superiority of the proposed DS-CapsNet in scene recognition tasks.

SeedSortNet: a rapid and highly effificient lightweight CNN based on visual attention for seed sorting.

Seed purity directly affects the quality of seed breeding and subsequent processing products. Seed sorting based on machine vision provides an effective solution to this problem. The deep learning technology, particularly convolutional neural networks (CNNs), have exhibited impressive performance in image recognition and classification, and have been proven applicable in seed sorting. However the huge computational complexity and massive storage requirements make it a great challenge to deploy them in real-time applications, especially on devices with limited resources. In this study, a rapid and highly efficient lightweight CNN based on visual attention, namely SeedSortNet, is proposed for seed sorting. First, a dual-branch lightweight feature extraction module Shield-block is elaborately designed by performing identity mapping, spatial transformation at higher dimensions and different receptive field modeling, and thus it can alleviate information loss and effectively characterize the multi-scale feature while utilizing fewer parameters and lower computational complexity. In the down-sampling layer, the traditional MaxPool is replaced as MaxBlurPool to improve the shift-invariant of the network. Also, an extremely lightweight sub-feature space attention module (SFSAM) is presented to selectively emphasize fine-grained features and suppress the interference of complex backgrounds. Experimental results show that SeedSortNet achieves the accuracy rates of 97.33% and 99.56% on the maize seed dataset and sunflower seed dataset, respectively, and outperforms the mainstream lightweight networks (MobileNetv2, ShuffleNetv2, etc.) at similar computational costs, with only 0.400M parameters (vs. 4.06M, 5.40M).

Photoactive Electro‐Controlled Visual Perception Memory for Emulating Synaptic Metaplasticity and Hebbian Learning

AbstractIn bionic technology, it has become an innovative process imitating the functionality and structuralism of human biological systems to exploit advanced artificial intelligent machines. Bionics plays a significant role in environmental protection, especially for its low energy loss. By fusing the concept of receptor‐like sensing component and synapse‐like memory, the photoactive electro‐controlled optical sensory memory (PE‐SM) is proposed and realized in a single device, which endows a simple methodology of reducing power consumption by photoactive electro‐control. The PE‐SM is the system built with the stacked atomically thick materials, in which rhenium diselenide serves as a robust photosensor, hexagonal boron nitride serves as a tunneling dielectric, and graphene serves as a charge‐storage layer. With the features of the PE‐SM, it performs synaptic metaplasticities under optical spikes. In addition, a simulated spiking neural network composed of 24 × 24 PE‐SMs is further presented in an unsupervised machine learning environment, performing image recognition via the Hebbian rule. The PE‐SM not only improves the neuromorphic computing efficiency but also simplifies the circuit‐size structure. Eventually, the concept of photoactive electro‐control can extend to other photosensitive 2D materials and provide a new approach of constructing either visual perception memory or photonic synaptic devices.

Confocal Laser Microscopy for in vivo Intraoperative Application: Diagnostic Accuracy of Investigator and Machine Learning Strategies

Background: Confocal laser microscopy (CLM) is one of the optical techniques that are promising methods of intraoperative in vivo real-time tissue examination based on tissue fluorescence. However, surgeons might struggle interpreting CLM images intraoperatively due to different tissue characteristics of different tissue pathologies in clinical reality. Deep learning techniques enable fast and consistent image analysis and might support intraoperative image interpretation. The objective of this study was to analyze the diagnostic accuracy of newly trained observers in the evaluation of normal colon and peritoneal tissue and colon cancer and metastasis, respectively, and to compare it with that of convolutional neural networks (CNNs). Methods: Two hundred representative CLM images of the normal and malignant colon and peritoneal tissue were evaluated by newly trained observers (surgeons and pathologists) and CNNs (VGG-16 and Densenet121), respectively, based on tissue dignity. The primary endpoint was the correct detection of the normal and cancer/metastasis tissue measured by sensitivity and specificity of both groups. Additionally, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated for the newly trained observer group. The interobserver variability of dignity evaluation was calculated using kappa statistic. The F1-score and area under the curve (AUC) were used to evaluate the performance of image recognition of the CNNs’ training scenarios. Results: Sensitivity and specificity ranged between 0.55 and 1.0 (pathologists: 0.66–0.97; surgeons: 0.55–1.0) and between 0.65 and 0.96 (pathologists: 0.68–0.93; surgeons: 0.65–0.96), respectively. PPVs were 0.75 and 0.90 in the pathologists’ group and 0.73–0.96 in the surgeons’ group, respectively. NPVs were 0.73 and 0.96 for pathologists’ and between 0.66 and 1.00 for surgeons’ tissue analysis. The overall interobserver variability was 0.54. Depending on the training scenario, cancer/metastasis tissue was classified with an AUC of 0.77–0.88 by VGG-16 and 0.85–0.89 by Densenet121. Transfer learning improved performance over training from scratch. Conclusions: Newly trained investigators are able to learn CLM images features and interpretation rapidly, regardless of their clinical experience. Heterogeneity in tissue diagnosis and a moderate interobserver variability reflect the clinical reality more realistic. CNNs provide comparable diagnostic results as clinical observers and could improve surgeons’ intraoperative tissue assessment.

Thai Sign Language Image Recognition for the Hearing-Impaired using Radial Inverse Force Histogram combined with Max-Min Boundary

Sign language image recognition is also a very interesting research topic. Because it can be applied to help normal people understand and use it as a communication tool for the hearing impaired. The objectives of this research were to: 1) study and analyze Thai Sign Language image recognition image data for hearing impaired, 2) develop Thai Sign Language image recognition system for hearing impaired by using new techniques, and 3) measure the efficiency of Thai sign language image recognition for hearing impaired using Radial Inverse Force Histogram and the Maximum and Minimum boundary values. The results of the research were as follows: 1) The study and analysis of image data of Thai Sign Language Image Recognition In this research, 62,694 sign language images were used, divided into 2 parts: 1) American Sign Language images, which consisted of 36 groups of images, namely 26 groups of letters (AZ) and 10 groups of numbers (0-9), and Part 2) Picture of Thai Sign Language consisting of 61 groups of images, including 44 groups of letters (ก-ฮ), 7 groups of vowels and 10 groups of numbers (0-9). Each group of pictures is rotated, enlarged, and Image promotion There were 6 sub-groups of images in various forms, divided into 2 parts: 70% of the images for training and 30% of the images for testing. 3) The results of measuring the efficiency of image recognition. It is divided into two parts: American Sign Language Image Recognition and Thai Sign Language Image Recognition. Compared with the Angular histogram method, the mean image accuracy was 0.86, the recall of the mean American sign language was 0.91, and the accuracy of the Thai Sign Language was 0.78. The recognition performance for Thai Sign Language images averaged 0.89, while the recognition efficiency was achieved when using radial inverse force histograms in combination with image similarity measurements with maximum-minimum boundary values. Accuracy for Mean American Sign Language was 0.99 and Remembrance for Mean American Sign was 1.00, while Accuracy for Mean Thai Sign Language was 0.89. Mean Remembrance for Thai Sign Language was 0.96. The results of the visual recognition performance measurement of both the American Sign Language and the Thai Sign Language images were very good compared to the Angular Histogram method.

Multi-label image recognition with two-stream dynamic graph convolution networks

Recent studies use Graph Convolution Networks (GCN) to model label correlation for multi-label images because of the outstanding performance of GCN in relational modeling tasks. However, the traditional GCN has low generalization, and the current state-of-the-arts' accuracy is poor. Therefore, we propose a Two-Stream Dynamic Graph Convolution Network (2S-DGCN) to improve the performance of multi-label image recognition. In 2S-DGCN, we first obtain the Up Confidence Score of prediction categories (UCS), the content-aware category and the label discriminant vector by a Semantic Attention Module (SAM) and a Dynamic Graph Convolution Network (DGCN) in upstream. Then fed the new graph feature nodes reconstructed by lateral embedding the content-aware category and the label discriminant vector into a DGCN to produce the Down Confidence Score of prediction categories (DCS) in downstream. Finally, the Final Confidence Score of prediction categories (FCS) for multi-label image recognition is synthesized by fusing the UCS and DCS. Extensive experiments on the public multi-label benchmarks achieve mAPs of 85.6% on MS-COCO and 95.4% on VOC 2007. The results of compared experiment and visualization demonstrate that our method has better performance than the current state-of-the-art methods.

Neural Mechanism of Noise Affecting Face Recognition

Face recognition is one of the most important cognitive functions for humans in social activities. The ability will be negatively affected when the face images deteriorate. However, the neural process of extracting facial information under challenging conditions is still poorly understood. Therefore, it is necessary to further understand the neurophysiological relevance of this effect. We examined patients with multiple subdural electrodes (ECoG) monitored for clinical purposes. During the experimental task, the patients were presented with face and house images with different noise levels and were asked to recognize the faces. We found a striking increase in high gamma band power (HGP; 60–160 Hz) when face images were shown. We localized the face-specific electrodes to the fusiform gyrus (FG) and surrounding cortices. For each subject, the behavioral performance and magnitudes of the HGP for the face-specific sites significantly both fit a sigmoid function and showed similar changes. Additionally, the curve profile of the average HGP magnitude across the face-specific sites was almost equal to the average behavior curve; the former could precisely track the behavioral performance. In general, these results suggest that the HGP in the FG is closely related to the performance of face image recognition.

A Collaborative Region Detection and Grading Framework for Forest Fire Smoke Using Weakly Supervised Fine Segmentation and Lightweight Faster-RCNN

Forest fires are serious disasters that affect countries all over the world. With the progress of image processing, numerous image-based surveillance systems for fires have been installed in forests. The rapid and accurate detection and grading of fire smoke can provide useful information, which helps humans to quickly control and reduce forest losses. Currently, convolutional neural networks (CNN) have yielded excellent performance in image recognition. Previous studies mostly paid attention to CNN-based image classification for fire detection. However, the research of CNN-based region detection and grading of fire is extremely scarce due to a challenging task which locates and segments fire regions using image-level annotations instead of inaccessible pixel-level labels. This paper presents a novel collaborative region detection and grading framework for fire smoke using a weakly supervised fine segmentation and a lightweight Faster R-CNN. The multi-task framework can simultaneously implement the early-stage alarm, region detection, classification, and grading of fire smoke. To provide an accurate segmentation on image-level, we propose the weakly supervised fine segmentation method, which consists of a segmentation network and a decision network. We aggregate image-level information, instead of expensive pixel-level labels, from all training images into the segmentation network, which simultaneously locates and segments fire smoke regions. To train the segmentation network using only image-level annotations, we propose a two-stage weakly supervised learning strategy, in which a novel weakly supervised loss is proposed to roughly detect the region of fire smoke, and a new region-refining segmentation algorithm is further used to accurately identify this region. The decision network incorporating a residual spatial attention module is utilized to predict the category of forest fire smoke. To reduce the complexity of the Faster R-CNN, we first introduced a knowledge distillation technique to compress the structure of this model. To grade forest fire smoke, we used a 3-input/1-output fuzzy system to evaluate the severity level. We evaluated the proposed approach using a developed fire smoke dataset, which included five different scenes varying by the fire smoke level. The proposed method exhibited competitive performance compared to state-of-the-art methods.

MetaAugment: Sample-Aware Data Augmentation Policy Learning

Automated data augmentation has shown superior performance in image recognition. Existing works search for dataset-level augmentation policies without considering individual sample variations, which are likely to be sub-optimal. On the other hand, learning different policies for different samples naively could greatly increase the computing cost. In this paper, we learn a sample-aware data augmentation policy efficiently by formulating it as a sample reweighting problem. Specifically, an augmentation policy network takes a transformation and the corresponding augmented image as inputs, and outputs a weight to adjust the augmented image loss computed by a task network. At training stage, the task network minimizes the weighted losses of augmented training images, while the policy network minimizes the loss of the task network on a validation set via meta-learning. We theoretically prove the convergence of the training procedure and further derive the exact convergence rate. Superior performance is achieved on widely-used benchmarks including CIFAR-10/100, Omniglot, and ImageNet.

Research on Ultrasonic Image Recognition Based on Optimization Immune Algorithm.

With the rapid development of science and technology, ultrasound has been paid more and more attention by people, and it is widely used in engineering, diagnosis, and detection. In this paper, an ultrasonic image recognition method based on immune algorithm is proposed for ultrasonic images, and its method is applied to medical ultrasound liver image recognition. Firstly, this paper grays out the ultrasound liver image and selects the region of interest of the image. Secondly, it extracts the feature based on spatial gray matrix independent matrix, spatial frequency decomposition, and fractal features. Then, the immune algorithm is used to classify and identify the normal liver, liver cirrhosis, and liver cancer ultrasound images. Finally, based on the deficiency of the immune algorithm, it is combined with the support vector machine to form an optimized immune algorithm, which improves the performance of ultrasonic liver image classification and recognition. The simulation shows that this paper can effectively classify the normal liver, liver cirrhosis, and liver cancer ultrasound images. Compared with the traditional immune algorithm, this paper combines the immune algorithm with the support vector machine, and the optimized immune algorithm can effectively improve the performance of ultrasonic liver image classification and recognition.

Optimizing TensorFlow Performance by Reconstructing the Convolution Routine

Using deep learning, we can currently build computational models composed of multiple processing layers to learn representations of data. Convolutional neural networks (CNNs) have been widely adopted to achieve significant performance in image recognition and classification. TensorFlow, an open-source deep learning framework from Google, uses profiling to select one convolution algorithm, from among several available, as the core of a CNN to deliver the best performance in terms of execution time and memory usage. However, the overhead from profiling is considerably significant, because TensorFlow executes and profiles all the available algorithms for the best selection whenever an application is launched. We observe that memory usage overshoots during profiling, which limits data parallelism, and thus, fails to deliver maximum performance. In this paper, we present a novel profiling method to reduce overhead by storing the profile result from the first run and reusing it from the second run on. Using Inception-V3, we achieved up to 1.12 times and 1.11 times higher throughput, compared to the vanilla TensorFlow and TensorFlow with XLA JIT compilation, respectively, without losing accuracy.

Connected Components-based Colour Image Representations of Vibrations for a Two-stage Fault Diagnosis of Roller Bearings Using Convolutional Neural Networks

Roller bearing failure is one of the most common faults in rotating machines. Various techniques for bearing fault diagnosis based on faults feature extraction have been proposed. But feature extraction from fault signals requires expert prior information and human labour. Recently, deep learning algorithms have been applied extensively in the condition monitoring of rotating machines to learn features automatically from the input data. Given its robust performance in image recognition, the convolutional neural network (CNN) architecture has been widely used to learn automatically discriminative features from vibration images and classify health conditions. This paper proposes and evaluates a two-stage method RGBVI-CNN for roller bearings fault diagnosis. The first stage in the proposed method is to generate the RGB vibration images (RGBVIs) from the input vibration signals. To begin this process, first, the 1-D vibration signals were converted to 2-D grayscale vibration Images. Once the conversion was completed, the regions of interest (ROI) were found in the converted 2-D grayscale vibration images. Finally, to produce vibration images with more discriminative characteristics, an algorithm was applied to the 2-D grayscale vibration images to produce connected components-based RGB vibration images (RGBVIs) with sets of colours and texture features. In the second stage, with these RGBVIs a CNN-based architecture was employed to learn automatically features from the RGBVIs and to classify bearing health conditions. Two cases of fault classification of rolling element bearings are used to validate the proposed method. Experimental results of this investigation demonstrate that RGBVI-CNN can generate advantageous health condition features from bearing vibration signals and classify the health conditions under different working loads with high accuracy. Moreover, several classification models trained using RGBVI-CNN offered high performance in the testing results of the overall classification accuracy, precision, recall, and F-score.

TsanKit: artificial intelligence for solder ball head-in-pillow defect inspection

In this paper, we propose an AI (Artificial Intelligence) solution for solder ball HIP (Head-In-Pillow) defect inspection. The HIP defect will affect the conductivity of the solder balls leading to intermittent failures. Due to the variable location and shape of the HIP defect, traditional machine vision algorithms cannot solve the problem completely. In recent years, Convolutional Neural Network (CNN) has an outstanding performance in image recognition and classification, but it is easy to cause overfitting problems due to insufficient data. Therefore, we combine CNN and the machine learning algorithm Support Vector Machine (SVM) to design our inspection process. Referring to the advantages of several state-of-the-art models, we propose our 3D CNN model and adopt focal loss as well as triplet loss to solve the data imbalance problem caused by rare defective data. Our inspection method has the best performance and fast testing speed compared with several classic CNN models and the deep learning inspection software SuaKIT.

The Use of Deep Learning and VR Technology in Film and Television Production From the Perspective of Audience Psychology.

As the development of artificial intelligence (AI) technology, the deep-learning (DL)-based Virtual Reality (VR) technology, and DL technology are applied in human-computer interaction (HCI), and their impacts on modern film and TV works production and audience psychology are analyzed. In film and TV production, audiences have a higher demand for the verisimilitude and immersion of the works, especially in film production. Based on this, a 2D image recognition system for human body motions and a 3D recognition system for human body motions based on the convolutional neural network (CNN) algorithm of DL are proposed, and an analysis framework is established. The proposed systems are simulated on practical and professional datasets, respectively. The results show that the algorithm's computing performance in 2D image recognition is 7–9 times higher than that of the Open Pose method. It runs at 44.3 ms in 3D motion recognition, significantly lower than the Open Pose method's 794.5 and 138.7 ms. Although the detection accuracy has dropped by 2.4%, it is more efficient and convenient without limitations of scenarios in practical applications. The AI-based VR and DL enriches and expands the role and application of computer graphics in film and TV production using HCI technology theoretically and practically.