Convolutional Neural Network For Recognition Research Articles

Visual manipulation relationship recognition in object-stacking scenes

• A new CNN architecture to recognize manipulation relationships. • A new dataset for visual manipulation relationship (VMR) recognition. • The first end-to-end CNN for VMR recognition based on RGB images. • (a) In real-world robotic grasping, visual manipulation relationships (VMR) are important for plan making to prevent potential damages (e.g. when neglecting VMRs, if the robot grasps the book first, the cup upon it will fall over and may be broken). • (b) We present a new convolutional neural network (CNN) architecture called Visual Manipulation Relationship Network (VMRN) to help robots detect targets and recognize the VMRs in real time. Object manipulation in object-stacking scenes is a significant but challenging skill for intelligent robots . In most cases, the relationships among objects should be considered before manipulation to prevent chaos and damages. However, the analysis of object relationships in object-stacking scenes, especially for robotic manipulation, remains to be unsolved. To this end, this paper presents a new convolutional neural network (CNN) architecture, called Visual Manipulation Relationship Network (VMRN), to recognize the visual manipulation relationships (VMR) between objects in real-time. By considering the manipulation relationships in object-stacking scenes, it ensures that the robot can complete manipulation tasks safely and reliably. The core of our model is the Object Pairing Pooling Layer (OP 2 L), which makes it possible to recognize objects and all possible VMRs in one forward process. Moreover, to train VMRN, we contribute a dataset named Visual Manipulation Relationship Dataset (VMRD) consisting of 4683 images with more than 16,000 object instances and the VMRs between each object pair. The experimental results show that the proposed network architecture can detect objects and predict VMRs.

CAPTCHA Recognition Using Deep Learning with Attached Binary Images

Websites can increase their security and prevent harmful Internet attacks by providing CAPTCHA verification for determining whether end-user is a human or a robot. Text-based CAPTCHA is the most common and designed to be easily recognized by humans and difficult to identify by machines or robots. However, with the dramatic advancements in deep learning, it becomes much easier to build convolutional neural network (CNN) models that can efficiently recognize text-based CAPTCHAs. In this study, we introduce an efficient CNN model that uses attached binary images to recognize CAPTCHAs. By making a specific number of copies of the input CAPTCHA image equal to the number of characters in that input CAPTCHA image and attaching distinct binary images to each copy, we build a new CNN model that can recognize CAPTCHAs effectively. The model has a simple structure and small storage size and does not require the segmentation of CAPTCHAs into individual characters. After training and testing the proposed CAPTCHA recognition CNN model, the achieved experimental results reveal the strength of the model in CAPTCHA character recognition.

Landslide detection by deep learning of non-nadiral and crowdsourced optical images

The recent development of mobile surveying platforms and crowdsourced geoinformation has produced a huge amount of non-validated data that are now available for research and application. In the field of risk analysis, with particular reference to landslide hazard, images generated by autonomous platforms (such as UAVs, ground-based acquisition systems, satellite sensors) and pictures obtained from web data mining are easily gathered and contribute to the fast surge in the amount of non-organized information that may engulf data storage facilities. Therefore, the high potential impact of such methods is severely reduced by the need of a massive amount of human intelligence tasks (HITs), which is necessary to filter and classify the data, whatever the final purpose. In this work, we present a new set of convolutional neural networks (CNNs) specifically designed for the automated recognition of landslides and mass movements in non-standard pictures that can be used in automated image classification, in supporting UAV autonomous guidance and in the filtering of data-mined information. Computer vision can be of great help in fostering the autonomous capability of intelligent systems to complement, or completely substitute, HITs. Image and object recognition are at the forefront of this research field. The deep learning procedure has been accomplished by applying transfer learning to some of the top-performer CNNs available in the literature. Results show that the deep learning machines, calibrated on a relevant dataset of validated images of landforms, may supply reliable predictions with computational time and resource requirements compatible with most of the UAV platforms and web data mining applications in landslide hazard studies. Average accuracy achieved by the proposed methods ranges between 87 and 90% and is consistently higher than that obtained by general-purpose state-of-the-art image recognition convolutional neural networks. The method can be applied to early warning, vulnerability assessment, residual risk estimation, model parameterisation and landslide mapping. Specific advantages will be the reduction of the present limitations in the intelligent guidance of landslide mapping drones, the classification of fake news, the validation of post-disaster information and the correct interpretation of an impending change in the environment.

Landslide Recognition by Deep Convolutional Neural Network and Change Detection

It is a technological challenge to recognize landslides from remotely sensed (RS) images automatically and at high speeds, which is fundamentally important for preventing and controlling natural landslide hazards. Many methods have been developed, but there remains room for improvement for stable, higher accuracy, and high-speed landslide recognition for large areas with complex land cover. In this article, a novel integrated approach combining a deep convolutional neural network (CNN) and change detection is proposed for landslide recognition from RS images. Logically, it comprises the following four parts. First, a CNN for landslide recognition is built based on training data sets from RS images with historical landslides. Second, the object-oriented change detection CNN (CDCNN) with a fully connected conditional random field (CRF) is implemented based on the trained CNN. Third, the preliminary CDCNN is optimized by the proposed postprocessing methods. Finally, the results are further enhanced by a set of information extraction methods, including trail extraction, source point extraction, and attribute extraction. Furthermore, in the implementation of the proposed approach, image block processing and parallel processing strategies are adopted. As a result, the speed has been improved significantly, which is extremely important for RS images covering large areas. The effectiveness of the proposed approach has been examined using two landslide-prone sites, Lantau Island and Sharp Peak, Hong Kong, with a total area of more than 70 km <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . Besides its high speed, the proposed approach has an accuracy exceeding 80%, and the experiments demonstrate its high practicability.

Model Architecture of CNN for Recognition the Pandava Mask

This research was conducted to observe the use of architectural model Convolutional Neural Networks (CNN) LeNEt, which was suitable to use for Pandava mask objects. The Data processing in the research was 200 data for each class or similar with 1000 trial data. Architectural model CNN LeNET used input layer 32x32, 64x64, 128x128, 224x224 and 256x256. The trial result with the input layer 32x32 succeeded, showing a faster time compared to the other layer. The result of accuracy value and validation was not under fitted or overfit. However, when the activation of the second dense process as changed from the relu to sigmoid, the result was better in sigmoid, in the tem of time, and the possibility of overfitting was less. The research result had a mean accuracy value of 0.96.

A deep learning approach to integrate convolutional neural networks in speaker recognition

We propose a novel usage of convolutional neural networks (CNNs) for the problem of speaker recognition. While being particularly designed for computer vision problems, CNNs have recently been applied for speaker recognition by using spectrograms as input images. We believe that this approach is not optimal as it may result in two cumulative errors in solving both a computer vision and a speaker recognition problem. In this work, we aim at integrating CNNs in speaker recognition without relying on images. We use Restricted Boltzmann Machines (RBMs) to extract speakers models as matrices and introduce a new way to model target and non-target speakers, in order to perform speaker verification. Thus, we use a CNN to discriminate between target and non-target matrices. Experiments were conducted with the THUYG-20 SRE corpus under three noise conditions: clean, 9 db, and 0 db. The results demonstrate that our method outperforms the state-of-the-art approaches by decreasing the error rate by up to 60%.

Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform

In this paper, a real-time signal processing frame-work based on a 60 GHz frequency-modulated continuous wave (FMCW) radar system to recognize gestures is proposed. In order to improve the robustness of the radar-based gesture recognition system, the proposed framework extracts a comprehensive hand profile, including range, Doppler, azimuth and elevation, over multiple measurement-cycles and encodes them into a feature cube. Rather than feeding the range-Doppler spectrum sequence into a deep convolutional neural network (CNN) connected with recurrent neural networks, the proposed framework takes the aforementioned feature cube as input of a shallow CNN for gesture recognition to reduce the computational complexity. In addition, we develop a hand activity detection (HAD) algorithm to automatize the detection of gestures in real-time case. The proposed HAD can capture the time-stamp at which a gesture finishes and feeds the hand profile of all the relevant measurement-cycles before this time-stamp into the CNN with low latency. Since the proposed framework is able to detect and classify gestures at limited computational cost, it could be deployed in an edge-computing platform for real-time applications, whose performance is notedly inferior to a state-of-the-art personal computer. The experimental results show that the proposed framework has the capability of classifying 12 gestures in real-time with a high F1-score.

Comparison of 3 deep learning neural networks for classifying the relationship between the mandibular third molar and the mandibular canal on panoramic radiographs.

The aim of this study was to compare time and storage space requirements, diagnostic performance, and consistency among 3 image recognition convolutional neural networks (CNNs) in the evaluation of the relationships between the mandibular third molar and the mandibular canal on panoramic radiographs. Of 600 panoramic radiographs, 300 each were assigned to noncontact and contact groups based on the relationship between the mandibular third molar and the mandibular canal. The CNNs were trained twice by using cropped image patches with sizes of 70 × 70 pixels and 140 × 140 pixels. Time and storage space were measured for each system. Accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUC) were determined. Intra-CNN and inter-CNN consistency values were calculated. Time and storage space requirements depended on the depth of CNN layers and number of learned parameters, respectively. The highest AUC values ranged from 0.88 to 0.93 in the CNNs created by 70 × 70 pixel patches, but there were no significant differences in diagnostic performance among any of the models with smaller patches. Intra-CNN and inter-CNN consistency values were good or very good for all CNNs. The size of the image patches should be carefully determined to ensure acquisition of high diagnostic performance and consistency.

A Review of Convolutional Neural Network Applied to Fruit Image Processing

Agriculture has always been an important economic and social sector for humans. Fruit production is especially essential, with a great demand from all households. Therefore, the use of innovative technologies is of vital importance for the agri-food sector. Currently artificial intelligence is one very important technological tool widely used in modern society. Particularly, Deep Learning (DL) has several applications due to its ability to learn robust representations from images. Convolutional Neural Networks (CNN) is the main DL architecture for image classification. Based on the great attention that CNNs have had in the last years, we present a review of the use of CNN applied to different automatic processing tasks of fruit images: classification, quality control, and detection. We observe that in the last two years (2019–2020), the use of CNN for fruit recognition has greatly increased obtaining excellent results, either by using new models or with pre-trained networks for transfer learning. It is worth noting that different types of images are used in datasets according to the task performed. Besides, this article presents the fundamentals, tools, and two examples of the use of CNNs for fruit sorting and quality control.

Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer.

The interaction between humans and an NAO robot using deep convolutional neural networks (CNN) is presented in this paper based on an innovative end-to-end pipeline method that applies two optimized CNNs, one for face recognition (FR) and another one for the facial expression recognition (FER) in order to obtain real-time inference speed for the entire process. Two different models for FR are considered, one known to be very accurate, but has low inference speed (faster region-based convolutional neural network), and one that is not as accurate but has high inference speed (single shot detector convolutional neural network). For emotion recognition transfer learning and fine-tuning of three CNN models (VGG, Inception V3 and ResNet) has been used. The overall results show that single shot detector convolutional neural network (SSD CNN) and faster region-based convolutional neural network (Faster R-CNN) models for face detection share almost the same accuracy: 97.8% for Faster R-CNN on PASCAL visual object classes (PASCAL VOCs) evaluation metrics and 97.42% for SSD Inception. In terms of FER, ResNet obtained the highest training accuracy (90.14%), while the visual geometry group (VGG) network had 87% accuracy and Inception V3 reached 81%. The results show improvements over 10% when using two serialized CNN, instead of using only the FER CNN, while the recent optimization model, called rectified adaptive moment optimization (RAdam), lead to a better generalization and accuracy improvement of 3%-4% on each emotion recognition CNN.

A CNN recognition method for early stage of 10 kV single core cable based on sheath current

Traditional analysis of cable early state recognition is mainly based on one or several threshold values of electrical characteristics, but the calculation of threshold is often affected by measurement accuracy and external disturbance, which inevitably reduces recognition accuracy. The development of artificial intelligence provides a new way to solve this problem. This paper presents a deep convolutional neural network (CNN) recognition method for early state of 10 kV single core cable based on sheath current. Firstly, waveform and energy characteristics which are extracted from the mutation information of sheath current by wavelet transform, are used to construct cable state recognition matrix. The mutational signal is detected by the cumulative sum (CU-SUM) method and intercepted by a set time window. Secondly, a 7-layer deep CNN is constructed according to the features of recognition matrix. Then the CNN model is trained by the adaptive moment estimation (Adam) method to get the recognition model of cable state. Finally, the proposed method is used to recognize cable early state by large number of samples which are obtained from the simulation of four cable states with PSCAD software. Compared with other methods, the results of simulation demonstrate that the proposed method has a high recognition accuracy.

Machine-Learning-Accelerated Perovskite Crystallization

Summary Perovskites have seen significant research interest in the last decade. As ternary and quaternary compounds, their chemical space is exceptionally large, yet perovskite development has been limited to a restricted set of chemical constituents often discovered through trial and error. Here, we report a high-throughput experimental framework for the discovery of new perovskite single crystals. We use machine learning (ML) to guide the sequence of ever-improved robotic synthetic trials. We perform high-throughput syntheses of perovskite single crystals with a protein crystallization robot and characterize the outcomes with the aid of convolutional neural network-based image recognition. We then use an ML model to predict the optimal conditions for the synthesis of a new perovskite single crystal, enabling us to report the first synthesis of (3-PLA)2PbCl4.This material exhibits strong blue emission, illustrating the applicability of the method in identifying new optoelectronic materials.

Real-time license plate detection and recognition using deep convolutional neural networks

Automatic License Plate Recognition (ALPR) is an important task with many applications in Intelligent Transportation and Surveillance systems. This work presents an end-to-end ALPR method based on a hierarchical Convolutional Neural Network (CNN). The core idea of the proposed method is to identify the vehicle and the license plate region using two passes on the same CNN, and then to recognize the characters using a second CNN. The recognition CNN massively explores the use of synthetic and augmented data to cope with limited training datasets, and our results show that the augmentation process significantly increases the recognition rate. In addition, we present a novel temporal coherence technique to better stabilize the OCR output in videos. Our method was tested with publicly available datasets containing Brazilian and European license plates, achieving accuracy rates better than competitive academic methods and a commercial system.

The Effect of Class Definitions on the Transferability of Adversarial Attacks Against Forensic CNNs

In recent years, convolutional neural networks (CNNs) have been widely used by researchers to perform forensic tasks such as image tampering detection. At the same time, adversarial attacks have been developed that are capable of fooling CNN-based classifiers. Understanding the transferability of adversarial attacks, i.e. an attacks ability to attack a different CNN than the one it was trained against, has important implications for designing CNNs that are resistant to attacks. While attacks on object recognition CNNs are believed to be transferrable, recent work by Barni et al. has shown that attacks on forensic CNNs have difficulty transferring to other CNN architectures or CNNs trained using different datasets. In this paper, we demonstrate that adversarial attacks on forensic CNNs are even less transferrable than previously thought even between virtually identical CNN architectures! We show that several common adversarial attacks against CNNs trained to identify image manipulation fail to transfer to CNNs whose only difference is in the class definitions (i.e. the same CNN architectures trained using the same data). We note that all formulations of class definitions contain the unaltered class. This has important implications for the future design of forensic CNNs that are robust to adversarial and anti-forensic attacks.

Identification of Abnormal Processes with Spatial-Temporal Data Using Convolutional Neural Networks

Identifying abnormal process operation with spatial-temporal data remains an important and challenging work in many practical situations. Although spatial-temporal data identification has been extensively studied in some domains, such as public health, geological condition, and environment pollution, the challenge associated with designing accurate and convenient recognition schemes is very rarely addressed in modern manufacturing processes. This paper proposes a general recognition framework for identifying abnormal process with spatial-temporal data by employing a convolutional neural network (CNN) model. Firstly, motivated by the pasting case study, the spatial-temporal data are transformed into process images for capturing spatial and temporal interrelationship. Then, the CNN recognition model is presented for identifying different types of these process images, leading to the identification of abnormal process with spatial-temporal data. The specific architecture parameters of CNN are determined step by step. According to the performance comparison with alternative methods, the proposed method is able to accurately identify the abnormal process with spatial-temporal data.

Using convolutional neural networks for recognition of objects varied in appearance in computer vision for intellectual robots

Abstract The paper describes an effort to train a convolutional neural network capable of reliably recognizing complex objects that are highly varied in their shapes and appearances in images. Neural networks show very good results on objects that have constant appearances but may have trouble recognizing abstract objects that appear in different shapes, art-styles and lack solid structure, for example, national flags. In an image, a flag may appear waving on a pole, as an element of clothes, in a form of stickers, etc. Due to these differences in appearance computer vision systems may show unsatisfactory results on these types of objects. However, detecting such objects is a necessary task in computer vision, especially for intelligent robots in order to understand the environment. The aim of the research is to apply convolutional neural networks for the detection of flags. In this research, we prepared training and testing sets of objects, trained a neural network for detection task, conducted testing experiments and measured the neural net’s performance. These results can be applied in cognitive and robotics technologies as well as general computer vision tasks.

Comparing performance of metaheuristic algorithms for finding the optimum structure of CNN for face recognition

Local and global based methods are two main trends for face recognition. Local approaches extract salient features by processing different parts of the image whereas global approaches find a general template for face of each person. Unfortunately, most global approaches work under controlled environments and they are sensitive to changes in the illumination. On the other hand, local approaches are more robust but finding their optimal parameters is a challenging task. This work proposes a new local-based approach that automatically tunes its parameters. The proposed method incorporates different techniques. In the first step, convolutional neural network (CNN) is employed as a trainable feature extraction procedure. In the second step, different metaheuristic methods are merged with CNN so that its best structure is found automatically. Finally, in the last step the decision is made by employing proper multi-class support vector machine (SVM). In this fashion a fully automated system is developed that is self-tuned and do not need manual adjustments. Simulation results demonstrate efficacy of the proposed method.

Gujarati Handwritten Character Recognition from Text Images

Today is the era of paperless office and governance. It comes with numerous advantages like increased productivity and efficiency, pervasiveness, storage optimization, robustness and eco-friendliness. Hence there is a need of converting paper documents into machine editable form. This leads to development of OCR (Optical Character Recognition). OCR is a technique to convert, mechanically or electronically an image, photo or scanned document of a handwritten text (HCR-Handwritten Character Recognition) or printed text (PCR- Printed Character Recognition) into digital text. HCR is a form of OCR that is specifically designed to recognize the handwritten text whereas PCR focuses on recognition of printed text. HCR is more challenging as compared to PCR due to diversity in human writing styles, size, curve, strokes and thickness of characters. Based on data acquisition mode, the OCR can either be online or offline. Offline recognition is performed in two ways: handwritten and printed [1]. In offline mode, the characters are on paper and captured using scanner or high-resolution camera whereas in online mode the pixel values of characters are captured by movement of cursor, pen or stylus. The HCR systems are readily available for foreign languages and many of the Indian languages like Bangla, Devanagari and Gurumukhi but for Gujarati language the HCR development is still in its infancy stage. This study focuses on development of an artificial intelligence based offline HCR system for Gujarati language. Important contribution of this study is data collection, of size 10,000 images from 250 number of people, of different age groups, of different professions. This paper describes a supervised classifier approach based on CNN (Convolutional Neural Networks) and MLP (Multi-Layer Perceptron) for recognition of handwritten Gujarati characters. A success rate of 97.21% is obtained using CNN and 64.48% using MLP. Lot of work has been done at character level, but very few has been done at word level recognition. Major focus of this study was on creating a continuous workflow for image to text conversion at word level.

Nonlinear gated channels networks for action recognition

Video-based convolutional neural networks (CNNs) involve a large amount of training parameters, leading to the enormous computational complexity, which thereby delays the network convergence. Therefore, training successful CNN for action recognition rapidly is non-trivial. In this paper, a novel encoding called nonlinear gated channels unit (NGCU) is proposed to facilitate network training by encoding the global channel-level relationship. Based on this, nonlinear gated channels network (NGCN) is constructed for the end-to-end encoding, and the corresponding convergence performance is evaluated on the standard benchmarks UCF101 and HMDB51. Experimental results demonstrate that the proposed method is conducive to the convergence process of CNN based action recognition models.

Improved softmax loss for deep learning‐based face and expression recognition

In recent years, deep convolutional neural networks (CNN) have been widely used in computer vision and significantly improved the performance of image recognition tasks. Most works use softmax loss to supervise the training of CNN and then adopt the output of last layer as features. However, the discriminative capability of the softmax loss is limited. Here, the authors analyse and improve the softmax loss by manipulating the cosine value and input feature length. As the approach does not change the principle of the softmax loss, the network can easily be optimised by typical stochastic gradient descent. The MNIST handwritten digits dataset is employed to visualise the features learned by the improved softmax loss. The CASIA-WebFace and FER2013 training set are adopted to train deep CNN for face and expression recognition, respectively. Results on both the LFW dataset and FER2013 test set show that the proposed softmax loss can learn more discriminative features and achieve better performance.