Traditional Image Processing Algorithms Research Articles

Pantograph Spark Fault Detection using YOLO

Pantograph-catenary is now the dominant form of current collection for modern electric trains because they can be used for higher voltages. Faults in pantograph-catenary systems threaten the operation and safety of railway transportation. They need to be continuously monitored and controlled to maintain safe transport. Pantograph may be damaged as a result of extreme weather conditions which can affect its normal operation, leading to failure of pantograph and overhead contact line systems. Poor contact between pantograph and overhead contact line causes thermal erosion to the wire. When the pantographs are exposed to air, they could deteriorate due to electrochemical reaction with the environment since they are made of metals. Movement of catenary lines and pantograph in high crosswinds has been found to cause the wire to be trapped in the pantograph. There is a serious issue regarding the quality of images generated by pantograph video monitoring system on high-speed railway trains which often shows inconsistencies of catenary faults. The application of traditional image processing and deep learning techniques have been unable to meet the requirements of spark detection. In this paper, a modern deep learning algorithm is proposed to detect sparks in the pantograph. Specifically, the YOLOv3 model is used to counter this problem that traditional image processing algorithms have been unable to. The results on a very large sample of data show the efficiency and real-time performance of the proposed method, which meets the requirements of pantograph spark detection in high-speed railway. Keywords : High-speed railway pantograph; Spark detection; Deep learning; YOLOv3; DOI: 10.7176/ISDE/12-3-02 Publication date: September 30 th 2021

Feature Extraction with Apparent to Semantic Channels for Object Detection

This paper focuses on using traditional image processing algorithms with some apparent-to-semantic features to improve the detection accuracy. Based on the optimization of Faster R-CNN algorithm, a mainstream framework in current object detection scenario, the multi-channel features are achieved by combining traditional image semantic feature algorithms (like Integral Channel Feature (ICF), Histograms of Gradient (HOG), Local Binary Pattern (LBF), etc.) and advanced semantic feature algorithms (like segmentation, heatmap, etc.). In order to realize the joint training of the original image and the above feature extraction algorithms, a unique network for increasing the accuracy of object detection and minimizing system weight called Multi-Channel Feature Network (MCFN) is proposed. The function of MCFN is to provide a multi-channel interface, which is not limited to the RGB component of a single picture, nor to the number of input channels. The experimental result shows the relationship between the number of additional channels, performance of model and accuracy. Compared with the basic Faster R-CNN structure, this result is based on the case of two additional channels. And the universal Mean Average Precision (mAP) can be improved by 2%-3%. When the number of extra channels is increased, the accuracy will not increase linearly. In fact, system performance starts to fluctuate in a range after the number of additional channels reaches six.

Human-Mimetic Estimation of Food Volume from a Single-View RGB Image Using an AI System.

It is well known that many chronic diseases are associated with unhealthy diet. Although improving diet is critical, adopting a healthy diet is difficult despite its benefits being well understood. Technology is needed to allow an assessment of dietary intake accurately and easily in real-world settings so that effective intervention to manage being overweight, obesity, and related chronic diseases can be developed. In recent years, new wearable imaging and computational technologies have emerged. These technologies are capable of performing objective and passive dietary assessments with a much simplified procedure than traditional questionnaires. However, a critical task is required to estimate the portion size (in this case, the food volume) from a digital image. Currently, this task is very challenging because the volumetric information in the two-dimensional images is incomplete, and the estimation involves a great deal of imagination, beyond the capacity of the traditional image processing algorithms. In this work, we present a novel Artificial Intelligent (AI) system to mimic the thinking of dietitians who use a set of common objects as gauges (e.g., a teaspoon, a golf ball, a cup, and so on) to estimate the portion size. Specifically, our human-mimetic system “mentally” gauges the volume of food using a set of internal reference volumes that have been learned previously. At the output, our system produces a vector of probabilities of the food with respect to the internal reference volumes. The estimation is then completed by an “intelligent guess”, implemented by an inner product between the probability vector and the reference volume vector. Our experiments using both virtual and real food datasets have shown accurate volume estimation results.

Fracture Extraction from Smooth Rock Surfaces Using Depth Image Segmentation

Due to the diversity of mineral particle features in rock, including sizes, shapes, arrangements, and structural connections, the color image of the rock sample surface at laboratory scale is quite complex. Thus, unacceptable misjudgement and information loss often occur when the traditional image processing algorithms are adopted. To improve the accuracy of fracture extraction, a method based on image processing algorithms is proposed in this paper to extract fractures from 3D point clouds. First, a high-precision original depth image is generated by gridding the rock surface point cloud data with the Kriging interpolation. The hill shading method is then applied to further clarify the fractures. Finally, the fractures are extracted from depth images and compared with color images. The results show that the integrity of the fracture skeleton is significantly improved and the error rate is reduced. In combination with color images, the physical fractures and other fracture-like features can be distinguished. The proposed method provides a new idea for extracting fractures in various destructive experiments on rocks, and may be developed for recognition of discontinuities at typical engineering scale.

Automotive Brake Part Inspection and Fault Localization using Deep Learning

Inspection of brake components is very essential to detect the damaged manufactured parts before it is assembled in any vehicle. Manual inspection of brakes is extremely difficult since most of defects are very minute and cannot be identified by human eyes. Therefore, automatic inspection of manufactured brakes is indispensible to prevent failure of brakes and accidents. Previously, various research articles perform inspection of brake through conventional image processing and traditional image processing algorithms. However, these techniques are capable of identifying a single fault only and are less robust to detecting numerous faults. Further, the existing techniques hardly localize the exact location of faults in the surface of brake. In order to over these drawbacks, in this research we utilize deep learning object detection algorithms namely Single Shot Detector and Faster RCNN to identify and localize the exact location of fault on the brake surface. Furthermore, the proposed system is capable to detect different types of faults in a single algorithm and is robust to brake’s material surface, environmental and lightening factors. The deep learning algorithms are trained using transfer learning on custom collected dataset. The proposed algorithms deliver an accuracy of 95.64% and mAP of 73.2% on cylindrical grey shade brakes.

Image processing based on the detection of external defects of fan tower Weld

With the continuous development of wind power generation technology and the continuous increase in the demand for electric energy, the height of the fan tower is more and more demanding. It is very important to detect the weld produced in the welding process of fan tower. In this paper, an algorithm for weld defect detection based on traditional image processing and convolutional neural network is proposed. Firstly, the traditional image processing algorithm is used to gray the weld image collected by industrial camera. Then, the gray image of welding seam is enhanced to improve the visual effect and clear the image, which is convenient for further processing and analysis of the image by computer. Finally, the image is used as the input of the trained convolution neural network to judge whether there are defects outside the weld.

Identification Algorithm Based on Key-Point Detection Network for Vital Parts of Infrared Aerial Target

The precision strike capability of an infrared-guided air-to-air missile to target the vital parts of a fighter is key to precision-guidance weapons. The traditional image processing algorithms select features and designs classifiers according to human prior knowledge, but this has some limitations. Therefore we propose an algorithm for identifying the vital parts of an infrared aerial target based on key-point detection networks. The algorithm uses the end-to-end deep learning network architecture and combines illumination with texture. The data set is augmented and enhanced in terms of lighting, texture and deformation. The entire image information is preprocessed simply as input, and a loss function with constraints is constructed and iterated with an optimization algorithm. Compared with the conventional algorithms with the same training, the average recognition rate of the trained network model increases by 10%. The vital parts of the infrared aerial target are identified at the speed of ≤ 10 ms/frame. The accuracy of recognition of the 4 vital parts proposed by us is more than 80%.

Neural Network Classification of Ice-Crystal Images Observed by an Airborne Cloud Imaging Probe

ABSTRACT In the atmosphere, cloud particles have different shapes. The study of cloud particle shapes plays an important role in understanding cloud precipitation processes, radiative transfer, and weather modification. The image resolution and data quality of cloud probes affect the accuracy of the classification of particle shapes. To solve the occlusion of the photosensitive edge of the particle image and achieve automatic, high-precision ice-crystal classification of airborne Cloud Imaging Probe (CIP) ice-crystal images, this study uses a traditional image processing algorithm for data quality control and applies artificial intelligence algorithms to classify ice-crystal images. At present, there are mainly two types of ice-crystal classification methods, one classifies the shape of ice crystals using a pattern parameterization scheme, and the other uses an artificial intelligence network model to classify the shape. Combined with data quality control, the dataset was tested on eight models, and the TL-EfficientNet-b6 model was found to be the most accurate. Therefore, the TL-EfficientNet-b6 classifier model was used in this study, which is a newly developed convolutional neural network (CNN) based on a transfer learning method. Experimental results show that the TL-EfficientNet-b6 model can reach 100% in the single-class precision of tiny and hexagonal ice crystals, and the average precision can reach 98%. These results are more accurate than those using traditional classification methods. This method could be valuable in cloud microphysics research and weather modification.

Gaussian Process Model of Surrounding Rock Classification Based on Digital Characterization of Rock Mass Structure and Its Application

As the important data basis of surrounding rock classification, rock mass structural information obtained by traditional image processing and feature extraction algorithms could not be quantitatively analyzed because of the uncertainty and geometric randomness of structural planes. In this paper, based on straight line detection, intelligent scissors, and morphological edge detection algorithms, the multiple interpretation system of rock mass image including linear bunching extraction, magnetic tracking extraction, and multiparameter characterization was researched and developed, and the actual distribution information and the related probability distribution model of structural planes could be obtained directly. On the basis of this, plenty of corresponding random rating-values meeting the probability distribution models of these evaluation indices were gained by Monte Carlo Simulation. The distribution probability affiliated with different rock mass grade was attained by inductive statistics, and the robust evaluation of surrounding rock classification could be carried out. Taking the robust results as learning samples, the response model of surrounding rock grade based on Gaussian process classification was established, making the evaluation of surrounding rock subclassification more rapid and robust.

Semantic segmentation of HeLa cells: An objective comparison between one traditional algorithm and four deep-learning architectures.

The quantitative study of cell morphology is of great importance as the structure and condition of cells and their structures can be related to conditions of health or disease. The first step towards that, is the accurate segmentation of cell structures. In this work, we compare five approaches, one traditional and four deep-learning, for the semantic segmentation of the nuclear envelope of cervical cancer cells commonly known as HeLa cells. Images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and four three deep learning architectures: VGG16, ResNet18, Inception-ResNet-v2, and U-Net. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The first three deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The U-Net architecture was trained from scratch with 36, 000 training images and labels of size 128 × 128. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%), and U-Net (92%, 56%).

Stress Distribution Analysis on Hyperspectral Corn Leaf Images for Improved Phenotyping Quality.

High-throughput imaging technologies have been developing rapidly for agricultural plant phenotyping purposes. With most of the current crop plant image processing algorithms, the plant canopy pixels are segmented from the images, and the averaged spectrum across the whole canopy is calculated in order to predict the plant’s physiological features. However, the nutrients and stress levels vary significantly across the canopy. For example, it is common to have several times of difference among Soil Plant Analysis Development (SPAD) chlorophyll meter readings of chlorophyll content at different positions on the same leaf. The current plant image processing algorithms cannot provide satisfactory plant measurement quality, as the averaged color cannot characterize the different leaf parts. Meanwhile, the nutrients and stress distribution patterns contain unique features which might provide valuable signals for phenotyping. There is great potential to develop a finer level of image processing algorithm which analyzes the nutrients and stress distributions across the leaf for improved quality of phenotyping measurements. In this paper, a new leaf image processing algorithm based on Random Forest and leaf region rescaling was developed in order to analyze the distribution patterns on the corn leaf. The normalized difference vegetation index (NDVI) was used as an example to demonstrate the improvements of the new algorithm in differentiating between different nitrogen stress levels. With the Random Forest method integrated into the algorithm, the distribution patterns along the corn leaf’s mid-rib direction were successfully modeled and utilized for improved phenotyping quality. The algorithm was tested in a field corn plant phenotyping assay with different genotypes and nitrogen treatments. Compared with the traditional image processing algorithms which average the NDVI (for example) throughout the whole leaf, the new algorithm more clearly differentiates the leaves from different nitrogen treatments and genotypes. We expect that, besides NDVI, the new distribution analysis algorithm could improve the quality of other plant feature measurements in similar ways.

Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales

Grain number is crucial for analysis of yield components and assessment of effects of cultivation measures. The grain number per spike and thousand-grain weight can be measured by counting grains manually, but it is time-consuming, tedious and error-prone. Previous image processing algorithms cannot work well with different backgrounds and different sizes. This study used deep learning methods to resolve the limitations of traditional image processing algorithms. Wheat grain image datasets were collected in the scenarios of three varieties, six background and two image acquisition devices with different heights, angles and grain numbers, 1 748 images in total. All images were processed through color space conversion, image flipping and rotation. The grain was manually annotated, and the datasets were divided into training set, validation set and test set. We used the TensorFlow framework to construct the Faster Region-based Convolutional Neural Network Model. Using the transfer learning method, we optimized the wheat grain detection and enumeration model. The total loss of the model was less than 0.5 and the mean average precision was 0.91. Compared with previous grain counting algorithms, the grain counting error rate of this model was less than 3% and the running time was less than 2 s. The model can be effectively applied under a variety of backgrounds, image sizes, grain sizes, shooting angles, and shooting heights, as well as different levels of grain crowding. It constitutes an effective detection and enumeration tool for wheat grain. This study provides a reference for further grain testing and enumeration applications.

Research and Prospect of Image Recognition Based on Convolutional Neural Network

This paper compares common image recognition technology, introduces three image recognition and classification technologies, and explains the most popular deep learning image recognition algorithm based on CNN in detail. As a new method of ANN, CNN combines ANN and deep learning technology. In the area of image recognition, the advantage of CNN is particularly prominent. Compared with the traditional image processing algorithm, CNN has higher recognition accuracy and speed, and can directly process the original image to avoid the complicated pre-processing image data. CNN is the first choice for image recognition.

AK-DL: A Shallow Neural Network Model for Diagnosing Actinic Keratosis with Better Performance Than Deep Neural Networks.

Actinic keratosis (AK) is one of the most common precancerous skin lesions, which is easily confused with benign keratosis (BK). At present, the diagnosis of AK mainly depends on histopathological examination, and ignorance can easily occur in the early stage, thus missing the opportunity for treatment. In this study, we designed a shallow convolutional neural network (CNN) named actinic keratosis deep learning (AK-DL) and further developed an intelligent diagnostic system for AK based on the iOS platform. After data preprocessing, the AK-DL model was trained and tested with AK and BK images from dataset HAM10000. We further compared it with mainstream deep CNN models, such as AlexNet, GoogLeNet, and ResNet, as well as traditional medical image processing algorithms. Our results showed that the performance of AK-DL was better than the mainstream deep CNN models and traditional medical image processing algorithms based on the AK dataset. The recognition accuracy of AK-DL was 0.925, the area under the receiver operating characteristic curve (AUC) was 0.887, and the training time was only 123.0 s. An iOS app of intelligent diagnostic system was developed based on the AK-DL model for accurate and automatic diagnosis of AK. Our results indicate that it is better to employ a shallow CNN in the recognition of AK.

Model Watermarking for Image Processing Networks

Deep learning has achieved tremendous success in numerous industrial applications. As training a good model often needs massive high-quality data and computation resources, the learned models often have significant business values. However, these valuable deep models are exposed to a huge risk of infringements. For example, if the attacker has the full information of one target model including the network structure and weights, the model can be easily finetuned on new datasets. Even if the attacker can only access the output of the target model, he/she can still train another similar surrogate model by generating a large scale of input-output training pairs. How to protect the intellectual property of deep models is a very important but seriously under-researched problem. There are a few recent attempts at classification network protection only.In this paper, we propose the first model watermarking framework for protecting image processing models. To achieve this goal, we leverage the spatial invisible watermarking mechanism. Specifically, given a black-box target model, a unified and invisible watermark is hidden into its outputs, which can be regarded as a special task-agnostic barrier. In this way, when the attacker trains one surrogate model by using the input-output pairs of the target model, the hidden watermark will be learned and extracted afterward. To enable watermarks from binary bits to high-resolution images, both traditional and deep spatial invisible watermarking mechanism are considered. Experiments demonstrate the robustness of the proposed watermarking mechanism, which can resist surrogate models learned with different network structures and objective functions. Besides deep models, the proposed method is also easy to be extended to protect data and traditional image processing algorithms.

Research and application of orthotopic DR chest radiograph quality control system based on artificial intelligence

With the change of medical diagnosis and treatment mode, the quality of medical image directly affects the diagnosis and treatment of the disease for doctors. Therefore, realization of intelligent image quality control by computer will have a greater auxiliary effect on the radiographer's filming work. In this paper, the research methods and applications of image segmentation model and image classification model in the field of deep learning and traditional image processing algorithm applied to medical image quality evaluation are described. The results demonstrate that deep learning algorithm is more accurate and efficient than the traditional image processing algorithm in the effective training of medical image big data, which explains the broad application prospect of deep learning in the medical field. This paper developed a set of intelligent quality control system for auxiliary filming, and successfully applied it to the Radiology Department of West China Hospital and other city and county hospitals, which effectively verified the feasibility and stability of the quality control system.

An Efficient and Accurate DDPG-Based Recurrent Attention Model for Object Localization

Using image processing algorithm to localize objects, which lack specific patterns and local features, has always been the research focus in industrial production. Compared with the traditional image processing algorithm, RAM (Recurrent Attention Model) in deep learning not only shows advantages in positioning accuracy and stability, but also has good adaptability in situations such as occlusion. However, RAM contains policy gradient (PG) algorithm, which is unstable in training process and has low convergence efficiency. To overcome this shortcoming, in order to improve the learning efficiency and stability of RAM, this paper proposes DDPG-based RAM. In addition, current random sampling algorithm in DDPG (Deep Deterministic Policy Gradient) does not make full use of the information contained in samples. Some samples are repeatedly learned, which slows down the convergence rate of the neural network model, and even causes the model to converge to the local optimal solution. To solve the above problems, a prioritized experience replay algorithm based on Gaussian sampling method is proposed. By constructing the localization and grasping simulation environment in V-rep, it is shown that compared with the traditional image algorithm, the proposed model algorithm in this paper has a greater improvement in localization accuracy, stability and model convergence speed.

One class based feature learning approach for defect detection using deep autoencoders

Detecting defects is an integral part of any manufacturing process. Most works still utilize traditional image processing algorithms to detect defects owing to the complexity and variety of products and manufacturing environments. In this paper, we propose an approach based on deep learning which uses autoencoders for extraction of discriminative features. It can detect different defects without using any defect samples during training. This method, where samples of only one class (i.e. defect-free samples) are available for training, is called One Class Classification (OCC). This OCC method can also be used for training a neural network when only one golden sample is available by generating many copies of the reference image by data augmentation. The trained model is then able to generate a descriptor—a unique feature vector of an input image. A test image captured by an Automatic Optical Inspection (AOI) camera is sent to the trained model to generate a test descriptor, which is compared with a reference descriptor to obtain a similarity score. After comparing the results of this method with a popular traditional similarity matching method SIFT, we find that in the most cases this approach is more effective and more flexible than the traditional image processing-based methods, and it can be used to detect different types of defects with minimum customization.

Benchmarking Image Processing Algorithms for Unmanned Aerial System-Assisted Crack Detection in Concrete Structures

This paper summarizes the results of traditional image processing algorithms for detection of defects in concrete using images taken by Unmanned Aerial Systems (UASs). Such algorithms are useful for improving the accuracy of crack detection during autonomous inspection of bridges and other structures, and they have yet to be compared and evaluated on a dataset of concrete images taken by UAS. The authors created a generic image processing algorithm for crack detection, which included the major steps of filter design, edge detection, image enhancement, and segmentation, designed to uniformly compare different edge detectors. Edge detection was carried out by six filters in the spatial (Roberts, Prewitt, Sobel, and Laplacian of Gaussian) and frequency (Butterworth and Gaussian) domains. These algorithms were applied to fifty images each of defected and sound concrete. Performances of the six filters were compared in terms of accuracy, precision, minimum detectable crack width, computational time, and noise-to-signal ratio. In general, frequency domain techniques were slower than spatial domain methods because of the computational intensity of the Fourier and inverse Fourier transformations used to move between spatial and frequency domains. Frequency domain methods also produced noisier images than spatial domain methods. Crack detection in the spatial domain using the Laplacian of Gaussian filter proved to be the fastest, most accurate, and most precise method, and it resulted in the finest detectable crack width. The Laplacian of Gaussian filter in spatial domain is recommended for future applications of real-time crack detection using UAS.

Detection and segmentation of iron ore green pellets in images using lightweight U-net deep learning network

In steel manufacturing industry, powdered iron ore is agglomerated in a pelletizing disk to form iron ore green pellets. The agglomeration process is usually monitored using a camera. As pellet size distribution is one of the major measures of product quality monitoring, pellets detection and segmentation from the image are the key steps to determine the pellet size. Traditional image processing algorithms are not only challenged by the complicated constitution of pellets, sediment and residuals in the image, but also by the harsh and unbalanced light reflection on the pellet centrum area and the background which results in tedious parameter adjustment work and pool performance. To solve these problems, we design a lightweight U-net deep learning network to automatically detect pellets from images and to obtain the probability maps of pellet contours. Compared to classic U-net, the proposed network has fewer parameters and introduces batch normalization layers, which greatly reduces the computing time and improves generalization ability of the network. A concentric circle model is then used to separate clumped contours of the pellets, and the pellets shapes are detected via ellipse fitting. The proposed method is verified using images captured from an industrial pelletizing disk, and its performance is compared with traditional methods and the classic U-net. Results show that the proposed method achieves better segmentation performance in DICE and ROC indexes and shows good robustness to uneven illumination. Tests on temporal image sequences demonstrate that the proposed method is effective in monitoring the pellet size distribution and the pellet shape as well. Results of this work have potential usage in online detection of iron ore green pellets and other types of particles.