Convolutional Neural Network-based Image Research Articles

Convolutional neural network-based image watermarking using discrete wavelet transform

Convolutional neural network-based image watermarking using discrete wavelet transform

Artificial-intelligence-based decision support tools for the differential diagnosis of colitis.

Whereas Artificial Intelligence (AI) based tools have recently been introduced in the field of gastroenterology, application in inflammatory bowel disease (IBD) is in its infancies. We established AI-based algorithms to distinguish IBD from infectious and ischemic colitis using endoscopic images and clinical data. First, we trained and tested a Convolutional Neural Network (CNN) using 1796 real-world images from 494 patients, presenting with three diseases (IBD [n=212], ischemic colitis [n=157], and infectious colitis [n=125]). Moreover, we evaluated a Gradient Boosted Decision Trees (GBDT) algorithm using five clinical parameters as well as a hybrid approach (CNN + GBDT). Patients and images were randomly split into two completely independent datasets. The proposed approaches were benchmarked against each other and three expert endoscopists on the test set. For the image-based CNN, the GBDT algorithm and the hybrid approach global accuracies were .709, .792, and .766, respectively. Positive predictive values were .602, .702, and .657. Global areas under the receiver operating characteristics (ROC) and precision recall (PR) curves were .727/.585, .888/.823, and .838/.733, respectively. Global accuracy did not differ between CNN and endoscopists (.721), but the clinical parameter-based GBDT algorithm outperformed CNN and expert image classification. Decision support systems exclusively based on endoscopic image analysis for the differential diagnosis of colitis, representing a complex clinical challenge, seem not yet to be ready for primetime and more diverse image datasets may be necessary to improve performance in future development. The clinical value of the proposed clinical parameters algorithm should be evaluated in prospective cohorts.

Large depth-of-field image fusion method for complex gyration class mechanical parts

The local regions of complex gyration class mechanical parts are largely different, which results in the problem that their surface images are partially clear. This problem directly affects the effectiveness of vision methods on detecting surface defects. To tackle this problem, a large depth-of-field (DoF) surface image fusion method is proposed to obtain a full-focus image of complex mechanical parts, such as gyration class mechanical parts. We design an image acquisition platform based on the characteristics of gyration class parts to acquire their surface images accurately and clearly. We then propose a feature detection-based image registration method, by which the registered image can represent the surface information of the part completely and accurately. Additionally, we adopt a convolutional neural network-based image fusion method to achieve a fused large DoF surface image. Experimental studies were conducted to evaluate the performance of the deep-learning-based methods. The experimental results show that the proposed method can completely and clearly fuse the large DoF surface images of complex gyration class mechanical parts. The quality of the fused images has a significant improvement, and the proposed method is significantly more efficient than traditional fusion methods and other deep-learning methods.

Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing

Lighting Search Algorithm With Convolutional Neural Network-Based Image Captioning System for Natural Language Processing

A Shine Muscat Grape Berry Detection and Grape Cluster Compactness Estimation for Assessment of Grape Quality Based on Instance Segmentation Methods

Highlights Mask R-CNN ResNet-101 was analyzed to have the highest detection accuracy for grape berries. Grape cluster compactness is one of the important factors that determines the quality of grapes. Indicator DGC was used to intuitively calculate the compactness of grape cluster. An average error of 1.6 mm occurred with the verification of diameter estimation algorithm. Abstract. Consumption and cultivation of grapes are constantly increasing because they are known as alkaline foods, which relieve fatigue by accelerating carbohydrate metabolism in the human body. Among these grapes, a Shine Muscat doesn’t have the bitter taste of ordinary grapes, has low acidity, and has a crunchy texture, which results in growing interest in modern society. As the consumption of Shine Muscat increases, an automated process to save labor and time becomes essential. Particularly, one challenge that needs to be solved is the determination of grape quality based solely on visual evaluations by experts, which can result in inconsistent pricing for consumers. In this study, an algorithm was adopted to detect the grape berries from a bunch of single grapes by acquiring RGB images of the harvested Shine Muscat. Mask R-CNN, a convolutional neural network-based image segmentation technique, was used with various backbones to compare and evaluate the performance of each model. Results showed that Mask R-CNN ResNet 101 had the highest AP (Average Precision) value of 0.961 among all models. In addition, indexes such as the size (diameter) of each grape berry, the area of the grape on the image, and the area of the empty space that are required to find the compactness of the grape cluster are obtained. In particular, it was analyzed that the average error value (mm) and percent error (%) of the diameter estimation algorithm developed in this study were 1.60 mm and 4.79%, respectively. Keywords: Keywords., Density of Grape Cluster (DGC), Diameter estimation, Grape berry, Grape cluster compactness, Mask R-CNN, Object detection.

Three-stage image forgery localization with shallow feature enhancement and attention

With the development of technology, it is becoming easier and easier for people to generate tampered images that are indistinguishable from the human eye, and the malicious use of these photos in news, academic papers, and criminal crimes has brought great harm to society. This paper proposes a deep convolutional neural network-based image forgery localization method to uncover the subtle differences between doctored images and real images. Specifically, the network achieves tampering localization through a three-stage enhancement scheme. First, the dilated convolution in the deep layer of the network is used to keep the feature map resolution constant, and the number of shallow convolutions is decreased to reduce the perceptual field, so the network focuses on local regions. Second, the feature enhancement module is used to fuse shallow features with deep features to effectively filter content information and highlight tampering features, making full use of local and global information to improve the generalization ability. Finally, the attention enhancement module reweights the convolutional feature maps in terms of channels and locations, respectively, to highlight the information regions around the forgery boundaries, thus guiding the network to capture more intrinsic features for image forgery. Extensive experimental results on several public datasets show that this method outperforms other state-of-the-art methods in image forgery localization.

Structured cerebellar connectivity supports resilient pattern separation.

The cerebellum is thought to helpdetect and correct errors between intended and executed commands1,2 and is critical for social behaviours, cognition and emotion3-6. Computations for motor control must be performed quickly to correct errors in real time and should be sensitive to small differences between patterns for fine error correction while being resilient to noise7. Influential theories of cerebellar information processing have largely assumed random network connectivity, which increases the encoding capacity of the network's first layer8-13. However, maximizing encoding capacity reduces the resilience to noise7. To understand how neuronal circuits address this fundamental trade-off, we mapped the feedforward connectivity in the mouse cerebellar cortex using automated large-scale transmission electron microscopy and convolutional neural network-based image segmentation. We found that both the input and output layers of the circuit exhibit redundant and selective connectivity motifs, which contrast with prevailing models. Numerical simulations suggest that these redundant, non-random connectivity motifs increase the resilience to noise at a negligible cost to the overall encoding capacity. This work reveals how neuronal network structure can support a trade-off between encoding capacity and redundancy, unveiling principles of biological network architecture with implications for the design of artificial neural networks.

Applying image recognition techniques to visual information mining in hospitality and tourism

Purpose This study aims to provide a critical reflection of the application of image recognition techniques in visual information mining in hospitality and tourism. Design/methodology/approach This study begins by reviewing the progress of image recognition and advantages of convolutional neural network-based image recognition models. Next, this study explains and exemplifies the mechanisms and functions of two relevant image recognition applications: object recognition and facial recognition. This study concludes by providing theoretical and practical implications and potential directions for future research. Findings After this study presents different potential applications and compares the use of image recognition with traditional manual methods, the main findings of this critical reflection revolve around the feasibility of the described techniques. Practical implications Knowledge on how to extract valuable visual information from large-scale user-generated photos to infer the online behavior of consumers and service providers and its influence on purchase decisions and firm performance is crucial to business practices in hospitality and tourism. Originality/value Visual information plays a crucial role in online travel agencies and peer-to-peer accommodation platforms from the side of sellers and buyers. However, extant studies relied heavily on traditional manual identification with small samples and subjective judgment. With the development of deep learning and computer vision techniques, current studies were able to extract various types of visual information from large-scale datasets with high accuracy and efficiency. To the best of the authors’ knowledge, this study is the first to offer an outlook of image recognition techniques for mining visual information in hospitality and tourism.

Multiple Vessel Detection in Harsh Maritime Environments

Abstract Recently, research concerning the navigation of autonomous surface vehicles (ASVs) has been increasing. However, a large-scale implementation of these vessels is still held back by several challenges such as multi-object tracking. Attaining accurate object detection plays a big role in achieving successful tracking. This article presents the development of a detection model with an image-based Convolutional Neural Network trained through transfer learning, a deep learning technique. To train, test, and validate the detector module, data were collected with the SENSE ASV by sailing through two nearby ports, Leixões and Viana do Castelo, and recording video frames through its on-board cameras, along with a Light Detection And Ranging, GPS, and Inertial Measurement Unit data. Images were extracted from the collected data, composing a manually annotated dataset with nine classes of different vessels, along with data from other open-source maritime datasets. The developed model achieved a class mAP@[.5 .95] (mean average precision) of 89.5% and a clear improvement in boat detection compared to a multi-purposed state-of-the-art detector, YOLO-v4, with a 22.9% and 44.3% increase in the mAP with an Intersection over Union threshold of 50% and the mAP@[.5 .95], respectively. It was integrated in a detection and tracking system, being able to continuously detect nearby vessels and provide sufficient information for simple navigation tasks.

Image Dehazing Algorithm Based on Deep Learning Coupled Local and Global Features

To address the problems that most convolutional neural network-based image defogging algorithm models capture incomplete global feature information and incomplete defogging, this paper proposes an end-to-end convolutional neural network and vision transformer hybrid image defogging algorithm. First, the shallow features of the haze image were extracted by a preprocessing module. Then, a symmetric network structure including a convolutional neural network (CNN) branch and a vision transformer branch was used to capture the local features and global features of the haze image, respectively. The mixed features were fused using convolutional layers to cover the global representation while retaining the local features. Finally, the features obtained by the encoder and decoder were fused to obtain richer feature information. The experimental results show that the proposed defogging algorithm achieved better defogging results in both the uniform and non-uniform haze datasets, solves the problems of dark and distorted colors after image defogging, and the recovered images are more natural for detail processing.

Evaluating time series encoding techniques for Predictive Maintenance

Predictive Maintenance has become an important component in modern industrial scenarios, as a way to minimize down-times and fault rate for different equipment. In this sense, while machine learning and deep learning approaches are promising due to their accurate predictive abilities, their data-heavy requirements make them significantly limited in real world applications. Since one of the main issues to overcome is lack of consistent training data, recent work has explored the possibility of adapting well-known deep-learning models for image recognition, by exploiting techniques to encode time series as images. In this paper, we propose a framework for evaluating some of the best known time series encoding techniques, together with Convolutional Neural Network-based image classifiers applied to predictive maintenance tasks. We conduct an extensive empirical evaluation of these approaches for the failure prediction task on two real-world datasets (PAKDD2020 Alibaba AI OPS Competition and NASA bearings), also comparing their performances with respect to the state-of-the-art approaches. We further discuss advantages and limitation of the exploited models when coupled with proper data augmentation techniques.

Machine learning technique for morphological classification of galaxies from SDSS. II. The image-based morphological catalogs of galaxies at 0.02<z<0.1

We applied the image-based approach with a convolutional neural network (CNN) model to the sample of low-redshift galaxies with –24m<Mr<–19.4m from the SDSS DR9. We divided it into two subsamples, SDSS DR9 galaxy dataset and Galaxy Zoo 2 (GZ2) dataset, considering them as the inference and training datasets, respectively. To determine the principal parameters of galaxy morphology defined within the GZ2 project, we classified the galaxies into five visual types and 34 morphological features of galaxies from the inference dataset, which do not match with GZ2 training dataset. As a result, we created the morphological catalog of 315782 galaxies at 0.02<z<0.1, where these classes and features were defined for the first time for 216148 galaxies by image-based CNN classifier. For the rest of galaxies the initial morphological classification was re-assigned as in the GZ2 project. Main results are presented in the catalog of 19468 completely rounded, 27321 rounded in-between, 3235 cigar-shaped, 4099 edge-on, 18615 spiral, and 72738 general low-redshift galaxies of the studied SDSS sample. Our method shows the promising performance of morphological classification attaining >93 % of accuracy for five classes morphology prediction except the cigar-shaped (~75 %) and completely rounded (~83 %) galaxies. As for the classification of galaxies by their detailed structural morphological features, our CNN model gives the accuracy in the range of 92–99 % depending on features, a number of galaxies with the given feature in the inference dataset, and the galaxy image quality. As a result, for the first time we assigned 34 morphological detailed features (bar, rings, number of spiral arms, mergers, etc.) for more than 160000 low-redshift galaxies from the SDSS DR9. We demonstrate that implication of the CNN model with adversarial validation and adversarial image data augmentation improves classification of smaller and fainter SDSS galaxies with mr <17.7.

A Hybrid Deep Learning Model for Automatic Modulation Classification

Automatic modulation classification (AMC) is one of the major challenges for cognitive radio (CR), which can enhance the spectrum utilization efficiency. In this study, a hybrid signal and image-based deep learning model is designed for AMC in CR. A convolutional neural network (CNN) is applied in both the deep learning models. The signal-based CNN (SBCNN) is designed with the optimal filter size for the prediction accuracy, which is used as a pre-training deep learning network to extract features with size <inline-formula> <tex-math notation="LaTeX">$24\times 1$ </tex-math></inline-formula>. The features extracted by SBCNN are converted into heat map images, which showed RGB images in the scale range of &#x2212;30 to &#x002B;30. Finally, the images are utilized for training and testing the image-based CNN (IBCNN). The dataset used for the experiment is DeepSig: RADIOML2018.01A, which is the latest version. For the IBCNN, the prediction accuracy is 1.96&#x0025;, 7.99&#x0025;, and 4.63&#x0025; higher at signal-to-noise ratio (SNR) 10 dB, and 3.26&#x0025;, 6.4&#x0025;, and 4.13&#x0025; higher at SNR 0 dB as compared to conventional models: ECNN, SCGNet, and LCNN, respectively.

Radiation Dose Reduction in Digital Mammography by Deep-Learning Algorithm Image Reconstruction: A Preliminary Study.

PurposeTo develop a denoising convolutional neural network-based image processing technique and investigate its efficacy in diagnosing breast cancer using low-dose mammography imaging.Materials and MethodsA total of 6 breast radiologists were included in this prospective study. All radiologists independently evaluated low-dose images for lesion detection and rated them for diagnostic quality using a qualitative scale. After application of the denoising network, the same radiologists evaluated lesion detectability and image quality. For clinical application, a consensus on lesion type and localization on preoperative mammographic examinations of breast cancer patients was reached after discussion. Thereafter, coded low-dose, reconstructed full-dose, and full-dose images were presented and assessed in a random order.ResultsLesions on 40% reconstructed full-dose images were better perceived when compared with low-dose images of mastectomy specimens as a reference. In clinical application, as compared to 40% reconstructed images, higher values were given on full-dose images for resolution (p < 0.001); diagnostic quality for calcifications (p < 0.001); and for masses, asymmetry, or architectural distortion (p = 0.037). The 40% reconstructed images showed comparable values to 100% full-dose images for overall quality (p = 0.547), lesion visibility (p = 0.120), and contrast (p = 0.083), without significant differences.ConclusionEffective denoising and image reconstruction processing techniques can enable breast cancer diagnosis with substantial radiation dose reduction.

Lightweight Single Image Super-Resolution With Similar Feature Fusion Block

Convolutional neural network-based image super-resolution methods have achieved great success in recent years. However, the huge memory and computational costs make most of the existing methods difficult to be applied to resource-constrained scenarios such as edge devices. To tackle this problem, we propose a generic, lightweight and efficient feature fusion block to replace the commonly used 1&#x002A;1 convolution. In addition, we propose the enhanced shallow residual blocks to improve the super-resolution performance. By combining these two novel blocks, we design an efficient similar feature fusion network for single image super-resolution, based on the observation that cross-layer features of the same channel usually have high similarities. The similar feature fusion block utilizes similarity as a guide for feature clustering, enabling efficient and high-performance cross-layer feature fusion. On the other hand, the enhanced shallow residual blocks are used as the base feature extraction model for the network to improve super-resolution performance in conjunction with the feature fusion module. In the enhanced shallow residual blocks, we combine convolution with identity connection to maintain the similarity of cross-layer features that are fed into the similar feature fusion block. The spatial attention mechanism is also introduced to reinforce the useful spatial features. Experimental results on the benchmark datasets show that the proposed method can achieve comparable results to state-of-the-art methods with a small number of parameters.

Dual Image-Based CNN Ensemble Model for Waste Classification in Reverse Vending Machine

A reverse vending machine motivates citizens to bring recyclable waste by rewarding them, which is a viable solution to increase the recycling rate. Reverse vending machines generally use near-infrared sensors, barcode sensors, or cameras to classify recycling resources. However, sensor-based reverse vending machines suffer from a high configuration cost and the limited scope of target objects, and conventional single image-based reverse vending machines usually make erroneous predictions about intentional fraud objects. This paper proposes a dual image-based convolutional neural network ensemble model to address these problems. For this purpose, we first created a prototype reverse vending machine and constructed an image dataset containing two cross-sections of objects, top and front view. Then, we chose convolutional neural network models widely used in image classification as the candidates for building an accurate and lightweight ensemble model. Considering the size and classification performance of candidates, we constructed the best-fit ensemble combination and evaluated its classification performance. The final ensemble model showed a classification accuracy higher than 95% for all target classes, including fraud objects. This result proves that our approach achieves better robustness against intentional fraud objects than single image-based models and thus can broaden the scope for target resources. The measurement results on lightweight embedded platforms also demonstrated that our model provides a short inference time that is enough to facilitate the real-time execution of reverse vending machines based on low-cost edge artificial intelligence devices.

Computed tomography angiography-based deep learning method for treatment selection and infarct volume prediction in anterior cerebral circulation large vessel occlusion.

BackgroundComputed tomography perfusion (CTP) is the mainstay to determine possible eligibility for endovascular thrombectomy (EVT), but there is still a need for alternative methods in patient triage.PurposeTo study the ability of a computed tomography angiography (CTA)-based convolutional neural network (CNN) method in predicting final infarct volume in patients with large vessel occlusion successfully treated with endovascular therapy.Materials and MethodsThe accuracy of the CTA source image-based CNN in final infarct volume prediction was evaluated against follow-up CT or MR imaging in 89 patients with anterior circulation ischemic stroke successfully treated with EVT as defined by Thrombolysis in Cerebral Infarction category 2b or 3 using Pearson correlation coefficients and intraclass correlation coefficients. Convolutional neural network performance was also compared to a commercially available CTP-based software (RAPID, iSchemaView).ResultsA correlation with final infarct volumes was found for both CNN and CTP-RAPID in patients presenting 6–24 h from symptom onset or last known well, with r = 0.67 (p < 0.001) and r = 0.82 (p < 0.001), respectively. Correlations with final infarct volumes in the early time window (0–6 h) were r = 0.43 (p = 0.002) for the CNN and r = 0.58 (p < 0.001) for CTP-RAPID. Compared to CTP-RAPID predictions, CNN estimated eligibility for thrombectomy according to ischemic core size in the late time window with a sensitivity of 0.38 and specificity of 0.89.ConclusionA CTA-based CNN method had moderate correlation with final infarct volumes in the late time window in patients successfully treated with EVT.

Prediction of Beck Depression Inventory Score in EEG: Application of Deep-Asymmetry Method

There is ongoing research on using electroencephalography (EEG) to predict depression. In particular, the deep learning method in which brain waves are used as inputs of a convolutional neural network (CNN) is being widely researched and has shown remarkable performance. We built a regression model to predict the severity score (Beck Depression Inventory [BDI]) of depressed patients as an extension of the deep-asymmetry method, which has shown promising performance in depression classification. Predicting the severity of depression is very important because the treatment and coping methods are different for each severity level. We imaged brain waves using the deep-asymmetry method, used them to train a two-dimensional CNN-based deep learning model, and achieved satisfactory performance. The EEG image-based CNN approach will make an important contribution to creating a highly interpretable model for predicting depression in the future.

A pilot study for fragment identification using 2D NMR and deep learning.

This paper presents a proof of concept of a method to identify substructures in 2D NMR spectra of mixtures using a bespoke image-based convolutional neural network application. This is done using HSQC and HMBC spectra separately and in combination. The application can reliably detect substructures in pure compounds, using a simple network. Results indicate that it can work for mixtures when trained on pure compounds only. HMBC data and the combination of HMBC and HSQC show better results than HSQC alone in this pilot study.

Adversarial Robustness of Deep Convolutional Neural Network-based Image Recognition Models: A Review

Deep convolutional neural networks have achieved great success in recent years. They have been widely used in various applications such as optical and SAR image scene classification, object detection and recognition, semantic segmentation, and change detection. However, deep neural networks rely on large-scale high-quality training data, and can only guarantee good performance when the training and test data are independently sampled from the same distribution. Deep convolutional neural networks are found to be vulnerable to subtle adversarial perturbations. This adversarial vulnerability prevents the deployment of deep neural networks in security-sensitive applications such as medical, surveillance, autonomous driving and military scenarios. This paper first presents a holistic view of security issues for deep convolutional neural network-based image recognition systems. The entire information processing chain is analyzed regarding safety and security risks. In particular, poisoning attacks and evasion attacks on deep convolutional neural networks are analyzed in detail. The root causes of adversarial vulnerabilities of deep recognition models are also discussed. Then, we give a formal definition of adversarial robustness and present a comprehensive review of adversarial attacks, adversarial defense, and adversarial robustness evaluation. Rather than listing existing research, we focus on the threat models for the adversarial attack and defense arms race. We perform a detailed analysis of several representative adversarial attacks on SAR image recognition models and provide an example of adversarial robustness evaluation. Finally, several open questions are discussed regarding recent research progress from our workgroup. This paper can be further used as a reference to develop more robust deep neural network-based image recognition models in dynamic adversarial scenarios.