Use Of Convolutional Neural Networks Research Articles

A powerful image synthesis and semi-supervised learning pipeline for site-specific weed detection

Precise and efficient weed detection in agricultural fields is the key for robotic weed control. Recent developments in convolutional neural networks (CNNs) have achieved significant success in this regard. CNNs that simultaneously localize and classify objects in images are the predominant forms that have been widely used to detect crops and weeds. However, the use of CNNs in agriculture, particularly for weed detection, has been impeded by a lack of large training dataset with ground truth annotations. Cut-and-paste image synthesis approach and semi-supervised learning are popular methods to alleviate the training data deficiency problem. In this paper, we propose a novel image synthesis and semi-supervised learning pipeline to train site-specific weed detection models without the need for manually labeled images. The CNN models trained by this pipeline achieve performance levels close to that of the supervised models. We investigated the behavior of the proposed pipeline by varying several key components and showed that color match between the training and testing images, training-time color augmentation, and iterative semi-supervised learning largely improve the model performance. These promising results can be used to guide the construction of a weed image database applicable to different weed detection scenarios.

Finicky transfer learning—A method of pruning convolutional neural networks for cracks classification on edge devices

AbstractHigh demand for computational power significantly limits the possibility of using modern deep learning methods in the environments where one has to deal with devices limited by the performance and the energy constraints. To address this issue, this paper proposes a novel method of combining the pruning and the transfer learning (TL) techniques for the purpose of delivering solid accuracy while simultaneously lowering the demand for energy and computing power. This method is referred to as finicky TL as it is finicky during the process of selecting filters from a pretrained feature extractor to compose a sparser architecture. The proposed filter selection process is based on an original approach utilizing the Jaccard similarity coefficient calculated between the activation maps and the masks obtained by semantic segmentation. This enables the use of convolutional neural networks, trained previously on a large generic dataset, in a crack classification task. The presented method significantly lowers the inference time while maintaining or even slightly increasing the classification accuracy, enabling real‐time operation on single‐board computers.

A convolutional neural network combined with positional and textural attention for the fully automatic delineation of primary nasopharyngeal carcinoma on non-contrast-enhanced MRI.

Convolutional neural networks (CNNs) have the potential to automatically delineate primary nasopharyngeal carcinoma (NPC) on magnetic resonance imaging (MRI), but currently, the literature lacks a module to introduce valuable pre-computed features into a CNN. In addition, most CNNs for primary NPC delineation have focused on contrast-enhanced MRI. To enable the use of CNNs in clinical applications where it would be desirable to avoid contrast agents, such as cancer screening or intra-treatment monitoring, we aim to develop a CNN algorithm with a positional-textural fully-connected attention (FCA) module that can automatically delineate primary NPCs on contrast-free MRI. This retrospective study was performed in 404 patients with NPC who had undergone staging MRI. A proposed CNN algorithm incorporated with our positional-textural FCA module (Aproposed ) was trained on manually delineated tumours (M1st ) to automatically delineate primary NPCs on non-contrast-enhanced T2-weighted fat-suppressed (NE-T2W-FS) images. The performance of Aproposed , three well-established CNNs, Unet (Aunet ), Attention-Unet (Aatt ) and Dense-Unet (Adense ), and a second manual delineation repeated to evaluate human variability (M 2 nd ) were measured by comparing to the reference standard M 1 st to obtain the Dice similarity coefficient (DSC) and average surface distance (ASD). The Wilcoxon rank test was used to compare the performance of Aproposed against Aunet , Aatt , Adense and M 2 nd . Aproposed showed a median DSC of 0.79 (0.10) and ASD of 0.66 (0.84) mm. It performed better than the well-established networks Aunet [DSC =0.75 (0.12) and ASD =1.22 (1.73) mm], Aatt [DSC =0.75 (0.10) and ASD =0.96 (1.16) mm] and Adense [DSC =0.71 (0.14) and ASD =1.67 (1.92) mm] (all P<0.01), but slightly worse when compared to M 2 nd [DSC =0.81 (0.07) and ASD =0.56 (0.80) mm] (P<0.001). The proposed CNN algorithm has potential to accurately delineate primary NPCs on non-contrast-enhanced MRI.

Discovering artistic influences of painters from expressionism, impressionism, and surrealism art movements using convolutional neural network

This paper explores the use of Convolutional Neural Network (CNN) to discover the artistic influences of painters from three major art movements - Expressionism, Impressionism, and Surrealism. Since artistic influence is a subjective inference by art experts, this paper offers a quantitative solution to this seemingly subjective problem. We propose two solutions. The first solution involves multi-class, artist-based classification using several CNN models including Alexnet, VGG16, and Resnet18. The idea is, if various models consistently misclassify certain artworks to another artist, then the models consider the artworks of the two to be similar. With that, artistic influence is drawn from consistent false positives in the confusion matrices. The second approach is retrieving visually similar paintings in a large-scale data set. Like the first approach, if artworks of two artists are deemed to be visually similar by a computer, then artistic influence can be drawn between the two. The retrieval process in finding similarities in paintings involves extracting features using a CNN, applying a dimension reduction technique, then computing the Euclidean distances between the features and returning the nearest neighbor. With a long list of artists with possible influences, a graph network is created to map the interactions on a wider point of view. The proposed methods were able to validate several historical claims of artistic influences and provide new possible artistic influences that have never been considered before by art historians.

Robust 3D Face Reconstruction Using One/Two Facial Images.

Being able to robustly reconstruct 3D faces from 2D images is a topic of pivotal importance for a variety of computer vision branches, such as face analysis and face recognition, whose applications are steadily growing. Unlike 2D facial images, 3D facial data are less affected by lighting conditions and pose. Recent advances in the computer vision field have enabled the use of convolutional neural networks (CNNs) for the production of 3D facial reconstructions from 2D facial images. This paper proposes a novel CNN-based method which targets 3D facial reconstruction from two facial images, one in front and one from the side, as are often available to law enforcement agencies (LEAs). The proposed CNN was trained on both synthetic and real facial data. We show that the proposed network was able to predict 3D faces in the MICC Florence dataset with greater accuracy than the current state-of-the-art. Moreover, a scheme for using the proposed network in cases where only one facial image is available is also presented. This is achieved by introducing an additional network whose task is to generate a rotated version of the original image, which in conjunction with the original facial image, make up the image pair used for reconstruction via the previous method.

Image Generation for 2D-CNN Using Time-Series Signal Features from Foot Gesture Applied to Select Cobot Operating Mode.

Advances in robotics are part of reducing the burden associated with manufacturing tasks in workers. For example, the cobot could be used as a “third-arm” during the assembling task. Thus, the necessity of designing new intuitive control modalities arises. This paper presents a foot gesture approach centered on robot control constraints to switch between four operating modalities. This control scheme is based on raw data acquired by an instrumented insole located at a human’s foot. It is composed of an inertial measurement unit (IMU) and four force sensors. Firstly, a gesture dictionary was proposed and, from data acquired, a set of 78 features was computed with a statistical approach, and later reduced to 3 via variance analysis ANOVA. Then, the time series collected data were converted into a 2D image and provided as an input for a 2D convolutional neural network (CNN) for the recognition of foot gestures. Every gesture was assimilated to a predefined cobot operating mode. The offline recognition rate appears to be highly dependent on the features to be considered and their spatial representation in 2D image. We achieve a higher recognition rate for a specific representation of features by sets of triangular and rectangular forms. These results were encouraging in the use of CNN to recognize foot gestures, which then will be associated with a command to control an industrial robot.

27403 Use of convolutional neural networks in skin lesion analysis using real world image and nonimage data

Introduction: Convolutional neural networks (CNNs) have demonstrated utility in high quality skin lesion image analysis. The use of CNNs in analyzing smartphone images needs further exploration to justify the development of real-time clinical decision support tools.

Computerized Tomography Image Feature under Convolutional Neural Network Algorithm Evaluated for Therapeutic Effect of Clarithromycin Combined with Salmeterol/Fluticasone on Chronic Obstructive Pulmonary Disease.

This study was to explore the use of convolutional neural network (CNN) for the classification and recognition of computerized tomography (CT) images of chronic obstructive pulmonary disease (COPD) and the therapeutic effect of clarithromycin combined with salmeterol/fluticasone. First, the clinical data of COPD patients treated in hospital from September 2018 to December 2020 were collected, and CT and X-ray images were also collected. CT-CNN and X ray-CNN single modal models were constructed based on the LeNet-5 model. The randomized fusion algorithm was introduced to construct a fused CNN model for the diagnosis of COPD patients, and the recognition effect of the model was verified. Subsequently, the three-dimensional reconstruction of the patient's bronchus was performed using the classified CT images, and the changes of CT quantitative parameters in COPD patients were compared and analyzed. Finally, COPD patients were treated with salmeterol/fluticasone (COPD-C) and combined with clarithromycin (COPD-T). In addition, the differences between patients' lung function indexes, blood gas indexes, St. George respiratory questionnaire (SGRQ) scores, and the number of acute exacerbations (AECOPD) before and after treatment were evaluated. The results showed that the randomized fusion model under different iteration times and batch sizes always had the highest recognition rate, sensitivity, and specificity compared to the two single modal CNN models, but it also had longer training time. After CT images were used to quantitatively evaluate the changes of the patient's bronchus, it was found that the area of the upper and lower lung lobes of the affected side of COPD patients and the ratio of the area of the tube wall to the bronchus were significantly changed. The lung function, blood gas index, and SGRQ score of COPD-T patients were significantly improved compared with the COPD-C group (P < 0.05), but there was no considerable difference in AECOPD (P > 0.05). In summary, the randomized fusion-based CNN model can improve the recognition rate of COPD, and salmeterol/fluticasone combined with clarithromycin therapy can significantly improve the clinical treatment effect of COPD patients.

Intellectual Tool to Compute Embodied Energy and Carbon Dioxide Emission for Building Construction Materials

Every material has its capacity to consume energy called embodied energy (EE) and to emit CO2 during the manufacturing process. The estimation procedure of EE and CO2 emission is predominantly based on the existing case examples from various nations across the globe. There is an impressive aperture in instances from developing nations concerning an appraisal of embodied energy and emission of CO2 for used building construction materials. The available tools are also not much user friendly and hence not everyone is capable to use it easily. From this point of view, this research aims to develop an intellectual tool to find out embodied energy and CO2 emission for building materials. This research presents a model for estimation of EE and emission of CO2 (E4-Model) for various building materials used in construction. This research focuses on construction of building in the Indian Construction Section. For the assessment of EE and CO2 emission, the concept of Convolutional Neural Network (CNN) is used and all the values are by weight of material. From the experimental analysis, it is observed that the estimation accuracy of the system is more than 99.62% for small as well as large buildings. The research concluded that the use of CNN is far better than the artificial neural networks in terms of system response time and complexity to estimate the EE and CO2 emission rates. In the research proposed E4-Model is applied and developed using the concept of deep learning and curve-fitting toolboxes within MATLAB Software. The actual emission for all the tested projects were found to be lesser than the estimated emission. The tool developed using CNN proved to work with high accuracy.

Computer Vision for Fire Detection on UAVs—From Software to Hardware

Fire hazard is a condition that has potentially catastrophic consequences. Artificial intelligence, through Computer Vision, in combination with UAVs has assisted dramatically to identify this risk and avoid it in a timely manner. This work is a literature review on UAVs using Computer Vision in order to detect fire. The research was conducted for the last decade in order to record the types of UAVs, the hardware and software used and the proposed datasets. The scientific research was executed through the Scopus database. The research showed that multi-copters were the most common type of vehicle and that the combination of RGB with a thermal camera was part of most applications. In addition, the trend in the use of Convolutional Neural Networks (CNNs) is increasing. In the last decade, many applications and a wide variety of hardware and methods have been implemented and studied. Many efforts have been made to effectively avoid the risk of fire. The fact that state-of-the-art methodologies continue to be researched, leads to the conclusion that the need for a more effective solution continues to arouse interest.

Convolutional Neural Network Based Equalizer for Improving the Reliability Performance of OFDM with Subcarrier Power Modulation

As the demand for higher data rates has rapidly been increasing day after day, researchers around the world have given serious attention and made significant efforts towards exploring new techniques that can improve the spectral efficiency of future wireless systems. Among these methods, the modulation technique named as orthogonal frequency division multiplexing with subcarrier power modulation (OFDM-SPM) is considered as a key potential candidate transmission method, which has the potential to effectively improve the per-user spectral efficiency of wireless networks. However, the reliability performance efficiency of OFDM-SPM is not that high, where it was found that the additional data stream conveyed by sub-carriers’ power has higher bit error rate (BER) performance compared to the data stream conveyed by conventional modulation schemes. To improve the reliability performance of OFDM-SPM furthermore, in this paper, we propose the use of Convolutional Neural Networks (CNN) based equalizer for OFDM-SPM. Simulation results show that Convolutional Neural Networks (CNN) based equalizer can improve the reliability performance of OFDM-SPM by 5-10 dB compared to the conventional OFDM-SPM scheme that does not use CNN.

Genomic prediction of growth traits in scallops using convolutional neural networks

Deep learning neural networks applied to the genomic prediction of complex traits have been of great interest in recent years. Previous studies primarily used simulated phenotypes or/and genotypes in plants and animals. The properties of deep learning models used in genomic selection are not well characterized and not well validated with real datasets. In the present study, we evaluated the performance of a class of deep learning methods called convolutional neural networks (CNNs) in the genomic prediction of four quantitative traits (e.g., shell length, shell height, shell width, and total weight) in a Bay scallop (Argopecten irradians irradians) population. The results were compared with those obtained from two linear models, RR-GBLUP and Bayes B, and multilayer perceptron neural networks (MLPs). One-convolutional layer CNNs with an optimal structure, which was obtained by using AIC or BIC method, had roughly comparable prediction accuracies on the four quantitive traits in the scallop population. Overall, CNNs outperformed RR-GBLUP, Bayes B and MLPs on shell height, shell width and total weight, and performed slightly worse than only Bayes B on shell length. MLPs gave the least accurate predictions on average among the four types of models. Because MLPs had far more parameters to estimate than the two linear models, and their predictions were challenged by the overfitting problem. Genomic prediction accuracy varied with SNP panel size and training population size.The impact of varied marker densities and two GWAS-based scenarios for SNP selection on genomic prediction accuracy was investigated as well. The present results provide evidence that supports the use of convolutional neural networks for genomic prediction of complex traits in scallops, yet the optimal structures of CNNs remained to be exploited in future studies.

Satellite and Scene Image Classification Based on Transfer Learning and Fine Tuning of ResNet50

Image classification has gained lot of attention due to its application in different computer vision tasks such as remote sensing, scene analysis, surveillance, object detection, and image retrieval. The primary goal of image classification is to assign the class labels to images according to the image contents. The applications of image classification and image analysis in remote sensing are important as they are used in various applied domains such as military and civil fields. Earlier approaches for remote sensing images and scene analysis are based on low-level feature representations such as color- and texture-based features. Vector of Locally Aggregated Descriptors (VLAD) and orderless Bag-of-Features (BoF) representations are the examples of mid-level approaches for remote sensing image classification. Recent trends for remote sensing and scene classification are focused on the use of Convolutional Neural Network (CNN). Keeping in view the success of CNN models, in this research, we aim to fine-tune ResNet50 by using network surgery and creation of network head along with the fine-tuning of hyperparameters. The learning of hyperparameters is tuned by using a linear decay learning rate scheduler known as piecewise scheduler. To tune the optimizer hyperparameter, Stochastic Gradient Descent with Momentum (SGDM) is used with the usage of weight learn and bias learn rate factor. Experiments and analysis are conducted on five different datasets, that is, UC Merced Land Use Dataset (UCM), RSSCN (the remote sensing scene classification image dataset), SIRI-WHU, Corel-1K, and Corel-1.5K. The analysis and competitive results exemplify that our proposed image classification-based model can classify the images in a more effective and efficient manner as compared to the state-of-the-art research.

Extrapolating shortwave geostationary satellite imagery of clouds into nighttime using longwave observations

The lack of shortwave (SW, visible, and near-infrared) geostationary satellite data at night results in degradation of many weather forecasts and real-time diagnostic products. We present a method to extrapolate SW GOES-16 advanced baseline imager data through night using nighttime longwave (LW, infrared) observations and the relationships between LW and SW data observed during the previous day. The method is not a forecast since it requires LW nighttime observations but can provide continuity through day, night, and satellite terminator hours. To provide performance statistics, the algorithm is applied during the day so the SW extrapolations can be compared to observations. Typical mean absolute errors (MAEs) range from 1.0% to 12.7% reflectance depending on the SW channel. These MAEs can be predicted using a diagnostic metric called 0-h MAE which quantifies the quality of the algorithm’s input data. In addition to quantitative error statistics, three case studies are presented, including an animation of extrapolated imagery from dusk through dawn. Considerations for future improvements include use of convolutional neural networks and/or object-based extrapolations where mesoscale features are extrapolated individually.

Robustness of digital camera identification with convolutional neural networks

This paper considers the area of digital forensics (DF). One of the problem in DF is the issue of identification of digital cameras based on images. This aspect has been attractive in recent years due to popularity of social media platforms like Facebook, Twitter etc., where lots of photographs are shared. Although many algorithms and methods for digital camera identification have been proposed, there is lack of research about their robustness. Therefore, in this paper the robustness of digital camera identification with the use of convolutional neural network is discussed. It is assumed that images may be of poor quality, for example, degraded by Poisson noise, Gaussian blur, random noise or removing pixels’ least significant bit. Experimental evaluation conducted on two large image datasets (including Dresden Image Database) confirms usefulness of proposed method, where noised images are recognized with almost the same high accuracy as normal images.

Is artificial intelligence really a future trend in health care?

Everyone who uses any digital platform in the daily routine has already been surprised by some sudden ad or product advertisement about which some information has been sought on the Internet. Coincidence? Of course not! This is just one example of how artificial intelligence is inserted into our daily lives. It is in the platforms for music streaming, movies, shopping for any product, in traffic applications, in the stock market. Each "like", each share, each post shows a pattern of consumer preference, a characteristic that can be used to direct advertisements in order to advertise or market a product to a specific target. This is already happening, it is not part of the future. Artificial intelligence is already part of our present. But how do these platforms manage to "guess" our preferences or tastes and hit exactly what we were looking for? In reality nothing is guessed, it is learned. Through computer modeling, these systems learn from the examples that we ourselves give them. We feed these systems on a daily basis. Just like children, who learn many things by example (languages, for instance) before they even go to school, these systems are also capable of learning. A child learns that a dog is different from a cat when it sees examples of several dogs and several cats. So a child can learn the differences between both animals. Algorithms learn the same way, through examples. This is what we call "machine learning," a sub-area of artificial intelligence (AI). It is an advance for society, but it must be applied with ethics and transparency (see the Netflix documentary Coded Bias). Moving away from the market sphere and thinking about health care, machine learning has also been widely employed, because these systems have the ability to learn using endless amount of patient and hospital data (Big Data). In this sense, AI-based systems have been developed aiming at improving patient care, from the organization of triage systems at clinics and hospitals, patient scheduling, organization of test result delivery, preventing errors in drug prescriptions, as well as predicting and assisting in disease diagnosis. The artificial intelligence literature in the medical field is already vast. In dentistry, research has focused on the use of convolutional neural networks (CNN) in dental radiology. Tools are produced for researchers and system developers that aim at assisting clinicians in imaging diagnosis, for example, of dental caries, periapical lesions, bone resorption, among other important outcomes. Some companies, in Brazil and worldwide, have already seen a potential market in the application of these neural networks, and are providing software to assist in the analysis of radiographic images. Far from being able to replace health professionals, this technology should be used to improve the work of dentists and bring more security in diagnosis. Trying to replace a health professional with artificial intelligence, especially in dentistry, is impossible and not productive at all (see Eric Topol's book Deep Medicine). Information technology as an ally will bring many benefits to dentistry, not only in radiology. The analysis of digital cohorts (electronic patient records) with machine learning algorithms can bring new insights to Science. Such algorithms are able to cross-reference thousands of predictive attributes with various endpoints to define which information is most relevant for qualitative analyses. It is the new advanced statistics. For this reason, it is especially important to emphasize the need to build a large-scale public dental dataset to make the clinical application of AI possible. The challenge now is to improve the quality of the datasets to build really accurate machine learning algorithms. Finally, it would be very useful for dentists if these developed machine learning systems become applications that could be widely available and spread to the dental community. The spectrum of AI is huge! Try doing a search today on some topic and wait for the algorithm to work! It will offer you all the information, based on the search example you yourself have offered! This is AI in our lives, no future, but a present! Keywords Artificial intelligence; Health care.

Use of Convolutional Neural Networks for the Detection of u-Serrated Patterns in Direct Immunofluorescence Images to Facilitate the Diagnosis of Epidermolysis Bullosa Acquisita

The u-serrated immunodeposition pattern in direct immunofluorescence (DIF) microscopy is a recognizable feature and confirmative for the diagnosis of epidermolysis bullosa acquisita (EBA). Due to unfamiliarity with serrated patterns, serration pattern recognition is still of limited use in routine DIF microscopy. The objective of this study was to investigate the feasibility of using convolutional neural networks (CNNs) for the recognition of u-serrated patterns that can assist in the diagnosis of EBA. The nine most commonly used CNNs were trained and validated by using 220,800 manually delineated DIF image patches from 106 images of 46 different patients. The data set was split into 10 subsets: nine training subsets from 42 patients to train CNNs and the last subset from the remaining four patients for a validation data set of diagnostic accuracy. This process was repeated 10 times with a different subset used for validation. The best-performing CNN achieved a specificity of 89.3% and a corresponding sensitivity of 89.3% in the classification of u-serrated DIF image patches, an expert level of diagnostic accuracy. Experiments and results show the effectiveness of CNN approaches for u-serrated pattern recognition with a high accuracy. The proposed approach can assist clinicians and pathologists in recognition of u-serrated patterns in DIF images and facilitate the diagnosis of EBA.

On the use of CNNs with patterned stride for medical image analysis

The use of deep learning techniques for early and accurate medical image diagnosis has grown significantly in recent years, with some encouraging results across many medical specialties, pathologies, and image types. One of the most popular deep neural network architectures is the convolutional neural network (CNN), widely used for medical image classification and segmentation, among other tasks. One of the configuration parameters of a CNN is called stride and it regulates how sparsely the image is sampled during the convolutional process. This paper explores the idea of applying a patterned stride strategy: pixels closer to the center are processed with a smaller stride concentrating the amount of information sampled, and pixels away from the center are processed with larger strides consequently making those areas to be sampled more sparsely. We apply this method to different medical image classification tasks and demonstrate experimentally how the proposed patterned stride mechanism outperforms a baseline solution with the same computational cost (processing and memory). We also discuss the relevance and potential future extensions of the proposed method.

Quantitative analysis of lithium in brine by laser-induced breakdown spectroscopy based on convolutional neural network

In this study, a simple and effective method for accurate determination of lithium in brine samples was developed by the combination of laser induced breakdown spectroscopy (LIBS) and convolutional neural network (CNN). Our results clearly demonstrate that the use of CNN could efficiently overcome the complex matrix effects, and thus allows for on-site Li quantitative determination in brine samples by LIBS. Specifically, two CNN models with different input data (M-CNN with matrix emission lines, and DP-CNN with double Li lines) were constructed based on the primary matrix features on spectrum and Boltzmann equation, respectively. It was observed that DP-CNN model could greatly improve the accuracy of Li analysis. We also compared the quantitative analysis capabilities of DP-CNN model with partial least squares regression (PLSR) and principal component analysis-support vector regression (PCA-SVR) model, and the results clearly showed DP-CNN offers the best quantification results (higher accuracy and less matrix interference). Finally, five real brine samples were successfully analyzed by the proposed DP-CNN model, confirming by the average absolute error of the prediction of 0.28 mg L−1 and the average relative error of 3.48%. These results clearly demonstrate that input data plays an important role in the training of CNN model in LIBS analysis, and the proposed DP-CNN provides an effective approach to solve the matrix effects encountered in LIBS for Li measurement in brine samples.

Deep Static and Dynamic Level Analysis: A Study on Infinite Mario

Automatic analysis of game levels can provide as- sistance to game designers and procedural content generation. We introduce a static-dynamic scale to categorize level analysis strategies, which captures the extent that the analysis depends on player simulation. Due to its ability to automatically learn intermediate representations for the task, a convolutional neural network (CNN) provides a general tool for both types of analysis. In this paper, we explore the use of CNN to analyze 1,437 Infinite Mario levels. We further propose a deep reinforcement learning technique for dynamic analysis, which allows the simulated player to pay a penalty to reduce error in its control. We empirically demonstrate the effectiveness of our techniques and complementarity of dynamic and static analysis.