Multiple Convolutional Neural Networks Research Articles

Earthquake damage detection and level classification method for wooden houses based on convolutional neural networks and onsite photos

AbstractThe results of earthquake damage certification (EDC) surveys are the basis of support measures for improving the lives of disaster victims. To address issues such as a limited workforce to perform EDC surveys and difficulties in judging the level of damage, a damage detection and level classification method for wooden houses using multiple convolutional neural network models is proposed. The proposed method, including detection, filtering, and classification models, was trained and validated based on photographs collected from EDC surveys in Uki City, Kumamoto Prefecture. Then, a software system, which deployed these models, was developed for the onsite EDC surveyors to detect damages shown in the photographs of the surveyed house and classify damage levels. The test results based on 32 target buildings indicate that the detection model achieved high recall in detecting damage. Moreover, the redundant detected regions can be precisely filtered by the filtering model. Finally, the classification model achieved relatively high overall accuracy in classifying the damage level.

Flow Field Estimation with Distortion Correction Based on Multiple Input Deep Convolutional Neural Networks and Hartmann–Shack Wavefront Sensing

Flow Field Estimation with Distortion Correction Based on Multiple Input Deep Convolutional Neural Networks and Hartmann–Shack Wavefront Sensing

Multi-scale deep learning ensemble for segmentation of endometriotic lesions

AbstractUltrasound is a readily available, non-invasive and low-cost screening for the identification of endometriosis lesions, but its diagnostic specificity strongly depends on the experience of the operator. For this reason, computer-aided diagnosis tools based on Artificial Intelligence techniques can provide significant help to the clinical staff, both in terms of workload reduction and in increasing the overall accuracy of this type of examination and its outcome. However, although these techniques are spreading rapidly in a variety of domains, their application to endometriosis is still very limited. To fill this gap, we propose and evaluate a novel multi-scale ensemble approach for the automatic segmentation of endometriosis lesions from transvaginal ultrasounds. The peculiarity of the method lies in its high discrimination capability, obtained by combining, in a fusion fashion, multiple Convolutional Neural Networks trained on data at different granularity. The experimental validation carried out shows that: (i) the proposed method allows to significantly improve the performance of the individual neural networks, even in the presence of a limited training set; (ii) with a Dice coefficient of 82%, it represents a valid solution to increase the diagnostic efficacy of the ultrasound examination against such a pathology.

XAI-FusionNet: Diabetic foot ulcer detection based on multi-scale feature fusion with explainable artificial intelligence

Diabetic foot ulcer (DFU) poses a significant threat to individuals affected by diabetes, often leading to limb amputation. Early detection of DFU can greatly improve the chances of survival for diabetic patients. This work introduces FusionNet, a novel multi-scale feature fusion network designed to accurately differentiate DFU skin from healthy skin using multiple pre-trained convolutional neural network (CNN) algorithms. A dataset comprising 6963 skin images (3574 healthy and 3389 ulcer) from various patients was divided into training (6080 images), validation (672 images), and testing (211 images) sets. Initially, three image preprocessing techniques - Gaussian filter, median filter, and motion blur estimation - were applied to eliminate irrelevant, noisy, and blurry data. Subsequently, three pre-trained CNN algorithms -DenseNet201, VGG19, and NASNetMobile - were utilized to extract high-frequency features from the input images. These features were then inputted into a meta-tuner module to predict DFU by selecting the most discriminative features. Statistical tests, including Friedman and analysis of variance (ANOVA), were employed to identify significant differences between FusionNet and other sub-networks. Finally, three eXplainable Artificial Intelligence (XAI) algorithms - SHAP (SHapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), and Grad-CAM (Gradient-weighted Class Activation Mapping) - were integrated into FusionNet to enhance transparency and explainability. The FusionNet classifier achieved exceptional classification results with 99.05% accuracy, 98.18% recall, 100.00% precision, 99.09% AUC, and 99.08% F1 score. We believe that our proposed FusionNet will be a valuable tool in the medical field to distinguish DFU from healthy skin.

A Multiple Attention Convolutional Neural Networks for Diesel Engine Fault Diagnosis.

Fault diagnosis can improve the safety and reliability of diesel engines. An end-to-end method based on a multi-attention convolutional neural network (MACNN) is proposed for accurate and efficient diesel engine fault diagnosis. By optimizing the arrangement and kernel size of the channel and spatial attention modules, the feature extraction capability is improved, and an improved convolutional block attention module (ICBAM) is obtained. Vibration signal features are acquired using a feature extraction model alternating between the convolutional neural network (CNN) and ICBAM. The feature map is recombined to reconstruct the sequence order information. Next, the self-attention mechanism (SAM) is applied to learn the recombined sequence features directly. A Swish activation function is introduced to solve "Dead ReLU" and improve the accuracy. A dynamic learning rate curve is designed to improve the convergence ability of the model. The diesel engine fault simulation experiment is carried out to simulate three kinds of fault types (abnormal valve clearance, abnormal rail pressure, and insufficient fuel supply), and each kind of fault varies in different degrees. The comparison results show that the accuracy of MACNN on the eight-class fault dataset at different speeds is more than 97%. The testing time of the MACNN is much less than the machine running time (for one work cycle). Therefore, the proposed end-to-end fault diagnosis method has a good application prospect.

Enhancing Plant Disease Detection using Advanced Deep Learning Models

Objective: The aim of this paper is to analyze the efficacy of employing multiple advanced convolutional neural networks (CNNs) for the purpose of enhancing the accuracy in detecting and classifying various plant diseases. Methods: The research involves the analysis of 7623 training images as well as 1906 validation images of different plant diseases and employed advanced deep learning models like DenseNet169, Xception, InceptionV3, MobileNetV2, and ResNet50V2 to classify them. At first the RGB images are converted to Grayscale and later to enhance their quality, few techniques have been used such as Otsu thresholding, noise removal, distance transform, and watershed techniques. Subsequently, contour features are extracted by calculating morphological values to obtain the necessary region that correspond to diseased areas in plant images. Findings: On evaluating the performance of the applied models on the basis of various metrics, MobileNetV2 and ResNetV2 achieved the highest validation accuracy scores of 99.42% each, with their respective loss values of 0.19 and 0.49. In terms of recall, precision, and F1 score, all models, except MobileNetV2 and InceptionV3, attained optimal scores of 0.99 each. Novelty: The novelty of this paper resides in the incorporation of multiple image segmentation techniques with fine tuning the parameters of advanced Convolutional Neural Network (CNN) models on the basis of various factors such as the number of images, size, channel, classes etc to generate the optimal results. Keywords: Agriculture, Plant diseases, Artificial Intelligence, Advanced CNN models, Watershed Technique

Improving Speech Recognition with Convolutional Neural Networks

This project explores advanced techniques in speech recognition, focusing on emotion identification using Convolutional Neural Networks for improved accuracy and real-time processing efficiency. Emotion recognition from speech signals plays a crucial role in various applications, including human-computer interaction, customer service, mental health monitoring, and entertainment. This project proposes an innovative approach to emotion recognition using Convolutional Neural Networks (CNNs) applied to speech data. By leveraging advanced deep learning techniques, the proposed system aims to accurately identify and classify emotions conveyed through vocal expressions. The project begins with a comprehensive review of existing literature on emotion recognition and speech processing, identifying key challenges and opportunities in the field. Building upon prior research, the project introduces a novel CNN architecture optimized for emotion recognition tasks. This architecture is designed to extract relevant features from speech signals and capture subtle nuances indicative of different emotional states. One of the distinguishing features of the proposed approach is its multi-modal integration, which combines information from both audio and visual modalities to enhance emotion recognition accuracy. In addition to analysing speech signals, the system incorporates visual cues such as facial expressions and gestures, providing a more comprehensive understanding of the speaker's emotional state. Real-time processing efficiency is prioritized in the design of the system, ensuring prompt and responsive emotion recognition in interactive applications. Optimization techniques such as model quantization and lightweight architecture design are employed to minimize computational overhead while maintaining high accuracy. To address the variability and subjectivity of emotional expression, the system incorporates user-specific adaptation mechanisms. Through continuous learning and feedback integration, the system dynamically adapts to individual speakers' speech patterns and emotional characteristics, enhancing its ability to accurately recognize emotions in diverse contexts. The project also explores ensemble learning strategies to improve robustness and generalization performance. By combining predictions from multiple CNN models trained on diverse datasets, the system achieves greater resilience to variations in emotional expression and environmental factors. Ethical considerations, including privacy protection and responsible data handling, are integral aspects of the project's design and implementation. Measures are implemented to ensure the ethical collection, storage, and usage of speech data, safeguarding user privacy and maintaining trust in the system. Overall, the proposed system represents a significant advancement in emotion recognition technology, offering a sophisticated and versatile solution for accurately identifying emotions from speech signals. By leveraging deep learning techniques, multi-modal integration, real-time processing optimization, user-specific adaptation, and ensemble learning, the system demonstrates promising potential for various practical applications requiring robust and context-aware emotion recognition capabilities. Keywords: Speech Recognition, Emotion Identification, Convolutional Neural Networks (CNNs), Real-time Processing, Multi-modal Integration, User-specific Adaptation, Ensemble Learning, Deep Learning, Emotional Expression, Ethical Data Handling

Enhancing the medical diagnosis of COVID-19 with learning based decision support systems

Since late December 2019, the COVID-19 pandemic has had substantial impact and long-lasting impact on numerous lives. The surge in patients has overwhelmed hospitals and exhausted essential resources such as masks and gloves. However, in response to this crisis, we have developed a robust solution that can ease the burden on emergency services and manage the influx of patients. Our proposed framework comprises deep learning and machine learning models that can predict and manage patient demand with high accuracy. The first model, is specifically designed to classify computed tomography (CT) scan images for COVID or non-COVID cases. We trained multiple convolutional neural network (CNN) models on a large dataset of CT scan images and evaluated their performance on a separate test set. Our evaluation showed that the ResNet50 model was the most effective, achieving an accuracy of 93.28%. The second model uses patient measurements dataset to predict the likelihood of intensive care unit (ICU) admission for COVID-19 patients. We experimented with the XGBoost machine learning algorithm and found that the accuracy score achieved 88.40%.

Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks

Deep learning and Convolutional Neural Networks (CNNs) have driven major transformations in diverse research areas. However, their limitations in handling low-frequency in-formation present obstacles in certain tasks like interpreting global structures or managing smooth transition images. Despite the promising performance of transformer struc-tures in numerous tasks, their intricate optimization com-plexities highlight the persistent need for refined CNN en-hancements using limited resources. Responding to these complexities, we introduce a novel framework, the Mul-tiscale Low-Frequency Memory (MLFM) Network, with the goal to harness the full potential of CNNs while keep-ing their complexity unchanged. The MLFM efficiently preserves low-frequency information, enhancing perfor-mance in targeted computer vision tasks. Central to our MLFM is the Low-Frequency Memory Unit (LFMU), which stores various low-frequency data and forms a parallel channel to the core network. A key advantage of MLFM is its seamless compatibility with various prevalent networks, requiring no alterations to their original core structure. Testing on ImageNet demonstrated substantial accuracy improvements in multiple 2D CNNs, including ResNet, MobileNet, EfficientNet, and ConvNeXt. Furthermore, we showcase MLFM's versatility beyond traditional image classification by successfully integrating it into image-to-image translation tasks, specifically in semantic segmenta-tion networks like FCN and U-Net. In conclusion, our work signifies a pivotal stride in the journey of optimizing the ef-ficacy and efficiency of CNNs with limited resources. This research builds upon the existing CNN foundations and paves the way for future advancements in computer vision. Our codes are available at https://github.com/AlphaWuSeu/MLFM.

Hear You Say You: An Efficient Framework for Marine Mammal Sounds’ Classification

Marine mammals and their ecosystem face significant threats from, for example, military active sonar and marine transportation. To mitigate this harm, early detection and classification of marine mammals are essential. While recent efforts have utilized spectrogram analysis and machine learning techniques, there remain challenges in their efficiency. Therefore, we propose a novel knowledge distillation framework, named XCFSMN, for this problem. We construct a teacher model that fuses the features extracted from an X-vector extractor, a DenseNet and Cross-Covariance attended compact Feed-Forward Sequential Memory Network (cFSMN). The teacher model transfers knowledge to a simpler cFSMN model through a temperature-cooling strategy for efficient learning. Compared to multiple convolutional neural network backbones and transformers, the proposed framework achieves state-of-the-art efficiency and performance. The improved model size is approximately 20 times smaller and the inference time can be 10 times shorter without affecting the model’s accuracy.

Abstract 4938: Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline

Abstract Colorectal cancer (CRC) stands as one of the deadliest cancer types worldwide. Identifying specific gene mutations crucial for targeted therapy is pivotal for CRC treatment. However, the current genetic testing method involves liquid biopsy, which is often cumbersome and cost-intensive. Histopathological whole-slide images (WSIs) are routinely generated for diagnostic purposes and can offer valuable insights into cellular heterogeneity. In this study, we aim to develop an integrated deep learning pipeline for predicting TP53 mutations in CRC, one of the most frequently mutated genes in CRC. We commenced by collecting 424 diagnostic WSIs from the TCGA-COAD cohorts. Each WSI was split as small patches at a resolution of 0.5 micrometers per pixel for following analyses. Quality control was implemented using Otsu thresholding and Gaussian blur filtering, both at the slide-level masking and patch-level filtering stages. Additionally, we applied the Macenko algorithm for color normalization to eliminate systematic errors. Following image preprocessing, we constructed a tissue-type classification model based on VGG19, which was trained and tested using the NCT-CRC-HE-100K and CRC-VAL-HE-7K datasets, both containing pixels with tissue types labeled by human experts. The trained classifier was then used to identify adenocarcinoma epithelium patches from TCGA-COAD. Subsequently, we applied multiple pre-trained Convolutional Neural Network (CNN) models, including ResNet-101, DenseNet-201, and Inception-ResNet-v2, to predict TP53 mutations for the identified adenocarcinoma epithelium patches from TCGA-COAD. A novel integration approach was employed to aggregate patch-level predictions into slide-level outcomes. We evaluated the trained models using various metrics, including accuracy, precision, recall, F1 score, AUC (Area Under the Curve), and Cohen's Kappa coefficient. The tissue-type classification model exhibited robust performance, achieving an overall accuracy of 94.7%, a Cohen's Kappa coefficient of 94%, a precision of 95.3%, a recall of 99.76%, and an F1 score of 97.48% for identifying adenocarcinoma epithelium patches. Regarding the TP53 mutation prediction models, ResNet-101, DenseNet-201, and Inception-ResNet-v2 exhibit the predictive ability to both patch level (AUC=65%) and slide level (AUC=70%). In summary, the proposed deep learning pipeline offers an efficient way to acquire gene mutation information from WSIs. Moreover, it demonstrates a capability to discover potential molecular aberration from the clinical images. Citation Format: Geng-Yun Tien, Liang-Chuan Lai, Tzu-Pin Lu, Mong-Hsun Tsai, Hsiang-Han Chen, Eric Y. Chuang. Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4938.

Analysis of dermoscopy images of multi-class for early detection of skin lesions by hybrid systems based on integrating features of CNN models.

Skin cancer is one of the most fatal skin lesions, capable of leading to fatality if not detected in its early stages. The characteristics of skin lesions are similar in many of the early stages of skin lesions. The AI in categorizing diverse types of skin lesions significantly contributes to and helps dermatologists to preserve patients' lives. This study introduces a novel approach that capitalizes on the strengths of hybrid systems of Convolutional Neural Network (CNN) models to extract intricate features from dermoscopy images with Random Forest (Rf) and Feed Forward Neural Networks (FFNN) networks, leading to the development of hybrid systems that have superior capabilities early detection of all types of skin lesions. By integrating multiple CNN features, the proposed methods aim to improve the robustness and discriminatory capabilities of the AI system. The dermoscopy images were optimized for the ISIC2019 dataset. Then, the area of the lesions was segmented and isolated from the rest of the image by a Gradient Vector Flow (GVF) algorithm. The first strategy for dermoscopy image analysis for early diagnosis of skin lesions is by the CNN-RF and CNN-FFNN hybrid models. CNN models (DenseNet121, MobileNet, and VGG19) receive a region of interest (skin lesions) and produce highly representative feature maps for each lesion. The second strategy to analyze the area of skin lesions and diagnose their type by means of CNN-RF and CNN-FFNN hybrid models based on the features of the combined CNN models. Hybrid models based on combined CNN features have achieved promising results for diagnosing dermoscopy images of the ISIC 2019 dataset and distinguishing skin cancers from other skin lesions. The Dense-Net121-MobileNet-RF hybrid model achieved an AUC of 95.7%, an accuracy of 97.7%, a precision of 93.65%, a sensitivity of 91.93%, and a specificity of 99.49%.

Convolutional neural network-based magnetic resonance image differentiation of filum terminale ependymomas from schwannomas

PurposePreoperative diagnosis of filum terminale ependymomas (FTEs) versus schwannomas is difficult but essential for surgical planning and prognostic assessment. With the advancement of deep-learning approaches based on convolutional neural networks (CNNs), the aim of this study was to determine whether CNN-based interpretation of magnetic resonance (MR) images of these two tumours could be achieved.MethodsContrast-enhanced MRI data from 50 patients with primary FTE and 50 schwannomas in the lumbosacral spinal canal were retrospectively collected and used as training and internal validation datasets. The diagnostic accuracy of MRI was determined by consistency with postoperative histopathological examination. T1-weighted (T1-WI), T2-weighted (T2-WI) and contrast-enhanced T1-weighted (CE-T1) MR images of the sagittal plane containing the tumour mass were selected for analysis. For each sequence, patient MRI data were randomly allocated to 5 groups that further underwent fivefold cross-validation to evaluate the diagnostic efficacy of the CNN models. An additional 34 pairs of cases were used as an external test dataset to validate the CNN classifiers.ResultsAfter comparing multiple backbone CNN models, we developed a diagnostic system using Inception-v3. In the external test dataset, the per-examination combined sensitivities were 0.78 (0.71–0.84, 95% CI) based on T1-weighted images, 0.79 (0.72–0.84, 95% CI) for T2-weighted images, 0.88 (0.83–0.92, 95% CI) for CE-T1 images, and 0.88 (0.83–0.92, 95% CI) for all weighted images. The combined specificities were 0.72 based on T1-WI (0.66–0.78, 95% CI), 0.84 (0.78–0.89, 95% CI) based on T2-WI, 0.74 (0.67–0.80, 95% CI) for CE-T1, and 0.81 (0.76–0.86, 95% CI) for all weighted images. After all three MRI modalities were merged, the receiver operating characteristic (ROC) curve was calculated, and the area under the curve (AUC) was 0.93, with an accuracy of 0.87.ConclusionsCNN based MRI analysis has the potential to accurately differentiate ependymomas from schwannomas in the lumbar segment.

PhosAF: An integrated deep learning architecture for predicting protein phosphorylation sites with AlphaFold2 predicted structures

Phosphorylation is indispensable in comprehending biological processes, while biological experimental methods for identifying phosphorylation sites are tedious and arduous. With the rapid growth of biotechnology, deep learning methods have made significant progress in site prediction tasks. Nevertheless, most existing predictors only consider protein sequence information, that limits the capture of protein spatial information. Building upon the latest advancement in protein structure prediction by AlphaFold2, a novel integrated deep learning architecture PhosAF is developed to predict phosphorylation sites in human proteins by integrating CMA-Net and MFC-Net, which considers sequence and structure information predicted by AlphaFold2. Here, CMA-Net module is composed of multiple convolutional neural network layers and multi-head attention is appended to obtaining the local and long-term dependencies of sequence features. Meanwhile, the MFC-Net module composed of deep neural network layers is used to capture the complex representations of evolutionary and structure features. Furthermore, different features are combined to predict the final phosphorylation sites. In addition, we put forward a new strategy to construct reliable negative samples via protein secondary structures. Experimental results on independent test data and case study indicate that our model PhosAF surpasses the current most advanced methods in phosphorylation site prediction.

Semantic-aware automatic extraction method for bottom sediment annotations in raster nautical charts

ABSTRACT The automatic extraction of bottom sediment annotations in large-scale raster nautical charts has limitations, including an imprecise semantic information description and low efficiency. To overcome them, we propose a convolutional neural network (CNN)-based method for the automatic extraction of bottom sediment annotations in raster nautical charts, using image processing techniques to improve it. First, an adaptive chart partitioning model that considers element completeness is constructed. Second, a principle for the unique identification of elements based on spatial conflicts is designed. Finally, a model for accurately extracting semantic information for bottom sediment annotations is established. To evaluate the effectiveness of the proposed method, we implemented a model based on the PyTorch framework and used the PIL library to analyze the results. We also conducted comparative experiments on multiple CNN models to recommend the selection of such models in the proposed method by comparing their classification and recognition performance. The experimental results indicate that (1) the proposed model can achieve high-precision extraction of bottom sediment annotations in raster nautical charts. (2) Furthermore, the proposed model generally has high recognition accuracy and semantic completeness, with better recognition precision than traditional pattern recognition methods.

Automatic Detection of Rice Blast Fungus Spores by Deep Learning-Based Object Detection: Models, Benchmarks and Quantitative Analysis

The severity of rice blast and its impacts on rice yield are closely related to the inoculum quantity of Magnaporthe oryzae, and automatic detection of the pathogen spores in microscopic images can provide a rapid and effective way to quantify pathogen inoculum. Traditional spore detection methods mostly rely on manual feature extraction and shallow machine learning models, and are mostly designed for the indoor counting of a single spore class, which cannot handle the interference of impurity particles in the field. This study achieved automatic detection of rice blast fungus spores in the mixture with other fungal spores and rice pollens commonly encountered under field conditions by using deep learning based object detection techniques. First, 8959 microscopic images of a single spore class and 1450 microscopic images of mixed spore classes, including the rice blast fungus spores and four common impurity particles, were collected and labelled to form the benchmark dataset. Then, Faster R-CNN, Cascade R-CNN and YOLOv3 were used as the main detection frameworks, and multiple convolutional neural networks were used as the backbone networks in training of nine object detection algorithms. The results showed that the detection performance of YOLOv3_DarkNet53 is superior to the other eight algorithms, and achieved 98.0% mean average precision (intersection over union > 0.5) and an average speed of 36.4 frames per second. This study demonstrated the enormous application potential of deep object detection algorithms in automatic detection and quantification of rice blast fungus spores.

An Intelligent Classification Framework for Complex PQDs Using Optimized KS-Transform and Multiple Fusion CNN

Intelligent classification of multiple power quality disturbances (PQDs) is a top priority in pollution control of the power grid. However, the large-scale application of renewable energy introduces lots of nonlinear and impact loads, which makes the PQDs more complex and challenges the effectiveness of conventional detection frameworks. In this paper, a novel framework based on optimized Kaiser window-based S-transform (OKST) and multiple fusion convolutional neural network (MFCNN) is proposed to identify multiple complex PQDs. First, the OKST is used for the time-frequency positioning of PQDs, where an improved control function is proposed to meet different detection requirements of time-frequency. Additionally, the parameters of the control function are adjusted automatically using maximum energy concentration. Then, the MFCNN based on residual networks (ResNets) is further proposed to extract and classify these time-frequency features automatically. In MFCNN, feature information is fused using different convolution kernels at a 2-dimensional level, which can effectively reduce information loss and improve classification performance. The network model is set up using the Pytorch platform, and the dataset containing 28 types of PQDs and 2 types of nonlinearly mixed PQDs is built to test our framework. The result shows that the proposed OKST-MFCNN obtains an average accuracy of 99.38% under the 20 dB noise level, which is more accurate and robust than some advanced PQDs detection frameworks. Moreover, the accuracy of 97.94% is achieved with satisfactory real-time performance in hardware platform experiments, proving its superior identification performance for complex PQDs.

P461 Deep learning model for endoscopic differential diagnosis between intestinal lymphoma and Crohn's disease: a multicenter study

Abstract Background Primary and secondary intestinal lymphoma (IL) are a relatively rare cancer, while easily misdiagnosed due to the unclear aetiology and unspecific clinical symptoms. Moreover, the endoscopic features of IL vary depending on pathological types, making it difficult to distinguish from Crohn's disease (CD). This study aims to develop deep learning models based on endoscopic images, facilitating the differential diagnosis of IL and CD. Methods We retrospectively recruited 271 patients with IL and 232 patients with CD from 6 tertiary centres in China. With out-of-focus or poor-bowel-prepared images excluded, we used 1739 IL and 1649 CD images for internal and external validation to build deep learning models. For internal validation, we used five-fold cross-validation with 1517 IL images and 1428 CD images from 5 centres randomly assigned without any patient overlap. 222 IL and 221 CD images from the 6th centre were used for independent external validation to measure the generalization ability of models. Further, we compared the performance of models with that of clinical physicians consisting of 2 junior physicians and 2 endoscopy specialists with more than ten years of clinical experience. We evaluated models using metrics such as accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUC) and used the kappa statistic to measure the inter-rater reliability between models and clinical physicians. Results We developed multiple convolutional neural network models, including ResNet18, ResNet34, ResNet50, VGG16, VGG19, DenseNet121, Inception v3, and Xception. The top 3 models in the internal validation were Inception v3, Xception, and DenseNet121. Inception v3 showed the best performance (accuracy: 83.1%; AUC: 0.910), while Xception performed better on average (average accuracy: 76.8%; average AUC: 0.834). In the external validation, the top 3 models were Inception v3, DenseNet121, and VGG19. Inception v3 achieved the highest accuracy of 81.9%, while DenseNet121 had the best AUC of 0.903. Compared with clinical physicians, Inception v3 performed better than junior physicians (average accuracy: 67.2%) and showed good agreement with the best endoscopy specialist (kappa: 0.604, p<0.05). Conclusion Convolutional neural network models using endoscopic images were able to differentiate PIL from CD, with better-diagnosing performance than junior physicians, good agreement with specialists, and good generation ability, which were expected to assist clinical physicians lacking experience to differentiate PIL and CD more precisely under endoscope.

Multi-objective compression for CNNs via evolutionary algorithm

Filter pruning has been successfully applied in convolutional neural networks (CNN) to reduce model redundancy and speed up inference. Current pruning algorithms require setting a target value for a pruned model, which is not conducive to the rapid implementation in practical application. Firstly, for a target model, it may be necessary to pay attention to multiple performances at the same time, but it is hard to keep the best trade-off among indicators. Secondly, it is time-consuming to find a suitable model by pruning in a trial-and-error way. Therefore, this paper attempts to combine the most concerning indicators into a search target, rather than producing a model that only contains some pre-defined constraint. Specifically, the multi-object differential evolution algorithm is introduced into the search process, which is used to obtain a set of sparse structures that have taken all useful indicators into account. Then select proper sub-network to retrain from search results according to actual requirements. Experiments on multiple advanced CNN networks show that our algorithm can compress different indicators while keeping comparable or better performance than prior works. On ImageNet, pruning based on ResNet50 reduces FLOPs by 60.4%, while the accuracy of top-1 and top-5 only loses 1.09% and 0.57%, respectively.

Research on Walnut (Juglans regia L.) Classification Based on Convolutional Neural Networks and Landsat-8 Remote Sensing Imagery

The study explores the use of convolutional neural networks (CNNs) and satellite remote sensing imagery for walnut analysis in Ganquan Township, Alar City, Xinjiang. The recent growth of walnut cultivation in Xinjiang presents challenges for manual data collection, making satellite imagery and computer vision algorithms a practical solution. Landsat-8 satellite images from Google Earth Engine underwent preprocessing, and experiments were conducted to enhance the ResNet model, resulting in improved accuracy and efficiency. Experiments were conducted to evaluate multiple CNN models and traditional methods, and the best detection method was chosen through comparisons. A comparison was drawn between traditional algorithms and convolutional neural network algorithms based on metrics such as precision, recall, f1-score, accuracy, and total time. The results indicated that although traditional methods were more efficient compared to CNN, they exhibited lower accuracy. In the context of this research, prioritizing efficiency at the cost of accuracy was deemed undesirable. Among the traditional algorithms employed in this study, k-NN produced the most favorable outcomes, with precision, recall, f1-score, and accuracy reaching 75.78%, 92.43%, 83.28%, and 84.46%, respectively, although these values were relatively lower than those of the CNN algorithm models. Within the CNN models, the ResNet model demonstrated superior performance, yielding corresponding results of 92.47%, 94.29%, 93.37%, and 93.27%. The EfficientNetV2 model also displayed commendable results, with precision, recall, and f1-score achieving 96.35%, 91.44%, and 93.83%. Nevertheless, it is worth noting that the classification efficiency of EfficientNetV2 fell significantly short of that of ResNet. Consequently, in this study, the ResNet model proved to be relatively more effective. Once optimized, the most efficient CNN model closely rivals traditional algorithms in terms of time efficiency for generating results while significantly surpassing them in accuracy. Through our studies, we discovered that once optimized, the most efficient CNN model closely rivals traditional algorithms in terms of time efficiency for generating results while significantly surpassing them in accuracy. In this study, empirical evidence demonstrates that integrating CNN-based methods with satellite remote sensing technology can effectively enhance the statistical efficiency of agriculture and forestry sectors, thus leading to substantial reductions in operational costs. These findings lay a solid foundation for further research in this field and offer valuable insights for other agricultural and forestry-related studies.