Pre-trained Network Research Articles

Leveraging deep learning for coastal monitoring: A VGG16-based approach to spectral and textural classification of coastal areas with sentinel-2A data

Coastal ecosystems are vital for the planet's health, providing essential habitats for diverse species and supporting human communities. However, these complex environments face increasing threats from climate change and human activities. Effective monitoring of these areas requires large-scale, efficient, and accurate methods. This study explores the potential of deep learning for automated coastal land cover classification using Sentinel-2A satellite imagery on the Google Earth Engine (GEE) platform. We investigate the impact of transfer learning and spectral band combinations on classification accuracy for five coastal types: artificial, bedrock, sandy, muddy, and vegetation-covered. Our findings demonstrate that a pre-trained VGG16 Convolutional Neural Network (CNN) with transfer learning significantly improves classification accuracy (average 19.3% increase) compared to using default weights. Notably, including the near-infrared (NIR) band in training data leads to superior results, particularly for artificial and bedrock coastlines, where the NIR band's effectiveness in separating land-water boundaries enhances classification accuracy. These results highlight the potential of deep learning for large-scale, automated coastal monitoring, informing applications in sustainable fisheries management, coastal vulnerability assessment, and marine ecosystem conservation.

Fault diagnosis of air compressors using transfer learning: A comparative study of pre-trained networks and hyperparameter optimization

Air compressors are critical components in many industries whose catastrophic failure results in huge financial losses and downtime leading to accidents. Hence, real time fault diagnosis of air compressor is essential to predict the health condition of air compressor and plan scheduled maintenance thereby reducing financial losses and accidents. Fault diagnosis using transfer learning aids in real time fault detection. In the present study, five air compressor conditions were considered namely, check valve fault, inlet and outlet reed valve fluttering fault, inlet reed valve fluttering fault, outlet reed valve fluttering fault, and good condition. The raw vibration data was converted to radar plot images that were pre-processed and classified using four pre-trained networks (ResNet-50, GoogLeNet, AlexNet, and VGG-16). The hyperparameters like epochs, batch size, optimizer, train-test split ratio, and learning rate were varied to find out the best network for air compressor fault diagnosis. ResNet-50 among all other pre-trained networks produced the maximum classification accuracy (average of five trials) of 98.72%.

Anti Face Spoofing System

Face spoofing, the act of using deceptive techniques such as printed photos or digital images to deceive facial recognition systems, poses a significant threat to the security of various applications, including biometric authentication and access control systems. This paper presents a concise yet effective approach to address the challenge of anti-face spoofing using Python within a limited codebase of 50 lines. The proposed solution leverages a combination of image processing techniques and machine learning algorithms to detect and prevent face spoofing attempts. A pre-trained deep neural network model for facial recognition is employed to extract essential facial features. Subsequently, the system utilizes image manipulation detection methods to identify anomalies indicative of face spoofing attacks. The implementation showcases the simplicity and efficiency of the proposed anti-face spoofing technique, demonstrating the potential for integration into real- world applications with minimal computational overhead. The concise Python code enables easy adoption and adaptation for developers aiming to enhance the security of facial recognition systems against face spoofing threats. The experimental results demonstrate the effectiveness of the approach in accurately distinguishing between genuine and spoofed facial images, thereby contributing to the robustness of facial recognition systems in the presence of adversarial attacks. Keywords - Liveness Detection, Presentation Attack Detection, Spoof Detection Algorithms, Biometric Security, Face Recognition Robustness, 3D Face Modeling, Texture Analysis, Deep Learning Anti- Spoofing, Multi-Spectral Imaging, Behavioral Biometrics, Challenge-Response Mechanisms, Optical Flow Analysis, Pulse Detection, Surface Reflectance, Depth Sensing, Motion Analysis, Pattern Recognition, Anti-Spoofing Networks, Temporal Feature Extraction, Anomaly Detection.

Deep SqueezeNet learning model for diagnosis and prediction of maize leaf diseases

The maize leaf diseases create severe yield reductions and critical problems. The maize leaf disease should be discovered early, perfectly identified, and precisely diagnosed to make greater yield. This work studies three main leaf diseases: common rust, blight, and grey leaf spot. This approach involves pre-processing, including sampling and labelling, while ensuring class balance and preventing overfitting via the SMOTE algorithm. The maize leaf dataset with augmentation was used to classify these diseases using several deep-learning pre-trained networks, including VGG16, Resnet34, Resnet50, and SqueezeNet. The model was evaluated using a maize leaf dataset that included various leaf classes, mini-batch sizes, and input sizes. Performance measures, recall, precision, accuracy, F1-score, and confusion matrix were computed for each network. The SqueezeNet learning model produces an accuracy of 97% in classifying four different classes of plant leaf datasets. Comparatively, the SqueezeNet learning model has improved accuracy by 2–5% and reduced the mean square error by 4–11% over VGG16, Resnet34, and Resnet50 deep learning models.

Fast video anomaly detection via context-aware shortcut exploration and abnormal feature distance learning

Surveillance systems are essential in computer vision, with video anomaly detection (VAD) critical for real-world security. Conventional approaches anticipate abnormal frames using normal patterns learned with normal training data and determine anomalies by comparing the output and input. However, they often incur high computational costs due to pre-trained networks and complex modules. Also, they suffer from the tendency to overlook anomalies due to strong generalizing ability. To address these, we propose two innovations: Anomaly Distance Learning (ADL) and Context-Aware Skip Connection (CONASkip), maximizing the distinction between normal and abnormal samples. ADL increases the feature gap by learning to separate normal and abnormal features, while CONASkip selectively connects layers to preserve normal information and differentiate output quality. These methods operate on self-supervised signals, replacing complex and costly modules. Additionally, ADL is excluded during testing, enabling fast inference. Our model achieves 130 frames per second and superior performance on three benchmarks, addressing real-world efficiency challenges.

Real-time sign language detection: Empowering the disabled community

Interaction and communication for normal human beings are easier than for a person with disabilities like speaking and hearing who may face communication problems with other people. Sign Language helps reduce this communication gap between a normal and disabled person. The prior solutions proposed using several deep learning techniques, such as Convolutional Neural Networks, Support Vector Machines, and K-Nearest Neighbors, have either demonstrated low accuracy or have not been implemented as real-time working systems. This system addresses both issues effectively. This work extends the difficulties faced while classifying the characters in Indian Sign Language(ISL). It can identify a total of 23 hand poses of the ISL. The system uses a pre-trained VGG16 Convolution Neural Network(CNN) with an attention mechanism. The model's training uses the Adam optimizer and cross-entropy loss function. The results demonstrate the effectiveness of transfer learning for ISL classification, achieving an accuracy of 97.5 % with VGG16 and 99.8 % with VGG16 plus attention mechanism.•Enabling quick and accurate sign language recognition with the help of trained model VGG16 with an attention mechanism.•The system does not require any external gloves or sensors, which helps to eliminate the need for physical sensors while simplifying the process with reduced costs.•Real-time processing makes the system more helpful for people with speaking and hearing disabilities, making it easier for them to communicate with other humans.

Transfer learning may explain pigeons’ ability to detect cancer in histopathology

Pigeons' unexpected competence in learning to categorize unseen histopathological images has remained an unexplained discovery for almost a decade (Levensonet al2015PLoS One10e0141357). Could it be that knowledge transferred from their bird's-eye views of the earth's surface gleaned during flight contributes to this ability? Employing a simulation-based verification strategy, we recapitulate this biological phenomenon with a machine-learning analog. We model pigeons' visual experience during flight with the self-supervised pre-training of a deep neural network on BirdsEyeViewNet; our large-scale aerial imagery dataset. As an analog of the differential food reinforcement performed in Levensonet al's study 2015PLoS One10e0141357), we apply transfer learning from this pre-trained model to the same Hematoxylin and Eosin (H&E) histopathology and radiology images and tasks that the pigeons were trained and tested on. The study demonstrates that pre-training neural networks with bird's-eye view data results in close agreement with pigeons' performance. These results support transfer learning as a reasonable computational model of pigeon representation learning. This is further validated with six large-scale downstream classification tasks using H&E stained whole slide image datasets representing diverse cancer types.

Introducing Methods for Analyzing and Detecting Concrete Cracks at the No. 3 Huaiyin Pumping Station in the South-to-North Water Diversion Project in China

Concrete cracks pose significant threats to concrete structures, causing immediate strength loss and leading to gradual erosion that compromises structural integrity. Therefore, accurate and automatic detection and classification of concrete cracks, along with the evaluation of their effects on target structures, are critically important. This study focuses on the No. 3 Huaiyin pumping station, a large-scale hydraulic structure on the Eastern Route of the South-to-North Water Diversion Project in Jiangsu, China. First, relevant field test literature is reviewed, and the finite element method is applied to investigate the effects of an existing crack on the No. 2 supporting wall. Using thermomechanically coupled numerical simulations, the distribution of tensile stress in the supporting wall is reported in two cases: without a crack and with an existing crack. The findings indicate that the increase in tensile stress due to the existing crack is relatively small and can be considered negligible for the No. 2 supporting wall. Next, the pretrained YOLOX network for the detection and classification of three types of cracks is proposed and retrained using collected concrete crack datasets. The mean average precision of the retrained YOLOX network for all three types of cracks reaches 80%. Finally, the retrained YOLOX network is applied to detect and classify cracks at the No. 3 Huaiyin pumping station. This automatic detection and classification approach will enhance the high-quality management of the pumping station because it is labor-saving and easy to deploy.

Weakly Supervised Lymph Nodes Segmentation Based on Partial Instance Annotations with Pre-trained Dual-branch Network and Pseudo Label Learning

Assessing the presence of potentially malignant lymph nodes aids in estimating cancer progression, and identifying surrounding benign lymph nodes can assist in determining potential metastatic pathways for cancer. For quantitative analysis, automatic segmentation of lymph nodes is crucial. However, due to the labor-intensive and time-consuming manual annotation process required for a large number of lymph nodes, it is more practical to annotate only a subset of the lymph node instances to reduce annotation costs. In this study, we propose a pre-trained Dual-Branch network with Dynamically Mixed Pseudo label (DBDMP) to learn from partial instance annotations for lymph nodes segmentation. To obtain reliable pseudo labels for lymph nodes that are not annotated, we employ a dual-decoder network to generate different outputs that are then dynamically mixed. We integrate the original weak partial annotations with the mixed pseudo labels to supervise the network. To further leverage the extensive amount of unannotated voxels, we apply a self-supervised pre-training strategy to enhance the model’s feature extraction capability. Experiments on the mediastinal Lymph Node Quantification (LNQ) dataset demonstrate that our method, compared to directly learning from partial instance annotations, significantly improves the Dice Similarity Coefficient (DSC) from 11.04% to 54.10% and reduces the Average Symmetric Surface Distance (ASSD) from 20.83 mm to 8.72 mm. The code is available at <a href='https://github.com/WltyBY/LNQ2023_training_code'>https://github.com/WltyBY/LNQ2023_training_code</a>

SLSG: Industrial image anomaly detection with improved feature embeddings and one-class classification

Industrial image anomaly detection under the setting of one-class classification has significant practical value. However, most existing models face challenges in extracting separable feature representations when performing feature embedding and in constructing compact descriptions of normal features when performing one-class classification. One direct consequence is that most models perform poorly in detecting logical anomalies that violate contextual relationships. Focusing on more effective and comprehensive anomaly detection, we propose a network based on self-supervised learning and self-attentive graph convolution (SLSG). SLSG uses a generative pre-training network to assist the encoder in learning the embedding of normal patterns and the reasoning of positional relationships. Subsequently, we introduce pseudo-prior knowledge of anomalies in SLSG using simulated abnormal samples. By comparing the simulated anomalies, SLSG can better summarize the normal patterns and narrow the hypersphere used for one-class classification. In addition, with the construction of a more general graph structure, SLSG comprehensively models the dense and sparse relationships among the elements in an image, which further strengthens the detection of logical anomalies. Extensive experiments on benchmark datasets show that SLSG achieves superior anomaly detection performance, demonstrating the effectiveness of our method.

Deep Convolutional Backbone Comparison for Automated PET Image Quality Assessment.

Pretraining deep convolutional network mappings using natural images helps with medical imaging analysis tasks; this is important given the limited number of clinically-annotated medical images. Many two-dimensional pretrained backbone networks, however, are currently available. This work compared 18 different backbones from 5 architecture groups (pretrained on ImageNet) for the task of assessing [18F]FDG brain Positron Emission Transmission (PET) image quality (reconstructed at seven simulated doses), based on three clinical image quality metrics (global quality rating, pattern recognition, and diagnostic confidence). Using two-dimensional randomly sampled patches, up to eight patients (at three dose levels each) were used for training, with three separate patient datasets used for testing. Each backbone was trained five times with the same training and validation sets, and with six cross-folds. Training only the final fully connected layer (with ~6,000-20,000 trainable parameters) achieved a test mean-absolute-error of ~0.5 (which was within the intrinsic uncertainty of clinical scoring). To compare "classical" and over-parameterized regimes, the pretrained weights of the last 40% of the network layers were then unfrozen. The mean-absolute-error fell below 0.5 for 14 out of the 18 backbones assessed, including two that previously failed to train. Generally, backbones with residual units (e.g. DenseNets and ResNetV2s), were suited to this task, in terms of achieving the lowest mean-absolute-error at test time (~0.45 - 0.5). This proof-of-concept study shows that over-parameterization may also be important for automated PET image quality assessments.

Analyzing deep learning algorithms with statistical methods

A number of farmers use human inspection to classify the quality of their fruits visually; this process could be done automatically with a Learning algorithm. This paper investigates the classification of apples, peaches, and oranges and verifies the quality of classification with the density of the edges detected in each image by the Sobel filter. The novelty of the study consists of comparing the classification accuracy of a convolutional neural network (CNN) and the statistical analysis of edge histograms. The edges are the main elements of what the CNN learns; this was the reason for verifying the deep learning algorithms with skewness, kurtosis, and standard deviation metrics extracted from the histogram of edge density detected by the Sobel filter. In the empirical process, four algorithms were proposed: a CNN with one convolutional layer, a DNN without convolutional layers, and two pre-trained networks, AlexNet and GoogLeNet. From clustering of the row data and analyzing the accuracy, the best classification is obtained by simple CNN for 20 epochs when the apples vs. oranges were classified.

Lumbar intervertebral disc detection and classification with novel deep learning models

Low back pain (LBP) is a prevalent spinal issue, affecting eight out of ten individuals. Notably, lumbar intervertebral disc (IVD) abnormalities frequently contribute to LBP. To diagnose LBP, Magnetic Resonance Imaging (MRI) is crucial for obtaining detailed spinal images. This paper employs deep learning (DL) to detect and locate lumbar IVD in sagittal MR images. It further classifies lumbar IVDs as healthy or herniated, utilizing both novel convolutional neural network (CNN) and conventional CNN models. The dataset utilized comprises MR images from 32 patients, with 10 exhibiting healthy discs and the remaining 22 posing a mix of healthy and herniated discs, totaling 160 lumbar discs, incorporating 112 healthy and 48 herniated discs. In this study, ResNet-50 architecture in the Novel Lumbar IVD detection (NLID) model served as the feature extractor to segment the five lumbar IVDs from MR images. The features extracted from ResNet-50 were input into YOLOv2 for the identification of the region of interest (ROI). The findings indicate that optimal performance was achieved at the 22nd Rectified Linear Unit (ReLU) activation layer, boasting a remarkable 99.59% average precision, 97.22% F1-score, 94.59% precision, and a perfect 100% recall. This commendable performance consistently held above the 85% threshold until the 22nd ReLU activation layer. Regarding imbalanced dataset classification, AlexNet emerged as the frontrunner among other pre-trained networks, boasting the highest test accuracy of 90.63%, and an impressive F1 score of 88.77%. Meanwhile, the Novel Lumbar IVD Classification (NLIC) model achieved superior results with 93.75% test accuracy, and 92.27% F1-score. In the setting of the balanced dataset, NLIC achieved 96.88% test accuracy, and 96.46% F1-score with fewer epochs compared to AlexNet, affirming the robustness of the novel trained-from-scratch network. These findings distinctly underscore the effectiveness of CNNs in both medical image segmentation and classification.

Imperceptible backdoor watermarks for speech recognition model copyright protection

As the demand for shared pre-trained deep neural network models continues to rise, safeguarding the intellectual property of models is also increasingly significant. While existing studies predominantly concentrate on protecting pre-trained image recognition models, limited research covers pre-trained speech recognition models. In this paper, we propose the black-box watermark method to authenticate the ownership of speech recognition models. This method can mitigate the risk of unauthorized AI services being created by attackers who gain access to the pre-trained model. Accordingly, we present three watermarking methods: Gaussian noise watermark, extreme frequency Gaussian noise watermark, and unrelated audio watermark. These generated watermarks, embedded into models through training or fine-tuning, exhibit remarkable fidelity and effectiveness, backed by rigorous experimental validation. Furthermore, our experiments reveal that the extreme frequency noise backdoor enhances the robustness of the watermark compared to the Gaussian noise and unrelated audio watermark.

MuCoCP: a priori chemical knowledge-based multimodal contrastive learning pre-trained neural network for the prediction of cyclic peptide membrane penetration ability.

There has been a burgeoning interest in cyclic peptide therapeutics due to their various outstanding advantages and strong potential for drug formation. However, it is undoubtedly costly and inefficient to use traditional wet lab methods to clarify their biological activities. Using artificial intelligence instead is a more energy-efficient and faster approach. MuCoCP aims to build a complete pre-trained model for extracting potential features of cyclic peptides, which can be fine-tuned to accurately predict cyclic peptide bioactivity on various downstream tasks. To maximize its effectiveness, we use a novel data augmentation method based on a priori chemical knowledge and multiple unsupervised training objective functions to greatly improve the information-grabbing ability of the model. To assay the efficacy of the model, we conducted validation on the membrane-permeability of cyclic peptides which achieved an accuracy of 0.87 and R-squared of 0.503 on CycPeptMPDB using semi-supervised training and obtained an accuracy of 0.84 and R-squared of 0.384 using a model with frozen parameters on an external dataset. This result has achieved state-of-the-art, which substantiates the stability and generalization capability of MuCoCP. It means that MuCoCP can fully explore the high-dimensional information of cyclic peptides and make accurate predictions on downstream bioactivity tasks, which will serve as a guide for the future de novo design of cyclic peptide drugs and promote the development of cyclic peptide drugs. All code used in our proposed method can be found at https://github.com/lennonyu11234/MuCoCP.

Anchor objects drive realism while diagnostic objects drive categorization in GAN generated scenes

Our visual surroundings are highly complex. Despite this, we understand and navigate them effortlessly. This requires transforming incoming sensory information into representations that not only span low- to high-level visual features (e.g., edges, object parts, objects), but likely also reflect co-occurrence statistics of objects in real-world scenes. Here, so-called anchor objects are defined as being highly predictive of the location and identity of frequently co-occuring (usually smaller) objects, derived from object clustering statistics in real-world scenes, while so-called diagnostic objects are predictive of the larger semantic context (i.e., scene category). Across two studies (N1 = 50, N2 = 44), we investigate which of these properties underlie scene understanding across two dimensions – realism and categorisation – using scenes generated from Generative Adversarial Networks (GANs) which naturally vary along these dimensions. We show that anchor objects and mainly high-level features extracted from a range of pre-trained deep neural networks (DNNs) drove realism both at first glance and after initial processing. Categorisation performance was mainly determined by diagnostic objects, regardless of realism, at first glance and after initial processing. Our results are testament to the visual system’s ability to pick up on reliable, category specific sources of information that are flexible towards disturbances across the visual feature-hierarchy.

Single-frame interferogram phase retrieval using a phase-shifting generative adversarial network with physics-based fine-tuning

Phase retrieval from a single-frame interferogram is a challenge in optical interferometry. This paper proposes an accurate physics-based deep learning method for one-shot phase retrieval. This approach involves both data-driven pre-training of a phase-shifting network and subsequent model-driven fine-tuning. The well-designed pre-training network is capable of simultaneously generating π/2, π, and 3π/2 phase-shifted versions of the input interferogram to facilitate phase extraction. Moreover, integrating the interferometric model into the testing dataset enables self-supervised fine-tuning, optimizing the use of both data and physics-based priors. Simulations and experiments demonstrate the effectiveness of the proposed method in overcoming the common generalization limitation of data-driven models and achieving accurate phase retrieval. The proposed method not only enhances the accuracy of phase retrieval but also improves the generalization capability, making it robust under experimental conditions for interferometric applications.

Hyperspectral Image Denoising by Pixel-Wise Noise Modeling and TV-Oriented Deep Image Prior

Model-based hyperspectral image (HSI) denoising methods have attracted continuous attention in the past decades, due to their effectiveness and interpretability. In this work, we aim at advancing model-based HSI denoising, through sophisticated investigation for both the fidelity and regularization terms, or correspondingly noise and prior, by virtue of several recently developed techniques. Specifically, we formulate a novel unified probabilistic model for the HSI denoising task, within which the noise is assumed as pixel-wise non-independent and identically distributed (non-i.i.d) Gaussian predicted by a pre-trained neural network, and the prior for the HSI image is designed by incorporating the deep image prior (DIP) with total variation (TV) and spatio-spectral TV. To solve the resulted maximum a posteriori (MAP) estimation problem, we design a Monte Carlo Expectation–Maximization (MCEM) algorithm, in which the stochastic gradient Langevin dynamics (SGLD) method is used for computing the E-step, and the alternative direction method of multipliers (ADMM) is adopted for solving the optimization in the M-step. Experiments on both synthetic and real noisy HSI datasets have been conducted to verify the effectiveness of the proposed method.

Fault diagnosis of monoblock centrifugal pumps using pre-trained deep learning models and scalogram images

The monoblock centrifugal pump (MCP) is widely utilized in a diverse range of applications encompassing residential and industrial usage. Sectors such as agriculture, civil projects, mine dewatering, and numerous other industrial applications have employed centrifugal pumps. Despite their extensive usage, these pumps are susceptible to faults and failures due to the presence of critical components that are prone to issues such as bearing faults, sealing problems, cavitation and impeller faults. Therefore, conducting timely fault diagnosis becomes crucial to ensure uninterrupted operation. To address this, the technique of transfer learning, a form of deep learning, is employed. This method entails utilizing prior knowledge from previous operations to improve fault diagnostic performance in monoblock centrifugal pumps. Specifically, scalogram images derived from vibration signals collected during experimental setups were used in fault diagnosis. The study classified faults in monoblock centrifugal pumps using fifteen pre-trained networks including DenseNet-201, GoogLeNet, VGG-19, InceptionResNetV2, Xception, ShuffleNet, VGG-16, InceptionV3, ResNet101, ResNet-50, EfficientNetb0, NasNetmobile, ResNet-18, AlexNet and MobileNet-v2. The highest classification accuracy was obtained by carefully adjusting the hyperparameters which were subsequently employed in the fault classification process. AlexNet, one of the pre-trained network models, showcased remarkable capabilities by achieving a perfect classification accuracy of 100% within a relatively fast computation time of 18 s. This approach employs a reliable and effective process for discovering defects from the start, lowering the risk of possible damage and ensuring the seamless operation of the system.

Sunpheno: A Deep Neural Network for Phenological Classification of Sunflower Images.

Leaf senescence is a complex trait which becomes crucial for grain filling because photoassimilates are translocated to the seeds. Therefore, a correct sync between leaf senescence and phenological stages is necessary to obtain increasing yields. In this study, we evaluated the performance of five deep machine-learning methods for the evaluation of the phenological stages of sunflowers using images taken with cell phones in the field. From the analysis, we found that the method based on the pre-trained network resnet50 outperformed the other methods, both in terms of accuracy and velocity. Finally, the model generated, Sunpheno, was used to evaluate the phenological stages of two contrasting lines, B481_6 and R453, during senescence. We observed clear differences in phenological stages, confirming the results obtained in previous studies. A database with 5000 images was generated and was classified by an expert. This is important to end the subjectivity involved in decision making regarding the progression of this trait in the field and could be correlated with performance and senescence parameters that are highly associated with yield increase.