Max Pooling Research Articles

Plant Disease Detection Using Machine Learning

In this project, we have developed a plant disease detection model using a machine learning approach on a dataset named “Plant Disease Expert” which is available on Kaggle. The dataset contains 199,611 images across 58 subdirectories. The splitting of the dataset in training, validation and testing is done in ratio 90:5:5. The data preprocessing included resizing the images, normalizing pixel values and applying augmentation techniques. We have used different libraries throughout the project such as NumPy, Pandas, Operating system, Time, Matplotlib, OpenCV, Shutil, TensorFlow, Keras, Seaborn, etc. EfficientNetB3 is used as the base model. EfficientNetB3 is a convolutional neural network architecture developed by Google Brain researchers [1]. It is known for its efficiency and effectiveness in image classification tasks. Additional custom layers such as max pooling, batch normalization, dense layers, and activation functions are used to optimize the performance. The adamax optimizer and categorical cross-entropy were used during training. The model was trained for 10 epochs with a batch size of 20 and a learning rate of 0.01. For evaluation, classification report, accuracy, precision, recall, F1-score and support were used. The model achieved an accuracy of 98.76% [1].

Prediction of cancer therapy related cardiac dysfunction by using a machine learning approach with cardiac magnetic resonance images

Abstract Background There is inconsistent data on the value of baseline and early cardiac function assessment on identifying patients at risk of cancer therapy related cardiac dysfunction (CTRCD). Whether machine learning (ML) approaches applied to cardiac MRI (CMR) images may help risk stratify patients is unclear. Purpose In women with HER2+ breast cancer receiving sequential anthracyclines and trastuzumab we sought to (i) assess if a ML approach applied to pre-cancer therapy and early post anthracycline CMR short axis (SAX) cines can help identify patients at risk for future CTRCD and (ii) whether this approach is superior to using clinical and manual quantitative CMR data. Methods Women with HER2+ breast cancer from the EMBRACE-MRI study underwent 5 CMR studies (pre-cancer therapy, post anthracyclines, 3, 6 and 12 months during trastuzumab). CTRCD was defined using the CREC criteria by end of treatment. Three models were created: 1) clinical data (demographics, cardiovascular risk factors and medications); 2) clinical and manual quantitative CMR data (right and left ventricular volumes and LVEF); and 3) ML model based on SAX cine CMR images only. We focused on the pre-cancer therapy CMR and 1st visit post-anthracycline CMR. All images underwent standardised formatting. A 3D convolutional neural networks model architecture was used. The model consisted of multiple convolutional layers followed by max pooling and global average pooling layers. The output of the convolutional layers were then fed into a dense layer of 128 nodes incorporating an L2 activity regularizer and a subsequent dropout layer. To train and tune the model, the process was iterated 50 times. In each iteration, the patient participant dataset was shuffled, and a random 80% of patients were selected for training and tuning. The tuned model was then applied to the remaining 20% of patients in each iteration, and this entire cycle of shuffling, training, and testing was repeated 50 times with replacement. Area Under the Curve (AUC), accuracy, sensitivity, and specificity were calculated on the test set and averaged across folds to determine each model’s performance. Results 135 women were included (mean age 51.1±9.2 years) mean doxorubicin equivalent was dose 204.9±12.5mg/m2. Cardiovascular risk factors were prevalent: 33 (24%) had a smoking history; 21 (15%) had hypertension. 37 (27%) developed CTRCD. The ML model using baseline CMR cine data only had a higher AUC (0.84 95% CI [0.83-0.86]) for the detection of CTRCD compared to clinical data (0.62 95% CI [0.59-0.65]) and clinical and CMR data (0.62 95% CI [0.60-0.65]). This further improved with the addition of post-anthracycline CMR data (Table 1). Conclusion A ML model examining early cardiac MRI data with short axis cine CMR images performed better than those using clinical or manual quantitative CMR data for the prediction of CTRCD. Further studies including external validation are needed.Table 1 Mean and 95% CIs

Towards the Development of the Clinical Decision Support System for the Identification of Respiration Diseases via Lung Sound Classification Using 1D-CNN

Respiratory disorders are commonly regarded as complex disorders to diagnose due to their multi-factorial nature, encompassing the interplay between hereditary variables, comorbidities, environmental exposures, and therapies, among other contributing factors. This study presents a Clinical Decision Support System (CDSS) for the early detection of respiratory disorders using a one-dimensional convolutional neural network (1D-CNN) model. The ICBHI 2017 Breathing Sound Database, which contains samples of different breathing sounds, was used in this research. During pre-processing, audio clips were resampled to a uniform rate, and breathing cycles were segmented into individual instances of the lung sound. A One-Dimensional Convolutional Neural Network (1D-CNN) consisting of convolutional layers, max pooling layers, dropout layers, and fully connected layers, was designed to classify the processed clips into four categories: normal, crackles, wheezes, and combined crackles and wheezes. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) was applied to the training data. Hyperparameters were optimized using grid search with k−fold cross-validation. The model achieved an overall accuracy of 0.95, outperforming state-of-the-art methods. Particularly, the normal and crackles categories attained the highest F1-scores of 0.97 and 0.95, respectively. The model’s robustness was further validated through 5−fold and 10−fold cross-validation experiments. This research highlighted an essential aspect of diagnosing lung sounds through artificial intelligence and utilized the 1D-CNN to classify lung sounds accurately. The proposed advancement of technology shall enable medical care practitioners to diagnose lung disorders in an improved manner, leading to better patient care.

Development of fig fruit ripeness classification using convolutional neural network

This study presents the design and evaluation of a deep convolutional neural network (CNN) model for accurately classifying fig ripeness stages. Traditionally, fruit ripeness classification has been conducted manually, which presents several drawbacks, including heavy reliance on human labor and inconsistencies in determining fruit ripeness. By leveraging advanced deep learning techniques, specifically CNNs, this research aims to automate the fig ripeness classification process. The CNN architecture was developed and trained using MATLAB software, targeting three ripeness categories: ripe, half-ripe, and unripe. The methodology involved pre-processing the fig images and configuring the CNN model with multiple convolutional, batch normalization, and max pooling layers specifically for fig classification tasks. The final CNN model achieved an impressive accuracy rate of 94.44%, significantly surpassing results from previously reported studies. The developed model is a promising tool for automating fig ripeness classification, contributing to advancements in precision agriculture and smart farming technologies.

Cross-attention guided loss-based deep dual-branch fusion network for liver tumor classification

Recently, convolutional neural networks (CNNs) and multiple instance learning (MIL) methods have been successfully applied to MRI images. However, CNNs directly utilize the whole image as the model input and the downsampling strategy (like max or mean pooling) to reduce the size of the feature map, thereby possibly neglecting some local details. And MIL methods learn instance-level or local features without considering spatial information. To overcome these issues, in this paper, we propose a novel cross-attention guided loss-based dual-branch framework (LCA-DB) to leverage spatial and local image information simultaneously, which is composed of an image-based attention network (IA-Net), a patch-based attention network (PA-Net) and a cross-attention module (CA). Specifically, IA-Net directly learns image features with loss-based attention to mine significant regions, meanwhile, PA-Net captures patch-specific representations to extract crucial patches related to the tumor. Additionally, the cross-attention module is designed to integrate patch-level features by using attention weights generated from each other, thereby assisting them in mining supplement region information and enhancing the interactive collaboration of the two branches. Moreover, we employ an attention similarity loss to further reduce the semantic inconsistency of attention weights obtained from the two branches. Finally, extensive experiments on three liver tumor classification tasks demonstrate the effectiveness of the proposed framework, e.g., on the LLD-MMRI–7, our method achieves 69.2%, 65.9% and 88.5% on the seven-class liver tumor classification tasks in terms of accuracy, F1 score and AUC, with the superior classification and interpretation performance over recent state-of-the-art methods. The source code of LCA-DB is available at https://github.com/Wangrui-berry/Cross-attention.

HỌC BIỂU DIỄN CÂU SỬ DỤNG MÔ HÌNH LSTM TRONG BÀI TOÁN TÌM KIẾM CÂU HỎI

Learning sentence representation with the full semantics of a document is a challenge in natural language processing problems because if the semantic representation vector of the sentence is suitable, it will increase the performance of finding similar question problems. In this paper, we propose implementing a series of LSTM models with different ways of extracting sentence representations and applying them to question retrieval to exploit the hidden semantics of sentences. These methods give sentence representation from hidden layers of the LSTM model. The results show that the technique using a combination of both Max Pooling and Mean Pooling gives the highest results on the 2017 SemEval dataset for the problem of finding similarity questions. Keywords: LSTM; Deep Learning; NLP; QA; learning sentence representation; CQA.

A Convolutional Neural Network Approach for Precision Fish Disease Detection

Background: Detecting and classifying fish diseases is crucial for maintaining the health and sustainability of aquaculture systems. This study employs deep learning techniques, particularly Convolutional Neural Networks (CNNs), to automate the detection of various fish diseases using image data. Methods: The study utilizes a carefully curated dataset sourced from the Kaggle database, comprising images representing seven distinct types of fish diseases, along with images of healthy fish. Data preprocessing techniques, including resizing, rescaling, denoising, sharpening, and smoothing, are applied to enhance image quality and facilitate accurate disease detection. Data augmentation is employed to increase the model's ability to generalize to unseen data. The CNN architecture is designed with cascading convolutional layers, ReLU activation functions, and pooling operations to extract high-level features associated with fish infections. The model architecture, implemented using the Keras Sequential API, includes convolutional layers, max pooling layers, and densely connected layers for classification. Results: Experimental results demonstrated promising performance across various disease categories, with high accuracy and balanced precision and recall values for most classes. The study also discussed the impact of climate change on fish disease incidence and underscores the importance of effective monitoring and management practices facilitated by technological innovations such as Big Data, IoT, sensors, and robotics in ensuring sustainable fisheries management.

Advanced deepfake detection with enhanced Resnet-18 and multilayer CNN max pooling

Advanced deepfake detection with enhanced Resnet-18 and multilayer CNN max pooling

PelviNet: A Collaborative Multi-agent Convolutional Network for Enhanced Pelvic Image Registration.

PelviNet introduces a groundbreaking multi-agent convolutional network architecture tailored for enhancing pelvic image registration. This innovative framework leverages shared convolutional layers, enabling synchronized learning among agents and ensuring an exhaustive analysis of intricate 3D pelvic structures. The architecture combines max pooling, parametric ReLU activations, and agent-specific layers to optimize both individual and collective decision-making processes. A communication mechanism efficiently aggregates outputs from these shared layers, enabling agents to make well-informed decisions by harnessing combined intelligence. PelviNet's evaluation centers on both quantitative accuracy metrics and visual representations to elucidate agents' performance in pinpointing optimal landmarks. Empirical results demonstrate PelviNet's superiority over traditional methods, achieving an average image-wise error of 2.8 mm, a subject-wise error of 3.2 mm, and a mean Euclidean distance error of 3.0 mm. These quantitative results highlight the model's efficiency and precision in landmark identification, crucial for medical contexts such as radiation therapy, where exact landmark identification significantly influences treatment outcomes. By reliably identifying critical structures, PelviNet advances pelvic image analysis and offers potential enhancements for broader medical imaging applications, marking a significant step forward in computational healthcare.

Hybrid CNN-LSTM for Predicting Diabetes: A Review.

Diabetes is a common and deadly chronic disease caused by high blood glucose levels that can cause heart problems, neurological damage, and other illnesses. Through the early detection of diabetes, patients can live healthier lives. Many machine learning and deep learning techniques have been applied for noninvasive diabetes prediction. The results of some studies have shown that the CNN-LSTM method, a combination of CNN and LSTM, has good performance for predicting diabetes compared to other deep learning methods. This paper reviews CNN-LSTM-based studies for diabetes prediction. In the CNNLSTM model, the CNN includes convolution and max pooling layers and is applied for feature extraction. The output of the max-pooling layer was fed into the LSTM layer for classification. The CNN-LSTM model performed well in extracting hidden features and correlations between physiological variables. Thus, it can be used to predict diabetes. The CNNLSTM model, like other deep neural network architectures, faces challenges such as training on large datasets and biological factors. Using large datasets can further improve the accuracy of detection. The CNN-LSTM model is a promising method for diabetes prediction, and compared with other deep-learning models, it is a reliable method.

Research on Person Re-Identification through Local and Global Attention Mechanisms and Combination Poolings.

This research proposes constructing a network used for person re-identification called MGNACP (Multiple Granularity Network with Attention Mechanisms and Combination Poolings). Based on the MGN (Multiple Granularity Network) that combines global and local features and the characteristics of the MGN branch, the MGNA (Multiple Granularity Network with Attentions) is designed by adding a channel attention mechanism to each global and local branch of the MGN. The MGNA, with attention mechanisms, learns the most identifiable information about global and local features to improve the person re-identification accuracy. Based on the constructed MGNA, a single pooling used in each branch is replaced by combination pooling to form MGNACP. The combination pooling parameters are the proportions of max pooling and average pooling in combination pooling. Through experiments, suitable combination pooling parameters are found, the advantages of max pooling and average pooling are preserved and enhanced, and the disadvantages of both types of pooling are overcome, so that poolings can achieve optimal results in MGNACP and improve the person re-identification accuracy. In experiments on the Market-1501 dataset, MGNACP achieved competitive experimental results; the values of mAP and top-1 are 88.82% and 95.46%. The experimental results demonstrate that MGNACP is a competitive person re-identification network, and that the attention mechanisms and combination poolings can significantly improve the person re-identification accuracy.

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification.

Addressing the challenges in effectively extracting multi-scale features and preserving pose information during hyperspectral image (HSI) classification, a Multi-Scale Depthwise Separable Capsule Network (MDSC-Net) is proposed in this article for HSI classification. Initially, hierarchical features are extracted by MDSC-Net through the employment of parallel multi-scale convolutional kernels, while computational complexity is reduced via depthwise separable convolutions, thus reducing the overall computational load and achieving efficient feature extraction. Subsequently, to enhance the translational invariance of features and reduce the loss of pose information, features of various scales are processed in parallel by independent capsule networks, with improvements in max pooling achieved through dynamic routing. Lastly, features of different scales are concatenated and integrated through the concatenate operation, thereby facilitating precise analysis of multi-level information in the hyperspectral image classification process. Experimental comparisons demonstrate that MDSC-Net achieves average accuracies of 94%, 98%, and 99% on the Kennedy Space Center, University of Pavia, and Salinas datasets, respectively, indicating a significant performance advantage over recent HSI classification models and validating the effectiveness of the proposed model.

Corrosion image classification method based on EfficientNetV2

Corrosion is one of the key factors leading to material failure, which can occur in facilities and equipment closely related to people's lives, causing structural damage and thus affecting the safety of people's lives and property. To identify corrosion more effectively across multiple facilities and equipment, this paper utilizes a corrosion binary classification dataset containing various materials to develop a CNN classification model for better detection and distinction of material corrosion, using a methodological paradigm of transfer learning and fine-tuning. The proposed model implementation initially uses data augmentation to enhance the dataset and employs different sizes of EfficientNetV2 for training, evaluated using Confusion Matrix, ROC curve, and the values of Precision, Recall, and F1-score. To further enhance the testing results, this paper focuses on the impact of using the Global Average Pooling layer versus the Global Max Pooling layer, as well as the number of fine-tuning layers. The results show that the Global Average Pooling layer performs better, and EfficientNetV2B0 with a fine-tuning rate of 20 %, and EfficientNetV2S with a fine-tuning rate of 15 %, achieve the highest testing accuracy of 0.9176, an ROC-AUC value of 0.97, and Precision, Recall, and F1-Score values exceeding 0.9. These findings can be served as a reference for other corrosion classification models which uses EfficientNetV2.

Interlayer information fusion-based and dual-attention improved U-Net for ABVS image sequence intelligent tumor segmentation

Breast cancer is the most common cancer among women. Accurate and intelligent ultrasound image tumor segmentation can help physicians efficiently determine the location, shape and features of tumors and then develop appropriate treatment plans. To this end, we propose a deep learning method combining interlayer information and dual attention improvement for accurate and intelligent segmentation of breast tumors in ABVS image sequences. We extracted the slices containing tumors in ABVS images and formed a three-channel image input network with the current frame and the corresponding before and after frames. The network combines interlayer information during computation, and the before and after frames can guide the current frame image for more accurate tumor segmentation. In addition, we improved the channel attention module by replacing the full connection with a one-dimensional convolution and applying global max pooling on the basis of the original channel attention module, which improved the segmentation performance of the module while reducing the number of parameters. The improved channel attention module was added to the U-Net skip-connection section to assign different weight coefficients to each feature channel according to the characteristics of the input three-channel image. Our method outperformed other mainstream tumor segmentation methods on the dataset in both centers, achieving a Dice similarity coefficient (DSC) of 84.97 %, a Jaccard coefficient (JC) of 74.63 %, a 95 % Hausdorff distance (95HD) of 6.22 mm on the inner center, and DSC of 80.79 %, JC of 69.74 %, 95HD of 9.82 mm on the outer dataset.

Low-light image enhancement using generative adversarial networks

In low-light environments, the amount of light captured by the camera sensor is reduced, resulting in lower image brightness. This makes it difficult to recognize or completely lose details in the image, which affects subsequent processing of low-light images. Low-light image enhancement methods can increase image brightness while better-restoring color and detail information. A generative adversarial network is proposed for low-quality image enhancement to improve the quality of low-light images. This network consists of a generative network and an adversarial network. In the generative network, a multi-scale feature extraction module, which consists of dilated convolutions, regular convolutions, max pooling, and average pooling, is designed. This module can extract low-light image features from multiple scales, thereby obtaining richer feature information. Secondly, an illumination attention module is designed to reduce the interference of redundant features. This module assigns greater weight to important illumination features, enabling the network to extract illumination features more effectively. Finally, an encoder-decoder generative network is designed. It uses the multi-scale feature extraction module, illumination attention module, and other conventional modules to enhance low-light images and improve quality. Regarding the adversarial network, a dual-discriminator structure is designed. This network has a global adversarial network and a local adversarial network. They determine if the input image is actual or generated from global and local features, enhancing the performance of the generator network. Additionally, an improved loss function is proposed by introducing color loss and perceptual loss into the conventional loss function. It can better measure the color loss between the generated image and a normally illuminated image, thus reducing color distortion during the enhancement process. The proposed method, along with other methods, is tested using both synthesized and real low-light images. Experimental results show that, compared to other methods, the images enhanced by the proposed method are closer to normally illuminated images for synthetic low-light images. For real low-light images, the images enhanced by the proposed method retain more details, are more apparent, and exhibit higher performance metrics. Overall, compared to other methods, the proposed method demonstrates better image enhancement capabilities for both synthetic and real low-light images.

Plug-and-play learnable EMD module for time-domain speech separation

This paper presents a learnable empirical mode decomposition (EMD) module tailored for single channel speech separation. The proposed EMD module exhibits a flexible plug-and-play characteristic, enabling its integration into various speech separation networks. Conventionally, non-learnable EMD algorithms, as a nonlinear time-domain adaptive techniques, aimed at decomposing speech signals into a set of oscillatory intrinsic mode functions (IMFs) and a residual component. However, the inherent non-differentiability of the interpolation operation in these algorithms restricts their integration into neural networks, thus the EMD algorithm only exists in the preprocessing step and cannot be integrated into the neural network to form an end-to-end trainable speech separation network. In order to mitigate this issue, we use the max pooling layer and the deconvolution network to replace the envelope solving algorithm in the EMD method and build a trainable EMD neural network module. Multiple sets of experiments have shown that the integrating of EMD module brings the improvement to different separation systems.

A Hybrid Deep BiLSTM-CNN for Hate Speech Detection in Multi-social media

Nowadays, means of communication among people have changed due to advancements in information technology and the rise of online multi-social media. Many people express their feelings, ideas, and emotions on social media sites such as Instagram, Twitter, Gab, Reddit, Facebook, and YouTube. However, people have misused social media to send hateful messages to specific individuals or groups to create chaos. For various governance authorities, manually identifying hate speech on various social media platforms is a difficult task to avoid such chaos. In this study, a hybrid deep-learning model, where bidirectional long short-term memory (BiLSTM) and convolutional neural network (CNN) are used to classify hate speech in textual data, is proposed. This model incorporates a GLOVE-based word embedding approach, dropout, L2 regularization, and global max pooling to get impressive results. Further, the proposed BiLSTM-CNN model has been evaluated on various datasets to achieve state-of-the-art performance that is superior to the traditional and existing machine learning methods in terms of accuracy, precision, recall, and F1-score.

Detecting image manipulation with ELA-CNN integration: a powerful framework for authenticity verification.

The exponential progress of image editing software has contributed to a rapid rise in the production of fake images. Consequently, various techniques and approaches have been developed to detect manipulated images. These methods aim to discern between genuine and altered images, effectively combating the proliferation of deceptive visual content. However, additional advancements are necessary to enhance their accuracy and precision. Therefore, this research proposes an image forgery algorithm that integrates error level analysis (ELA) and a convolutional neural network (CNN) to detect the manipulation. The system primarily focuses on detecting copy-move and splicing forgeries in images. The input image is fed to the ELA algorithm to identify regions within the image that have different compression levels. Afterward, the created ELA images are used as input to train the proposed CNN model. The CNN model is constructed from two consecutive convolution layers, followed by one max pooling layer and two dense layers. Two dropout layers are inserted between the layers to improve model generalization. The experiments are applied to the CASIA 2 dataset, and the simulation results show that the proposed algorithm demonstrates remarkable performance metrics, including a training accuracy of 99.05%, testing accuracy of 94.14%, precision of 94.1%, and recall of 94.07%. Notably, it outperforms state-of-the-art techniques in both accuracy and precision.

Self-adaptive selection graph pooling based fault diagnosis method under few samples and noisy environment

Neural network (NN)-based methods are extensively used for intelligent fault diagnosis in industrial systems. Nevertheless, due to the limited availability of faulty samples and the presence of noise interference, most existing NN-based methods perform limited diagnosis performance. In response to these challenges, a self-adaptive selection graph pooling method is proposed. Firstly, graph encoders with sharing parameters are designed to extract local structure-feature information (SFI) of multiple sensor-wise sub-graphs. Then, the temporal continuity of the SFI is maintained through time-by-time concatenation, resulting in a global sensor graph and reducing the dependency on data volume from the perspective of adding prior knowledge. Subsequently, leveraging a self-adaptive node selection mechanism, the noise interference of redundant and noisy sensor-wise nodes in the graph is alleviated, allowing the networks to concentrate on the fault-attention nodes. Finally, the local max pooling and global mean pooling of the node-selection graph are incorporated in the readout module to get the multi-scale graph features, which serve as input to a multi-layer perceptron for fault diagnosis. Two experimental studies involving different mechanical and electrical systems demonstrate that the proposed method not only achieves superior diagnosis performance with limited data, but also maintains strong anti-interference ability in noisy environments. Additionally, it exhibits good interpretability through the proposed self-adaptive node selection mechanism and visualization methods.

MP-DGCNN for the semantic segmentation of Chinese ancient building point clouds

Point cloud semantic segmentation is a key step in the scan-to-HBIM process. In order to reduce the information in the process of DGCNN, this paper proposes a Mix Pooling Dynamic Graph Convolutional Neural Network (MP-DGCNN) for the segmentation of ancient architecture point clouds. The proposed MP-DGCNN differs from DGCNN mainly in two aspects: (1) to more comprehensively characterize the local topological structure of points, the edge features are redefined, and distance and neighboring points are added to the original edge features; (2) based on a Multilayer Perceptron (MLP), an internal feature adjustment mechanism is established, and a learnable mix pooling operator is designed by fusing adaptive pooling, max pooling, average pooling, and aggregation pooling, to learn local graph features from the point cloud topology. To verify the proposed algorithm, experiments are conducted on the Qutan Temple point cloud dataset, and the results show that compared with PointNet, PointNet++, DGCNN, GACNet and LDGCNN, the MP-DGCNN segmentation network achieves the highest OA, mIOU and mAcc, reaching 90.19%,65.34% and 79.41%, respectively.