Multi-path Convolutional Neural Network Research Articles

Enhancement of brain tumor classification from MRI images using multi-path convolutional neural network with SVM classifier

The use of Convolutional Neural Networks (CNNs) has brought significant progress to the field of medical image segmentation and classification. This study aims to explore the potential of CNNs in classifying brain MRI images, which is crucial for faster identification of brain tumor types to speed up the treatment process. Many researchers are working hard to synthesize neural networks with less complexity and higher accuracy. In this regard, we start with a single-path CNN structure from the literature as a basis and halve the number of filters of three convolutional layers down to 4, 8 and 16. Then, we improve the accuracy of brain tumor classification by modifying a multi-path CNN structure constructed from our simplified single-path CNNs. The greatest improvement in the network's accuracy and sensitivity occurred when we replaced the conventional SoftMax classifier with a Support Vector Machine (SVM) classifier. Comparison of the proposed structure to the work of other researchers demonstrates a notable increment in accuracy of brain tumor classification by more than 10 % of the literature, meanwhile decreasing the complexity of the neural network structure. In this work, to verify the robustness and efficiency of our approach, the final proposed dual-path CNN with SVM classifier network is testified utilizing two MRI datasets. The proposed structure achieved an accuracy of 98.3 % on the First dataset, 98.2 % on the Second dataset, and 99.1 % on the combination of the First and the Second datasets. In addition, other assessment measures including Recall, Specificity, Precision, and F1 score are extracted to be 99.5 %, 98.9 %, 97.3 %, and 98.4 %, respectively.

PM-CNN: microbiome status recognition and disease detection model based on phylogeny and multi-path neural network.

The human microbiome, found throughout various body parts, plays a crucial role in health dynamics and disease development. Recent research has highlighted microbiome disparities between patients with different diseases and healthy individuals, suggesting the microbiome's potential in recognizing health states. Traditionally, microbiome-based status classification relies on pre-trained machine learning (ML) models. However, most ML methods overlook microbial relationships, limiting model performance. To address this gap, we propose PM-CNN (Phylogenetic Multi-path Convolutional Neural Network), a novel phylogeny-based neural network model for multi-status classification and disease detection using microbiome data. PM-CNN organizes microbes based on their phylogenetic relationships and extracts features using a multi-path convolutional neural network. An ensemble learning method then fuses these features to make accurate classification decisions. We applied PM-CNN to human microbiome data for status and disease detection, demonstrating its significant superiority over existing ML models. These results provide a robust foundation for microbiome-based state recognition and disease prediction in future research and applications. PM-CNN software is available at https://github.com/qdu-bioinfo/PM_CNN.

Novel MobileNet based Multipath Convolutional Neural Network for defect detection in fabrics

Automatic fabric defect detection and classification is the most important process in the textile industry to ensure the fabric quality. In the existing systems, a learning based method is used for detecting defects in plain weave fabrics. In this paper, a novel MobileNet based Multipath Convolutional Neural Network (MMPCNN) architecture is proposed for detection and classification of simple and complex patterned fabric defects. In the proposed MMPCNN architecture, MobileNet model is used in the first path. In this, Gabor filter bank is used instead of conventional filters in the first convolution layer. A simple convolutional neural network architecture with Gray Level Co-occurrence Matrix (GLCM) features as an input is used in the second path of the MMPCNN architecture. Gabor filters are more useful for analyzing the texture with different orientations and scales. Each Gabor filter parameter has its own impact on analyzing the texture and extracting the information from the texture. Therefore, in this paper, the use of Gabor filter parameters in MMPCNN architecture is analyzed. The proposed model is experimented on the TILDA textile image database and it is able to achieve 100% accuracy with reduced trainable parameters for fabric defect detection and classification.

A novel insect and pest identification model based on a weighted multipath convolutional neural network and generative adversarial network

Timely identification of insects and their management play a significant role in sustainable agriculture development. The proposed hybrid model integrates a weighted multipath convolutional neural network and generative adversarial network to identify insects efficiently. To address the shortcomings of single-path networks, this novel model takes input from numerous iterations of the same image to learn more specific features. To avoid redundancy produced due to multipath, weights have been assigned to each path. For Xie2 dataset, the model shows 3.75%, 2.74%, 1.54%, 1.76%, 1.76%, 2.74 %, and 2.14% performance improvement from AlexNet, ResNet50, ResNet101, GoogleNet, VGG-16, VGG-19, and simple CNN respectively. To the best of our knowledge, no researchers have used a multipath convolution neural network in insect identification.

A novel approach for insect-pest identification using multipath convolutional neural network

A novel approach for insect-pest identification using multipath convolutional neural network

Solid Waste Management Scenario in India and Illegal Dump Detection Using Deep Learning: An AI Approach towards the Sustainable Waste Management

The study is presented in four sections. The first section defines the municipal solid waste and solid waste management system. The second section illustrates the descriptive statistical analysis of waste generation patterns in India. The average waste generation in India was 160,038.9 tons per day in 2021; 95% of this total waste was collected and transported to the disposal sites. Based on scientific studies and observations, the per capita waste generation rate in 2018 was 0.490–0.626 g per day. In the last one and a half decades (1999–2000 to 2015–2016), Delhi and Bangalore have shown the highest percentage growth of 2075% and 1750%, respectively, in total waste generation among the highest population cities. The analysis of waste generation patterns concludes urbanization is a major factor that highly influences the waste generation rate. The third section describes the major issues in current solid waste management services. Some of these issues are the unavailability of web portals for citizens, no real-time monitoring of bins, collection vehicles and illegal dumping. These issues are identified based on the survey performed in a city and analysis of related research studies and scientific reports. We determined that illegal dumping is one of these major concerns and needs a technological solution. In the fourth section, we propose a multipath convolutional neural network (mp-CNN) to detect and localize the waste dumps on streets and roadsides. We constructed our dataset to train and test the proposed model, as no benchmark dataset is publicly available to obtain this objective. We applied the weakly supervised learning approach to training the model. In this approach, mp-CNN was trained according to the image class; in our case, it is two (waste and non-waste). In the testing phase, the model showed the performance evaluation matrices 97.82% of precision, 98.86% of recall, 98.34% of F1 score, 98.33% of accuracy, and 98.63% of AUROC for this binary classification. Due to the scarcity of benchmark datasets, waste localization results cannot be presented quantitatively. So, we performed a survey to compare the overlapping of the mask generated by the model with the region waste in the actual image. The average score for the generated mask obtained a score of 3.884 on a scale of 5. Based on the analysis of model performance evaluation parameters, precision-recall curve, receiver characteristic operator curve, and comparison of mask generated by the model over waste with corresponding actual images show that mp-CNN performs remarkably good in detection, classification, and localization of waste regions. Finally, two conceptual architectures in the context of developing countries are suggested to demonstrate the future practical applications of the mp-CNN model.

Multi-path Convolutional Neural Network to Identify Tumorous Sub-classes for Breast Tissue from Histopathological Images

Malignancy is one of the leading causes of death. It is on the rise in the developed and low-income countries with survival rates of less than 40%. However, early diagnosis may increase survival chances. Histopathology images acquired from the biopsy are a popular method for cancer diagnosis. In this work, we propose a deep convolutional neural network-based method that helps classify breast cancer tumor subtypes from histopathology images. The model is trained on the BreakHis dataset but is also tested on images from other datasets. The model is trained to recognized eight different tumor subtypes, and also to perform binary classification (malignant/non-malignant). The CNN model combines an encoder–decoder architecture and a parallel feed-forward network with attention mechanism. The proposed model provides state-of-the-art scores. Comparing with the other models, the accuracy of the proposed model is higher at different magnification and patient levels. The implementation is available at github.com/rangan2510/Residual\(\_\)Unet

A Graph and Attentive Multi-Path Convolutional Network for Traffic Prediction

Traffic prediction is an important and yet highly challenging problem due to the complexity and constantly changing nature of traffic systems. To address the challenges, we propose a graph and attentive multi-path convolutional network (GAMCN) model to predict traffic conditions such as traffic speed across a given road network into the future. Our model focuses on the spatial and temporal factors that impact traffic conditions. To model the spatial factors, we propose a variant of the graph convolutional network (GCN) named LPGCN to embed road network graph vertices into a latent space, where vertices with correlated traffic conditions are close to each other. To model the temporal factors, we use a multi-path convolutional neural network (CNN) to learn the joint impact of different combinations of past traffic conditions on the future traffic conditions. Such a joint impact is further modulated by an attention generated from an embedding of the prediction time, which encodes the periodic patterns of traffic conditions. We evaluate our model on real-world road networks and traffic data. The experimental results show that our model outperforms state-of-art traffic prediction models by up to 18.9% in terms of prediction errors and 23.4% in terms of prediction efficiency.

Learning Multiscale Temporal–Spatial–Spectral Features via a Multipath Convolutional LSTM Neural Network for Change Detection With Hyperspectral Images

Change detection (CD) with hyperspectral images (HSIs) can be effectively performed using deep learning networks (DLNs) by taking advantage of HSIs for their abundant spectral and spatial information and the excellent performance of DLN in machine learning. By modeling the temporal dependence of multiscale representative features, more discriminative information reflecting land use and land cover (LULC) changes can be obtained by suppressing less correlated information while improving the robustness of pseudo-changes caused by imaging noises. However, preserving time-dependent multiscale representative features while extracting spatial–spectral features based on conventional DLN is difficult, mainly due to the structural limitation of conventional DLN. A multipath convolutional long short-term memory (LSTM) multipath convolutional long short-term memory neural network (MP-ConvLSTM) taking advantage of LSTM and convolutional neural network (CNN) through the designed parallel architecture to learn multilevel temporal dependencies of bitemporal HSIs, therefore, was proposed for extracting multiscale temporal–spatial–spectral features by combining hidden states from different paths of ConvLSTM in the present study. In the proposed MP-ConvLSTM, the efficient channel attention (ECA) module was introduced to refine features of different paths, and Siamese CNN was adopted to reduce HSIs’ dimensionality and extract preliminary features to build up an end-to-end trainable model for CD with HSIs. The validity of the MP-ConvLSTM was evaluated using the binary and multiclass CD datasets. The CD accuracy of the proposed MP-ConvLSTM was visually and statistically evaluated by different criteria and compared with those derived from several state-of-the-art (SOTA) CD algorithms. The experiments demonstrated that our proposed model not only outperformed those SOTA CD models but also exhibited better tradeoff between complexity and accuracy in general.

Multitask Deep Learning for Segmentation and Lumbosacral Spine Inspection

Multi-task learning has achieved notable progress in many medical applications. In this paper, we propose a multi-task neural network, MRNet, for segmentation and spinal parameter inspection. It is developed based on the multi-path convolutional neural network for the robust detection of obscure regions on X-ray images. The proposed MRNet has two branches. One is for the segmentation of lumbar vertebrae, sacrum, and femoral heads. It shares the main features with the second branch for detection and classification by supervised learning. The output of the second branch is used to estimate the parameters for lumbosacral spine inspection. We conduct this research on our dataset collected and annotated by doctors for model training and performance evaluation. The datasets are used to train our MRNet as well as other networks for performance evaluation. Compared to the state-of-the-art techniques, the proposed MRNet is capable of X-ray image segmentation and parameter estimation with very limited training data. The results have demonstrated the feasibility of our MRNet for the segmentation of lumbar vertebrae, as well as the automated parameter prediction for lumbosacral spine inspection. Code is available at https://github.com/LuanTran07/BiLUnet-Lumbar-Spine.

A novel multiscale and multipath convolutional neural network based age-related macular degeneration detection using OCT images

Background and Objective: One of the significant retinal diseases that affected older people is called Age-related Macular Degeneration (AMD). The first stage creates a blur effect on vision and later leads to central vision loss. Most people overlooked the primary stage blurring and converted it into an advanced stage. There is no proper treatment to cure the disease. So the early detection of AMD is essential to prevent its extension into the advanced stage. This paper proposes a novel deep Convolutional Neural Network (CNN) architecture to automate AMD diagnosis early from Optical Coherence Tomographic (OCT) images.Methods: The proposed architecture is a multiscale and multipath CNN with six convolutional layers. The multiscale convolution layer permits the network to produce many local structures with various filter dimensions. The multipath feature extraction permits CNN to merge more features regarding the sparse local and fine global structures. The performance of the proposed architecture is evaluated through ten-fold cross-validation methods using different classifiers like support vector machine, multi-layer perceptron, and random forest.Results: The proposed CNN with the random forest classifier gives the best classification accuracy results. The proposed method is tested on data set 1, data set 2, data set 3, data set 4, and achieved an accuracy of 0.9666, 0.9897, 0.9974, and 0.9978 respectively, with random forest classifier. Also, we tested the combination of first three data sets and achieved an accuracy of 0.9902.Conclusions: An efficient algorithm for detecting AMD from OCT images is proposed based on a multiscale and multipath CNN architecture. Comparison with other approaches produced results that exhibit the efficiency of the proposed algorithm in the detection of AMD. The proposed architecture can be applied in rapid screening of the eye for the early detection of AMD. Due to less complexity and fewer learnable parameters.

Diabetic retinopathy classification based on multipath CNN and machine learning classifiers.

Eye care professionals generally use fundoscopy to confirm the occurrence of Diabetic Retinopathy (DR) in patients. Early DR detection and accurate DR grading are critical for the care and management of this disease. This work proposes an automated DR grading method in which features can be extracted from the fundus images and categorized based on severity using deep learning and Machine Learning (ML) algorithms. A Multipath Convolutional Neural Network (M-CNN) is used for global and local feature extraction from images. Then, a machine learning classifier is used to categorize the input according to the severity. The proposed model is evaluated across different publicly available databases (IDRiD, Kaggle (for DR detection), and MESSIDOR) and different ML classifiers (Support Vector Machine (SVM), Random Forest, and J48). The metrics selected for model evaluation are the False Positive Rate (FPR), Specificity, Precision, Recall, F1-score, K-score, and Accuracy. The experiments show that the best response is produced by the M-CNN network with the J48 classifier. The classifiers are evaluated across the pre-trained network features and existing DR grading methods. The average accuracy obtained for the proposed work is 99.62% for DR grading. The experiments and evaluation results show that the proposed method works well for accurate DR grading and early disease detection.

Polarimetric Multipath Convolutional Neural Network for PolSAR Image Classification

Scatter targets of complex land covers in polarimetric synthetic aperture radar (PolSAR) images are often randomly oriented and cause randomly fluctuating echoes, which brings a challenge to PolSAR image classification. Therefore, many existing methods have alleviated this problem through orientation compensation. However, there are still two obstacles that limit the improvement of classification accuracy. On the one hand, generally, these methods process PolSAR images with fixed polarization rotation angles, which is experience-dependent and inflexible. On the other hand, for the different land covers of a PolSAR image, the existing methods do not consider these rotation angles separately. For the first obstacle, we design a group of convolution kernels called polarization rotation kernels (PRKs) and utilize them to build the polarimetric convolutional neural network (CNN) (PolCNN). The PolCNN is the base network of our final model, and it can learn polarization rotation angles adaptively. For the second obstacle, we extend the PolCNN into a multipath structure, the final model polarimetric multipath CNN (PolMPCNN). The polarization rotation angles of different land covers are directly related to the networks of different paths within the PolMPCNN. Furthermore, we also put forward the two-scale sampling and the stagewise training algorithm in order that our PolMPCNN can fit different scales of PolSAR targets and pays more attention to difficult training samples. Experiments on real PolSAR images show that the proposed model achieves the best classification results with an extremely low sampling rate of 0.1%.

AutoTICI: Automatic Brain Tissue Reperfusion Scoring on 2D DSA Images of Acute Ischemic Stroke Patients.

The Thrombolysis in Cerebral Infarction (TICI) score is an important metric for reperfusion therapy assessment in acute ischemic stroke. It is commonly used as a technical outcome measure after endovascular treatment (EVT). Existing TICI scores are defined in coarse ordinal grades based on visual inspection, leading to inter- and intra-observer variation. In this work, we present autoTICI, an automatic and quantitative TICI scoring method. First, each digital subtraction angiography (DSA) acquisition is separated into four phases (non-contrast, arterial, parenchymal and venous phase) using a multi-path convolutional neural network (CNN), which exploits spatio-temporal features. The network also incorporates sequence level label dependencies in the form of a state-transition matrix. Next, a minimum intensity map (MINIP) is computed using the motion corrected arterial and parenchymal frames. On the MINIP image, vessel, perfusion and background pixels are segmented. Finally, we quantify the autoTICI score as the ratio of reperfused pixels after EVT. On a routinely acquired multi-center dataset, the proposed autoTICI shows good correlation with the extended TICI (eTICI) reference with an average area under the curve (AUC) score of 0.81. The AUC score is 0.90 with respect to the dichotomized eTICI. In terms of clinical outcome prediction, we demonstrate that autoTICI is overall comparable to eTICI.

Automated Detection of Age-Related Macular Degeneration from OCT Images Using Multipath CNN

Age-related macular degeneration (AMD) is an eye disorder that can have harmful effects on older people. AMD affects the macula, which is the core portion of the retina. Hence, early diagnosis is necessary to prevent vision loss in the elderly. To this end, this paper proposes a novel multipath convolutional neural network (CNN) architecture for the accurate diagnosis of AMD. The architecture proposed is a multipath CNN with five convolutional layers used to classify AMD or normal images. The multipath convolution layer enables many global structures to be generated with a large filter kernel. In this proposed network, the sigmoid function is used as the classifier. The proposed CNN network is trained on the Mendeley dataset and evaluated on four datasets-the Mendeley, OCTID, Duke, and SD-OCT Noor datasets- and it achieved accuracies of 99.60%, 99.61%, 96.67%, and 93.87%, respectively. Although the proposed model is only trained on the Mendeley dataset, it achieves good detection accuracy when evaluated with other datasets. This indicates that the proposed model has the capacity to detect AMD. These results demonstrate the efficiency of the proposed algorithm in detecting AMD compared to other approaches. The proposed CNN can be applied in real-time due to its reduced complexity and learnable parameters.

BiLuNetICP: A Deep Neural Network for Object Semantic Segmentation and 6D Pose Recognition

The ability of understanding a scene and predicting the pose of objects has attracted significant interests in recent years. Specifically, it is used with visual sensors to provide the information for a robotic manipulator to interact with the target. Thus, 6D pose estimation and object recognition from point clouds or RGB-D images are important tasks for visual servoing. In this article, we propose a learning based approach to perform 6D pose estimation for robotic manipulation using a BiLuNetICP pipeline. It consists of a multi-path convolutional neural network (CNN) for semantic segmentation on RGB images. The network extracts the object mask and uses it to merge with the depth information to perform 6D pose estimation by the Iterative Closest Point (ICP) algorithm. We collected our own dataset for training and evaluate with Intersection over Union (IoU). The proposed method is able to provide better results compared with Unet++ using a small amount of training data. For the robotic grasping application, we test and evaluate our approach using a HIWIN 6-axis robot with Asus Xtion Live 3D camera and our structured-light depth camera. The experimental results demonstrate its efficiency in computation and the high success rate in grasping.

Multi-path convolutional neural network in fundus segmentation of blood vessels

There is a close correlation between retinal vascular status and physical diseases such as eye lesions. Retinal fundus images are an important basis for diagnosing diseases such as diabetes, glaucoma, hypertension, coronary heart disease, etc. Because the thickness of the retinal blood vessels is different, the minimum diameter is only one or two pixels wide, so obtaining accurate measurement results becomes critical and challenging. In this paper, we propose a new method of retinal blood vessel segmentation that is based on a multi-path convolutional neural network, which can be used for computer-based clinical medical image analysis. First, a low-frequency image characterizing the overall characteristics of the retinal blood vessel image and a high-frequency image characterizing the local detailed features are respectively obtained by using a Gaussian low-pass filter and a Gaussian high-pass filter. Then a feature extraction path is constructed for the characteristics of the low- and high-frequency images, respectively. Finally, according to the response results of the low-frequency feature extraction path and the high-frequency feature extraction path, the whole blood vessel perception and local feature information fusion coding are realized, and the final blood vessel segmentation map is obtained. The performance of this method is evaluated and tested by DRIVE and CHASE_DB1. In the experimental results of the DRIVE database, the evaluation indexes accuracy (Acc), sensitivity (SE), and specificity (SP) are 0.9580, 0.8639, and 0.9665, respectively, and the evaluation indexes Acc, SE, and SP of the CHASE_DB1 database are 0.9601, 0.8778, and 0.9680, respectively. In addition, the method proposed in this paper could effectively suppress noise, ensure continuity after blood vessel segmentation, and provide a feasible new idea for intelligent visual perception of medical images.

Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion

Mechanical harvesting requires agricultural robot to detect fruits automatically. However, effective and accurate detection of cucumber by computer vision system is still a challenge due to similarity between cucumber color and that of branches and leaves, shape irregularity and complex growing environment. To improve the practicability and accuracy of the automatic recognition models, this paper proposed a novel cucumber region detection method using multi-path convolutional neural network (MPCNN), combined with color component selection and support vector machine (SVM). In this method, the cucumber image was transformed into color space to obtain 15 color components and the weight information of relevant features was analyzed by I-RELIEF. In parallel, to remove part of the background area, the OSTU algorithm was applied to segment the G component and Maximally Stable Extrernal Regions (MSER) was used to obtain the mask image. In order to maximize the differences between cucumber and leaf, promoting the classification accuracy of SVM, the top three components of the weight were input into the deep learning module to extract and fuse features. In final, cucumber recognition was realized by combining SVM classification with mask image. The recognition results show that more than 90% pixels of cucumber images are correctly classified, and the misidentified pixels are less than 22%. The ratio between the two indicators is over 4, demonstrating the satisfactory performance of the proposed method and highlighting its promising applications in mechanical cucumber harvesting.

CNN-based Broad Learning with Efficient Incremental Reconstruction Model for Facial Emotion Recognition

Abstract Convolutional neural network-based broad learning with efficient incremental reconstruction model (CNNBL) is proposed to recognize emotions in human-robot interaction. It aims to extract deep and abstract features from facial emotional images, and reduce the influence of the complex structure and slow network updates on facial emotion recognition in deep learning. Feature extraction is carried out by convolution and maximum pooling, and then the ridge regression algorithm is used for emotion recognition. When the network needs to expand, the network is dynamically updated by incremental learning algorithm. We verified the experimental performance through k-fold cross validation. According to the recognition results, the accuracy on JAFFE database of our proposal is greater than that of the state of the art, such as the Local Binary Patterns with Softmax and Deep Attentive Multi-path convolutional neural network.

A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images.

Automatic identification of brain lesions from magnetic resonance imaging (MRI) scans of stroke survivors would be a useful aid in patient diagnosis and treatment planning. It would also greatly facilitate the study of brain-behavior relationships by eliminating the laborious step of having a human expert manually segment the lesion on each brain scan. We propose a multi-modal multi-path convolutional neural network system for automating stroke lesion segmentation. Our system has nine end-to-end UNets that take as input 2-dimensional (2D) slices and examines all three planes with three different normalizations. Outputs from these nine total paths are concatenated into a 3D volume that is then passed to a 3D convolutional neural network to output a final lesion mask. We trained and tested our method on datasets from three sources: Medical College of Wisconsin (MCW), Kessler Foundation (KF), and the publicly available Anatomical Tracings of Lesions After Stroke (ATLAS) dataset. To promote wide applicability, lesions were included from both subacute (1 to 5 weeks) and chronic (> 3 months) phases post stroke, and were of both hemorrhagic and ischemic etiology. Cross-study validation results (with independent training and validation datasets) were obtained to compare with previous methods based on naive Bayes, random forests, and three recently published convolutional neural networks. Model performance was quantified in terms of the Dice coefficient, a measure of spatial overlap between the model-identified lesion and the human expert-identified lesion, where 0 is no overlap and 1 is complete overlap. Training on the KF and MCW images and testing on the ATLAS images yielded a mean Dice coefficient of 0.54. This was reliably better than the next best previous model, UNet, at 0.47. Reversing the train and test datasets yields a mean Dice of 0.47 on KF and MCW images, whereas the next best UNet reaches 0.45. With all three datasets combined, the current system compared to previous methods also attained a reliably higher cross-validation accuracy. It also achieved high Dice values for many smaller lesions that existing methods have difficulty identifying. Overall, our system is a clear improvement over previous methods for automating stroke lesion segmentation, bringing us an important step closer to the inter-rater accuracy level of human experts.