Inception Module Research Articles

A Quantum Computing-Based Accelerated Model for Image Classification Using a Parallel Pipeline Encoded Inception Module

Image classification is typically a research area that trains an algorithm for accurately identifying subjects in images that have never been seen before. Training a model to recognize images within a dataset is significant as image classification generally has several applications in medicine, face detection, image reconstruction, etc. In spite of such applications, the main difficulty in this area involves the computation in the classification process, which is vast, leading to slow speed of classification. Moreover, as conventional image classification approaches have fallen short in terms of attaining high accuracy, an optimal model is needed. To resolve this, quantum computing has been developed. Due to their parallel computing ability, quantum-based algorithms could accomplish the classification of vast amounts of image data. This has theoretically confirmed the feasibility and advantages of incorporating a quantum computing-based system with traditional image classification methodologies. Considering this, the present study quantizes the layers of the proposed parallel encoded Inception module to improvise the network performance. This study exposes the flexibility of DL (deep learning)-based quantum state computational methodologies for missing computations by creating a pipeline for denoising, state estimation, and imputation. Furthermore, controlled parameterized rotations are regarded for entanglement, a vital component in quantum perceptron structure. The proposed approach not only possesses the unique features of quantum mechanics, but it also maintains the weight sharing of the kernel. Finally, the MNIST (Modified National Institute of Standards and Technology) and Fashion MNIST image classification outcomes are attained by measuring the quantum state. Overall performance is assessed to prove its effectiveness in image classification.

DCDA-Net: Dual-convolutional dual-attention network for obstructive sleep apnea diagnosis from single-lead electrocardiograms

Obstructive sleep apnea (OSA) is a breathing-related chronic disease in which the soft palate and tongue collapse and block the upper airway for at least 10 s during sleep. It can lead to many heart diseases such as hypertension, myocardial infarction, and coronary heart syndrome if not detected early. Artificial intelligence has facilitated the diagnosis of many diseases in healthcare. Polysomnography is a widely used but unpleasant, time-consuming, technically demanding, and financially expensive procedure to detect OSA. Some previous methods have detected OSA using time-domain information from an electrocardiogram (ECG), whereas others have used frequency-domain information. The limitations of these two approaches can be handled using the data’s time–frequency representation. Nevertheless, there is room for enhancing the detection accuracy of OSA using the time–frequency representation approach. Therefore, we propose a novel technique that takes the ECG signal and detects R-peaks from the QRS complexes. Afterward, we interpolate those R-peaks by linear interpolation and get an interpolated-R signal. Then we magnify the interpolated-R signal corresponding to the apnea and normal frequency ranges. After magnification in the time domain, we transformed the magnified version into a scalogram. We also transformed the original one-minute ECG signal into a spectrogram after denoising. Overall, we used ECG signals to generate scalograms and spectrograms for 2 dimensional convolutional neural network (2D CNN) to classify obstructive sleep apnea. For apnea classification, we proposed a dual convolutional dual attention network (DCDA-Net) that includes a dual convolutionally modified inception module, a spatial attention module, and a channel attention module. Finally, we apply a support vector machine to the probability scores obtained from DCDA-Net based on the scalogram and spectrogram. Extensive experimental results using the open PhysioNet apnea ECG dataset confirm the effectiveness of our method in terms of accuracy and F1 score of 98% and 97.5%, respectively, which outperforms state-of-the-art methods.

Deep convolutional architecture-based hybrid learning for sleep arousal events detection through single-lead EEG signals.

Detecting arousal events during sleep is a challenging, time-consuming, and costly process that requires neurology knowledge. Even though similar automated systems detect sleep stages exclusively, early detection of sleep events can assist in identifying neuropathology progression. An efficient hybrid deep learning method to identify and evaluate arousal events is presented in this paper using only single-lead electroencephalography (EEG) signals for the first time. Using the proposed architecture, which incorporates Inception-ResNet-v2 learning transfer models and optimized support vector machine (SVM) with the radial basis function (RBF) kernel, it is possible to classify with a minimum error level of less than 8%. In addition to maintaining accuracy, the Inception module and ResNet have led to significant reductions in computational complexity for the detection of arousal events in EEG signals. Moreover, in order to improve the classification performance of the SVM, the grey wolf algorithm (GWO) has optimized its kernel parameters. This method has been validated using pre-processed samples from the 2018 Challenge Physiobank sleep dataset. In addition to reducing computational complexity, the results of this method show that different parts of feature extraction and classification are effective at identifying sleep disorders. The proposed model detects sleep arousal events with an average accuracy of 93.82%. With the lead present in the identification, the method becomes less aggressive in recording people's EEG signals. According to this study, the suggested strategy is effective in detecting arousals in sleep disorder clinical trials and may be used in sleep disorder detection clinics.

Lightweight Multiscale CNN Model for Wheat Disease Detection

Wheat disease detection is crucial for disease diagnosis, pesticide application optimization, disease control, and wheat yield and quality improvement. However, the detection of wheat diseases is difficult due to their various types. Detecting wheat diseases in complex fields is also challenging. Traditional models are difficult to apply to mobile devices because they have large parameters, and high computation and resource requirements. To address these issues, this paper combines the residual module and the inception module to construct a lightweight multiscale CNN model, which introduces the CBAM and ECA modules into the residual block, enhances the model’s attention to diseases, and reduces the influence of complex backgrounds on disease recognition. The proposed method has an accuracy rate of 98.7% on the test dataset, which is higher than classic convolutional neural networks such as AlexNet, VGG16, and InceptionresnetV2 and lightweight models such as MobileNetV3 and EfficientNetb0. The proposed model has superior performance and can be applied to mobile terminals to quickly identify wheat diseases.

CUSS-Net: A Cascaded Unsupervised-Based Strategy and Supervised Network for Biomedical Image Diagnosis and Segmentation.

Biomedical image segmentation and classification are critical components in a computer-aided diagnosis system. However, various deep convolutional neural networks are trained by a single task, ignoring the potential contribution of mutually performing multiple tasks. In this paper, we propose a cascaded unsupervised-based strategy to boost the supervised CNN framework for automated white blood cell (WBC) and skin lesion segmentation and classification, called CUSS-Net. Our proposed CUSS-Net consists of an unsupervised-based strategy (US) module, an enhanced segmentation network named E-SegNet, and a mask-guided classification network called MG-ClsNet. On the one hand, the proposed US module produces coarse masks that provide a prior localization map for the proposed E-SegNet to enhance it in locating and segmenting a target object accurately. On the other hand, the enhanced coarse masks predicted by the proposed E-SegNet are then fed into the proposed MG-ClsNet for accurate classification. Moreover, a novel cascaded dense inception module is presented to capture more high-level information. Meanwhile, we adopt a hybrid loss by combining a dice loss and a cross-entropy loss to alleviate the imbalance training problem. We evaluate our proposed CUSS-Net on three public medical image datasets. Experiments show that our proposed CUSS-Net outperforms representative state-of-the-art approaches.

Parallel matters: Efficient polyp segmentation with parallel structured feature augmentation modules

AbstractThe large variations of polyp sizes and shapes and the close resemblances of polyps to their surroundings call for features with long‐range information in rich scales and strong discrimination. This article proposes two parallel structured modules for building those features. One is the Transformer Inception module (TI) which applies Transformers with different reception fields in parallel to input features and thus enriches them with more long‐range information in more scales. The other is the Local‐Detail Augmentation module (LDA) which applies the spatial and channel attentions in parallel to each block and thus locally augments the features from two complementary dimensions for more object details. Integrating TI and LDA, a new Transformer encoder based framework, Parallel‐Enhanced Network (PENet), is proposed, where LDA is specifically adopted twice in a coarse‐to‐fine way for accurate prediction. PENet is efficient in segmenting polyps with different sizes and shapes without the interference from the background tissues. Experimental comparisons with state‐of‐the‐arts methods show its merits.

Damage location diagnosis of frame structure based on wavelet denoising and convolution neural network implanted with Inception module and LSTM

Accurate diagnosis of the damage location of the frame structure is very important for the overall damage assessment and subsequent maintenance of the frame structure. However, the frame structure generally works in the noise environment, which increases the difficulty of health monitoring and fault diagnosis of frame structure based on vibration data. In order to realize the accurate damage location diagnosis of structural frame under noise environment, this paper proposes a fault diagnosis method based on wavelet denoising, convolutional neural network, Inception module, and long short-term memory (LSTM) on the basis of complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN). In order to verify the effectiveness and superiority of the method proposed in this paper, the 4-story steel frame model of the University of British Columbia is taken as the research object, and the experiments are carried out with the method proposed in this paper, and under the same conditions, the comparative experiments are carried out with other similar methods. The experimental results show that the method proposed in this paper not only has high accuracy, but also has strong anti-noise ability, and its performance is better than other similar methods. Therefore, the fault diagnosis method proposed in this paper can effectively perform the accurate diagnosis of damage location of frame structure under noise environment.

An AI-based monitoring system for external disturbance detection and classification near a buried pipeline

External threats including excavation, drilling, construction, vandalism, or sabotage can result in pipeline leak or rupture. To improve pipeline safety, an AI-based monitoring system for buried pipeline external disturbance detection and classification is proposed. The system is designed to detect and recognize potential threats prior to the occurrence of damage, offering localized continuous non-invasive structural health monitoring. In the proposed system, an accelerometer measures the operational vibration of the buried pipeline due to excitation by both internal flow turbulence and external disturbances. In this paper, the characteristics of both internal and external disturbances are investigated including pressure perturbations, drilling, jumping, running, and walking. The designed monitoring system consists of two processing phases. In the first phase, a disturbance indicator is developed using common time–frequency analysis techniques to generate a Spectral Index Function (SIF) and a Severity Index (SI), and these are used as inputs to a Quadratic Support Vector Machine (Q-SVM) classifier. Four classifications are used and overall accuracies of above 99% are achieved. In the second phase, contributions due to normal operating conditions are removed using a wavelet denoising technique. Morphological features of external disturbances are represented by grayscale images created by performing Hilbert Spectral Analysis (HSA) on wavelet denoised time series. A four-layer Convolutional Neural Network (CNN) classifier with inception modules is designed to perform the classification task. The results show that external disturbances can be successfully classified with high accuracy (96.1%). The robustness of the monitoring system is examined using data sets collected from different accelerometer locations. The results show that the designed monitoring system has a high level of robustness.

MIU-Net: MIX-Attention and Inception U-Net for Histopathology Image Nuclei Segmentation

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Mixture lightweight transformer for scene understanding

In adapting Transformer from language to computer vision, the major obstacles are the high computational complexity and large model size of Transformer blocks, derived from the great quantity of visual tokens and high resolutions of input entities. To address these challenges, this paper presents a mixture lightweight Transformer (MLT) backbone for image understanding, where each Transformer block, called SH-Transformer, adopts Single-Head Self-Attention (SHSA) and Convolutional Inception Module (CIM). Unlike previous Transformers that compute Multi-Head Self-Attention (MHSA), SHSA limits the representation of input token into a single head, resulting in a low-dimensional embedding that greatly reduces computational complexity. In spite of adding a small number of model parameters, SHSA greatly minimizes the number of input tokens. As complementary of SHSA that only investigates global interactions, CIM is designed to explore multi-scale local information using lightweight convolutions in a multi-path parallel manner. Experimental results reveal that MLT yields competitive or state-of-the-art results with respect to recent transformers while keeping smaller model size and lower computational costs for different visual tasks, including image classification, semantic segmentation and object detection. Particularly, the proposed method has 4.2% improvement to the tiny version of Pyramid Vision Transformer on image classification of top-1 accuracy on ImageNet-1K.

Security Detection Design for Laboratory Networks Based on Enhanced LSTM and AdamW Algorithms

With the addition of multimedia big data, the diversification of network data types becomes more prominent, the proportion of unstructured data increases sharply, and the requirements of data application show the characteristics of rapid change. Laboratory servers hold a large amount of core experimental data, which is at risk of being compromised in the event of a cyberattack, and the rapid pace of information technology has made cyberattacks complex. In the face of the great challenges posed by continuously changing networks to network security, this paper proposes a network exception detection approach that combines an improved inception module incorporating an attention mechanism and a Bi-LSTM. The inception module with the attention mechanism enhances the adaptability of the neural network to different spatial feature scales in the network stream, weakens irrelevant non-critical features, and exploits the advantages of the Bi-LSTM in terms of temporal features of the network stream to effectively improve the accuracy of the detection of network attacks.

Grow-push-prune: Aligning deep discriminants for effective structural network compression

Most of today’s popular deep architectures are hand-engineered to be generalists. However, this design procedure usually leads to massive redundant, useless, or even harmful features for specific tasks. Unnecessarily high complexities render deep nets impractical for many real-world applications, especially those without powerful GPU support. In this paper, we attempt to derive task-dependent compact models from a deep discriminant analysis perspective. We propose an iterative and proactive approach for classification tasks which alternates between (1) a pushing step, with an objective to simultaneously maximize class separation, penalize co-variances, and push deep discriminants into alignment with a compact set of neurons, and (2) a pruning step, which discards less useful or even interfering neurons. Deconvolution is adopted to reverse ‘unimportant’ filters’ effects and recover useful contributing sources. A simple network growing strategy based on the basic Inception module is proposed for challenging tasks requiring larger capacity than what the base net can offer. Experiments on the MNIST, CIFAR10, and ImageNet datasets demonstrate our approach’s efficacy. On ImageNet, by pushing and pruning our grown Inception-88 model, we achieve more accurate models than Inception nets generated during growing, residual nets, and popular compact nets at similar sizes. We also show that our grown Inception nets (without hard-coded dimension alignment) clearly outperform residual nets of similar complexities.

Detecting diseases in apple tree leaves using FPN–ISResNet–Faster RCNN

ABSTRACT Apple leaf diseases typified by small disease spots are generally difficult to detect in images. This study proposes a deep learning model called the feature pyramid networks (FPNs) –inception squeeze-and-excitation ResNet (ISResNet)–Faster RCNN (region with convolutional neural network) model to improve the accuracy of detecting apple leaf diseases. Apple leaf diseases were identified, evaluated, and validated by using the FPN–ISResNet–Faster RCNN. The results were compared with those obtained by the single-shot multibox detector (SSD), Faster RCNN, and ISResNet–Faster RCNN. The detection accuracies obtained by using different feature extraction networks, positions and numbers of SE, inception modules, scales of FPN structures, and scales of anchor frames were also compared. The results showed that the values of average precision (AP), and APs with the thresholds of the intersection over union of 0.5 and 0.75 (AP50 and AP75), obtained from the FPN–ISResNet–Faster RCNN were 62.71%, 93.68%, and 70.94%, respectively, which are higher than those of the SSD, VGG–Faster RCNN, GoogleNet–Faster RCNN, ResNet50–Faster RCNN, ResNeXt–Faster RCNN, and ISResNet–Faster RCNN. FPN–ISResNet–Faster RCNN was shown to be able to detect diseases in apple leaves with high accuracy and generalizability.

Improving generalisation and accuracy of on-line milling chatter detection via a novel hybrid deep convolutional neural network

Unstable chatter seriously reduces the quality of machined workpiece and machining efficiency. In order to improve productivity, on-line chatter detection has attracted much interest in the past decades. Nevertheless, traditional methods are inevitably flawed due to the manually extracted features. Deep learning methods possess outstanding feature learning and classification capabilities, but the generalisation and accuracy are severely affected by the labelling and training of data. To address this, this paper proposed a novel hybrid deep convolutional neural network method combining an Inception module and a Squeeze-and-Excitation ResNet block (SR-block), namely ISR-CNN. The Inception module can automatically extract multi-scale features of cutting force signal to enrich the feature map. The SR-block can assign weights to different feature channels, thus suppressing useless feature maps and improving the model accuracy. Meanwhile, the introduction of SR-block also reduces the risk of gradient disappearance and speeds up the training of network. The generalisation and accuracy of the model is guaranteed by combining the two modules without training with transition state data. Milling tests were carried out on a wedge-shaped workpiece using different cutting parameters and tool overhang lengths to verify the accuracy and generalisability of the proposed method. The results showed that the proposed method outperforms other methods by achieving classification accuracy of on the validation and test sets 100% and 97.8%, respectively. In comparison to existing methods, the proposed method can correctly identify each machining state, including the transition states. Furthermore, the proposed method identifies the onset of chatter earlier than other methods, which is beneficial for chatter suppression.

NEAT: A Resilient Deep Representational Learning for Fault Detection Using Acoustic Signals in IIoT Environment

Fault diagnostics involving the Internet-of-Things (IoT) sensors and edge devices is a challenging task due to their limited energy and computational capabilities. Another challenge concerning IoT sensors or devices is the incursion of noise when used in an industrial environment. The noisy samples affect the decision support system that could lead to financial and operational losses. This article proposes a noisy encoder using artificial intelligence of things (NEAT) architecture for fault diagnosis in IoT edge devices. NEAT combines autoencoders and Inception module to co-train the clean and noisy samples for solving the said problem. Experimental results on benchmark data sets reveal that the NEAT architecture is noise resilient in comparison to the existing works. Furthermore, we also show that the NEAT architecture has lightweight characteristics as it yields a lower number of parameters, weight storage, training, and testing times that support its real-life applicability in an Industrial IoT environment.

Music Emotion Recognition Based on a Neural Network with an Inception-GRU Residual Structure

As a key field in music information retrieval, music emotion recognition is indeed a challenging task. To enhance the accuracy of music emotion classification and recognition, this paper uses the idea of inception structure to use different receptive fields to extract features of different dimensions and perform compression, expansion, and recompression operations to mine more effective features and connect the timing signals in the residual network to the GRU module to extract timing features. A one-dimensional (1D) residual Convolutional Neural Network (CNN) with an improved Inception module and Gate Recurrent Unit (GRU) was presented and tested on the Soundtrack dataset. Fast Fourier Transform (FFT) was used to process the samples experimentally and determine their spectral characteristics. Compared with the shallow learning methods such as support vector machine and random forest and the deep learning method based on Visual Geometry Group (VGG) CNN proposed by Sarkar et al., the proposed deep learning method of the 1D CNN with the Inception-GRU residual structure demonstrated better performance in music emotion recognition and classification tasks, achieving an accuracy of 84%.

DeepBreastCancerNet: A Novel Deep Learning Model for Breast Cancer Detection Using Ultrasound Images

Breast cancer causes hundreds of women’s deaths each year. The manual detection of breast cancer is time-consuming, complicated, and prone to inaccuracy. For Breast Cancer (BC) detection, several imaging methods are explored. However, sometimes misidentification leads to unnecessary treatment and diagnosis. Therefore, accurate detection of BC can save many people from unnecessary surgery and biopsy. Due to recent developments in the industry, deep learning’s (DL) performance in processing medical images has significantly improved. Deep Learning techniques successfully identify BC from ultrasound images due to their superior prediction ability. Transfer learning reuses knowledge representations from public models built on large-scale datasets. However, sometimes Transfer Learning leads to the problem of overfitting. The key idea of this research is to propose an efficient and robust deep-learning model for breast cancer detection and classification. Therefore, this paper presents a novel DeepBraestCancerNet DL model for breast cancer detection and classification. The proposed framework has 24 layers, including six convolutional layers, nine inception modules, and one fully connected layer. Also, the architecture uses the clipped ReLu activation function, the leaky ReLu activation function, batch normalization and cross-channel normalization as its two normalization operations. We observed that the proposed model reached the highest classification accuracy of 99.35%. We also compared the performance of the proposed DeepBraestCancerNet approach with several existing DL models, and the experiment results showed that the proposed model outperformed the state-of-the-art. Furthermore, we validated the proposed model using another standard, publicaly available dataset. The proposed DeepBraestCancerNet model reached the highest accuracy of 99.63%.

Temporal augmented contrastive learning for micro-expression recognition

Micro-expressions (MEs) can reveal the hidden but real emotion and are usually caused spontaneously. However, the characteristics of subtlety and temporariness with the lack of sufficient ME datasets make it hard for recognition. In this paper, we propose an adaptively temporal augmented momentum contrastive learning to alleviate these problems. For the small scale, we pre-train the model on a new interpolated dataset via contrastive learning with momentum contrast (MoCo). For the subtle and rapid facial movements, we augment the temporal dynamics using an adaptive dropout operation to redundant frames. Specifically, we use a recursive way to create a new interpolated dataset from raw datasets firstly. Then we design a shallow model with an inflated inception module as the encoder of the contrastive learning. Afterward, we pre-train the model on the new dataset via momentum contrastive learning. During the pre-training, we propose adaptively temporal augmentation via generative adversarial learning. After the pre-training, we take the encoder out and finetune it for recognition. Finally, we perform extensive experiments and ablation studies on three ME datasets. The results demonstrate the effectiveness of the MoCo-like pre-training and the temporal augmentation for recognition. Moreover, the pre-trained model outperforms other state-of-the-art models based on optical flow.

Using Deep Convolutional Neural Networks to Classify the Discharge Current of a Cold Atmospheric-Pressure Plasma Jet

Atmospheric-pressure plasma discharge can vary with conditions, such as working gas or discharge type, and the resulting discharge current has quite different electrical features. Hence, the provision of real-time and cost-effective monitoring of atmospheric-pressure plasma discharge is possible via current classification with a deep learning model. In addition, the current generated by an atmospheric-pressure plasma jet (APPJ) can be easily obtained, so it is suitable to use a data-driven convolutional neural network (CNN) to extract and analyze the current characteristics. This study presents two CNN classification applications of the current generated by an APPJ. First, we used the time-series classification approach known as InceptionTime to predict the APPJ working gas from helium, argon, and nitrogen. Second, InceptionTime was also used to predict the APPJ discharge type as Townsend discharge or glow discharge. InceptionTime leveraged the use of the inception module to reduce dimensions and avoid overfitting. Moreover, the use of four different sizes of filters along with the maxpooling layer provided sufficient receptive fields to analyze the entire current signal. We achieved 100% accuracy on a testing set of 1125 current signals, which did not include the training data during the working gas prediction; we also achieved 86.3% accuracy on a testing set of 750 current signals, which did not include the training data during the discharge type prediction. This study proves that the CNN can extract features from the discharge current waveforms and successfully predict the discharge type and working gas of an APPJ. The high-speed computing characteristics of deep learning may also facilitate real-time monitoring or diagnosis of APPJ.

Registration of 3D medical images based on unsupervised cooperative cascade of deep networks

In this paper, a deformable registration network (DR-Net) and a multi-scale cascading strategy are designed for the registration of largely deformed 3D medical images. Our DR-Net appears as a U-shaped convolutional neural network with a pyramidal input module (PIM), a light weighted sequential Inception module and an SCAM convolutional attention module. Our multi-scale cooperative cascading strategy integrates the deformation field information within and between sub-networks at different scales to synthesize the cascaded deformation fields. To cooperatively train the cascaded network, not only the output of the final network layer but also the multi-scale outputs from different layers of the decoder in the last cascaded sub-network are used to calculate loss function. As compared with the VoxelMorph and IVTN, the average dice similarity coefficients (Dice) achieved with our DR-Net are 2.4% and 2.5% higher on the Sliver dataset and are 2.5% and 2.4% higher on the LiTS dataset. The average Dice coefficients achieved with our multi-scale cascading strategy of three DR-Nets are 1.6% and 1.9% higher than those of the VM-CR3 and are 1.5% and 1.7% higher than those of the IVTN-CR3 on these two datasets, respectively. These results show that not only our proposed DR-Net itself but also the cascade of them outperform the state-of-the-art methods and their cascades in registration accuracy.