Complexity Of Convolutional Neural Networks Research Articles

Research on convolutional neural network to realize high-quality dynamic holographic display

There are still some challenges in the computation speed and reproduction quality of holographic display. In this paper, a method is proposed to realize a high-quality 3D holographic display by using a complex value convolutional neural network. Firstly, the layer-based method is used to cut and layer the object. The amplitude and phase of each layer of the object are calculated by a complex value network at the same time. Then, the output of the complex value type of the network is converted into a phase-only hologram, and each layer of the hologram is superimposed to reproduce the complete 3D object. In addition, a novel color convolutional neural network is used to generate R, G and B three-color holograms simultaneously, reducing the time required for three-channel recalculation of holograms. The proposed method only takes 16 ms to generate a hologram from single-layer object, the average peak signal-to-noise ratio of the reconstructed image is 28 dB, and the structural similarity exceeds 0.99, which realizes a high-quality dynamic holographic display. The experimental results confirm the feasibility of the proposed method and provide a new method for real-time holographic display of 3D object.

Exploring the Impact of Model Complexity on Laryngeal Cancer Detection.

Background: Laryngeal cancer accounts for a third of all head and neck malignancies, necessitating timely detection for effective treatment and enhanced patient outcomes. Machine learning shows promise in medical diagnostics, but the impact of model complexity on diagnostic efficacy in laryngeal cancer detection can be ambiguous. Methods: In this study, we examine the relationship between model sophistication and diagnostic efficacy by evaluating three approaches: Logistic Regression, a small neural network with 4 layers of neurons and a more complex convolutional neural network with 50 layers and examine their efficacy on laryngeal cancer detection on computed tomography images. Results: Logistic regression achieved 82.5% accuracy. The 4-Layer NN reached 87.2% accuracy, while ResNet-50, a deep learning architecture, achieved the highest accuracy at 92.6%. Its deep learning capabilities excelled in discerning fine-grained CT image features. Conclusion: Our study highlights the choices involved in selecting a laryngeal cancer detection model. Logistic regression is interpretable but may struggle with complex patterns. The 4-Layer NN balances complexity and accuracy. ResNet-50 excels in image classification but demands resources. This research advances understanding affect machine learning model complexity could have on learning features of laryngeal tumor features in contrast CT images for purposes of disease prediction.

Speech enhancement using deep complex convolutional neural network (DCCNN) model

Speech enhancement using deep complex convolutional neural network (DCCNN) model

Breast Cancer Detection and Localizing the Mass Area Using Deep Learning

Breast cancer presents a substantial health obstacle since it is the most widespread invasive cancer and the second most common cause of death in women. Prompt identification is essential for effective intervention, rendering breast cancer screening a critical component of healthcare. Although mammography is frequently employed for screening purposes, the manual diagnosis performed by pathologists can be laborious and susceptible to mistakes. Regrettably, the majority of research prioritizes mass classification over mass localization, resulting in an uneven distribution of attention. In response to this problem, we suggest a groundbreaking approach that seeks to identify and pinpoint cancers in breast mammography pictures. This will allow medical experts to identify tumors more quickly and with greater precision. This paper presents a complex deep convolutional neural network design that incorporates advanced deep learning techniques such as U-Net and YOLO. The objective is to enable automatic detection and localization of breast lesions in mammography pictures. To assess the effectiveness of our model, we carried out a thorough review that included a range of performance criteria. We specifically evaluated the accuracy, precision, recall, F1-score, ROC curve, and R-squared error using the publicly available MIAS dataset. Our model performed exceptionally well, with an accuracy rate of 93.0% and an AUC (area under the curve) of 98.6% for the detection job. Moreover, for the localization task, our model achieved a remarkably high R-squared value of 97%. These findings highlight that deep learning can boost the efficiency and accuracy of diagnosing breast cancer. The automation of breast lesion detection and classification offered by our proposed method bears substantial benefits. By alleviating the workload burden on pathologists, it facilitates expedited and accurate breast cancer screening processes. As a result, the proposed approach holds promise for improving healthcare outcomes and bolstering the overall effectiveness of breast cancer detection and diagnosis.

Deep learning and saliency-based parking IoT classification under different weather conditions

The world’s population is growing at an exponential rate, which has created a slew of new issues in our daily lives. Among these challenges is the difficulty of finding available parking spaces. To overcome this obstacle, an effective system for detecting vacant parking spaces is crucial, particularly in densely populated areas. Even though there are numerous parking detecting systems on the market today, they all have their own set of restrictions. There are primarily two types of parking systems available. The first is a hardware-based system that uses hardware to detect whether a space is vacant or occupied, and the second is a smart parking system that uses cameras to classify parking status. Vision-based parking systems are becoming increasingly popular due to benefits such as flexibility and cost efficiency. However, due to challenging weather conditions and various occlusions caused by objects, such systems are challenging to develop. The limitations of a vision-based parking system must be addressed for it to become more accurate. To date, many researchers have attempted to incorporate deep learning-based parking systems with various algorithms but none of them has focused on reducing the complexity of Convolutional Neural Network (CNN) training through unsupervised pre-processing steps. In this study, an attempt is made to reduce the complexity of CNN by integrating salient features as a pre-processing step for normalizing weather conditions. It has been observed that salient features-based pre-processing makes the system more robust in terms of accuracy and time efficiency.

Design of Fully Spectral CNNs for Efficient FPGA-Based Acceleration.

Computing convolutional layers in the frequency domain using fast Fourier transformation (FFT) has been demonstrated to be effective in reducing the computational complexity of convolutional neural networks (CNNs). Nevertheless, the main challenge of this approach lies in the frequent and repeated transformations between the spatial and frequency domains due to the absence of nonlinear functions in the spectral domain, as such it makes the benefit less attractive for low-latency inference, especially on embedded platforms. To overcome the drawbacks in the existing FFT-based convolution, we propose a fully spectral CNN using a novel spectral-domain adaptive rectified linear unit (ReLU) layer, which completely removes the compute-intensive transformations between the spatial and frequency domains within the network. The proposed fully spectral CNNs maintain the nonlinearity of the spatial CNNs while taking into account the hardware efficiency. We then propose a deeply customized and compute-efficient hardware architecture to accelerate the fully spectral CNN inference on field programmable gate array (FPGA). Different hardware optimizations, such as spectral-domain intralayer and interlayer pipeline techniques, are introduced to further improve the performance of throughput. To achieve a load-balanced pipeline, a design space exploration (DSE) framework is proposed to optimize the resource allocation between hardware modules according to the resource constraints. On an Intel's Arria 10 SX160 FPGA, our optimized accelerator achieves a throughput of 204 Gop/s with 80% of compute efficiency. Compared with the state-of-the-art spatial and FFT-based implementations on the same device, our accelerator is 4×∼ 6.6× and 3.0×∼ 4.4× faster while maintaining a similar level of accuracy across different benchmark datasets.

Energy Complexity of Convolutional Neural Networks.

The energy efficiency of hardware implementations of convolutional neural networks (CNNs) is critical to their widespread deployment in low-power mobile devices. Recently, a number of methods have been proposed for providing energy-optimal mappings of CNNs onto diverse hardware accelerators. Their estimated energy consumption is related to specific implementation details and hardware parameters, which does not allow for machine-independent exploration of CNN energy measures. In this letter, we introduce a simplified theoretical energy complexity model for CNNs, based on only a two-level memory hierarchy that captures asymptotically all important sources of energy consumption for different CNN hardware implementations. In this model, we derive a simple energy lower bound and calculate the energy complexity of evaluating a CNN layer for two common data flows, providing corresponding upper bounds. According to statistical tests, the theoretical energy upper and lower bounds we present fit asymptotically very well with the real energy consumption of CNN implementations on the Simba and Eyeriss hardware platforms, estimated by the Timeloop/Accelergy program, which validates the proposed energy complexity model for CNNs.

Language Dissemination Paths and Modes Aided by Computer Technology

The expansion of technology and computer science, as well as advancements in language instruction and learning methodologies, has enabled computer-assisted language learning technologies to tackle this challenge. In the field of Chinese learning, a few language learning computerized systems in the country and abroad concentrate mainly on language, grammar acquisition only have one or two assessment indicators as basis of evaluation, that definite functional flaws provide a general assessment to learners' pronunciation. In this manuscript, Language Dissemination Paths and Modes Aided by Computer Technology (LDPM-QICCNN-KOA) are proposed. The input data are collected from Chinese Corpus dataset. Then the data is given into unscented trainable kalman filter for preprocessing the input data. Then the preprocessed data are provided to QICCNN for Language Dissemination. In general, the based Quantum-inspired Complex Convolutional Neural Network doesn’t express adapting optimization approaches to determine optimal parameters to ensure exact identification. Hence, KOA utilized to enhance Quantum-inspired Complex Convolutional Neural Network, which accurately done the Language Dissemination Paths and Modes. The proposed LDPM-QICCNN-KOA method is executed on python. Then performance of proposed technique is analyzed with other existing methods. The proposed technique attains 26.36%, 20.69% and 35.29% higher accuracy; 19.23%, 23.56%, and 33.96% higher F1-Score; 26.28%, 31.26%, and 19.66% higher precision when comparing with the existing methods such as research on network oral English teaching system depend on machine learning (LDPM-DBN), nonlinear network speech recognition structure in deep learning algorithm (LDPM-DNN), research on open oral English scoring system depend on neural network (LDPM-BPNN).

Detection of tuberculosis using customized MobileNet and transfer learning from chest X-ray image

One of the most contagious diseases in the world, tuberculosis (TB) is brought on by the bacteria Mycobacterium tuberculosis. This hazardous scenario can cause life losses and requires expert doctors and several hours to detect the disease. Using the MobileNet transfer learning model, a computationally lightweight model has been proposed in this study. The optimal model for the diagnosis of tuberculosis has been determined after testing numerous variants on the base model with pre-trained weights. A computationally light transfer learning model is proposed to obtain the maximum overall accuracy of 98.66%. The improvement over the best existing model is quite significant. The transfer learning model (DenseNet) utilized in this existing model is based on a very complex convolutional neural network (CNN), and as a result, the model requires greater amounts of time. The performance of the other existing models is relatively less in comparison to our proposed model, and the methods have a number of other drawbacks. Our goal in this work is to create a more accurate model that requires less computational effort. When compared to previous models, our model has a very less number of trainable parameters, which causes the model to converge more quickly and predict more accurately. Our approach also has the benefit of being simply able to modify its weights when the system is further updated with new datasets. Additionally, because of its lightweight architecture, it can be installed on mobile devices as well as used in web-based applications with ease. To analyze and validate the proposed method, we have collected data from Kaggle and used MC, CHN and NIH datasets.

Low complex CNN for semiconductor wafer defect detection

Fabrication of semiconductor wafers is a complex process and chances of defect wafers are high. Because of defective wafers the circuit patterns will not be created correctly and it is necessary to identify them. Manual identification of defects are time consuming and expensive. Deep learning methods are widely used for defect detection. In this paper we propose a simple Convolutional Neural Network (CNN) model for classification of nine defects in wafers. A custom CNN consisting of 9 layers is used for the classification of defects as Center, Donut, Edge-Loc, Edge-Ring, Loc, Random, Scratch, Near-full, and None. Performance of the model is evaluated using WM-811K dataset. Results shows that the model classifies the defects with high confidence score and an accuracy of 99.1% is achieved using this method. Further, the convolution operation in the CNN is realized using Coordinate Rotation Digital Computer (CORDIC) algorithm. The model is implemented in Field Programmable Gate Arrays (FPGA) and proved less complex method and consume less computational power than conventional methods.

Improved Complex Convolutional Neural Network Based on SPIRiT and Dense Connection for Parallel MRI Reconstruction

To accelerate the data acquisition speed of magnetic resonance imaging (MRI) and improve the reconstructed MR images’ quality, we propose a parallel MRI reconstruction model (SPIRiT-Net), which combines the iterative self-consistent parallel imaging reconstruction model (SPIRiT) with the cascaded complex convolutional neural networks (CCNNs). More specifically, this model adopts the SPIRiT model for reconstruction in the k-space domain and the cascaded CCNNs with dense connection for reconstruction in the image domain. Meanwhile, this model introduces the data consistency layers for better reconstruction in both the image domain and the k-space domain. The experimental results on two clinical knee datasets as well as the fastMRI brain dataset under different undersampling patterns show that the SPIRiT-Net model achieves better reconstruction performance in terms of visual effect, peak signal-to-noise ratio, and structural similarity over SPIRiT, Deepcomplex, and DONet. It will be beneficial to the diagnosis of clinical medicine.

Source localization in deep ocean based on complex convolutional neural network

To solve the problem that phase information cannot be used effectively in underwater acoustic localization, a deep learning network based on complex convolutional neural network is proposed in this paper. The complex convolution layer is used to effectively utilize the phase features favorable to the source localization problem, and the phase information is used to improve the feature extraction performance. Through simulation, the performance of the network for source localization in deep-sea direct arrival region under different SNR conditions is analyzed. The results show that the complex convolutional network proposed in this paper can locate the sound source with less computation and has better performance under the condition of low SNR.

Forwardformer: Efficient Transformer With Multi-Scale Forward Self-Attention for Day-Ahead Load Forecasting

Accurate load forecasting can maintain the safety and stability of power grids. The mainstream models are based on complex recurrent or convolutional neural networks (RNNs, CNNs), and in recent years they are often used in combination with the attention mechanism. The shortcomings of these models are that they cannot get rid of sequential computation and fail to capture long-term dependence. For better application in day-ahead load forecasting (DALF), we propose a new network architecture, i.e., Forwardformer, which is implemented by imposing some effective improvements on the Transformer (a pioneering network model with applications in Natural Language Processing (NLP)). The core of Forwardformer is the multi-scale forward self-attention (MSFSA) and the correction structure of the encoder-dual decoder, which confer better computational efficiency and forecasting accuracy. Meanwhile, to improve forecasting accuracy on special days (weekends, holidays, etc.), the MSFSA configures dilated attention and global attention for them, respectively. Experiments performed on datasets from China and America demonstrated that the Forwardformer requires less runtime while being superior in forecasting accuracy. Especially in terms of weekends and holidays, it has outstanding advantages and provides a new solution to the DALF problem.

EEGProgress: A fast and lightweight progressive convolution architecture for EEG classification

Because of the intricate topological structure and connection of the human brain, extracting deep spatial features from electroencephalograph (EEG) signals is a challenging and time-consuming task. The extraction of topological spatial information plays a crucial role in EEG classification, and the architecture of the spatial convolution greatly affects the performance and complexity of convolutional neural network (CNN) based EEG classification models. In this study, a progressive convolution CNN architecture named EEGProgress is proposed, aiming to efficiently extract the topological spatial information of EEG signals from multi-scale levels (electrode, brain region, hemisphere, global) with superior speed. To achieve this, the raw EEG data is permuted using the empirical topological permutation rule, integrating the EEG data with numerous topological properties. Subsequently, the spatial features are extracted by a progressive feature extractor including prior, electrode, region, and hemisphere convolution blocks, progressively extracting the deep spatial features with reduced parameters and speed. Finally, the comparison and ablation experiments under both cross-subject and within-subject scenarios are conducted on a public dataset to verify the performance of the proposed EEGProgress and the effectiveness of the topological permutation. The results demonstrate the superior feature extraction ability of the proposed EEGProgress, with an average increase of 4.02% compared to other CNN-based EEG classification models under both cross-subject and within-subject scenarios. Furthermore, with the obtained average testing time, FLOPs, and parameters, the proposed EEGProgress outperforms other comparison models in terms of model complexity.

Which Convolutional Neural Network Architecture is the Best at Detecting Alzheimer’s Disease: A head‐to‐head Comparison Using Relevance Maps

AbstractBackgroundAlzheimer’s disease (AD) is a neurodegenerative disorder which causes gradual and irreversible damage to the brain. These patterns of brain damage can be detected using T1‐weighted MRI scans by modern neuroimaging methods. Recent developments in convolutional neural networks (CNN) have achieved promising results in various computer‐vision tasks. Feature attribution methods such as layer‐wise relevance propagation (LRP) allow tracing back the information flow in CNNs to derive relevance heatmaps, which approximate the contribution of the input image regions on the model decision. We addressed the open question, which of the most common CNN architectures is best suited for medical image detection, i.e. AD classification based on MRI data.MethodFour CNN architectures are widely used in the literature: AlexNet, VGG, ResNet, and DenseNet. We adapted these CNN architectures to be used with 3D brain MRI data and trained the models on a heterogeneous dataset with N>2200 from four large studies. We applied tenfold cross‐validation and additionally evaluated the results on an independent test dataset.ResultThe more complex CNN architectures DenseNet and ResNet with ‘skip connections’, provided the best results for both group separation tasks of ‐ AD vs Cognitively Normal (CN), and Mild Cognitive Impairment (MCI) vs CN, although the overall differences in accuracy did not reach statistical significance. From the mean relevance maps obtained using the LRP relevance propagation method (see Fig. 1.), we found that DenseNet focused primarily on the medial temporal lobe and posterior cingulate cortex atrophy. ResNet’s mean relevance maps were more noisy and heterogeneous.ConclusionDenseNet provided the most focused relevance maps, best matching apriori expectations of brain regions contributing to the detection of AD. Its ‘dense connections’ between the layers enabled a highly efficient information flow at various scales. Our evaluation setup also demonstrated the value of a holistic evaluation of models, where relevance maps are used in combination with classical performance metrics.

Which Convolutional Neural Network Architecture is the Best at Detecting Alzheimer’s Disease: A head‐to‐head Comparison Using Relevance Maps

AbstractBackgroundAlzheimer’s disease (AD) is a neurodegenerative disorder which causes gradual and irreversible damage to the brain. These patterns of brain damage can be detected using T1‐weighted MRI scans by modern neuroimaging methods. Recent developments in convolutional neural networks (CNN) have achieved promising results in various computer‐vision tasks. Feature attribution methods such as layer‐wise relevance propagation (LRP) allow tracing back the information flow in CNNs to derive relevance heatmaps, which approximate the contribution of the input image regions on the model decision. We addressed the open question, which of the most common CNN architectures is best suited for medical image detection, i.e. AD classification based on MRI data.MethodFour CNN architectures are widely used in the literature: AlexNet, VGG, ResNet, and DenseNet. We adapted these CNN architectures to be used with 3D brain MRI data and trained the models on a heterogeneous dataset with N>2200 from four large studies. We applied tenfold cross‐validation and additionally evaluated the results on an independent test dataset.ResultThe more complex CNN architectures DenseNet and ResNet with ‘skip connections’, provided the best results for both group separation tasks of ‐ AD vs Cognitively Normal (CN), and Mild Cognitive Impairment (MCI) vs CN, although the overall differences in accuracy did not reach statistical significance. From the mean relevance maps obtained using the LRP relevance propagation method (see Fig. 1.), we found that DenseNet focused primarily on the medial temporal lobe and posterior cingulate cortex atrophy. ResNet’s mean relevance maps were more noisy and heterogeneous.ConclusionDenseNet provided the most focused relevance maps, best matching apriori expectations of brain regions contributing to the detection of AD. Its ‘dense connections’ between the layers enabled a highly efficient information flow at various scales. Our evaluation setup also demonstrated the value of a holistic evaluation of models, where relevance maps are used in combination with classical performance metrics.

EEG-Based Seizure Detection Using Variable-Frequency Complex Demodulation and Convolutional Neural Networks

Epilepsy is a complex neurological disorder characterized by recurrent and unpredictable seizures that affect millions of people around the world. Early and accurate epilepsy detection is critical for timely medical intervention and improved patient outcomes. Several methods and classifiers for automated epilepsy detection have been developed in previous research. However, the existing research landscape requires innovative approaches that can further improve the accuracy of diagnosing and managing patients. This study investigates the application of variable-frequency complex demodulation (VFCDM) and convolutional neural networks (CNN) to discriminate between healthy, interictal, and ictal states using electroencephalogram (EEG) data. For testing this approach, the EEG signals were collected from the publicly available Bonn dataset. A high-resolution time–frequency spectrum (TFS) of each EEG signal was obtained using the VFCDM. The TFS images were fed to the CNN classifier for the classification of the signals. The performance of CNN was evaluated using leave-one-subject-out cross-validation (LOSO CV). The TFS shows variations in its frequency for different states that correspond to variation in the neural activity. The LOSO CV approach yields a consistently high performance, ranging from 90% to 99% between different combinations of healthy and epilepsy states (interictal and ictal). The extensive LOSO CV validation approach ensures the reliability and robustness of the proposed method. As a result, the research contributes to advancing the field of epilepsy detection and brings us one step closer to developing practical, reliable, and efficient diagnostic tools for clinical applications.

GeneNet: Transfer learning-based hybrid African buffalo optimization with genetic algorithm for gene expression based cancer classification

Gene therapy is one of the advanced medical diagnosis environments, which is used to identify the solutions to various cancers by identifying the perfect gene expressions. The computer aided detection of gene expressions is a research problem, where cancer type is identified by artificial intelligence methods. However, the conventional machine learning models were failed to provide the accurate cancer classification and resulted in poor quantitative performance. Therefore, this work focuses on the implementation of gene expression-based cancer classification (GECC) network, here after referred as GeneNet. Initially, the proposed GeneNet performs the data preprocessing to normalize the gene-expression data by replacing the unknown characters and missing symbols. Then, the visual geometry group developed VGG16 model for extraction of correlated, and structural features from pre-processed dataset. In addition, hybrid African buffalo optimization with genetic algorithm (HABO-GA) is used to select the optimal features through natural inspired hunting (searching) properties. Finally, low complex convolutional neural network (LC-CNN) model is trained with HABO-GA features, which also performs the multi-class cancer classification from test data. The simulations are conducted on publicly available GECC datasets such as lymphography, ovarian, leukemia, lung, and prostate cancers discloses the superiority of proposed GeneNetas compared to state-of-the-art approaches with 86.6 %, 100 %, 76.92 %, 100 %, and 100 % of accuracy, respectively.

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

Convolutional Neural Networks (CNNs) have been widely deployed, while traditional cloud data-centers based applications suffer from the bandwidth and latency network demand when applying to Industrial-Internet-of-Things (IIoT) fields. It is critical to migrate the CNN inference to edge devices for efficiency and security concerns. However, it is challenging to deploy complex CNNs on resource-constraint IIoT edge devices due to a large number of parameters and intensive floating-point computations. In this paper, we propose ABM-SpConv-SIMD, an on-device inference optimization framework, aiming at accelerating the network inference by fully utilizing the low-cost and common CPU resource. ABM-SpConv SIMD first adopts a model optimizer with pruning and quantization, which produces Sparse Convolutional models. And then, the Accumulation-Before-Multiplication mechanism is proposed to reduce multiplication operations. Additionally, the SIMD instructions, which are commonly available on cost-effective edge devices, are employed to improve the performance of convolutions. We have implemented ABM-SpConv-SIMD base on the Arm Compute Library software framework and evaluated on Hikey970 and Raspberry Pi devices with two representative models AlexNet and ResNet50. The results show that the ABM SpConv-SIMD can significantly improve the performance, and achieve on average of 1.96x and 1.73x speedup respectively over the baseline implementation with negligible loss of accuracy.

Retraction: Magnetic resonance imaging reconstruction algorithm under complex convolutional neural network in diagnosis and prognosis of cerebral infarction.

Retraction: Magnetic resonance imaging reconstruction algorithm under complex convolutional neural network in diagnosis and prognosis of cerebral infarction.