Convolutional Neural Network Processing Research Articles

Investigation of multi-input convolutional neural networks for the prediction of particleboard mechanical properties.

This study explored the potential of building an image-based quality control system for particleboard manufacturing. Single-layer particleboards were manufactured under 27 operating conditions and their modulus of elasticity (MOE) and the modulus of rupture (MOR) were determined. Subsequently, images of the upper surface, lower surface, and cross-section of each specimen were collected. Two types of convolutional neural networks (CNNs) were designed: a single-input CNN processing one image and a multi-input CNN capable of analyzing multiple images simultaneously. Their prediction accuracies were then compared. Among the single-input CNNs, the cross-sectional image yielded the best prediction accuracy for both the MOE and MOR. For multi-input CNNs, the combination of the upper surface and cross-sectional images produced the highest scores when the model merged the information from each image at early stage, outperforming single-input CNNs. Adding density information to multi-input CNNs significantly improved prediction accuracy for both MOE and MOR, achieving optimal results. Regression activation maps were constructed to visualize the image features that were strongly correlated with the predicted results. For MOE prediction, the precise location of phenol formaldehyde (PF) resin and particle alignment were crucial. For MOR prediction, the interface between particles and PF resin was the key.

Towards the Automation of Non-destructive Fault Recognition: Enhancement of Robustness and Accuracy Through AI Powered Acoustic and Thermal Signal Analysis in Time, Frequency- and Time-Frequency Domains

Non-destructive inspection and analysis techniques are crucial for quality assessment and defect analysis in various industries. They enable for screening and monitoring of parts and products without alteration or impact, facilitating the exploration of material interactions and defect formation. With increasing complexity in microelectronic technologies, high reliability, robustness and thus, successful failure analysis is essential. Machine learning (ML) approaches have been developed and evaluated for the analysis of acoustic echo signals and time-resolved thermal responses for assessing their ability for defect detection. In the present paper different ML architectures were evaluated, including 1D and 2D convolutional neural networks (CNNs) after transforming time domain data into the spectral- and wavelet domains. Results showed that 2D CNNs processing data in wavelet domain representation performed best, however at the expense of additional computational effort. Furthermore, ML-based analysis was explored for lock-in thermography to detect and locate defects in the axial dimension based on thermal emissions. While promising, further research is needed to fully realize its potential.

Exposing video surveillance object forgery by combining TSF features and attention-based deep neural networks

Recently, forensics has encountered a new challenge with video surveillance object forgery. This type of forgery combines the characteristics of popular video copy-move and splicing forgeries, failing most existing video forgery detection schemes. In response to this new forgery challenge, this paper proposes a Video Surveillance Object Forgery Detection (VSOFD) method including three parts components: (i) The proposed method presents a special-combined extraction technique that incorporates Temporal-Spatial-Frequent (TSF) perspectives for TSF feature extraction. Furthermore, TSF features can effectively represent video information and benefit from feature dimension reduction, improving computational efficiency. (ii) The proposed method introduces a universal, extensible attention-based Convolutional Neural Network (CNN) baseline for feature processing. This CNN processing architecture is compatible with various series and parallel feed-forward CNN structures, considering these structures as processing backbones. Therefore, the proposed CNN architecture benefits from various state-of-the-art structures, leading to addressing each independent TSF feature. (iii) The method adopts an encoder-attention-decoder RNN framework for feature classification. By incorporating temporal characteristics, the framework can further identify the correlations between the adjacent frames to classify the forgery frames better. Finally, experimental results show that the proposed network can achieve the best F1 = 94.69 % score, increasing at least 5–12 % from the existing State-Of-The-Art (SOTA) VSOFD schemes and other video forensics.

Identification of Plant Leaf Disease Using CNN and Image Processing

Agriculture is an important sector for the growing people of the world to fulfil the minimum requirements of food. Identifying plant infection in the agricultural sector is complex. If the detection is wrong, there is more damage to crop production and economic loss of the market. Leaf infection identification need a large number of labors, command of plant disorders and requires a lot of observing time. Therefore, this analysis explains detection of plant leaf disease by utilizing CNN and image processing. Alex Net and ResNet-50 are Convolutional Neural Network (CNN) models. First, this method is performed on Kaggle datasets of potato and tomato plants to examine the characteristics of an infected leaves. Then, feature extraction and categorization method is executed on dataset pictures to observe leaf disorders using Alex Net and ResNet-50 models by executing image processing. Observational outputs demonstrate potential of described method, in which it reaches a complete accuracy of 98% and 95% of Alex Net and ResNet50. The output shows that described model particularly predicts defective plant leaves from healthy leaf images.

Currency Classification Using Deep Learning

Among the uses of machine learning is the recognition of facial expressions. Based on the features that are derived from an image, it assigns a facial expression to one of the classes of facial expressions. Convolutional Neural Network (CNN) is a classification technique that may also be used to identify patterns in an image. We used the CNN approach to identify facial expressions in our proposed study. To increase the precision of facial emotion recognition, the wavelet transform is used after CNN processing. Seven distinct facial expressions are included in the facial expression image dataset that was obtained from Kaggle. The findings of the face expression recognition experiment utilizing CNN and wavelet transform show that the accuracy is improved, and the output is audible. Index Terms: Facial and Convolutional Neural Nets

Reflective microring-resonator-based microwave photonic sensor incorporating a self-attention assisted convolutional neural network

In this paper, a reflective microring resonator (MRR)-based microwave photonic (MWP) sensor incorporating a self-attention convolutional neural network (CNN) is presented. An MRR cascaded with an inverse-designed optical reflector is adopted as the sensor probe to allow for utilizing the responses generated from both the clockwise and counterclockwise resonant modes. Through the MWP interrogation, the cascaded resonant modes can be transformed into distinctive deep radio-frequency (RF) spectral notches under different modulator bias conditions. By using a self-attention assisted CNN processing to leverage both the local and global features of the RF spectra, a sensing model with improved accuracy can be established. As a proof of concept, the proposed scheme is experimentally demonstrated in temperature sensing. Even with a small dataset, the root-mean-square error of the sensing model established after training is achieved at 0.026°C, which shows a 10-fold improvement in sensing accuracy compared to that of the traditional linear fitting model.

6G Traffic Prediction with a Novel Parallel Convolutional Neural Networks Architecture and Matrix Format Method Integration

In the evolving world of wireless communication, sixth generation (6G) networks represent a significant leap forward. Beyond its high-speed and reliable communication, 6G integrates Artificial Intelligence (AI), making networks intelligent entities. This elevates the infrastructure of smart cities and other ecosystems. A critical factor in 6G's success is real-time traffic analysis. As 6G aims to interconnect billions of devices, it faces unprecedented traffic patterns. Practical traffic analysis ensures optimal performance, resource distribution, and energy efficiency. It also supports the network in handling vital sectors like healthcare and transportation by anticipating congestion and prioritizing crucial data. However, traditional traffic analysis techniques designed for earlier generations cannot accommodate 6G's demands. With 6G's integration of diverse technologies, understanding traffic becomes more challenging. Recent advancements have incorporated deep learning architectures, notably Convolutional Neural Networks (CNNs), for traffic analysis. While these models show potential, adapting them to 6G's specifics remains challenging. This research presents a unique parallel CNN architecture for 6G traffic prediction. It converts network data into an image using the Matrix Format Method (MFM), making it suitable for CNN processing. This innovation addresses the limitations of traditional methods and meets 6G's requirements. Compared to other models, our parallel CNN architecture highlights enhanced performance, promising increased traffic prediction accuracy. It also paves the way for improved resource allocation, energy management, and quality of service in 6G environments.

Automated Detection of Cerebral Microbleeds on Two-dimensional Gradient-recalled Echo T2* Weighted Images Using a Morphology Filter Bank and Convolutional Neural Network

We present a novel algorithm for the automated detection of cerebral microbleeds (CMBs) on 2D gradient-recalled echo T2* weighted images (T2*WIs). This approach combines a morphology filter bank with a convolutional neural network (CNN) to improve the efficiency of CMB detection. A technical evaluation was performed to ascertain the algorithm's accuracy. In this retrospective study, 60 patients with CMBs on T2*WIs were included. The gold standard was set by three neuroradiologists based on the Microbleed Anatomic Rating Scale guidelines. Images with CMBs were extracted from the training dataset comprising 30 cases using a morphology filter bank, and false positives (FPs) were removed based on the threshold of size and signal intensity. The extracted images were used to train the CNN (Vgg16). To determine the effectiveness of the morphology filter bank, the outcomes of the following two methods for detecting CMBs from the 30-case test dataset were compared: (a) employing the morphology filter bank and additional FP removal and (b) comprehensive detection without filters. The trained CNN processed both sets of initial CMB candidates, and the final CMB candidates were compared with the gold standard. The sensitivity and FPs per patient of both methods were compared. After CNN processing, the morphology-filter-bank-based method had a 95.0% sensitivity with 4.37 FPs per patient. In contrast, the comprehensive method had a 97.5% sensitivity with 25.87 FPs per patient. Through effective CMB candidate refinement with a morphology filter bank and FP removal with a CNN, we achieved a high CMB detection rate and low FP count. Combining a CNN and morphology filter bank may facilitate the accurate automated detection of CMBs on T2*WIs.

Analog Convolutional Operator Circuit for Low-Power Mixed-Signal CNN Processing Chip

In this paper, we propose a compact and low-power mixed-signal approach to implementing convolutional operators that are often responsible for most of the chip area and power consumption of Convolutional Neural Network (CNN) processing chips. The convolutional operators consist of several multiply-and-accumulate (MAC) units. MAC units are the primary components that process convolutional layers and fully connected layers of CNN models. Analog implementation of MAC units opens a new paradigm for realizing low-power CNN processing chips, benefiting from less power and area consumption. The proposed mixed-signal convolutional operator comprises low-power binary-weighted current steering digital-to-analog conversion (DAC) circuits and accumulation capacitors. Compared with a conventional binary-weighted DAC, the proposed circuit benefits from optimum accuracy, smaller area, and lower power consumption due to its symmetric design. The proposed convolutional operator takes as input a set of 9-bit digital input feature data and weight parameters of the convolutional filter. It then calculates the convolutional filter’s result and accumulates the resulting voltage on capacitors. In addition, the convolutional operator employs a novel charge-sharing technique to process negative MAC results. We propose an analog max-pooling circuit that instantly selects the maximum input voltage. To demonstrate the performance of the proposed mixed-signal convolutional operator, we implemented a CNN processing chip consisting of 3 analog convolutional operators, with each operator processing a 3 × 3 kernel. This chip contains 27 MAC circuits, an analog max-pooling, and an analog-to-digital conversion (ADC) circuit. The mixed-signal CNN processing chip is implemented using a CMOS 55 nm process, which occupies a silicon area of 0.0559 mm2 and consumes an average power of 540.6 μW. The proposed mixed-signal CNN processing chip offers an area reduction of 84.21% and an energy reduction of 91.85% compared with a conventional digital CNN processing chip. Moreover, another CNN processing chip is implemented with more analog convolutional operators to demonstrate the operation and structure of an example convolutional layer of a CNN model. Therefore, the proposed analog convolutional operator can be adapted in various CNN models as an alternative to digital counterparts.

Klasifikasi Hama Serangga pada Pertanian Menggunakan Metode Convolutional Neural Network

Insect pest attacks pose a serious threat that can potentially cause significant losses in agricultural production. Therefore, the effective recognition and control of insect pests are crucial for maintaining agricultural productivity and quality of yields. With the advancement of computer technology and artificial intelligence, computer technology can be utilized to automatically recognize images in object recognition, particularly for insect pest classification using the Convolutional Neural Network (CNN) method with the Xception architecture. CNN is one of the types of deep feed-forward artificial neural networks widely used in digital image analysis and can process data in the form of grid patterns. CNN consists of three types of layers: convolutional layer, pooling layer, and fully connected layer. The use of CNN in this research aims to facilitate the classification of insect pests. The CNN process involves stages of training, testing, and validation on insect pests to determine the classification of images of various insect pest species. This research utilizes 1363 image samples with 13 classes of insect pests. The training process of CNN involves several parameters such as batch size, number of epochs, learning rate, and optimizer. The experiment's results indicate that the best accuracy achieved by this model is 93.81% during the training phase and 81.75% during the validation phase. This demonstrates that the model successfully performs insect pest classification using the CNN method.

Two-Stage Deep Single-Image Super-Resolution With Multiple Blur Kernels for Internet of Things

Single image super-resolution (SISR) aims to reconstruct a high-resolution image from a single low-resolution (LR) image. Although convolutional neural network (CNN)-based SISR methods greatly enhance image restoration, they face critical challenges. First, SISR models using CNNs process image patches uniformly regardless of importance, causing spatial inefficiency in computation and representation. However, due to resource constraints, edge devices in the Internet of Things (IoT) cannot bear heavy computations or large memory storage. Second, most of the existing SISR methods are designed only for the widely adopted bicubic degradation and cannot handle LR images with arbitrary blur kernels, resulting in poor recovery performance. To address these issues, in this paper we propose a two-stage semantic and spatial deep super-resolution (SSDSR) model suitable for the IoT environment. The proposed SSDSR model is capable of handling a variety of blur kernels (e.g., isotropic Gaussian, motion and disk blur) by effectively using their prior information. Moreover, the Semantic Feature Extraction (SFE) module enables the proposed model to focus on key areas of LR images rather than treating all pixels equally, which significantly reduces the computational load. The semantic information from the SFE module and the spatial information from the Spatial Attention module are fused adaptively, allowing the proposed model to extract key information in LR images, thereby increasing the representation capacity of the CNN and improving image recovery. Compared with state-of-the-art SISR methods on benchmark datasets, the proposed SSDSR model demonstrates superior performance. When run in real time on an IoT edge device, our model exhibits high computational efficiency and excellent image quality.

Improving the Performance of CNN Accelerator Architecture under the Impact of Process Variations

Convolutional neural network (CNN) accelerators are popular specialized platforms for efficient CNN processing. As semiconductor manufacturing technology scales down to nano scale, process variation dramatically affects the chip’s quality. Process variation causes delay variation within the chip due to transistor parameter differences. CNN accelerators adopt a large number of processing elements (PEs) for parallel computing, which are highly susceptible to process variation effects. Fast CNN processing desires consistent performance among PEs; otherwise the processing speed is limited by the slowest PE within the chip. In this work, we first quantitatively model and analyze the impact of process variation on CNN accelerators’ operating frequency. We further analyze the utilization of CNN accelerators and the characteristics of CNN models. We then leverage the PE underutilization to propose a sub-matrix reformation mechanism and leverage the pixel similarity of images to propose a weight transfer technique. Both techniques are able to tolerate the low-frequency PEs and achieve performance improvement at chip level. Furthermore, a novel resilience-aware mapping technique that exploits the diversity in the importance of weights is also proposed to improve the performance. Evaluation results show that our techniques are able to achieve significant processing speed improvement with negligible accuracy loss.

Automatic snoring detection using a hybrid 1D–2D convolutional neural network

Snoring, as a prevalent symptom, seriously interferes with life quality of patients with sleep disordered breathing only (simple snorers), patients with obstructive sleep apnea (OSA) and their bed partners. Researches have shown that snoring could be used for screening and diagnosis of OSA. Therefore, accurate detection of snoring sounds from sleep respiratory audio at night has been one of the most important parts. Considered that the snoring is somewhat dangerously overlooked around the world, an automatic and high-precision snoring detection algorithm is required. In this work, we designed a non-contact data acquire equipment to record nocturnal sleep respiratory audio of subjects in their private bedrooms, and proposed a hybrid convolutional neural network (CNN) model for the automatic snore detection. This model consists of a one-dimensional (1D) CNN processing the original signal and a two-dimensional (2D) CNN representing images mapped by the visibility graph method. In our experiment, our algorithm achieves an average classification accuracy of 89.3%, an average sensitivity of 89.7%, an average specificity of 88.5%, and an average AUC of 0.947, which surpasses some state-of-the-art models trained on our data. In conclusion, our results indicate that the proposed method in this study could be effective and significance for massive screening of OSA patients in daily life. And our work provides an alternative framework for time series analysis.

COAC: Cross-Layer Optimization of Accelerator Configurability for Efficient CNN Processing

To achieve high accuracy, convolutional neural networks (CNNs) are increasingly growing in complexity and diversity in layer types and topologies. This makes it very challenging to efficiently deploy such networks on custom processor architectures for resource-scarce edge devices. Existing mapping exploration frameworks enable searching for the optimal execution schedules or hardware mappings of individual network layers, by optimizing each layer’s spatial (dataflow parallelization) and temporal unrolling (TU, execution order). However, these tools fail to take into account the overhead of supporting different unrolling schemes within a common hardware architecture. Using a fixed unrolling scheme across all layers is also not ideal, as this misses significant opportunities for energy and latency savings from optimizing the mapping of diverse layer types. A balanced approach assesses the right amount of mapping flexibility needed across target neural networks, while taking into account the overhead to support multiple unrollings. This article, therefore, presents cross-layer optimization of accelerator configurability (COAC), a cross-layer design space exploration and mapping framework to optimize the flexibility of neural processing architectures by balancing configurability overhead against resulting energy and latency savings for end-to-end inference. COAC does not only provide a systematical analysis of the architectural overhead in function of the supported spatial unrollings (SUs), but also builds an automated flow to find the best unrolling combination(s) for efficient end-to-end inference with limited hardware overhead. Results demonstrate that architectures with carefully optimized flexibility can achieve up to 38% energy-delay-product (EDP) savings for a set of six neural networks at the expense of a relative area increase of 9.5%.

Convolutional Neural Networks using FPGA-based Pipelining

In order to speed up convolutional neural networks (CNNs), this study gives a complete overview of the use of FPGA-based pipelining for hardware acceleration of CNNs. These days, most people use convolutional neural networks (CNNs) to perform computer vision tasks like picture categorization and object recognition. The processing and memory demands of CNNs, however, can be excessive, especially for real-time applications. In order to speed up CNNs, FPGA-based pipelining has emerged as a viable option thanks to its parallel processing capabilities and low power consumption. The examination describes the fundamentals of FPGA-based pipelining and the basic structure of convolutional neural networks (CNNs). The current best practises for developing pipelined accelerators for CNNs on FPGAs are then reviewed, covering topics like partitioning and pipelining. Area and power limits, memory needs, and latency considerations are only some of the difficulties and trade-offs discussed in the article. In addition, the survey evaluates and contrasts the various pipelined FPGA accelerators for CNNs in terms of performance, energy consumption, and resource utilisation. Future directions and potential research areas are also discussed in the paper, such as the use of approximate computing techniques, the integration of reconfigurable architectures with emerging memory technologies, and the exploration of hybrid architectures that combine FPGAs and other hardware accelerators. This survey was created to aid researchers and practitioners in developing efficient and effective hardware accelerators for neural networks by providing a thorough overview of current trends and issues in FPGA-based pipelining for CNNs.

A hybrid method for cutting tool RUL prediction based on CNN and multistage Wiener process using small sample data

Based on the multistage and nonlinear characteristics of cutting tool wear, a hybrid method for cutting tool remaining useful life (RUL) prediction based on convolutional neural network (CNN) and multistage Wiener process using small sample data is proposed in this paper. First, wavelet transform is used to analyse the vibration data collected for cutting tools in the time–frequency domain to obtain an initial image data set. A CCVAE (conditional variational autoencoder with CNN) network is built to expand the initial data set and solve the problem of unbalanced data in different stages of cutting tool wear. The expanded data is used as the input of the CNN to monitor the tool wear state and amount of wear. Then, a nonlinear multistage Wiener process is established to describe the cutting tool wear degradation process and achieve accurate RUL prediction for the cutting tool. Specifically, a nonlinear Wiener process model corresponding to different degradation stages based on the change points between the cutting tool wear states output by the CNN is established. Maximum likelihood estimation and Bayesian methods are used to estimate and update the parameters, respectively, and the RUL value and corresponding probability density function (PDF) are obtained under different wear conditions. Finally, through experimental research and comparative analysis, it is found that the multistage nonlinear Wiener model accurately simulates cutting tool wear degradation, which verifies the feasibility and performance of the method proposed in this paper.

ATTENTION BASED CONVOLUTIONAL NEURAL NETWORK FOR FACIAL EXPRESSION RECOGNITION

Many deep learning-based methods have been suggested in recent years to improve facial expression recognition performance, but they are unable to extract the small facial changes in skin textures, such as wrinkles and furrows, which indicate changes in expression. We put forward an attention mechanism-based Convolutional Neural Network (CNN) for face expression recognition to get around this issue. The attention module, classification module feature extraction module and the reconstruction module make up the architectures four components. The feature extraction module, which consists of two distinct CNN processing streams-one for raw images and the other for Local Binary Pattern (LBP) feature maps. The LBP features extract image texture data before capturing the minute facial movements, which can enhance network efficiency. The neural network can pay more attention to useful characteristics by using an attention mechanism. To improve the attention model and produce improved results, we combine LBP features and convolutional features. We use the CK+ and Oulu-CASIA datasets to test the proposed methodology. The experimental findings show that the suggested method is workable and efficient. KEYWORDS: Convolution neural network, Local binary pattern, Facial expression recognition.

Breast Cancer Classification Using Equivariance Transition in Group Convolutional Neural Networks

In computer vision, rotation equivariance and translation invariance are properties of a representation that preserve the geometric structure of a transformed input. These properties are achieved in Convolutional Neural Networks (CNNs) through data augmentation. However, achieving these properties remains a challenge. This is because CNNs are not equivariance under rotation. In this study, a novel deep neural network architecture combining a group convolutional neural network (G-CNN) built with a special Euclidean (SE2) motion group and discrete cosine transform (DCT) is proposed. The former is based on the group theory and uses SE2 to guarantee equivariance in 2D images, whereas the latter is used to encode and parameterize the model space. To restore and preserve the equivariance property of the transformed and convoluted images, the DCT was used as a rotation-invariant module. These combined techniques are employed to improve breast cancer classification and data efficiency in the CNNs processing pipeline. The developed model is tested on the rotated MNIST datasets to assess its performance. Finally, the model is applied to mammography images and achieved a high computational performance and improved inference generation in breast cancer classification with an accuracy of 94.84%.

Ocelli: Efficient Processing-in-Pixel Array Enabling Edge Inference of Ternary Neural Networks

Convolutional Neural Networks (CNNs), due to their recent successes, have gained lots of attention in various vision-based applications. They have proven to produce incredible results, especially on big data, that require high processing demands. However, CNN processing demands have limited their usage in embedded edge devices with constrained energy budgets and hardware. This paper proposes an efficient new architecture, namely Ocelli includes a ternary compute pixel (TCP) consisting of a CMOS-based pixel and a compute add-on. The proposed Ocelli architecture offers several features; (I) Because of the compute add-on, TCPs can produce ternary values (i.e., −1, 0, +1) regarding the light intensity as pixels’ inputs; (II) Ocelli realizes analog convolutions enabling low-precision ternary weight neural networks. Since the first layer’s convolution operations are the performance bottleneck of accelerators, Ocelli mitigates the overhead of analog buffers and analog-to-digital converters. Moreover, our design supports a zero-skipping scheme to further power reduction; (III) Ocelli exploits non-volatile magnetic RAMs to store CNN’s weights, which remarkably reduces the static power consumption; and finally, (IV) Ocelli has two modes, including sensing and processing. Once the object is detected, the architecture switches to the typical sensing mode to capture the image. Compared to the conventional pixels, it achieves an average 10% efficiency on its lane detection power consumption compared with existing edge detection algorithms. Moreover, considering different CNN workloads, our design shows more than 23% power efficiency over conventional designs, while it can achieve better accuracy.

Application of Convolutional Neural Network to Defect Diagnosis of Drill Bits

Drilling, one of the most used machining processes, has wide application in different industrial fields. Monitoring the system health and operation status of the drilling process is essential for maintaining production efficiency. In this study, a convolutional neural network (CNN), a deep-learning method, is applied to the defect diagnosis of drill bits. Four drill bits with different health conditions were used to drill holes in an aluminum block, and a vibration sensor collected the signals. Vibration spectrograms generated using short-time Fourier transform were applied to a 2D CNN algorithm, and they were then reconstructed into a 1D data set and applied to a 1D CNN algorithm. The input data size was reduced significantly compared to the raw vibration data after the data-reconstruction process. As a result, the 2D CNN process shows a diagnostic accuracy of 97.33%. On the other hand, the 1D CNN provides a diagnostic accuracy of 96.6%, but it only requires 2/3 of the computational time required by the 2D CNN.