Feed-forward Convolutional Neural Networks Research Articles

Decoding Stress with Computer Vision-Based Approach Using Audio Signals for Psychological Event Identification during COVID-19

Interpreting psychological events can be costly and quite complex. It is simple to translate such experiences into a person's spoken and nonverbal cues. The suggested model investigates a computer vision-based method for using an individual's audio signal to identify stressful psychological events. Different people's input speech signals are recorded and compared to the common questionnaire. A series of inquiries pertaining to the second stage of COVID-19 events are included in the questionnaire set. Through additional processing, these speech signals are converted into frequency components by means of the Fast Fourier transformation (FFT) method. A long short-term memory module processes each frequency component and produces temporal information from each frequency band. The features of speech signals are extracted into the temporal frames by this module. The VGG 16 algorithm is used to further classify each temporal frame into stress and un-stress classes. A classifier with 16 layers of architecture is called VGG 16. A feed-forward convolutional neural network called VGG 16 is used to divide the vast array of speech signal features into classes: stressed and unstressed. The proposed model attempts to recognize speech signals as stress indicators. A standard set of questionnaires with a series of interrogation-style questions has been used to develop the stress symptoms in an individual's mind. The audio signals generated by each person's responses are recorded and subsequently analyzed for stress and un-stress classes. The proposed model was able to identify stress in speech signals with 98% accuracy. The time and cost implications of the suggested model are relevant. Medical research is typically costly and time-consuming.LSTM; VGG 16; CNN model; data preprocessing; speech signal.

Image‐based characterization of flocculation processes through PLS inspired representation learning in convolutional neural networks

AbstractMonitoring of flocculation processes such as those used in downstream processing of a fermentation broth is essential for process control. One approach is to apply microscopic imaging combined with image analysis for characterizing the state of the process. In this work, we investigate and compare the use of supervised feedforward convolutional neural network (CNN) architectures to predict the process states from the image information and compare the results with the traditional alternative of characterizing flocs based on manually engineered image features guided by human expertise. From a well‐defined image data set representing six process states, the objective is to establish end‐to‐end classification models which are accurate but at the same time learn meaningful latent variable space representations. Specifically, we evaluate three different CNN architectures with varying degrees of regularization and compare results with logistic regression models based on inputs from two different traditional feature engineering methods. By applying global average pooling as a structural regularizer to the CNN architecture, we significantly improve the generalization performance in comparison with the classification accuracies of the traditional feature engineered models. Furthermore, we show that by imposing a projection to latent structures (PLS) like regularization framework onto the CNN, it can also learn a latent variable representation that mimics the features selected by human expertise.

Emergent human-like covert attention in feedforward convolutional neural networks

Covert attention allows the selection of locations or features of the visual scene without moving the eyes. Cues and contexts predictive of a target's location orient covert attention and improve perceptual performance. The performance benefits are widely attributed to theories of covert attention as a limited resource, zoom, spotlight, or weighting of visual information. However, such concepts are difficult to map to neuronal populations. We show that a feedforward convolutional neural network (CNN) trained on images to optimize target detection accuracy and with no explicit incorporation of an attention mechanism, a limited resource, or feedback connections learns to utilize cues and contexts in the three most prominent covert attention tasks (Posner cueing, set size effects in search, and contextual cueing) and predicts the cue/context influences on human accuracy. The CNN's cueing/context effects generalize across network training schemes, to peripheral and central pre-cues, discrimination tasks, and reaction time measures, and critically do not vary with reductions in network resources (size). The CNN shows comparable cueing/context effects to a model that optimally uses image information to make decisions (Bayesian ideal observer) but generalizes these effects to cue instances unseen during training. Together, the findings suggest that human-like behavioral signatures of covert attention in the three landmark paradigms might be an emergent property of task accuracy optimization in neuronal populations without positing limited attentional resources. The findings might explain recent behavioral results showing cueing and context effects across a variety of simple organisms with no neocortex, from archerfish to fruit flies.

LightNet: a novel lightweight convolutional network for brain tumor segmentation in healthcare.

Diagnosis, treatment planning, surveillance, and the monitoring of clinical trials for brain diseases all benefit greatly from neuroimaging-based tumor segmentation. Recently, Convolutional Neural Networks (CNNs) have demonstrated promising results in enhancing the efficiency of image-based brain tumor segmentation. Most current work on CNNs, however, is devoted to creating increasingly complicated convolution modules to improve performance, which in turn raises the computing cost of the model. This work proposes a simple and effective feed-forward CNN, LightNet (Light Network). Based on multi-path and multi-level, it replaces traditional convolutional methods with light operations, which reduces network parameters and redundant feature maps. In the up-sampling stage, a light channel attention module is added to achieve richer multi-scale and spatial semantic feature information extraction of brain tumor. The performance of the network is evaluated in the Multimodal Brain Tumor Segmentation Challenge (BraTS 2015) dataset, and results are presented here alongside other high-performing CNNs. Results show comparable accuracy with other methods but with increased efficiency, segmentation performance, and reduced redundancy and computational complexity. The result is a high-performing network with a balance between efficiency and accuracy, allowing, for example, better energy performance on mobile devices.

Deep Learning based Enhanced Adaptive Despeckling for Ultrasound Images

Considering the inadequacies observed in traditional medical ultrasound image de-speckling techniques, this research presents an innovative solution: a feedforward convolutional neural network (CNN) model coupled with an adaptive multi-exposure fusion framework. The study initiates by curating a specialized ultrasound image training dataset.It then proposes a multi-exposure fusion framework with adaptive enhancement factors to improve image quality for more efficient feature extraction. The proposed method trains a speckle model through the neural network and achieves the extraction of a de-speckled image. Experimental results unequivocally demonstrate the method's unparalleled effectiveness in speckle noise reduction within medical ultrasound images while concurrently preserving intricate image details, thus exemplifying its potential in clinical applications.

A reconstruction and convolution operations enabled variant vision transformer with gastroscopic images for automatic locating of polyps in Internet of Medical Things

Gastric polyps are an important cause of gastric disease. At present, the computer-aided diagnosis technology based on convolutional neural network (CNN) can automatically locate the position of polyps from the gastroscopic image, which improves the efficiency of doctors. However, due to the small polyp area in the gastroscopic image, the CNN-based method has a high rate of missed detection. To solve the above problems, in this work, we propose a reconstruction and convolution operations enabled variant vision transformer (RCVViT) to automatically locate the position of polyps in gastroscopic images. The RCVViT model uses the vision transformer model as a benchmark model. By using the self-attention mechanism, contextual information can be considered, and irregularly shaped polyps or polyps with small areas can be effectively detected. The feedforward neural network (FNN) and CNN are used to flatten each image patch data into a one-dimensional vector. The advantage of combining the FNN and CNN is that the local feature information and structural information of the polyp area are considered. In addition, we use an Internet of Medical Things (IoMT) platform to collect and analyze patients’ medical data to make timely diagnosis of patients’ diseases. Finally, our multiple experimental results on real gastroscopic datasets demonstrate the superiority of the RCVViT model.

Invited Session III: Neural network models of the visual system: Exploring the function of different forms of visual recurrence with artificial neural networks.

Behavioral studies suggest that recurrence in the visual system is important for processing degraded stimuli. There are two broad anatomical forms this recurrence can take, lateral or feedback, each with different assumed functions. I'll discuss work wherein I add four different kinds of recurrence-two of each anatomical form-to a feedforward convolutional neural network and find all forms capable of increasing the ability of the network to classify noisy digit images. By using several analysis tools frequently applied to neural data, the distinct strategies used by different networks were identified. The analyses used here can be applied to real neural recordings to identify the strategies at play in the brain. An analysis of an fMRI dataset weakly supports the predictive feedback model but points to a need for higher-resolution cross-regional data to understand recurrent visual processing.

Acoustic Scene Classification using Deep Fisher network

Acoustic Scene Classification (ASC) is the task of assigning a semantic label to an audio recording, based on the surrounding environment. In this work, a Fisher network is introduced for ASC. The proposed method mimics the working mechanism of a feed-forward Convolutional Neural Network (CNN) where, output of a layer is fed as an input to the succeeding layer. The Fisher network consists of a feature extraction step followed by a Fisher layer. The Fisher layer has three sub-layers, namely, Fisher Vector (FV) encoder, temporal pyramid and normalization layers along with feature reduction layer. Gammatone Time Cepstral Coefficients (GTCCs) and Mel-spectrograms are the features encoded as Fisher vector representation in FV encoder sub-layer. Temporal information of the Fisher vectors is retained using temporal pyramid sub-layer. After temporal pyramids are extracted from Fisher vectors, they are available as a feature vector. Irrelevant dimensions of the temporal pyramids are reduced further using Principal Component Analysis (PCA) in normalization and PCA sub-layers. The proposed model is evaluated on five DCASE datasets, TUT Urban Acoustic Scenes 2018 and Mobile, DCASE 2019 Acoustic Scene Classification Task 1(a) and Task 1(b), TAU Urban Acoustic Scenes 2020 datasets. The overall classification accuracy is 93%, 91%, 92%, 91% and 89% for TUT 2018, TUT Mobile 2018, DCASE Task 1(a) 2019, DCASE Task 1(b) 2019, and TAU Urban Acoustic Scenes 2020 datasets, respectively. The proposed model performed much better than the state-of-the-art ASC systems.

CNNParted: An open source framework for efficient Convolutional Neural Network inference partitioning in embedded systems

Applications such as autonomous driving or assistive robotics heavily rely on the usage of Deep Neural Networks. In particular, Convolutional Neural Networks (CNNs) provide precise and reliable results in image processing tasks like camera-based object detection or semantic segmentation. However, to achieve even better results, CNNs are becoming more and more complex. Deploying these networks in distributed embedded systems thereby imposes new challenges, due to additional constraints regarding performance and energy consumption in the near-sensor compute platforms, i.e. the sensor nodes. Processing all data in the central node, however, is disadvantageous since raw data of camera consumes large bandwidth and running CNN inference of multiple tasks requires certain performance. Moreover, sending raw data over the interconnect is not advisable for privacy reasons. Hence, offloading CNN workload to the sensor nodes in the system can lead to reduced traffic on the link and a higher level of data security.However, due to the limited hardware-resources on the sensor nodes, partitioning CNNs has to be done carefully to meet overall latency requirements and energy constraints. Therefore, we present CNNParted, an open-source framework for efficient, hardware-aware CNN inference partitioning targeting embedded AI applications. It automatically searches for potential partitioning points in the CNN to find a beneficial workload distribution between sensor nodes and a central edge node. Thereby, CNNParted not only analyzes the CNN architecture but also takes hardware components, such as dedicated hardware accelerators and memories, into consideration to evaluate inference partitioning regarding latency and energy consumption.Exemplary, we apply CNNParted to three commonly used feed forward CNNs in embedded systems. Thereby, the framework first searches for several potential partitioning points and then evaluates the latter regarding inference latency and energy consumption. Based on the results, beneficial partitioning points can be identified depending on the system constraints. Using the framework, we are able to find and evaluate 10 potential partitioning points for FCN ResNet-50, 13 partitioning points for GoogLeNet, and 8 partitioning points for SqueezeNet V1.1 within 520 s, 330 s, and 140 s, respectively, on an AMD EPYC 7702P running 8 concurrent threads. For GoogLeNet, we determine two partitioning points that provide a good trade-off between required bandwidth, latency and energy consumption. We also provide insights into further interesting findings that can be derived from the evaluation results.

Prediction of Sulfur Removal from Iron Concentrate Using Column Flotation Froth Features: Comparison of k-Means Clustering, Regression, Backpropagation Neural Network, and Convolutional Neural Network

Froth feature extraction plays a significant role in the monitoring and control of the flotation process. Image-based soft sensors have received a great deal of interest in the flotation process due to their low-cost and non-intrusive properties. This study proposes data-driven soft sensor models based on froth images to predict the key performance indicators of the flotation process. The ability of multiple linear regression (MLR), the backpropagation neural network (BPNN), the k-means clustering algorithm, and the convolutional neural network (CNN) to predict the amount of sulfur removal from iron ore concentrate in the column flotation process was examined. A total of 99 experimental results were used to develop the predictive models. Extracted froth features including color, bubble shape and size, texture, stability, and velocity were used to train the traditional predictive models, whereas in the CNN model the froth images were directly fed into the model. The results comparison indicated that the three-layered feedforward NN model (17-10-1 topology) and CNN model provided better predictions than the MLR and k-means algorithm. The BPNN model displayed a correlation coefficient of 0.97 and a root mean square error of 4.84% between the actual data and network output for both training and the testing datasets. The error percentages of the CNN, BPNN, MLR and k-means models were 10, 11, 15 and 18%, respectively. This study can become a key technical support for the application of intelligent models in the control of the operational variables for the flotation process used to desulfurize iron concentrate.

Robust lane line segmentation based on group feature enhancement

Training a robust deep model for lane line segmentation is challenging due to the complex and changeable scenarios in autonomous driving, such as poor lighting conditions, ambiguous lane lines, and inclement weather. Without further extraction of effective deep features, such models often fail when faced with a large number of unstructured lane lines. In this article, we propose a grouped feature enhancement(GFE) method, ENet-GFE that uses the similarity between feature channels to further extract effective features while reducing noise. We group feature channels, evaluate the importance of each sub-feature in the channel and space, and learn within each group based on the relevance and similarity of the content to obtain more effective output features by using the semantic similarity between the lane line feature information and feature channels. GFE can be easily incorporated into a feedforward convolutional neural network(CNN) without adding too much computing overhead. We validated the proposed method on the widely used CULane lane line segmentation dataset and the latest autonomous driving dataset, OpenMPD. In experiments, ENet-GFE exhibited the most advanced performance. Currently, it is one of the lightest models.

Deep Learning Models for Time-History Prediction of Vehicle-Induced Bridge Responses: A Comparative Study

Time-history responses of the bridge induced by the moving vehicle provide crucial information for bridge design, operation, maintenance, etc. As inspired by this, this work attempts to provide a new paradigm for vehicle–bridge interaction (VBI) by highlighting the comparison of different deep learning algorithms applied to the prediction of time-history responses of the bridge under vehicular loads. Particularly, three deep learning architectures with few and measurable input features developed by using fully-connected feedforward neural network, long short-term memory (LSTM) network, and convolutional neural network (CNN) are proposed on the basis of the governing equation of bridge vibrations. Three VBI systems with various vehicle models are developed and further validated to produce reliable training data. To examine the accuracy of the predictive models, two advanced metrics are exploited for time-history estimate. Moreover, the proposed deep learning models are comprehensively investigated through a parametric study on the influential factors associated with the VBI system and network architecture. The results show that deep feedforward neural network (DFNN), LSTM network, and CNN can be applied in VBI analysis to estimate the bridge time-history response. The three neural networks have comparable prediction accuracies. When considering the irregularity excitation, CNN is found to be the most efficient predictive model, while DFNN needs the least training time under perfect bridge surface condition.

Recurrent Connections in the Primate Ventral Visual Stream Mediate a Trade-Off Between Task Performance and Network Size During Core Object Recognition.

The computational role of the abundant feedback connections in the ventral visual stream is unclear, enabling humans and nonhuman primates to effortlessly recognize objects across a multitude of viewing conditions. Prior studies have augmented feedforward convolutional neural networks (CNNs) with recurrent connections to study their role in visual processing; however, often these recurrent networks are optimized directly on neural data or the comparative metrics used are undefined for standard feedforward networks that lack these connections. In this work, we develop task-optimized convolutional recurrent (ConvRNN) network models that more correctly mimic the timing and gross neuroanatomy of the ventral pathway. Properly chosen intermediate-depth ConvRNN circuit architectures, which incorporate mechanisms of feedforward bypassing and recurrent gating, can achieve high performance on a core recognition task, comparable to that of much deeper feedforward networks. We then develop methods that allow us to compare both CNNs and ConvRNNs to finely grained measurements of primate categorization behavior and neural response trajectories across thousands of stimuli. We find that high-performing ConvRNNs provide a better match to these data than feedforward networks of any depth, predicting the precise timings at which each stimulus is behaviorally decoded from neural activation patterns. Moreover, these ConvRNN circuits consistently produce quantitatively accurate predictions of neural dynamics from V4 and IT across the entire stimulus presentation. In fact, we find that the highest-performing ConvRNNs, which best match neural and behavioral data, also achieve a strong Pareto trade-off between task performance and overall network size. Taken together, our results suggest the functional purpose of recurrence in the ventral pathway is to fit a high-performing network in cortex, attaining computational power through temporal rather than spatial complexity.

A FeedForward–Convolutional Neural Network to Detect Low-Rate DoS in IoT

The lack of standardization and the heterogeneous nature of the Internet of Things (IoT) has exacerbated the issue of security and privacy. In literature, to improve security at the network layer of the IoT architecture, the possibility of using Software-Defined Networking (SDN) was explored. SDN is also plagued by network threats that affect conventional networks. One such threat to a network is the Low-Rate Denial of Service (LR DoS) attack, where the attacker sends precise traffic bursts that force a TCP flow to enter a retransmission timeout state. LR DoS attacks are difficult to detect as their attack signature is similar to benign network traffic. The existing AI-based detection algorithms in the literature are signature-based, and their efficacy in detecting unknown LR DoS attacks was not explored. In this work, an AI-based anomaly detection scheme called FeedForward–Convolutional Neural Network (FFCNN) is proposed to detect LR DoS attacks in IoT-SDN. The Canadian Institute of Cybersecurity Denial of Service 2017 (CIC DoS 2017) dataset is used for the study. An iterative wrapper-based feature selection using Support Vector Machine (SVM) is used to derive the significant features required for detection. The performance of FFCNN is compared to the machine learning algorithms-J48, Random Forest, Random Tree, REP Tree, SVM, and Multi-Layer Perceptron (MLP). The performance of the models is measured using the metrics accuracy, precision, recall, F1 score, detection time per flow, and ROC curves. The empirical analysis shows that FFCNN outperforms other machine learning algorithms on all metrics.

Identifying specular highlights: Insights from deep learning.

Specular highlights are the most important image feature for surface gloss perception. Yet, recognizing whether a bright patch in an image is due to specular reflection or some other cause (e.g., texture marking) is challenging, and it remains unclear how the visual system reliably identifies highlights. There is currently no image-computable model that emulates human highlight identification, so here we sought to develop a neural network that reproduces observers’ characteristic successes and failures. We rendered 179,085 images of glossy, undulating, textured surfaces. Given such images as input, a feedforward convolutional neural network was trained to output an image containing only the specular reflectance component. Participants viewed such images and reported whether or not specific pixels were highlights. The queried pixels were carefully selected to distinguish between ground truth and a simple thresholding of image intensity. The neural network outperformed the simple thresholding model—and ground truth—at predicting human responses. We then used a genetic algorithm to selectively delete connections within the neural network to identify variants of the network that approximated human judgments even more closely. The best resulting network shared 68% of the variance with human judgments—more than the unpruned network. As a first step toward interpreting the network, we then used representational similarity analysis to compare its inner representations to a wide variety of hand-engineered image features. We find that the network learns representations that are similar not only to directly image-computable predictors but also to more complex predictors such as intrinsic or geometric factors, as well as some indications of photo-geometrical constraints learned by the network. However, our network fails to replicate human response patterns to violations of photo-geometric constraints (rotated highlights) as described by other authors.

Multiscale Brain-Like Neural Network for Saliency Prediction on Omnidirectional Images

Current top-performing saliency prediction methods of omnidirectional images (ODIs) depend on deep feedforward convolutional neural networks (CNNs), benefiting from their powerful multiscale representation ability. Although these methods adopt deep feedforward CNNs to achieve superb performance in saliency prediction task, they have the following limitations: 1) these deep feedforward CNNs are difficult to map to ventral stream structure of the brain visual system due to their vast number of layers and missing biologically important connections, such as recurrence and 2) most deep feedforward CNNs represent the multiscale features in a layerwise manner. To tackle these issues, models that could learn multiscale features yet share the similarities with human brain are needed. In this article, we propose a novel multiscale brain-like network (MBN) model to predict saliency of head fixations on ODIs. Specifically, our proposed model consists of two major modules: 1) a brain-like CORnet-S module and 2) a multiscale feature extraction module. The CORnet-S module is a lightweight backbone network with four anatomically mapped areas (V1, V2, V4, and IT) and it can simulate the visual processing mechanism of ventral visual stream in the human brain. The multiscale feature extraction module is inspired by the multiscale brain structure, which represents multiscale features at a granular level and increases the range of receptive fields for each network layer. Extensive experiments and ablation studies conducted on two major benchmarks demonstrate the superiority of the proposed MBN model over the state-of-the-art methods.

Mix-loss trained bias-removed blind image denoising network

We studied the modern deep convolutional neural networks used for image denoising, where RGB input images are transformed into RGB output images via feed-forward convolutional neural networks that use a loss defined in the RGB color space. Considering the difference between human visual perception and objective evaluation metrics such as PSNR or SSIM, we propose a data augmentation technique and demonstrate that it is equivalent to defining a perceptual loss function. We trained a network based on this and obtained visually pleasing denoised results. We also combine an unsupervised design and the bias-free network to deal with the overfitting due to the absence of clean images, and improve performance when the noise level exceeds the training range.

Mobile Health App for Adolescents: Motion Sensor Data and Deep Learning Technique to Examine the Relationship between Obesity and Walking Patterns

With the prevalence of obesity in adolescents, and its long-term influence on their overall health, there is a large body of research exploring better ways to reduce the rate of obesity. A traditional way of maintaining an adequate body mass index (BMI), calculated by measuring the weight and height of an individual, is no longer enough, and we are in need of a better health care tool. Therefore, the current research proposes an easier method that offers instant and real-time feedback to the users from the data collected from the motion sensors of a smartphone. The study utilized the mHealth application to identify participants presenting the walking movements of the high BMI group. Using the feedforward deep learning models and convolutional neural network models, the study was able to distinguish the walking movements between nonobese and obese groups, at a rate of 90.5%. The research highlights the potential use of smartphones and suggests the mHealth application as a way to monitor individual health.

Deep CNN with Hybrid Binary Local Search and Particle Swarm Optimizer for Exudates Classification from Fundus Images.

Diabetic retinopathy is a chronic condition that causes vision loss if not detected early. In the early stage, it can be diagnosed with the aid of exudates which are called lesions. However, it is arduous to detect the exudate lesion due to the availability of blood vessels and other distractions. To tackle these issues, we proposed a novel exudates classification from the fundus image known as hybrid convolutional neural network (CNN)-based binary local search optimizer-based particle swarm optimization algorithm. The proposed method from this paper exploits image augmentation to enlarge the fundus image to the required size without losing any features. The features from the resized fundus images are extracted as a feature vector and fed into the feed-forward CNN as the input. Henceforth, it classifies the exudates from the fundus image. Further, the hyperparameters are optimized to reduce the computational complexities by utilization of binary local search optimizer (BLSO) and particle swarm optimization (PSO). The experimental analysis is conducted on the public ROC and real-time ARA400 datasets and compared with the state-of-art works such as support vector machine classifiers, multi-modal/multi-scale, random forest, and CNN for the performance metrics. The classification accuracy is high for the proposed work, and thus, our proposed outperforms all the other approaches.

Have We Solved Edge Detection? A Review of State-of-the-Art Datasets and DNN Based Techniques

Recent Deep Neural Networks (DNNs) based edge detection methods claim to beat human performance on small scale datasets like BSDS500. But is the problem of edge detection really solved? To answer this question, we review the existing dataset limitations as well as the generalization capabilities of the proposed architectures. To this end, we develop a Synthetic Textured Masks Dataset (STMD) that contains 28,000 gray scale images. The performance of several edge detection methods is severely degraded on STMD. To further validate these results we propose a baseline Single Scale Feed Forward Edge Detector (SFED), which is a simple 9-layer feed-forward convolutional neural network with no pooling layers. The performance of SFED is better than most state-of-the-art architectures on BSDS500 and is superior to all the compared architectures on STMD. These results show that most of the architectural advancements of existing architectures are at the cost of generalizability where if we change the dataset set distribution (both training and testset), the performance become significantly degraded and therefore the problem of edge detection is still far away from being solved. There are also severe limitations of existing datasets in the field, and STMD provides a framework for designing and testing better edge detection architectures for novel application areas, such as, medical imaging and self-driving cars.