Computational Neural Network Research Articles

Neural Network and Cloud Computing for Predicting ECG Waves from PPG Readings

In this paper, we have recently created self-driving cars and self-parking systems in human-friendly cars that can provide high safety and high convenience functions by recognizing the internal and external situations of automobiles in real time by incorporating next-generation electronics, information communication, and function control technologies. And with the development of connected cars, the ITS (Intelligent Transportation Systems) market is expected to grow rapidly. Intelligent Transportation System (ITS) is an intelligent transportation system that incorporates technologies such as electronics, information, communication, and control into the transportation system, and aims to implement a next-generation transportation system suitable for the information society. By combining the technologies of connected cars and Internet of Things with software features and operating systems, future cars will serve as a service platform to connect the surrounding infrastructure on their own. This study creates a research methodology based on the Enhanced Security Model in Self-Driving Cars model. As for the types of attacks, Availability Attack, Man in the Middle Attack, Imperial Password Use, and Use Inclusive Access Control attack defense methodology are used. Along with the commercialization of 5G, various service models using advanced technologies such as autonomous vehicles, traffic information sharing systems using IoT, and AI-based mobility services are also appearing, and the growth of smart transportation is accelerating. Therefore, research was conducted to defend against hacking based on vulnerabilities of smart cars based on artificial intelligence blockchain.

Photonic-aware neural networks

Photonics-based neural networks promise to outperform electronic counterparts, accelerating neural network computations while reducing power consumption and footprint. However, these solutions suffer from physical layer constraints arising from the underlying analog photonic hardware, impacting the resolution of computations (in terms of effective number of bits), requiring the use of positive-valued inputs, and imposing limitations in the fan-in and in the size of convolutional kernels. To abstract these constraints, in this paper we introduce the concept of Photonic-Aware Neural Network (PANN) architectures, i.e., deep neural network models aware of the photonic hardware constraints. Then, we devise PANN training schemes resorting to quantization strategies aimed to obtain the required neural network parameters in the fixed-point domain, compliant with the limited resolution of the underlying hardware. We finally carry out extensive simulations exploiting PANNs in image classification tasks on well-known datasets (MNIST, Fashion-MNIST, and Cifar-10) with varying bitwidths (i.e., 2, 4, and 6 bits). We consider two kernel sizes and two pooling schemes for each PANN model, exploiting 2times 2 and 3times 3 convolutional kernels, and max and average pooling, the latter more amenable to an optical implementation. 3times 3 kernels perform better than 2times 2 counterparts, while max and average pooling provide comparable results, with the latter performing better on MNIST and Cifar-10. The accuracy degradation due to the photonic hardware constraints is quite limited, especially on MNIST and Fashion-MNIST, demonstrating the feasibility of PANN approaches on computer vision tasks.

Assessment for Different Neural Networks with FeatureSelection in Classification Issue.

In general, the investigation of NN (neural network) computing systems requires the management of a significant number of simultaneous distinct algorithms, such as parallel computing, fault tolerance, classification, and data optimization. Supervised learning for NN originally comes from certain parameters, such as self-revised learning, input learning datasets, and multiple second learning processes. Specifically, the operation continues to adjust the NN connection synapses’ weight to achieve a self-learning computer system. The current article is aimed at developing the CC (correlation coefficient) assignment scheme adaptively joint with the FS (feature selection) categories to pursue the solutions utilized in solving the restrictions of NN computing. The NN computing system is expected to solve high-dimensional data, data overfitting, and strict FS problems. Hence, the Fruits-360 dataset is applied in the current article, that is, the variety of fruits, the sameness of color, and the differences in appearance features are utilized to examine the NN system accuracy, performance, and loss rate. Accordingly, there are 120 different kinds with a total of 20,860 fruit image datasets collected from AlexNet, GoogLeNet, and ResNet101, which were implemented in the CC assignment scheme proposed in this article. The results are employed to verify that the accuracy rate can be improved by reducing strict FS. Finally, the results of accuracy rate from the training held for the three NN frameworks are discussed. It was discovered that the GoogLeNet model presented the most significant FS performance. The demonstrated outcomes validate that the proposed CC assignment schemes are absolutely worthwhile in designing and choosing an NN training model for feature discrimination. From the simulation results, it has been observed that the FS-based CC assignment improves the accurate rate of recognition compared to the existing state-of-the-art approaches.

Photoenhanced Electroresistance at Dislocation-Mediated Phase Boundary

Ferroelectric tunneling junctions have attracted intensive research interest due to their potential applications in high-density data storage and neural network computing. However, the prerequisite of an ultrathin ferroelectric tunneling barrier makes it a great challenge to simultaneously implement the robust polarization and negligible leakage current in a ferroelectric thin film, both of which are significant for ferroelectric tunneling junctions with reliable operating performance. Here, we observe a large tunneling electroresistance effect of ∼1.0 × 104% across the BiFeO3 nanoisland edge, where the intrinsic ferroelectric polarization of the nanoisland makes a major contribution to tuning the barrier height. This phenomenon is beneficial from the artificially designed tunneling barrier between the nanoscale top electrode and the inclined conducting phase boundary, which is located between the rhombohedral-island and tetragonal-film matrix and arranged with the dislocation array. More significantly, the tunneling electroresistance effect is further improved to ∼1.6 × 104% by the introduction of photoinduced carriers, which are separated by the flexoelectric field arising from the dislocations.

Minimum signed digit approximation for faster and more efficient convolutional neural network computation on embedded devices

In the era of smart Internet-of-Things, convolutional neural network (CNN) models with low computational overhead are crucial for low-latency applications in resource-constrained embedded devices. The performance and efficiency of multiplication operations play a vital role in accelerating and optimizing CNN computation. In this article, we propose the MA4C technique, which reduces CNN computation overhead by converting a pretrained CNN’s parameters into approximated minimum signed digit (MSD) representations. MSD representation contains fewer non-zero digits on average compared to the binary representation of a number. The proposed scheme approximates the MSD representations by only considering a specified number of most significant digits. The MA4C technique reduces the computational complexity of multipliers by reducing the number of partial sums. The proposed MSD approximation was applied on various DNN models, and their performance was analyzed for different datasets, varying CNN depth, and network configuration. Implementation of our proposed method on FPGA reduced the logic circuits and multiplier latency by up to 4.2× and 1.2× respectively compared to an 8-bit Booth multiplier for most CNN models.

Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study.

BackgroundAutomated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces.ObjectiveWe designed a strategy to gamify the collection and labeling of child emotion–enriched images to boost the performance of automatic child emotion recognition models to a level closer to what will be needed for digital health care approaches.MethodsWe leveraged our prototype therapeutic smartphone game, GuessWhat, which was designed in large part for children with developmental and behavioral conditions, to gamify the secure collection of video data of children expressing a variety of emotions prompted by the game. Independently, we created a secure web interface to gamify the human labeling effort, called HollywoodSquares, tailored for use by any qualified labeler. We gathered and labeled 2155 videos, 39,968 emotion frames, and 106,001 labels on all images. With this drastically expanded pediatric emotion–centric database (>30 times larger than existing public pediatric emotion data sets), we trained a convolutional neural network (CNN) computer vision classifier of happy, sad, surprised, fearful, angry, disgust, and neutral expressions evoked by children.ResultsThe classifier achieved a 66.9% balanced accuracy and 67.4% F1-score on the entirety of the Child Affective Facial Expression (CAFE) as well as a 79.1% balanced accuracy and 78% F1-score on CAFE Subset A, a subset containing at least 60% human agreement on emotions labels. This performance is at least 10% higher than all previously developed classifiers evaluated against CAFE, the best of which reached a 56% balanced accuracy even when combining “anger” and “disgust” into a single class.ConclusionsThis work validates that mobile games designed for pediatric therapies can generate high volumes of domain-relevant data sets to train state-of-the-art classifiers to perform tasks helpful to precision health efforts.

Resistance Drift-Reduced Multilevel Storage and Neural Network Computing in Chalcogenide Phase Change Memories by Bipolar Operation

Phase change materials, which has been focused on the non-volatile memory field, show the possibility to carry out data storage and computing in the same physical location. However, the resistance drift behavior of phase change memory has been a huge barrier not only to traditional binary memory application for a long time, but to multi-level storage and therefore the neural network computing. Here, a bipolar programming scheme is exploited to achieve drift-reduced intermediate states and convolutional neural network (CNN) computations in Ge <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> Sb <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> Te <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sub> (GST) based memory cells. Experiments show that the resistance drift phenomena under bipolar programming have been reduced. Furthermore, the impact of bipolar operation on CNN for inference is investigated. This work provides effective means for implementing phase change neuromorphic processor with enhanced stability.

Rubik: A Hierarchical Architecture for Efficient Graph Neural Network Training

The graph convolutional network (GCN) emerges as a promising direction to learn the inductive representation in graph data commonly used in widespread applications, such as E-commerce, social networks, and knowledge graphs. However, learning from graphs is nontrivial because of its mixed computation model involving both graph analytics and neural network computing. To this end, we decompose the GCN learning into two hierarchical paradigms: 1) graph-level and 2) node-level computing. Such a hierarchical paradigm facilitates the software and hardware accelerations for GCN learning. We propose a lightweight graph reordering methodology, incorporated with a GCN accelerator architecture that equips a customized cache design to fully utilize the graph-level data reuse. We also propose a mapping methodology aware of data reuse and task-level parallelism to handle various graphs inputs effectively. The results show that Rubik accelerator design improves energy efficiency by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$26.3\times $ </tex-math></inline-formula> – <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1375.2\times $ </tex-math></inline-formula> than GPU platforms across different datasets and GCN models.

Accurate Machine-learning Atmospheric Retrieval via a Neural-network Surrogate Model for Radiative Transfer

Atmospheric retrieval determines the properties of an atmosphere based on its measured spectrum. The low signal-to-noise ratios of exoplanet observations require a Bayesian approach to determine posterior probability distributions of each model parameter, given observed spectra. This inference is computationally expensive, as it requires many executions of a costly radiative transfer (RT) simulation for each set of sampled model parameters. Machine learning (ML) has recently been shown to provide a significant reduction in runtime for retrievals, mainly by training inverse ML models that predict parameter distributions, given observed spectra, albeit with reduced posterior accuracy. Here we present a novel approach to retrieval by training a forward ML surrogate model that predicts spectra given model parameters, providing a fast approximate RT simulation that can be used in a conventional Bayesian retrieval framework without significant loss of accuracy. We demonstrate our method on the emission spectrum of HD 189733 b and find good agreement with a traditional retrieval from the Bayesian Atmospheric Radiative Transfer (BART) code (Bhattacharyya coefficients of 0.9843–0.9972, with a mean of 0.9925, between 1D marginalized posteriors). This accuracy comes while still offering significant speed enhancements over traditional RT, albeit not as much as ML methods with lower posterior accuracy. Our method is ∼9× faster per parallel chain than BART when run on an AMD EPYC 7402P central processing unit (CPU). Neural-network computation using an NVIDIA Titan Xp graphics processing unit is 90×–180× faster per chain than BART on that CPU.

GA-based fuzzy neural network control for ROV

Abstract Aiming at the problem of uncertainty and static instability of underwater robot system, a motion control algorithm of underwater robot based on fuzzy neural network optimized by genetic algorithm is proposed in this paper. The algorithm makes use of the characteristics of neural network which can predict and approximate the target value well and the complementarity of fuzzy function to neural network, and uses genetic algorithm to optimize the parameters of fuzzy neural network membership function, so as to reduce the computation of neural network. The experimental results show that the controller has good robustness and enhances the stability of the control system.

TCTAP A-053 Multilayer Convolutional Neural Network Computer Vision Is Effective in Automatic OCT Image Processing.

TCTAP A-053 Multilayer Convolutional Neural Network Computer Vision Is Effective in Automatic OCT Image Processing.

An Effective Algorithm to Improve Recommender Systems using Evolutionary Computation Algorithms and Neural Network

The growing Internet access and easy access to it have resulted in a significant increase in e-content, which, along with many benefits, has caused problems for users. Internet users simply cannot find the content they need from this massive amount of data. Users are faced with a lot of suggestions for choosing goods, buying items, selecting music and videos, and more. Advantage systems can be used to overcome these problems. Today, with the spread of people’s use of cyberspace, such as web sites and social networks, and increasing the need for conscious and clever selection of people, recommender systems has been extensively investigated. Although the neural network can identify the connections between the inputs and outputs of a dataset, but in order to achieve the proper performance of the neural network, a proper structure should be considered. We will use the mantle algorithm to determine this structure. The mantle algorithm is a form of traditional genetic algorithm that uses local search to reduce the time to achieve optimal response. Genetic algorithms are created to search across the search space, while the local search, the neighborhood of the neighborhood, finds every response found by the genetic algorithm to find better answers. This algorithm seeks to find the optimal values for the parameters of the neural network method, so optimal solutions of the memetic algorithm is considered to be used to set parameters for the neural network method. The results of this study show the desirable performance of the proposed approach in this study.

Analysis of Comprehensive Artificial Neural Network Computer Media Aided Construction of Economic Forecasting Model

As a complex system engineering, economic forecasting involves many fields and disciplines, and its accuracy largely depends on the selected forecasting method and forecasting model. The construction of the traditional economic prediction model mainly depends on the theories of economic statistics and metrology, which cannot meet the prediction needs of the new era. ANN model has a more flexible nonlinear quality and learning mode so that it can effectively adapt to the prediction scenario of a nonlinear economic system. ANN is an abstract model of the human brain neuronal network based on the information-processing perspective and composed of different networks to achieve a specific representation of logical strategies. ANN’s self-learning function, associative storage function, and the ability to find optimal solutions at high speed have led to its in-depth application and research in an increasing number of fields. In addition, ANN research work has continued to advance, has made great progress in many fields represented by economic forecasting, has successfully solved many practical problems, showing a more intelligent adaptation in the field of economics, and therefore has an important research value. Based on this, this paper first analyzes the economic system prediction, then studies the ANN medium model and learning algorithm, and finally verifies the effectiveness of the ANN economic prediction model integrating computer medium.

Fine Geological Modeling of Complex Fault Block Reservoir Based on Deep Learning

Nowadays, people’s demand for underground mineral resources is increasing, and geological disasters have occurred frequently in recent years. Geological disasters refer to geological effects or geological phenomena that are formed under the action of natural or man-made factors, causing loss of human life and property, and damage to the environment; such as landslides, collapses, mudslides, and ground subsidence. Under such a background, people must accelerate the exploration of complex geological structures. This paper is aimed at using the methods and concepts of deep reinforcement learning. Deep learning is to learn the inherent laws and representation levels of sample data. The information obtained during these learning processes is of great help to the interpretation of data such as text and images. In this way, the fine geology of complex fault-block reservoirs is modeled and studied. Geological structures and phenomena are discussed through convolutional neural network models and computer techniques. At the same time, the multitask bird recognition network is used to extract and classify geological images, so as to construct geological model maps with different spatial structures. Finally, the quality of the fault reconstruction model, the calculation of reservoir geological simulation reserves, and the evaluation of the water injection development effect of complex fault blocks are analyzed. In the evaluation of the development effect of water injection in complex fault blocks, comparing the relationship curve between the actual comprehensive water content and the oil recovery factor with the standard curve, the comprehensive water content of the initial block increased rapidly. Through timely and dynamic water allocation and comprehensive management, the water cut rising speed is controlled. The current comprehensive water cut of the reservoir is between 60% and 80%, the actual curve is between 25% and 35%, and the estimated waterflooding recovery is about 30%.

History Dependence in a Chemical Reaction Network Enables Dynamic Switching.

This work describes an enzymatic autocatalytic network capable of dynamic switching under out-of-equilibrium conditions. The network, wherein a molecular fuel (trypsinogen) and an inhibitor (soybean trypsin inhibitor) compete for a catalyst (trypsin), is kept from reaching equilibria using a continuous flow stirred tank reactor. A so-called 'linear inhibition sweep' is developed (i.e., a molecular analogue of linear sweep voltammetry) to intentionally perturb the competition between autocatalysis and inhibition, and used to demonstrate that a simple molecular system, comprising only three components, is already capable of a variety of essential neuromorphic behaviors (hysteresis, synchronization, resonance, and adaptation). This research provides the first steps in the development of a strategy that uses the principles in systems chemistry to transform chemical reaction networks into platforms capable of neural network computing.

A Customized NoC Architecture to Enable Highly Localized Computing-on-the-Move DNN Dataflow

The ever-increasing computation complexity of fast-growing Deep Neural Networks (DNNs) has requested new computing paradigms to overcome the memory wall in conventional Von Neumann computing architectures. The emerging Computing-In-Memory (CIM) architecture has been a promising candidate to accelerate neural network computing. However, data movement between CIM arrays may still dominate the total power consumption in conventional designs. This brief proposes a flexible CIM processor architecture named Domino and “Computing-On-the-Move” (COM) dataflow, to enable stream computing and local data access to significantly reduce data movement energy. Meanwhile, Domino employs customized distributed instruction scheduling within Network-on-Chip (NoC) to implement inter-memory computing and attain mapping flexibility. The evaluation with prevailing DNN models shows that Domino achieves 1.77-to- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.37\times $ </tex-math></inline-formula> power efficiency over several state-of-the-art CIM accelerators and improves the throughput by 1.28-to- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$13.16\times $ </tex-math></inline-formula> .

A progressive genetic-based neural architecture search

PurposeRecently, more and more attention has been put forth on the application and deep learning, due to the widespread practicability of neural network computation. The purpose of this paper is developing an effective algorithm to automatically discover the optimal neural network architecture for several real applications.Design/methodology/approachThe author proposes a novel algorithm, namely, progressive genetic-based neural architecture search (PG-NAS), as a solution to efficiently find the optimal neural network structure for given data. PG-NAS also employs several operations to effectively shrink the search space to reduce the computation cost and improve the accuracy validation.FindingsThe proposed PG-NAS could be utilized on several tasks for discovering the optimal network structure. The author reduces the demand of manual settings when implementing artificial intelligence (AI) models; hence, PG-NAS requires less human intervention than traditional machine learning. The average and top-1 metrics, such as error, loss and accuracy, are used to measure the discovered neural architectures of the proposed model over all baselines. The experimental results show that, with several real datasets, the proposed PG-NAS model consistently outperforms the state-of-the-art models in all metrics.Originality/valueGenerally, the size and the complexity of the search space for the neural network dominates the performance of computation time and resources. In this study, PG-NAS utilizes genetic operations to effectively generate the compact candidate set, i.e. fewer combinations need to be generated when constructing the candidate set. Moreover, by the proposed selector in PG-NAS, the non-promising network structure could be significantly pruned off. In addition, the accuracy derivation of each combination in the candidate set is also a performance bottleneck. The author develops a predictor network to efficiently estimate the accuracy to avoid the time-consuming derivation. The learning of the prediction process is also adjusted dynamically; this adaptive learning of the predictor could capture the pattern of training data effectively and efficiently. Furthermore, the proposed PG-NAS algorithm is applied on several real datasets to show its practicability and scalability.

A Multifault‐Tolerant Training Scheme for Nonideal Memristive Neural Networks

Memristor crossbar is extensively investigated as an energy‐efficient accelerator for neural network (NN) computations. However, hardware implementation of NNs using realistic memristors is challenging due to the ubiquity of faults (mainly classified into hard and soft faults) in memristors. Herein, a hardware‐friendly, low‐power multifault‐tolerant training (MFTT) scheme capable of addressing both hard and soft faults simultaneously for memristive NNs is proposed. The MFTT scheme consists of multifault detection, targeted weight pruning, and in situ training with the Manhattan update rule. Specifically, multifault detection is first conducted to detect both hard and large soft faults. The detected faulty weights are subsequently pruned to prevent them from disturbing the NN. The sparsified NN after pruning is in situ trained so that the hard and large soft faults can be effectively tolerated using the sparsity and self‐adaptivity of NNs. In addition, the remaining small soft faults can be well tolerated by the Manhattan update rule. Experimentally, MFTT demonstrates the lowest accuracy losses among several representative fault‐tolerant schemes not only in the hard fault‐only (when the hard fault ratio exceeds 10%) and soft fault‐only (when the soft faults are large) cases, but also in the case where both types of faults coexist.

A 15-Gbps BiCMOS XNOR gate for fast recognition of COVID-19 in binarized neural networks

The COVID-19 pandemic is spreading around the world causing more than 177 million cases and over 3.8 million deaths according to the European Centre for Disease Prevention and Control. The virus has devastating effects on economies, health, and well-being of worldwide population. Due to the high increase in daily cases, the available number of COVID-19 test kits in under-developed countries is scarce. Hence, it is vital to implement an effective screening method of patients using chest radiography since the equipment already exists. With the presence of automatic detection systems, any abnormalities in chest radiography that characterizes COVID-19 can be identified. Several artificial-intelligence algorithms have been proposed to detect the virus. However, neural networks training is considered to be time-consuming. Since computations in training neural networks are spent on floating-point multiplications, high computational power is required. Multipliers consume the most space and power among all arithmetic operators in deep neural networks. This paper proposes a 15 Gbps high-speed bipolar-complementary-metal-oxide-semiconductor (BiCMOS) exclusive-nor (XNOR) gate to replace multipliers in binarized neural networks. The proposed gate can be implemented on BiCMOS-based field-programmable gate arrays (FPGAs). This will significantly improve the response time in identifying chest abnormalities in CT scans and X-rays.

CIS Publication Spotlight [Publication Spotlight

<i>A Survey of Stochastic Computing Neural Networks for Machine Learning Applications</i>, by Y. Liu, S. Liu, Y. Wang, F. Lombardi, and J. Han, <i>IEEE Transactions on Neural Networks and Learning Systems</i>, Vol. 32, No. 7, July 2021, pp. 2809&#x2013;2824.