Computational Neural Network Research Articles

Neural network estimation of kinetic parameters in distributed activation energy model (DAEM) without a priori assumptions for parallel reaction system

In this study, a new estimation method for the kinetic parameters in a distributed activation energy model (DAEM) was designed and developed. In the proposed method, the conversion estimation by the DAEM is regarded as a feedforward computation of a three-layer neural network, and the kinetic parameters of the DAEM are estimated by optimization of the neural network. The proposed method does not require an a priori assumption of the kinetic parameters or mechanism of a parallel reaction system. First, we created reaction data using numerical simulations, and a kinetic analysis using the neural network was performed. The neural network predicted conversion X very accurately; however, reactions with low contributions to the parallel reaction system also appeared. Next, we carried out a kinetic analysis using the neural network with the lower limit on the contribution of the ith reaction Vi*/V*. The lower limit on Vi*/V* did not influence the prediction accuracy of X and had a significant effect on reducing the reactions with low Vi*/V* and the estimation accuracies of the kinetic parameters in the DAEM. The optimal value of the lower limit on Vi*/V* was determined to be 1.0×10−5–1.0×10−4 when using the neural network with 64 hidden layer nodes. Moreover, the prediction accuracy of the proposed method was compared with those of conventional methods — the single-Gaussian method (1-DAEM), double-Gaussian method (2-DAEM), and Miura and Maki method. The kinetic parameters estimated using the proposed method were closer to the true values than those obtained using conventional methods. Moreover, the X value predicted by the proposed method was more accurate than that predicted by conventional methods. The influence of approximation on training data creation was also examined. The estimation accuracy of the neural network was still high but slightly deteriorated when the neural network was optimized using the reaction data created without the approximation.

Toward a cerebello-thalamo-cortical computational model of spinocerebellar ataxia

Computational neural network modelling is an emerging approach for optimization of drug treatment of neurological disorders and fine-tuning of rehabilitation strategies. In the current study, we constructed a cerebello-thalamo-cortical computational neural network model to simulate a mouse model of cerebellar ataxia (pcd5J mice) by manipulating cerebellar bursts through reduction of GABAergic inhibitory input. Cerebellar output neurons were projected to the thalamus and bidirectionally connected with the cortical network. Our results showed that reduction of inhibitory input in the cerebellum orchestrated the cortical local field potential (LFP) dynamics to generate specific motor outputs of oscillations of the theta, alpha, and beta bands in the computational model as well as in mouse motor cortical neurons. The therapeutic potential of deep brain stimulation (DBS) was tested in the computational model by increasing the sensory input to restore cortical output. Ataxia mice showed normalization of the motor cortex LFP after cerebellum DBS. We provide a novel approach to computational modelling to investigate the effect of DBS by mimicking cerebellar ataxia involving degeneration of Purkinje cells. Simulated neural activity coincides with findings from neural recordings of ataxia mice. Our computational model could thus represent cerebellar pathologies and provide insight into how to improve disease symptoms by restoring neuronal electrophysiological properties using DBS.

Emergent computations in trained artificial neural networks and real brains

Synaptic plasticity allows cortical circuits to learn new tasks and to adapt to changing environments. How do cortical circuits use plasticity to acquire functions such as decision-making or working memory? Neurons are connected in complex ways, forming recurrent neural networks, and learning modifies the strength of their connections. Moreover, neurons communicate emitting brief discrete electric signals. Here we describe how to train recurrent neural networks in tasks like those used to train animals in neuroscience laboratories and how computations emerge in the trained networks. Surprisingly, artificial networks and real brains can use similar computational strategies.

AINNS: All-Inclusive Neural Network Scheduling Via Accelerator Formalization

Driven by the rapid development of accelerators and diverse efficiency requirements of the naturally heterogeneous neural network computation, recent years have seen increased heterogeneity in neural network accelerator systems in terms of network structures, accelerator dataflows and implementations. However, existing research fails to schedule and map the heterogeneous neural networks on heterogeneous accelerators efficiently. They rely on clumpy exhaustive search or complicated ad hoc mapping approaches due to the semantic gap between the networks and accelerators. This paper proposes a systematic method to transform various accelerators into standard parameterized containers of the neural network loops, which builds a direct connection between the computation and the underlying hardware resources. This enables us to match the neural networks with accelerators based on their essential characteristics (e.g., reuse opportunities and bandwidth requirements) without diving into the detailed architectures. To this end, we propose AINNS, an all-inclusive neural network scheduler, that automatically schedules and maps the NN computation on heterogeneous accelerators with just one universal algorithm. Our experimental results show the proposed AINNS not only performs well in the traditional neural network acceleration but also improves the system throughput and energy efficiency by 1.8x and 1.7x respectively in the most challenging heterogeneous acceleration system.

Hand Gesture Recognition

Abstract: This work presents a computer-vision-based application for recognizing hand gestures. A live video feed is captured by a camera, and a still image is extracted from that feed with the aid of an interface. At least once per count hand gesture (one, two, three, four, and five), the system is trained. After that, the system is given a test gesture to see if it can identify it. Several algorithms that are capable of distinguishing a hand gesture were studied. It was determined that the highest rate of accuracy was achieved by using the computational neural network known as the Alexnet algorithm. Traditionally, systems have used data gloves or markers as a means of input. We are free to use the system however we like. In this way, the user can make natural hand gestures in front of the camera. The system implemented serves as an extendable basis for future work toward a fully robust hand gesture recognition system, which is still the subject of intensive research and development.

VFR: The Underwater Acoustic Target Recognition Using Cross-Domain Pre-Training with FBank Fusion Features

Underwater acoustic target recognition is a hot research area in acoustic signal processing. With the development of deep learning, feature extraction and neural network computation have become two major steps of recognition. Due to the complexity of the marine environment, traditional feature extraction cannot express the characteristics of the targets well. In this paper, we propose an underwater acoustic target recognition approach named VFR. VFR adopts a novel feature extraction method by fusing three-dimensional FBank features, and inputs the extracted features into a residual network, instead of the classical CNN network, plus cross-domain pre-training to perform target recognition. The experimental results show that VFR achieves 98.5% recognition accuracy on the randomly divided ShipsEar dataset and 93.8% on the time-divided dataset, respectively, which are better than state-of-the-art results.

Numerical and Levenberg–Marquardt backpropagation neural networks computation of ternary nanofluid flow across parallel plates with Nield boundary conditions

Numerical and Levenberg–Marquardt backpropagation neural networks computation of ternary nanofluid flow across parallel plates with Nield boundary conditions

AcousticIA, a deep neural network for multi-species fish detection using multiple models of acoustic cameras

Acoustic cameras, or imaging sonars, are high-potential devices for many applications in aquatic ecology, notably for fisheries management and population monitoring. However, how to extract such data into high-value information without a time-consuming entire data set reading by an operator is still a challenge. Moreover, the analysis of acoustic imaging, due to its low signal-to-noise ratio, is a perfect training ground for experimenting with new approaches, especially concerning deep learning techniques. We present hereby a novel approach that takes advantage of both convolutional neural network (CNN) and classical computer vision (CV) techniques, able to detect fish passages in acoustic video streams. The pipeline pre-treats the acoustic images to localise the signals of interest and to improve the detection performances. The YOLOv3-based model was trained with fish data from multiple species recorded by the two most frequently used models of acoustic cameras, the DIDSON and ARIS, including species of high ecological interest, as Atlantic salmon or European eels. The pre-treatment of images greatly improves the model performance, increasing its F1-score from 0.52 to 0.69. The model we developed provides satisfying results detecting almost 80% of fish passages and minimising the false-positive rate. On a validation data set, 40 h of videos and around 1 800 fish passages, the efficiency increases with the fish sizes, notably reaching a recall higher than 95% for Atlantic salmon. Conversely, the model appears much less efficient for eel detections on ARIS videos than on DIDSON data (31% recall vs 75%).

The Influence of the Number of Spiking Neurons on Synaptic Plasticity.

The main advantages of spiking neural networks are the high biological plausibility and their fast response due to spiking behaviour. The response time decreases significantly in the hardware implementation of SNN because the neurons operate in parallel. Compared with the traditional computational neural network, the SNN use a lower number of neurons, which also reduces their cost. Another critical characteristic of SNN is their ability to learn by event association that is determined mainly by postsynaptic mechanisms such as long-term potentiation. However, in some conditions, presynaptic plasticity determined by post-tetanic potentiation occurs due to the fast activation of presynaptic neurons. This violates the Hebbian learning rules that are specific to postsynaptic plasticity. Hebbian learning improves the SNN ability to discriminate the neural paths trained by the temporal association of events, which is the key element of learning in the brain. This paper quantifies the efficiency of Hebbian learning as the ratio between the LTP and PTP effects on the synaptic weights. On the basis of this new idea, this work evaluates for the first time the influence of the number of neurons on the PTP/LTP ratio and consequently on the Hebbian learning efficiency. The evaluation was performed by simulating a neuron model that was successfully tested in control applications. The results show that the firing rate of postsynaptic neurons post depends on the number of presynaptic neurons pre, which increases the effect of LTP on the synaptic potentiation. When post activates at a requested rate, the learning efficiency varies in the opposite direction with the number of pres, reaching its maximum when fewer than two pres are used. In addition, Hebbian learning is more efficient at lower presynaptic firing rates that are divisors of the target frequency of post. This study concluded that, when the electronic neurons additionally model presynaptic plasticity to LTP, the efficiency of Hebbian learning is higher when fewer neurons are used. This result strengthens the observations of our previous research where the SNN with a reduced number of neurons could successfully learn to control the motion of robotic fingers.

Quantum Fourier Convolutional Network

The neural network and quantum computing are both significant and appealing fields, with their interactive disciplines promising for large-scale computing tasks that are untackled by conventional computers. However, both developments are restricted by the scope of the hardware development. Nevertheless, many neural network algorithms had been proposed before GPUs became powerful enough for running very deep models. Similarly, quantum algorithms can also be proposed as knowledge reserve before real quantum computers are easily accessible. Specifically, taking advantage of both the neural networks and quantum computation and designing quantum deep neural networks (QDNNs) for acceleration on the Noisy Intermediate-Scale Quantum (NISQ) processors is also an important research problem. As one of the most widely used neural network architectures, convolutional neural network (CNN) remains to be accelerated by quantum mechanisms, with only a few attempts having been demonstrated. In this article, we propose a new hybrid quantum-classical circuit, namely, Quantum Fourier Convolutional Network (QFCN). Our model achieves exponential speedup compared with classical CNN theoretically and improves over the existing best result of quantum CNN. We demonstrate the potential of this architecture by applying it on different deep learning tasks, including traffic prediction and image classification.

Deep Learning Based Efficient Pneumonia Detection in Chest X-Ray Images

Medical motion plays an essential role in clinical treatment. The deep neural network is an emerging machine learning method that has proven its potential for different classification tasks. Notably, the convolutional neural network dominates with the best results on varying image classification tasks. However, medical image datasets are hard to collect because it needs a lot of professional expertise to label them. Although it can be detected and treated with very less sophisticated instruments and medication. Therefore, this paper how to apply the Convolutional Neural Network (CNN) based algorithm on a chest X-ray dataset to classify pneumonia. The objective and automated detection of pneumonia represents a serious challenge in medical imaging because the signs of the illness are not obvious in CT or X-ray scans. Further on, it is also an important task, since millions of people die of pneumonia every year. Deep learning-based methods have shown good generalization traits over various problem domains, which prompts researchers around the globe to work tirelessly and come up with more efficient and effective models than earlier. However, this robust nature comes at the cost of high computational resources and, in general, it requires a huge amount of data to train the model efficiently. The latter requirement sometimes cannot be fulfilled, especially in the biomedical field. The main goal is to propose a solution for the above-mentioned problem, using a novel deep neural network architecture. The proposed novelty consists of the use of dropout in the convolutional part of the network. The proposed method was trained and tested on a set of labeled images. This project aims to introduce a deep learning technology based on the computational neural network, which can realize automatic diagnosis of patients with pneumonia in X-ray images.

Компьютерное моделирование на основе нейросети как инструмент получения криминологически значимой информации по оценке состояния преступности (на материалах Республики Казахстан)

The purpose of the study is to obtain the missing information related to the assessment of the degree of influence of changes in some socio-economic factors on the state of crime and its individual manifestations. The motivation for the study is related to the initiative of the President of the Republic of Kazakhstan Kassym Jomart Tokayev to increase the minimum wage. The justification for the increase in the minimum wage was a calculation that showed an increase in gross domestic product by 1.5%. The method of research is computer modeling based on the functioning of a neural network. The creation and training of the neural network was based on official statistical data of the Republic of Kazakhstan. At the initial stage of the study, the effectiveness of the chosen method was tested in comparison with other methods of information processing and analysis. The test showed a higher accuracy of calculation using a computer neural network. The dependence of changes in the minimum wage with an increase in gross domestic product and a decrease in the crime rate was confirmed. At the next, main stage of the study, there was a need to improve the neural network by optimizing the input matrix intended for training. Optimization lies in the fact that the learning matrix was formed by those values of socio-economic parameters, the impact of which on the level of crime and crime associated with manifestations of terrorism and extremism is maximal. Test comparisons of calculation results using neural network training optimization showed more accurate data compared to standard neural network training. Using a more advanced neural network, modeling of the expanded impact of changes in the minimum wage on the level of crime and criminal activity of an extremist and terrorist nature was carried out. The simulation results showed that the dependence of changes in the crime rate on changes in the minimum wage has a more complex nature of influence, in which it is important to determine its optimal value. The increase in the minimum wage has practically no effect on crimes of special gravity. They can mitigate the manifestations of peak activity of crimes of particular severity, but they do not have a significant impact on this type of crime. The research group notes the universality of the described software tools. Its application can be significantly expanded and used in various applications related to law enforcement. The authors declare no conflicts of interests.

Calculation of Standard Entropy of Binary Solid Compounds by Neural Network Computation

In the present study, a layer type neural network computation was applied to estimate the standard entropy of binary solid oxides, sulfides and halides. Independent variables to influence the thermodynamics property associated with dispersion or randomness in the crystals were used as input parameters for the calculation. 325 substances involving 12 input parameters were applied to the calculation. The regression computation enabled reproduction of training data cited in learning process and prediction of test data not used in the learning process with high accuracy.

Computation and implementation of large scalable Spiking Neural Network

Photonic neural networks are a highly sought-after area of research due to their potential for high-performance complex computing. Unlike artificial neural networks, which use simple nonlinear maps, biological neurons transmit information and perform computations through spikes that depend on spike time and/or rate. Through comprehensive studies and experiments, a strong foundation has been laid for the development of photonic neural networks. We have recently developed a large-scale spiking neural network, which serves as a proof-of-concept experiment for novel bio-inspired learning concepts. This breakthrough is significant because it demonstrates the potential of using photonic neural networks for advanced computing and highlights the importance of incorporating biological principles into artificial intelligence research.

L-GhostNet: Extract Better Quality Features

A lightweight image recognition model, L-GhostNet based on improved GhostNet, is proposed to address the problems of extensive computation and high storage cost of deep convolutional neural networks. The model incorporated learning group convolution and improved CA into GhostNet to reduce the calculation and number of parameters and improve the flexibility of the network. At the same time, the pruning ratio in the learning group convolution is increased to control the end time of pruning in the whole process; the improved CA uses a fully connected layer to replace the convolutional layer, which can make the connection between the two dimensions tighter and increase the flexibility of the model. Experiments on datasets in various fields, such as grape leaf recognition, gesture recognition, face recognition, rice recognition, and CIFAR-10, show that L-GhostNet has slightly improved accuracy, reduced computation by more than 44%, decreased the number of parameters by more than 33%, and improved FPS by 26% on all datasets compared to GhostNet. Compared with other commonly used lightweight network models, MobileNets and ShuffleNets, it has the best overall performance with the lowest FLOPs, highest accuracy, and fewer parameters on all datasets at the same level of FLOPs.

Fast Algorithms for Deep Octonion Networks.

This brief presents the results of a study of the possibilities of reducing the arithmetic complexity of computing basic operations in octonionic neural networks and also proposes new algorithmic solutions for efficiently performing these operations. Here, we primarily mean the operation of multiplying octonions, the operation of computing the dot product of two octonion-valued vectors, and the operation of multiple multiplications of an octonion by several other octonions. In order to reduce the computational complexity of these operations, it is proposed to use the fast Walsh-Hadamard transform, which is well known in digital signal processing. Using this transform reduces the number of multiplications and additions of real numbers required to perform computations. Thus, the use of the proposed algorithms will speed up computations in octonion-valued neural networks.

Hierarchical Model Compression via Shape-Edge Representation of Feature Maps—an Enlightenment From the Primate Visual System

The cumbersome computation of deep neural networks (DNNs) limits the practical deployment of DNNs on resource-constrained mobile multimedia devices. To deploy DNNs on devices with limited power and computing resources, model compression techniques are leveraged to accelerate the networks, where network pruning can improve the inference efficiency for deployed DNNs by removing redundant weights and structures. As one of the important components of DNNs, the feature maps (FMs) can be leveraged to evaluate the importance of network structures for DNN pruning. However, previous methods neglect to fully explore the characteristics of FMs to prune the DNNs. In this paper, we investigate the high capacity and resource efficient analogy-ventral dual-pathway primates visual system to propose a hierarchical pruning framework (dubbed as HPSE). In an efficient primate visual system (PVS), the analog pathway analyzes low-frequency information to facilitate the high-frequency information inference in ventral stream. In HPSE, we extract the low-frequency shape information and high-frequency edge information from FMs to present a novel pruning pipeline that resembles the analysis mechanism of PVS. In particular, we first imitate the analogy pathway to group different FMs in each layer by calculating the shape-feature overlap. Secondly, we leverage the edge information modulated by the grouping results of the first step to prune the network. The effectiveness of HPSE is verified by pruning various DNNs on different benchmarks, and the results have achieved satisfactory performances. For example, for ResNet-56 on CIFAR-10, HPSE reduces 52.9% of FLOPs with a slight accuracy improvement; for ResNet-50 on ImageNet, we achieve 54.3%-FLOPs drop with only 0.49% Top-1 accuracy loss.

STUDY OF THE PASSENGER TRAFFIC PARAMETERS IN AIR TRANSPORT

The article is an overview of the issue of calculating. The study mainly used the publications of scientists in the chosen field of research, published in recent years, which provides the research with modern scientific achievements in this area. All the publications used are recognized by the scientific community and included in scientometric databases, including Scopus. To obtain the results, the method of system analysis is applied to the methods and approaches used by the authors in solving the problems of calculating the parameters of passenger flows, determining the factors of their formation and modeling such flows. The article formulates the approaches and methods outlined by the authors and reveals certain advantages of the solutions proposed by the authors. The results of the analysis of the development of scientific thought on the issues under consideration are the formation of generalized conclusions regarding the methods of modeling certain transport processes. It is proved that modern scientists pay considerable attention to the modeling of passenger flows in time using mathematical modeling, in particular. Popular methods include neural network methods, computer modeling, and methods for solving problems inherent in engineering networks. As a result, a model was obtained for calculating such parameters as: payback period of the project for the purchase of transport for the route, net and net discounted profit flows, costs of maintaining personnel for route maintenance, passenger correspondence matrix, passenger traffic on the route between network nodes, transferability parameters, and many others. The obtained results make it possible to make systemic decisions on the feasibility of certain actions in the route system and provide probable options for the operation of the route system in its management

3D Point Cloud Semantic Segmentation System Based on Lightweight FPConv

In this paper, we proposed a 3D point cloud semantic segmentation system based on lightweight FPConv. In 3D point cloud mapping, data is depicted in a 3D space to represent 3D imagery data. These maps are collected through direct measurements; all points in a 3D point cloud map corresponds to a measurement point and, therefore, contains a large amount of data. Data in 3D point cloud maps are stored in point clouds, and they are extracted using 3D image processing or deep learning. However, because of the non-structured and high-dimensional properties of point clouds, the development of 3-D image recognition applications in the field of computer vision warrants further exploration. Large-scale neural networks are highly accurate, but they have the disadvantages of high computation complexity and low portability. Therefore, the present study proposed a 3D point cloud semantic segmentation system based on lightweight FPConv. The proposed network combines depth-wise separate convolution, quantization, and Winograd convolution technology to lighten and accelerate neural network computation. The performance of the presented network was verified using the Stanford 3D Large-Scale Indoor Spaces (S3DIS) large scene database provided by Stanford 3D AI Lab. The results reveals the excellent performance of the proposed model.

Highly Linear and Symmetric Analog Neuromorphic Synapse Based on Metal Oxide Semiconductor Transistors with Self‐Assembled Monolayer for High‐Precision Neural Network Computation

AbstractThis work presents an analog neuromorphic synapse device consisting of two oxide semiconductor transistors for high‐precision neural networks. One of the two transistors controls the synaptic weight by charging or discharging the storage node, which leads to a conductance change in the other transistor. The programmed weight maintains for more than 300 s as electrons in the storage node are well preserved due to the extremely low off current of the oxide transistor. Ideal synaptic behaviors are achieved by utilizing superior properties of oxide transistors such as a high on/off ratio, low off current, and large‐area uniformity. To further improve the synaptic performance, self‐assembled monolayer treatment is applied for reducing the transistor conductance. The reduction of on current reduces the power consumption, and the reduced off current improves the retention characteristics. There is no noticeable decrease in simulated neural network accuracy even when the measured device‐to‐device variation is intentionally increased by 200%, indicating the possibility of large‐array operation with the synapse device.