Computational Neural Network Research Articles

ZnO-based artificial synaptic diodes with zero-read voltage for neural network computing

Brain-inspired neuromorphic sensory devices play a crucial role in addressing the limitations of von Neumann systems in contemporary computing. Currently, synaptic devices rely on memristors and thin-film transistors, requiring the establishment of a read voltage. A built-in electric field exists within the p–n junction, enabling the operation of zero-read-voltage synaptic devices. In this study, we propose an artificial synapse utilizing a ZnO diode. Typical rectification curves characterize the formation of ZnO diodes. ZnO diodes demonstrate distinct synaptic properties, including paired-pulse facilitation, paired-pulse depression, long-term potentiation, and long-term depression modulations, with a read voltage of 0 V. An artificial neural network is constructed to simulate recognition tasks using MNIST and Fashion-MNIST databases, achieving test accuracy values of 92.36% and 76.71%, respectively. This research will pave the way for advancing zero-read-voltage artificial synaptic diodes for neural network computing.

Study of Weight Quantization Associations over a Weight Range for Application in Memristor Devices

The development of hardware-based cognitive computing systems critically hinges upon the integration of memristor devices capable of versatile weight expression across a spectrum of resistance levels while preserving consistent electrical properties. This investigation aims to explore the practical implementation of a digit recognition system utilizing memristor devices with minimized weighting levels. Through the process of weight quantization for digits represented by 25 or 49 input signals, the study endeavors to ascertain the feasibility of digit recognition via neural network computation. The integration of memristor devices into the system architecture is poised to streamline the representation of the resistors required for weight expression, thereby facilitating the realization of neural-network-based cognitive systems. To minimize the information corruption in the system caused by weight quantization, we introduce the concept of “weight range” in this work. The weight range is the range between the maximum and minimum values of the weights in the neural network. We found that this has a direct impact on weight quantization, which reduces the number of digits represented by a weight below a certain level. This was found to help maintain the information integrity of the entire system despite the reduction in weight levels. Moreover, to validate the efficacy of the proposed methodology, quantized weights are systematically applied to an array of double-layer neural networks. This validation process involves the construction of cross-point array circuits with dimensions of 25 × 10 and 10 × 10, followed by a meticulous examination of the resultant changes in the recognition rate of randomly generated numbers through device simulations. Such endeavors contribute to advancing the understanding and practical implementation of hardware-based cognitive computing systems leveraging memristor devices and weight quantization techniques.

Implementation of Artificial Neural Networks on Field-Programmable Gate Arrays

Implementing Artificial Neural Networks (ANNs) on Field-Programmable Gate Arrays (FPGAs) provides a promising solution for achieving high-performance, low-latency, and energy-efficient computations in complex tasks. This paper investigates the methodology for mapping ANNs onto FPGAs, focusing on critical aspects such as architecture selection, hardware design, and optimization techniques. By harnessing the parallel processing capabilities and reconfigurability of FPGAs, neural network computations are significantly accelerated, making them ideal for real-time applications like image processing and embedded systems. The implementation process addresses key considerations, including fixed-point arithmetic, memory management, and dataflow optimization, while employing advanced techniques such as pipelining, quantization, and pruning. The research compares the accuracy and performance speedup of ANNs on CPUs versus FPGAs, revealing that FPGA-based simulations are 4680 times faster than CPU-based simulations using MATLAB, without compromising prediction accuracy.

Applications of Machine Learning: Classifying Wheat Seeds through Computer Neural Networks

In the past few decades, the incompetent processes of traditional mechanical seed sorting have given rise to new, innovative methods, along with the integration of smart technology in agriculture. The present study created a multi-classification machine learning model that classifies wheat seeds based on given characteristics by giving highly accurate predictions for future samples based on given data. Due to various uses of different types of seeds, effective sorting is an integral preparatory stage for food processing. The dataset used in the study included seven features (directly measured characteristics of seeds), three labels (types of seeds), and around two hundred examples. The seven features include area, perimeter, compactness, kernel length, kernel width, asymmetry coefficient, and kernel groove. The first model was created through neural networks trained on all seven characteristics, reaching an overall accuracy of 0.94. Implementing a decision tree method, feature selection was applied to the data, which distinguished kernel groove, kernel width, and asymmetry coefficient as the three most important features. A new model was constructed with only the three selected features, which reduced the risk of overfitting, eliminated noise, and yielded a higher accuracy (0.96) than the previous model. Both multi-classification models used the softmax activation function and results were compared; feature selection noticeably reduced any inaccuracies, including categorical cross entropy loss. The main incentive of the study was to develop an effective model to sort wheat seeds (to replace traditional mechanical sorting) and provide reasonable recommendations for practical use through an analysis of feature selection.

Optimization-Based Vibration Control Analysis of Dampers in High-Rise Building

Tuned mass dampers (TMDs) are a useful control of vibrations for tall buildings and can be considered equal to burdens such as breezes and earthquakes. The better limits for TMD are to reduce the earthquake vibrations of tall buildings including soil–structure interaction (SSI) influences. This study proposes the multi-objective Black Widow Optimization (BWO) with Elman neural network (ENN) computation that is brought on to find the optimal limits of TMD. Considering that this proposed model analyzed the mass, stiffness, and damping ratio of TMD to get the maximum accelerations and displacement of 40 story building model, from the execution results, starting from this proposed BWO–Elman model to the expected methodology, the most outrageous TMD limits are better, likewise, the reduction of acceleration and displacement are 25% and 19.86% in the proposed study. This study assists the researchers with a better understanding of earthquake vibration and leads the fashioners to achieve superior TMD for tall designs. The impact of vibration control on the displacements and accelerations of both controlled and uncontrolled buildings is investigated in this work. It uses a MATLAB software technique to analyze their modal replies, moreover, this proposed optimal TMD model is compared with conventional techniques by acceleration and stiffness parameters.

Orthogonal Matrix-Autoencoder-Based Encoding Method for Unordered Multi-Categorical Variables with Application to Neural Network Target Prediction Problems

Neural network models, such as BP, LSTM, etc., support only numerical inputs, so data preprocessing needs to be carried out on the categorical variables to convert them into numerical data. For unordered multi-categorical variables, existing encoding methods may produce dimensional catastrophes and may also introduce additional order misrepresentation and distance bias in neural network computation. To solve the above problems, this paper proposes an unordered multi-categorical variable encoding method O-AE using orthogonal matrix for encoding and encoding representation learning and dimensionality reduction via an autoencoder. Bayesian optimization is used for hyperparameter optimization of the autoencoder. Finally, seven experiments were designed with the basic O-AE, Bayesian optimization of the hyperparameters of the autoencoder for O-AE, and other encoding methods to encode unordered multi-categorical variables in five datasets, and they were input into a BP neural network to carry out target prediction experiments. The results show that the experiments using O-AE and O-AE-b have better prediction results, proving that the method proposed in this paper is highly feasible and applicable and can be an optional method for the data processing of unordered multi-categorical variables.

An efficient ANN SoC for detecting Alzheimer's disease based on recurrent computing

Alzheimer's Disease (AD) is an irreversible, degenerative condition that, while incurable, can have its progression slowed or impeded. While there are numerous methods utilizing neural networks for AD detection, there is a scarcity of High-performance AD detection chips. Moreover, excessively complex neural networks are not conducive to on-chip implementation and clinical applications. This study addresses the challenges of high misdiagnosis rates and significant hardware costs inherent in traditional AD detection techniques. A novel and efficient AD detection framework based on a recurrent computational strategy is proposed. The framework harnesses an Artificial Neural Network (ANN) embedded within a System on Chip (SoC) to perform sophisticated Electroencephalogram (EEG) analysis. The approach began by employing a reduced IEEE754 single-precision encoding method to hardware-encode the preprocessed EEG data, thereby minimizing the memory storage area. Next, data remapping techniques were utilized to ensure the continuity of the input data read addresses and reduce the memory access pressure during neural network computations. Subsequently, hierarchical and Processing Element (PE) reuse technologies were leveraged to perform the multiply-accumulate operations of the ANN. Finally, a step function was chosen to establish binary classification circuits dedicated to AD detection. Experimental results indicate that the optimized SoC achieves a 70 % reduction in area and a 50 % reduction in power consumption compared to traditional designs. For various neural network models, the detection model proposed in this paper incurs less overhead, with a training speed 3 to 4 times faster than other traditional models, and a high accuracy rate of 98.53 %.

(Invited) Analog Computation-in-Memory (CiM) for AI and Neuromorphic Computing

[Invited talk]This paper overviews analog Computation-in-Memory (CiM) with non-volatile memories for AI applications and neuromorphic computing. In the edge IoT/mobile devices, CiM will be heterogeneously integrated with digital processors such as CPUs to realize the inference of AI with high energy efficiency.This talk also introduces the approximate CiM. Inspired by the approximate computing, by tolerating some degree of memory errors, the trade-off of performance, energy and cost is resolved.Finally, this paper covers various AI algorithms and neuromorphic computing such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), event driven Spiking Neural Network (SNN) and Reservoir Computing.

Innovative thermal management in the presence of ferromagnetic hybrid nanoparticles

In the present work, a simple intelligence-based computation of artificial neural networks with the Levenberg-Marquardt backpropagation algorithm is developed to analyze the new ferromagnetic hybrid nanofluid flow model in the presence of a magnetic dipole within the context of flow over a stretching sheet. A combination of cobalt and iron (III) oxide (Co-Fe2O3) is strategically selected as ferromagnetic hybrid nanoparticles within the base fluid, water. The initial representation of the developed ferromagnetic hybrid nanofluid flow model, which is a system of highly nonlinear partial differential equations, is transformed into a system of nonlinear ordinary differential equations using appropriate similarity transformations. The reference data set of the possible outcomes is obtained from bvp4c for varying the parameters of the ferromagnetic hybrid nanofluid flow model. The estimated solutions of the proposed model are described during the testing, training, and validation phases of the backpropagated neural network. The performance evaluation and comparative study of the algorithm are carried out by regression analysis, error histograms, function fitting graphs, and mean squared error results. The findings of our study analyze the increasing effect of the ferrohydrodynamic interaction parameter β\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\beta$$\\end{document} to enhance the temperature and velocity profiles, while increasing the thermal relaxation parameter α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\alpha$$\\end{document} decreases the temperature profile. The performance on MSE was shown for the temperature and velocity profiles of the developed model about 9.1703e−10, 7.1313ee−10, 3.1462e−10, and 4.8747e−10. The accuracy of the artificial neural networks with the Levenberg-Marquardt algorithm method is confirmed through various analyses and comparative results with the reference data. The purpose of this study is to enhance understanding of ferromagnetic hybrid nanofluid flow models using artificial neural networks with the Levenberg-Marquardt algorithm, offering precise analysis of key parameter effects on temperature and velocity profiles. Future studies will provide novel soft computing methods that leverage artificial neural networks to effectively solve problems in fluid mechanics and expand to engineering applications, improving their usefulness in tackling real-world problems.

Solution of state transfer matrix based on DNA strand displacement circuits

In recent years, DNA strand displacement (DSD) circuits have been developed in leaps and bounds. The high storage and parallelism of DNA give it an inherent advantage in the field of DNA computations. DSD is widely used in analog computations and neural network computations. However, there are few studies on solving state transfer matrix using DSD circuits. Aiming at the above problems, a scheme for solution of state transfer matrix based on DSD circuits is proposed. In this paper, the summation, subtraction, multiplication, division and exponential reaction modules are built by the DSD reactions. Based on the reaction modules, DNA chemical reaction networks of state transfer matrix are constructed. DSD circuits are built by cascading the DNA chemical reaction networks. The solution of the state transfer matrix is implemented through the DSD circuits. The Visual DSD is used to verify the practicality of DSD circuits. This scheme may provide a reference to analog computations based on DSD circuits.

Implementation and research of gesture recognition in HCI field

Since the first Neural Network computer was made by Marvin Minsky and his schoolmates in 1951, the development of artificial intelligence (AI)has undergone various and huge changes. It generally evolves into a field that has infinite possibilities and a major branch of computer science that can not be ignored. With the trend of computerization and the rapid evolution of the Internet of Things (IoT), gesture recognition emerged. Various prototypes are blossoming in the laboratories, and some of them have become certain products that have practical applications in later days. At the same time, more and more deep learning technology has been applied to gesture recognition systems that greatly improve the quality of the service. The purpose of this review is to summarize and analyze the existing algorithms of Dynamic Gesture Recognition systems, which have several different methods that are based on multiple signal extractors. This study focuses on the 3D Convolutional Neural Network(CNN), which plays an important role in the Dynamic Gesture Recognition optimization algorithm and data analysis algorithm. In this paper, we reviewed past papers in the gesture recognition field, which include the sEMG method, microwave method, and vision recognition method. During the process of data collecting and summarizing the past papers, we mainly focused on the accuracy differences between recognition methods and the efficiency differences in data processing algorithms. (CNN-based, LSTM based, etc.) Then, we analyzed the main difficulties and challenges of these methods, which are briefly listed in the Introduction part. Data processing algorithms are also being studied, and a horizontal comparison between CNN-based, LSTM-based, and transformer-based algorithms is also being made. Besides this, for those problems that already have a reliable solution, we also summarized the possible solutions and listed them out. We found that the gesture recognition system has already been systematically studied and is partly used in some fields. However, the algorithm and modeling methods can still be optimized and it also needs further study to be more widely used.

Recent Trends in Application of Memristor in Neuromorphic Computing: A Review

Abstract: Recently memristors have emerged as a form of nonvolatile memory that is based on the principle of ion transport in solid electrolytes under the impact of an external electric field. It is perceived as one of the key elements to building next-generation computing systems owing to its peculiar resistive switching characteristics. The switching mechanism in a memristor is mainly governed by filamentary conduction. Further, it can be employed as a memory as well as a logic element, which makes it an ideal candidate for building innovative computer architecture. Moreover, it is capable of mimicking the characteristics of biological synapses, which makes it an ideal candidate for developing a Neuromorphic system. In this review to begin with the switching mechanism of the memristor, primarily focusing on filamentary conduction, is discussed. Few SPICE models of memristor are reviewed, and their critical comparison is performed, which are widely used to build computing systems. An in-depth study on the various crossbar memory architecture augmented with memristors is reviewed. Finally, the application of memristors in neuromorphic computing and hardware implementation of Artificial Neural Networks (ANN) employing memristors is discussed.

Special Issue: Selected papers from the AIxIA 2023 Workshops

The 2023 edition of the AIxIA Conference, held in Rome, brought together a large number of researchers and practitioners to discuss the most recent and important advancements in Artificial Intelligence (AI). The conference featured 19 workshops, organized by 77 experts, attracting 248 submissions and resulting in 16 proceedings. This special issue presents extended versions of selected papers initially showcased at these workshops. Each paper underwent rigorous review and represents a diverse array of topics, reflecting the multifaceted nature of the Italian AI community. The topics covered include ethical foundations to symbiotic AI, symbolic knowledge extraction from black-box models, creative influence prediction using graph theory, AI approaches to multidimensional poverty prediction, an assessment of AI-based supports for informal caregivers, deep learning-based EEG denoising, AI-assisted board-game-based learning, large language models for assessment and feedback in higher education, geometric reasoning in the Traveling Salesperson Problem, defeasible reasoning in weighted knowledge bases, and conditional computation in neural networks. These contributions demonstrate the innovative and interdisciplinary research within the AI community, offering valuable insights and advancing the field.

Conditional computation in neural networks: Principles and research trends

This article summarizes principles and ideas from the emerging area of applying conditional computation methods to the design of neural networks. In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input. Examples include the dynamic selection of, e.g., input tokens, layers (or sets of layers), and sub-modules inside each layer (e.g., channels in a convolutional filter). We first provide a general formalism to describe these techniques in an uniform way. Then, we introduce three notable implementations of these principles: mixture-of-experts (MoEs) networks, token selection mechanisms, and early-exit neural networks. The paper aims to provide a tutorial-like introduction to this growing field. To this end, we analyze the benefits of these modular designs in terms of efficiency, explainability, and transfer learning, with a focus on emerging applicative areas ranging from automated scientific discovery to semantic communication.

Comparative analysis of single-board computers for the development of a microarchitectural computing system for fire detection

Objectives. The purpose of the work is to select the basic computing microplatform of the onboard microarchitectural computing complex for the detection of anomalous situations in the territory of the Republic of Belarus from space on the basis of artificial intelligence methods.Methods. The method of comparative analysis is used to select a computing platform. A series of performance tests and comparative analysis (benchmarking) are performed on the selected equipment. The methods of comparative and benchmarking analysis are performed in accordance with the terms of reference to the current project.Results. A comparative analysis and performance testing of Raspberry Pi 4 Model B and Cool Pi 4 Model B single-board computers, as well as AI-accelerator Google Coral USB Accelerator with Google Edge TPU have been performed. The comparative analysis showed that Raspberry Pi 4 Model B and Cool Pi 4 Model B fully meet the terms of reference to the current project. At the same time Cool Pi 4 Model B handles neural network calculations well, but four times slower than similar calculations on Google Coral USB Accelerator. Neural network computations on the Raspberry Pi 4 Model B are 22 times slower than similar computations on the Google Coral USB Accelerator. Cool Pi 4 Model B outperforms Raspberry Pi 4 Model B by the factor of two to three for data copying and compression and almost six times faster for neural network computations.Conclusion. Despite the fact that Raspberry Pi 4 Model B meets the terms of reference to the project as a computational basis, when developing an on-board microarchitectural computing system for detecting anomalous situations, it is worth using more powerful alternatives with built-in AI-accelerators (e.g., Radxa Rock 5 Model A) or with an additional external AI-accelerator (e.g., a combination of Cool Pi 4 Model B and Google Coral USB Accelerator). Using a Raspberry Pi 4 Model B with an additional AI-accelerator is also acceptable and will speed up computations by several dozen times. AI-accelerators provide the fastest neural network computations, but there are features related to the novelty of the technology that will be explored in further development.

The Mechanics Underpinning Non-Deterministic Computation in Cortical Neural Networks

The Mechanics Underpinning Non-Deterministic Computation in Cortical Neural Networks

Machine learning in biological physics: From biomolecular prediction to design

Machine learning has been proposed as an alternative to theoretical modeling when dealing with complex problems in biological physics. However, in this perspective, we argue that a more successful approach is a proper combination of these two methodologies. We discuss how ideas coming from physical modeling neuronal processing led to early formulations of computational neural networks, e.g., Hopfield networks. We then show how modern learning approaches like Potts models, Boltzmann machines, and the transformer architecture are related to each other, specifically, through a shared energy representation. We summarize recent efforts to establish these connections and provide examples on how each of these formulations integrating physical modeling and machine learning have been successful in tackling recent problems in biomolecular structure, dynamics, function, evolution, and design. Instances include protein structure prediction; improvement in computational complexity and accuracy of molecular dynamics simulations; better inference of the effects of mutations in proteins leading to improved evolutionary modeling and finally how machine learning is revolutionizing protein engineering and design. Going beyond naturally existing protein sequences, a connection to protein design is discussed where synthetic sequences are able to fold to naturally occurring motifs driven by a model rooted in physical principles. We show that this model is “learnable” and propose its future use in the generation of unique sequences that can fold into a target structure.

Symmetric and Energy‐Efficient Conductance Update in Ferroelectric Tunnel Junction for Neural Network Computing

Symmetric and Energy‐Efficient Conductance Update in Ferroelectric Tunnel Junction for Neural Network Computing

Marine Diesel Engine Fault Detection Based on Xilinx ZYNQ SoC

Marine diesel engines are the preferred power equipment for ships and are the most important component among the numerous electromechanical devices on board. Accidents involving these engines can potentially cause immeasurable damage to the vessel, making fault detection in marine diesel engines crucial. This design enables the detection and reporting of faults in marine diesel engines at the earliest possible time through the computation of convolutional neural networks, which is of great significance for ensuring the safe navigation of ships. For this functionality, the Xilinx ZYNQ-7000 XC7Z010 is selected as the main control chip, and the LoRa wireless network is used as the transmission module. The FreeRTOS embedded operating system is ported, with sensor data collection completed on the PS side of the ZYNQ chip and algorithm acceleration calculations on the PL side. Data are then transmitted to the host computer via the LoRa module paired with a custom protocol. Experimental test results show that the program provides stable data transmission, with each module of the algorithm generally accelerating by more than 95% and an accuracy rate of 92.86%. Additionally, the host computer can display the received data in real time. The custom protocol’s header also allows for precise judgments about the completeness and origin of messages, facilitating the expansion of other SOC’s message uplink and the host computer’s message downlink.

Parallel photonic chip for nanosecond end-to-end image processing, transmission, and reconstruction

Image processing, transmission, and reconstruction constitute a major proportion of information technology. The rapid expansion of ubiquitous edge devices and data centers has led to substantial demands on the bandwidth and efficiency of image processing, transmission, and reconstruction. The frequent conversion of serial signals between the optical and electrical domains, coupled with the gradual saturation of electronic processors, has become the bottleneck of end-to-end machine vision. Here, we present an optical parallel computational array chip (OPCA chip) for end-to-end processing, transmission, and reconstruction of optical intensity images. By proposing constructive and destructive computing modes on the large-bandwidth resonant optical channels, a parallel computational model is constructed to implement end-to-end optical neural network computing. The OPCA chip features a measured response time of 6 ns and an optical bandwidth of at least 160 nm. Optical image processing can be efficiently executed with minimal energy consumption and latency, liberated from the need for frequent optical–electronic and analog–digital conversions. The proposed optical computational sensor opens the door to extremely high-speed processing, transmission, and reconstruction of visible contents with nanoseconds response time and terahertz bandwidth.