Implementation Of Neural Networks Research Articles

Skyrmion-mediated nonvolatile ternary memory

Multistate memory systems have the ability to store and process more data in the same physical space as binary memory systems, making them a potential alternative to existing binary memory systems. In the past, it has been demonstrated that voltage-controlled magnetic anisotropy (VCMA) based writing is highly energy-efficient compared to other writing methods used in non-volatile nano-magnetic binary memory systems. In this study, we introduce a new, VCMA-based and skyrmion-mediated non-volatile ternary memory system using a perpendicular magnetic tunnel junction (p-MTJ) in the presence of room temperature thermal perturbation. We have also shown that ternary states {− 1, 0, + 1} can be implemented with three magnetoresistance values obtained from a p-MTJ corresponding to ferromagnetic up, down, and skyrmion state, with 99% switching probability in the presence of room temperature thermal noise in an energy-efficient way, requiring ~ 2 fJ energy on an average for each switching operation. Additionally, we show that our proposed ternary memory demonstrates an improvement in area and energy by at least 2X and ~ 104X respectively, compared to state-of-the-art spin-transfer torque (STT)-based non-volatile magnetic multistate memories. Furthermore, these three states can be potentially utilized for energy-efficient, high-density in-memory quantized deep neural network implementation.

Research on the Inhibition and Transmission Properties of Photonic Spiking Dynamics in Semiconductor Ring Lasers

Significant progress has been made in the research of all-optical neural networks in recent years. In this paper, we theoretically explore the properties of a neural system composed of semiconductor ring lasers (SRLs). Our study demonstrates that external optical signals generated by a tunable laser (TL) are injected into the first semiconductor ring laser photonic neuron (SRL1). Subsequently, the responses of SRL1 in the clockwise (CW) and counterclockwise (CCW) directions are unidirectionally injected into the CW and CCW directions of the second semiconductor ring laser photonic neuron (SRL2), respectively, which then exhibits similar spiking inhibition behaviors. Numerical simulations reveal that the spiking inhibition behavior of the SRL response can be precisely controlled by adjusting the perturbation time and intensity of the external injection signal, and this behavior is highly repeatable. Most importantly, we successfully achieve the stable transmission of these responses between the two SRL photonic neurons. These inhibition behaviors are analogous to those of biological neurons, but with a response speed reaching the sub-nanosecond level. Additionally, we indicate that SRL photonic neurons undergo a refractory-period-like phenomenon when subjected to two consecutive perturbations. These findings highlight the immense potential for the design and implementation of future all-optical neural networks, providing critical theoretical foundations and support for them.

Tunable intermediate states for neuromorphic computing with spintronic devices

In the pursuit of advancing neuromorphic computing, our research presents a novel method for generating and precisely controlling intermediate states within heavy metal/ferromagnet systems. These states are engineered through the interplay of a strong in-plane magnetic field and an applied charge current. We provide a method for fine-tuning these states by introducing a small out-of-plane magnetic field, allowing for the modulation of the system’s probabilistic response to varying current levels. We also demonstrate the implementation of a spiking neural network (SNN) with a tri-state spike timing-dependent plasticity (STDP) learning rule using our devices. Our research furthers the development of spintronics and informs neural system design. These intermediate states can serve as synaptic weights or neuronal activations, paving the way for multi-level neuromorphic computing architectures.

Probabilistic Bayesian Neural Networks for olive phenology prediction in precision agriculture

Plant phenology is the study of cyclical events in a plant life cycle such as leaf bud burst, flowering, and fruiting. In this article the problem of olive phenology prediction is addressed through the use of Deep Learning. Although Neural Networks have already been used in this area, to the best of our knowledge, this is the first implementation of Probabilistic Bayesian Neural Networks for olive phenology prediction. This architecture gives particular emphasis to estimating the model uncertainty, both aleatoric and epistemic. The Bayesian Inference method, more precisely the Variational Inference one, is compared with the Monte Carlo Dropout technique, which is known to be a less computationally intensive approximation of Variational Inference. For validation purposes, models performance is compared to the state-of-the-art results.

Reconfigurable Neuromorphic Computing with 2D Material Heterostructures for Versatile Neural Information Processing.

Reconfigurable neuromorphic computing holds promise for advancing energy-efficient neural network implementation and functional versatility. Previous work has focused on emulating specific neural functions rather than an integrated approach. We propose an all two-dimensional (2D) material-based heterostructure capable of performing multiple neuromorphic operations by reconfiguring output terminals in response to stimuli. Specifically, our device can synergistically emulate the key neural elements of the synapse, neuron, and dendrite, which play important and interrelated roles in information processing. Dendrites, the branches that receive and transmit presynaptic action potentials, possess the ability to nonlinearly integrate and filter incoming signals. The proposed heterostructure allows reconfiguration between different operation modes, demonstrating its potential for diverse computing tasks. As a proof of concept, we show that the device can perform basic Boolean logic functions. This highlights its applicability to complex neural-network-based information processing problems. Our integrated neuromorphic approach may advance the development of versatile, low-power neuromorphic hardware.

Bidirectional Power Sharing Control with Current-Fed Dual Active Bridge Converter: Hybrid Technique

This paper proposes a hybrid method for controlling bidirectional power sharing among grids and dc microgrids. The interlinking converter (IC) and current-fed dual active bridge (CFDAB) converters connect the grid and dc microgrid. The proposed approach is the joint implementation of golden eagle optimization and radial basis function neural network, hence, named the GEO-RBFNN approach. The bidirectional power distribution control signal analysis significantly diminishes power distribution deviation and circulating current. This power-sharing control system neglects the need for battery backup since the required amount of power is brought from or handled by an AC utility. The dc microgrid incorporates the dc to dc converter, from which boost converters use sources and buckconverters use supply loads. The control of bidirectional power sharing among the microgrids and the grid is attained with the aid of the GEO-RBFNN algorithm. It delivers certain aids, like higher predictability, highly reliable results, high speed, and less complexity. The outcome is investigated on the MATLAB platform. According to the comparison outcome, the proposed method delivers higher performance than the existing system. The proposed method gives higher comparison outcomes than the current methods, which can conquer the associated problems. The proposed method-based settling time is 3.356 ms, and the rise time is 1.555 ms. The efficiency based on the proposed method is 97.54%, which is higher than the existing methods.

Unveiling the Phase Diagram and Reaction Paths of the Active Model B with the Deep Minimum Action Method.

Nonequilibrium phase transitions are notably difficult to analyze because their mechanisms depend on the system's dynamics in a complex way due to the lack of time-reversal symmetry. To complicate matters, the system's steady-state distribution is unknown in general. Here, the phase diagram of the active Model B is computed with a deep neural network implementation of the geometric minimum action method (gMAM). This approach unveils the unconventional reaction paths and nucleation mechanism in dimensions 1, 2, and 3, by which the system switches between the homogeneous and inhomogeneous phases in the binodal region. Our main findings are (i)the mean time to escape the phase-separated state is (exponentially) extensive in the system size L, but it increases nonmonotonically with L in dimension 1; (ii)the mean time to escape the homogeneous state is always finite, in line with the recent work of Cates and Nardini [Phys. Rev. Lett. 130, 098203 (2023)PRLTAO0031-900710.1103/PhysRevLett.130.098203]; (iii)at fixed L, the active term increases the stability of the homogeneous phase, eventually destroying the phase separation in the binodal for large but finite systems. Our results are particularly relevant for active matter systems in which the number of constituents hardly goes beyond 10^{7} and where finite-size effects matter.

Implementation of wavelet method and backpropagation neural network on road crack detection based on image processing

Road crack detection is critical to road infrastructure maintenance, requiring sophisticated and accurate approaches. This research explores the utilization of a combination of Wavelet and Convolutional Neural Network (CNN) methods to improve efficiency and accuracy in detecting cracks in road images. The wavelet method was chosen for its capability to capture information at different scales, enabling improved feature extraction. Meanwhile, CNN was utilized to comprehend the spatial context and tackle image complexity. The research involves several stages, including data collection, pre-processing, decomposition using the Wavelet method, forming of the CNN architecture model, training, testing, and evaluating the result. The tested images involve three main types of cracks: alligator, linear, and images without cracks. The testing results show that the developed model is capable of classifying cracks with an F1-score of 0.96, recall of 0.96, and precision of 0.96. In real-time detection of road cracks, the testing obtained an F1-score of 0.84, recall of 0.92, and precision of 0.77. This research contributes to the advancement of road crack detection technology by leveraging the capabilities of Wavelet and CNN, enhancing the accuracy and efficiency of crack detection in road maintenance.

Enhancing the expressivity of quantum neural networks with residual connections

In noisy intermediate-scale quantum era, the research on the combination of artificial intelligence and quantum computing has been greatly developed. Here we propose a quantum circuit-based algorithm to implement quantum residual neural networks, where the residual connection channels are constructed by introducing auxiliary qubits to data-encoding and trainable blocks in quantum neural networks. We prove that when this particular network architecture is applied to a l-layer data-encoding, the number of frequency generation forms extends from one, namely the difference of the sum of generator eigenvalues, to O(l2)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{{{{{{\\mathcal{O}}}}}}}}({l}^{2})$$\\end{document}, and the flexibility in adjusting Fourier coefficients can also be improved. It indicates that residual encoding can achieve better spectral richness and enhance the expressivity of various parameterized quantum circuits. Extensive numerical demonstrations in regression tasks and image classification are offered. Our work lays foundation for the complete quantum implementation of classical residual neural networks and offers a quantum feature map strategy for quantum machine learning.

Investigating resource-efficient neutron/gamma classification ML models targeting eFPGAs

There has been considerable interest and resulting progress in implementing machine learning (ML) models in hardware over the last several years from the particle and nuclear physics communities. A big driver has been the release of the Python package, hls4ml, which has enabled porting models specified and trained using Python ML libraries to register transfer level (RTL) code. So far, the primary end targets have been commercial field-programmable gate arrays (FPGAs) or synthesized custom blocks on application specific integrated circuits (ASICs). However, recent developments in open-source embedded FPGA (eFPGA) frameworks now provide an alternate, more flexible pathway for implementing ML models in hardware. These customized eFPGA fabrics can be integrated as part of an overall chip design. In general, the decision between a fully custom, eFPGA, or commercial FPGA ML implementation will depend on the details of the end-use application. In this work, we explored the parameter space for eFPGA implementations of fully-connected neural network (fcNN) and boosted decision tree (BDT) models using the task of neutron/gamma classification with a specific focus on resource efficiency. We used data collected using an AmBe sealed source incident on Stilbene, which was optically coupled to an OnSemi J-series silicon photomultiplier (SiPM) to generate training and test data for this study. We investigated relevant input features and the effects of bit-resolution and sampling rate as well as trade-offs in hyperparameters for both ML architectures while tracking total resource usage. The performance metric used to track model performance was the calculated neutron efficiency at a gamma leakage of 10-3. The results of the study will be used to aid the specification of an eFPGA fabric, which will be integrated as part of a test chip.

Recent Trends in Application of Memristor in Neuromorphic Computing: A Review

Abstract: Recently memristors have emerged as a form of nonvolatile memory that is based on the principle of ion transport in solid electrolytes under the impact of an external electric field. It is perceived as one of the key elements to building next-generation computing systems owing to its peculiar resistive switching characteristics. The switching mechanism in a memristor is mainly governed by filamentary conduction. Further, it can be employed as a memory as well as a logic element, which makes it an ideal candidate for building innovative computer architecture. Moreover, it is capable of mimicking the characteristics of biological synapses, which makes it an ideal candidate for developing a Neuromorphic system. In this review to begin with the switching mechanism of the memristor, primarily focusing on filamentary conduction, is discussed. Few SPICE models of memristor are reviewed, and their critical comparison is performed, which are widely used to build computing systems. An in-depth study on the various crossbar memory architecture augmented with memristors is reviewed. Finally, the application of memristors in neuromorphic computing and hardware implementation of Artificial Neural Networks (ANN) employing memristors is discussed.

Unveiling Anomalies in Surveillance Videos

Abstract: The implementation of Convolutional Neural Network (CNN) algorithm for the detection of anomalies like fire and potholes in images or videos. Leveraging deep learning technique, the model aims to accurately identify fire, potholes, etc. within various contexts. The CNN architecture is trained on a dataset comprising diverse images to enhance its ability to generalize. This proposed project not only addresses safety concerns by promoting helmet usage and proper maintenance of roads but also contributes to the broader field of computer vision and object detection. This implementation involves training the CNN on annotated datasets, fine-tuning the model for optimal performance, and integrating it into a practical application for efficient anomaly identification. With a focus on real-time detection, the system utilizes image processing techniques and a trained CNN model to analyze visual data from cameras or video feeds. The system's versatility allows integration into surveillance systems, industrial sites, or any environment where fire detection is crucial for safety. By automating the detection process, the project contributes to minimizing human error and ensuring consistent monitoring. The proposed solution holds the potential to significantly impact safety protocols, particularly in industries where protective headgear is paramount. Potholes pose a significant threat to both drivers and pedestrians, leading to accidents and infrastructure damage. The proposed solution leverages CNN's ability to effectively process visual data, making it well-suited for image recognition tasks. Our approach involves training the CNN on a diverse dataset of road images to enable it to accurately identify potholes. The model will learn distinctive features and patterns associated with potholes, allowing for robust detection under various environmental conditions. Real-time implementation on embedded systems or cameras along roadways will enable instantaneous identification and alerting.

Integrating selective flocculation techniques for enhanced efficiency in manufacturing processes: A novel approach through artificial neural network modeling

The use of medium and low-grade iron ore is gradually becoming more important due to the depletion of high-grade iron ore reserves and stringent environmental acts/rules. Slimes from iron ore washing were discarded in tailing dams; however, there is currently consideration for recovering iron values from ultra-fines as well. There are enormous fine dumps that are still unutilised. Hence, this study attempt to delve the optimization of iron ore slimes an indeed requirement for manufacturing and design in industries. Leveraging a flocculation process, coupled with the implementation of an Artificial Neural Network (ANN) predictive model, the Kiriburu processing plant serves as the primary source for iron ore slime samples. Chemical analyses of the collected iron samples reveal a composition featuring 58.24 % iron content, 3.47 % Al2O3, 4.72 % SiO2, and 5.18 % LOI (Loss on Ignition). The investigation explores the performance of the flocculation technique under varying pH levels, different pulp densities, and diverse flocculant dosages. Furthermore, the varying parameters selected are pH from 6 to 11, pulp density from 1 % to 15 %, and flocculant dose from 0.03 to 0.27 mg/g. The study's findings showcase a substantial improvement in the Fe grade of iron ore, escalating from 58.24 % to 66.12 %, with an impressive recovery rate of 82.54 % achieved using a flocculant dosage of 0.09 mg/g at pH 10. Additionally, a performance assessment of the selective flocculation method for iron ore slimes is conducted using an ANN predictive model, with recovery as the pivotal parameter. The input parameters for this model encompass pH, pulp density, and flocculant dosages. Employing a three-layer ANN model with a 3–3–1 architecture and utilizing feed-forward back propagation, the study demonstrates a close alignment between predicted values and experimental data, confirming the model's effectiveness for practical manufacturing applications. Information regarding the potential applications of the model's iron ore slime beneficiation efficacy for the manufacturing sector should be considered. This could entail lower waste, more effectiveness, or cost savings. Emphasise any possible ramifications for sustainability or the environment that would make the study pertinent in a larger perspective.

End-to-end codesign of Hessian-aware quantized neural networks for FPGAs

We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) hardware. Our approach leverages Hessian-aware quantization of NNs, the Quantized Open Neural Network Exchange intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA firmware. This makes efficient NN implementations in hardware accessible to nonexperts in a single open sourced workflow that can be deployed for real-time machine-learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40-MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on FPGA hardware within the strict area and latency requirements. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions.

A Study on h‐BN Resistive Switching Temporal Response

AbstractPrevious work that studied hexagonal boron nitride (h‐BN) memristor DC resistive‐switching characteristics is extended to include an experimental understanding of their dynamic behavior upon programming or synaptic weight update. The focus is on the temporal resistive switching response to driving stimulus (programming voltage pulses) effecting conductance updates during training in neural network crossbar implementations. Test arrays are fabricated at the wafer level, enabled by the transfer of CVD‐grown few‐layer (8 layer) or multi‐layer (18 layer) h‐BN films. A comprehensive study of their temporal response under various conditions–voltage pulse amplitude, edge rate (pulse rise/fall times), and temperature–provides new insights into the resistive switching process toward optimized devices and improvements in their implementation of artificial neural networks. The h‐BN memristors can achieve multi‐state operation through ultrafast pulsed switching (< 25 ns) with high energy efficiency (≈10 pJ pulse−1).

Designing machine learning based intelligent network for assessment of heat transfer performance of ternary hybrid nanofluid flow between a cone and a disk: Case of MLP feed forward neural network

In the current study, authors have studied the heat transfer through ternary hybrid nanofluid (THNF) between the gap of a disk and cone, where both are co-rotating with regard to the other. The authors have developed a mathematical model of THNF flow inside a gap between a cone and a disk with the effects of magnetic field, Hall effects, and radiation parameters. The unique aspect of this study is the implementation of a powerful artificial neural network (ANN) that exhibits improved robustness and accurate predictive performance for the Nusselt number at the surface of both the cone and disk. Through the use of proper similarity transformations, the model equations are transformed into a collection of nonlinear ODEs. Using MATLAB and an inbuilt bvp4c function, these equations are numerically solved. The main objective of this research is to examine the impacts of cone and disk rotation within the co-rotation system. The heat transfer rate is seen to be higher at the surface of the disk. It is followed from the analysis of the ANN model that the regression value for each case varies between 0.99574 and 1, and in many cases, it is exactly equal to 1. In addition, the mean square error (MSE) for every dataset is notably very close to zero. The effect of the angle of the conical gap is also one of the noteworthy subjects due to the significance of its effects in several engineering sector applications.

Multi-objective Evolutionary Neural Architecture Search for Recurrent Neural Networks

Artificial neural network (NN) architecture design is a nontrivial and time-consuming task that often requires a high level of human expertise. Neural architecture search (NAS) serves to automate the design of NN architectures and has proven to be successful in automatically finding NN architectures that outperform those manually designed by human experts. NN architecture performance can be quantified based on multiple objectives, which include model accuracy and some NN architecture complexity objectives, among others. The majority of modern NAS methods that consider multiple objectives for NN architecture performance evaluation are concerned with automated feed forward NN architecture design, which leaves multi-objective automated recurrent neural network (RNN) architecture design unexplored. RNNs are important for modeling sequential datasets, and prominent within the natural language processing domain. It is often the case in real world implementations of machine learning and NNs that a reasonable trade-off is accepted for marginally reduced model accuracy in favour of lower computational resources demanded by the model. This paper proposes a multi-objective evolutionary algorithm-based RNN architecture search method. The proposed method relies on approximate network morphisms for RNN architecture complexity optimisation during evolution. The results show that the proposed method is capable of finding novel RNN architectures with comparable performance to state-of-the-art manually designed RNN architectures, but with reduced computational demand.

Query-guided generalizable medical image segmentation

The practical implementation of deep neural networks in clinical settings faces hurdles due to variations in data distribution across different centers. While the incorporation of query-guided Transformer has improved performance across diverse tasks, the full scope of their generalization capabilities remains unexplored. Given the ability of the query-guided Transformer to dynamically adjust to individual samples, fulfilling the need for domain generalization, this paper explores the potential of query-based Transformer for cross-center generalization and introduces a novel Query-based Cross-Center medical image Segmentation mechanism (QuCCeS). By integrating a query-guided Transformer into a U-Net-like architecture, QuCCeS utilizes attribution modeling capability of query-guided Transformer decoder for segmentation in fluctuating scenarios with limited data. Additionally, QuCCeS incorporates an auxiliary task with adaptive sample weighting for coarse mask prediction. Experimental results demonstrate QuCCeS’s superior generalization on unseen domains.

Implementation of Backpropagation Artificial Neural Network for Electricity Load Forecasting in Jember District

The increase in population and various kinds of human activities in the world has made it possible for changes to increase the need for electrical power with demand that is not the same at any time. Based on this description, this research will propose research on the theme of electricity load forecasting as a preventive measure to determine future electricity load needs. Research was assisted using MATLAB data processing software to process research data. Three forecasting models were carried out, namely day, night and day-night conditions. From these three forecasting models, parameters such as epoch, number of input layers, number of hidden layers, activation function, and etc. The data is divided into two parts, training data and test data with a ratio of 70: 30. Test results using the backpropagation artificial neural network method show the highest MSE values for the three forecasting models, day, night, and day-night, are, 0.0039, 0.0041, and 0.002 while the lowest MSE values were in the three models are, 6.77E-04, 0.001, and 0.0011.

Advancing Environmental Sustainability through Implementation of Artificial Neural Networks for Wastewater Treatment Model Prediction and Remediation

Advancing Environmental Sustainability through Implementation of Artificial Neural Networks for Wastewater Treatment Model Prediction and Remediation