Implementation Of Neural Networks Research Articles

An Optimised CNN Hardware Accelerator Applicable to IoT End Nodes for Disruptive Healthcare

In the evolving landscape of computer vision, the integration of machine learning algorithms with cutting-edge hardware platforms is increasingly pivotal, especially in the context of disruptive healthcare systems. This study introduces an optimized implementation of a Convolutional Neural Network (CNN) on the Basys3 FPGA, designed specifically for accelerating the classification of cytotoxicity in human kidney cells. Addressing the challenges posed by constrained dataset sizes, compute-intensive AI algorithms, and hardware limitations, the approach presented in this paper leverages efficient image augmentation and pre-processing techniques to enhance both prediction accuracy and the training efficiency. The CNN, quantized to 8-bit precision and tailored for the FPGA’s resource constraints, significantly accelerates training by a factor of three while consuming only 1.33% of the power compared to a traditional software-based CNN running on an NVIDIA K80 GPU. The network architecture, composed of seven layers with excessive hyperparameters, processes downscale grayscale images, achieving notable gains in speed and energy efficiency. A cornerstone of our methodology is the emphasis on parallel processing, data type optimization, and reduced logic space usage through 8-bit integer operations. We conducted extensive image pre-processing, including histogram equalization and artefact removal, to maximize feature extraction from the augmented dataset. Achieving an accuracy of approximately 91% on unseen images, this FPGA-implemented CNN demonstrates the potential for rapid, low-power medical diagnostics within a broader IoT ecosystem where data could be assessed online. This work underscores the feasibility of deploying resource-efficient AI models in environments where traditional high-performance computing resources are unavailable, typically in healthcare settings, paving the way for and contributing to advanced computer vision techniques in embedded systems.

Integrative physiology-coupled pilot-centered flight simulation

Maintaining the pilot’s physiological performance envelope within the limits of human capabilities may be crucial for avoiding hazardous physiological episodes in fighter aircraft that compromise safety. The main physiological episode of interest is impaired pilot respiration, better known as hypoxia, caused by a high fraction of inspired oxygen (FiO2) at high altitudes and variation in accelerative gravitational forces (g-forces). This research focuses on coupling FlightGear and Pulse Physiology Engine (Pulse) simulations to recreate and understand hypoxic events related to a combination of high g-forces and high-FiO2. By coupling interactive simulations based on FlightGear and Pulse, hypoxia was recreated from two scenarios: simulated accelerative atelectasis, achieved by using high g-forces output from FlightGear with a modified Pulse tension pneumothorax scenario, and the combination of high g-forces and high-FiO2, based on a prototype OBOGS simulation. Each scenario resulted in hypoxic events after executing three consecutive simulations, validated by the ratio mismatch of pulmonary ventilation ([Formula: see text]) and perfusion ([Formula: see text]) rates, [Formula: see text], due to a reduction in ventilation. Eventually, a detailed atelectasis simulation founded on multi-physics finite elements (FE) is planned in future work, where real-time efficiency is feasible through a deep neural network implementation trained on FE experiments. Using neural networks based on biometrics and pre-existing health conditions, if any, will result in the development of a predictive analytics model to determine whether pilots are susceptible to hypoxia prior to flight.

MolGraph: a Python package for the implementation of molecular graphs and graph neural networks with TensorFlow and Keras.

Molecular machine learning (ML) has proven important for tackling various molecular problems, such as predicting molecular properties based on molecular descriptors or fingerprints. Since relatively recently, graph neural network (GNN) algorithms have been implemented for molecular ML, showing comparable or superior performance to descriptor or fingerprint-based approaches. Although various tools and packages exist to apply GNNs in molecular ML, a new GNN package, named MolGraph, was developed in this work with the motivation to create GNN model pipelines highly compatible with the TensorFlow and Keras application programming interface (API). MolGraph also implements amodule to accommodate the generation of small molecular graphs, which can be passed to a GNN algorithm to solve a molecular ML problem. To validate the GNNs, benchmarking was conducted using the datasets from MoleculeNet, as well as three chromatographic retention time datasets. The benchmarking results demonstrate that the GNNs performed in line with expectations. Additionally, the GNNs proved useful for molecular identification and improved interpretability of chromatographic retention time data. MolGraph is available at https://github.com/akensert/molgraph . Installation, tutorials and implementation details can be found at https://molgraph.readthedocs.io/en/latest/ .

Deep learning-driven multi-omics sequential diagnosis with Hybrid-OmniSeq: Unraveling breast cancer complexity

Breast cancer results from an uncontrolled growth of breast tissue. Many methods of diagnosis are using multi-omics data to better understand the complexity of breast cancer. The new strategy laid out in this work, called “Hybrid-OmniSeq,” is a deep learning-based multi-omics data analysis technology that uses molecular subtypes of breast cancer gene to increase the precision and effectiveness of breast cancer diagnosis. For preprocessing, the BC-VM procedure is utilized, and for molecular subtype analysis, the BC-MSA procedure is utilized. The implementation of Deep Neural Network (DNN) technology in conjunction with Sequential Forward Floating Selection (SFFS) and Truncated Singular Value Decomposition (TSVD) entropy enable adaptive learning from multi-omics gene data. Five machine learning classifiers are used for classification purpose. Hybrid-OmniSeq uses a variety of machine learning classifiers in a thorough analytical process to achieve remarkable diagnostic accuracy. Deep Learning-based multi-omics sequential approach was evaluated using METABRIC RNA-seq data sets of intrinsic subtypes of breast cancer. According to test results, Logistic Regression (LR) had ER (Estrogen Receptor) status values of 94.51%, ER status values of 96.33%, and HER2 (Human Epidermal growth factor Receptor) status values of 92.3%; Random Forest (RF) had ER status values of 93.77%, ER status values of 95.23%, and HER2 status values of 93.4%. LR and RF increase the cancer detection accuracy for all subtypes when compared to alternative machine learning classifiers or the majority voting method, providing a comprehensive understanding of the underlying causes of breast cancer.

A Neural Controller Design for Enhancing Stability of a Single Machine Infinite Bus Power System

This paper examines the formulation and implementation of a neuro-controller for the excitation system of synchronous generators in a Single-Machine Infinite Bus (SMIB) power system. The SMIB model is employed as a fundamental model of a power system, thereby facilitating the assessment and comparison of disparate control strategies with the objective of enhancing system stability. The goal of this study is to enhance the stability of the SMIB power system through the implementation of an Artificial Neural Network (ANN) neuro-controller, providing a comparison of its performance to that of a Power System Stabilizer (PSS) and a Proportional-Integral-Derivative (PID) controller. The proposed neuro-controller will be integrated into the generator's excitation system and will be designed to regulate the excitation voltage in response to fluctuations in the system's operational parameters. To this end, an ANN is calibrated to account for the singularity of the generator's excitation level and terminal voltage. The Levenberg-Marquardt algorithm is employed to ascertain the optimal weight coefficients for the ANN. To assess the performance of the neuro-controller, simulations were conducted using MATLAB/Simulink. The simulations encompass a comprehensive range of operational scenarios, including diverse disturbances and alterations in the reference voltage level. Subsequently, the neuro-controller's outputs are evaluated in comparison to the PSS and PID controllers, as these are the prevailing controllers used to enhance voltage regulation and transient stability in power systems. This paper presents the results of an analysis of the neuro-controller's impact on the system's robustness, voltage variation amplitude, and generator dynamic performance during faults. Simulation results demonstrate that the application of an ANN-based neuro-controller yields superior outcomes in voltage regulation and transient stability compared to the conventional controllers PSS and PID. Furthermore, the neuro-controller is distinguished by accelerated response times and enhanced precision in voltage level regulation. The neuro-controller represents a superior approach to the control of a power system, particularly in the context of SMIB, which would ultimately result in enhanced performance and stability.

Backpropagation Shuffled Leaping Neural Network Implementation in Classification of DDoS Packet Flow Traffic in Data Mining

Backpropagation Shuffled Leaping Neural Network Implementation in Classification of DDoS Packet Flow Traffic in Data Mining

State-of-the-Art Deep Learning CT Reconstruction Algorithms in Abdominal Imaging.

The implementation of deep neural networks has spurred the creation of deep learning reconstruction (DLR) CT algorithms. DLR CT techniques encompass a spectrum of deep learning-based methodologies that operate during the different steps of the image creation, prior to or after the traditional image formation process (eg, filtered backprojection [FBP] or iterative reconstruction [IR]), or alternatively by fully replacing FBP or IR techniques. DLR algorithms effectively facilitate the reduction of image noise associated with low photon counts from reduced radiation dose protocols. DLR methods have emerged as an effective solution to ameliorate limitations observed with prior CT image reconstruction algorithms, including FBP and IR algorithms, which are not able to preserve image texture and diagnostic performance at low radiation dose levels. An additional advantage of DLR algorithms is their high reconstruction speed, hence targeting the ideal triad of features for a CT image reconstruction (ie, the ability to consistently provide diagnostic-quality images and achieve radiation dose imaging levels as low as reasonably possible, with high reconstruction speed). An accumulated body of evidence supports the clinical use of DLR algorithms in abdominal imaging across multiple CT imaging tasks. The authors explore the technical aspects of DLR CT algorithms and examine various approaches to image synthesis in DLR creation. The clinical applications of DLR algorithms are highlighted across various abdominal CT imaging domains, with emphasis on the supporting evidence for diverse clinical tasks. An overview of the current limitations of and outlook for DLR algorithms for CT is provided. ©RSNA, 2024.

Physics-consistent input convex neural network-driven reinforcement learning control for multi-zone radiant ceiling heating and cooling systems: An experimental study

Radiant ceiling heating and cooling system is a technology used for space heating and cooling. Owing to the variable weather conditions, occupant behavior, and thermal lag of the system, it is challenging to design a control strategy to reduce air-conditioning energy consumption while maintaining the thermal environment. This study is the first pilot implementation of a physics-consistent input convex neural network (PCICNN)-driven reinforcement learning (RL) approach for real-world multi-zone radiant ceiling heating and cooling systems. A multi-zone PCICNN based on a graph neural network (GNN) was developed to simulate the zone temperature. The radiant panel load was simulated using the physics-based ε-NTU method. The PCICNN-driven RL agent was based on the soft actor-critic algorithm and trained in an environment model comprising the PCICNN and ε-NTU models. The proposed controller was deployed in real-time on one floor of an office building for one month. The real-world implementation showed that the proposed PCICNN-driven RL control can reduce the radiant panel cooling load by up to 33% compared with the inherited baseline control strategy under similar weather conditions. This study provides a comprehensive demonstration of real-world data-driven building controls and leverages future research on advanced building control.

Dual‐Layer‐Per‐Array Operation Using Local Polarization Switching of Ferroelectric Tunnel FETs for Massive Neural Networks

AbstractA novel dual‐layer‐per‐array (DLPA) operation is proposed and experimentally demonstrated by using ferroelectric tunnel field‐effect transistors (FeTFETs) as synaptic transistors for high‐density and low‐power neuromorphic computing applications for the first time. Through local polarization switching (LPS), two distinct synaptic weight sets are stored and processed by utilizing the symmetrical and independent forward and ambipolar current regions within a single FeTFET. The stored weights of each region are multiplied by input gate voltages in the two‐layer operation modes of TFETs: forward and ambipolar layer, enabling separate vector‐matrix multiplication (VMM) operations within a single device array and doubling computational density. It is expected that the DLPA operation of FeTFETs will facilitate the implementation of large neural networks by reducing the footprint of conventional neuromorphic hardware by 50%.

Emergent Self-Adaptation in an Integrated Photonic Neural Network for Backpropagation-Free Learning.

Plastic self-adaptation, nonlinear recurrent dynamics and multi-scale memory are desired features in hardware implementations of neural networks, because they enable them to learn, adapt, and process information similarly to the way biological brains do. In this work, these properties occurring in arrays of photonic neurons are experimentally demonstrated. Importantly, this is realized autonomously in an emergent fashion, without the need for an external controller setting weights and without explicit feedback of a global reward signal. Using a hierarchy of such arrays coupled to a backpropagation-free training algorithm based on simple logistic regression, a performance of 98.2% is achieved on the MNIST task, a popular benchmark task looking at classification of written digits. The plastic nodes consist of silicon photonics microring resonators covered by a patch of phase-change material that implements nonvolatile memory. The system is compact, robust, and straightforward to scale up through the use of multiple wavelengths. Moreover, it constitutes a unique platform to test and efficiently implement biologically plausible learning schemes at a high processingspeed.

An IoT‐Based 5G Wireless Sensor Network Employs a Secure Routing Methodology Leveraging DCNN Processing

ABSTRACTThe security principles of the fifth generation (5G) are anticipated to include robust cryptography models, information security models, and machine learning (ML) powered Intrusion Detection Systems (IDS) specifically designed for Internet of Things (IoT) based wireless sensor networks (WSNs). Nevertheless, the existing security models fall short in addressing the dynamic network characteristics of WSNs. In this context, the suggested system introduces a secure and collaborative multi‐watchdog system through the implementation of deep convolutional neural network (DCNN) and distributed particle filtering evaluation scheme (DPFES). The proposed system utilizes deep learning (DL) techniques to create a dynamic multi‐watchdog system that safeguards each sensor node by monitoring its transmissions. Furthermore, the proposed approach includes secure data‐centric and node‐centric evaluation methods that are crucial for enhancing the security of 5G‐based IoT‐WSN networks. The network evaluation processes based on DL facilitate the creation of a secure multi‐watchdog system within dense IoT‐WSN environments. This system enables the deployment of active watchdog IDS agents as needed. The proposed approach includes various components such as a system dynamics model, cooperative watchdog model, Dual Line Minimum Connected Dominating Set (DL‐MCDS), and DL‐based event analysis procedures. From a technical perspective, the system is driven by the implementation of DPFES, which utilizes particle filtering frameworks to analyze network events and establish a secure 5G environment. The system has been successfully implemented, and its results have been compared with those of other similar works. The performance of the proposed cooperative multi‐watchdog system demonstrates a significant improvement of and compared to other techniques.

Deep Neural Network Implementation for Appliances Identification by Using Current and Voltage Signatures

Deep Neural Network Implementation for Appliances Identification by Using Current and Voltage Signatures

Enhancing Uniformity, Read Voltage Margin, and Retention in Three-Dimensional and Self-Rectifying Vertical Pt/Ta2O5/Al2O3/TiN Memristors.

This study introduces a Ta2O5-based self-rectifying memristor (SRM) with an Al2O3 interfacial layer adopted to improve switching uniformity, read voltage margin, and long-term retention. The Pt/Ta2O5/Al2O3/TiN (PTAT) device exhibits a 105 rectification ratio, 104 on/off ratio, 2 × 106 endurance, and retention of 104 s at 150 °C. A 3-layer 4 × 4 vertical resistive random access memory structure exhibits uniform switching parameters. The coefficient of variation (CV) for device-to-device measurements is 0.23 for the low resistance state (LRS) and 0.22 for the high resistance state (HRS), while for cycle-to-cycle measurements, the CV is 0.38 for the LRS and 0.11 for the HRS. Finally, the present study demonstrates the superior performance of the PTAT devices in the context of hardware-aware training for a fully connected neural network implementation. These advancements position the PTAT device as a promising candidate for high-density three-dimensional storage class memory and low-power neural networks, offering the consistent performance and reliability necessary for future high-density storage applications.

Computational study with neural networks to double diffusion in Prandtl thermal nanofluid flow adjacent to a stretching surface design with numerical treatment

The study focuses on the implementation of neural networks in analyzing the behavior of Prandtl nanofluid near a extending surface in the existence of dual diffusion. The research investigates the merged consequence of thermal and concentration gradients on the fluid flow and heat transfer characteristics using advanced computational techniques. The investigation delves into the phenomenon of double diffusion in the flow of Prandtl nanofluid near a stretching surface (DD-PNSS), employing the Levenberg-Marquardt scheme with Artificial Neural Networks (LMS-ANNs). The application of similarity variables converts the non-linear partial differential equations into non-linear ordinary differential equations. Through the application of the Lobatto IIIa formula in a three-stage process, various data sets are generated for the LMS-ANNs by varying parameters such as the Prandtl fluid parameter ([Formula: see text]), Prandtl number (Pr), Brownian motion parameter (Nb), thermophoresis parameter (Nt), and Dufour-solutal Lewis number (Ld). The range of parameter α is from 2 to 3.5, parameter β from 0.1 to 0.9, Nb is from 0.1 to 0.4, Nt is from 0.1 to 0.7, Le is from 1 to 3 and Ld is from 0.1 to 0.5. The intrinsic behavior of embedded parameters on dimensionless velocity, temperature, solutal concentration, and nanoparticle concentration profiles is graphically evaluated. The proposed LMS-ANNs model is meticulously tested, validated, and trained using a multi-stage approach, and its performance is compared to established references to ensure its reliability. The effectiveness of the suggested LMS-ANNs model is further affirmed through regression analysis, Mean Squared Error (MSE) evaluation, and histogram studies, showcasing an exceptional accuracy level ranging from 10−08 to 10−10, setting it apart from alternative approaches and reference models.

Implementation of Distributed Machine Learning in Cloud Calculation: Evidence from Consumer Behavior Prediction

Abstract. In the field of machine learning, distributed machine learning (DML) has gained massive popularity in research and appeared in a wide range of applications including transportation, medicine, and business. Distributed machine learning showed huge potential in processing large amounts of data while keeping the data private. In this paper, a distributed implementation of a deep neural network (DNN) with a large dataset from a telecom company, aiming to predict consumer behavior based on usage data is carried out. Various datasets with distinct heterogeneity settings were applied to the network, simulating a real-world cloud application scenario. Comparisons between both the centralized model and the distributed models were made to analyze the impact on model performance with device heterogeneity and data heterogeneity. These result indicate that under certain circumstances, DML models showed better accuracy and lower loss compared to the centralized model. The paper provided some insights and made some assumptions on how the data and the parameters affect the model performance.

The backpropagation algorithm implemented on spiking neuromorphic hardware

The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.

Anisotropic induced polarization modeling with neural networks and effective medium theory

Interpreting three-dimensional induced polarization (IP) data requires rock physics models that reflect the omnipresent anisotropy of the Earth’s crust. The Generalized Effective Medium Theory of Induced Polarization (GEMTIP) can model the IP signatures of rocks with polarizable mineral inclusions. However, it is computationally demanding because it requires numerically solving the depolarization tensors of each inclusion. We aim to streamline GEMTIP simulations by (1) integrating anisotropic background conductivity and triaxial ellipsoidal inclusions in the model and (2) estimating the depolarization tensors with a neural network. We validate the neural network predictions against known solutions for spherical and spheroidal inclusions, and we test our method using data from an actual rock sample. The neural network shows a relative sensitivity of 56% to inclusion shape and 44% to host rock anisotropy and is up to 100,000 times faster than numerical integration. We release the pre-trained neural network implementation as an open-source Python package, thereby providing a new method to interpret the IP signatures of anisotropic rocks.

OPTIMIZING AUTHENTICATION SECURITY IN INTELLIGENT SYSTEMS THROUGH VISUAL BIOMETRICS FOR ENHANCED EFFICIENCY

Context. The primary objective of this article is to explore aspects related to ensuring security and enhancing the efficiency of authentication processes in intelligent systems through the application of visual biometrics. The focus is on advancing and refining authentication systems by employing sophisticated biometric identification methods. Objective. A specialized intelligent system has been developed, utilizing a Siamese neural network to establish secure user authentication within the existing system. Beyond incorporating fundamental security measures such as hashing and secure storage of user credentials, the contemporary significance of implementing two-factor authentication is underscored. This approach significantly fortifies user data protection, thwarting most contemporary hacking methods and safeguarding against data breaches. The study acknowledges certain limitations in its approach, possibly affecting the generalizability of the findings. These limitations provide avenues for future research and exploration, contributing to the ongoing evolution of authentication methodologies in intelligent systems. Method. The two-factor authentication system integrates facial recognition technology, employing visual biometrics for heightened security compared to alternative two-factor authentication methods. Various implementations of the Siamese neural network, utilizing Contrastive loss function and Triplet loss function, were evaluated. Subsequently, a neural network employing the Triplet loss function was implemented and trained. Results. The article emphasizes the practical implications of the developed intelligent system, showcasing its effectiveness in minimizing the risk of unauthorized access to user accounts. The integration of contemporary authentication methodologies ensures a secure and robust user authentication process. Conclusions. The implementation of facial recognition technology in authentication processes has broader social implications. It contributes to a more secure digital environment by preventing unauthorized access, ultimately safeguarding user privacy and data. The study’s originality lies in its innovative approach to authentication, utilizing visual biometrics within a Siamese neural network framework. The developed intelligent system represents a valuable contribution to the field, offering an effective and contemporary solution to user authentication challenges.

Classification of Pneumonia via a Hybrid ZFNet-Quantum Neural Network Using a Chest X-ray Dataset.

Deep neural networks (DNNs) have made significant contributions to diagnosing pneumonia from chest X-ray imaging. However, certain aspects of diagnosis and planning can be further enhanced through the implementation of a quantum deep neural network (QDNN). Therefore, we introduced a technique that integrates neural networks with quantum algorithms named the ZFNet-quantum neural network for detecting pneumonia using 5863 X-ray scans with binary cases. The hybrid model efficiently pre-processes complex and high-dimensional data by extracting significant features from the ZFNet model. These significant features are given to the quantum circuit algorithm and further embedded into a quantum device. The parameterized quantum circuit algorithm using qubits, superposition theorem, and entanglement phenomena generates 4 features from 4098 features extracted from images via a deep transfer learning model. Moreover, to validate the outcome measures of the proposed technique, we used various PennyLane quantum devices to detect pneumonia and normal control images. By using the Adam optimizer, which exploits an adaptive learning rate that is fixed to 10-6 and six layers of a quantum circuit composed of quantum gates, the proposed model achieves an accuracy of 96.5%, corresponding to 25 epochs. The integrated ZFNet-quantum learning network outperforms the deep transfer learning network in terms of testing accuracy, as the accuracy gained by the convolutional neural network (CNN) is 94%. Therefore, we use a hybrid classical-quantum model to detect pneumonia in which a variational quantum algorithm enhances the outcomes of a ZFNet transfer learning method. This approach is an efficient and automated method for detecting pneumonia and could significantly enhance outcome measures related to the speed and accuracy of the network in the clinical and healthcare sectors.

Delayed-feedback oscillators replicate the dynamics of multiplex networks: Wavefront propagation and stochastic resonance

The widespread development and use of neural networks have significantly enriched a wide range of computer algorithms and promise higher speed at lower cost. However, the imitation of neural networks by means of modern computing substrates is highly inefficient, whereas physical realization of large scale networks remains challenging. Fortunately, delayed-feedback oscillators, being much easier to realize experimentally, represent promising candidates for the empirical implementation of neural networks and next generation computing architectures. In the current research, we demonstrate that coupled bistable delayed-feedback oscillators emulate a multilayer network, where one single-layer network is connected to another single-layer network through coupling between replica nodes, i.e. the multiplex network. We show that all the aspects of the multiplexing impact on wavefront propagation and stochastic resonance identified in multilayer networks of bistable oscillators are entirely reproduced in the dynamics of time-delay oscillators. In particular, varying the coupling strength allows suppressing and enhancing the effect of stochastic resonance, as well as controlling the speed and direction of both deterministic and stochastic wavefront propagation. All the considered effects are studied in numerical simulations and confirmed in physical experiments, showing an excellent correspondence and disclosing thereby the robustness of the observed phenomena.