Parallel Artificial Neural Networks Research Articles

Optimisation Challenge for a Superconducting Adiabatic Neural Network That Implements XOR and OR Boolean Functions.

In this article, we consider designs of simple analog artificial neural networks based on adiabatic Josephson cells with a sigmoid activation function. A new approach based on the gradient descent method is developed to adjust the circuit parameters, allowing efficient signal transmission between the network layers. The proposed solution is demonstrated on the example of a system that implements XOR and OR logical operations.

Pseudograin decomposition of short fiber-reinforced plastics for two-step homogenization using machine learning approach

Decomposing the representative volume element (RVE) of short fiber-reinforced plastics (SFRPs) into several pseudograins (PGs) is essential for understanding its effective mechanical behavior. However, conventional PG decomposition methodologies are limited by their high computational costs due to iteration-based algorithms. To address this, we propose a machine learning-assisted PG decomposition procedure that utilizes a series–parallel artificial neural network (ANN) system to facilitate the time-consuming decomposition process. To validate the effectiveness of our proposal, we implemented a two-step homogenization framework of SFRP that consists of the series–parallel ANN system, Mori-Tanaka model, and Voigt model into ABAQUS user material subroutine (UMAT). The elastic modulus values predicted by the UMAT are found to be in good agreement with both DIGIMAT-MF and experimental values, while also maintaining low computational time.

Enhanced Mild-Slope Wave Model with Parallel Implementation and Artificial Neural Network Support for Simulation of Wave Disturbance and Resonance in Ports

Ensuring sea surface tranquility within port basins is of paramount importance for safe and efficient port operations and vessels’ accommodation. The present study aims to introduce a robust numerical model based on mild-slope equations, capable of accurately simulating wave disturbance and resonance in ports. The model is further enhanced by the integration of an artificial neural network (ANN) to address partial reflection, and its efficiency is optimized by developing a parallel algorithm based on OpenMP, allowing for a reduction in the required simulation times for real port areas spanning several kilometers horizontally. Numerous numerical experiments focusing on wave reflection against a vertical wall were conducted to develop the ANN. This neural network was designed to determine the appropriate value of the eddy viscosity coefficient, a crucial parameter in the momentum equation of the mild-slope model, tailored to incident wave characteristics. The model’s validity was confirmed through rigorous validation against experimental measurements, covering wave disturbance, rectangular harbor resonance, and Bragg resonance. The model consistently demonstrated a more than satisfactory performance across all considered scenarios. In a practical application, the model was deployed in the Port of Rethymno, Crete Island, Greece, effectively capturing and describing dominant phenomena within the port area. The implementation of a parallel algorithm significantly reduced the simulation times by ~92%, compared to the serial algorithm, thereby enhancing the model’s efficiency and applicability in real-world port environments.

How to achieve the fast computation for voxel-based irregular structures by few finite elements?

We propose machine learning (ML) based smart interpolation functions to enhance the finite element computation (named as smart-I finite element) for the heterogeneous structures (e.g., alloy material, concrete material and porous rock) that is composed of many microstructures known as voxel-based irregular shape (VI-shape). This element can directly describe the geometries with different VI-shapes and material properties without the complex meshing process by traditional finite element method (FEM). To achieve the smart interpolation in such element, a machine learning algorithm containing multiple parallel artificial neural networks (ANNs) and DeepONets assembled by the convolutional architectures is constructed for the fast prediction of interpolation functions. On this basis, the smart-I finite element is combined with classic FEM element to derive the global equilibrium equations and obtain the full-field variable distribution. Numerical tests with various material, geometrical as well as loading parameters prove that, the smart-I finite element enhanced FEM (SFEE-FEM) is able to conduct the accurate computation with a small number of elements, and thus greatly reduces the time cost. The smart-I finite element proposed by this work breaks through the limitation of traditional element shape with regular boundary (e.g., straight line), and may pave a new way to implement the FEM calculation on voxel-based geometries efficiently.

Electric vehicle battery pack state of charge estimation using parallel artificial neural networks

Electric vehicle battery pack state of charge estimation using parallel artificial neural networks

Anticipating gelation and vitrification with medium amplitude parallel superposition (MAPS) rheology and artificial neural networks

Anticipating qualitative changes in the rheological response of complex fluids (e.g., a gelation or vitrification transition) is an important capability for processing operations that utilize such materials in real-world environments. One class of complex fluids that exhibits distinct rheological states are soft glassy materials such as colloidal gels and clay dispersions, which can be well characterized by the soft glassy rheology (SGR) model. We first solve the model equations for the time-dependent, weakly nonlinear response of the SGR model. With this analytical solution, we show that the weak nonlinearities measured via medium amplitude parallel superposition (MAPS) rheology can be used to anticipate the rheological aging transitions in the linear response of soft glassy materials. This is a rheological version of a technique called structural health monitoring used widely in civil and aerospace engineering. We design and train artificial neural networks (ANNs) that are capable of quickly inferring the parameters of the SGR model from the results of sequential MAPS experiments. The combination of these data-rich experiments and machine learning tools to provide a surrogate for computationally expensive viscoelastic constitutive equations allows for rapid experimental characterization of the rheological state of soft glassy materials. We apply this technique to an aging dispersion of Laponite® clay particles approaching the gel point and demonstrate that a trained ANN can provide real-time detection of transitions in the nonlinear response well in advance of incipient changes in the linear viscoelastic response of the system.

High‐speed adaptive analog filter based on fully analog artificial neural network

SummaryThis paper presents an innovative concept of a high‐speed and higher order adaptive filter. A fully analog artificial neural network handles the adaption by using a filter bank for filtering and learning through analog feedback inspired by backpropagation. There is no clock control used in this concept, which makes real‐time adaption possible even for high frequencies. Its functionality is shown by several electrical circuit simulations involving component inaccuracies as well as a commonly used filter example.

Structural optimization design of machine tools based on parallel artificial neural networks and genetic algorithms

Structural optimization design of machine tools based on parallel artificial neural networks and genetic algorithms

Knowledge-Based Neural Network for Thinned Array Modeling With Active Element Patterns

To speed up the process for designing finite thinned arrays, an efficient artificial neural network (ANN) model considering mutual coupling effects is proposed in this communication. Array elements are divided into three categories based on the active element pattern (AEP) technique, and subarrays are constructed to extract the AEP of each category of elements as training samples. The proposed model including parallel ANN branches takes into account the mutual coupling between elements and avoids the calculation of the radiation pattern of the entire array. To improve modeling performance, furthermore, a prior knowledge input technique is used to reduce the complexity of the input–output relationship that an ANN has to learn. The existing knowledge is obtained by back propagation (BP) ANN modeling. Once the knowledge-based model is well trained, it is repeatedly called by the genetic algorithm (GA) for the optimal solution as a substitute for the full-wave simulation. Numerical examples of a U-shaped slot thinned array and a dual-layer patch thinned array are utilized to verify the efficiency of the proposed scheme.

Flow regime identification and gas volume fraction prediction in two-phase flows using a simple gamma-ray gauge combined with parallel artificial neural networks

Volume fraction and regime identification are considered as two main goals in the multiphase flow measurement. By developing the applications of artificial neural networks (ANN), new approaches have been introduced in the multiphase measurements. In the present work, prediction of gas volume fraction and identification of five different flow regimes including Bubble, Dispersed, Pluged, Annular, and Slug regimes were carried out using the simplest form of a radiation measurement system and proposing a simple structure of ANN including two independent MLP networks which are working in parallel. All data used for training and testing the networks were recorded using a simple radiation-based setup involving a137Cs gamma-ray source and one NaI(Tl) detector in the experimental setup with real dynamic conditions of fluids in a test loop. Extracted features from the recorded spectrum by the detector include full-energy peak and total counts. Overall results showed that all introduced flow regimes have been successfully identified by one of the proposed ANNs while GVFs have been predicted precisely using another one with a mean relative error of less than 3%.

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

An artificial neural network chip based on two-dimensional semiconductor

Recently, research on two-dimensional (2D) semiconductors has begun to translate from the fundamental investigation into rudimentary functional circuits. In this work, we unveil the first functional MoS2 artificial neural network (ANN) chip, including multiply-and-accumulate (MAC), memory and activation function circuits. Such MoS2 ANN chip is realized through fabricating 818 field-effect transistors (FETs) on a wafer-scale and high-homogeneity MoS2 film, with a gate-last process to realize top gate structured FETs. A 62-level simulation program with integrated circuit emphasis (SPICE) model is utilized to design and optimize our analog ANN circuits. To demonstrate a practical application, a tactile digit sensing recognition was demonstrated based on our ANN circuits. After training, the digit recognition rate exceeds 97%. Our work not only demonstrates the protentional of 2D semiconductors in wafer-scale integrated circuits, but also paves the way for its future application in AI computation.

Hybrid optical-electronic neural network with pseudoinverse learning for classification inference

Constructing optical neural networks as analog artificial neural networks becomes a research highlight of implementing machine learning in a physical system. However, the training of an optical neural network with backpropagation error correction requires a complicated and less-efficient computational process. Here, we introduce a hybrid optical-electronic neural network to produce efficient artificial learning of handwritten digits without the backpropagation process. In the proposal, one can illuminate an input image with incident light and map the input image to a feature vector according to the transmitted light intensity. One can then encode the feature vector as an array of light and use an optical matrix multiplication unit to multiply the feature vector by a learning weight matrix. This learning weight matrix is realized by the spatial light modulator, which is constructed from the pseudoinverse learning method. The output light intensity through the optical multiplication unit represents the recognition result of the input image. The proposed neural network without backpropagation achieves sufficient accuracy of handwritten digits classification, exposing the advantages of training speed acceleration and energy efficiency improvement.

Field-effect at electrical contacts to two-dimensional materials.

The inferior electrical contact to two-dimensional (2D) materials is a critical challenge for their application in post-silicon very large-scale integrated circuits. Electrical contacts were generally related to their resistive effect, quantified as contact resistance. With a systematic investigation, this work demonstrates a capacitive metal-insulator-semiconductor (MIS) field-effect at the electrical contacts to 2D materials: The field-effect depletes or accumulates charge carriers, redistributes the voltage potential, and gives rise to abnormal current saturation and nonlinearity. On one hand, the current saturation hinders the devices’ driving ability, which can be eliminated with carefully engineered contact configurations. On the other hand, by introducing the nonlinearity to monolithic analog artificial neural network circuits, the circuits’ perception ability can be significantly enhanced, as evidenced using a coronavirus disease 2019 (COVID-19) critical illness prediction model. This work provides a comprehension of the field-effect at the electrical contacts to 2D materials, which is fundamental to the design, simulation, and fabrication of electronics based on 2D materials. Electronic Supplementary MaterialSupplementary material (results of the simulation and SEM) is available in the online version of this article at 10.1007/s12274-021-3670-y.

Design of classifiers based on ANN approximations of traditional methods

AbstractIn this paper, we suggest basing the development of classification methods on traditional techniques but approximating them in whole or in part with artificial neural networks (ANNs). Compared to a direct ANN approach, the underlying traditional method is often easier to analyse. Classification failures can be better understood and corrected for, while at the same time faster execution can be obtained due to the parallel ANN structure. Furthermore, such a two‐step design philosophy partly eliminates the ‘guesswork’ associated with the design of ANNs. The expected gain is thus that the benefits from the traditional field, and the ANN field can be obtained by combining the best features from both fields. We illustrate our approach by working through an explicit example, namely, a Nearest Neighbour classifier applied to a subset of the MNIST database of handwritten digits. Two different approaches are discussed for how to translate the traditional method into an ANN. The first approach is based on a constructive implementation which directly reflects the original algorithm. The second approach uses ANNs to approximate the whole, or part of, the original method. An important part of the approach is to show how improvements can be introduced. In line with the presented philosophy, this is done by extending the traditional method in several ways followed by ANN approximation of the modified algorithms. The extensions are based on a windowed version of the nearest neighbour algorithm. We show that the improvements carry over to the ANN implementations. We further investigate the stability of the solutions by modifying the training set. It is shown that the errors do not change significantly. This also holds true for the ANN approximations providing confidence that the two‐step strategy is robust.

A power-efficient and re-configurable analog artificial neural network classifier

This paper presents a power-efficient and re-configurable radial basis function-artificial neural network (RBF-ANN) for real-time pattern classification tasks. We have developed a MATLAB-based behavioral model, which can be employed to facilitate the off-chip learning of the proposed network with a hybrid learning algorithm. Besides, a generic design methodology is presented to ease implementing different scales of neural networks. For the RBF-ANN, an analog tunable Gaussian kernel circuit is used as an activation neuron while a current-mode computation is employed to improve the power efficiency. The proposed network is designed and simulated in 0.18 μm X-FAB CMOS process for a non-linear XOR-pattern classification and voice activity detector (VAD), respectively. It is found that the network's weights obtained using the developed model in MATLAB are well-matched with the Spectre simulation (Virtuoso) results with a maximum relative error of 7.1%. Consequently, the proposed model can be effectively employed for the off-chip training of RBF-ANN. Besides, the preliminary simulation results of the VAD depict that the classification accuracy reaches as large as 95% in a noisy environment of babble with an acoustic signal to noise ratio (SNR) of 10 dB. Meanwhile, the power efficiency of the proposed network gets significantly improved compared to the recent prior arts.

Design and validation of an artificial neural network based on analog circuits

This paper focuses on the design and validation of an analog artificial neural network. Basic building blocks of the analog ANN have been constructed in UMC 90 nm device technology. Performance metrics of the building blocks have been demonstrated through circuit simulations. The weights of the ANN have been estimated through an automated back-propagation algorithm, which is running circuit simulations during weight optimization. Two case studies, the operation an XOR logic gate and a full adder circuit have been captured using the proposed analog ANN. Monte Carlo analysis of the XOR gate reveals that the analog ANN operates with an accuracy of 99.85%.

Utilizing historical data for corporate credit rating assessment

Corporate credit rating assessment is one of the crucial problems of credit risk management; it will help the financial institutions and government decide whether to issue debts. Recent studies focusing on the prediction of credit rating by using artificial intelligence (AI) techniques have shown impressive results compared to traditional statistical methods. Although the AI techniques can be used to assess credit risk, the prediction accuracy is still worth improving further, as even a small improvement in credit rating prediction accuracy leads to significant loss reduction in the industry. In this paper, we propose new learning analytic methods to enhance the prediction accuracy of credit rating. First, we devise the metrics based on the credit rating history of the firms, and expand the feature space with new input variables. This approach can be applied to any conventional AI methods for improvement of prediction accuracy. Second, we develop a novel learning algorithm that is designed to take into account historical financial data. We propose the parallel artificial neural networks (PANNs) ensemble model that creates several independent artificial neural networks (ANNs); each ANN deals with financial performance of the firms for each year, and the final output of PANNs is aggregated by ensemble learning. In our experiment, three different real-world datasets are used to validate the performance of our proposed approach. Consequently, the experimental results show that our proposed approach achieved competitive results compared to conventional AI techniques.

A Fully-Integrated Analog Machine Learning Classifier for Breast Cancer Classification

We propose a fully integrated common-source amplifier based analog artificial neural network (ANN). The performance of the proposed ANN with a custom non-linear activation function is demonstrated on the breast cancer classification task. A hardware-software co-design methodology is adopted to ensure good matching between the software AI model and hardware prototype. A 65 nm prototype of the proposed ANN is fabricated and characterized. The prototype ANN achieves 97% classification accuracy when operating from a 1.1 V supply with an energy consumption of 160 fJ/classification. The prototype consumes 50 μ W power and occupies 0.003 mm 2 die area.

Design and specification of analog artificial neural network

In this paper, we have implemented, using Matlab Simulink an analog artificial neural network for breast cancer classification. Simulated results with ideal building blocks exhibit a total error of classification of 2.6%. Thanks to this value, we have modified Simulink models of the building blocks (i.e. multiplier, activation function and its derivative) in order to take into account their non-idealities. This study allows to determine their influence on the classification quality and to extract some specifications of these building blocks.