Hyperbolic Tangent Activation Function Research Articles

Predicting and Analyzing Electric Bicycle Adoption to Enhance Urban Mobility in Belgrade Using ANN Models

The development of alternative environmentally friendly modes of transportation is becoming an increasingly promising solution in traffic-congested and polluted urban areas. E-bikes, as one of them, are recognized as an ecologically sustainable means of transportation that has significant potential to replace motorized modes of transportation that can improve urban mobility. Relying on artificial intelligence and considering an ecological approach when considering the acceptability of e-bikes by setting a direct question for users influences the development of an innovative way of understanding and evaluating the use of more sustainable modes of transportation. In this regard, this study aims to elucidate the main variables influencing the acceptability of e-bike use using artificial neural network (ANN) models—multilayer perceptron (MLP) and radial basis function (RBF). For training and testing the models, data from a random sample obtained through an online questionnaire, which was answered by 626 residents of Belgrade (Serbia), were used. A multilayer perceptron with nine and seven neurons in two hidden layers, respectively, hyperbolic tangent activation function in the hidden layer and identity function in the output layer, gave better results than the radial basis function model. With an accuracy of 89%, a precision of 83%, a recall of 79%, and an area under the receiver operating characteristic (ROC) curve of 0.927, the multilayer perceptron model recognized the influential variables in predicting acceptability. The results of the model indicate that the mileage traveled, the frequency of motorcycle use, the respondents’ awareness of the pollution in Belgrade, and the age of the respondents have the greatest influence on the acceptability of using e-bikes. In addition to majority acceptability (69.8%), the results obtained by the model can represent a useful basis for decision-makers when defining strategies for the development and application of e-bikes while reducing traffic congestion and environmental pollution in Belgrade.

Mobile Robot Motion Behavior Based on Hopfield Neural Network Dynamics Under Electromagnetic Radiation

The nonlinear characteristic of activation function is a critical aspect that affects the performance of artificial neural networks (ANNs). Despite its importance, the impact of neuronal nonlinearity variations on the dynamics of ANNs has not been thoroughly studied in previous research. This paper aims to fill this gap by exploring the influence of nonlinearity changes in activation function due to memristive electromagnetic induction on the dynamical characteristics of ANNs. We first propose a memristor model with sinusoidal conductance function. Then, the effects on the biological neuron under the electromagnetic induction are simulated by replacing the gradient of hyperbolic tangent activation function with the conductance of the proposed memristor. Furthermore, a Hopfield neural network (HNN) is constructed by taking the nonlinearity changes of activation function into consideration and its dynamical behaviors are analyzed through Lyapunov exponent spectra, bifurcation diagram, biparameter-based dynamic maps and phase portraits. The numerical simulation results show complex dynamical behaviors such as single-scroll chaotic spiral attractor, double-scroll chaotic attractors and initial-value-dependent offset boosted attractors, demonstrating that nonlinearity changes of activation function induced by the electromagnetic induction have a significant impact on the dynamics of HNN. To further verify our findings, we implement the proposed HNN in hardware and find that the results are in line with our numerical simulations. Finally, we apply the chaotic HNN in the mobile robot path planning task, which achieves higher coverage rate than the random number-based random walk method.

An improved long short-term memory with denoising autoencoder for solving text classification problems

In the past decade, text classification has been a popular research field for automatically categorizing documents into relevant categories. Predicting classes for input samples based on their features is the goal of classification. To limit the dimension of the feature space for precise text categorization, feature selection approaches are widely used for recognizing important and affective features while ignoring unimportant, weak, and chaotic components. This paper proposes a new model referred to as DAE-LSTM for reducing data dimension through the use of a denoising autoencoder. (Long Short-Term Memory (LSTM) is one of the popular text classification approaches, particularly when dealing with sequential datasets. Additionally, the Rectified Linear Unit (Relu) activation function was used instead of the hyperbolic tangent activation function (tanh) to improve the accuracy of the model. Four benchmark datasets were used to evaluate the proposed model against 4 cutting-edge text classification methods, including conventional Bidirectional LSTM, Bidirectional GRU, CNN-LSTM, and CNN-GRU. The suggested model has been found to perform much better than current methods using several kinds of performance assessment measures.

Synthesizing Lithuanian voice replacement for laryngeal cancer patients with Pareto-optimized flow-based generative synthesis network

This study presents a Pareto optimized flow-based generative network for speech synthesis - the P-GLOW model in Lithuanian speech synthesis for substituting original voices affected by cancer-related pathologies. Comparing this pure Lithuanian model with an English model trained on Lithuanian phonemes, the study emphasizes the impact of language-specific characteristics on performance, illustrating the advantages of tailored, language-specific models for speech substitution for impaired persons. Our methodology integrates generative models with an architecture based on 12 coupling layers, 1x1 invertible convolutions, and dilated convolutions with gated hyperbolic tangent activation functions. This design facilitates the effective transformation of waveform inputs into Mel spectrograms, enabling the synthesis of high-quality speech audio. The proposed, Pareto optimized, native Lithuanian model achieved a MOS of 4.2 ± 0.1 and an SMOS of 3.9 ± 0.2, while the baseline model (English language model trained on top with Lithuanian phonemes) scored lower with a MOS of 3.8 ± 0.2 and an SMOS of 3.4 ± 0.3. In terms of the Mel Cepstral Distortion (MCD), Voiced/Unvoiced Decision Error (VDE), Glottal-to-Noise Excitation Ratio (GPE), and Frame Fidelity Error (FFE) scores, the proposed method achieved an MCD of 3.2 dB ± 0.2, a VDE of 5.6% ± 1.0, a GPE of 7.9% ± 1.4, and an FFE of 9.7% ± 2.0, outperforming the baseline method which scored higher with an MCD of 4.6 dB ± 0.3, a VDE of 10.8% ± 1.9, a GPE of 14.5% ± 2.4, and an FFE of 18.7% ± 2.8. Log Likelihood Ratio (LLR), Weighted Spectral Slope (WSS), and Perceptual Evaluation of Speech Quality (PESQ) scores also showed strong improvement. The proposed method achieved an LLR of 0.5 ± 0.05, a WSS of 0.3 ± 0.03, and a PESQ of 4.2 ± 0.1, while the baseline method scored lower with an LLR of 0.8 ± 0.08, a WSS of 0.5 ± 0.05, and a PESQ of 3.6 ± 0.2. The Speech Intelligibility Index (SII) and Short-Time Objective Intelligibility (STOI) metrics for restored alaryngeal speech were also high. The proposed method achieved an SII of 0.75 ± 0.05 and an STOI of 0.85 ± 0.03, while the baseline method scored lower with an SII of 0.65 ± 0.05 and an STOI of 0.75 ± 0.04. The evaluation results demonstrate that our approach was able to effectively synthesize high-quality Lithuanian speech audio restoring the input signal from distorted alaryngeal speech.

Comparative Analysis of the Predictive Performance of an ANN and Logistic Regression for the Acceptability of Eco-Mobility Using the Belgrade Data Set

To solve the problem of environmental pollution caused by road traffic, alternatives to vehicles with internal combustion engines are often proposed. As such, eco-mobility microvehicles have significant potential in the fight against environmental pollution, but only on the condition that they are widely accepted and that they replace the vehicles that predominantly pollute the environment. With this in mind, this study aims to elucidate the main variables that influence the acceptability of these vehicles, using prediction models based on binary logistic regression and a multilayer artificial neural network—a multilayer perceptron (ANN). The data of a random sample obtained via an online questionnaire, answered by 503 inhabitants of Belgrade (Serbia), were used for training and testing the model. A multilayer perceptron with 9 and 7 neurons in two hidden layers, a hyperbolic tangent activation function in the hidden layer, and an identity function in the output layer performed slightly better than the binary logistic regression model. With an accuracy of 85%, a precision of 79%, a recall of 81%, and an area under the ROC curve of 0.9, the multilayer perceptron model recognized the influential variables in predicting acceptability. The results of the model indicate that a respondent’s relationship to their current environmental pollution, the frequency of their use of modes of transport such as bicycles and motorcycles, their mileage for commuting, and their personal income have the greatest influence on the acceptability of using eco-mobility vehicles.

USE OF MACHINE LEARNING, FIXED AND MIXED MODELS FOR VOLUME ESTIMATION IN FLOODPLAIN FOREST IN THE AMAZON ESTUARY

The present study aimed to evaluate the performance of artificial neural networks (ANNs), support vector machines (SVM), and fixed and mixed models to estimate the volume of individual trees in an Amazonian estuary floodplain forest. The forest inventory was conducted in an area located in the district of Itatupã - Gurupá, Pará, Brazil. The Schumacher and Hall linear model was used in its fixed and mixed form to estimate the volume. A total of 240 ANNs and 32 SVM configurations were trained, with 4 variations of the input variables: diameter at breast height (D), commercial height (hm), diameter classes (CCD), commercial height classes (CChm),) and species (SP), with the volume (V) being the output variable. The ANNs were trained in the Neuro 4.0.6 software program, and the analyses of the fixed and mixed regression models and SVM were performed in the R software program. The AIC (Akaike’s Information Criterion) and BIC (Bayesian Information Criterion) information criteria were used for the regression models, while the following were used for comparisons of all models: correlation coefficient (rYŶ), bias, root mean squared error (RMSE), and residual distribution analysis. The best statistical metrics were obtained by the ANN of the RPROP- algorithm with D+hm+CCD+CChm inputs, with eight neurons in the hidden layer and hyperbolic tangent activation function, with 0.9827 of rYŶ RMSE of 0.2439, bias of 0.0080 and residual graphic distribution tending to homogeneity.

Prediction for nonlinear time series by improved deep echo state network based on reservoir states reconstruction

With the aim to enhance prediction accuracy for nonlinear time series, this paper put forward an improved deep Echo State Network based on reservoir states reconstruction driven by a Self-Normalizing Activation (SNA) function as the replacement for the traditional Hyperbolic tangent activation function to reduce the model’s sensitivity to hyper-parameters. The Strategy was implemented in a two-state reconstruction process by first inputting the time series data to the model separately. Once, the time data passes through the reservoirs and is activated by the SNA activation function, the new state for the reservoirs is created. The state is input to the next layer, and the concatenate states module saves. Pairs of states are selected from the activated multi-layer reservoirs and input into the state reconstruction module. Multiple input states are transformed through the state reconstruction module and finally saved to the concatenate state module. Two evaluation metrics were used to benchmark against three other ESNs with SNA activation functions to achieve better prediction accuracy.

Echo State Property upon Noisy Driving Input

The echo state property (ESP) is a key concept for understanding the working principle of the most widely used reservoir computing model, the echo state network (ESN). The ESP is achieved most of the operation time under general conditions, yet the property is lost when a combination of driving input signals and intrinsic reservoir dynamics causes unfavorable conditions for forgetting the initial transient state. A widely used treatment, setting the spectral radius of the weight matrix below the unity, is not sufficient as it may not properly account for the nature of driving inputs. Here, we characterize how noisy driving inputs affect the dynamical properties of an ESN and the empirical evaluation of the ESP. The standard ESN with a hyperbolic tangent activation function is tested using the MNIST handwritten digit datasets at different additive white Gaussian noise levels. The correlations among the neurons, input mapping, and memory capacity of the reservoir nonlinearly decrease with the noise level. These trends agree with the deterioration of the MNIST classification accuracy against noise. In addition, the ESP index for noisy driving input is developed as a tool to help easily assess ESPs in practical applications. Bifurcation analysis explicates how the noise destroys an asymptotical convergence in an ESN and confirms that the proposed index successfully captures the ESP against noise. These results pave the way for developing noise-robust reservoir computing systems, which may promote the validity and utility of reservoir computing for real-world machine learning applications.

Laboratory measuring of contact angles for liquid/solid surface and analysis with artificial neural network

AbstractIn this study, the advancing and receding contact angles of three types of pure liquid on four different surfaces have been measured. The contact angles of the droplet have been measured by analysing the captured photo by tilted plate method. Droplet behaviour has been modelled by two experimental equations. Additionally, the data has been modelled by an artificial neural network using genetic algorithm and a hyperbolic tangent activation function. The input parameters are density, molecular weight of pure liquid and solid, viscosity and surface tension of pure liquid, roughness of the solid surface and the two outputs are advancing and receding contact angles of the droplet. Number of 81 data points were used for training, 27 data for validation and 28 data for testing. The topography of {7,7,2} for artificial neural network has been proposed. The resulting RMS errors were 8%, 8% and 7% for training, validation and testing, respectively.

Modeling of multicomponent adsorption equilibria of phenol and ciprofloxacin on pristine, acid-modified and thermo-oxidatively-aged polyethylene terephthalate microplastics

Abstract Multicomponent systems are representative of the most common real situations as many industrial discharges contain a mixture of several pollutants. This study examines the concurrent adsorption of phenol (PHE) and ciprofloxacin (CIP) onto three types of polyethylene terephthalate microplastics (PET MPs): pristine, acid-modified, and thermal-oxidatively aged. Using extended Langmuir (EL), extended Freundlich (EF) isotherms, and a new artificial neural network (ANN) model, equilibrium adsorption capacities were predicted. The EL isotherm fit for pristine and aged PET MPs, while EF fit for modified PET MPs. Monolayer adsorption capacities ranged from 342.10–3715.73 mg/g for PHE and 2518.23–14498.79 mg/g for CIP, exceeding single-component adsorption. The ANN model used one hidden layer with 3 neurons for pristine and aged PET MPs, and 2 hidden layers with five neurons for modified PET MPs, with a hyperbolic tangent activation function. Models showed excellent performance metrics, including R2 values of 0.989–0.999, RMSE of 0.001–0.413, and AAE of 0.009–0.327. Synergistic interactions were observed in the binary system, with PET MPs showing higher selectivity toward CIP. The study demonstrates the effectiveness of PET MPs for binary adsorption of PHE and CIP in aqueous solutions, highlighting their potential for multicomponent pollutant removal.

Multilayer Perceptron Artificial Neural Networks and Tree Models as Multifactorial Binary Predictors of Heart Disease and Failure

BACKGROUND: Cardiovascular diseases (CVDs) are a significant global health concern, causing an estimated 17.9 million deaths annually, which represents 31% of worldwide deaths. A significant proportion of CVD deaths are due to heart attacks and strokes, with one-third of these deaths occurring prematurely in individuals under 70 years old. Heart failure is a notable event within CVDs and emerges when the heart cannot efficiently pump blood to fulfill the body's requirements. This complex syndrome's origins are multifactorial and often arise from conditions such as hypertension, diabetes, and hyperlipidemia. Large datasets with multiple features offer an opportunity for machine learning to aid in the early detection and prediction of heart failure. METHODS: The study employed an unmatched case-control retrospective design. Supervised machine learning models were utilized, notably Multilayer Perceptron Artificial Neural Networks (MLP-ANNs) and decision tree-based models, to predict heart failure disease using data from 918 patients. The open licensed dataset, a combination from five independent heart datasets, comprises 11 demographic and clinical features related to patient status. The MLP-ANN, equipped with a hidden layer and a hyperbolic tangent activation function, was trained on 70% of the data and tested on the remaining 30%. Additionally, the study evaluated the decision tree model's performance through split-sample validation and 10-fold cross-validation. RESULTS: The ANN model demonstrated an accuracy of 88.10% in predicting heart disease (AUROC = 0.942) based on six factors and five covariates. The cross-validated tree model achieved an overall predictive accuracy of 84.3%. In contrast, the split-sample validated tree model, which used a balanced 50-50 data split for training and testing, attained an accuracy of 82.0%. OldPeak (ST depression induced by exercise relative to rest) had the highest normalized importance calculated from the MLP ANN model. CONCLUSION: Machine learning predictions have gained importance in healthcare, presenting potential benefits in early detection and intervention, leading to improved patient outcomes and reduced healthcare expenses. The study revealed that ANNs outperform decision tree models in accuracy for the dataset in use. Furthermore, the research emphasized the significance of the clinical feature "Oldpeak" in predicting heart failure through ANNs. ANNs can discern intricate relationships between variables and recognize non-linear interactions, a capability sometimes missed by decision tree models. However, the efficacy of machine learning models remains dependent on the quality and volume of the available data.

Efficient Algorithm for Solving Fuzzy Singularly Perturbed Volterra Integro-Differential Equation

In this paper, we design a fuzzy neural network to solve fuzzy singularly perturbed Volterra integro-differential equation by using a High Performance Training Algorithm such as the Levenberge-Marqaurdt (TrianLM) and the sigmoid function of the hidden units which is the hyperbolic tangent activation function. A fuzzy trial solution to fuzzy singularly perturbed Volterra integro-differential equation is written as a sum of two components. The first component meets the fuzzy requirements, however, it does not have any fuzzy adjustable parameters. The second component is a feed-forward fuzzy neural network with fuzzy adjustable parameters. The proposed method is compared with the analytical solutions. We find that the proposed method has excellent accuracy in findings, a lower error rate, and faster convergence than other typical methods.

Voxel representation of brain images inpainting via Regional Pixel Semantic Network and pyramidal attention AE - Quantile differential mechanism model

Medical image inpainting holds significant importance in enhancing the quality of medical images by restoring missing areas, thereby rendering them suitable for diagnostic purposes. While several techniques have been previously proposed for medical image inpainting, they are not suitable for distorted images containing metallic implants due to their limited consideration of known shaped masking. To overcome this limitation, a novel Vectorized Box Interpolation with Arbitrary Auto-Rand Augment Masking technique has been proposed which involves scaling and vectorizing images to expand their details and generating asymmetrically shaped masking in an automatic random format. One of the challenging tasks in this regard is the precise detection of lost regions, which is addressed through the introduction of the Regional Pixel Semantic Network. This technique employs the locally shared features (LSF) based region sensing with FCN (fully convolutional network) segmentation, which performs automatic segmentation based on neighboring pixel local dependency and regional features to determine the location of masked regions. During the reconstruction of missing parts, a significant challenge posed is the inability to recognize proximity in encoding owing to the generation of shadow-like regions on the feature map. To address this issue, a novel Multilayered DRC Regularized Pyramidal Attention AE Model has been proposed which employs dilated convolution with coherent pyramidal attention for feature extraction and improves image resolution using a Laplacian convolutional layer. Moreover, the realness of the generated image is determined using the Quantile Differential Mechanism model, where in the Quantile Differential Partial Convolutional Discriminator utilizes the hyperbolic tangent activation function in the partial convolutional layer to calculate recognition accuracy. As a result, the proposed method achieves high percentages for accuracy (98 %), precision (97 %), sensitivity (96 %), recall (95 %), and F-measure (96 %) thereby outperforming existing methods. Overall, this proposed method effectively handles distorted images with metallic implants, accurately detects lost regions, and improves the reconstructed image quality.

Neural network-based method for determining vessel position by seabed relief

A seabed relief-based vessel position fixing model on the basis of a four-layered feedforward neural network is proposed. Hidden neurons have hyperbolic tangent activation functions. The model is constructed for 1-D case that can be considered as vessel motion throw a narrow channel or alongside fairway axis. A sequence of spot soundings is given for the network input. The linear coordinate registered for the last sounding forms the network output. The training set is formed by means of the intentional pseudorandom alteration of input samples in accordance with suspected limits of sea level variations and the constant error of its measurements. The validation set is not used. The Adamax algorithm is implemented for the neural network training. The maximum of absolute value of the prediction error is used as a performance criterion of the net. Modeling has been conducted with the Python programming language. The Tensorflow library is used for the creation, training and testing of the neural network. The depth is modelled as a piecewise polynomial function of the coordinate. The results of neural network testing with the use of noised input samples let to state that the neural net can determine a ship position by means of soundings with acceptable accuracy. Different combinations of the sea level error and the number of hidden neurons have been considered. For each of such combinations the network accuracy indicators have been calculated. The best results are obtained for the network with 100 hidden neurons per each layer.

Stochastic Gradient Deep Multilayer Neural Network based Linear Congruential Generative Cryptosystem for Secured Data Communication in Cloud

Cloud computing is a kind of distributed computing that use a vast network of interconnected resources accessible over the internet. Security is a crucial concern in cloud computing due to the fact that users save their data on the cloud for convenient access from any location and at any time. Consequently, many users are worried about safeguarding their sensitive data in an unsafe location. Therefore, cloud computing architecture requires an innovative cryptographic method that ensures the secrecy, authenticity, integrity, and non-repudiation of data transfer in the cloud. A new technique called SEMcrypt, which stands for Stochastic Gradient DEep Multilayer Neural Network based Linear Congruential Generative Cryptography, has been developed to enhance secure data transmission. SEMcrypt ensures higher data confidentiality and reduces the time required for communication between the cloud user (i.e., patient) and the server. The SEMcrypt approach has two distinct processes: categorization and secure data transport. Initially, the data is gathered from the patients and is used as input for the stochastic gradient regularised deep multilayer neural network. The deep neural network consists of one input layer, two hidden layers, and one output layer. At first, the information is gathered from the patients and sent to the input layer. Next, the patient data that has been gathered is examined in hidden layer 1 using the generalised Tikhonov regularisation function. The patient data that has been analysed is sent to hidden layer 2. The hyperbolic tangent activation function is used at that layer to classify the patient data. Subsequently, the categorised data undergoes encryption via the use of Linear Congruential Generative Goldwasser-Micali encryption, ensuring safe transfer of the data. Subsequently, the encrypted data is sent to the cloud server. The patient data is encrypted on the server side using the Linear Congruential Generative Goldwasser-Micali decryption technique to prevent unauthorised access or assaults. Consequently, the authorised recipient receives the unaltered information, which is then kept in the database for further analysis. Secured data transmission is achieved by ensuring better levels of data confidentiality and reducing the time required for the process. The experimental assessment focuses on criteria such as the time it takes to generate keys, the level of data confidentiality and integrity, the computing time, and the accuracy of categorization. The empirical findings demonstrate that our suggested SEMcrypt approach delivers efficient performance outcomes by attaining superior levels of data confidentiality and integrity within a minimal timeframe.

Complex Dynamical Characteristics of the Fractional-Order Cellular Neural Network and Its DSP Implementation

A new fractional-order cellular neural network (CNN) system is solved using the Adomian decomposition method (ADM) with the hyperbolic tangent activation function in this paper. The equilibrium point is analyzed in this CNN system. The dynamical behaviors are studied as well, using a phase diagram, bifurcation diagram, Lyapunov Exponent spectrum (LEs), and spectral entropy (SE) complexity algorithm. Changing the template parameters and the order values has an impact on the dynamical behaviors. The results indicate that rich dynamical properties exist in the system, such as hyperchaotic attractors, chaotic attractors, asymptotic periodic loops, complex coexisting attractors, and interesting state transition phenomena. In addition, the digital circuit implementation of this fractional-order CNN system is completed on a digital signal processing (DSP) platform, which proves the accuracy of ADM and the physical feasibility of the CNN system. The study in this paper offers a fundamental theory for the fractional-order CNN system as it applies to secure communication and image encryption.

Seabed relief-based vessel position fixing with a neural network

A seabed relief-based vessel position fixing system on the basis of a neural network is proposed. The neural network satisfies the conditions of the universal approximation theorem and has one hidden layer. The hidden neurons have hyperbolic tangent activation functions. The model is constructed for 1-D case that can be considered as vessel motion throw a narrow channel or fairway. A sequence of depth derivatives (in relation to the coordinate) is fed to the network input. The depth is assumed to be measured with an echo-sounder. The vessel linear coordinate registered for the last depth derivative is formed on the network output. The training set contains not only data presumably registered at the stage of preliminary depth survey but also their noise-added versions obtained with the use of a random number generator. The validation set contains the survey data only. The Adamax algorithm is implemented for the neural network training. The maximum of absolute value of the prediction error is used as a performance criterion of the net. Modeling has been conducted in Python with Tensorflow. The depth is considered to be a polynomial function of the coordinate at each path region. So, the depth derivatives can be calculated analytically. As the result it is possible to state that the neural network predicts a vessel position with acceptable accuracy even in input signal noise conditions. Moreover, the neural network architecture of the radial-basis functions has been examined, but it is not possible to achieve acceptable accuracy by using it. The conducted investigations of the influences of a mini-batch size and learning rate values on the accuracy has shown that these parameters have the significant impact and an issue of their choice remains opened and actual in the framework of the task.

Deep learning framework for sensor array precision and accuracy enhancement

In the upcoming years, artificial intelligence is going to transform the practice of medicine in most of its specialties. Deep learning can help achieve better and earlier problem detection, while reducing errors on diagnosis. By feeding a deep neural network (DNN) with the data from a low-cost and low-accuracy sensor array, we demonstrate that it becomes possible to significantly improve the measurements’ precision and accuracy. The data collection is done with an array composed of 32 temperature sensors, including 16 analog and 16 digital sensors. All sensors have accuracies between 0.5 - 2.0;^{^circ } {text{C}}. 800 vectors are extracted, covering a range from to 30 to 45;^circ hbox {C}. In order to improve the temperature readings, we use machine learning to perform a linear regression analysis through a DNN. In an attempt to minimize the model’s complexity in order to eventually run inferences locally, the network with the best results involves only three layers using the hyperbolic tangent activation function and the Adam Stochastic Gradient Descent optimizer. The model is trained with a randomly-selected dataset using 640 vectors (80% of the data) and tested with 160 vectors (20%). Using the mean squared error as a loss function between the data and the model’s prediction, we achieve a loss of only 1.47x10^{-4} on the training set and 1.22x10^{-4} on the test set. As such, we believe this appealing approach offers a new pathway towards significantly better datasets using readily-available ultra low-cost sensors.

Activation Function Effects and Simplified Implementation for Hopfield Neural Network

The activation function is crucial in the Hopfield neural network (HNN) to restrict the input–output relation of each neuron. The physical realizability and simplicity of the hardware circuit of activation function are beneficial to promote the practical engineering application of the HNN. However, the HNN commonly used hyperbolic tangent activation function involves a complex hardware circuit implementation. This paper discusses a piecewise-linear activation function (PWL-AF) with simplified circuit implementation and a tri-neuron small-world HNN is built as a paradigm. The hardware implementation circuit of the HNN is greatly simplified, benefited from the PWL-AF with a simple analog circuit. Meanwhile, the dynamics related to the PWL-AF and initial conditions are numerically explored. The numerical results demonstrate that the PWL-AF-based HNN can produce dynamical behaviors like the HNN based on the hyperbolic tangent activation function. Nevertheless, the multistability with up to six kinds of coexisting multiple attractors emerged because of the PWL-AF breakpoint. This can give more flexible and potential aspects in multistability-based engineering applications. Especially, the PWL-AF breakpoint value simultaneously acts as the offset booster and amplitude controller in regulating the offset boosting and amplitude rescaling of neuron states. Afterwards, an analog circuit with three straightforward operational amplifiers (op-amp)-based circuit modules is designed for the PWL-AF, and a PCB-based analog circuit is thereby implemented for the tri-neuron small-world HNN. The hardware experiments agree with the numerical simulations, implying the feasibility of the PWL-AF simplification for the HNN.

IDEA: Artificial neural network models for 11-species air properties at thermochemical equilibrium

Accurate prediction of high-temperature air properties is essential in many aerodynamic applications under hypersonic flight conditions. Various curve-fit models using piecewise polynomial fittings have been commonly adopted to approximate equilibrium air properties at high temperatures. Several shortcomings including low accuracy, lack of diversity, and discontinuity at curve-fit boundary still remain, causing numerical troubles in computational procedures. To address the issues, IDEA, an open-source C++ library that enables fast and accurate computations of the equilibrium air properties and their first and second derivatives, is newly developed based on the artificial neural network (ANN).IDEA, which stands for the Infinitely Differentiable Equilibrium Air, predicts thermodynamic and transport properties of 11-species (N2, O2, N, O, NO, NO+, N+, O+, N++, O++, and e−) thermochemical equilibrium air at the temperature range up to 25,000 K and density range from 10−7 to 103 amagats. The training data is constructed from the kinetic molecular theory using the equilibrium constant method with the rigid-rotor, harmonic-oscillator model. As the name suggests, IDEA's models are infinitely differentiable in the application range; thus, they have enhanced convergence in computational fluid dynamics (CFD) when using gradient-based methods. Using a newly developed training process based on the Levenberg–Marquardt algorithm with weighted mean squared error loss, IDEA provides more accurate and diverse property models with much fewer parameters than previous piecewise polynomial fitting models. In addition, the proposed training method offers easy extensions to various property models with different species data. IDEA provides C interfaces that can be used for programs in various computer languages, such as C/C++, Fortran, Python, and MATLAB. IDEA's modeling routines are thread-safe, so they can be safely used for parallel programs without performance loss. The accuracy and enhanced convergence of IDEA is demonstrated via several high-speed flow computations Program summaryProgram title: IDEACPC library link to program files:https://doi.org/10.17632/84rhtfz9n2.1Developers' repository link:https://github.com/HojunYouKr/IDEALicensing provisions: BSD-3-ClauseProgramming language: C++11Nature of problem: Accurate prediction of equilibrium air properties is essential in many aerodynamic applications under hypersonic flight conditions. Direct calculation of air properties by the kinetic molecular theory requires time-consuming iterative methods to solve nonlinear equations of species concentrations. Various piecewise polynomial models have been developed to avoid such iterations. However, the regression error of the polynomial models requires improvement, and the lack of diversity of the polynomial models makes iterative computations inevitable, causing computational inefficiency. Furthermore, these polynomial models are not continuously differentiable, which deteriorates the convergence characteristics of computational fluid dynamics (CFD) solvers.Solution method: ANN is chosen to model the equilibrium properties of air owing to its excellent capability as a universal function approximator and better adaptability to a large dataset than kernel methods. The hyperbolic tangent activation function makes the ANN model infinitely differentiable, which enables to use a superlinear training process and thus improves convergence characteristics. The newly proposed superlinear training process based on the Levenberg–Marquardt algorithm with weighted mean squared error loss provides more accurate and diverse property models with much fewer parameters than the existing polynomial models. As a result, the proposed ANN model is completely free from iterative computations.