Activation Function Research Articles

Artificial neural network hyperparameters optimization for predicting the thermal conductivity of MXene/graphene nanofluids

BackgroundThe critical role of thermal conductivity (TC) as a significant thermo-physical property in MXene/graphene-based nanofluids for photovoltaic/thermal systems has motivated recent research into developing precision predictive models. The multilayer perceptron neural network (MLPNN) has emerged as an eminent AI algorithm for this task. MethodsThis study employs Bayesian optimization, random search (RS), and grid search (GS) to fine-tune MLPNN hyperparameters—hidden layers, neurons, activation functions, standardization, and regularization—to elevate TC modeling efficiency. The proposed methodology unfolds in sequential phases: data analysis, data pre-processing, and introduction of MLPNN, GS, RS, Bayesian approach, and their integration algorithm. The next phase entails developing predictive models and presenting optimal cases. Lastly, the final models undergo statistical evaluation and graphical comparison for a thorough analysis. FindingsResults manifest that the GS-MLPNN model excels, achieving the lowest testing data error (MAPE = 0.5261%) and high conformity with empirical data (R = 0.99941). Meanwhile, the RS method adjusts hyperparameters with negligible precision loss (MAPE = 0.6046%, R = 0.99887). Contrarily, Bayesian optimization lags, increasing errors (MAPE = 3.1981%) and lower correlation (R = 0.98099), suggesting its relative inefficacy for this specific application. The optimized models provide efficient predictions, significantly reducing the financial/computing costs associated with experimental/numerical analysis.

Construction of robust superhydrophobic surfaces with electrothermal, photothermal properties using simple spraying method

In this study, robust superhydrophobic surfaces with electrothermal, photothermal properties were fabricated by simple spraying method. In more detail, multiwall carbon nanotube (MWCNT) and silica were deliberately sprayed onto the wet poly(amide imide) (PAI) surface to construct intensive conductive network with hierarchical porous structure. The composite coating showed superhydrophobic with a water contact angle (CA) of greater than 155o and sliding angle (SA) of lower than 5o, conductivity of approaching to 1 kΩ sq-1. As expected, the composite coating possessed passive anti-icing property owing to its high CA, and the freezing time at -20 °C was obviously delayed from 150 to 600 s. On the other hand, the surface temperature respectively increased by 40 °C and 35 °C under the assistance of electric (voltage of 36 V) or light (0.23 W cm-2 irradiation) power, and the active anti-icing function can still work even under a such low temperature of -15 °C. It is worth noting that the heat generated by Joule effect was fixed on the conductive surfaces, and thus the composite coating showed good thermal insulation due to the low thermal conductivity of PAI substrate (0.298 W/mK) and hierarchical porous structure of composite coating. In addition, the surfaces showed high mechanical robust and durability, and the superhydrophobic and conductive properties were nearly unaffected even they suffered from lots of tests such as sandpaper abrasion, ultrasonic treatment, strong acid or strong alkali immerse, UV irradiation, and hot water jet impacting. Therefore, this study offered a simple way to fabricate electrothermal and photothermal surfaces for practical anti-icing/de-icing applications.

The synergistic effect of polyphenols and polypeptides for plant-based bioplastic film – Enhanced UV resistance, antioxidant and antibacterial performance

The exceptional biodegradability and active biological functions of bio-based packaging materials have attracted increasing interest. In this study, a bioplastic film was developed by introducing simultaneously polyphenols (tea polyphenols, TPs) and peptides (nisin) into a soy protein isolate/sodium alginate (SPI/SA) based film-forming matrix. The research results revealed that the dynamic coordinated interaction between TPs and nisin enhanced mechanical properties, UV-resistance, and thermal stability of bioplastic films. Furthermore, the bioplastic film exhibited antibacterial activity and antioxidant properties. Significantly, biofilm growth of Staphylococcus aureus treated with TPs-5/Nisin-5 bioplastic film was inhibited by 91.12% compared to the blank group. The shelf life of beef with TPs-5/Nisin-5 bioplastic film was prolonged by 2 days because of the synergistic effect of TPs and nisin. Additionally, the bioplastic film biodegraded in the natural environment about 21 days. This environmentally friendly regeneration strategy and the integration of advantageous functions provided ideas for the development of active food packaging.

A method for counting fish based on improved YOLOv8

In industrial aquaculture, accurately counting fish in real-time is crucial for optimizing feeding strategies, preventing disease, and managing water quality. Current methods utilizing sensors, acoustics, machine learning, and density map regression face challenges such as high costs, invasiveness, and computational complexity. To address these limitations, we propose the YOLOv8n-MEMAGD method for accurate real-time fish counting. This method enhances YOLOv8 by incorporating the GELU activation function, MPDIoU for localization loss, and the C2f-FAM and C2f-MSCA modules into the backbone network. Additionally, the neck network is redesigned with a gather-and-distribution mechanism. Experimental results under land-based industrial aquaculture conditions demonstrate that YOLOv8n-MEMAGD achieved a mean absolute error (MAE) of 2.28 and a root mean square error (RMSE) of 2.84, even in challenging conditions such as fish overlap, aggregation, and background confounding. Compared to YOLOv8n, the proposed method increased average precision (AP50) for fish detection by 6.2 %, and significantly reduced MAE and RMSE by 61.4 % and 65.2 %, respectively, compared to CSRNet. Additionally, the method achieved a frame rate of 61 frames per second (FPS) when the number of fish ranged from 79 to 91, representing a 390.4 % increase over CSRNet. By comparing heatmaps, the proposed method demonstrates more effective detection of fish edge contours than current advanced algorithms. In conclusion, the proposed method shows promise for application in aquacultural scenarios with higher turbidity and larger number of fish.

Automatic localization of image semantic patches for crop disease recognition

Crop disease recognition plays a crucial role in agricultural production. However, disease images are large in scale and have a lot of redundant information, which reduces the effectiveness of deep neural networks in extracting diseases. To address the above issues and considering that not all image regions are relevant to disease recognition, this study proposes an efficient crop disease recognition method with dynamic reduction of image redundancy. The method is a two-stage process. In the first stage, we employ the lightweight CA-AnchorNet, which incorporates coordinate attention, to swiftly generate a feature map of the affected crop areas. Subsequently, class activation maps (CAMs) are utilized to identify the disease feature regions, highlighting areas that exhibit class discriminability. These regions are then mapped to a higher resolution from the original lower-resolution image, and the target patch is extracted. In the second stage, these local semantic patches, characterized by reduced spatial redundancy, are fed into the lightweight PatchNet for accurate recognition. PatchNet incorporates the Inception-C module and the ACON-C activation function. These features enhance the model's ability to express multi-scale and non-linear characteristics of crop disease features. This method does not require manually annotated region boxes and achieves an identification accuracy of 99.86 % on a 12-class crop disease dataset with complex environments. The parameter count is only 0.98 M. This method has the characteristics of accurate localization and low parameter count, and can be used for effective and high-precision recognition of crop diseases in complex environments.

Global attractive set for quaternion-valued neural networks with neutral items

This paper investigated the global attractive set for quaternion-valued neural networks (QVNNs) with leakage delay, time-varying delay, and neutral items. Based on various basic conditions of activation function, the global attractive set and global exponential attractive set of QVNNs are given combined with novel analytical techniques and Lyapunov theory. The QVNNs are studied by a direct method, without any decomposition. The time delay can be non-differential, which makes the results more pragmatic. Restrictions on the activation function of the neutral item are relaxed. The neutral activation function can be bounded or unbounded, which makes the results more practical. Two simulation examples are given to verify the validity of the theory results.

Utilization of genetic algorithm in tuning the hyper-parameters of hybrid NN-based side-slip angle estimators

This paper proposes a solution to enhance and compare different neural network (NN)-based side-slip angle estimators. The feed-forward neural networks (FFNNs), recurrent neural networks, long short-term memory units (LSTMs), and gated recurrent units are investigated. However, there is a lack in the selection criteria of the architectures’ hyper-parameters. Therefore, the genetic algorithm is integrated with the NN-based estimators to find the optimal hyper-parameters for the studied architectures. The tuned hyper-parameters in this work include the number of neurons, number of layers, activation function, optimizer type, and learning rate. The objective function of the optimization problem is minimizing the root-mean-square error (RMSE) on multiple testing data. The optimal models are further included in the design of a hybrid NN estimator with Kalman filter. In the hybrid estimators, the optimal NN estimators are used as virtual sensors to correct the prediction of the side-slip angle resulting from the mathematical lateral vehicle model. Eventually, the performance of the best selected model is evaluated in terms of different metrics; mean RMSE, mean error variance, mean training time, and mean estimation time. LSTMs are found to achieve the lowest mean RMSE while being tested on highly generalized data yielding the highest training and estimation time. However, FFNNs achieve the lowest RMSE while being tested on low generalized data and the lowest training and estimation time. Meanwhile, it is observed that the hybrid estimators achieved lower RMSE with great enhancement compared to the non-hybridized ones proving the effectiveness of the proposed approach and increasing the side-slip estimation generalization ability in unknown environments with high uncertainties, which are not covered by the training dataset for the NNs estimators.

An automated lightweight approach for detecting dead fish in a recirculating aquaculture system

Accurate methods for detecting dead fish are of great significance for fisheries because of their potential to improve production and reduce water pollution. Small-scale fisheries typically apply manual observation, but this approach is labor-intensive and time-consuming, which limits its application on large-scale farms. Advances in computer vision tools have provided the foundation for revolutionary methods of automated identification of individual-animal behavior. However, computer vision systems often suffer from unstable accuracy, low efficiency, and limited detection capability. Thus, the aim of this study was to design a deep learning system, designated “Deadfish-YOLO”, to detect dead fish based solely on standard underwater camera images with no additional hardware. First, eight selected rearing tanks were monitored by a self-developed computer version system. From these tank images, a dataset containing 18,114 frames was built by manual labeling. Next, a lightweight backbone network was generated with YOLOv4 to ensure fast computations, and an attention mechanism was introduced into the model to suppress unimportant features. Finally, the ReLU-memristor-like activation function was adopted to improve neural-network performance. The accuracy and processing speed of Deadfish-YOLO were superior to those of other state-of-the-art single- and two-stage detection models; Deadfish-YOLO ran at 85 frames per second with a mean average precision of 0.946 and an average ratio of 0.924 between intersection and union. These results demonstrate that Deadfish-YOLO may be used to automatically monitor dead fish in real circulating aquaculture systems. In addition, the results of this study should facilitate the wider application of artificial-intelligence-based animal monitoring in aquaculture.

A neural network computational procedure for the novel designed singular fifth order nonlinear system of multi-pantograph differential equations

The current investigations present the numerical solutions of the novel singular nonlinear fifth-order (SNFO) system of multi-pantograph differential model (SMPDM), i.e., SNFO–SMPDM. The novel SNFO–SMPDM is obtained using the sense of the second kind of typical Emden–Fowler and prediction differential models. The features of shape factor, pantograph along with singular points are provided for all four obtained classes of the SNFO–SMPDM. The extensive use of the singular models is observed in the engineering and mathematical systems, e.g., inverse systems and viscoelasticity or creep systems. For the correctness of the proposed novel SNFO–SMPDM, one case of each class is numerically handled by applying supervised neural networks (SNNs) along with the optimization of Levenberg–Marquardt backpropagation scheme (LMBS), i.e., SNNs–LMBS. A dataset using the traditional variational iteration scheme is designed to compare the proposed results of each case of SNFO–SMPDM. The obtained approximate solutions of each class using the novel SNFO-SMPDM are presented based on the training (80 %), authentication (10 %) and testing (10 %) measures to evaluate the mean square error. Fifteen numbers of neurons, and sigmoid activation function are used in this SNN process. To authenticate the competence, and precision of SNFO–SMPDM, the numerical simulations are accessible by applying the relative measures of regression, error histogram plots, and correlation.

A Fixed-Time Noise-Tolerance ZNN Model for Time-Variant Inequality-Constrained Quaternion Matrix Least-Squares Problem.

Presently, numerical algorithms for solving quaternion least-squares problems have been intensively studied and utilized in various disciplines. However, they are unsuitable for solving the corresponding time-variant problems, and thus few studies have explored the solution to the time-variant inequality-constrained quaternion matrix least-squares problem (TVIQLS). To do so, this article designs a fixed-time noise-tolerance zeroing neural network (FTNTZNN) model to determine the solution of the TVIQLS in a complex environment by exploiting the integral structure and the improved activation function (AF). The FTNTZNN model is immune to the effects of initial values and external noise, which is much superior to the conventional zeroing neural network (CZNN) models. Besides, detailed theoretical derivations about the global stability, the fixed-time (FXT) convergence, and the robustness of the FTNTZNN model are provided. Simulation results indicate that the FTNTZNN model has a shorter convergence time and superior robustness compared to other zeroing neural network (ZNN) models activated by ordinary AFs. At last, the construction method of the FTNTZNN model is successfully applied to the synchronization of Lorenz chaotic systems (LCSs), which shows the practical application value of the FTNTZNN model.

LDLR-mediated uptake of lipoproteins fuels T-cell activation and immunological function

LDLR-mediated uptake of lipoproteins fuels T-cell activation and immunological function

Enhanced Results on Sampled-Data Synchronization for Chaotic Neural Networks With Actuator Saturation Using Parameterized Control.

This article investigates a novel sampled-data synchronization controller design method for chaotic neural networks (CNNs) with actuator saturation. The proposed method is based on a parameterization approach which reformulates the activation function as the weighted sum of matrices with the weighting functions. Also, controller gain matrices are combined by affinely transformed weighting functions. The enhanced stabilization criterion is formulated in terms of linear matrix inequalities (LMIs) based on the Lyapunov stability theory and weighting function's information. As shown in the comparison results of the bench marking example, the presented method much outperforms previous methods, and thus the enhancement of the proposed parameterized control is verified.

Artificial Neural Network Model to Predict OBrix and pH of Banana Based on Color Parameters

Artificial neural network (ANN) was used to predict internal quality parameters (oBrix and pH) of lady finger banana. This research consisted of three stages, namely: (1) capturing images of lady finger banana using a computer vision system; (2) measurement of oBrix and pH of the banana; (3) ANN architecture analysis using the Matlab R2019a application. The ANN architectural model consisted of 3 output models, namely: (1) oBrix values; (2) pH value; (3) oBrix and pH values. The ANN architecture analysis was carried out through two phases. Phase I consisted of 45 experimental units and phase II with 35 experimental units. The best ANN architecture to be used as a prediction model for oBrix and pH of golden banana fruit is ANN architecture model 3 with the number of neurons inside the hidden layer = 3; activation function in hidden layer = logsig; activation function inside the output layer = logsig; data transformation range 0 – 1; learning rate value = 0.01; learning algorithm = tradingda; with MSE (mean square error), MAE (mean absolute error) performance and R correlation coefficient from training results of 0.0954; 0.2619 and 0.6538; test results 0.0392; 0.1606 and 0.7000 and validation results 0.0289; 0.1474 and 0.7889. Keywords: Artificial neural networks, Color, Computer vision system, CVS, RGB.

Combination of fermentation and enzymolysis enhances bioactive components and function of de-oiled rice bran.

De-oiled rice bran (DORB), a substantial yet underutilized byproduct of rice processing, boasts a rich composition of active ingredients but suffers from limited application. Previous studies have indicated that enzymatic or fermentation treatments enhanced these active components. In this study, lactobacilli and complex enzymes were employed to co-treat DORB, involving the determination of the changes in active components and functionalities of DORB extract (DORBE) before and after this treatment. Following fermentation-enzymolysis, the total phenol and total flavonoid contents in DORBE were significantly increased by 43.59% and 55.10%, reaching 19.66 and 34.34 g kg-1, respectively. Antioxidant tests in vitro demonstrated that the co-treatment enhanced the scavenging activities of DPPH, hydroxyl and ABTS radicals. Porcine intestinal epithelial cell experiments revealed that, compared to DORBE, the fermentation and enzymolysis DORBE (FDORBE) exhibited significantly improved cell viability and catalase activity as well as scavenging capacity for reactive oxygen species and malondialdehyde after induction by H2O2. Furthermore, FDORBE restored the decreased mRNA expression levels of Nrf2, HO-1 and NQO1 in the nuclear factor erythroid 2-related factor 2 (Nrf2) pathway stimulated by H2O2. Fermentation-enzymolysis co-treatment increases the contents of bioactive components of DORBE and enhances its antioxidant capacity, leading to a better protection against intestinal disorders induced by oxidative stress, suggesting that this co-treatment is a rational and effective strategy to increase the value of grains and promotes the use of DORB as a functional feed in animal production. © 2024 Society of Chemical Industry.

The Influence of Particle Swarm Optimization‐Back Propagation Neural Network Hyperparameter Selection on the Prediction Accuracy of Converter Endpoint Temperature

The converter is a complex, high temperature, high pressure reactor with limited internal moitoring. At present, data‐driven models mainly focus on the prediction differences between algorithms, and there is relatively little analysis of the impact of different hyperparameters on prediction accuracy. Taking a 120 t converter in a Chinese steel plant as an example, this paper explores the application of particle swarm optimization‐back propagation neural network (PSO‐BP) in converter temperature prediction. First, the Pauta criterion or Box plot method was used to preprocess the data by prescreening. Subsequently, the influence of the activation function, learning rate, and number of hidden layer nodes of BP on the prediction accuracy of the endpoint temperature were explored. Then we investigated the influence of PSO parameters on the optimal result of BP initial value. Comparing the temperature prediction hit rate before and after optimization, the BP model has hit rates of 63.64%, 79.22%, and 87.45% within ±10, ±15, and ±20 °C, respectively, and the PSO‐BP model has hit rates of 68.40%, 84.85%, and 94.81%, respectively. In comparison, PSO‐BP extracts data features more accurately, has higher stability, and has better accuracy in predicting the endpoint temperature of the converter.

A novel green learning artificial intelligence model for regional electrical load prediction

Load prediction is crucial to unit commitment and scheduling in electricity systems; however, load prediction models consume considerable electricity. Most conventional prediction methods are based on deep learning (DL) models, such as long short-term memory and gated recurrent units. The nonlinear activation functions and backpropagation mechanisms used in these methods result in high computational complexity and correspondingly high energy consumption. This paper proposes a prediction model based on a green learning (GL) framework aimed at reducing energy consumption. The proposed GL model replaces the activation functions conventionally used for feature extraction with a hybrid scheme combining categorical and numerical features, such as seasonal climate features and multi-autoregression terms. The proposed GL model also eliminates the backpropagation methods used to optimize hyperparameters by employing seasonal centroids and the quantile auto-regression forest algorithm for classification/regression. In a case study, the proposed GL model significantly outperformed conventional DL models in terms of energy consumption without compromising accuracy. The energy savings of the proposed GL model are equivalent to the annual electricity consumption of 211 ∼ 565 households, increasing with model penetration. This paper demonstrates the importance of energy-saving models in mitigating the linkage effect of industrial carbon emissions. It also explores policy implications for the implementation of GL models.

Comparison between abdominal core muscle strength in males with and without frozen shoulder: A pilot study

Background: The impact of adhesive capsulitis starts from the structures surrounding the shoulder complex and extends to overall deconditioning and affects neuromuscular control due to reduced daily living activities which may be passed along further in the kinetic chain. Context: Studies show there may be a certain link between core muscle activation and shoulder function, however, the literature is scanty in this regard. Aims: To compare the abdominal core muscle strength in males with and without frozen shoulder. Settings and Design: In this observational comparative study, 20 males (40-60 yrs) from a tertiary care hospital visiting the physiotherapy department for the treatment of frozen shoulder were included along with age, Body Mass Index (BMI) and Waist to Hip Ratio (WHR) matched healthy controls. Methods and Material: Abdominal core muscle strength was measured using a pressure biofeedback unit while progressive limb loading was performed. Statistical analysis used: The ordinal data was compared using SPSS Version 24 and Man Whitney U and Wilcoxon W were obtained. Results: The abdominal core muscle strength of male patients with frozen shoulder was weaker compared with healthy controls (p > 0.0001) Conclusions: The abdominal core muscle strength was significantly weaker in the males with frozen shoulder compared to age, BMI and WHR matched healthy controls. Thus Inclusion of core muscle strengthening in the rehabilitation of patients with frozen shoulder may lead to better outcomes in rehabilitation.

Microstructure versus topography: the impact of crystallographic substrate modification during ultrashort pulsed direct laser interference patterning on the antibacterial properties of Cu

Introduction: Topographic surface patterning in the micro- and nanometer scale has evolved into a well applied approach in surface functionalization following biomimetic blueprints from nature. Depending on the production process an additional impact of process-related substrate modification has to be considered in functional surface optimization. This is especially true in case of antimicrobial applications of Cu surfaces where a modification of the substrate properties might impact bactericidal efficiency.Methods: In this regard, the effect of ultrashort pulsed direct laser interference patterning on the microstructure of pure Cu and resulting antimicrobial properties was investigated alongside line-like patterning in the scale of single bacterial cells.Results and Discussion: The process-induced microstructure modification was shown to play an important role in corrosion processes on Cu surfaces in saline environment, whereas the superficial microstructure impacts both corrosive interaction and ion emission. Surprisingly, antimicrobial efficiency is not predominantly following deviating trends in Cu ion release rates but rather depends on surface topography and wettability, which was shown to be impacted by the substrate microstructure state, as well. This highlights the need of an in-depth understanding on how different surface properties are simultaneously modulated during laser processing and how their interaction has to be designed to acquire an effective surface optimization e.g., to agitate active antimicrobial surface functionalization.

Advanced State of Charge Estimation Using Deep Neural Network, Gated Recurrent Unit, and Long Short-Term Memory Models for Lithium-Ion Batteries under Aging and Temperature Conditions

Accurate estimation of the state of charge (SoC) of lithium-ion batteries is crucial for battery management systems, particularly in electric vehicle (EV) applications where real-time monitoring ensures safe and robust operation. This study introduces three advanced algorithms to estimate the SoC: deep neural network (DNN), gated recurrent unit (GRU), and long short-term memory (LSTM). The DNN, GRU, and LSTM models are trained and validated using laboratory data from a lithium-ion 18650 battery and simulation data from Matlab/Simulink for a LiCoO2 battery cell. These models are designed to account for varying temperatures during charge/discharge cycles and the effects of battery aging due to cycling. This paper is the first to estimate the SoC by a deep neural network using a variable current profile that provides the SoC curve during both the charge and discharge phases. The DNN model is implemented in Matlab/Simulink, featuring customizable activation functions, multiple hidden layers, and a variable number of neurons per layer, thus providing flexibility and robustness in the SoC estimation. This approach uniquely integrates temperature and aging effects into the input features, setting it apart from existing methodologies that typically focus only on voltage, current, and temperature. The performance of the DNN model is benchmarked against the GRU and LSTM models, demonstrating superior accuracy with a maximum error of less than 2.5%. This study highlights the effectiveness of the DNN algorithm in providing a reliable SoC estimation under diverse operating conditions, showcasing its potential for enhancing battery management in EV applications.

Crop disease diagnosis and prediction using two-stream hybrid convolutional neural networks

Crop diseases significantly impact yield and quality, posing a direct threat to food security. The application of Convolutional Neural Networks (CNN) in crop disease recognition has notably improved diagnosis accuracy and efficiency. This study presents an innovative crop disease classification model based on the VGG-16 network. Enhancements include the incorporation of Batch Normalization (BN) and a novel activation function synergizing with Exponential Linear Units (ELU), improving model convergence speed and accuracy. Additionally, Global Average Pooling (GAP) is integrated to streamline the network architecture, and the InceptionV2 module is introduced to extract leaf disease features from different dimensions, enhancing model robustness. Validation on the PlantVillage dataset shows an accuracy rate of 98.89%, demonstrating the model's competitiveness and its potential to support sustainable agricultural production.