Activation Function Research Articles

Detection of breast cancer by deep belief network with improved activation function

Detection of breast cancer by deep belief network with improved activation function

Neural network-based algorithm for door handle recognition using RGBD cameras

The ability to recognize and interact with a variety of doorknob designs is an important component on the path to true robot adaptability, allowing robotic systems to effectively interact with a variety of environments and objects The problem addressed in this paper is to develop and implement a method for recognizing the position of a door handle by a robot using data from an RGBD camera. To achieve this goal, we propose a revolutionary approach designed for autonomous robots that allows them to identify and manipulate door handles in different environments using data obtained from RGBD cameras. This was achieved by creating and annotating a complete dataset consisting of 5000 images of door handles from different angles, with the coordinates of the vertices of the bounding rectangles labeled. The architectural basis of the proposed approach is based on MobileNetV2, combined with a special decoder that optimally increases the resolution to 448 pixels. A new activation function specially designed for this neural network is implemented to ensure increased accuracy and efficiency of raw data processing. The most important achievement of this study is the model's ability to work in real-time, processing up to 16 images per second. This research paves the way for new advancements in the fields of robotics and computer vision, making a substantial contribution to the practical deployment of autonomous robots in a myriad of life's spheres.

Night target detection algorithm based on improved YOLOv7

Aiming at the problems of error detection and missing detection in night target detection, this paper proposes a night target detection algorithm based on YOLOv7(You Only Look Once v7). The algorithm proposed in this paper preprocesses images by means of square equalization and Gamma transform. The GSConv(Group Separable Convolution) module is introduced to reduce the number of parameters and the amount of calculation to improve the detection effect. ShuffleNetv2_×1.5 is introduced as the feature extraction Network to reduce the number of Network parameters while maintaining high tracking accuracy. The hard-swish activation function is adopted to greatly reduce the delay cost. At last, Scylla Intersection over Union function is used instead of Efficient Intersection over Union function to optimize the loss function and improve the robustness. Experimental results demonstrate that the average detection accuracy of the proposed improved YOLOv7 model is 88.1%. It can effectively improve the detection accuracy and accuracy of night target detection.

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs.

Flexible computation is a hallmark of intelligent behavior. However, little is known about how neural networks contextually reconfigure for different computations. In the present work, we identified an algorithmic neural substrate for modular computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses revealed learned computational strategies mirroring the modular subtask structure of the training task set. Dynamical motifs, which are recurring patterns of neural activity that implement specific computations through dynamics, such as attractors, decision boundaries and rotations, were reused across tasks. For example, tasks requiring memory of a continuous circular variable repurposed the same ring attractor. We showed that dynamical motifs were implemented by clusters of units when the unit activation function was restricted to be positive. Cluster lesions caused modular performance deficits. Motifs were reconfigured for fast transfer learning after an initial phase of learning. This work establishes dynamical motifs as a fundamental unit of compositional computation, intermediate between neuron and network. As whole-brain studies simultaneously record activity from multiple specialized systems, the dynamical motif framework will guide questions about specialization and generalization.

Multi-physics information-integrated neural network for fatigue life prediction of additively manufactured Hastelloy X superalloy

ABSTRACT The inherent ‘black-box’ characteristic of neural networks renders the physical interpretability of fatigue life prediction challenging, resulting in physically inconsistent prediction results. In response to this challenge, a multi-physics information-integrated neural network framework is proposed for fatigue life prediction. Rooted in the continuous damage mechanics with embedded additive manufacturing process parameters (AM-CDM), coupled with activation functions and loss functions that amalgamate physical information, a multi-tiered and multi-physical information source network framework emerges. The framework incorporates the outcomes of the AM-CDM model as input features for domain knowledge enhancement, embeds the fatigue limit within the activation function, and establishes a loss function integrating the physical model. Through the integration of multiple physical information, the model can be restricted by different physical constraints during training, thereby improving its prediction accuracy and physical consistency. Validation of the proposed framework is conducted using laser powder bed fusion (LPBF)-fabricated Hastelloy X subjected to varying orientations, post-treatments and test temperatures. The findings demonstrate that the prediction accuracy of an integrating-physics-informed neural network (IPINN) is improved by 27.51%, and its life prediction results are capped at the fatigue limit life, yielding a better physical consistency. The IPINN framework provides a novel perspective for fatigue life prediction.

Airborne transient electromagnetic imaging method based on wavelet neural network

The airborne transient electromagnetic method is a crucial technique for near-surface exploration in rugged terrains. In this study, we introduce a multi-input, single-output double-hidden wavelet neural network design for airborne transient electromagnetic pseudo-resistivity imaging. We construct uniform half-space, stratified stratum, and three-dimensional geoelectric models. By evaluating various performance indicators of neural networks that employ different wavelet basis functions as activation functions, we identify the most suitable wavelet basis functions. The quasi-resistivity is computed using both the wavelet neural network and the backpropagation neural network and then juxtaposed with traditional apparent resistivity. Our findings indicate that the wavelet neural network's quasi-resistivity aligns more closely with the resistivity of the real model. It is also more responsive to low resistivity anomalies than the conventional apparent resistivity translation algorithm. The wavelet approach moderates the undershoot or overshoot occurrences during abrupt stratum changes, offering a more accurate representation of subterranean electrical properties. When compared to the backpropagation neural network, the wavelet neural network provides a superior fit for the model, rendering a smoother quasi-resistivity depth curve. Therefore, it stands out as an improved method for pseudo-resistivity imaging. By processing survey data via the trained wavelet neural network, we find that the all-time apparent resistivity and quasi-resistivity align well with real-world situations. The wavelet neural network prominently showcases the low-resistance calculation results, offering a broader resistivity range. This clarity enhances anomaly detection, confirming the wavelet neural network's applicability and practicality for airborne transient electromagnetic imaging.

MC-ShuffleNetV2: A lightweight model for maize disease recognition

Maize has a long history of cultivation and is renowned for its high yield, superior quality, and adaptability. Currently, maize holds a significant position in grain cultivation and occupies a significant place in the agricultural structure. However, maize is susceptible to various diseases during its growth process, which can have a significant impact on the quality and yield. Traditional machine learning is heavily reliant on feature extraction, whereas deep learning has demonstrated notable success in image recognition for computer vision.The use of bloated models and the resulting wastage of computational resources represent significant challenges. The paper proposes a lightweight model, MC-ShuffleNetV2 (Mish + Convolutional Block Attention Module + ShuffleNetV2), to meet the practical needs of convolutional neural networks in maize disease image recognition. The model has designed with a focus on network lightweighting and accurate feature extraction. The model was constructed upon the foundation of the high-performance ShuffleNetV2 1 × network. The Convolutional Block Attention Module was integrated into the network architecture to enhance the model’s adaptive expressiveness. The depthwise separable convolution kernel of the depth-separable module was modified from a 3 × 3 kernel to a 5 × 5 kernel. This modification was implemented with the objective of expanding the image receptive field and extracting more detailed features of the image. It was necessary to modify the activation function in each stage for Mish. The model was compressed through the application of pruning operations. In the maize disease dataset test, the accuracy of the test set recognition accuracy of the network model constructed in this paper reaches 99.86 %, the model parameters are only 873,936, and the FLOPs (Floating-point Operations) are only 1,751,286. Compared with LeNet, AlexNet, MobileNetV2, and EfficientNetV2 models, the MC-ShufflenetV2 model’s recognition ability and size have obvious advantages, and it is more conducive to the actual deployment of the agricultural mobile terminal.

Investigating and Engineering an 1,2-Propanediol-Responsive Transcription Factor-Based Biosensor.

Transcription factor (TF)-based biosensors have arisen as powerful tools in the advancement of metabolic engineering. However, with the emergence of numerous bioproduction targets, the variety of applicable TF-based biosensors remains severely limited. In this study, we investigated and engineered an 1,2-propanediol (1,2-PD)-responsive transcription activator, PocR, from Salmonella typhimurium to enrich the current biosensor repertoire. Heterologous characterization of PocR in E. coli revealed a significantly limited operational range and dynamic range, primarily attributed to the leaky binding between PocR and its corresponding promoters in the absence of the 1,2-PD inducer. Promiscuity characterization uncovered the minor responsiveness of PocR toward glycerol and 1,2-butanediol (1,2-BD). Using AlphaFold-predicted structure and protein mutagenesis, we preliminarily explored the underlying mechanism of PocR. Based on the investigated mechanism, we engineered a PcoR-F46R/G105D variant with an altered inducer specificity to glycerol, as well as a PocR-ARE (Q107A/S192R/A203E) variant with nearly a 4-fold higher dynamic range (6.7-fold activation) and a 20-fold wider operational range (0-20 mM 1,2-PD). Finally, we successfully converted PocR to a repressor through promoter engineering. Integrating the activation and repression functions established a versatile 1,2-PD-induced bifunctional regulation system based on PocR-ARE. Our work showcases the exploration and exploitation of an underexplored type of transcriptional activator capable of recruiting RNA polymerase. It also expands the biosensor toolbox by providing a 1,2-PD-responsive bifunctional regulator and glycerol-responsive activator.

Effects of virtual reality-based robot therapy combined with task-oriented therapy on upper limb function and cerebral cortex activation in patients with stroke.

This study aimed to investigate the effects of virtual reality (VR)-based robot therapy combined with task-oriented therapy on cerebral cortex activation and upper limb function in patients with stroke. This study included 46 patients with hemiplegia within 1 year of stroke onset. Patients were divided into an experimental group (n = 23) and a control group (n = 23) using a computer randomization program. The experimental group received VR-based robot and task-oriented therapies, whereas the control group received only task-oriented therapy. All participants received interventions for 40 minutes per session, 5 times a week, for 8 weeks. For the pre- and post-evaluation of all participants, the Fugl-Meyer Assessment for the upper extremity, manual function test, motor activity log, and Jebsen-Taylor Hand Function Test were used to evaluate changes in upper limb function and motor-evoked potential amplitudes were measured to compare cerebral cortex activation. In comparison to the control group, experimental group demonstrated an improvement in the function of the upper limb (P < .01) and activation of the cerebral cortex (P < .01). The combined intervention of VR-based robot and task-oriented therapies is valuable for improving upper limb function and cerebral cortex activation in patients with stroke.

The R-mAtrIx Net

We provide a novel neural network architecture that can: i) output R-matrix for a given quantum integrable spin chain, ii) search for an integrable Hamiltonian and the corresponding R-matrix under assumptions of certain symmetries or other restrictions, iii) explore the space of Hamiltonians around already learned models and reconstruct the family of integrable spin chains which they belong to. The neural network training is done by minimizing loss functions encoding Yang–Baxter equation, regularity and other model-specific restrictions such as hermiticity. Holomorphy is implemented via the choice of activation functions. We demonstrate the work of our neural network on the spin chains of difference form with two-dimensional local space. In particular, we reconstruct the R-matrices for all 14 classes. We also demonstrate its utility as an Explorer, scanning a certain subspace of Hamiltonians and identifying integrable classes after clusterisation. The last strategy can be used in future to carve out the map of integrable spin chains with higher dimensional local space and in more general settings where no analytical methods are available.

Application of Artificial Intelligence to the Alert of Explosions in Colombian Underground Mines

AbstractThe use of Artificial Intelligence (AI), particularly of Artificial Neural Networks (ANN), in alerting possible scenarios of methane explosions in Colombian underground mines is illustrated by the analysis of an explosion that killed twelve miners. A combination of geological analysis, a detailed characterization of samples of coal dust and scene evidence, and an analysis with physical modeling tools supported the hypothesis of the existence of an initial methane explosion ignited by an unprotected tool that was followed by a coal dust explosion. The fact that one victim had a portable methane detector at the moment of the methane explosion suggested that the ubiquitous use of these systems in Colombian mines could be used to alert regulatory agencies of a possible methane explosion. This fact was illustrated with the generation of a database of possible readouts of methane concentration based on the recreation of the mine atmosphere before the explosion with Computational Fluid Dynamics (CFD). This database was used to train and test an ANN that included an input layer with two nodes, two hidden layers, each with eight nodes, and an output layer with one node. The inner layers applied a rectified linear unit activation function and the output layer a Sigmoid function. The performance of the ANN algorithm was considered acceptable as it correctly predicted the need for an explosion alert in 971.9 per thousand cases and illustrated how AI can process data that is currently discarded but that can be of importance to alert about methane explosions.

A Novel Error-Based Adaptive Feedback Zeroing Neural Network for Solving Time-Varying Quadratic Programming Problems

This paper introduces a novel error-based adaptive feedback zeroing neural network (EAF-ZNN) to solve the time-varying quadratic programming (TVQP) problem. Compared to existing variable gain ZNNs, the EAF-ZNN dynamically adjusts the parameter to adaptively accelerate without increasing to very large values over time. Unlike adaptive fuzzy ZNN, which only considers the current convergence error, EAF-ZNN ensures regulation by introducing a feedback regulation mechanism between the current convergence error, the historical cumulative convergence error, the change rate of the convergence error, and the model gain parameter. This regulation mechanism promotes effective neural dynamic evolution, which results in high convergence rate and accuracy. This paper provides a detailed analysis of the convergence of the model, utilizing four distinct activation functions. Furthermore, the effect of changes in the proportional, integral, and derivative factors in the EAF-ZNN model on the rate of convergence is explored. To assess the superiority of EAF-ZNN in solving TVQP problems, a comparative evaluation with three existing ZNN models is performed. Simulation experiments demonstrate that the EAF-ZNN model exhibits a superior convergence rate. Finally, the EAF-ZNN model is compared with the other three models through the redundant robotic arms example, which achieves smaller position error.

Tuning the activation function to optimize the forecast horizon of a reservoir computer

Abstract Reservoir computing is a machine learning framework where the readouts from a nonlinear system (the reservoir) are trained so that the output from the reservoir, when forced with an input signal, reproduces a desired output signal. A common implementation of reservoir computers is to use a recurrent neural network as the reservoir. The design of this network can have significant effects on the performance of the reservoir computer. In this paper we study the effect of the node activation function on the ability of reservoir computers to learn and predict chaotic time series. We find that the Forecast Horizon (FH), the time during which the reservoir’s predictions remain accurate, can vary by an order of magnitude across a set of 16 activation functions used in machine learning. By using different functions from this set, and by modifying their parameters, we explore whether the entropy of node activation levels or the curvature of the activation functions determine the predictive ability of the reservoirs. We find that the FH is low when the activation function is used in a region where it has low curvature, and a positive correlation between curvature and FH. For the activation functions studied we find that the largest FH generally occurs at intermediate levels of the entropy of node activation levels. Our results show that the performance of reservoir computers is very sensitive to the activation function shape. Therefore, modifying this shape in hyperparameter optimization algorithms can lead to improvements in reservoir computer performance.

Heterogeneous wireless network selection using feed forward double hierarchy linguistic neural network

Network selection in heterogeneous wireless networks (HWNs) is a complex issue that requires a thorough understanding of service features and user preferences. This is because the various wireless access technologies have varying capabilities and limitations, and the best network for a voice, video, and data service depends on a variety of factors. For selecting the optimal network in HWNs, varying factors such as the user’s position, accessible network resources, service quality requirements, and user preferences must be considered. The classical decision making procedure is very difficult and uncertain to select the desirable HWNs for voice, video, and data. Therefore, we develop a novel decision making model based on feed-forward neural networks under the double hierarchy linguistic information for the selection of the best HWNs for voice, video, and data. In this article, we introduce a novel feed-forward double hierarchy linguistic neural network using the Hamacher t-norm and t-conorm. Further, the feed-forward double hierarchy linguistic neural network applies to the decision making model for the selection of the best HWNs for voice, video, and data. In this novel decision making model, we first take the given data about HWNs and use the converting function to convert the given data into a double hierarchy linguistic term set. We calculate the hidden layer and output layer information by using Hamacher aggregation operations. Finally, we use the sigmoid activation function on the output layer information to decide on the best HWNs for voice, video, and data according to ranking. The proposed approach is compared with other existing models of decision making and the results of the comparison show that the proposed technique is applicable and reliable for the decision support model.

GAINN: The Galaxy Assembly and Interaction Neural Networks for High-redshift JWST Observations

We present the Galaxy Assembly and Interaction Neural Networks (Gainn), a series of artificial neural networks for predicting the redshift, stellar mass, halo mass, and mass-weighted age of simulated galaxies based on James Webb Space Telescope (JWST) photometry. Our goal is to determine the best neural network for predicting these variables at 11 < z < 15. The parameters of the optimal neural network can then be used to estimate these variables for real, observed galaxies. The inputs of the neural networks are JWST filter magnitudes of a subset of five broadband filters (F150W, F200W, F277W, F356W, and F444W) and two medium-band filters (F162M and F182M). We compare the performance of the neural networks using different combinations of these filters, as well as different activation functions and numbers of layers. The best neural network predicted redshift with a normalized rms error of 0.010−0.001+0.003 , stellar mass with rms = 0.089−0.022+0.044 , halo mass with a mean-squared error of 0.022−0.008+0.014 , and mass-weighted age with rms = 12.466−2.408+5.065 . We also test the performance of Gainn on real data from MACS0647JD, an object observed by JWST. Predictions from Gainn for the first projection of the object (JD1) have normalized bias 〈Δz〉 < 0.00228, which is significantly smaller than found with template-fitting methods. We find that the optimal filter combination is F277W, F356W, F162M, and F200W when considering both theoretical accuracy and observational resources from JWST.

Firefly Interval Selection Combined With Extreme Learning Machine for Spectral Quantification of Complex Samples

ABSTRACTFirefly algorithm (FA) combined with extreme learning machine (ELM) is developed for spectral interval selection and quantitative analysis of complex samples. The method firstly segments the spectra into a certain number of intervals. Vectors with 1 and 0, which represent the interval selected or not, are used as the inputs of the FA. The RMSEP value predicted by ELM model is used as the fitness function of the FA. The activation function and number of hidden layer nodes of ELM, number of spectral intervals, population number, environmental absorbance, and constant of FA are optimized. The predictive performance of FA‐ELM is compared with full‐spectrum PLS, ELM, genetic algorithm‐ELM (GA‐ELM), and particle swarm optimization‐ELM (PSO‐ELM) by one ultraviolet (UV) spectrum dataset of gasoil and three near‐infrared (NIR) spectral datasets of corn, wheat, and tablet samples. The results show that FA‐ELM has a better performance compared with its competitors in predicting monoaromatics, water, wheat kernel texture, and active pharmaceutical ingredients (APIs) in gasoil, corn, wheat, and tablet samples.

The antagonistic role of an E3 ligase pair in regulating plant NLR-mediated autoimmunity and fungal pathogen resistance

Plant immune homeostasis is achieved through a balanced immune activation and suppression, enabling effective defense while averting autoimmunity. In Arabidopsis, disrupting a mitogen-activated protein (MAP) kinase cascade triggers nucleotide-binding leucine-rich-repeat (NLR) SUPPRESSOR OF mkk1/2 2 (SUMM2)-mediated autoimmunity. Through an RNAi screen, we identify PUB5, a putative plant U-box E3 ligase, as a critical regulator of SUMM2-mediated autoimmunity. In contrast to typical E3 ligases, PUB5 stabilizes CRCK3, a calmodulin-binding receptor-like cytoplasmic kinase involved in SUMM2 activation. A closely related E3 ligase, PUB44, functions oppositely with PUB5 to degrade CRCK3 through monoubiquitylation and internalization. Furthermore, CRCK3, highly expressed in roots and conserved across plant species, confers resistance to Fusarium oxysporum, a devastating soil-borne fungal pathogen, in both Arabidopsis and cotton. These findings demonstrate the antagonistic role of an E3 ligase pair in fine-tuning kinase proteostasis for the regulation of NLR-mediated autoimmunity and highlight the function of autoimmune activators in governing plant root immunity against fungal pathogens.

Female Sexual Function following Pelvic Organ Prolapse Reconstruction: A Case Serial Study

Background: Pelvic organ prolapse (POP) may reduce women's quality of life and is one of the most important aspects in the occurrence of sexual dysfunction. Objective: This study aimed to assess sexual function in POP women with pelvic floor dysfunction pre and post-surgery. Methods: This study was a prospective cohort study. Using the total sampling technique at Arifin Achmad Hospital, Indonesia, the study recruited women who underwent POP surgery with active sexual function from July 2020 to December 2021. The pelvic floor disorder was evaluated through Pelvic Floor Distress Inventory (PFDI) questionnaires validated in the Indonesian language, pre, and post-surgery. The assessment of sexual function was carried out using the Indonesian linguistic validation of the Female Sexual Function Index (FSFI) questionnaires before and after surgery. Results: There was a decrease in the average value of pelvic floor disorder scores before and six months after surgery (P&lt; 0.05). However, the sexual function increased with a cut-off score of FSFI &gt; 20.25 at 3 and 6 months post-surgery, respectively as much as 61.1% (11/18 subjects) and 72.2% (13/18 subjects), compared to the pre-surgery sexual function scores by 7.8% (5/18 subjects). Out of the 6 domains of sexual function, an increase was found in the sexual arousal domain (P&lt;0.05). Conclusion: There is a decrease in pelvic floor disorder and an increase in sexual function, particularly an increment in sexual arousal at six months post-surgery.

Round-robin scheduling protocol-based stabilization for discrete-time Markov jumping neural networks

This paper focuses on the stabilization problem of discrete-time Markov jumping neural networks (DMJNNs) under communication constraints between the actuators and the remote controller. To reduce the communication burden, a round-robin scheduling protocol is introduced to allocate the right to a single actuator to exclusively use the communication channel at a given time while ensuring that multiple nodes can take turns to occupy the channel for signal transmission. Two criteria for stochastic analysis of the closed-loop network model are presented based on different assumptions of the activation functions. Then, numerically tractable design methods for the required controller gains are developed. Finally, the three-mode DMJNN model is used as an example to illustrate the complementarity and effectiveness of the presented analysis results and design methods.

Construction of rGO and GSH Electrochemical Sensor by Electrodeposition for Naringenin Sensing

Naringenin (NRG), a flavanone compound present in citrus fruits, has a variety of beneficial physiological active functions such as antioxidant, anti-inflammatory, and hypoglycaemic. In this study, an sensor was constructed by electrodeposition and used for the electrochemical study of NRG. Reduced graphene oxide (rGO) and glutathione (GSH) showed the ability to synergistically amplify NRG signals and possessed good linearity in the concentration range of 10.00–1200.00 μmol l−1. The linear equation is Ip = 0.0776logc + 0.9353 (R2 = 0.9901), and the limit of detection is 3.33 μmol l−1. The sensor performed well in terms of reproducibility, stability, and selectivity, which in turn enabled the detection of NRG in tomatoes. The average recovery of the sensor is 95.68% to 111.92%, with RSD less than 11.89%. The results were also verified by Ultraviolet–visible spectroscopy(UV-vis). Furthermore, density-functional theory was employed to analyse the front track of the NRG, speculating that the NRG underwent a transfer of two electrons and two protons.