Improved Activation Function Research Articles

Federated learning with differential privacy via fast Fourier transform for tighter-efficient combining.

Spurred by the simultaneous need for data privacy protection and data sharing, federated learning (FL) has been proposed. However, it still poses a risk of privacy leakage in it. This paper, an improved Differential Privacy (DP) algorithm to protect the federated learning model. Additionally, the Fast Fourier Transform (FFT) is used in the computation of the privacy budget [Formula: see text], to minimize the impact of limited arithmetic resources and numerous users on the effectiveness of training model. Moreover, instead of direct analyses of the privacy budget ε through various methods, Privacy Loss Distribution (PLD) and privacy curves are adopted, while the number of artificial assignments hyperparameters is reduced, and the grid parameters delineated for FFT use are improved. The improved algorithm tightens parameter bounds and minimizes human factors' influence with minimal efficiency impact. It decreases the errors caused by truncation and discreteness of PLDs while expanding the discreteness interval to reduce the calculation workload. Furthermore, an improved activation function using a temper sigmoid with only one parameter [Formula: see text], smooths the accuracy curve and mitigates drastically fluctuating scenarios during model training. Finally, simulation results on real datasets show that our improved DP algorithm, which accounts for long trailing, facilitates a better balance between privacy and utility in federated learning models.

A Fixed-Time Noise-Tolerance ZNN Model for Time-Variant Inequality-Constrained Quaternion Matrix Least-Squares Problem.

Presently, numerical algorithms for solving quaternion least-squares problems have been intensively studied and utilized in various disciplines. However, they are unsuitable for solving the corresponding time-variant problems, and thus few studies have explored the solution to the time-variant inequality-constrained quaternion matrix least-squares problem (TVIQLS). To do so, this article designs a fixed-time noise-tolerance zeroing neural network (FTNTZNN) model to determine the solution of the TVIQLS in a complex environment by exploiting the integral structure and the improved activation function (AF). The FTNTZNN model is immune to the effects of initial values and external noise, which is much superior to the conventional zeroing neural network (CZNN) models. Besides, detailed theoretical derivations about the global stability, the fixed-time (FXT) convergence, and the robustness of the FTNTZNN model are provided. Simulation results indicate that the FTNTZNN model has a shorter convergence time and superior robustness compared to other zeroing neural network (ZNN) models activated by ordinary AFs. At last, the construction method of the FTNTZNN model is successfully applied to the synchronization of Lorenz chaotic systems (LCSs), which shows the practical application value of the FTNTZNN model.

Detection of breast cancer by deep belief network with improved activation function

SummaryBreast cancer is the most prevalent kind of tumor to occur in females and the primary cause of death for women. Early detection is perhaps the most successful strategy to minimize breast cancer mortality. Early diagnosis necessitates a consistent and efficient diagnostics method that allows doctors to differentiate benign from malignant breast cancers without a surgical sample. The goal of this endeavor is to develop a sophisticated breast cancer diagnosis method. The primary goal of the paper is to reduce the death rate among women by promoting early detection of breast cancer. First, pre‐processing techniques such as median filtering and contrast limiting adaptive histogram equalization are used to the obtained raw images. By doing this, the machine‐learning model's computational complexity is decreased and the image quality is enhanced. K‐means clustering is used to segregate the pre‐processed image. Additionally, features including the enhanced local vector pattern, grey‐level co‐occurrence matrix and local vector patterns are produced in the course of the feature extraction stage. Finally, an optimized deep belief network (DBN) is carrying out the classification process. To boosts the classification accuracy, activation function of DBN (tanh, softmax, ReLu) as well as its weight function is optimized by the proposed grey wolf updated whale optimization algorithm The accuracy of the greywolf updated whale optimization algorithm+DBN is above 93% for datasets 1 and 2 when compared to extant models. Finally, calculation of the performance validates the proposed model's performance.

Oil painting image style recognition based on ResNet-NTS network

With the improvement of economic level, cultural and artistic pursuits have become the new life style of people. For painting exhibition style appreciation, the study uses deep learning and neural networks to classify and recognize the style identification of oil painting images. Firstly, the classification technology of residual networks is used to collect features and enhance regions in images of Western oil painting art. Secondly, the residual network is optimized and improved. Finally, attention mechanism and improved activation function are introduced to achieve the recognition effect of oil painting image style. Through comparative experiments, the results show that the improved residual network has an accuracy of 69.38% in style recognition of oil painting datasets. After optimizing with cosine annealing decay strategy, the network testing system model has the highest recognition accuracy of 74.41% for oil painting style data. Finally, the residual network and the network test system are respectively used in the style recognition of Xizang Thangka art dataset, and the accuracy of style recognition of the improved residual network and the network test system is 87.39% and 95.28%, respectively. Therefore, it proves that the residual network and network test system model based on the residual network and network test system model have superiority in the effect of oil painting style recognition, and improve technical support for art dissemination and management.

An Improved Activation Function in Convolution Neural Network to Estimate the Hazardous Air Pollutant Based on Images

An Improved Activation Function in Convolution Neural Network to Estimate the Hazardous Air Pollutant Based on Images

Quantitative identification of debonding defects in building façades based on UAV-thermography using a two-stage network integrating dual attention mechanism

The debonding defects in building façades pose a serious threat to the safety of residents. In this paper, a two-stage quantitative network for debonding defect identification quickly and accurately based on deep learning is proposed. Firstly, the rotor UAV equipped with an infrared thermal imager is applied as the working platform to detect the debonding defects in building façades. Then, the target detection network combining dual attention mechanism, improved activation function, and bilinear interpolation has been proposed to accurately recognize infrared images and suppress background interference. Further, the semantic segmentation network with channel attention mechanism has been proposed to obtain more accurate defect area boundaries and shape information. Finally, compared with the classical deep learning networks, the results show that the improved algorithm can accurately identify the type and shape information of debonding defects.

Image blind motion deblurring method with longitudinal channel and wavelet dynamic convolution

Deep learning methods have achieved great success in image deblurring. However, these methods also exist two limitations, the detailed information loss caused by the image encoding, and the poor robustness caused by fixed weight parameters. To address the two issues, we propose an image deblurring method based on DeblurGAN, which is good at removing motion blurs. Firstly, two structures are specially designed, respectively. On one hand, the longitudinal channel network structure is proposed. It can directly fuse the original image features into each feature layer of the encoder, so as to retain more detailed information of the original image. On the other hand, the wavelet dynamic convolution module is applied to the feature extraction process. It can generate dynamic wavelet kernels according to different input images rather than fixed convolution parameters. In addition, we use the improved activation function and loss function to retrain the network. The quantitative experimental results show that the PSNR and SSIM metrics of our method can reach 29.9605 and 0.9177 on the GoPro test set, respectively. The qualitative experimental results indicate that our method can retain the original image details well, and achieve promising robustness.

Knowledge features enhanced intelligent fault detection with progressive adaptive sparse attention learning for high-power diesel engine

High-power diesel engines are core power equipment in some key fields, and fault diagnosis is of great significance for improving their long-term operational reliability and safety. The lack of sufficient fault samples, known as few-shot conditions, has resulted in the performance degradation of most current intelligent diagnostic methods under complex and variable working conditions. Moreover, the end-to-end deep learning diagnostic models based on implicit layer features do not have clear physical meanings, making it difficult for intelligent diagnostic models to be recognized by equipment managers in key fields. Mechanism-based fusion methods can effectively alleviate the above problems by incorporating fault-sensitive features with clear physical meanings. However, the lack of sufficient knowledge-related features that can characterize fault information is a challenge facing current mechanism fusion methods. In this work, we propose a progressive adaptive sparse attention mechanism-guided knowledge feature incremental mining method. Fault-sensitive segments are obtained based on attention-supervised learning with an improved activation function. Both knowledge-related features and hidden layer features from these segments are incorporated into the feature library. A differential constraint is designed between the distribution of knowledge features and hidden layer features, encouraging the model to focus on learning new knowledge-related features in the subsequent iteration process. Finally, a feature set containing a large amount of knowledge features is constructed to enhance fault diagnosis. Testing on a V12 diesel engine test bench under the conditions of rich and few-shot data sets, shows that the proposed method significantly enhances the fault recognition performance of diesel engines under complex variable working conditions and few-shot data set conditions.

Efficacy of the combination therapy of platelet-rich plasma and hyaluronic acid on improving knee pain and dysfunction in patients with moderate-to-severe KOA: a protocol for a randomised controlled trial

Introduction54% of patients with moderate-to-severe knee osteoarthritis (KOA) still reported persistent pain and functional loss after conservative treatment according to guidelines. As an emerging treatment, platelet-rich plasma (PRP) has been...

Estimating Body Weight in Captive Rabbits Based on Improved Mask RCNN

Automated body weight (BW) estimation is an important indicator to reflect the automation level of breeding, which can effectively reduce the damage to animals in the breeding process. In order to manage meat rabbits accurately, reduce the frequency of manual intervention, and improve the intelligent of meat rabbit breeding, this study constructed a meat rabbit weight estimation system to replace manual weighing. The system consists of a meat rabbit image acquisition robot and a weight estimation model. The robot stops at each cage in turn and takes a top view of the rabbit through an RGB camera. The images from the robot are automatically processed in the weight estimation model, which consists of the meat rabbit segmentation network based on improved Mask RCNN and the BW fitting network. Attention mechanism, PointRend algorithm, and improved activation function are proposed to improve the performance of Mask RCNN. Six morphological parameters (relative projected area, contour perimeter, body length, body width, skeleton length, and curvature) are extracted from the obtained mask, and are sent into the BW fitting network based on SVR-SSA-BPNN. The experiment shows that the system achieves a 4.3% relative error and 172.7 g average absolute error in BW estimation for 441 rabbits, while the meat rabbit segmentation network achieves a 99.1% mean average precision (mAP) and a 98.7% mean pixel accuracy (MPA). The system provides technical support for automatic BW estimation of meat rabbits in commercial breeding, which is helpful to promote precision breeding.

Flame smoke detection algorithm based on YOLOv5 in petrochemical plant

PurposeFire smoke detection in petrochemical plant can prevent fire and ensure production safety and life safety. The purpose of this paper is to solve the problem of missed detection and false detection in flame smoke detection under complex factory background.Design/methodology/approachThis paper presents a flame smoke detection algorithm based on YOLOv5. The target regression loss function (CIoU) is used to improve the missed detection and false detection in target detection and improve the model detection performance. The improved activation function avoids gradient disappearance to maintain high real-time performance of the algorithm. Data enhancement technology is used to enhance the ability of the network to extract features and improve the accuracy of the model for small target detection.FindingsBased on the actual situation of flame smoke, the loss function and activation function of YOLOv5 model are improved. Based on the improved YOLOv5 model, a flame smoke detection algorithm with generalization performance is established. The improved model is compared with SSD and YOLOv4-tiny. The accuracy of the improved YOLOv5 model can reach 99.5%, which achieves a more accurate detection effect on flame smoke. The improved network model is superior to the existing methods in running time and accuracy.Originality/valueAiming at the actual particularity of flame smoke detection, an improved flame smoke detection network model based on YOLOv5 is established. The purpose of optimizing the model is achieved by improving the loss function, and the activation function with stronger nonlinear ability is combined to avoid over-fitting of the network. This method is helpful to improve the problems of missed detection and false detection in flame smoke detection and can be further extended to pedestrian target detection and vehicle running recognition.

Balance optimization method of energy shipping based on Hopfield neural network

It is of great significance for the optimization of transportation strategy to study the methods of energy shipping scheduling. Based on the Hopfield neural network (HNN) theory, this paper proposes Hopfield neural network energy transportation path optimization algorithm with improved activation function, which solves the problems of poor mapping ability, low flexibility and high sensitivity of neurons near zero to input of traditional activation function. The improved activation function can reduce the derivative value of the activation function and the sensitivity of neurons near zero to the input value by flexibly adjusting the steepness, positioning and mapping range of energy sea transportation at the same time. The experimental results show that: under the same initial conditions, the model proposed in this paper shows better error code performance, and the data length of the required transmission sequence is shorter. When sending 8PSK (Phase Shift Keying) signal with a data length of N = 40, the error rate of the 2-path synthesized random channel is less than 0.112. The error code performance and convergence speed of the energy shipping path planning algorithm are improved effectively.

Deep Learning for the Detection and Classification of Diabetic Retinopathy with an Improved Activation Function.

Diabetic retinopathy (DR) is an eye disease triggered due to diabetes, which may lead to blindness. To prevent diabetic patients from becoming blind, early diagnosis and accurate detection of DR are vital. Deep learning models, such as convolutional neural networks (CNNs), are largely used in DR detection through the classification of blood vessel pixels from the remaining pixels. In this paper, an improved activation function was proposed for diagnosing DR from fundus images that automatically reduces loss and processing time. The DIARETDB0, DRIVE, CHASE, and Kaggle datasets were used to train and test the enhanced activation function in the different CNN models. The ResNet-152 model has the highest accuracy of 99.41% with the Kaggle dataset. This enhanced activation function is suitable for DR diagnosis from retinal fundus images.

Extended dissipativity-based synchronization of Markov jump neural networks subject to partially known transition and mode detection information

This paper addresses the extended dissipativity-based synchronization problem of Markov jump neural networks with partially known probability information by using a detector from the hidden Markov model, where the partially known probability may exist in one of the transition probability matrix and detection probability matrix, or both of them simultaneously. By using such a hidden Markov model, an extended stochastic dissipative synchronization criterion for neural networks with partially known probability information is established, and a novel design method is given with the help of an improved activation function dividing method. Finally, the validity of the proposed approach is demonstrated by two numerical examples.

Postoperative Cognitive Behavioral Intervention in Patients with Coronary Heart Disease Based on the Effect of Nursing Process Reengineering.

With the development of society and economy, patients with coronary heart disease increasingly hope to receive more humanized services after surgery, and the reengineering of the nursing process is precisely based on humanized management. In order to create a nursing intervention that helps in boosting total nursing gains and lowering disasters, the primary element of the nursing business process reengineering is to reintegrate or condense the creative work flow that was fragmented by specialization into a whole job. The “patient” is positioned in the center of nursing process reengineering, with service quality as the main guiding principle. It follows the management principles of continuous regulation and innovation, and cultivates and promotes the development of the nursing field from the perspective of “sublation.” In the method part, this article introduces and supplements related information on nursing process reengineering, coronary heart disease, and postoperative cognitive behavioral intervention. Algorithmically, this paper proposes 3D RPN structure, loss function, capsule network, and improved activation function algorithm for coronary heart disease region extraction. In the part of experiment and results, this article collects general data of patients and compares memory ability, grades of self-management behaviors of patients treated with coronary heart disease, scores of quality control standards before and after nursing process reengineering, scores of patients' mental state, and comparison of postoperative physical indicators. According to the test results, the test group's diastolic and systolic blood pressure declined by more than 20, their total cholesterol dropped from 6.23 to 5.28, and their triglyceride levels dropped from 1.82 to 1.39 after the stent was implanted. Triglycerides reduced from 1.82 to 1.39, and total cholesterol dropped from 6.15 to 5.98. It indicates postoperative memory in patients with coronary heart disease. Abilities, including recognition, number, touch memory, and associative memory, decline significantly, but it can be improved after receiving the nursing process reengineering. It can be seen that nursing process reengineering can play a positive role in the postoperative cognitive behavior of patients with coronary heart disease.

Flatness Prediction of Cold Rolled Strip Based on Deep Neural Network with Improved Activation Function.

With the improvement of industrial requirements for the quality of cold rolled strips, flatness has become one of the most important indicators for measuring the quality of cold rolled strips. In this paper, the strip production data of a 1250 mm tandem cold mill in a steel plant is modeled by an improved deep neural network (the improved DNN) to improve the accuracy of strip shape prediction. Firstly, the type of activation function is analyzed, and the monotonicity of the activation function is deemed independent of the convexity of the loss function in the deep network. Regardless of whether the activation function is monotonic, the loss function is not strictly convex. Secondly, the non-convex optimization of the loss functionextended from the deep linear network to the deep nonlinear network, is discussed, and the critical point of the deep nonlinear network is identified as the global minimum point. Finally, an improved Swish activation function based on batch normalization is proposed, and its performance is evaluated on the MNIST dataset. The experimental results show that the loss of an improved Swish function is lower than that of other activation functions. The prediction accuracy of a deep neural network (DNN) with an improved Swish function is 0.38% more than that of a deep neural network (DNN) with a regular Swish function. For the DNN with the improved Swish function, the mean square error of the prediction for the flatness of cold rolled strip is reduced to 65% of the regular DNN. The accuracy of the improved DNN is up to and higher than the industrial requirements. The shape prediction of the improved DNN will assist and guide the industrial production process, reducing the scrap yield and industrial cost.

Deep Learning for Typhoon Intensity Classification Using Satellite Cloud Images

Abstract A tropical cyclone, also known as a typhoon, is one of the most destructive weather phenomena. Its intense cyclonic eddy circulations often cause serious damage to coastal areas. Accurate classification or prediction for typhoon intensity is crucial to disaster warning and mitigation management. But typhoon intensity-related feature extraction is a challenging task as it requires significant preprocessing and human intervention for analysis, and its recognition rate is poor due to various physical factors such as tropical disturbance. In this study, we built a Typhoon-CNNs framework, an automatic classifier for typhoon intensity based on a convolutional neural network (CNN). The Typhoon-CNNs framework utilized a cyclical convolution strategy supplemented with dropout zero-set, which extracted sensitive features of existing spiral cloud bands (SCBs) more effectively and reduces the overfitting phenomenon. To further optimize the performance of Typhoon-CNNs, we also proposed the improved activation function (T-ReLU) and the loss function (CE-FMCE). The improved Typhoon-CNNs was trained and validated using more than 10 000 multiple sensor satellite cloud images from the National Institute of Informatics. The classification accuracy reached to 88.74%. Compared with other deep learning methods, the accuracy of our improved Typhoon-CNNs was 7.43% higher than ResNet50, 10.27% higher than InceptionV3, and 14.71% higher than VGG16. Finally, by visualizing hierarchic feature maps derived from Typhoon-CNNs, we can easily identify the sensitive characteristics such as typhoon eyes, dense-shadowing cloud areas, and SCBs, which facilitate classifying and forecasting typhoon intensity.

Usage of intelligent medical aided diagnosis system under the deep convolutional neural network in lumbar disc herniation

In order to improve the diagnosis efficiency of lumbar disc herniation (LDH) and reduce the impact of manual intervention in the traditional computer-aided diagnosis (CAD) system, the magnetic resonance imaging (MRI) images of LDH patients were selected as the research objects in this study. Firstly, the conditional deep convolutional generative adversarial network (CDCGAN) was constructed, and the influence of the improved T-ReLu activation function on the classification effect of the model was analyzed comparatively. Secondly, the model was applied to classification of MRI images of LDH patients to analyzed comparatively the effects of different parameters on the classification accuracy. Then, an aided diagnosis system of LDH covering the MRI feature extraction, 3D modeling, and CDCGAN model classification was built, and the model was tested with the real data. Finally, the Spearman correlation was adopted to analyze the correlation between the MRI quantitative indexes based on the aided diagnosis system of LDH and the course of LDH. It was found that the classification accuracy of the CDCGAN model with improved activation function on the Cifar-100 data set was 96.07%, and the classification accuracy of MRI images of LDH patients was 94.41% when the parameter N was 85. After the cross-validation, it was found that the diagnostic accuracy of the aided diagnosis system of LDH constructed in this study was 94.15% on LDH diseases. In addition, it was found that the Kyphotic angle of the herniated dise value and the relative signal intensity value in the MRI quantitative indicators showed a very significant negative correlation with the prevalence of LDH (P < 0.01), while the index of disc herniation, nucleus protrusion rate, ratio of horizontal deviation angle, and the ratio between the protruded part and the dural sac showed extremely positive correlations with the course of LDH (P < 0.01). It suggested that applying the CDCGAN model to the aided diagnosis system of LDH based on the MRI quantitative indicators could improve the accuracy of diagnosis of LDH.

Intervention of Artificial Neural Network with an Improved Activation Function to Predict the Performance and Emission Characteristics of a Biogas Powered Dual Fuel Engine

Biogas is a significant renewable fuel derived by sources of biological origin. One of today’s research issues is the effect of biofuels on engine efficiency. The experiments on the engine are complicated, time consuming and expensive. Furthermore, the evaluation cannot be carried out beyond the permissible limit. The purpose of this research is to build an artificial neural network successfully for dual fuel diesel engine with a view to overcoming experimental difficulties. Authors used engine load, bio-gas flow rate and n-butanol concentration as input parameters to forecast target variables in this analysis, i.e., smoke, brake thermal efficiency (BTE), carbon monoxide (CO), hydrocarbon (HC), nitrous-oxide (NOx). Estimated values and results of experiments were compared. The error analysis showed that the built model has quite accurately predicted the experimental results. This has been described by the value of Coefficient of determination (R2), which varies between 0.8493 and 0.9863 with the value of normalized mean square error (NMSE) between 0.0071 and 0.1182. The potency of the Nash-Sutcliffe coefficient of efficiency (NSCE) ranges from 0.821 to 0.8898 for BTE, HC, NOx and Smoke. This research has effectively emulated the on-board efficiency, emission, and combustion features of a dual-fuel biogas diesel engine taking the Swish activation mechanism in artificial neural network (ANN) model.

Nonlinear All-Optical Diffractive Deep Neural Network with 10.6 μm Wavelength for Image Classification

A photonic artificial intelligence chip is based on an optical neural network (ONN), low power consumption, low delay, and strong antiinterference ability. The all-optical diffractive deep neural network has recently demonstrated its inference capabilities on the image classification task. However, the size of the physical model does not have miniaturization and integration, and the optical nonlinearity is not incorporated into the diffraction neural network. By introducing the nonlinear characteristics of the network, complex tasks can be completed with high accuracy. In this study, a nonlinear all-optical diffraction deep neural network (N-D2NN) model based on 10.6 μm wavelength is constructed by combining the ONN and complex-valued neural networks with the nonlinear activation function introduced into the structure. To be specific, the improved activation function of the rectified linear unit (ReLU), i.e., Leaky-ReLU, parametric ReLU (PReLU), and randomized ReLU (RReLU), is selected as the activation function of the N-D2NN model. Through numerical simulation, it is proved that the N-D2NN model based on 10.6 μm wavelength has excellent representation ability, which enables them to perform classification learning tasks of the MNIST handwritten digital dataset and Fashion-MNIST dataset well, respectively. The results show that the N-D2NN model with the RReLU activation function has the highest classification accuracy of 97.86% and 89.28%, respectively. These results provide a theoretical basis for the preparation of miniaturized and integrated N-D2NN model photonic artificial intelligence chips.