Leaky ReLU Research Articles

A modified residual network for detection and classification of Alzheimer’s disease

<p>Alzheimer's disease (AD) is a brain disease that significantly declines a person's ability to remember and behave normally. By applying several approaches to distinguish between various stages of AD, neuroimaging data has been used to extract different patterns associated with various phases of AD. However, because the brain patterns of older adults and those in different phases are similar, researchers have had difficulty classifying them. In this paper, the 50-layer ResNet is modified by adding extra convolution layers to make the extracted features more diverse. Besides, the activation function (ReLU) was replaced with (Leaky ReLU) because ReLU takes the negative parts of its input, drops them to zero, and retains the positive parts. These negative inputs may contain useful feature information that could aid in the development of high-level discriminative features. Thus, Leaky ReLU was used instead of ReLU to prevent any potential loss of input information. In order to train the network from scratch without encountering the issue of overfitting, we added a dropout layer before the fully connected layer. The proposed method successfully classified the four stages of AD with an accuracy of 97.49 % and 98 % for precision, recall, and f1-score.</p>

Controlled Growth of Wafer-Scale Transition Metal Dichalcogenides with a Vertical Composition Gradient for Artificial Synapses with High Linearity.

Artificial synapses are promising for dealing with large amounts of data computing. Great progress has been made recently in terms of improving the on/off current ratio, the number of states, and the energy efficiency of synapse devices. However, the nonlinear weight update behavior of a synapse caused by the uncertain direction of the conductive filament leads to complex weight modulation, which degrades the delivery accuracy of information. Here we propose a strategy to improve the weight update behavior of synapses using chemical-vapor-deposition-grown transition metal dichalcogenides (TMDCs) with a vertical composition gradient, where the sulfur concentration decreases gradually along the thickness direction of TMDCs and thus forms a certain direction of the conduction filament for synapse devices. It is worth noting that the devices show an excellent linear conductance of potentiation and depression with a high linearity of 0.994 (surpassing most state-of-the-art synapses), have a large number of states, and are able to fabricate synapse arrays with wafer-scale. Furthermore, the devices based on the TMDCs with the vertical composition gradient exhibit an asymmetric feature of potentiation and depression behaviors with high linearity and follow the simulated linear Leaky ReLU function, resulting in a high recognition accuracy of 94.73%, which overcomes the unreliability issue in the Sigmoid function due to the vanishing gradient phenomenon. This study not only provides a universal method to grow TMDCs with a vertical composition gradient but also contributes to exploring highly linear synapses toward neuromorphic computing.

Short-Term Electrical Load Demand Forecasting Based on LSTM and RNN Deep Neural Networks

As the development of smart grids is increasing, accurate electric load demand forecasting is becoming more important for power systems, because it plays a vital role to improve the performance of power companies in terms of less operating cost and reliable operation. Short-term load forecasting (STLF), which focuses on the prediction of few hours to one week ahead predictions and is also beneficial for unit commitment and cost-effective operation of smart power grids, is receiving increasing attention these days. Development and selection of an accurate forecast model from different artificial intelligence (AI)-based techniques and meta-heuristic algorithms for better accuracy is a challenging task. Deep Neural Network (DNN) is a group of intelligent computational algorithms which have a viable approach for modeling across multiple hidden layers and complex nonlinear relationships between variables. In this paper, a model for STLF using deep learning neural network (DNN) with feature selection is proposed. A wide range of intelligent forecast models was designed and tested based on multiple activation functions, such as hyperbolic tangent (tanh), different variants of rectifier linear unit (ReLU), and sigmoid. Among the others, DNN with leaky ReLu produced the best forecast accuracy. Regarding the precision of the methods used in this research work, certain output measures, such as absolute percentage error (MAPE), mean square error (MSE), and root mean square error (RMSE) are used. There was also a reliance on multiple parametric and variable details to determine the capability of the smart load forecasting techniques.

YOLO-DSD: A YOLO-Based Detector Optimized for Better Balance between Accuracy, Deployability and Inference Time in Optical Remote Sensing Object Detection

Many deep learning (DL)-based detectors have been developed for optical remote sensing object detection in recent years. However, most of the recent detectors are developed toward the pursuit of a higher accuracy, but little toward a balance between accuracy, deployability and inference time, which hinders the practical application for these detectors, especially in embedded devices. In order to achieve a higher detection accuracy and reduce the computational consumption and inference time simultaneously, a novel convolutional network named YOLO-DSD was developed based on YOLOv4. Firstly, a new feature extraction module, a dense residual (DenseRes) block, was proposed in a backbone network by utilizing a series-connected residual structure with the same topology for improving feature extraction while reducing the computational consumption and inference time. Secondly, convolution layer–batch normalization layer–leaky ReLu (CBL) ×5 modules in the neck, named S-CBL×5, were improved with a short-cut connection in order to mitigate feature loss. Finally, a low-cost novel attention mechanism called a dual channel attention (DCA) block was introduced to each S-CBL×5 for a better representation of features. The experimental results in the DIOR dataset indicate that YOLO-DSD outperforms YOLOv4 by increasing mAP0.5 from 71.3% to 73.0%, with a 23.9% and 29.7% reduction in Params and Flops, respectively, but a 50.2% improvement in FPS. In the RSOD dataset, the mAP0.5 of YOLO-DSD is increased from 90.0~94.0% to 92.6~95.5% under different input sizes. Compared with the SOTA detectors, YOLO-DSD achieves a better balance between the accuracy, deployability and inference time.

An Intelligent Rice Yield Trait Evaluation System Based on Threshed Panicle Compensation.

High-throughput phenotyping of yield-related traits is meaningful and necessary for rice breeding and genetic study. The conventional method for rice yield-related trait evaluation faces the problems of rice threshing difficulties, measurement process complexity, and low efficiency. To solve these problems, a novel intelligent system, which includes an integrated threshing unit, grain conveyor-imaging units, threshed panicle conveyor-imaging unit, and specialized image analysis software has been proposed to achieve rice yield trait evaluation with high throughput and high accuracy. To improve the threshed panicle detection accuracy, the Region of Interest Align, Convolution Batch normalization activation with Leaky Relu module, Squeeze-and-Excitation unit, and optimal anchor size have been adopted to optimize the Faster-RCNN architecture, termed ‘TPanicle-RCNN,’ and the new model achieved F1 score 0.929 with an increase of 0.044, which was robust to indica and japonica varieties. Additionally, AI cloud computing was adopted, which dramatically reduced the system cost and improved flexibility. To evaluate the system accuracy and efficiency, 504 panicle samples were tested, and the total spikelet measurement error decreased from 11.44 to 2.99% with threshed panicle compensation. The average measuring efficiency was approximately 40 s per sample, which was approximately twenty times more efficient than manual measurement. In this study, an automatic and intelligent system for rice yield-related trait evaluation was developed, which would provide an efficient and reliable tool for rice breeding and genetic research.

Landscape Analysis for Shallow Neural Networks: Complete Classification of Critical Points for Affine Target Functions

In this paper, we analyze the landscape of the true loss of neural networks with one hidden layer and ReLU, leaky ReLU, or quadratic activation. In all three cases, we provide a complete classification of the critical points in the case where the target function is affine and one-dimensional. In particular, we show that there exist no local maxima and clarify the structure of saddle points. Moreover, we prove that non-global local minima can only be caused by ‘dead’ ReLU neurons. In particular, they do not appear in the case of leaky ReLU or quadratic activation. Our approach is of a combinatorial nature and builds on a careful analysis of the different types of hidden neurons that can occur.

Digital design of regional characteristic apparel pattern driven by GAN

In the links of apparel product development and production, apparel pattern design cannot reduce its marginal cost through economies of scale because of its creative characteristics. With the world entering the era of industry 4.0, machine learning can provide services for apparel design. This research takes the Chinese characteristic tachisme pattern as the research object and puts forward a new design method of regional characteristic apparel pattern driven by Generative Adversarial Networks (GAN). Firstly, the main framework based on GAN including discrimination and generation modules is established. Aiming at the training difficulties of regional characteristic apparel pattern sample situation with small quantity and disordered specification, the image self-amplification and normalization pre-processing module is added to the model. Secondly, by adding the Batch Normalization mechanism, Leaky ReLU and RMSProp algorithm, the problems of gradient disappearance and overfitting in the experiment are solved, and the convergence speed of the model is improved. Finally, the HSV colour model algorithm is introduced into the loss function to indicate the training process, so that the artistic expression characteristics of the generated results are closer to the human visual perception experience. Through index evaluation comparison, result authenticity investigation and product design practice, we prove the superiority and practicality of the proposed method in this paper. The new design method theoretically solves the scale economy dilemma of the previous apparel pattern design methods and provides reference ideas for more application scenarios currently trapped in the real-time presentation of design results.

A Deep Convolutional Generative Adversarial Networks-Based Method for Defect Detection in Small Sample Industrial Parts Images

Online defect detection in small industrial parts is of paramount importance for building closed loop intelligent manufacturing systems. However, high-efficiency and high-precision detection of surface defects in these manufacturing systems is a difficult task and poses a major research challenge. The small sample size of industrial parts available for training machine learning algorithms and the low accuracy of computer vision-based inspection algorithms are the bottlenecks that restrict the development of efficient online defect detection technology. To address these issues, we propose a small sample gear face defect detection method based on a Deep Convolutional Generative Adversarial Network (DCGAN) and a lightweight Convolutional Neural Network (CNN) in this paper. Initially, we perform data augmentation by using DCGAN and traditional data enhancement methods which effectively increase the size of the training data. In the next stage, we perform defect classification by using a lightweight CNN model which is based on the state-of-the-art Vgg11 network. We introduce the Leaky ReLU activation function and a dropout layer in the proposed CNN. In the experimental evaluation, the proposed framework achieves a high score of 98.40%, which is better than that of the classic Vgg11 network model. The method proposed in this paper is helpful for the detection of defects in industrial parts when the available sample size for training is small.

Predicting future community intrusions using a novel type and encryption mechanism architecture for attack node mitigation

The recent exponential rise in the number of cyber-attacks has demanded intensive study into community intrusion detection, prediction, and mitigation systems. Even though there are a variety of intrusion detection technologies available, predicting future community intrusions is still a work in progress. Existing approaches rely on statistical and/or superficial device mastery techniques to solve the problem, and as a result, feature selection and engineering are required. The truth is that no single classifier can provide the highest level of accuracy for all five types of training data set. Cyber-attack detection is a technique for detecting cyber-attacks as they emerge on a laptop or network device, intending to compromise the gadget's security. As a result, using a novel type and encryption mechanism, this paper offered a unique architecture for attack node mitigation. The input UNSW-NB15 dataset is first acquired and divided into training and testing statistics. First and foremost, the information is pre-processed and capabilities are retrieved in the training section. The Taxicab Woodpecker Mating Algorithm (TWMA) is then used to select the critical characteristics. The attacked and non-attacked information are then classified using the BRELU-ResNet (Bernoulli's Leaky Rectified Linear Unit - Residual Neural Community) classifier. The encrypted at Ease Hash Probability-Based Elliptic-Curve Cryptography (ESHP-ECC) technique is used to encrypt the ordinary facts, which are subsequently kept in the security log report. Following that, using Euclidean distance, the shortest course distance is estimated. Finally, the records are decrypted using a set of principles known as Decrypted Relaxed Hash Probability-Based Elliptic-Curve Cryptography (DSHP-ECC). If the input appears in the log file during testing, it is regarded as attacked data and is prevented from being transmitted. If it isn't found, the procedure of detecting cyber-attacks continues.

A Novel CovidDetNet Deep Learning Model for Effective COVID-19 Infection Detection Using Chest Radiograph Images

The suspected cases of COVID-19 must be detected quickly and accurately to avoid the transmission of COVID-19 on a large scale. Existing COVID-19 diagnostic tests are slow and take several hours to generate the required results. However, on the other hand, most X-rays or chest radiographs only take less than 15 min to complete. Therefore, we can utilize chest radiographs to create a solution for early and accurate COVID-19 detection and diagnosis to reduce COVID-19 patient treatment problems and save time. For this purpose, CovidDetNet is proposed, which comprises ten learnable layers that are nine convolutional layers and one fully-connected layer. The architecture uses two activation functions: the ReLu activation function and the Leaky Relu activation function and two normalization operations that are batch normalization and cross channel normalization, making it a novel COVID-19 detection model. It is a novel deep learning-based approach that automatically and reliably detects COVID-19 using chest radiograph images. Towards this, a fine-grained COVID-19 classification experiment is conducted to identify and classify chest radiograph images into normal, COVID-19 positive, and pneumonia. In addition, the performance of the proposed novel CovidDetNet deep learning model is evaluated on a standard COVID-19 Radiography Database. Moreover, we compared the performance of our approach with hybrid approaches in which we used deep learning models as feature extractors and support vector machines (SVM) as a classifier. Experimental results on the dataset showed the superiority of the proposed CovidDetNet model over the existing methods. The proposed CovidDetNet outperformed the baseline hybrid deep learning-based models by achieving a high accuracy of 98.40%.

A deep learning approach for detecting drill bit failures from a small sound dataset

Monitoring the conditions of machines is vital in the manufacturing industry. Early detection of faulty components in machines for stopping and repairing the failed components can minimize the downtime of the machine. In this article, we present a method for detecting failures in drill machines using drill sounds in Valmet AB, a company in Sundsvall, Sweden that supplies equipment and processes for the production of pulp, paper, and biofuels. The drill dataset includes two classes: anomalous sounds and normal sounds. Detecting drill failure effectively remains a challenge due to the following reasons. The waveform of drill sound is complex and short for detection. Furthermore, in realistic soundscapes, both sounds and noise exist simultaneously. Besides, the balanced dataset is small to apply state-of-the-art deep learning techniques. Due to these aforementioned difficulties, sound augmentation methods were applied to increase the number of sounds in the dataset. In this study, a convolutional neural network (CNN) was combined with a long-short-term memory (LSTM) to extract features from log-Mel spectrograms and to learn global representations of two classes. A leaky rectified linear unit (Leaky ReLU) was utilized as the activation function for the proposed CNN instead of the ReLU. Moreover, an attention mechanism was deployed at the frame level after the LSTM layer to pay attention to the anomaly in sounds. As a result, the proposed method reached an overall accuracy of 92.62% to classify two classes of machine sounds on Valmet’s dataset. In addition, an extensive experiment on another drilling dataset with short sounds yielded 97.47% accuracy. With multiple classes and long-duration sounds, an experiment utilizing the publicly available UrbanSound8K dataset obtains 91.45%. Extensive experiments on our dataset as well as publicly available datasets confirm the efficacy and robustness of our proposed method. For reproducing and deploying the proposed system, an open-source repository is publicly available at https://github.com/thanhtran1965/DrillFailureDetection_SciRep2022.

Identification of Smoke from Straw Burning in Remote Sensing Images with the Improved YOLOv5s Algorithm

Controlling straw burning is important for ensuring the ambient air quality and for sustainable agriculture. Detecting burning straw is vital for managing and controlling straw burning. Existing methods for detecting straw combustion mainly look for combustion products, especially smoke. In this study, the improved You Only Look Once version 5 (YOLOv5s) algorithm was used to detect smoke in Sentinel-2 images captured by remote sensing. Although the original YOLOv5s model had a faster detection speed, its detection accuracy was poor. Thus, a convolutional block attention module was added to the original model. In addition, in order to speed up the convergence of the model, this study replaced the leaky Rectified Linear Unit (leaky ReLU) activation function with the Mish activation function. The accuracy of the improved model was approximately 4% higher for the same detection speed. The improved YOLOv5s had a higher detection accuracy and speed compared to common target detection algorithms, such as RetinaNet, mask Region-Based Convolutional Neural Network (R-CNN), Single-Shot Multibox Detector (SSD), and faster R-CNN. The improved YOLOv5s analyzed an image in 2 ms. In addition, mAP50 exceeded 94%, demonstrating that with this study’s improved method, smoke can be quickly and accurately identified. This work may serve as a reference for improving smoke detection, and for the effective management and control of straw burning.

Deep Neural Networks for Form-Finding of Tensegrity Structures

Analytical paradigms have limited conventional form-finding methods of tensegrities; therefore, an innovative approach is urgently needed. This paper proposes a new form-finding method based on state-of-the-art deep learning techniques. One of the statical paradigms, a force density method, is substituted for trained deep neural networks to obtain necessary information of tensegrities. It is based on the differential evolution algorithm, where the eigenvalue decomposition process of the force density matrix and the process of the equilibrium matrix are not needed to find the feasible sets of nodal coordinates. Three well-known tensegrity examples including a 2D two-strut, a 3D-truncated tetrahedron and an icosahedron tensegrity are presented for numerical verifications. The cases of the ReLU and Leaky ReLU activation functions show better results than those of the ELU and SELU. Moreover, the results of the proposed method are in good agreement with the analytical super-stable lines. Three examples show that the proposed method exhibits more uniform final shapes of tensegrity, and much faster convergence history than those of the conventional one.

Impact of activation functions and number of layers on detection of exudates using circular Hough transform and convolutional neural networks

Convolutional neural networks (CNN), which are used for object detection, are widely used in image recognition and segmentation fields due to their good performance. In this study, a method is proposed for the detection of exudate from DR lesions based on image processing and CNN model with different activation functions and layer numbers. Activation functions are one of the main reasons why the CNN model is complicated in hierarchical structure. CNN models have real artificial intelligence capabilities, especially thanks to non-linear activation functions. Although ReLU is the most commonly used activation function in CNN models, it has some shortcomings in practice. The most important shortcoming is that if the input value is negative, the learning process will slow down due to the inability to take the derivative. In order to solve this problem, the effect of the activation function in the CNN model on a real world problem and its image was investigated in this study. By applying circular Hough transform, one of the image processing methods, the OD region, which has a similar structure to the exuding regions, was determined and removed from the image, and then the detection of the exuding regions was carried out using the CNN model. Exudate detection was performed using ReLU, ELU, Leaky ReLU, Softplus and Swish activation functions in the CNN model and the results were compared. Experimental results in DiaretDB0 and DiaretDB1 databases show that the Swish activation function has a better performance than the others.

Enhancement: SiamFC Tracker Algorithm Performance Based on Convolutional Hyperparameters Optimization and Low Pass Filter

Over the past few decades, convolutional neural networks (CNNs) have achieved outstanding results in addressing a broad scope of computer vision problems. Despite these improvements, fully convolutional Siamese neural networks (FCSNN) still hardly adapt to complex scenes, such as appearance change, scale change, similar objects interference, etc. The present study focuses on an enhanced FCSNN based on convolutional block hyperparameters optimization, a new activation function (ModReLU) and Gaussian low pass filter. The optimization of hyperparameters is an important task, as it has a crucial ascendancy on the tracking process performance, especially when it comes to the initialization of weights and bias. They have to work efficiently with the following activation function layer. Inadequate initialization can result in vanishing or exploding gradients. In the first method, we propose an optimization strategy for initializing weights and bias in the convolutional block to ameliorate the learning of features so that each neuron learns as much as possible. Next, the activation function normalizes the output. We implement the convolutional block hyperparameters optimization by setting the convolutional weights initialization to constant, the bias initialization to zero and the Leaky ReLU activation function at the output. In the second method, we propose a new activation, ModReLU, in the activation layer of CNN. Additionally, we also introduce a Gaussian low pass filter to minimize image noise and improve the structures of images at distinct scales. Moreover, we add a pixel-domain-based color adjustment implementation to enhance the capacity of the proposed strategies. The proposed implementations handle better rotation, moving, occlusion and appearance change problems and improve tracking speed. Our experimental results clearly show a significant improvement in the overall performance compared to the original SiamFC tracker. The first proposed technique of this work surpasses the original fully convolutional Siamese networks (SiamFC) on the VOT 2016 dataset with an increase of 15.42% in precision, 16.79% in AUPC and 15.93% in IOU compared to the original SiamFC. Our second proposed technique also reveals remarkable advances over the original SiamFC with 18.07% precision increment, 17.01% AUPC improvement and an increase of 15.87% in IOU. We evaluate our methods on the Visual Object Tracking (VOT) Challenge 2016 dataset, and they both outperform the original SiamFC tracker performance and many other top performers.

Animal Fur Recognition Algorithm Based on Feature Fusion Network

China is a big country in animal fur industry. The total production and consumption of fur are increasing year by year. However, the recognition of fur in the fur production process still mainly relies on the visual identification of skilled workers, and the stability and consistency of products cannot be guaranteed. In response to this problem, this paper proposes a feature fusion-based animal fur recognition network on the basis of typical convolutional neural network structure, relying on rapidly developing deep learning techniques. This network superimposes texture feature - the most prominent feature of fur image - into the channel dimension of input image. The output feature map of the first layer convolution is inverted to obtain the inverted feature map and concat it into the original output feature map, then Leaky ReLU is used for activation, which makes full use of the texture information of fur image and the inverted feature information. Experimental results show that the algorithm improves the recognition accuracy by 9.08% on Fur_Recognition dataset and 6.41% on CIFAR-10 dataset. The algorithm in this paper can change the current situation that fur recognition relies on manual visual method to classify, and can lay foundation for improving the efficiency of fur production technology.

Improvement of noisy images filtered by bilateral process using a multi-scale context aggregation network

Deep learning has recently received a lot of attention as a feasible solution to a variety of artificial intelligence difficulties. Convolutional neural networks (CNNs) outperform other deep learning architectures in the application of object identification and recognition when compared to other machine learning methods. Speech recognition, pattern analysis, and image identification, all benefit from deep neural networks. When performing image operations on noisy images, such as fog removal or low light enhancement, image processing methods such as filtering or image enhancement are required. The study shows the effect of using Multi-scale deep learning Context Aggregation Network CAN on Bilateral Filtering Approximation (BFA) for de-noising noisy CCTV images. Data-store is used tomanage our dataset, which is an object or collection of data that are huge to enter in memory, it allows to read, manage, and process data located in multiple files as a single entity. The CAN architecture provides integral deep learning layers such as input, convolution, back normalization, and Leaky ReLu layers to construct multi-scale. It is also possible to add custom layers like adaptor normalization (µ) and adaptive normalization (Lambda) to the network. The performance of the developed CAN approximation operator on the bilateral filtering noisy image is proven when improving both the noisy reference image and a CCTV foggy image. The three image evaluation metrics (SSIM, NIQE, and PSNR) evaluate the developed CAN approximation visually and quantitatively when comparing the created de-noised image over the reference image.Compared with the input noisy image, these evaluation metrics for the developed CAN de-noised image were (0.92673/0.76253, 6.18105/12.1865, and 26.786/20.3254) respectively

A Hybrid Deep Learning-Based Approach for Brain Tumor Classification

Brain tumors (BTs) are spreading very rapidly across the world. Every year, thousands of people die due to deadly brain tumors. Therefore, accurate detection and classification are essential in the treatment of brain tumors. Numerous research techniques have been introduced for BT detection as well as classification based on traditional machine learning (ML) and deep learning (DL). The traditional ML classifiers require hand-crafted features, which is very time-consuming. On the contrary, DL is very robust in feature extraction and has recently been widely used for classification and detection purposes. Therefore, in this work, we propose a hybrid deep learning model called DeepTumorNet for three types of brain tumors (BTs)—glioma, meningioma, and pituitary tumor classification—by adopting a basic convolutional neural network (CNN) architecture. The GoogLeNet architecture of the CNN model was used as a base. While developing the hybrid DeepTumorNet approach, the last 5 layers of GoogLeNet were removed, and 15 new layers were added instead of these 5 layers. Furthermore, we also utilized a leaky ReLU activation function in the feature map to increase the expressiveness of the model. The proposed model was tested on a publicly available research dataset for evaluation purposes, and it obtained 99.67% accuracy, 99.6% precision, 100% recall, and a 99.66% F1-score. The proposed methodology obtained the highest accuracy compared with the state-of-the-art classification results obtained with Alex net, Resnet50, darknet53, Shufflenet, GoogLeNet, SqueezeNet, ResNet101, Exception Net, and MobileNetv2. The proposed model showed its superiority over the existing models for BT classification from the MRI images.

A Hybrid Channel-Communication-Enabled CNN-LSTM Model for Electricity Load Forecasting

Smart grids provide a unique platform to the participants of energy markets to tweak their offerings based on demand-side management. Responding quickly to the needs of the market can help to improve the reliability of the system, as well as the cost of capital investments. Electric load forecasting is important because it is used to make and run decisions about the power grid. However, people use electricity in nonlinear ways, which makes the electric load profile a complicated signal. Even though there has been a lot of research done in this field, an accurate forecasting model is still needed. In this regard, this article proposed a hybrid cross-channel-communication (C3)-enabled CNN-LSTM model for accurate load forecasting which helps decision making in smart grids. The proposed model is the combination of three different models, i.e., a C3 block to enable channel communication of a CNN (convolutional neural networks) model, two convolutional layers to extract the features and an LSTM (long short-term memory network) model for forecasting. In the proposed hybrid model, Leaky ReLu (rectified linear unit) was used as activation function instead of sigmoid. The channel communication in CNN model makes the proposed model very light and efficient. Extensive experimentation was done on electricity load data. The results show the model’s high efficiency. The proposed model shows 98.3% accuracy and 0.4560 MAPE error.

Construction and Simulation of Market Risk Warning Model Based on Deep Learning

The market is intricate and complicated, and the existing risk warning models have problems of low efficiency and poor generalization in predicting market risk data. Aiming at the problems, this study takes stock market risk warning as the research object and proposes a market risk warning model based on LSTM-VaR. 15 variables in the three categories of basic transaction data, statistical technical indicators, and moving interval data are selected as the stock market characteristic indicators, the LSTM(Long Short Term Memory) prediction model is constructed and the standard deviation of stock returns is predicted. Based on the predicted results, the probability distribution of return rate under the conditional distribution is obtained, and the VaR(Value at Risk) is calculated. 1% and 5% sample quantiles are taken as the warning line, and the LSTM-VaR warning model is obtained. The results show that the RMSE value of the model is the smallest, which is 0.013762, when the activation function of the LSTM-VaR model is the Leaky ReLu function, the training periods epochs are 10, the time window length N is 9, the batch size is 8, the number of neurons in each layer is 50, the dropout probability is 0.1, and adam is used as the optimizer. Compared with traditional prediction models such as MLP, the proposed model has better performance and can well realize market risk warning.