Gradient Descent Adaptive Learning Rate Research Articles

Sentiment Analysis: An empirical comparison between various training algorithms for Artificial Neural Network

The proliferated increase in the commercial benefits of sentiment analysis accumulated a huge interest in the domain of sentiment classification. Sentiment analysis categorises a given text into positive or negative class. In this paper, we present an empirical comparison between different training algorithms gradient descent (GD), gradient descent with momentum backpropagation (GDM), gradient descent adaptive learning rate backpropagation (GDA), gradient descent with momentum and adaptive learning rate backpropagation (GDX), and Levenberg-Marquardt backpropagation (LM), used for training the neural network for the domain of sentiment classification. The performance of all the methods is compared and evaluated using three balanced binary datasets from various domains with different features using various performance metrics such as accuracy, precision, recall, f-score, mean squared error, and training time. The experiments are performed five times with different random seed values using 10-fold cross-validation. The results indicate that GDX and LM outperform other methods in terms of classification accuracy.

Implementation of Artificial Intelligence for Classification of Frogs in Bioacoustics

This research presents the implementation of artificial intelligence (AI) for classification of frogs in symmetry of the bioacoustics spectral by using the feedforward neural network approach (FNNA) and support vector machine (SVM). Recently, the symmetry concept has been applied in physics, and in mathematics to help make mathematical models tractable to achieve the best learning performance. Owing to the symmetry of the bioacoustics spectral, feature extraction can be achieved by integrating the techniques of Mel-scale frequency cepstral coefficient (MFCC) and mentioned machine learning algorithms, such as SVM, neural network, and so on. At the beginning, the raw data information for our experiment is taken from a website which collects many kinds of frog sounds. This in fact saves us collecting the raw data by using a digital signal processing technique. The generally proposed system detects bioacoustic features by using the microphone sensor to record the sounds of different frogs. The data acquisition system uses an embedded controller and a dynamic signal module for making high-accuracy measurements. With regard to bioacoustic features, they are filtered through the MFCC algorithm. As the filtering process is finished, all values from ceptrum signals are collected to form the datasets. For classification and identification of frogs, we adopt the multi-layer FNNA algorithm in machine learning and the results are compared with those obtained by the SVM method at the same time. Additionally, two optimizer functions in neural network include: scaled conjugate gradient (SCG) and gradient descent adaptive learning rate (GDA). Both optimization methods are used to evaluate the classification results from the feature datasets in model training. Also, calculation results from the general central processing unit (CPU) and Nvidia graphics processing unit (GPU) processors are evaluated and discussed. The effectiveness of the experimental system on the filtered feature datasets is classified by using the FNNA and the SVM scheme. The expected experimental results of the identification with respect to different symmetry bioacoustic features of fifteen frogs are obtained and finally distinguished.

PM<sub>10</sub> Forecasting Using Soft Computing Techniques

Air quality forecasting has acquired great significance in environmental sciences due to its adverse affects on humans and the environment. The artificial neural network is one of the most common soft computing techniques that can be applied for modeling such complex problem. This study designed air quality forecasting model using three-layer FFNN's and recurrent Elman network to forecast PM10 air pollutant concentrations 1 day advance in Yilan County, Taiwan. Then, the optimal model is selected based on testing performance measurements (RMSE, MAE, r, IA and VAF) and learning time. This study used an hourly historical data set from 1/1/2009-31/12/2011 collected by Dongshan station. The data was entirely pre-processed and cleared form missing and outlier values then transformed into daily average values. The final results showed that the three-layer FFNN with One Step Secant (OSS) training algorithm achieved better results than Elman network with Gradient Descent adaptive learning rate (GDX) training algorithm. Where, the FFNN required the less training time and achieved better performance in forecasting PM10 concentrations. Also, the testing performance measurements shown that the selected daily average input variables in previous day (PM_2.5), relative humidity, PM10, temperature, wind direction and speed is critical to give better forecasting accuracy. Whereas, the testing measurements RMSE = 6.23 μg/m₃, MAE = 4.75 μg/m₃, r = 0.943, IA = 0.964 and VAF = 88.80 in PM₁₀ FFNN forecasting model that used OSS training algorithm.

Implementasi Jaringan Syaraf Tiruan Recurrent Dengan Metode Pembelajaran Gradient Descent Adaptive Learning Rate Untuk Pendugaan Curah Hujan Berdasarkan Peubah Enso

The use of technology of technology Artificial Neural Network (ANN) in prediction of rainfall can be done using the learning approach. ANN prediction accuracy measured by the coefficient of determination (R2) and Root Mean Square Error (RMSE).This research employ a recurrent optimized heuristic Artificial Neural Network (ANN) Recurrent Elman gradient descent adaptive learning rate approach using El-Nino Southern Oscilation (ENSO) variable, namely Wind, Southern Oscillation Index (SOI), Sea Surface Temperatur (SST) dan Outgoing Long Wave Radiation (OLR) to forecast regional monthly rainfall. The patterns of input data affect the performance of Recurrent Elman neural network in estimation process. The first data group that is 75% training data and 25% testing data produce the maximum R2 69.2% at leap 0 while the second data group that is 50% training data & 50% testing data produce the maximum R2 53.6%.at leap 0 Our result on leap 0 is better than leap 1,2 or 3.