Neural Network Learning Algorithm Research Articles

Hybrid spam message detection using convolutional neural network and long short-term memory techniques

Short Message Service (SMS) is a feature of a mobile phone that enable convenient and instant way of sending electronic messages between users. As SMS usage increases fraudulent text messages, known as spam, are becoming more common. Spam SMS may result in leaking personal information, invasion of privacy or accessing unauthorized data from mobile devices. Users of mobile phones can mistakingly give away personal information with the assumption that they are sharing it with the right recipients. This work propose a SMS spam detection method that combines convolutional neural network (CNN) and long short term memory (LSTM) deep learning algorithms. The CNN is used for feature extraction while the LSTM classifies the message. The SMS spam dataset, collected from online repository, is used to train the model. Word embeddings is used to vectorize the words in the message to make it suitable for the model. The result obtained from the implementation outperforms other machine learning algorithms with an accuracy of 99.77%.

The Research of Short-term Electric Load Forecasting based on Machine Learning Algorithm

The ability of electric load forecasting become the key to measuring electric planning and dispatching, and improving the accuracy of electric load forecasting has become one hot spot topic of scholars in recent years. The traditional electric load forecasting algorithm mainly include statistical learning algorithm. The electric load forecasting algorithm based on traditional forecasting algorithm, machine learning algorithm and neural network algorithm greatly improve the convergence approximation effect and accuracy. In recent years, the electric load forecasting algorithms based on the deep learning algorithm has greatly reduced the error and improved the measurement accuracy and robustness, such as RNN, GRU, LSTM, DBN, TCN, and so on, as well as the combination algorithm based on deep learning. This paper introduces the classical algorithm of deep learning in electric load forecasting. The main purpose is to explore the error and fitting degree of algorithms established by different algorithms, and provide reference for the selection of forecasting algorithms for various types of electric load forecasting.

Artificial Intelligence Approach for Early Detection of Brain Tumors Using MRI Images

Artificial intelligence (AI) is one of the most promising approaches to health innovation. The use of AI in image recognition considerably extends findings beyond the constraints of human sight. The application of AI in medical imaging, which relies on picture interpretation, is beneficial for automatic diagnosis. Diagnostic radiology is evolving from a subjective perceptual talent to a more objective science thanks to AI. Automatic object detection in medical images is an essential AI technology in medicine. The problem of detecting brain tumors at an early stage is well advanced with convolutional neural network (CNN) and deep learning algorithms (DLA). The problem is that those algorithms require a training phase with a big database of more than 500 images and time-consuming with a complex computational and expensive infrastructure. This study proposes a classical automatic segmentation method for detecting brain tumors in the early stage using MRI images. It is based on a multilevel thresholding technique on a harmony search algorithm (HSO); the algorithm was developed to suit MRI brain segmentation, and parameters selection was optimized for the purpose. Multiple thresholds, based on the variance and entropy functions, break the histogram into multiple portions, and different colors are associated with each portion. To eliminate the tiny arias supposed as noise and detect brain tumors, morphological operations followed by a connected component analysis are utilized after segmentation. The brain tumor detection performance is judged using performance parameters such as Accuracy, Dice Coefficient, and Jaccard index. The results are compared to those acquired manually by experts in the field. The results were further compared with different CNN and DLA approaches using Brain Images dataset called the “BraTS 2017 challenge”. The average Dice Index was used as a performance measure for the comparison. The results of the proposed approach were found to be competitive in accuracy to those obtained by CNN and DLA methods and much better in terms of execution time, computational complexity, and data management.

Prediction of narrow HT-SMA thermal hysteresis behaviour using explainable machine learning

The ability to tailor narrow high-temperature thermal hysteresis behaviour in Ti-Ni-based high-transformation temperature shape memory alloys (HT-SMAs) is crucial for the development of the next-generation actuators intended for use in extreme conditions. Equally, the need and the development of tools to accurately predict shape memory alloys (SMA) transformation temperatures also becomes emergent. As such, the purpose of this study is to curate a small dataset from the experimental peer-reviewed literature and develop a practical data-driven machine learning (ML) based model to predict narrow hysteresis in HT-SMA transformations. This study compiles a small dataset based on experimental data and develops at least twelve candidate models using common machine learning and artificial neural network algorithms. Our results show that the XGBoost-based model exhibits the best performance in predicting the narrow thermal hysteresis behaviour in Ti-Ni-based HT-SMAs (i.e., the highest R2 = 0.893 and the lowest RMSE = 5.4). Further, using preselected case studies, the study demonstrates the utility of the relative feature importance, the accumulated local effects (ALE), and the SHapley Additive exPlanations (SHAP) model-agnostic techniques to interpret predicted outcomes and establish a more transparent relationship between the input materials composition descriptors and the predictions. These analyses demonstrate that the explainable ML approach offers practitioners a more transparent design tool, not only to predict the narrow hysteresis behaviour in HT-SMA transformations, but also a deeper understanding of how the input materials composition descriptors can contribute to its magnitude and/or directionality of the narrowing hysteresis.

A critical review of machine learning approaches for predicting diabetes mellitus

Diabetes is a complex, chronic metabolic condition that progresses gradually and causes the patient to have increased blood glucose levels. The three most prevalent kinds of diabetes are type 1, type 2, and gestational. It results in serious health problems such renal problems, heart attacks, foot ulcers, and visual impairment. One of the primary symptoms is hyperglycemia. Early illness progression detection aids in the reduction of additional consequences. The classification of quantitative data most typically uses machine learning techniques. It makes it easier to pinpoint the important elements that are most responsible for the development of diseases. During the pre-processing stage, the least important properties are removed. The classification or forecasting of the condition Diabetes Mellitus using conventional. The discussion and analysis of supervised machine learning techniques focused on their predictive accuracy. The most popular and effective machine learning model among the several extant algorithms is examined using a variety of tools, datasets, and measures. One of the most effective machine learning models is the Deep learning algorithm of artificial neural networks, which is favored by many scientists and researchers in the field of health.

Fast Computational Approach to the Levenberg-Marquardt Algorithm for Training Feedforward Neural Networks

Abstract This paper presents a parallel approach to the Levenberg-Marquardt algorithm (LM). The use of the Levenberg-Marquardt algorithm to train neural networks is associated with significant computational complexity, and thus computation time. As a result, when the neural network has a big number of weights, the algorithm becomes practically ineffective. This article presents a new parallel approach to the computations in Levenberg-Marquardt neural network learning algorithm. The proposed solution is based on vector instructions to effectively reduce the high computational time of this algorithm. The new approach was tested on several examples involving the problems of classification and function approximation, and next it was compared with a classical computational method. The article presents in detail the idea of parallel neural network computations and shows the obtained acceleration for different problems.

Brain-constrained neural modeling explains fast mapping of words to meaning.

Although teaching animals a few meaningful signs is usually time-consuming, children acquire words easily after only a few exposures, a phenomenon termed "fast-mapping." Meanwhile, most neural network learning algorithms fail to achieve reliable information storage quickly, raising the question of whether a mechanistic explanation of fast-mapping is possible. Here, we applied brain-constrained neural models mimicking fronto-temporal-occipital regions to simulate key features of semantic associative learning. We compared networks (i) with prior encounters with phonological and conceptual knowledge, as claimed by fast-mapping theory, and (ii) without such prior knowledge. Fast-mapping simulations showed word-specific representations to emerge quickly after 1-10 learning events, whereas direct word learning showed word-meaning mappings only after 40-100 events. Furthermore, hub regions appeared to be essential for fast-mapping, and attention facilitated it, but was not strictly necessary. These findings provide a better understanding of the critical mechanisms underlying the human brain's unique ability to acquire new words rapidly.

Predicting money laundering sanctions using machine learning algorithms and artificial neural networks

ABSTRACT This article used machine learning (ML) and artificial neural network (ANN) algorithms to predict the likelihood of a country being sanctioned by the Basel Institute on Governance for not adhering to anti-money laundering (AML) standards. Data for this paper came from the Basel AML Index and the World Bank. The results showed that the logistic regression and support vector machine (SVM) classifiers had the highest performance and balanced accuracy scores in sanction prediction. Additionally, these two algorithms also had the highest precision, specificity, and F1 scores, indicating that they were robust in their predictions of money laundering sanctions. In contrast to the ML classifiers, the ANN model had the highest sensitivity and receiver operating characteristic scores for money laundering sanctions. The strongest predictors of sanctions are financial transparency, political and legal risks, unemployment rate, and money laundering and terrorist financing risks. These findings reinforce the potential practical applications of ML and ANN models in predicting sanctions.

A comparative study of prediction of compressive strength of ultra‐high performance concrete using soft computing technique

AbstractConcrete which is the most commercialized construction material and thus it plays a key role in this era of development and hence its evolution is of utmost importance and therefore the evolution of the quality of concrete to that of its highly evolved type namely, ultra‐high performance concrete (UHPC) is undeniably the boon to this sector. Though, the correlations between the technical characteristics of UHPC and the composition of its mixture are complicated, nonlinear, and complex to characterize using standard statistical techniques. This paper is intended to use both deep neural network and ensemble machine learning algorithms namely gradient boosting, extreme gradient boosting, random forest regressor, extra tree regressor, and voting regressor trained with an 810 UHPC mixture collections with 15 input variables to predict its compressive strength. After adjusting a regression model, the prediction efficiency and generalization ability of the developed models are validated using a number of performance parameters. It was established that all employed models performed better at forecasting result, the extra tree regressor was the most accurate followed by extreme gradient boosting.

Statistical learning algorithms for dendritic neuron model artificial neural network based on sine cosine algorithm

Training of dendritic neuron model artificial neural networks is generally achieved by using nonlinear least square methods. The distribution of random error terms is ignored in training algorithms although error terms are random variables. Maximum likelihood estimators can be obtained for dendritic neuron model artificial neural networks by using some indefinite symmetric probability distributions. In this study, statistical learning algorithms are proposed for dendritic neuron model artificial neural networks. Maximum likelihood estimators for dendritic neuron model artificial neural networks are obtained by using Normal, Cauchy, Logistic, Gumbel and Laplace distributions. The Sine cosine algorithm is used for maximization of the likelihood function under error terms that have Normal, Cauchy, Logistic, Gumbel and Laplace distributions. The proposed learning algorithms are applied to Istanbul Stock Exchange time series data sets. At the end of the analysis of application results, the performance of the proposed method is statistically better than well-known deep and shallow artificial neural networks.

Durian Farm Threats Identification through Convolution Neural Networks and Multimedia Mobile Development

— The difficulties to drive away the durian farm threatens animals such as wild boars, monkeys, foxes, and squirrels during nighttime often experienced by durian farmers. Therefore, the Pro Durian application is proposed that allows farmers to identify durian threats through a camera phone with an alert feature activation when the system detects an animal to drive away those animals. The application implements a deep learning algorithm of Convolutional Neural Network (CNN)-YOLO3in order to receive the best output results in identifying the different datasets of durian farm threats. The classification accuracies reached 80% in detecting the animal’s images. Keywords— Durian Farm, Recognition Image, TensorFlow lite, Android Studio, Convolution Neural Network

Landslide susceptibility evaluation based on active deformation and graph convolutional network algorithm

Disastrous landslides have become a focus of the world’s attention. Landslide susceptibility evaluation (LSE) can predict where landslides may occur and has caught the attention of scientists all over the world. This work establishes integrated criteria of potential landslide recognition and combines the historical landslides and newly-identified potential landslides to improve the accuracy, rationality, and practicability of a LSE map. Moreover, slope units can well reflect the topographic constraint to landslide occurrence and development, and Graph Convolutional Network (GCN) can well portray the topological and feature relation among various slope units. The combination of slope units and GCN is for the first time employed in LSE. This work focuses on Wanzhou District, a famous landslide-serious region in the Three Gorges reservoir area, and employs multisource data to conduct potential landslide recognition and LSE and to reveal the distribution characteristics of high landslide susceptibility. Some new viewpoints are suggested as follows. 1) The established criteria of potential landslide recognition consist of the characteristics of active deformation, stratum and lithology, tectonics, topography, micro-geomorphology, environment, meteorology, earthquakes, and human engineering activity. These criteria can well eliminate 4 types of false alarm regions and is successfully validated by field survey. 2) 34 potential landslides are newly discovered, and the movement of these potential landslides were controlled or induced by the combined action of soft-hard interbedding rock mass, steep topography, frequent tectonic movement, strong fluvial erosion, abundant precipitation, and intensive road and building construction. 3) The GCN algorithm reaches a relatively high accuracy (AUC: 0.941) and outperforms the other representative machine learning algorithms of Convolutional Neural Network (AUC: 0.926), Support Vector Machine (AUC: 0.835), and CART Tree (AUC: 0.762). 4) High landslide susceptibility is caused by the coupled action of weathered rock cavities, soft rock and swelling soil, strong river erosion, abundant rainfall, and intensive human engineering activity.

Dynamic evolution of changbaishan volcanism in Northeast China illuminated by machine learning

Previous studies propose that there is a mantle upwelling that generated the Cenozoic basalts in Changbaishan. However, the dominant source and mechanism of the mantle upwelling remains highly debated. Here we apply machine learning algorithms of Random Forest and Deep Neural Network to train models using global island arc and ocean island basalts data. The trained models predict that Changbaishan basalts are highly influenced by slab-derived fluid. More importantly, the fluid effect decreases with no (87Sr/86Sr)0 and εNd(t) changes between 5 Ma and 1 Ma, then enhances with increasing εNd(t) and decreasing (87Sr/86Sr)0 after 1 Ma. We propose that a gap opened at about 5 Ma and the hot sub-slab oceanic asthenosphere rose through the gap after 1 Ma, generating the basalts enriched in fluid mobile elements and with the addition of depleted mantle component derived from the sub-slab oceanic asthenosphere.

Transport Stream Optimization Based on Neural Network Learning Algorithms

The article discusses a description of the practical application of traffic flow management automation technologies and the use of artificial intelligence to solve traffic optimization problems, traffic monitoring, describes its goals, tasks, indicates the necessary functionality of the traffic monitoring automation system. The defining performance indicators of the use of automated traffic control systems are described. Various methods of calculations used in the control, regulation, analysis and management of traffic flows are demonstrated. The gradations of service levels and levels of convenience in estimating throughput are considered. Analyzed various criteria for the regulation and control of road and transport situations in general.

Evolutionary Augmentation Policy Optimization for Self-Supervised Learning

Self-supervised Learning (SSL) is a machine learning algorithm for pretraining Deep Neural Networks (DNNs) without requiring manually labeled data. The central idea of this learning technique is based on an auxiliary stage aka pretext task in which labeled data are created automatically through data augmentation and exploited for pretraining the DNN. However, the effect of each pretext task is not well studied or compared in the literature. In this paper, we study the contribution of augmentation operators on the performance of self supervised learning algorithms in a constrained settings. We propose an evolutionary search method for optimization of data augmentation pipeline in pretext tasks and measure the impact of augmentation operators in several SOTA SSL algorithms. By encoding different combination of augmentation operators in chromosomes we seek the optimal augmentation policies through an evolutionary optimization mechanism. We further introduce methods for analyzing and explaining the performance of optimized SSL algorithms. Our results indicate that our proposed method can find solutions that outperform the accuracy of classification of SSL algorithms which confirms the influence of augmentation policy choice on the overall performance of SSL algorithms. We also compare optimal SSL solutions found by our evolutionary search mechanism and show the effect of batch size in the pretext task on two visual datasets.

Identification and detection of brain tumour using deep learning-based classification MRI applied using neural network and machine learning algorithm

Identification and detection of brain tumour using deep learning-based classification MRI applied using neural network and machine learning algorithm

Identification and Detection of Brain Tumour Using Deep Learning-based Classification MRI Applied Using Neural Network and Machine Learning Algorithm

Identification and Detection of Brain Tumour Using Deep Learning-based Classification MRI Applied Using Neural Network and Machine Learning Algorithm

An Analysis of Prediction of Students’ Results Using Deep Learning

Many efforts have already been carried out for education mining in the past. Many techniques and models are already developed to predict and identify students’ performance, learning behavior, status, and education level. However, there is no exact solution for the student’s result prediction since it is affected by the level of the student, the field of study, the location of the data collection, different sizes and nature of data, etc. Different research shows that there can be up to 10% difference in the accuracy of results with and without a feature selection process. Thus, the proposed model designs a better model for student result prediction using feature selection and deep learning techniques. The proposed dissertation task compares and analyzes Correlation-based Feature Selection (CFS), Chi-Square ([Formula: see text]), Genetic Algorithm (GA), Information Gain (IG), Maximum Relevance Minimum Redundancy (mRMR), ReliefF, and Recursive Feature Elimination (RFE) feature selection techniques with the Classification and Regression Tree (CART), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Convolution Neural Network (CNN) and Long Short-Term Memory (LSTM) machine learning algorithms. In the proposed model, a feature selection process CFS and prediction using CNN is recommended. The recommended model (CFS–CNN) is tested with a primary dataset collected from bachelor-level students. The recommended model provides improved performance compared to old techniques. The major contribution of the proposed dissertation is to design a better model for the prediction of students’ results using demographic data and past examination results.

Hybrid Deep Learning Based Attack Detection for Imbalanced Data Classification

Internet of Things (IoT) is the most widespread and fastest growing technology today. Due to the increasing of IoT devices connected to the Internet, the IoT is the most technology under security attacks. The IoT devices are not designed with security because they are resource constrained devices. Therefore, having an accurate IoT security system to detect security attacks is challenging. Intrusion Detection Systems (IDSs) using machine learning and deep learning techniques can detect security attacks accurately. This paper develops an IDS architecture based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) deep learning algorithms. We implement our model on the UNSW-NB15 dataset which is a new network intrusion dataset that categorizes the network traffic into normal and attacks traffic. In this work, interpolation data preprocessing is used to compute the missing values. Also, the imbalanced data problem is solved using a synthetic data generation method. Extensive experiments have been implemented to compare the performance results of the proposed model (CNN+LSTM) with a basic model (CNN only) using both balanced and imbalanced dataset. Also, with some state-of-the-art machine learning classifiers (Decision Tree (DT) and Random Forest (RF)) using both balanced and imbalanced dataset. The results proved the impact of the balancing technique. The proposed hybrid model with the balance technique can classify the traffic into normal class and attack class with reasonable accuracy (92.10%) compared with the basic CNN model (89.90%) and the machine learning (DT 88.57% and RF 90.85%) models. Moreover, comparing the proposed model results with the most related works shows that the proposed model gives good results compared with the related works that used the balance techniques.

Research advances in simple and compact optical imaging techniques

Computational imaging enables optical imaging systems to acquire more information with miniaturized setups. Computational imaging can avoid the object-image conjugate limitation of the imaging system, and introduce encoding and decoding processes based on physical optics to achieve more efficient information transmission. It can simultaneously increase the amount of information and reduce the complexity of the system, thereby paving the way for miniaturizing imaging systems. Based on computational imaging, the simple and compact optical imaging techniques are developed, which is also called simple optics. To develop miniaturized optical imaging elements and integrated systems, simple optics utilizes the joint design of optical system and image processing algorithms, thereby realizing high-quality imaging that is comparable to complex optical systems. The imaging systems are of small-size, low-weight, and low-power consumption. With the development of micro-nano manufacturing, the optical elements have evolved from a single lens or a few lenses, to flat/planar optical elements, such as diffractive optical elements and metasurface optical elements. As a result, various lensless and metalens imaging systems have emerged. Owing to the introduction of encoding process and decoding process, an optical imaging model is developed to represent the relationship between the target object and the acquired signal, from which the computational reconstruction is used to restore the image. In the image restoration part, the algorithms are discussed in three categories, i.e. the classic algorithm, the model-based optimization iterative algorithm, and the deep learning (neural network) algorithm. Besides, the end-to-end optimization is highlighted because it introduces a new frame to minimize the complexity of optical system. In this review, the imaging techniques realized by simple optics are also discussed, such as depth imaging, high-resolution and super-resolution imaging, large field of view imaging, and extended depth of field imaging, as well as their important roles in developing consumer electronics, unmanned driving, machine vision, security monitoring, biomedical devices and metaverse. Last but not least, the challenges and future developments are prospected.