Multi-output deep learning for high-frequency prediction of air and surface temperature in Kuwait.

We describe the sentiment analysis experiments that were performed on the Lithuanian Internet comment dataset using traditional machine learning (Naïve Bayes Multinomial—NBM and Support Vector Machine—SVM) and deep learning (Long Short-Term Memory—LSTM and Convolutional Neural Network—CNN) approaches. The traditional machine learning techniques were used with the features based on the lexical, morphological, and character information. The deep learning approaches were applied on the top of two types of word embeddings (Vord2Vec continuous bag-of-words with negative sampling and FastText). Both traditional and deep learning approaches had to solve the positive/negative/neutral sentiment classification task on the balanced and full dataset versions. The best deep learning results (reaching 0.706 of accuracy) were achieved on the full dataset with CNN applied on top of the FastText embeddings, replaced emoticons, and eliminated diacritics. The traditional machine learning approaches demonstrated the best performance (0.735 of accuracy) on the full dataset with the NBM method, replaced emoticons, restored diacritics, and lemma unigrams as features. Although traditional machine learning approaches were superior when compared to the deep learning methods; deep learning demonstrated good results when applied on the small datasets.

Deep learning, a state-of-the-art machine learning approach, has shown outstanding performance over traditional machine learning in identifying intricate structures in complex high-dimensional data, especially in the domain of computer vision. The application of deep learning to early detection and automated classification of Alzheimer's disease (AD) has recently gained considerable attention, as rapid progress in neuroimaging techniques has generated large-scale multimodal neuroimaging data. A systematic review of publications using deep learning approaches and neuroimaging data for diagnostic classification of AD was performed. A PubMed and Google Scholar search was used to identify deep learning papers on AD published between January 2013 and July 2018. These papers were reviewed, evaluated, and classified by algorithm and neuroimaging type, and the findings were summarized. Of 16 studies meeting full inclusion criteria, 4 used a combination of deep learning and traditional machine learning approaches, and 12 used only deep learning approaches. The combination of traditional machine learning for classification and stacked auto-encoder (SAE) for feature selection produced accuracies of up to 98.8% for AD classification and 83.7% for prediction of conversion from mild cognitive impairment (MCI), a prodromal stage of AD, to AD. Deep learning approaches, such as convolutional neural network (CNN) or recurrent neural network (RNN), that use neuroimaging data without pre-processing for feature selection have yielded accuracies of up to 96.0% for AD classification and 84.2% for MCI conversion prediction. The best classification performance was obtained when multimodal neuroimaging and fluid biomarkers were combined. Deep learning approaches continue to improve in performance and appear to hold promise for diagnostic classification of AD using multimodal neuroimaging data. AD research that uses deep learning is still evolving, improving performance by incorporating additional hybrid data types, such as—omics data, increasing transparency with explainable approaches that add knowledge of specific disease-related features and mechanisms.

Offensive language identification (OLI) in user generated text is automatic detection of any profanity, insult, obscenity, racism or vulgarity that degrades an individual or a group. It is helpful for hate speech detection, flame detection and cyber bullying. Due to immense growth of accessibility to social media, OLI helps to avoid abuse and hurts. In this paper, we present deep and traditional machine learning approaches for OLI. In deep learning approach, we have used bi-directional LSTM with different attention mechanisms to build the models and in traditional machine learning, TF-IDF weighting schemes with classifiers namely Multinomial Naive Bayes and Support Vector Machines with Stochastic Gradient Descent optimizer are used for model building. The approaches are evaluated on the OffensEval@SemEval2019 dataset and our team SSN_NLP submitted runs for three tasks of OffensEval shared task. The best runs of SSN_NLP obtained the F1 scores as 0.53, 0.48, 0.3 and the accuracies as 0.63, 0.84 and 0.42 for the tasks A, B and C respectively. Our approaches improved the base line F1 scores by 12%, 26% and 14% for Task A, B and C respectively.

The prediction accuracy of hourly air temperature is generally poor because of random changes, long time series, and the nonlinear relationship between temperature and other meteorological elements, such as air pressure, dew point, and wind speed. In this study, two deep-learning methods—a convolutional neural network (CNN) and long short-term memory (LSTM)—are integrated into a network model (CNN–LSTM) for hourly temperature prediction. The CNN reduces the dimensionality of the time-series data, while LSTM captures the long-term memory of the massive temperature time-series data. Training and validation sets are constructed using 60,133 hourly meteorological data (air temperature, dew point, air pressure, wind direction, wind speed, and cloud amount) obtained from January 2000 to October 2020 at the Yinchuan meteorological station in China. Mean absolute error (MAE), mean absolute percentage error (MAPE), and goodness of fit are used to compare the performances of the CNN, LSTM, and CNN–LSTM models. The results show that MAE, MAPE, RMSE, and PBIAS from the CNN–LSTM model for hourly temperature prediction are 0.82, 0.63, 2.05, and 2.18 in the training stage and 1.02, 0.8, 1.97, and −0.08 in the testing stage. Average goodness of fit from the CNN–LSTM model is 0.7258, higher than the CNN (0.5291), and LSTM (0.5949) models. The hourly temperatures predicted by the CNN–LSTM model are highly consistent with the measured values, especially for long time series of hourly temperature data.

In online courses, discussion forums play a key role in enhancing student interaction with peers and instructors. Due to large enrolment sizes, instructors often struggle to respond to students in a timely manner. To address this problem, both traditional Machine Learning (ML) (e.g., Random Forest) and Deep Learning (DL) approaches have been applied to classify educational forum posts (e.g., those required urgent responses vs. that did not). However, there lacks an in-depth comparison between these two kinds of approaches. To better guide people to select an appropriate model, we aimed at providing a comparative study on the effectiveness of six frequently-used traditional ML and DL models across a total of seven different classification tasks centering around two datasets of educational forum posts. Through extensive evaluation, we showed that (i) the up-to-date DL approaches did not necessarily outperform traditional ML approaches; (ii) the performance gap between the two kinds of approaches can be up to 3.68% (measured in F1 score); (iii) the traditional ML approaches should be equipped with carefully-designed features, especially those of common importance across different classification tasks. Based on the derived findings, we further provided insights to help instructors and educators construct effective classifiers for characterizing educational forum discussions, which, ultimately, would enable them to provide students with timely and personalized learning support.

Deep learning has attracted a great amount of interest in recent years and has become a rapidly emerging field in artificial intelligence. In medical image analysis, deep learning methods have produced promising results comparable to and, in some cases, superior to human experts. Nevertheless, researchers have also noted the limitations and challenges of the deep learning approaches, especially in model selection and interpretability. This paper compares the efficacy of deep learning and traditional machine learning techniques in detecting cognitive impairment (CI) associated with Alzheimer's disease (AD) using brain MRI scans. We base our study on 894 brain MRI scans provided by the open access OASIS platform. In particular, we explore two deep learning approaches: 1) a 3D convolutional neural network (3D-CNN) and 2) a hybrid model with a CNN plus LSTM (CNN-LSTM) architecture. We further examine the performance of five traditional machine learning algorithms based on features extracted from the MRI images using the FreeSurfer software. Our experimental results demonstrate that the deep learning models achieve higher Precision and Recall, while the traditional machine learning methods deliver more stability and better performance in Specificity and overall accuracy. Our findings could serve as a case study to highlight the challenges in adopting deep learning-based approaches.

The self-regulated recognition of human activities from time-series smartphone sensor data is a growing research area in smart and intelligent health care. Deep learning (DL) approaches have exhibited improvements over traditional machine learning (ML) models in various domains, including human activity recognition (HAR). Several issues are involved with traditional ML approaches; these include handcrafted feature extraction, which is a tedious and complex task involving expert domain knowledge, and the use of a separate dimensionality reduction module to overcome overfitting problems and hence provide model generalization. In this article, we propose a DL-based approach for activity recognition with smartphone sensor data, i.e., accelerometer and gyroscope data. Convolutional neural networks (CNNs), autoencoders (AEs), and long short-term memory (LSTM) possess complementary modeling capabilities, as CNNs are good at automatic feature extraction, AEs are used for dimensionality reduction and LSTMs are adept at temporal modeling. In this study, we take advantage of the complementarity of CNNs, AEs, and LSTMs by combining them into a unified architecture. We explore the proposed architecture, namely, “ConvAE-LSTM”, on four different standard public datasets (WISDM, UCI, PAMAP2, and OPPORTUNITY). The experimental results indicate that our novel approach is practical and provides relative smartphone-based HAR solution performance improvements in terms of computational time, accuracy, F1-score, precision, and recall over existing state-of-the-art methods.

In big data applications such as ambient healthcare-supported living, accurate human activity identification may be of tremendous use. In current years, research that is associated with the study of human behavior has been the focus of greater interest among engineers working in the field of computer vision, in particular research that makes use of deep learning. Deep learning (DL) strategies have made significant contributions to the advancement of research in HAR (Human Activity Identification). When it comes to the automated feature extraction, these deep learning algorithms perform far better than traditional machine learning (ML) approaches. In recent years, a great number of DL models have been shown to be state-of-the-art systems that can effectively identify basic and complicated human actions in order to deal with the HAR. In trained neural networks, pruning techniques of neural networks may reduce the number of available parameters, which in turn improves the computing speed of inference, all without compromising correctness of the model. By applying the neural network pruning approach to human activity identification based on deep learning, this study demonstrates its usefulness. PAMAP2 data were analyzed using an architecture called long short-term memory (LSTM), which was used to identify everyday human activities. Before training a LSTM network, the data was preprocessed. PAMAP2 is a publicly available benchmark complex activity dataset that is used to assess the proposed architecture. The results of the experiments indicate that suggested LSTM model achieves the best level of accuracy (97.42%) on the same dataset as other baseline DL models, showing a considerable improvement over other model. The excellent performance of LSTM was also validated by our investigations when comparison to the performances provided by the other base models utilised. The findings of the experiments demonstrate that the approach we have suggested is capable of accurately predicting the social actions of humans.

With the growth of online review sites, we can use the opportunity to find out what other people think. Sentiment analysis is a popular text classification task in the data mining domain. In this research, we develop a text classification machine learning solution that can classify customer reviews into positive or negative class by predicting the overall sentiment of the review text on a document level. For this purpose, we carry out the experiment with two approaches, i.e. traditional machine learning approach and deep learning. In the first approach, we utilize four traditional machine learning algorithms with TF-IDF model using n-gram approach. These classifiers are multinomial naive Bayes, logistic regression, k-nearest neighbour and random forest. Out of these four classifiers, logistic regression achieves the highest accuracy. The second approach is to utilize deep learning methodologies with the word2vec approach, for which we develop a sequential deep learning neural network. The accuracy we achieve with deep learning is much lower than our traditional machine learning approach. After finding out the best performing approach and the classifier, the next step of the work is to build our final model with logistic regression using some advanced machine learning methodologies, i.e. Synthetic Minority Over-sampling for data balancing issues, Shuffle Split cross-validation. The accuracy of the final logistic regression model is approximately 87% which is 3% higher from the initial experimentation. Our finding in this research work is that, in smaller dataset scenarios, traditional machine learning would outperform deep learning models in terms of accuracy and other evaluation metrics. Another finding in this work is, by addressing data balancing issues in the dataset the accuracy of the model can be improved.

Abstract. Predictions of average surface temperature of a sparsely vegetated West-African savannah by both single and dual source models of surface energy partitioning are compared. Within the single source model, the ``excess resistance" to heat transfer away from the canopy (compared to momentum absorption) is characterised by parameter kB-1, where k is the von Kármán constant and B is the Stanton number. Two values of this parameter are used; first kB-1 = 2 (a value often used within surface energy balance models but primarily applicable to permeable vegetation types) and then 12.4 (a value applicable to the savannah in question, which consists more of bluff roughness elements). As expected, the latter parameterisation generates better predictions of surface temperature. To make accurate predictions of surface temperature using a dual source model, then that model’s in-canopy aerodynamic resistance must be increased. Information on this increase is found through direct model intercomparison with the single source model parameterised with kB-1 = 12.4. Keywords: Penman-Monteith equation; Surface temperature; Canopy resistance; Savannah; Dual-Source model

Fused deposition modeling (FDM) is a widely used additive manufacturing (AM) process valued for its versatility in rapid prototyping and its 2.5D part-generation capabilities. However, the quality and mechanical properties of FDM-printed parts are highly sensitive to variations in process parameters, such as material properties, temperature, and printing speed, leading to challenges in maintaining consistent part performance. This study addresses real-time sensor-based anomaly detection in FDM by using time-series data collected from a printing device. We compared two approaches: traditional handcrafted feature extraction methods and deep learning (DL) models, which automatically extract features by transforming signals into images suitable for machine vision algorithms. Specifically, we designed and evaluated bespoke hybrid models that combine convolutional neural networks (CNNs) and long short-term memory (LSTM) units (CNN-LSTM) to monitor the FDM process by utilizing acoustic and vibration signals for anomaly detection. Experimental results show that while traditional machine learning methods, particularly support vector machines (SVMs), achieved slightly higher raw classification metrics, statistical analysis confirmed that these differences were not significant. Moreover, CNN-LSTM models demonstrate notable advantages in terms of computational efficiency, robustness to noise, and future scalability, making them strong candidates for real-time and industrial monitoring applications.

Sensor-based indoor air temperature prediction using deep ensemble machine learning: An Australian urban environment case study

Machine and Deep Learning applied to galaxy morphology - A comparative study

Sentiment analysis (SA) aims to extract users’ opinions automatically from their posts and comments. Almost all prior works have used machine learning algorithms. Recently, SA research has shown promising performance in using the deep learning approach. However, deep learning is greedy and requires large datasets to learn, so it takes more time for data annotation. In this research, we proposed a semiautomatic approach using Naïve Bayes (NB) to annotate a new dataset in order to reduce the human effort and time spent on the annotation process. We created a dataset for the purpose of training and testing the classifier by collecting Saudi dialect tweets. The dataset produced from the semiautomatic model was then used to train and test deep learning classifiers to perform Saudi dialect SA. The accuracy achieved by the NB classifier was 83%. The trained semiautomatic model was used to annotate the new dataset before it was fed into the deep learning classifiers. The three deep learning classifiers tested in this research were convolutional neural network (CNN), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). Support vector machine (SVM) was used as the baseline for comparison. Overall, the performance of the deep learning classifiers exceeded that of SVM. The results showed that CNN reported the highest performance. On one hand, the performance of Bi-LSTM was higher than that of LSTM and SVM, and, on the other hand, the performance of LSTM was higher than that of SVM. The proposed semiautomatic annotation approach is usable and promising to increase speed and save time and effort in the annotation process.

Protocol specification, testing and verification