Machine Learning Techniques For Classification Research Articles

Image classification using deep and classical machine learning models on small datasets: a complete comparative

One of the most important challenges in the Machine and Deep Learning areas today is to build good models using small datasets, because sometimes it is not possible to have large ones. Several techniques have been proposed in the literature to address this challenge. This paper aims at studying the different available Deep Learning techniques and performing a thorough experimentation to analyze which technique or combination thereof improves the performance and effectiveness of the models. A complete comparison with classical Machine Learning techniques was carried out, to contrast the results obtained using both techniques when working with small datasets. Thirteen algorithms were implemented and trained using three different small datasets (MNIST, Fashion MNIST, and CIFAR-10). Each experiment was evaluated using a well-established set of metrics (Accuracy, Precision, Recall, F1, and the Matthews correlation coefficient). The experimentation allowed concluding that it is possible to find a technique or combination of them to mitigate a lack of data, but this depends on the nature of the dataset, theamount of data, and the metrics used to evaluate them.

A Comprehensive Survey on Deep Learning Methods in Human Activity Recognition

Human activity recognition (HAR) remains an essential field of research with increasing real-world applications ranging from healthcare to industrial environments. As the volume of publications in this domain continues to grow, staying abreast of the most pertinent and innovative methodologies can be challenging. This survey provides a comprehensive overview of the state-of-the-art methods employed in HAR, embracing both classical machine learning techniques and their recent advancements. We investigate a plethora of approaches that leverage diverse input modalities including, but not limited to, accelerometer data, video sequences, and audio signals. Recognizing the challenge of navigating the vast and ever-growing HAR literature, we introduce a novel methodology that employs large language models to efficiently filter and pinpoint relevant academic papers. This not only reduces manual effort but also ensures the inclusion of the most influential works. We also provide a taxonomy of the examined literature to enable scholars to have rapid and organized access when studying HAR approaches. Through this survey, we aim to inform researchers and practitioners with a holistic understanding of the current HAR landscape, its evolution, and the promising avenues for future exploration.

An effective ensemble classification framework for intrusion detection system using LSTM autoencoder

In intrusion detection, the curse of dimensionality and the trade-off between maintaining a low false alarm rate and achieving a high detection rate are significant challenges. This research suggests a unique strategy based on dimensionality reduction methods to improve the performance of network intrusion detection systems (NIDS). Compressing high-dimensional network traffic data using a Long Short-Term Memory Autoencoder (LSTMAE) allows the reduced characteristics to be submitted to a classifier to identify anomalies that may indicate an attack. Using standard datasets, including Network Security Laboratory - Knowledge Discovery in Datasets (NSL-KDD), UNSW-NB15, and Canadian Institute for Cyber Security - Intrusion Detection Systems (CICIDS2017), the proposed model is tested with classifiers like Random Forest (RF) and LightGBM (Light Gradient Boosting Machines). It is hoped that by adopting this method, NIDS response times may be improved while costs associated with storing and processing data are minimized. Precision, recall, F-score, accuracy, detection rate (DR), and false alarm rate (FAR) are only a few of the performance measures used to assess the quality of the suggested models. The experimental findings show that the proposed LSTMAE model reduces prediction errors more effectively than classic machine learning techniques such as Random Forest (RF), Gradient Boosting (GB), Support Vector Machines (SVM), Deep Belief Networks (DBN), Deep Neural Networks (DNN), Autoencoder (AE), and Long Short-Term Memory (LSTM). The results also show that the proposed solution outperforms the state-of-the-art methods of detection accuracy and computing complexity using accuracy, precision, recall, F1_Score, detection rate, and FAR.

Performance Comparison of Apache Spark and Hadoop for Machine Learning based iterative GBTR on HIGGS and Covid-19 Datasets

In the realm of distributed computing frameworks, such as Apache Spark and MapReduce Hadoop, the efficacy of these frameworks varies across diverse applications and algorithms contingent upon distinctive evaluation metrics and critical parameters. This research paper diligently scrutinizes the extant body of research that compares these two frameworks concerning said evaluation metrics and parameters. Subsequently, it conducts empirical investigations to authenticate the performance of these frameworks in the context of an iterative Gradient Boosting Tree Regression (GBTR) algorithm. Remarkably, the comparative analyses in previous studies encompass a spectrum of iterative machine learning regression and classification techniques, batch processing, SQL, and Graph processing algorithms. Furthermore, numerous investigations have explored the application of machine learning algorithms encompassing logistic regression, Page Rank, K-Means, KNN, and the HiBench suite. This paper presents the comparison between the two distributed computing platforms on iterative GBTR for classification task on the HIGGS dataset from the physics domain and for the regression task on the Covid-19 dataset from the healthcare domain. The empirical findings corroborate that Apache Spark exhibits superior execution speed in iterative tasks when the available physical memory significantly exceeds the dataset size. Conversely, Hadoop outperforms Spark when dealing with substantial datasets or constrained physical memory resources.

A Comparison Study of Deep Learning Methodologies for Music Emotion Recognition.

Classical machine learning techniques have dominated Music Emotion Recognition. However, improvements have slowed down due to the complex and time-consuming task of handcrafting new emotionally relevant audio features. Deep learning methods have recently gained popularity in the field because of their ability to automatically learn relevant features from spectral representations of songs, eliminating such necessity. Nonetheless, there are limitations, such as the need for large amounts of quality labeled data, a common problem in MER research. To understand the effectiveness of these techniques, a comparison study using various classical machine learning and deep learning methods was conducted. The results showed that using an ensemble of a Dense Neural Network and a Convolutional Neural Network architecture resulted in a state-of-the-art 80.20% F1 score, an improvement of around 5% considering the best baseline results, concluding that future research should take advantage of both paradigms, that is, combining handcrafted features with feature learning.

Toward an improved ensemble of multi-source daily precipitation via joint machine learning classification and regression

Accurate estimation of precipitation at local to global scales can considerably enhance our understanding of climate system dynamics. While numerous precipitation products are available as indispensable tools for investigating precipitation and its associated processes, none can consistently provide the lowest estimation error across environmental conditions. The multiple source precipitation ensemble (MSPE) methods have been considered a vital solution. A new MSPE framework is proposed here, which simultaneously uses machine learning (ML) classification and regression techniques within an automatic workflow (MSPEaml). Six precipitation products and their ensembles based on different MSPE strategies were evaluated at 2365 gauged and 800 randomly selected ungauged sites over China. Results revealed significant precision inconsistencies among the products primarily due to their different data sources and retrieval algorithms; while MSPEaml can effectively reduce the random and classification errors of estimated precipitation according to the Kling-Gupta efficiency and Heidke skill score. The improvements demonstrated the unique features of MSPEaml, particularly the necessity of the joint use of ML classifiers and regressors and assigning spatiotemporal dynamic weights for merging precipitation data. Moreover, MSPEaml can substantially improve its generalizability through a simple binning procedure, making it applicable under more complex conditions. The varying contributions of predictor variables (indicated by Shapely values) in different ML models identified the complexity of the MSPE issue and further the importance of designing proper ML models according to specific targets. The proposed MSPE framework is expected to be a suitable solution for assembling multiple precipitation data sources with different time periods and scales.

Detecting cyberbullying using deep learning techniques: a pre-trained glove and focal loss technique.

This study investigates the effectiveness of various deep learning and classical machine learning techniques in identifying instances of cyberbullying. The study compares the performance of five classical machine learning algorithms and three deep learning models. The data undergoes pre-processing, including text cleaning, tokenization, stemming, and stop word removal. The experiment uses accuracy, precision, recall, and F1 score metrics to evaluate the performance of the algorithms on the dataset. The results show that the proposed technique achieves high accuracy, precision, and F1 score values, with the Focal Loss algorithm achieving the highest accuracy of 99% and the highest precision of 86.72%. However, the recall values were relatively low for most algorithms, indicating that they struggled to identify all relevant data. Additionally, the study proposes a technique using a convolutional neural network with a bidirectional long short-term memory layer, trained on a pre-processed dataset of tweets using GloVe word embeddings and the focal loss function. The model achieved high accuracy, precision, and F1 score values, with the GRU algorithm achieving the highest accuracy of 97.0% and the NB algorithm achieving the highest precision of 96.6%.

Analysis of selected deep features with CNN-SVM-based for bread wheat seed classification

The main ingredient of flour is processed wheat. Wheat is an agricultural product that is harvested once a year. It may be necessary to choose the variety of wheat for growing wheat and efficient harvesting. The variety of wheat is important for its economic value, taste, and crop yield. Although there are many varieties of wheat, they are very similar in colour, size, and shape, and it requires expertise to distinguish them by eye. This is very time consuming and can lead to human error. Using computer vision and artificial intelligence, such problems can be solved more quickly and objectively. In this study, an attempt was made to classify five bread wheat varieties belonging to different cultivars using Convolutional Neural Network (CNN) models. Three approaches have been proposed for classification. First, pre-trained CNN models (ResNet18, ResNet50, and ResNet101) were trained for bread wheat cultivars. Second, the features extracted from the fc1000 layer of the pre-trained CNN models ResNet18, ResNet50, and ResNet101 were classified using a support vector machine (SVM) classifier with different kernel features from machine learning techniques for classification with different variants. Finally, SVM methods were used in the second stage to classify the features obtained from the fc1000 layer of the pre-trained CNN models with an optimal set of features that can represent all features using the minimum redundancy maximum relevance (mRMR) feature selection algorithm.The accuracies obtained in the first, second, and last phases are as follows. In the first phase, the most successful method in classifying wheat grains was the ResNet18 model with 97.57%. In the second phase, the ResNet18 + ResNet50 + ResNet101 + Quadratic SVM model was the most successful model in classification using the features obtained from the ResNet CNN models with 94.08%.The accuracy for classification with the 1000 most effective features selected by the feature selection algorithm was 94.51%. Although the classification with features is slightly lower than deep learning, the classification time is much shorter and is 93%. This result confirms the great effectiveness of CNN models for wheat grain classification.

From programming-to-modeling-to-prompts smart ubiquitous applications

Since its introduction by Mark Weiser, ubiquitous computing has received increased interest in the dawn of technological advancement. Supported by wireless technology advancement, embedded systems, miniaturization, and the integration of various intelligent and communicative devise, context-aware ubiquitous applications actively and intelligently use rich contextual information to assist their users. However, their designs are subject to continuous changes imposed by external factors. Nowadays, software engineering, particularly in the fields of Model-Driven Engineering, displays a strong tendency towards developing applications for pervasive computing. This trend is also fueled by the rise of generative artificial intelligence, paving the way for a new generation of no-code development tools and models specifically trained on open-source code repositories to generate applications from their descriptions. The specificities of our approach lies in starting with a graphical model expressed using a domain-specific language (DSL) composed of symbols and formal notations. This allows for graphically instantiating and editing applications, guiding and assisting experts from various engineering fields in defining ubiquitous applications that are eventually transformed into peculiar models. We believe that creating intelligent models is the best way to promote software development efficiency. We have used and evaluated recurrent neural networks, leveraging the recurrence of processing the same contextual information collected within this model, and enabling iterative adaptation to future evolutions in ubiquitous systems. We propose a prototype instantiated by our meta-model which tracks the movements of individuals who were positive for COVID-19 and confirmed to be contagious. Different deep learning models and classical machine learning techniques are considered and compared for the task of detection/classification of COVID-19. Results obtained from all techniques were evaluated with confusion matrices, accuracy, precision, recall and F1-score. In summary, most of the results are very impressive. Our deep learning approach used a RNN architecture produced up to 92.1% accuracy. With the recent development of OpenAI Codex, optimized for programming languages, we provided the same requirements to the Codex model and asked it to generate the source code for the COVID-19 application, comparing it with the application generated by our workshop.

NuInsSeg: A fully annotated dataset for nuclei instance segmentation in H&E-stained histological images

In computational pathology, automatic nuclei instance segmentation plays an essential role in whole slide image analysis. While many computerized approaches have been proposed for this task, supervised deep learning (DL) methods have shown superior segmentation performances compared to classical machine learning and image processing techniques. However, these models need fully annotated datasets for training which is challenging to acquire, especially in the medical domain. In this work, we release one of the biggest fully manually annotated datasets of nuclei in Hematoxylin and Eosin (H&E)-stained histological images, called NuInsSeg. This dataset contains 665 image patches with more than 30,000 manually segmented nuclei from 31 human and mouse organs. Moreover, for the first time, we provide additional ambiguous area masks for the entire dataset. These vague areas represent the parts of the images where precise and deterministic manual annotations are impossible, even for human experts. The dataset and detailed step-by-step instructions to generate related segmentation masks are publicly available on the respective repositories.

Improving multilayer perceptron neural network using two enhanced moth-flame optimizers to forecast iron ore prices

Abstract The quality of the output produced by the multi-layer perceptron neural network depends on the careful selection of its weights and biases. The gradient descent technique is commonly used for choosing MLP’s optimal configuration, but it can suffer from being stuck in local optima and slow convergence toward promising regions in the search space. In this article, we propose two new optimization algorithms based on the moth-flame optimization algorithm (MFO), which mimics moths’ special navigation methods at night. We use these algorithms to enhance the performance of the training process of the MLP neural network. To demonstrate the effectiveness of our approach, we apply it to the problem of predicting iron ore prices, which plays an important role in the continuous development of the steel industry. We use a large number of features to predict the iron ore price, and we select a promising set of features using two feature reduction methods: Pearson’s correlation and a newly proposed categorized correlation. Surprisingly, new features not mentioned in the literature are discovered, and some are discarded. The time series dataset used has been extracted from several sources and pre-processed to fit the proposed model. We compare our two proposed MFO algorithms, the roulette wheel moth-flame optimization algorithm and the global best moth-flame optimization algorithm, against four swarm intelligence algorithms and five classical machine learning techniques when predicting the iron ore price. The results acquired indicate the superior performance of the suggested algorithms concerning prediction accuracy, root-mean-square error, mean-square error, average absolute relative deviation, and mean absolute error. Overall, our work presents a promising approach for improving the performance of MLP neural networks, and it demonstrates its effectiveness in the challenging problem of predicting iron ore prices.

Machine learning classifiers for fall detection leveraging LoRa communication network

Today, health monitoring relies heavily on technological advancements. This study proposes a low-power wide-area network (LPWAN) based, multinodal health monitoring system to monitor vital physiological data. The suggested system consists of two nodes, an indoor node, and an outdoor node, and the nodes communicate via long range (LoRa) transceivers. Outdoor nodes use an MPU6050 module, heart rate, oxygen pulse, temperature, and skin resistance sensors and transmit sensed values to the indoor node. We transferred the data received by the master node to the cloud using the Adafruit cloud service. The system can operate with a coverage of 4.5 km, where the optimal distance between outdoor sensor nodes and the indoor master node is 4 km. To further predict fall detection, various machine learning classification techniques have been applied. Upon comparing various classifier techniques, the decision tree method achieved an accuracy of 0.99864 with a training and testing ratio of 70:30. By developing accurate prediction models, we can identify high-risk individuals and implement preventative measures to reduce the likelihood of a fall occurring. Remote monitoring of the health and physical status of elderly people has proven to be the most beneficial application of this technology.

Selective Partitioned Regression for Accurate Kidney Health Monitoring

The number of people diagnosed with advanced stages of kidney disease have been rising every year. Early detection and constant monitoring are the only minimally invasive means to prevent severe kidney damage or kidney failure. We propose a cost-effective machine learning-based testing system that can facilitate inexpensive yet accurate kidney health checks. Our proposed framework, which was developed into an iPhone application, uses a camera-based bio-sensor and state-of-the-art classical machine learning and deep learning techniques for predicting the concentration of creatinine in the sample, based on colorimetric change in the test strip. The predicted creatinine concentration is then used to classify the severity of the kidney disease as healthy, intermediate, or critical. In this article, we focus on the effectiveness of machine learning models to translate the colorimetric reaction to kidney health prediction. In this setting, we thoroughly evaluated the effectiveness of our novel proposed models against state-of-the-art classical machine learning and deep learning approaches. Additionally, we executed a number of ablation studies to measure the performance of our model when trained using different meta-parameter choices. Our evaluation results indicate that our selective partitioned regression (SPR) model, using histogram of colors-based features and a histogram gradient boosted trees underlying estimator, exhibits much better overall prediction performance compared to state-of-the-art methods. Our initial study indicates that SPR can be an effective tool for detecting the severity of kidney disease using inexpensive lateral flow assay test strips and a smart phone-based application. Additional work is needed to verify the performance of the model in various settings.

Carpal Tunnel Syndrome Automated Diagnosis: A Motor vs. Sensory Nerve Conduction-Based Approach.

The objective of this study was to evaluate the effectiveness of machine learning classification techniques applied to nerve conduction studies (NCS) of motor and sensory signals for the automatic diagnosis of carpal tunnel syndrome (CTS). Two methodologies were tested. In the first methodology, motor signals recorded from the patients' median nerve were transformed into time-frequency spectrograms using the short-time Fourier transform (STFT). These spectrograms were then used as input to a deep two-dimensional convolutional neural network (CONV2D) for classification into two categories: patients and controls. In the second methodology, sensory signals from the patients' median and ulnar nerves were subjected to multilevel wavelet decomposition (MWD), and statistical and non-statistical features were extracted from the decomposed signals. These features were utilized to train and test classifiers. The classification target was set to three categories: normal subjects (controls), patients with mild CTS, and patients with moderate to severe CTS based on conventional electrodiagnosis results. The results of the classification analysis demonstrated that both methodologies surpassed previous attempts at automatic CTS diagnosis. The classification models utilizing the motor signals transformed into time-frequency spectrograms exhibited excellent performance, with average accuracy of 94%. Similarly, the classifiers based on the sensory signals and the extracted features from multilevel wavelet decomposition showed significant accuracy in distinguishing between controls, patients with mild CTS, and patients with moderate to severe CTS, with accuracy of 97.1%. The findings highlight the efficacy of incorporating machine learning algorithms into the diagnostic processes of NCS, providing a valuable tool for clinicians in the diagnosis and management of neuropathies such as CTS.

Applying machine learning classification techniques for disease diagnosis from medical imaging data using Transformer based Attention Guided CNN (TAGCNN)

Applying machine learning classification techniques for disease diagnosis from medical imaging data using Transformer based Attention Guided CNN (TAGCNN)

The five-factor model analysed by machine learning classification techniques

ABSTRACT This study compares the classification performance of traditional factor models and ML to explore the assumption of linearity in risk premiums. We apply ML techniques to the variables used in the Fama – French five-factor (FF5) model to classify the top 30% and bottom 30% of stocks based on whether their prices are expected to increase (winners) or decrease (losers), respectively, over a specified time period. The analysis showed that the ML model’s classification Accuracy outperforms the FF5 classification, which suggests that the higher returns obtained using the FF5 model can be attributed to its effectiveness in classifying stocks with exceptionally high or low returns, rather than the comprehensive use of its variables for overall classification purposes.

Maximizing efficiency in solar ammonia–water absorption refrigeration cycles: Exergy analysis, concentration impact, and advanced optimization with GBRT machine learning and FHO optimizer

A detailed analysis of energy and exergy is conducted on a single-effect solar ammonia–water (NH3–H2O) absorption refrigeration cycle (ARC) using TRNSYS and EES software. Considering the physical and chemical exergies, the exergy destruction rate (ĖD) in each component of the system is calculated, highlighting its contribution to the overall ĖD. The study explores the effects of varying refrigerant mass flow rate (ṁᵣ) and ammonia concentration in strong and weak solutions (Xs and Xw) on key performance parameters, including coefficient of performance (COP), exergy efficiency (ĖD), and overall ĖD across a range of generator temperatures (Tg). In this study, a gradient boosting regression tree (GBRT) is employed as a supervised machine-learning technique for classification and regression problems, utilizing boosting to enhance conventional decision tree predictions. The Fire Hawk Optimizer (FHO) approach is also utilized to optimize performance parameters, maximizing COP and ηηEwhile minimizing Tg and ĖD. The GBRT models are developed using available experimental and simulation data, revealing relationships between variables (ṁᵣ, Xs, Xw, and Tg) and outcomes (COP, ĖD, and overall ĖD). The results revealed that the generator exhibits considerable ĖD regardless of operating conditions, underscoring its pivotal role in the ARC. It emerges as the primary ĖD contributor (50 %), followed by the evaporator (17 %) and the absorber (15 %). However, ĖD associated with the recooler, pump, and expansion valves is negligible in comparison. Optimization results reveal that, when minimizing Tg and ĖD, the highest COP and ĖD at Tg of 373.15 K reach 0.8081 and 0.46, respectively.

Kinematic Analysis of Human Gait in Healthy Young Adults Using IMU Sensors: Exploring Relevant Machine Learning Features for Clinical Applications.

Gait is the manner or style of walking, involving motor control and coordination to adapt to the surrounding environment. Knowing the kinesthetic markers of normal gait is essential for the diagnosis of certain pathologies or the generation of intelligent ortho-prostheses for the treatment or prevention of gait disorders. The aim of the present study was to identify the key features of normal human gait using inertial unit (IMU) recordings in a walking test. Gait analysis was conducted on 32 healthy participants (age range 19-29 years) at speeds of 2 km/h and 4 km/h using a treadmill. Dynamic data were obtained using a microcontroller (Arduino Nano 33 BLE Sense Rev2) with IMU sensors (BMI270). The collected data were processed and analyzed using a custom script (MATLAB 2022b), including the labeling of the four relevant gait phases and events (Stance, Toe-Off, Swing, and Heel Strike), computation of statistical features (64 features), and application of machine learning techniques for classification (8 classifiers). Spider plot analysis revealed significant differences in the four events created by the most relevant statistical features. Among the different classifiers tested, the Support Vector Machine (SVM) model using a Cubic kernel achieved an accuracy rate of 92.4% when differentiating between gait events using the computed statistical features. This study identifies the optimal features of acceleration and gyroscope data during normal gait. The findings suggest potential applications for injury prevention and performance optimization in individuals engaged in activities involving normal gait. The creation of spider plots is proposed to obtain a personalised fingerprint of each patient's gait fingerprint that could be used as a diagnostic tool. A deviation from a normal gait pattern can be used to identify human gait disorders. Moving forward, this information has potential for use in clinical applications in the diagnosis of gait-related disorders and developing novel orthoses and prosthetics to prevent falls and ankle sprains.

Predicting students’ academic progress and related attributes in first-year medical students: an analysis with artificial neural networks and Naïve Bayes

BackgroundDropout and poor academic performance are persistent problems in medical schools in emerging economies. Identifying at-risk students early and knowing the factors that contribute to their success would be useful for designing educational interventions. Educational Data Mining (EDM) methods can identify students at risk of poor academic progress and dropping out. The main goal of this study was to use machine learning models, Artificial Neural Networks (ANN) and Naïve Bayes (NB), to identify first year medical students that succeed academically, using sociodemographic data and academic history.MethodsData from seven cohorts (2011 to 2017) of admitted medical students to the National Autonomous University of Mexico (UNAM) Faculty of Medicine in Mexico City were analysed. Data from 7,976 students (2011 to 2017 cohorts) of the program were included. Information from admission diagnostic exam results, academic history, sociodemographic characteristics and family environment was used. The main dataset included 48 variables. The study followed the general knowledge discovery process: pre-processing, data analysis, and validation. Artificial Neural Networks (ANN) and Naïve Bayes (NB) models were used for data mining analysis.ResultsANNs models had slightly better performance in accuracy, sensitivity, and specificity. Both models had better sensitivity when classifying regular students and better specificity when classifying irregular students. Of the 25 variables with highest predictive value in the Naïve Bayes model, percentage of correct answers in the diagnostic exam was the best variable.ConclusionsBoth ANN and Naïve Bayes methods can be useful for predicting medical students’ academic achievement in an undergraduate program, based on information of their prior knowledge and socio-demographic factors. Although ANN offered slightly superior results, Naïve Bayes made it possible to obtain an in-depth analysis of how the different variables influenced the model. The use of educational data mining techniques and machine learning classification techniques have potential in medical education.

Machine Learning Classification Workflow and Datasets for Ionospheric VLF Data Exclusion

Machine learning (ML) methods are commonly applied in the fields of extraterrestrial physics, space science, and plasma physics. In a prior publication, an ML classification technique, the Random Forest (RF) algorithm, was utilized to automatically identify and categorize erroneous signals, including instrument errors, noisy signals, outlier data points, and the impact of solar flares (SFs) on the ionosphere. This data communication includes the pre-processed dataset used in the aforementioned research, along with a workflow that utilizes the PyCaret library and a post-processing workflow. The code and data serve educational purposes in the interdisciplinary field of ML and ionospheric physics science, as well as being useful to other researchers for diverse objectives.