Traditional Learning Research Articles

Early and Realistic Exploitability Prediction of Just-Disclosed Software Vulnerabilities: How Reliable Can It Be?

With the rate of discovered and disclosed vulnerabilities escalating, researchers have been experimenting with machine learning to predict whether a vulnerability will be exploited. Existing solutions leverage information unavailable when a CVE is created, making them unsuitable just after the disclosure. This paper experiments with early exploitability prediction models driven exclusively by the initial CVE record, i.e., the original description and the linked online discussions. Leveraging NVD and Exploit Database, we evaluate 72 prediction models trained using six traditional machine learning classifiers, four feature representation schemas, and three data balancing algorithms. We also experiment with five pre-trained large language models (LLMs). The models leverage seven different corpora made by combining three data sources, i.e., CVE description, Security Focus , and BugTraq . The models are evaluated in a realistic , time-aware fashion by removing the training and test instances that cannot be labeled “neutral” with sufficient confidence. The validation reveals that CVE descriptions and Security Focus discussions are the best data to train on. Pre-trained LLMs do not show the expected performance, requiring further pre-training in the security domain. We distill new research directions, identify possible room for improvement, and envision automated systems assisting security experts in assessing the exploitability.

Robust Structure-Aware Graph-based Semi-Supervised Learning: Batch and Recursive Processing

Graph-based semi-supervised learning plays an important role in large scale image classification tasks. However, the problem becomes very challenging in the presence of noisy labels and outliers. Moreover, traditional robust semi-supervised learning solutions suffers from prohibitive computational burdens thus cannot be computed for streaming data. Motivated by that, we present a novel unified framework robust structure-aware semi-supervised learning called Unified RSSL (URSSL) for batch processing and recursive processing robust to both outliers and noisy labels. Particularly, URSSL applies joint semi-supervised dimensionality reduction with robust estimators and network sparse regularization simultaneously on the graph Laplacian matrix iteratively to preserve the intrinsic graph structure and ensure robustness to the compound noise. First, in order to relieve the influence from outliers, a novel semi-supervised robust dimensionality reduction is applied relying on robust estimators to suppress outliers. Meanwhile, to tackle noisy labels, the denoised graph similarity information is encoded into the network regularization. Moreover, by identifying strong relevance of dimensionality reduction and network regularization in the context of robust semi-supervised learning (RSSL), a two-step alternative optimization is derived to compute optimal solutions with guaranteed convergence. We further derive our framework to adapt to large scale semi-supervised learning particularly suitable for large scale image classification and demonstrate the model robustness under different adversarial attacks. For recursive processing, we rely on reparameterization to transform the formulation to unlock the challenging problem of robust streaming-based semi-supervised learning. Last but not least, we extend our solution into distributed solutions to resolve the challenging issue of distributed robust semi-supervised learning when images are captured by multiple cameras at different locations. Extensive experimental results demonstrate the promising performance of this framework when applied to multiple benchmark datasets with respect to state-of-the-art approaches for important applications in the areas of image classification and spam data analysis.

Machine learning in predicting pathological complete response to neoadjuvant chemoradiotherapy in rectal cancer using MRI: a systematic review and meta-analysis.

To evaluate the performance of machine learning models in predicting pathological complete response (pCR) to neoadjuvant chemoradiotherapy (nCRT) in patients with rectal cancer using magnetic resonance imaging. We searched PubMed, Embase, Cochrane Library, and Web of Science for studies published before March 2024. The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) was used to assess the methodological quality of the included studies, random-effects models were used to calculate sensitivity and specificity, I2 values were used for heterogeneity measurements, and subgroup analyses were carried out to detect potential sources of heterogeneity. A total of 1699 patients from 24 studies were included. For machine learning models in predicting pCR to nCRT, the meta-analysis calculated a pooled area under the curve (AUC) of 0.91 (95% CI, 0.88-0.93), pooled sensitivity of 0.83 (95% CI, 0.74-0.89), and pooled specificity of 0.86 (95% CI, 0.80-0.91). We investigated 6 studies that mainly contributed to heterogeneity. After performing meta-analysis again excluding these 6 studies, the heterogeneity was significantly reduced. In subgroup analysis, the pooled AUC of the deep-learning model was 0.93 and 0.89 for the traditional statistical model; the pooled AUC of studies that used diffusion-weighted imaging (DWI) was 0.90 and 0.92 in studies that did not use DWI; the pooled AUC of studies conducted in China was 0.93, and was 0.83 in studies conducted in other countries. This systematic study showed that machine learning has promising potential in predicting pCR to nCRT in patients with locally advanced rectal cancer. Compared to traditional machine learning models, although deep-learning-based studies are less predominant and more heterogeneous, they are able to obtain higher AUC. Compared to traditional machine learning models, deep-learning-based studies are able to obtain higher AUC, although they are less predominant and more heterogeneous. Together with clinical information, machine learning-based models may bring us closer towards precision medicine.

Fracture prediction in a Swiss cohort.

Fracture prediction is essential in managing patients with osteoporosis, and is an integral component of many fracture prevention guidelines. We aimed to identify the most relevant clinical fracture risk factors in contemporary populations by training and validating short- and long-term fracture risk prediction models in two cohorts. We used traditional and machine learning survival models to predict risks of vertebral, hip and any fractures on the basis of clinical risk factors, T-scores and treatment history among participants in a nationwide Swiss osteoporosis registry (N = 5944 postmenopausal women, median follow-up of 4.1years between January 2015 and October 2022; a total of 1190 fractures during follow-up). The independent validation cohort comprised 5474 postmenopausal women from the UK Biobank with 290 incident fractures during follow-up. Uno's C-index and the time-dependent area under the receiver operating characteristics curve were calculated to evaluate the performance of different machine learning models (Random survival forests and eXtreme Gradient Boosting). In the independent validation set, the C-index was 0.74 [0.58, 0.86] for vertebral fractures, 0.83 [0.7, 0.94] for hip fractures and 0.63 [0.58, 0.69] for any fractures at year 2, and these values further increased for longer estimations of up to 7years. In comparison, the 10-year fracture probability calculated with FRAX® Switzerland was 0.60 [0.55, 0.64] for major osteoporotic fractures and 0.62 [0.49, 0.74] for hip fractures. The most important variables identified with Shapley additive explanations (SHAP) values were age, T-scores and prior fractures, while number of falls was an important predictor of hip fractures. Performances of both traditional and machine learning models showed similar C-indices. We conclude that fracture risk can be improved by including the lumbar spine T-score, trabecular bone score, numbers of falls and recent fractures, and treatment information has a significant impact on fracture prediction.

Predicting treatment response in multicenter non-small cell lung cancer patients based on federated learning

BackgroundMulticenter non-small cell lung cancer (NSCLC) patient data is information-rich. However, its direct integration becomes exceptionally challenging due to constraints involving different healthcare organizations and regulations. Traditional centralized machine learning methods require centralizing these sensitive medical data for training, posing risks of patient privacy leakage and data security issues. In this context, federated learning (FL) has attracted much attention as a distributed machine learning framework. It effectively addresses this contradiction by preserving data locally, conducting local model training, and aggregating model parameters. This approach enables the utilization of multicenter data with maximum benefit while ensuring privacy safeguards. Based on pre-radiotherapy planning target volume images of NSCLC patients, a multicenter treatment response prediction model is designed by FL for predicting the probability of remission of NSCLC patients. This approach ensures medical data privacy, high prediction accuracy and computing efficiency, offering valuable insights for clinical decision-making.MethodsWe retrospectively collected CT images from 245 NSCLC patients undergoing chemotherapy and radiotherapy (CRT) in four Chinese hospitals. In a simulation environment, we compared the performance of the centralized deep learning (DL) model with that of the FL model using data from two sites. Additionally, due to the unavailability of data from one hospital, we established a real-world FL model using data from three sites. Assessments were conducted using measures such as accuracy, receiver operating characteristic curve, and confusion matrices.ResultsThe model’s prediction performance obtained using FL methods outperforms that of traditional centralized learning methods. In the comparative experiment, the DL model achieves an AUC of 0.718/0.695, while the FL model demonstrates an AUC of 0.725/0.689, with real-world FL model achieving an AUC of 0.698/0.672.ConclusionsWe demonstrate that the performance of a FL predictive model, developed by combining convolutional neural networks (CNNs) with data from multiple medical centers, is comparable to that of a traditional DL model obtained through centralized training. It can efficiently predict CRT treatment response in NSCLC patients while preserving privacy.

Enhanced analysis of tabular data through Multi-representation DeepInsight

Tabular data analysis is a critical task in various domains, enabling us to uncover valuable insights from structured datasets. While traditional machine learning methods can be used for feature engineering and dimensionality reduction, they often struggle to capture the intricate relationships and dependencies within real-world datasets. In this paper, we present Multi-representation DeepInsight (MRep-DeepInsight), a novel extension of the DeepInsight method designed to enhance the analysis of tabular data. By generating multiple representations of samples using diverse feature extraction techniques, our approach is able to capture a broader range of features and reveal deeper insights. We demonstrate the effectiveness of MRep-DeepInsight on single-cell datasets, Alzheimer's data, and artificial data, showcasing an improved accuracy over the original DeepInsight approach and machine learning methods like random forest, XGBoost, LightGBM, FT-Transformer and L2-regularized logistic regression. Our results highlight the value of incorporating multiple representations for robust and accurate tabular data analysis. By leveraging the power of diverse representations, MRep-DeepInsight offers a promising new avenue for advancing decision-making and scientific discovery across a wide range of fields.

Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition.

Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.

Energy-Efficient Edge and Cloud Image Classification with Multi-Reservoir Echo State Network and Data Processing Units.

In an era dominated by Internet of Things (IoT) devices, software-as-a-service (SaaS) platforms, and rapid advances in cloud and edge computing, the demand for efficient and lightweight models suitable for resource-constrained devices such as data processing units (DPUs) has surged. Traditional deep learning models, such as convolutional neural networks (CNNs), pose significant computational and memory challenges, limiting their use in resource-constrained environments. Echo State Networks (ESNs), based on reservoir computing principles, offer a promising alternative with reduced computational complexity and shorter training times. This study explores the applicability of ESN-based architectures in image classification and weather forecasting tasks, using benchmarks such as the MNIST, FashionMnist, and CloudCast datasets. Through comprehensive evaluations, the Multi-Reservoir ESN (MRESN) architecture emerges as a standout performer, demonstrating its potential for deployment on DPUs or home stations. In exploiting the dynamic adaptability of MRESN to changing input signals, such as weather forecasts, continuous on-device training becomes feasible, eliminating the need for static pre-trained models. Our results highlight the importance of lightweight models such as MRESN in cloud and edge computing applications where efficiency and sustainability are paramount. This study contributes to the advancement of efficient computing practices by providing novel insights into the performance and versatility of MRESN architectures. By facilitating the adoption of lightweight models in resource-constrained environments, our research provides a viable alternative for improved efficiency and scalability in modern computing paradigms.

Applications of machine learning in urodynamics: A narrative review.

Machine learning algorithms as a research tool, including traditional machine learning and deep learning, are increasingly applied to the field of urodynamics. However, no studies have evaluated how to select appropriate algorithm models for different urodynamic research tasks. We undertook a narrative review evaluating how the published literature reports the applications of machine learning in urodynamics. We searched PubMed up to December 2023, limited to the English language. We selected the following search terms: artificial intelligence, machine learning, deep learning, urodynamics, and lower urinary tract symptoms. We identified three domains for assessment in advance of commencing the review. These were the applications of urodynamic studies examination, applications of diagnoses of dysfunction related to urodynamics, and applications of prognosis prediction. The machine learning algorithm applied in the field of urodynamics can be mainly divided into three aspects, which are urodynamic examination, diagnosis of urinary tract dysfunction and prediction of the efficacy of various treatment methods. Most of these studies were single-center retrospective studies, lacking external validation, requiring further validation of model generalization ability, and insufficient sample size. The relevant research in this field is still in the preliminary exploration stage; there are few high-quality multi-center clinical studies, and the performance of various models still needs to be further optimized, and there is still a distance from clinical application. At present, there is no research to summarize and analyze the machine learning algorithms applied in the field of urodynamics. The purpose of this review is to summarize and classify the machine learning algorithms applied in this field and to guide researchers to select the appropriate algorithm model for different task requirements to achieve the best results.

A Survey of Machine Learning in Edge Computing: Techniques, Frameworks, Applications, Issues, and Research Directions

Internet of Things (IoT) devices often operate with limited resources while interacting with users and their environment, generating a wealth of data. Machine learning models interpret such sensor data, enabling accurate predictions and informed decisions. However, the sheer volume of data from billions of devices can overwhelm networks, making traditional cloud data processing inefficient for IoT applications. This paper presents a comprehensive survey of recent advances in models, architectures, hardware, and design requirements for deploying machine learning on low-resource devices at the edge and in cloud networks. Prominent IoT devices tailored to integrate edge intelligence include Raspberry Pi, NVIDIA’s Jetson, Arduino Nano 33 BLE Sense, STM32 Microcontrollers, SparkFun Edge, Google Coral Dev Board, and Beaglebone AI. These devices are boosted with custom AI frameworks, such as TensorFlow Lite, OpenEI, Core ML, Caffe2, and MXNet, to empower ML and DL tasks (e.g., object detection and gesture recognition). Both traditional machine learning (e.g., random forest, logistic regression) and deep learning methods (e.g., ResNet-50, YOLOv4, LSTM) are deployed on devices, distributed edge, and distributed cloud computing. Moreover, we analyzed 1000 recent publications on “ML in IoT” from IEEE Xplore using support vector machine, random forest, and decision tree classifiers to identify emerging topics and application domains. Hot topics included big data, cloud, edge, multimedia, security, privacy, QoS, and activity recognition, while critical domains included industry, healthcare, agriculture, transportation, smart homes and cities, and assisted living. The major challenges hindering the implementation of edge machine learning include encrypting sensitive user data for security and privacy on edge devices, efficiently managing resources of edge nodes through distributed learning architectures, and balancing the energy limitations of edge devices and the energy demands of machine learning.

A novel transformer-based neural network under model interpretability for improving wheat yield estimation using remotely sensed multi-variables

Timely and accurate yield prediction before wheat harvest is of great significance for food policy formulation and national economic development. Deep learning method gains importance on crop yield estimation and growth monitoring with the rapid development of deep learning models combined with remotely sensed data. However, the problems of nonlinear parameter optimization and long-term dependence in time series data have restricted the improvement of yield estimation accuracy. Transformer model completely abandons the traditional deep learning architecture and is gradually applied to time series tasks due to the advantage in modeling long-term dependence. This study introduced a novel transformer-based deep learning framework to estimate winter wheat yield in the Guanzhong Plain utilizing remotely sensed multi-variables, namely leaf area index (LAI), fraction of photosynthetically active radiation (FPAR) and vegetation temperature condition index (VTCI). The proposed model, called SSA-LSTM-transformer (SLTF), was developed for dealing with above problems under the automatic optimization capability of the sparrow search algorithm (SSA) and the long-term memory advantage of the long short-term memory (LSTM) structure. The findings demonstrated that the SLTF model achieved better estimation accuracy at the county level (R2 = 0.72, RMSE = 488.68 kg/ha) compared with the single transformer model (R2 = 0.61, RMSE = 573.99 kg/ha). The analysis on SLTF model’s performance at sampling sites over different disasters suggested that the model can effectively learn the effects of diseases, lodge and pests on crop yield estimation, which had good generalization ability at all sampling sites (R2 = 0.45, RMSE = 738.63 kg/ha). The Shapley Additive exPlanations (SHAP) approach was employed to evaluate the relative importance of remotely sensed multi-variables to yield and to understand how each input variable influences the estimated yield for global interpretability and local interpretability of the SLTF model. It was found that FPAR played the largest roles in yield estimation with the highest importance value, and FPAR and LAI from late April to late May and VTCI from late March to mid-April were regarded as important features for the estimated yield with high importance value. In conclusion, our findings indicated the potential of SLTF model for winter wheat yield estimation, which contributes to promoting further application of remotely sensed technology for agricultural production.

Bangla song genre recognition using artificial neural network

<p>Music has a control over human moods and it can make someone calm or excited. It allows us to feel all emotions we experience. Nowadays, people are often attached with their phones and computers listening to music on Spotify, Soundcloud or any other internet platform. Music Information retrieval plays an important role for music recommendation according to lyrics, pitch, pattern of choices, and genre. In this study, we have tried to recognize the music genre for a better music recommendation system. We have collected an amount of 1820 Bangla songs from six different genres including Adhunik, Rock, Hip hop, Nazrul, Rabindra and Folk music. We have started with some traditional machine learning algorithms having K-Nearest Neighbor, Logistic Regression, Random Forest, Support Vector Machine and Decision Tree but ended up with a deep learning algorithm named Artificial Neural Network with an accuracy of 78% for recognizing music genres from six different genres. All mentioned algorithms are experimented with transformed mel-spectrograms and Mean Chroma Frequency Values of that raw amplitude data. But we found that music Tempo having Beats per Minute value with two previous features present better accuracy.</p>

STG-MTL: scalable task grouping for multi-task learning using data maps

Multi-Task Learning (MTL) is a powerful technique that has gained popularity due to its performance improvement over traditional Single-Task Learning (STL). However, MTL is often challenging because there is an exponential number of possible task groupings, which can make it difficult to choose the best one because some groupings might produce performance degradation due to negative interference between tasks. That is why existing solutions are severely suffering from scalability issues, limiting any practical application. In our paper, we propose a new data-driven method that addresses these challenges and provides a scalable and modular solution for classification task grouping based on a re-proposed data-driven features, Data Maps, which capture the training dynamics for each classification task during the MTL training. Through a theoretical comparison with other techniques, we manage to show that our approach has the superior scalability. Our experiments show a better performance and verify the method’s effectiveness, even on an unprecedented number of tasks (up to 100 tasks on CIFAR100). Being the first to work on such number of tasks, our comparisons on the resulting grouping shows similar grouping to the mentioned in the dataset, CIFAR100. Finally, we provide a modular implementation 3 3 https://github.com/ammarSherif/STG-MTL. for easier integration and testing, with examples from multiple datasets and tasks.

MiniSeg: An Extremely Minimum Network Based on Lightweight Multiscale Learning for Efficient COVID-19 Segmentation.

The rapid spread of the new pandemic, i.e., coronavirus disease 2019 (COVID-19), has severely threatened global health. Deep-learning-based computer-aided screening, e.g., COVID-19 infected area segmentation from computed tomography (CT) image, has attracted much attention by serving as an adjunct to increase the accuracy of COVID-19 screening and clinical diagnosis. Although lesion segmentation is a hot topic, traditional deep learning methods are usually data-hungry with millions of parameters, easy to overfit under limited available COVID-19 training data. On the other hand, fast training/testing and low computational cost are also necessary for quick deployment and development of COVID-19 screening systems, but traditional methods are usually computationally intensive. To address the above two problems, we propose MiniSeg, a lightweight model for efficient COVID-19 segmentation from CT images. Our efforts start with the design of an attentive hierarchical spatial pyramid (AHSP) module for lightweight, efficient, effective multiscale learning that is essential for image segmentation. Then, we build a two-path (TP) encoder for deep feature extraction, where one path uses AHSP modules for learning multiscale contextual features and the other is a shallow convolutional path for capturing fine details. The two paths interact with each other for learning effective representations. Based on the extracted features, a simple decoder is added for COVID-19 segmentation. For comparing MiniSeg to previous methods, we build a comprehensive COVID-19 segmentation benchmark. Extensive experiments demonstrate that the proposed MiniSeg achieves better accuracy because its only 83k parameters make it less prone to overfitting. Its high efficiency also makes it easy to deploy and develop. The code has been released at https://github.com/yun-liu/MiniSeg.

Analysis of the emotional coloring of text using machine and deep learning methods

The presented scientific article is a comprehensive study of machine learning and deep learning methods in the context of emotion recognition in text data. The main goal of the study is to conduct a comprehensive analysis and comparison of various machine learning and deep learning methods to classify emotions in text. During the work, special attention was paid to the analysis of traditional machine learning algorithms, such as multinomial naive Bayes (MNB), multilayer perceptron (MLP), and support vector machine (SVM), as well as the use of deep learning methods based on long short-term memory (LSTM). The experimental part of the study involves the analysis of different data sets covering a variety of text styles and contexts. The experimental results are analyzed in detail, identifying the advantages and limitations of each method. The article provides practical recommendations for choosing the optimal method depending on the specific tasks and context of the application. The data obtained is important for the development of intelligent systems that can effectively adapt to the emotional aspects of interaction with users. Overall, this work makes a significant contribution to the field of emotion recognition in text and provides a basis for further research in this area.

On the diagnosis of chronic kidney disease using a machine learning-based interface with explainable artificial intelligence

Chronic Kidney Disease (CKD) is increasingly recognised as a major health concern due to its rising prevalence. The average survival period without functioning kidneys is typically limited to approximately 18 days, creating a significant need for kidney transplants and dialysis. Early detection of CKD is crucial, and machine learning methods have proven effective in diagnosing the condition, despite their often opaque decision-making processes. This study utilised explainable machine learning to predict CKD, thereby overcoming the 'black box' nature of traditional machine learning predictions. Of the six machine learning algorithms evaluated, the extreme gradient boost (XGB) demonstrated the highest accuracy. For interpretability, the study employed Shapley Additive Explanations (SHAP) and Partial Dependency Plots (PDP), which elucidate the rationale behind the predictions and support the decision-making process. Moreover, for the first time, a graphical user interface with explanations was developed to diagnose the likelihood of CKD. Given the critical nature and high stakes of CKD, the use of explainable machine learning can aid healthcare professionals in making accurate diagnoses and identifying root causes.

Analyzing electroencephalograph signals for early Alzheimer’s disease detection: deep learning vs. traditional machine learning approaches

Alzheimer’s disease (AD) stands as a progressive neurodegenerative disorder with a significant global public health impact. It is imperative to establish early and accurate diagnoses of AD to facilitate effective interventions and treatments. Recent years have witnessed the emergence of machine learning (ML) and deep learning (DL) techniques, displaying promise in various medical domains, including AD diagnosis. This study undertakes a comprehensive contrast between conventional machine learning methods and advanced deep learning strategies for early AD diagnosis. Conventional ML algorithms like support vector machines, decision trees, and K nearest neighbor have been extensively employed for AD diagnosis through relevant feature extraction from heterogeneous data sources. Conversely, deep learning techniques such as multilayer perceptron (MLP) and convolutional neural networks (CNNs) have demonstrated exceptional aptitude in autonomously uncovering intricate patterns and representations from unprocessed data like EEG data. The findings reveal that while traditional ML methods may perform adequately with limited data, deep learning techniques excel when ample data is available, showcasing their potential for early and precise AD diagnosis. In conclusion, this research paper contributes to the ongoing discourse surrounding the choice of appropriate methodologies for early Alzheimer’s disease diagnosis.

Deep Neural Networks and Tabular Data: A Survey.

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

Stochastic calculus-guided reinforcement learning: A probabilistic framework for optimal decision-making

Stochastic Calculus-guided Reinforcement learning (SCRL) is a new way to make decisions in situations where things are uncertain. It uses mathematical principles to make better choices and improve decision-making in complex situations. SCRL works better than traditional Stochastic Reinforcement Learning (SRL) methods. In tests, SCRL showed that it can adapt and perform well. It was better than the SRL methods. SCRL had a lower dispersion value of 63.49 compared to SRL's 65.96. This means SCRL had less variation in its results. SCRL also had lower risks than SRL in the short- and long-term. SCRL's short-term risk value was 0.64, and its long-term risk value was 0.78. SRL's short-term risk value was much higher at 18.64, and its long-term risk value was 10.41. Lower risk values are better because they mean less chance of something going wrong. Overall, SCRL is a better way to make decisions when things are uncertain. It uses math to make smarter choices and has less risk than other methods. Also, different metrics, viz training rewards, learning progress, and rolling averages between SRL and SCRL, were assessed, and the study found that SCRL outperforms well compared to SRL. This makes SCRL very useful for real-world situations where decisions must be made carefully.•By leveraging mathematical principles derived from stochastic calculus, SCRL offers a robust framework for making informed choices and enhancing performance in complex scenarios.•In comparison to traditional SRL methods, SCRL demonstrates superior adaptability and efficacy, as evidenced by empirical tests.

Enhancing self-regulated learning and critical thinking in STEM through flipped classroom models

One of the causes of low proficiency among high school students in mathematical critical thinking is their lack of willingness to learn independently. This study investigated the mathematical critical thinking skills and self-regulated learning of high school students through the Flipped Classroom STEM approach. The research employed a sequential explanatory design, beginning with quasi-experimental pre-tests and post-tests for quantitative assessment of the learning process, followed by qualitative methods such as questionnaires, interviews, and observations for deeper analysis. Data analysis utilized quantitative techniques, including normality and homogeneity tests and hypothesis testing in SPSS, alongside qualitative methods for data reduction, data presentation, and conclusion. The findings revealed that flipped classroom learning significantly improved students' abilities in critical mathematical thinking, particularly highlighting the importance of self-regulated learning. There was a notable difference in post-test scores compared to those from traditional learning methods. Moreover, the study found that students with higher self-regulated learning skills were more effective in critical thinking, underscoring the effectiveness of the flipped classroom approach in fostering independent learning and enhancing essential thinking abilities in mathematics education.