Machine Learning Metrics Research Articles

A review of model evaluation metrics for machine learning in genetics and genomics.

Machine learning (ML) has shown great promise in genetics and genomics where large and complex datasets have the potential to provide insight into many aspects of disease risk, pathogenesis of genetic disorders, and prediction of health and wellbeing. However, with this possibility there is a responsibility to exercise caution against biases and inflation of results that can have harmful unintended impacts. Therefore, researchers must understand the metrics used to evaluate ML models which can influence the critical interpretation of results. In this review we provide an overview of ML metrics for clustering, classification, and regression and highlight the advantages and disadvantages of each. We also detail common pitfalls that occur during model evaluation. Finally, we provide examples of how researchers can assess and utilise the results of ML models, specifically from a genomics perspective.

Birds, bats and beyond: evaluating generalization in bioacoustics models

In the context of passive acoustic monitoring (PAM) better models are needed to reliably gain insights from large amounts of raw, unlabeled data. Bioacoustics foundation models, which are general-purpose, adaptable models that can be used for a wide range of downstream tasks, are an effective way to meet this need. Measuring the capabilities of such models is essential for their development, but the design of robust evaluation procedures is a complex process. In this review we discuss a variety of fields that are relevant for the evaluation of bioacoustics models, such as sound event detection, machine learning metrics, and transfer learning (including topics such as few-shot learning and domain generalization). We contextualize these topics using the particularities of bioacoustics data, which is characterized by large amounts of noise, strong class imbalance, and distribution shifts (differences in the data between training and deployment stages). Our hope is that these insights will help to inform the design of evaluation protocols that can more accurately predict the ability of bioacoustics models to be deployed reliably in a wide variety of settings.

Deciphering the environmental chemical basis of muscle quality decline by interpretable machine learning models

BackgroundSarcopenia is known as a decline in skeletal muscle quality and function that is associated with age. Sarcopenia is linked to diverse health problems, including endocrine-related diseases. Environmental chemicals (ECs), a broad class of chemicals released from industry, may influence muscle quality decline. ObjectivesIn this work, we aimed to simultaneously elucidate the associations between muscle quality decline and diverse EC exposures based on the data from the 2011–2012 and 2013–2014 survey cycles in the National Health and Nutrition Examination Survey (NHANES) project using machine learning models. MethodsSix machine learning models were trained based on the EC and non-EC exposures from NHANES to distinguish low from normal muscle quality index status. Different machine learning metrics were evaluated for these models. The Shapley additive explanations (SHAP) approach was used to provide explainability for machine learning models. ResultsRandom forest (RF) performed best on the independent testing data set. Based on the testing data set, ECs can independently predict the binary muscle quality status with good performance by RF (area under the receiver operating characteristic curve = 0.793; area under the precision-recall curve = 0.808). The SHAP ranked the importance of ECs for the RF model. As a result, several metals and chemicals in urine, including 3-phenoxybenzoic acid and cobalt, were more associated with the muscle quality decline. ConclusionsAltogether, our analyses suggest that ECs can independently predict muscle quality decline with a good performance by RF, and the SHAP-identified ECs can be closely related to muscle quality decline and sarcopenia. Our analyses may provide valuable insights into ECs that may be the important basis of sarcopenia and endocrine-related diseases in United States populations.

Whose work matters? A tool for identifying and developing more inclusive physics textbooks

The lack of representational diversity and role models in physics, including in our textbooks and curricular materials, is an oft-cited contributing factor to the continuing dramatic under-representation of women and people of color in physics. In this work, we develop an automated, Python-based tool for identifying the names and demographics of scientists who are mentioned in indices and chapters of physics textbooks, enabling authors, publishers, and users of physics textbooks to rapidly analyze the demographics of these texts. We quantitatively validate the automated tool using standard machine learning metrics, attaining high accuracy, precision, recall, and F1 scores. The tool is then used to demonstrate two of the many potential applications: examining whose work is mentioned in the entire collection of textbooks used in a representative four-year undergraduate physics major curriculum as well as an analysis of the demographics of scientists mentioned in a selection of ten introductory physics textbooks. Both of the sample analyses result in a similar portrait, showing that the undergraduate physics textbooks examined in this work focus overwhelmingly on work attributed to White men of European, British, and North American descent. This work points to an urgent need for the physics education community, including textbook publishers, authors, and adopters, to work together to broaden our portrayals of physics to reflect the vast diversity of scientists, both historically and contemporaneously, who are working in this field.

Opportunities to Reduce the Risk of Cardiovascular Death by Improving Machine Learning Methods

Improving algorithms for automatic recognition of electrocardiograms requires increasing of training dataset, which is not always possible due to the rarity of certain cardiac diseases or ethical issues. It is possible to improve the algorithms for generating synthetic electrocardiograms using mechanistic models and generative-descriptive neural networks (GANs). At the same time, when evaluating the effectiveness of the proposed solutions, various authors offer different quality assessment metrics from subjective expert assessment to the squared mean error. We compare two approaches to generating synthetic electrocardiograms: pseudo-ECG generation using a one-dimensional cardiomyocyte model and GAN based on long-term short-term memory in terms of machine learning metrics: accuracy, recall, f1-score. We solve the problem of binary classification with bagging method, class 0 — normal sinus rhythm, class 1 — atrial fibrillation. We found that classifier trained on synthetic ECGs generated using GAN is slightly more effective compared to pseudo-ECGs generated using mechanistic models ones. At the same time, it turned out that GANs are not stable enough. The generation of synthetic electrocardiograms using both mechanistic models and GAN can be used to enrich the training set in case of recognition of rare cardiac diseases.

Seismically Informed Reference Models Enhance AI‐Based Earthquake Prediction Systems

AbstractGiven the robust nonlinear regression capabilities of Artificial Intelligence (AI) technology, its commendable performance in numerous geophysical tasks is expected. Yet, AI technology suffers from (a) its “black box” nature and (b) the fact that some complicated artificial neural networks (ANNs) claiming superior performance do not surpass some simple geophysical models that clearly describe the underlying physical processes. Numerous reports rely on standard machine learning metrics, often using a spatially uniform Poisson (SUP) distribution as their reference. A good performance just means that the artificial neural network (ANN) outperforms this basic reference, potentially offering little novelty to the scientific community. Worse, this can lead to spurious inference. We demonstrate this by using the monthly average human‐made Nighttime Light Map and the cumulative energy of earthquakes in various space‐time units as inputs for an Long short‐term memory model. The goal is to predict earthquakes with a magnitude of M ≥ 5.0 across the entire Chinese Mainland. With the SUP reference model, the ANN concludes that human‐made Nighttime Light possesses substantial earthquake prediction capability. This is evidently flawed reasoning. We show that this stems from the poor reference model and this spurious inference disappears when using a better benchmark consisting of a spatially varying Poisson (SVP) model informed from statistical seismology. This is implemented by weighting the punishments/rewards of our ANN associated with failed/successful predictions by prior probabilities provided by the stronger SVP model. Scores obtained with the time‐space Molchan diagram demonstrate the strong performance improvement obtained by training ANN with a better reference model.

Robust mortality prediction on a recirculating aquaculture system.

Aquaculture presents itself as one of the most rapidly developing means of sustainable production of animal protein to feed ever-growing populations. Recirculating aquaculture systems offer higher control and fewer inconveniences than traditional systems, making them an attractive option for fish production. Although the sector's digitalization is in its early stages, its application should increase its rentability while conserving the environment. This paper aims to promote the sector's evolution by assessing parameter importance in mortality with tree-based machine learning models, verifying the method's natural robustness and how it compares to a specially devised one, and at the same time evaluating the concept's relevance in predicting categorical mortality values. In particular, to better understand the aquaculture production process through a systematic data evaluation, an exploration based on real-time data acquisition is fully needed. Moreover, algorithm robustness is a key ingredient in this application since measurements are greatly affected by errors. This invalidates the application of traditional machine learning methods, where models are sensitive to production data variations and sensor noise. The study found the parameters that play relevant roles in the production phases, such as pH and nitrate concentration. While the obtained predictive metrics are still sub-optimal, further enhancements could be achieved through rigorous analysis of feature engineering, fine-tuning model hyperparameters, and exploring more advanced algorithms. Additionally, incorporating larger and more diverse datasets, refining data pre-processing techniques, and iteratively optimizing the model architecture may contribute to significant improvements in predictive performance. Despite that, the impact costs of using adjusted machine learning metrics are clear, as are the importance of data rounding in pre-processing and directions for improvement regarding data acquisition and transformation.

A Novel Brain-Machine Safety System Based on Drowsiness Detection using the PNN and MLP Algorithms

This work aims to develop a safety system capable of detecting drowsiness, in order to reduce the accident rate, the main idea is based on the classification of the cerebral signal captured by the electroencephalograph, after processing to eliminate all noise, the resulting data will then be segmented into two classes, the first category is devoted to the state of wakefulness and the second will be devoted to drowsiness. This categorization is based on the combination of features such as mean and standard deviation, and performed by two neural network algorithms, the probabilistic neural network and the multilayer perceptron, whose performance is evaluated according to machine learning metrics such as accuracy, error, specificity, sensitivity, F1 indicator and ROC curve representation. The binary classification results are highly significant in terms of class discrimination, with both models achieving excellent performance, with the MLP classifier outperforming the PNN model with an accuracy of 96.10% and an accuracy rate of 98.43%. This suggests that the MLP model can be effectively applied to critical tasks where good discrimination is required, such as detecting drowsiness in drivers to avoid accidents.

Critical success index or F measure to validate the accuracy of administrative healthcare data identifying epilepsy in deceased adults in Scotland

BackgroundMethods to undertake diagnostic accuracy studies of administrative epilepsy data are challenged by lack of a way to reliably rank case-ascertainment algorithms in order of their accuracy. This is because it is difficult to know how to prioritise positive predictive value (PPV) and sensitivity (Sens). Large numbers of true negative (TN) instances frequently found in epilepsy studies make it difficult to discriminate algorithm accuracy on the basis of negative predictive value (NPV) and specificity (Spec) as these become inflated (usually >90%). This study demonstrates the complementary value of using weather forecasting or machine learning metrics critical success index (CSI) or F measure, respectively, as unitary metrics combining PPV and sensitivity. We reanalyse data published in a diagnostic accuracy study of administrative epilepsy mortality data in Scotland. MethodCSI was calculated as 1/[(1/PPV) + (1/Sens) – 1]. F measure was calculated as 2.PPV.Sens/(PPV + Sens). CSI and F values range from 0 to 1, interpreted as 0 = inaccurate prediction and 1 = perfect accuracy. The published algorithms were reanalysed using these and their accuracy re-ranked according to CSI in order to allow comparison to the original rankings. ResultsCSI scores were conservative (range 0.02–0.826), always less than or equal to the lower of the corresponding PPV (range 39%–100%) and sensitivity (range 2%–93%). F values were less conservative (range 0.039–0.905), sometimes higher than either PPV or sensitivity, but were always higher than CSI. Low CSI and F values occurred when there was a large difference between PPV and sensitivity, e.g. CSI was 0.02 and F was 0.039 in an instance when PPV was 100% and sensitivity was 2%. Algorithms with both high PPV and sensitivity performed best in terms of CSI and F measure, e.g. CSI was 0.826 and F was 0.905 in an instance when PPV was 90% and sensitivity was 91%. ConclusionCSI or F measure can combine PPV and sensitivity values into a convenient single metric that is easier to interpret and rank in terms of diagnostic accuracy than trying to rank diagnostic accuracy according to the two measures themselves. CSI or F prioritise instances where both PPV and sensitivity are high over instances where there are large differences between PPV and sensitivity (even if one of these is very high), allowing diagnostic accuracy thresholds based on combined PPV and sensitivity to be determined. Therefore, CSI or F measures may be helpful complementary metrics to report alongside PPV and sensitivity in diagnostic accuracy studies of administrative epilepsy data.

Machine Learning for Credit Risk Prediction: A Systematic Literature Review

In this systematic review of the literature on using Machine Learning (ML) for credit risk prediction, we raise the need for financial institutions to use Artificial Intelligence (AI) and ML to assess credit risk, analyzing large volumes of information. We posed research questions about algorithms, metrics, results, datasets, variables, and related limitations in predicting credit risk. In addition, we searched renowned databases responding to them and identified 52 relevant studies within the credit industry of microfinance. Challenges and approaches in credit risk prediction using ML models were identified; we had difficulties with the implemented models such as the black box model, the need for explanatory artificial intelligence, the importance of selecting relevant features, addressing multicollinearity, and the problem of the imbalance in the input data. By answering the inquiries, we identified that the Boosted Category is the most researched family of ML models; the most commonly used metrics for evaluation are Area Under Curve (AUC), Accuracy (ACC), Recall, precision measure F1 (F1), and Precision. Research mainly uses public datasets to compare models, and private ones to generate new knowledge when applied to the real world. The most significant limitation identified is the representativeness of reality, and the variables primarily used in the microcredit industry are data related to the Demographic, Operation, and Payment behavior. This study aims to guide developers of credit risk management tools and software towards the existing ability of ML methods, metrics, and techniques used to forecast it, thereby minimizing possible losses due to default and guiding risk appetite.

Automated Image Quality and Protocol Adherence Assessment of Examinations in Teledermatology: First Results.

Introduction: Image quality and acquisition protocol adherence assessment is a neglected area in teledermatology. We examine if it is feasible to use deep learning methods to automate the assessment of the adherence of examinations to image acquisition protocols. In this study, we focused on the quality criteria of two image acquisition protocols: (1) approximation image and (2) panoramic image, as these are present in all teledermatology examination protocols currently used by the Santa Catarina State Integrated Telemedicine and Telehealth System (STT/SC). Methods: We use a data set of 36,102 teledermatological examinations performed at the STT/SC during 2021. As our validation process, we adopted standard machine learning metrics and an inter-rater agreement (IRA) study with 11 dermatologists. For the approximation image protocol, we used the Mask-Region based Convolutional Neural Network (RCNN) Object Detection Deep Learning (DL) architecture to identify the presence of a lesion identification tag and a ruler used to provide a frame reference of the lesion. For the panoramic image protocol, we used DensePose, a pose estimation DL, architecture to assess the presence of a whole patient body and its orientation. A combination of the two approaches was additionally validated through an IRA study between specialists. Results: Mask-RCNN achieved a score of 96% mean average precision (mAP), while DensePose presented 75% mAP. IRA achieved a level of agreement of 96.68% with the Krippendorff alpha score. Conclusions: Our results show the feasibility of using deep learning to automate the image quality and protocol adherence assessment in teledermatology, before the specialist's manual analysis of the examination.

Towards practical object detection for weed spraying in precision agriculture.

Weeds pose a persistent threat to farmers' yields, but conventional methods for controlling weed populations, like herbicide spraying, pose a risk to the surrounding ecosystems. Precision spraying aims to reduce harms to the surrounding environment by targeting only the weeds rather than spraying theentire field with herbicide. Such an approach requires weeds to first be detected. With the advent of convolutional neural networks, there has been significant research trialing such technologies on datasets of weeds and crops. However, the evaluation of the performance of these approaches has often been limited to the standard machine learning metrics. This paper aims to assess the feasibility of precision spraying via a comprehensive evaluation of weed detection and spraying accuracy using two separate datasets, different image resolutions, and several state-of-the-art object detection algorithms. A simplified model of precision spraying is proposed to compare the performance of different detection algorithms while varying the precision of the spray nozzles. The key performance indicators in precision spraying that this study focuses on are a high weed hit rate and a reduction in herbicide usage. This paper introduces two metrics, namely, weed coverage rate and area sprayed, to capture these aspects of the real-world performance of precision spraying and demonstrates their utility through experimental results. Using these metrics to calculate the spraying performance, it was found that 93% of weeds could be sprayed by spraying just 30% of the area using state-of-the-art vision methods to identify weeds.

Active Object Learning for intelligent social robots

The applications of social robots have continued to rise in number over the years: there are now mainly used in nursing homes and hospitals, performing different tasks like assistance, physical and cognitive rehabilitation, and supervision. However, robots’ ability to learn from humans remains a problem to be solved. This paper presents the development of an Active Object Learning architecture in which the robot can identify unknown objects and autonomously initiate the learning process, supported by user interactions. We study the feasibility of generating a real-time dataset, including techniques for reducing image repetition and online classification model training. The work considers the analysis of machine learning metrics such as F1-Score, accuracy, confidence, and hit rate as the robot learns new objects. The accuracy of the classifier remains constant as the number of classes increases, indicating that the regularisation methods reduce overfitting and decrease the generalisation gap. The results prove that the robot can perform the classifications correctly and differentiate between known and unknown objects. The paper also includes a case study to demonstrate the feasibility of the proposed system integrated into a real social robot.

Generating Synthetic Dataset for ML-Based IDS Using CTGAN and Feature Selection to Protect Smart IoT Environments

Networks within the Internet of Things (IoT) have some of the most targeted devices due to their lightweight design and the sensitive data exchanged through smart city networks. One way to protect a system from an attack is to use machine learning (ML)-based intrusion detection systems (IDSs), significantly improving classification tasks. Training ML algorithms require a large network traffic dataset; however, large storage and months of recording are required to capture the attacks, which is costly for IoT environments. This study proposes an ML pipeline using the conditional tabular generative adversarial network (CTGAN) model to generate a synthetic dataset. Then, the synthetic dataset was evaluated using several types of statistical and ML metrics. Using a decision tree, the accuracy of the generated dataset reached 0.99, and its lower complexity reached 0.05 s training and 0.004 s test times. The results show that synthetic data accurately reflect real data and are less complex, making them suitable for IoT environments and smart city applications. Thus, the generated synthetic dataset can further train models to secure IoT networks and applications.

Improving the quality ofhospital sterilization process using failure modes andeffects analysis, fuzzy logic, andmachine learning: experience intertiary dental centre.

Activities practiced in the hospital generate several types of risks. Therefore, performing the risk assessment is one of the quality improvement keys in the healthcare sector. For this reason, healthcare managers need to design and perform efficient risk assessment processes. Failure modes and effects analysis (FMEA) is one of the most used risk assessment methods. The FMEA is a proactive technique consisting of the evaluation of failure modes associated with a studied process using three factors: occurrence, non-detection, and severity, in order to obtain the risk priority number using fuzzy logic approach and machine learning algorithms, namely the support vector machine and the k-nearest neighbours. The proposed model is applied in the case of the central sterilization unit of a tertiary national reference centre of dental treatment, where its efficiency is evaluated compared to the classical approach. These comparisons are based on expert advice and machine learning performance metrics. Our developed model proved high effectiveness throughout the results of the expert's vote (she agrees with 96% fuzzy-FMEA results against 6% with classical FMEA results). Furthermore, the machine learning metrics show a high level of accuracy in both training data (best rate is 96%) and testing data (90%). This study represents the first study that aims to perform artificial intelligence approach to risk management in the Moroccan healthcare sector. The perspective of this study is to promote the application of the artificial intelligence in Moroccan health management, especially in the field of quality and safety management.

A deep learning based sensor fusion method to diagnose tightening errors

Modern smart assembly lines commonly include electric tools with built-in sensors to tighten safety-critical joints. These sensors generate data that are subsequently analyzed by human experts to diagnose potential tightening errors. Previous research aimed to automate the diagnosing process by developing diagnosing models based on tightening theory and calibration of the friction coefficient in specific lab setups. Generalizing these results is difficult and often unsuccessful since friction coefficients vary between lab and production environments. To overcome this problem, this paper presents a novel methodology that builds multi-label classification deep learning models for diagnosing tightening errors using production data. The proposed methodology comprises three key contributions, i.e., the Labrador method, the Model Combo (MoBo) framework, and a heuristic evaluation method. Labrador is an elastic deep learning based sensor fusion method that (1) uses feature encoders to extract features; (2) conducts data-level and/or feature-level sensor fusion in both time and frequency domains; and (3) performs multi-label classification to detect and diagnose tightening errors. MoBo is a configurable and modular framework that supports Labrador in identifying optimal feature encoders. With MoBo and Labrador, one can easily explore and design a bounded search space for sensor fusion strategies (SFSs) and feature encoders. In order to identify the optimal solution within the defined search space, this paper introduces a heuristic method. By evaluating the trade-off between machine learning (ML) metrics (e.g., accuracy, subset accuracy, and F1) and operational (OP) metrics (e.g., inference latency), the proposed method identifies the most suitable solution depending on the requirements of individual use cases. In the experimental evaluation, we adopt the proposed methodology to identify the most suitable multi-label classification solutions for diagnosing tightening errors. To optimize ML metrics, the identified solution achieved 99.69% accuracy, 93.39% subset accuracy, 97.39% F1, and 6.68ms inference latency. To optimize OP metrics, the identified solution achieved 99.66% accuracy, 92.65% subset accuracy, 97.28% F1, and 2.41ms inference latency.

An advanced ensemble modeling approach for predicting carbonate reservoir porosity from seismic attributes

This study uses a machine learning (ML) ensemble modeling approach to predict porosity from multiple seismic attributes in one of the most promising Main Dolomite hydrocarbon reservoirs in NW Poland. The presented workflow tests five different model types of varying complexity: K-nearest neighbors (KNN), random forests (RF), extreme gradient boosting (XGB), support vector machine (SVM), single layer neural network with multilayer perceptron (MLP). The selected models are additionally run with different configurations originating from the pre-processing stage, including Yeo–Johnson transformation (YJ) and principal component analysis (PCA). The race ANOVA method across resample data is used to tune the best hyperparameters for each model. The model candidates and the role of different pre-processors are evaluated based on standard ML metrics – coefficient of determination (R2), root mean squared error (RMSE), and mean absolute error (MAE). The model stacking is performed on five model candidates: two KNN, two XGB, and one SVM PCA with a marginal role. The results of the ensemble model showed superior accuracy over single learners, with all metrics (R2 0.890, RMSE 0.0252, MAE 0.168). It also turned out to be almost three times better than the neural net (NN) results obtained from commercial software on the same testing set (R2 0.318, RMSE 0.0628, MAE 0.0487). The spatial distribution of porosity from the ensemble model indicated areas of good reservoir properties that overlap with hydrocarbon production fields. This observation completes the evaluation of the ensemble technique results from model metrics. Overall, the proposed solution is a promising tool for better porosity prediction and understanding of heterogeneous carbonate reservoirs from multiple seismic attributes.

Machine learning models to predict outcomes at 30-days using Global Leadership Initiative on Malnutrition combinations with and without muscle mass in people with cancer.

Equipment to assess muscle mass is not available in all health services. Yet we have limited understanding of whether applying the Global Leadership Initiative on Malnutrition (GLIM) criteria without an assessment of muscle mass affects the ability to predict adverse outcomes. This study used machine learning to determine which combinations of GLIM phenotypic and etiologic criteria are most important for the prediction of 30-day mortality and unplanned admission using combinations including and excluding low muscle mass. In a cohort of 2801 participants from two cancer malnutrition point prevalence studies, we applied the GLIM criteria with and without muscle mass. Phenotypic criteria were assessed using ≥5% unintentional weight loss, body mass index, subjective assessment of muscle stores from the PG-SGA. Aetiologic criteria included self-reported reduced food intake and inflammation (metastatic disease). Machine learning approaches were applied to predict 30-day mortality and unplanned admission using models with and without muscle mass. Participants with missing data were excluded, leaving 2494 for analysis [49.6% male, mean (SD) age: 62.3 (14.2) years]. Malnutrition prevalence was 19.5% and 17.5% when muscle mass was included and excluded, respectively. However, 48 (10%) of malnourished participants were missed if muscle mass was excluded. For the nine GLIM combinations that excluded low muscle mass the most important combinations to predict mortality were (1) weight loss and inflammation and (2) weight loss and reduced food intake. Machine learning metrics were similar in models excluding or including muscle mass to predict mortality (average accuracy: 84% vs. 88%; average sensitivity: 41% vs. 38%; average specificity: 85% vs. 89%). Weight loss and reduced food intake was the most important combination to predict unplanned hospital admission. Machine learning metrics were almost identical in models excluding or including muscle mass to predict unplanned hospital admission, with small differences observed only if reported to one decimal place (average accuracy: 77% vs. 77%; average sensitivity: 29% vs. 29%; average specificity: 84% vs. 84%). Our results indicate predictive ability is maintained, although the ability to identify all malnourished patients is compromised, when muscle mass is excluded from the GLIM diagnosis. This has important implications for assessment in health services where equipment to assess muscle mass is not available. Our findings support the robustness of the GLIM approach and an ability to apply some flexibility in excluding certain phenotypic or aetiologic components if necessary, although some cases will be missed.

Network Intrusion Prediction using Machine Learning

Intrusion Detection Systems are designed to safeguard the security needs of enterprise networks against cyber-attacks. However, networks suffer from several limitations, such as generating a high volume of low-quality alerts. While there are a variety of intrusion detection solutions available, the prediction of network intrusion events is still under active investigation. Over the past, statistical methods have dominated the design of attack prediction methods. The analysis of dataset by Supervised Machine Learning Technique(SMLT) to capture several information’s like, variable identification, univariate analysis, bivariate and multivariate analysis, missing value treatments etc. Then, we going to implement a machine-learning algorithm to get better accuracy. After that, we can build looks like a web page by using Django Framework. Once the user passed the information then the Admin can see the output, and after he will send the response to the user. The result shows the effectiveness of the machine learning metrics which are accuracy, precision, Recall, F1 Score, Sensitivity, and Specificity

Is attention all you need for intraday Forex trading?

AbstractThe main objective of this paper is to analyse whether the Transformer neural network, which has become one of the most influential algorithms in Artificial Intelligence over the last few years, exhibits predictive capabilities for high‐frequency Forex data. The prediction task is to classify short‐term Forex movements for six currency pairs and five different time intervals from 60 to 720 min. We find that the Transformer exhibits high predictive power in the context of intraday Forex trading. This performance is slightly better than for the carefully selected benchmark – ResNet‐LSTM, which currently is a state‐of‐the‐art algorithm. Since intraday Forex trading based on deep learning models is largely unexplored, we offer insight on which currency pair and time interval are amenable to devising a profitable trading strategy. We also show that high predictive accuracy can be misleading in real world trading for short time intervals, as models trained on OHLC data tend to report the highest accuracy when the spread cost is the highest. This renders assessment based on typical machine learning metrics overly optimistic. Therefore, it is critical to backtest frequent intraday Forex trading strategies with realistic cost assumptions, which is rarely the case in empirical literature. Lastly, sensitivity analysis shows that the length of the time interval used for training does not play a critical role in the Transformer's predictive capabilities, whereas features derived from technical analysis are essential.