Learning Metrics Research Articles

Central Arterial Dynamic Evaluation from Peripheral Blood Pressure Waveforms Using CycleGAN: An In Silico Approach.

Arterial stiffness is a major condition related to many cardiovascular diseases. Traditional approaches in the assessment of arterial stiffness supported by machine learning techniques are limited to the pulse wave velocity (PWV) estimation based on pressure signals from the peripheral arteries. Nevertheless, arterial stiffness can be assessed based on the pressure-strain relationship by analyzing its hysteresis loop. In this work, the capacity of deep learning models based on generative adversarial networks (GANs) to transfer pressure signals from the peripheral arterial region to pressure and area signals located in the central arterial region is explored. The studied signals are from a public and validated virtual database. Compared to other works in which the assessment of arterial stiffness was performed via PWV, in the present work the pressure-strain hysteresis loop is reconstructed and evaluated in terms of classical machine learning metrics and clinical parameters. Least-square GAN (LSGAN) and Wasserstein GAN with gradient penalty (WGAN-GP) adversarial losses are compared, yielding better results with LSGAN. LSGAN mean ± standard deviation of error for pressure and area pulse waveforms are 0.8 ± 0.4 mmHg and 0.1 ± 0.1 cm2, respectively. Regarding the pressure-strain elastic modulus, it is achieved a mean absolute percentage error of 6.5 ± 5.1%. GAN-based deep learning models can recover the pressure-strain loop of central arteries while observing pressure signals from peripheral arteries.

The use of machine learning and deep learning techniques to assess proprioceptive impairments of the upper limb after stroke

BackgroundRobots can generate rich kinematic datasets that have the potential to provide far more insight into impairments than standard clinical ordinal scales. Determining how to define the presence or absence of impairment in individuals using kinematic data, however, can be challenging. Machine learning techniques offer a potential solution to this problem. In the present manuscript we examine proprioception in stroke survivors using a robotic arm position matching task. Proprioception is impaired in 50–60% of stroke survivors and has been associated with poorer motor recovery and longer lengths of hospital stay. We present a simple cut-off score technique for individual kinematic parameters and an overall task score to determine impairment. We then compare the ability of different machine learning (ML) techniques and the above-mentioned task score to correctly classify individuals with or without stroke based on kinematic data.MethodsParticipants performed an Arm Position Matching (APM) task in an exoskeleton robot. The task produced 12 kinematic parameters that quantify multiple attributes of position sense. We first quantified impairment in individual parameters and an overall task score by determining if participants with stroke fell outside of the 95% cut-off score of control (normative) values. Then, we applied five machine learning algorithms (i.e., Logistic Regression, Decision Tree, Random Forest, Random Forest with Hyperparameters Tuning, and Support Vector Machine), and a deep learning algorithm (i.e., Deep Neural Network) to classify individual participants as to whether or not they had a stroke based only on kinematic parameters using a tenfold cross-validation approach.ResultsWe recruited 429 participants with neuroimaging-confirmed stroke (< 35 days post-stroke) and 465 healthy controls. Depending on the APM parameter, we observed that 10.9–48.4% of stroke participants were impaired, while 44% were impaired based on their overall task score. The mean performance metrics of machine learning and deep learning models were: accuracy 82.4%, precision 85.6%, recall 76.5%, and F1 score 80.6%. All machine learning and deep learning models displayed similar classification accuracy; however, the Random Forest model had the highest numerical accuracy (83%). Our models showed higher sensitivity and specificity (AUC = 0.89) in classifying individual participants than the overall task score (AUC = 0.85) based on their performance in the APM task. We also found that variability was the most important feature in classifying performance in the APM task.ConclusionOur ML models displayed similar classification performance. ML models were able to integrate more kinematic information and relationships between variables into decision making and displayed better classification performance than the overall task score. ML may help to provide insight into individual kinematic features that have previously been overlooked with respect to clinical importance.

Distance and Direction Based Deep Discriminant Metric Learning for Kinship Verification

Image-based kinship verification is an important task in computer vision and has many applications in practice, such as missing children search and family album construction, among others. Due to the differences in age, gender, expression and appearance, there usually exists a large discrepancy between the facial images of parent and child. This makes kinship verification a challenging task. In this article, we propose a Distance and Direction Based Deep Discriminant Metric Learning (D 4 ML) approach for kinship verification. The basic idea of D 4 ML is to make full use of the discriminant information contained in the facial images of parent and child such that the network can learn more a discriminating distance metric. Specifically, D 4 ML learns the metric by utilizing the discriminant information from two perspectives: distance-based perspective and direction-based perspective. From the distance-based perspective, the designed loss function is used to minimize the distance between images having kinship and maximize the distance between images without kinship. In practice, the gender difference and large age gap may significantly increase the distance between facial images of parent and child. Therefore, learning the metric only from a distance-based perspective is insufficient. Considering that two vectors with a large distance may appear with high similarity in direction, D 4 ML also employs the direction-based loss function in the training process. Both kinds of loss function work together to improve the discriminability of the learned metric. Experimental results on four small size publicly available datasets demonstrate the effectiveness of our approach. Source code of our approach can be found at https://github.com/lclhenu/D4ML .

Botnet attacks detection in IoT environment using machine learning techniques

IoT devices with weak security designs are a serious threat to organizations. They are the building blocks of Botnets, the platforms that launch organized attacks that are capable of shutting down an entire infrastructure. Researchers have been developing IDS solutions that can counter such threats, often by employing innovation from other disciplines like artificial intelligence and machine learning. One of the issues that may be encountered when machine learning is used is dataset purity. Since they are not captured from perfect environments, datasets may contain data that could affect the machine learning process, negatively. Algorithms already exist for such problems. Repeated Edited Nearest Neighbor (RENN), Encoding Length (Explore), and Decremental Reduction Optimization Procedure 5 (DROP5) algorithm can filter noises out of datasets. They also provide other benefits such as instance reduction which could help reduce larger Botnet datasets, without sacrificing their quality. Three datasets were chosen in this study to construct an IDS: IoTID20, N-BaIoT and MedBIoT. The filtering algorithms, RENN, Explore, and DROP5 were used on them to filter noise and reduce instances. Noise was also injected and filtered again to assess the resilience of these filters. Then feature optimizations were used to shrink the dataset features. Finally, machine learning was applied on the processed dataset and the resulting IDS was evaluated with the standard supervised learning metrics: Accuracy, Precision, Recall, Specificity, F-Score and G-Mean. Results showed that RENN and DROP5 filtering delivered excellent results. DROP5, in particular, managed to reduce the dataset substantially without sacrificing accuracy. However, when noise got injected, the DROP5 accuracy went down and could not keep up. Of the three dataset, N-BaIoT delivers the best accuracy overall across the learning techniques.

Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing

To process and transfer large amounts of data in emerging wireless services, it has become increasingly appealing to exploit distributed data communication and learning. Specifically, edge learning (EL) enables local model training on geographically disperse edge nodes and minimizes the need for frequent data exchange. However, the current design of separating EL deployment and communication optimization does not yet reap the promised benefits of distributed signal processing, and sometimes suffers from excessive signalling overhead, long processing delay, and unstable learning convergence. In this paper, we provide an overview on practical distributed EL techniques and their interplay with advanced communication optimization designs. In particular, typical performance metrics for dual-functional learning and communication networks are discussed. Also, recent achievements of enabling techniques for the dual-functional design are surveyed with exemplifications from the mutual perspectives of "communications for learning" and "learning for communications." The application of EL techniques within a variety of future communication systems are also envisioned for beyond 5G (B5G) wireless networks. For the application in goal-oriented semantic communication, we present a first mathematical model of the goal-oriented source entropy as an optimization problem. In addition, from the viewpoint of information theory, we identify fundamental open problems of characterizing rate regions for communication networks supporting distributed learning-and-computing tasks. We also present technical challenges as well as emerging application opportunities in this field, with the aim of inspiring future research and promoting widespread developments of EL in B5G.

Sentiment Analysis and Classification Using Deep Semantic Information and Contextual Knowledge

Sentiment analysis (AS) is one of the basic research directions in natural language processing (NLP), it is widely adopted for news, product review, and politics. Aspect-based sentiment analysis (ABSA) aims at identifying the sentiment polarity of a given target context, previous existing model of sentiment analysis possesses the issue of the insufficient exaction of features which results in low accuracy. Hence this research work develops a deep-semantic and contextual knowledge networks (DSCNet). DSCNet tends to exploit the semantic and contextual knowledge to understand the context and enhance the accuracy based on given aspects. At first temporal relationships are established then deep semantic knowledge and contextual knowledge are introduced. Further, a deep integration layer is introduced to measure the importance of features for efficient extraction of different dimensions. Novelty of DSCNet model lies in introducing the deep contextual. DSCNet is evaluated on three datasets i.e., Restaurant, Laptop, and Twitter dataset considering different deep learning (DL) metrics like precision, recall, accuracy, and Macro-F1 score. Also, comparative analysis is carried out with different baseline methods in terms of accuracy and Macro-F1 score. DSCNet achieves 92.59% of accuracy on restaurant dataset, 86.99% of accuracy on laptop dataset and 78.76% of accuracy on Twitter dataset.

Hydrophobicity identifies false positives and false negatives in peptide-MHC binding.

Major Histocompability Complex (MHC) Class I molecules allow cells to present foreign and endogenous peptides to T-Cells so that cells infected by pathogens can be identified and killed. Neural networks tools such as NetMHC-4.0 and NetMHCpan-4.1 are used to predict whether peptides will bind to variants of MHC molecules. These tools are trained on data gathered from binding affinity and eluted ligand experiments. However, these tools do not track hydrophobicity, a significant biochemical factor relevant to peptide binding, in their predictions. A previous study had concluded that the peptides predicted to bind to HLA-A*0201 by NetMHC-4.0 were much more hydrophobic than expected. This paper expands that study by also focusing on HLA-B*2705 and HLA-B*0801, which prefer binding hydrophilic and balanced peptides respectively. The correlation of hydrophobicity of 9-mer peptides with their predicted binding strengths to these various HLAs was investigated. Two studies were performed, one using the data that the two neural networks were trained on, and the other using a sample of the human proteome. NetMHC-4.0 was found to have a statistically significant bias towards predicting highly hydrophobic peptides as strong binders to HLA-A*0201 and HLA-B*2705 in both studies. Machine Learning metrics were used to identify the causes for this bias: hydrophobic false positives and hydrophilic false negatives. These results suggest that the retraining the neural networks with biochemical attributes such as hydrophobicity and better training data could increase the accuracy of their predictions. This would increase their impact in applications such as vaccine design and neoantigen identification.

The politics of good enough data. Developments, dilemmas and deadlocks in the production of global learning metrics

The indicator framework established for the monitoring of SDG4 is unambiguous on the need to advance in the production of global learning data. However, the production of SDG4 learning metrics has been riddled with technical and political difficulties. Drawing on a combination of documentary analysis, interviews and non-participant observation, this paper reconstructs the process of negotiation of the data suppliers, statistical routines and reporting standards necessary for the production of SDG4 learning metrics. The paper thus offers insight into the mechanics of global quantification, and on the transformative impact of such processes on the agendas and relationships of partaking organizations.

The Concept of Applied Strategic Analysis of Learning and Growth

Learning and growth is one of the essential tools that companies ought to use in today's market economy in order to ensure their sustainable development in the long-term and, consequently, to achieve their maximum market value. This article is predominantly of theoretical nature and considers the possibility of using the applied strategic analysis technique that was developed by the author to analyze learning and growth in the course of a study of strategic aspects of learning and growth. The technique is based on the balanced scorecard system of the same name. Methodologically, the study is based on Kaplan and Norton's balanced scorecard concept and the author's concept of applied strategic analysis. Findings – It is shown that applied strategic analysis of learning and growth envisages comparative evaluation, diagnostics of deviations and predictions of the values of the balanced scorecard metrics of learning and growth in the dimension of strategic objectives. It includes analysis of the level of motivation in employees, the volume of authority delegation and whether their personal goals match company goals; analysis of the expansion capacity of information systems, and analysis of the quality of retraining and the creativity level of employees. The applied strategic analysis of learning and growth with a comparative evaluation of the metrics of the level of motivation in employees, the volume of delegation to them and of the match between their personal and corporate goals, and concludes with a prediction of the values of factor measures of the quality of retraining and creativity development in employees. The author comes to the conclusion that applied strategic analysis of learning and growth is a new and sufficiently effective instrument for studying the strategic aspects of companies' performance that is associated with the training and development of their personnel. The instrument provides analytical support to strategic management as regards personnel training and development in today's conditions. Its results might be useful when working out managerial decisions for the company's short-term, medium-term and long-term plans in the area of learning and growth.

Multiclass spatial predictions of borehole yield in southern Mali by means of machine learning classifiers

Study regionRegions of Bamako, Kati and Kangaba, southwestern Mali Study focusMachine learning-based mapping of borehole yield. Three algorithms were trained on an imbalanced multiclass database of boreholes, while twenty variables were used as predictors for borehole yield. All models returned balanced and geometric scores in the order of 0.80, with area under the receiver operating characteristic curve up to 0.87. Three main methodological conclusions are drawn: (a) The evaluation of different machine learning classifiers and various resampling strategies and the subsequent selection of the best performing ones is shown to be a good strategy in this type of studies; (b) ad hoc calibration tools, such as data on borehole success rates, provide an apt complement to standard machine learning metrics; and (c) a multiclass approach with an unbalanced database represents a greater challenge than predicting a bivariate outcome, but potentially results in a finer depiction of field conditions. New hydrological insights for the regionAlluvial sediments were found to be the most productive areas, while the Mandingue Plateau has the lowest groundwater potential. The piedmont areas showcase an intermediate groundwater prospect. Elevation, basement depth, slope and geology rank among the most important variables. Lower values of clay content, slopes and elevations, and higher values of basement depth and saturated thickness were linked to the most productive class.

Measuring the effects of course modularizations in online courses for life-long learners

Many participants in Massive Open Online Courses are full-time employees seeking greater flexibility in their time commitment and the available learning paths. We recently addressed these requirements by splitting up our 6-week courses into three 2-week modules followed by a separate exam. Modularizing courses offers many advantages: Shorter modules are more sustainable and can be combined, reused, and incorporated into learning paths more easily. Time flexibility for learners is also improved as exams can now be offered multiple times per year, while the learning content is available independently. In this article, we answer the question of which impact this modularization has on key learning metrics, such as course completion rates, learning success, and no-show rates. Furthermore, we investigate the influence of longer breaks between modules on these metrics. According to our analysis, course modules facilitate more selective learning behaviors that encourage learners to focus on topics they are the most interested in. At the same time, participation in overarching exams across all modules seems to be less appealing compared to an integrated exam of a 6-week course. While breaks between the modules increase the distinctive appearance of individual modules, a break before the final exam further reduces initial interest in the exams. We further reveal that participation in self-paced courses as a preparation for the final exam is unlikely to attract new learners to the course offerings, even though learners' performance is comparable to instructor-paced courses. The results of our long-term study on course modularization provide a solid foundation for future research and enable educators to make informed decisions about the design of their courses.

Clustering of trauma patients based on longitudinal data and the application of machine learning to predict recovery

Predicting recovery after trauma is important to provide patients a perspective on their estimated future health, to engage in shared decision making and target interventions to relevant patient groups. In the present study, several unsupervised techniques are employed to cluster patients based on longitudinal recovery profiles. Subsequently, these data-driven clusters were assessed on clinical validity by experts and used as targets in supervised machine learning models. We present a formalised analysis of the obtained clusters that incorporates evaluation of (i) statistical and machine learning metrics, (ii) clusters clinical validity with descriptive statistics and medical expertise. Clusters quality assessment revealed that clusters obtained through a Bayesian method (High Dimensional Supervised Classification and Clustering) and a Deep Gaussian Mixture model, in combination with oversampling and a Random Forest for supervised learning of the cluster assignments provided among the most clinically sensible partitioning of patients. Other methods that obtained higher classification accuracy suffered from cluster solutions with large majority classes or clinically less sensible classes. Models that used just physical or a mix of physical and psychological outcomes proved to be among the most sensible, suggesting that clustering on psychological outcomes alone yields recovery profiles that do not conform to known risk factors.

Comparison of Machine Learning Algorithms Used for Skin Cancer Diagnosis

The paper presents a comparison of automatic skin cancer diagnosis algorithms based on analyses of skin lesions photos. Two approaches are presented: the first one is based on the extraction of features from images using simple feature descriptors, and then the use of selected machine learning algorithms for the purpose of classification, and the second approach uses selected algorithms belonging to the subgroup of machine learning—deep learning, i.e., convolutional neural networks (CNN), which perform both the feature extraction and classification in one algorithm. The following algorithms were analyzed and compared: Logistic Regression, k-Nearest Neighbors, Naive Bayes, Decision Tree, Random Forest, and Support Vector Machine, and four CNN–VGG-16, ResNet60, InceptionV3, and Inception-ResNetV2 In the first variant, before the classification process, the image features were extracted using 4 different feature descriptors and combined in various combinations in order to obtain the most accurate image features vector, and thus the highest classification accuracy. The presented approaches have been validated using the image dataset from the ISIC database, which includes data from two categories—benign and malignant skin lesions. Common machine learning metrics and saved values of training time were used to evaluate the effectiveness and the performance (computational complexity) of the algorithms.

NNCD-IQA: A new neural networks based compressed database for image quality assessment

Objective and subjective quality assessment is still a challenging problem in various image processing tasks. For instance, in the context of image compression, most of the conducted studies have focused on image datasets encoded using standard algorithms such as JPEG and JPEG2000. In this paper, we propose to further investigate the quality assessment issue in the presence of neural networks-based compressed images. More precisely, a new database of compressed images has been firstly built using JPEG2000 standard as well as four recent neural networks based coding schemes. Then, subjective experiments are performed to obtain the mean opinion scores of the generated distorted images. Finally, an extensive evaluation and analysis of objective image quality assessment metrics is achieved. For instance, in addition to conventional and machine learning metrics, we have considered different deep learning based models, which have been trained on our database. The new subjective database with its associated mean opinion scores as well as the learned models are publicly available at https://github.com/zakopz/NNCD-IQA-Database . The obtained results show the interest of deep learning based metrics in the context of neural networks-based compressed images.

Sojourn Time Minimization of Successful Jobs

Due to a growing interest in deep learning applications [5], compute-intensive and long-running (hours to days) training jobs have become a significant component of datacenter workloads. A large fraction of these jobs is often exploratory, with the goal of determining the best model structure (e.g., the number of layers and channels in a convolutional neural network), hyperparameters (e.g., the learning rate), and data augmentation strategies for the target application. Notably, training jobs are often terminated early if their learning metrics (e.g., training and validation accuracy) are not converging, with only a few completing successfully. For this motivating application, we consider the problem of scheduling a set of jobs that can be terminated at predetermined checkpoints with known probabilities estimated from historical data. We prove that, in order to minimize the time to complete the first K successful jobs on a single server, optimal scheduling does not require preemption (even when preemption overhead is negligible) and provide an optimal policy; advantages of this policy are quantified through simulation. Related Work. While job scheduling has been investigated extensively in many scenarios (see [6] and [2] for a survey of recent result), most policies require that the cost of waiting times of each job be known at scheduling time; in contrast, in our setting the scheduler does not know which job will be the K-th successful job, and sojourn times of subsequent jobs do not contribute to the target metric. For example, [4, 3] minimize makespan (i.e., the time to complete all jobs) for known execution times and waiting time costs; similarly, Gittins index [1] and SR rank [7] minimize expected sojourn time of all jobs, i.e., both successfully completed jobs and jobs terminated early. Unfortunately, scheduling policies not distinguishing between these two types of jobs may favor jobs where the next stage is short and leads to early termination with high probability, which is an undesirable outcome in our applications of interest.

Global Citizenship Competencies of Filipino Students: Using Machine Learning to Explore the Structure of Cognitive, Affective, and Behavioral Competencies in the 2019 Southeast Asia Primary Learning Metrics

While the Philippines is still building its global citizenship curriculum, there are global citizenship competencies already articulated in existing curriculum guides. Using data from a nationally representative sample of Grade 5 students in the Southeast Asia Primary Learning Metrics (SEA-PLM) assessment, we explored Filipino learners’ current global competencies. We used machine learning approaches to determine the best models to predict the six SEA-PLM global competency indices; models generated by Multilayer Perceptrons performed better than other techniques. Shapley Additive Explanations approach was applied to identify variables that had the most impact on the model of each global competency index. Some variables were important predictors across the indices: concern about pollution, feeling connected to people from other countries, beliefs about the importance of learning about other countries, how countries relate to each other, and how natural disasters in other countries affect the Philippines are variables that were associated with global competency indices. Willingness to participate in classroom debates also positively predicted the indices but willingness to participate in classroom elections negatively predicted indices related to knowledge and behavior intention indices. We discuss how patterns in Filipino students’ emerging global competencies can guide curriculum development.

Neural network approximations for Calabi-Yau metrics

Ricci flat metrics for Calabi-Yau threefolds are not known analytically. In this work, we employ techniques from machine learning to deduce numerical flat metrics for K3, the Fermat quintic, and the Dwork quintic. This investigation employs a simple, modular neural network architecture that is capable of approximating Ricci flat Kähler metrics for Calabi-Yau manifolds of dimensions two and three. We show that measures that assess the Ricci flatness and consistency of the metric decrease after training. This improvement is corroborated by the performance of the trained network on an independent validation set. Finally, we demonstrate the consistency of the learnt metric by showing that it is invariant under the discrete symmetries it is expected to possess.

Development and validation of comprehensive clinical outcome prediction models for acute ischaemic stroke in anterior circulation based on machine learning

The current prediction models for the clinical outcome of acute ischaemic stroke (AIS) remain insufficient for individualized patient management strategies. We aimed to investigate machine learning (ML) performance in the clinical outcome prediction of AIS in anterior circulation and evaluate the clinical outcome by combining the quantitative evaluation indicators of perfusion features and basic clinical information. Four ML classifiers, support vector machine (SVM), naive Bayes (NB), logistic regression (LR), and random forest (RF) were trained on a cohort of 389 adult patients (training cohort [70 %]; external validation cohort [30 %]) from the Acute Stroke Center Registry of Huashan Hospital. Model performance was compared by a range of learning metrics. Most imaging parameters were strongly correlated with the outcome (range, 0.57 to 0.81), and the correlation between relative cerebral blood flow (rCBF) < 30 % and clinical outcome was the strongest (ρ = 0.81). As the reference parameters increased, the performance of the four models was greatly improved [SVM (AUC: from 0.79 to 0.99, F1-score: from 0.61 to 0.90), RF (AUC: from 0.88 to 0.98, F1-score: from 0.71 to 0.96), LR (AUC: from 0.80 to 0.97, F1-score: from 0.64 to 0.95), and NB (AUC: from 0.74 to 0.97, F1-score: from 0.66 to 0.92)]. The ensemble classifier model with all parameters had the highest F1-score (0.97). All the ML models, jointly considering the basic clinical information and quantitative evaluation indicators of computed tomography perfusion (CTP), showed good performance in the prediction of clinical outcome of AIS in anterior circulation.

A framework to enhance generalization of deep metric learning methods using general discriminative feature learning and class adversarial neural networks

Deep Metric Learning (DML) methods automatically extract features from data and learn a non-linear transformation from the input to a semantically embedding space. Many DML methods focused to enhance the discrimination power of the learned metric by proposing novel sampling strategies or loss functions. This approach is very helpful when both the training and test examples are selected from the same set of categories. However, it is less effective in many applications of DML such as image retrieval and person-reidentification. Here, the DML should learn general semantic concepts from observed classes and employ them to rank or identify objects from unseen categories. Neglecting the generalization ability of the learned representation and just emphasizing to learn a more discriminative embedding on the observed classes may lead to the overfitting problem. To address this limitation, we propose a framework to enhance the generalization power of existing DML methods in a Zero-Shot Learning (ZSL) setting by general yet discriminative representation learning and employing a class adversarial neural network. To learn a general representation, we employ feature maps of intermediate layers in a deep neural network and enhance their discrimination power through an attention mechanism. Besides, a class adversarial network is utilized to force the deep model to seek class invariant features. We evaluate our work on widely used machine vision datasets in a ZSL setting. Extensive experimental results confirm that our framework can improve the generalization of existing DML methods, and it consistently outperforms baseline DML algorithms on unseen classes.

Behavioral metrics of reward learning in nonhuman primates provide a framework for human neurophysiological recordings

Behavioral metrics of reward learning in nonhuman primates provide a framework for human neurophysiological recordings