Context Of Machine Learning Research Articles

Deep Neural Network-Aided Soft-Demapping in Coherent Optical Systems: Regression Versus Classification

We examine here what type of predictive modelling, classification, or regression, using neural networks (NN), fits better the task of soft-demapping based post-processing in coherent optical communications, where the transmission channel is nonlinear and dispersive. For the first time, we present possible drawbacks in using each type of predictive task in a machine learning context, considering the nonlinear coherent optical channel equalization/soft-demapping problem. We study two types of equalizers based on the feed-forward and recurrent NNs, for several transmission scenarios, in linear and nonlinear regimes of the optical channel. We point out that even though from the information theory perspective the cross-entropy loss (classification) is the most suitable option for our problem, the NN models based on the cross-entropy loss function can severely suffer from learning problems. The latter translates into the fact that regression-based learning is typically superior in terms of delivering higher Q-factor and achievable information rates. In short, we show by empirical evidence that loss functions based on cross-entropy may not be necessarily the most suitable option for training communication systems in practical scenarios when overfitting- and vanishing gradients-related problems come into play.

A Fuzzy Approach to Drum Cymbals Classification

The many factors that influence the sound of a cymbal, combined with the external aspects that modify its sound perception, make the study of the acoustics of these instruments more challenging. Within the context of machine learning, most researches involving cymbal classification from their sounds aim to identify those instruments according to their types. However, there is a lack of studies investigating the acoustic elements of cymbals using machine learning techniques as tools. Hence, this paper proposes to classify cymbals according to their constitutive materials, since the metallic alloy assumes a significant portion is responsible for their acoustics. In addition, there is an interest in evaluating a fuzzy logic approach as a classifier applied to three sets of attributes, formed from temporal features and Mel Frequency Cepstral Coefficients extracted from audio signals, comparing triangular and Gaussian membership functions. For this, 276 audios, referring to 4 drum cymbals, were collected from a standardized procedure for capturing the sounds that considered variations in microphones and environments. As a result, the implemented model achieved 94.72% of average accuracy with a standard deviation of 2.51%, considering the Gaussian membership function and Mel Frequency Cepstral Coefficients as the audio descriptor.

Search-based fairness testing for regression-based machine learning systems

ContextMachine learning (ML) software systems are permeating many aspects of our life, such as healthcare, transportation, banking, and recruitment. These systems are trained with data that is often biased, resulting in biased behaviour. To address this issue, fairness testing approaches have been proposed to test ML systems for fairness, which predominantly focus on assessing classification-based ML systems. These methods are not applicable to regression-based systems, for example, they do not quantify the magnitude of the disparity in predicted outcomes, which we identify as important in the context of regression-based ML systems.Method:We conduct this study as design science research. We identify the problem instance in the context of emergency department (ED) wait-time prediction. In this paper, we develop an effective and efficient fairness testing approach to evaluate the fairness of regression-based ML systems. We propose fairness degree, which is a new fairness measure for regression-based ML systems, and a novel search-based fairness testing (SBFT) approach for testing regression-based machine learning systems. We apply the proposed solutions to ED wait-time prediction software.Results:We experimentally evaluate the effectiveness and efficiency of the proposed approach with ML systems trained on real observational data from the healthcare domain. We demonstrate that SBFT significantly outperforms existing fairness testing approaches, with up to 111% and 190% increase in effectiveness and efficiency of SBFT compared to the best performing existing approaches.Conclusion:These findings indicate that our novel fairness measure and the new approach for fairness testing of regression-based ML systems can identify the degree of fairness in predictions, which can help software teams to make data-informed decisions about whether such software systems are ready to deploy. The scientific knowledge gained from our work can be phrased as a technological rule; to measure the fairness of the regression-based ML systems in the context of emergency department wait-time prediction use fairness degree and search-based techniques to approximate it.

Moral dilemmas for moral machines

Autonomous systems are being developed and deployed in situations that may require some degree of ethical decision-making ability. As a result, research in machine ethics has proliferated in recent years. This work has included using moral dilemmas as validation mechanisms for implementing decision-making algorithms in ethically-loaded situations. Using trolley-style problems in the context of autonomous vehicles as a case study, I argue (1) that this is a misapplication of philosophical thought experiments because (2) it fails to appreciate the purpose of moral dilemmas, and (3) this has potentially catastrophic consequences; however, (4) there are uses of moral dilemmas in machine ethics that are appropriate and the novel situations that arise in a machine-learning context can shed some light on philosophical work in ethics.

Manifolds of quasi-constant SOAP and ACSF fingerprints and the resulting failure to machine learn four-body interactions.

Atomic fingerprints are commonly used for the characterization of local environments of atoms in machine learning and other contexts. In this work, we study the behavior of two widely used fingerprints, namely, the smooth overlap of atomic positions (SOAP) and the atom-centered symmetry functions (ACSFs), under finite changes of atomic positions and demonstrate the existence of manifolds of quasi-constant fingerprints. These manifolds are found numerically by following eigenvectors of the sensitivity matrix with quasi-zero eigenvalues. The existence of such manifolds in ACSF and SOAP causes a failure to machine learn four-body interactions, such as torsional energies that are part of standard force fields. No such manifolds can be found for the overlap matrix (OM) fingerprint due to its intrinsic many-body character.

The Markov Random Field in Materials Applications: A synoptic view for signal processing and materials readers

The Markov random field (MRF) is one of the most widely used models in image processing, constituting a prior model for addressing problems such as image segmentation, object detection, and reconstruction. What is not often appreciated is that the MRF owes its origin to the physics of solids, making it an ideal prior model for processing microscopic observations of materials. While both fields know of their respective interpretations of the MRF, each knows very little about the other’s version of it. Hence, both fields have “blind spots,” where some concepts readily appreciated by one field are completely obscured from the other. With this in mind, the objectives of this article are to 1) develop a synoptic view of the MRF, the related Gibbs distribution, and the Hammersley–Clifford theorem that links them, in such a way that signal processing and materials readers will see them from the same perspective; and 2) explain physics-based regularization using the MRF and describe how it can provide insight into the performance of MRF-based segmentation methods. While the MRF has already been used in many machine learning contexts, we will use a simpler, more transparent method to illustrate the fundamental behavior of the MRF, with the understanding that this behavior should be inherent in more complex learning approaches.

MACHINE LEARNING IN THE FIELD OF MANUFACTURING

Artificial intelligence is already a pulsating reality in society, and all major companies are trying to apply its use to their products. In this context, the manufacturing sector could not be left behind. The industry has more and more alternatives to choose from when it comes to implementing artificial intelligence techniques. In this aspect, Machine Learning can be of great use when it comes to solving the challenges that arise in manufacturing. Therefore, this article presents a brief context of Machine Learning, followed by its current applications in the manufacturing sector together with a historical evolutionary analysis of the publications related to Machine Learning in the manufacturing sector. It ends with some conclusions regarding the current state of Machine Learning in the scientific community and in companies. Keywords: Machine Learning, Review, Manufacturing.

Dense prediction of label noise for learning building extraction from aerial drone imagery

ABSTRACT Label noise is a commonly encountered problem in learning building extraction tasks; its presence can reduce performance and increase learning complexity. This is especially true for cases where high resolution aerial drone imagery is used, as the labels may not perfectly correspond/align with the actual objects in the imagery. In general machine learning and computer vision context, labels refer to the associated class of data, and in remote sensing-based building extraction refer to pixel-level classes. Dense label noise in building extraction tasks has rarely been formalized and assessed. We formulate a taxonomy of label noise models for building extraction tasks, which incorporates both pixel-wise and dense models. While learning dense prediction under label noise, the differences between the ground truth clean label and observed noisy label can be encoded by error matrices indicating locations and type of noisy pixel-level labels. In this work, we explicitly learn to approximate error matrices for improving building extraction performance; essentially, learning dense prediction of label noise as a subtask of a larger building extraction task. We propose two new model frameworks for learning building extraction under dense real-world label noise, and consequently two new network architectures, which approximate the error matrices as intermediate predictions. The first model learns the general error matrix as an intermediate step and the second model learns the false positive and false-negative error matrices independently, as intermediate steps. Approximating intermediate error matrices can generate label noise saliency maps, for identifying labels having higher chances of being mis-labelled. We have used ultra-high-resolution aerial images, noisy observed labels from OpenStreetMap, and clean labels obtained after careful annotation by the authors. When compared to the baseline model trained and tested using clean labels, our intermediate false positive-false negative error matrix model provides Intersection-Over-Union gain of 2.74% and F1-score gain of 1.75% on the independent test set. Furthermore, our proposed models provide much higher recall than currently used deep learning models for building extraction, while providing comparable precision. We show that intermediate false positive-false negative error matrix approximation can improve performance under label noise.

On the classification of simple and complex biological images using Krawtchouk moments and Generalized pseudo-Zernike moments: a case study with fly wing images and breast cancer mammograms.

In image analysis, orthogonal moments are useful mathematical transformations for creating new features from digital images. Moreover, orthogonal moment invariants produce image features that are resistant to translation, rotation, and scaling operations. Here, we show the result of a case study in biological image analysis to help researchers judge the potential efficacy of image features derived from orthogonal moments in a machine learning context. In taxonomic classification of forensically important flies from the Sarcophagidae and the Calliphoridae family (n = 74), we found the GUIDE random forests model was able to completely classify samples from 15 different species correctly based on Krawtchouk moment invariant features generated from fly wing images, with zero out-of-bag error probability. For the more challenging problem of classifying breast masses based solely on digital mammograms from the CBIS-DDSM database (n = 1,151), we found that image features generated from the Generalized pseudo-Zernike moments and the Krawtchouk moments only enabled the GUIDE kernel model to achieve modest classification performance. However, using the predicted probability of malignancy from GUIDE as a feature together with five expert features resulted in a reasonably good model that has mean sensitivity of 85%, mean specificity of 61%, and mean accuracy of 70%. We conclude that orthogonal moments have high potential as informative image features in taxonomic classification problems where the patterns of biological variations are not overly complex. For more complicated and heterogeneous patterns of biological variations such as those present in medical images, relying on orthogonal moments alone to reach strong classification performance is unrealistic, but integrating prediction result using them with carefully selected expert features may still produce reasonably good prediction models.

A machine learning based asset pricing factor model comparison on anomaly portfolios

We frame asset pricing linear factor models in a machine learning context and consider related comparisons of their predictive performance against ordinary least squares linear regression over a dataset of anomaly portfolios. Specific regression models involved in the comparison include regularized linear, support vector machines, neural networks, and tree based models among others. Performance metrics are presented on a model, portfolio group, and sequential basis, and the strongest predictors are recommended as alternative techniques for the problem of excess return forecasting.

Entropy, cross-entropy, relative entropy: Deformation theory (a)

Attempts at generalizing Shannon entropy and Kullback-Leibler divergence (relative entropy) led to a plenthora of deformation models in theoretical physics, including q-model, κ-model, etc. Naudts and Zhang (Inf. Geom., 1 (2018) 79) established that these models can be unified under two notions: deformed ϕ-exponential family (Naudts, J., J. Inequal. Pure Appl. Math., 5 (2004) 102) and conjugate -embedding (Zhang J., Neural Comput., 16 (2004) 159) of probability functions. Conjugate -embedding has a gauge freedom which, upon its fixing, subsumes the U-model of Eguchi (Sugaku Expositions, 19 (2006) 197) proposed in a statistical machine learning context. The generalization by -entropy, -cross-entropy, -divergence, when applied to the ϕ-exponential family, yields either a Hessian structure or a conformal Hessian structure under different gauge selections —this “splitting” is the hallmark when deforming the exponential family with its dually flat (Hessian) geometry. This letter provides a unified information geometric perspective of deformation of the exponential model, with calculations for Tsallis q-model.

A MACHINE LEARNING BASED ASSET PRICING FACTOR MODEL EXTENSION COMPARISON ON ANOMALY PORTFOLIOS

We frame linear factor models for asset pricing in a machine learning context and consider a numerical comparison of their performance against ordinary least squares linear regression over a dataset of anomaly portfolios. Specific regression models involved in the comparison include regularized linear, support vector machines, neural networks, and tree based models. Performance metrics are presented on a model and portfolio group basis, and the strongest predictors are recommended as alternative methods for the problem of excess return forecasting.

Response-adaptive trial designs with accelerated Thompson sampling.

In a clinical trial, sometimes it is desirable to allocate as many patients as possible to the best treatment, in particular, when a trial for a rare disease may contain a considerable portion of the whole target population. The Gittins index rule is a powerful tool for sequentially allocating patients to the best treatment based on the responses of patients already treated. However, its application in clinical trials is limited due to technical complexity and lack of randomness. Thompson sampling is an appealing approach, since it makes a compromise between optimal treatment allocation and randomness with some desirable optimal properties in the machine learning context. However, in clinical trial settings, multiple simulation studies have shown disappointing results with Thompson samplers. We consider how to improve short-run performance of Thompson sampling and propose a novel acceleration approach. This approach can also be applied to situations when patients can only be allocated by batch and is very easy to implement without using complex algorithms. A simulation study showed that this approach could improve the performance of Thompson sampling in terms of average total response rate. An application to a redesign of a preference trial to maximize patient's satisfaction is also presented.

Item response theory as a feature selection and interpretation tool in the context of machine learning.

Optimizing the number and utility of features to use in a classification analysis has been the subject of many research studies. Most current models use end-classifications as part of the feature reduction process, leading to circularity in the methodology. The approach demonstrated in the present research uses item response theory (IRT) to select features independent of the end-classification results without the biased accuracies that this circularity engenders. Dichotomous and polytomous IRT models were used to analyze 30 histological breast cancer features from 569 patients using the Wisconsin Diagnostic Breast Cancer data set. Based on their characteristics, three features were selected for use in a machine learning classifier. For comparison purposes, two machine learning-based feature selection protocols were run-recursive feature elimination (RFE) and ridge regression-and the three features selected from these analyses were also used in the subsequent learning classifier. Classification results demonstrated that all three selection processes performed comparably. The non-biased nature of the IRT protocol and information provided about the specific characteristics of the features as to why they are of use in classification help to shed light on understanding which attributes of features make them suitable for use in a machine learning context.

Annotating affective dimensions in user-generated content

In an era where user-generated content becomes ever more prevalent, reliable methods to judge emotional properties of these kinds of complex texts are needed, for example for developing corpora in machine learning contexts. In this study, we focus on Dutch Twitter messages, a genre which is high in emotional content and frequently investigated in the field of computational linguistics. We compare three annotation methods to annotate the emotional dimensions valence, arousal and dominance in 300 Tweets, namely rating scales, pairwise comparison and best–worst scaling. We evaluate the annotation methods on the criterion of inter-annotator agreement, based on judgments of 18 annotators in total. On this dataset, best–worst scaling has the highest inter-annotator agreement. We find that the difference in agreement is largest for dominance and smallest for valence, suggesting that the benefit of best–worst scaling becomes more pronounced as the annotation task gets more difficult. However, we also find that best–worst scaling is particularly more time-consuming than are rating scale and pairwise comparison annotations. This leads us to conclude that, in particular when dealing with computational models, a comparative assessment of quality versus costs needs to be made.

Functional analysis of generalized linear models under non-linear constraints with applications to identifying highly-cited papers

This article introduces a versatile functional form for Generalized Linear Models (GLMs) through a simple, yet effective, transformation of the current framework. The models are applied through a new hierarchical bayesian estimation procedure for logistic regression to highly-cited papers in the Management Information Systems (MIS) field. The results are uniformly better, in regards to model fit and inference for in-sample and out-of-sample data, for simulation studies and real-world data applications, requiring very little time to convergence to true population parameters. In simulation studies, I show that the method contains the true parameters nearly three times as often as widely used existing GLMs, and does so while having confidence intervals that are 54.50% smaller, while requiring around two-thirds the number of MCMC iterations as existing bayesian methods. In Scientometric applications the methodology is shown to be highly robust with predictive/classification accuracy, either equaling or exceeding existing methods for identifying highly-cited articles including Artificial Neural Networks (ANN). Thus, the method is shown to be robust to the amount of asymmetry (or symmetry) of the probability of success (or failure) and robust to unbalanced samples and varying Data Generating Processes. Further, the methodology is equivalent to current methods if the data support them and is therefore complementary to existing methods, without loss of interpretability of model parameters. For the MIS field it finds that Popularity Parameter (PP) of an article Keywords can predict whether a paper will be highly-cited (top 25% of highly-cited articles) between two to three years after publication and beyond. Furthermore, given the small number of iterations needed for convergence, the methodology can also be used as a baseline method in Big Data (BD) settings for both Artificial Intelligence (AI) and Machine Learning (ML) contexts as well.

Reaction-based machine learning representations for predicting the enantioselectivity of organocatalysts.

Hundreds of catalytic methods are developed each year to meet the demand for high-purity chiral compounds. The computational design of enantioselective organocatalysts remains a significant challenge, as catalysts are typically discovered through experimental screening. Recent advances in combining quantum chemical computations and machine learning (ML) hold great potential to propel the next leap forward in asymmetric catalysis. Within the context of quantum chemical machine learning (QML, or atomistic ML), the ML representations used to encode the three-dimensional structure of molecules and evaluate their similarity cannot easily capture the subtle energy differences that govern enantioselectivity. Here, we present a general strategy for improving molecular representations within an atomistic machine learning model to predict the DFT-computed enantiomeric excess of asymmetric propargylation organocatalysts solely from the structure of catalytic cycle intermediates. Mean absolute errors as low as 0.25 kcal mol−1 were achieved in predictions of the activation energy with respect to DFT computations. By virtue of its design, this strategy is generalisable to other ML models, to experimental data and to any catalytic asymmetric reaction, enabling the rapid screening of structurally diverse organocatalysts from available structural information.

Neutrosophic-based machine learning context for the trustworthiness of devices in the internet of things

The industrial sector is among the most suited sectors that may considerably advantage from the implementation of the ideas and technology of the Industrial Internet of Things (IIoT), and it is one of the most competitive industries in the world. The increased use of automated processes in manufacturing sectors results in a wide variety of applications based on IIoT. These applications call for the efficient integration of a wide variety of different systems and the execution of smooth operations across all machines. The issue of integration and smooth operation presents IIoT as a new subject of study in smart manufacturing. This carries with it several problems, including those on security, accountability, confidence, and dependability. As part of the Industrial Internet of Things (IIoT), many devices will be linked to one another and interact with one another through wireless and internet infrastructure. When this kind of situation plays out, the reliability of the IIoT devices becomes a key component in the process of preventing injection by hostile machines. As a result, an intelligent computer model is required to effectively cluster and categorize the level of trustworthiness possessed by the IIoT devices. In this article, we describe a trust model for the Internet of Things (IIoT) that is based on the neutrosophic TOPSIS and is utilized by IIoT apps to determine the trust score of IIoT devices. The reliability of devices is evaluated by the model that was constructed using the historical knowledge, chronological knowledge, and network behavior information that is received from IIoT devices. In addition to that, the model suggests KNN, and a Decision tree to categorize the attributes that were collected.

Beyond kappa: an informational index for diagnostic agreement in dichotomous and multivalue ordered-categorical ratings

Agreement measures are useful tools to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. Cohen’s kappa (κ) certainly is the most popular agreement method between two raters, and proved its effectiveness in the last sixty years. In spite of that, this method suffers from some alleged issues, which have been highlighted since the 1970s; moreover, its value is strongly dependent on the prevalence of the disease in the considered sample. This work introduces a new agreement index, the informational agreement (IA), which seems to avoid some of Cohen’s kappa’s flaws, and separates the contribution of the prevalence from the nucleus of agreement. These goals are achieved by modelling the agreement—in both dichotomous and multivalue ordered-categorical cases—as the information shared between two raters through the virtual diagnostic channel connecting them: the more information exchanged between the raters, the higher their agreement. In order to test its fair behaviour and the effectiveness of the method, IA has been tested on some cases known to be problematic for κ, in the machine learning context and in a clinical scenario to compare ultrasound (US) and automated breast volume scanner (ABVS) in the setting of breast cancer imaging.Graphical To evaluate the agreement between the two raters mathfrak {X} and mathfrak {Y} we create an agreement channel, based on Shannon Information Theory, that directly connects the random variables X and Y, that express the raters outcomes. They are the terminals of the chain X⇔ diagnostic test performed bymathfrak {X} ⇔ patient conditionmathfrak {D} ⇔ diagnostic test performed bymathfrak {Y} ⇔ Y

Adaptive context‐aware service optimization in mobile cloud computing accounting for security aspects

SummaryIn this article, we present an original agent‐based adaptive task scheduling system which optimizes the performance of services in the mobile cloud computing environment using machine learning mechanisms and context information. The system learns how to allocate resources appropriately: how to schedule services/tasks optimally between the mobile device and the cloud. Decisions are made taking into account the context (e.g., network connection type, location, security level). In this study, a supervised learning agent architecture and service selection algorithm are proposed to solve this problem. Adaptation is performed online on a mobile device. To verify the solution proposed, appropriate software has been developed and a series of experiments has been conducted. Results demonstrate that owing to the experience gathered and the learning process performed, the decision module becomes more efficient in assigning the task to either the mobile device or cloud resources. In the face of presented improvements, the security issues inherent in the context of mobile services/applications and cloud computing are further discussed. As threats associated with mobile data offloading are a serious concern, often ruling out the utilization of cloud services, we propose a more security focused approach for our solution, preferably without hindering performance.