Supervised Learning Procedure Research Articles

Anomaly Detection in Hobbing Tool Images: Using An Unsupervised Deep Learning Approach in Manufacturing Industry

This study explores the application of the PatchCore algorithm for anomaly classification in hobbing tools, an area of keen interest in industrial artificial intelligence application. Despite utilizing limited training images, the algorithm demonstrates capability in recognizing a variety of anomalies, promising to reduce the time-intensive labeling process traditionally undertaken by domain experts. The algorithm demonstrated an accuracy of 92%, precision of 84%, recall of 100%, and a balanced F1 score of 91%, showcasing its proficiency in identifying anomalies. However, the investigation also highlights that while the algorithm effectively identifies anomalies, it doesn't primarily recognize domain-specific wear issues. Thus, the presented approach is used only for pre-classification, with domain experts subsequently segmenting the images indicating significant wear. The intention is to employ a supervised learning procedure to identify actual wear. This premise will be further investigated in future research studies.

An Integrated Model of Collaborative Skill Acquisition: Anticipation, Control Tuning, and Role Adoption.

We studied collaborative skill acquisition in a dynamic setting with the game Co-op Space Fortress. While gaining expertise, the majority of subjects became increasingly consistent in the role they adopted without being able to communicate. Moreover, they acted in anticipation of the future task state. We constructed a collaborative skill acquisition model in the cognitive architecture ACT-R that reproduced subject skill acquisition trajectory. It modeled role adoption through reinforcement learning and predictive processes through motion extrapolation and learned relevant control parameters using both a reinforcement learning procedure and a new to ACT-R supervised learning procedure. This is the first integrated cognitive model of collaborative skill acquisition and, as such, gives us valuable insights into the multiple cognitive processes that are involved in learning tocollaborate.

Effective RNN-Based Forecasting Methodology Design for Improving Short-Term Power Load Forecasts: Application to Large-Scale Power-Grid Time Series

This article introduces a carefully-engineered forecasting methodology for day-ahead electric power load forecasts evaluated using the European Network of Transmission System Operators for Electricity (ENTSO-E). Two steps were employed to configure the desired forecasting methodology: First, a straightforward processing pipeline is proposed to enable systematic preprocessing of raw multivariate time-discrete power data extracted from the ENTSO-E repository, including a stride-based sliding window approach to generate time series-based batches ready for the supervised learning procedure. Second, the lightweight type of recurrent neural network method, namely gated recurrent units (GRU), is selected and carefully calibrated to yield accurate multi-step forecasts, which was trained using the preprocessed multivariate time series data to render day-ahead power load forecasts. The forecasting estimates generated by the proposed GRU model are evaluated using a set of regression-based metrics to assess the models’ precisions. The empirical results show that the proposed forecasting methodology yields outstanding day-ahead power load forecasting performance regarding the enterprise-class measured data compared to a statistical model, namely autoregressive integrated moving average with exogenous variables (ARIMAX), as well as the actual day-ahead forecasts generated by the ENTSO-E platform.

Robust probabilistic modeling for single-cell multimodal mosaic integration and imputation via scVAEIT

Recent advances in single-cell technologies enable joint profiling of multiple omics. These profiles can reveal the complex interplay of different regulatory layers in single cells; still, new challenges arise when integrating datasets with some features shared across experiments and others exclusive to a single source; combining information across these sources is called mosaic integration. The difficulties lie in imputing missing molecular layers to build a self-consistent atlas, finding a common latent space, and transferring learning to new data sources robustly. Existing mosaic integration approaches based on matrix factorization cannot efficiently adapt to nonlinear embeddings for the latent cell space and are not designed for accurate imputation of missing molecular layers. By contrast, we propose a probabilistic variational autoencoder model, scVAEIT, to integrate and impute multimodal datasets with mosaic measurements. A key advance is the use of a missing mask for learning the conditional distribution of unobserved modalities and features, which makes scVAEIT flexible to combine different panels of measurements from multimodal datasets accurately and in an end-to-end manner. Imputing the masked features serves as a supervised learning procedure while preventing overfitting by regularization. Focusing on gene expression, protein abundance, and chromatin accessibility, we validate that scVAEIT robustly imputes the missing modalities and features of cells biologically different from the training data. scVAEIT also adjusts for batch effects while maintaining the biological variation, which provides better latent representations for the integrated datasets. We demonstrate that scVAEIT significantly improves integration and imputation across unseen cell types, different technologies, and different tissues.

An information-theoretic study of rainfall time series through the Dempster–Shafer approach over a meteorological subdivision of India

Abstract The present paper reports a study, where we have developed a methodology to understand the relative uncertainty associated with the rainfall amount corresponding to summer monsoon (JJAS) and post monsoon (OND) for the period 1871–2016 over northeast India. After calculating the partial correlation between two random variables after the removal of the effect of the third one, we have standardized all the realizations of the random variables. Subsequently, after applying the Dempster–Shafer theory, we have obtained joint basic assignments through two judging criteria for the fuzzy sets representing the closeness of the observed values to two measures of central tendency for different window sizes obtained from the original time series. The study revealed a higher rate of increase in the uncertainty with a change in the window size for OND than in the case of JJAS. This study finally concluded that this approach could generate some idea about the most advantageous ratio of training and test cases for predictive models with supervised learning procedures.

Supervised learning for analysing movement patterns in a virtual reality experiment.

The projection into a virtual character and the concomitant illusionary body ownership can lead to transformations of one’s entity. Both during and after the exposure, behavioural and attitudinal changes may occur, depending on the characteristics or stereotypes associated with the embodied avatar. In the present study, we investigated the effects on physical activity when young students experience being old. After assignment (at random) to a young or an older avatar, the participants’ body movements were tracked while performing upper body exercises. We propose and discuss the use of supervised learning procedures to assign these movement patterns to the underlying avatar class in order to detect behavioural differences. This approach can be seen as an alternative to classical feature-wise testing. We found that the classification accuracy was remarkably good for support vector machines with linear kernel and deep learning by convolutional neural networks, when inserting time sub-sequences extracted at random and repeatedly from the original data. For hand movements, associated decision boundaries revealed a higher level of local, vertical positions for the young avatar group, indicating increased agility in their performances. This occurrence held for both guided movements as well as achievement-orientated exercises.

A Real-Time Semi-Supervised Deep Tone Mapping Network

Tone mapping operators (TMOs) can compress the range of high dynamic range (HDR) images so that they can be displayed normally on the low dynamic range (LDR) devices. Recent TMOs based on deep neural networks can produce impressive results, but there are still some shortcomings. On the one hand, their supervised learning procedure requires a high-quality paired dataset which is hard to be accessed. On the other hand, they are too slow and heavy to meet the needs of practical applications. This paper proposes a real-time deep semi-supervised learning TMO to solve the above problems. The proposed method learns in a semi-supervised manner by combining the adversarial loss, cycle consistency loss, and the pixel-wise loss. The first two can simulate the image distributions in the real world from the unpaired LDR data and the latter can learn the guidance of paired LDR labels. In this way, the proposed method only requires HDR sources, unpaired high-quality LDR images, and a few well tone-mapped HDR-LDR pairs as training data. Furthermore, the proposed method divides tone mapping into luminance mapping and saturation adjustment and then processes them simultaneously. By this strategy, we can reconstruct each component more precisely. Based on the aforementioned improvements, we propose a lightweight tone mapping network that is efficient in tone mapping task (up to 5000x parameters-saving and 27x time-saving compared to the learning-based TMOs). Both quantitative and qualitative results demonstrate that the proposed method performs favorable against state-of-the-art TMOs.

KFC: A clusterwise supervised learning procedure based on the aggregation of distances

ABSTRACT Nowadays, many machine learning procedures are available on the shelve and may be used easily to calibrate predictive models on supervised data. However, when the input data consists of more than one unknown cluster, linked to different underlying predictive models, fitting a model is a more challenging task. We propose, in this paper, a three-step procedure to automatically solve this problem. The first step aims at catching the clustering structure of the input data, which may be characterized by several statistical distributions. For each partition, the second step fits a specific predictive model based on the data in each cluster. The overall model is computed by a consensual aggregation of the models corresponding to the different partitions. A comparison of the performances on different simulated and real data assesses the excellent performance of our method in a large variety of prediction problems.

Forecasting hurricane‐related power outages via locally optimized random forests

Standard supervised learning procedures are validated against a test set that is assumed to have come from the same distribution as the training data. However, in many problems, the test data may have come from a different distribution. We consider the case of having many labeled observations from one distribution, P1, and wanting to make predictions at unlabeled points that come from P2. We combine the high predictive accuracy of random forests with an importance sampling scheme, where the splits and predictions of the base trees are done in a weighted manner, which we call locally optimized random forests. These weights correspond to a non‐parametric estimate of the likelihood ratio between the training and test distributions. To estimate these ratios with an unlabeled test set, we make the covariate shift assumption, where the differences in distribution are only a function of the training distributions. This methodology is motivated by the problem of forecasting power outages during hurricanes. The extreme nature of the most devastating hurricanes means that typical validation set ups will produce models that underestimate their devastating effects. Our method provides a data‐driven means of adapting a machine learning method to deal with extreme events.

A generative spiking neural-network model of goal-directed behaviour and one-step planning.

In mammals, goal-directed and planning processes support flexible behaviour used to face new situations that cannot be tackled through more efficient but rigid habitual behaviours. Within the Bayesian modelling approach of brain and behaviour, models have been proposed to perform planning as probabilistic inference but this approach encounters a crucial problem: explaining how such inference might be implemented in brain spiking networks. Recently, the literature has proposed some models that face this problem through recurrent spiking neural networks able to internally simulate state trajectories, the core function at the basis of planning. However, the proposed models have relevant limitations that make them biologically implausible, namely their world model is trained ‘off-line’ before solving the target tasks, and they are trained with supervised learning procedures that are biologically and ecologically not plausible. Here we propose two novel hypotheses on how brain might overcome these problems, and operationalise them in a novel architecture pivoting on a spiking recurrent neural network. The first hypothesis allows the architecture to learn the world model in parallel with its use for planning: to this purpose, a new arbitration mechanism decides when to explore, for learning the world model, or when to exploit it, for planning, based on the entropy of the world model itself. The second hypothesis allows the architecture to use an unsupervised learning process to learn the world model by observing the effects of actions. The architecture is validated by reproducing and accounting for the learning profiles and reaction times of human participants learning to solve a visuomotor learning task that is new for them. Overall, the architecture represents the first instance of a model bridging probabilistic planning and spiking-processes that has a degree of autonomy analogous to the one of real organisms.

Characterization of e-Government adoption in Europe

The digital divide in Europe has not yet been bridged and thus more contributions towards understanding the factors affecting the different dimensions involved are required. This research offers some insights into the topic by analyzing the e-Government adoption or practical use of e-Government across Europe (26 EU countries). Based on the data provided by the statistical office of the European Union (Eurostat), we defined two indexes, the E-Government Use Index (EGUI) and an extreme version of it taking into account only null or complete use (EGUI+), and characterized the use/non use of e-Government tools using supervised learning procedures in a selection of countries with different e-Government adoption levels. These procedures achieved an average accuracy of 73% and determined the main factors related to the practical use of e-Government in each of the countries, e.g. the frequency of buying goods over the Internet or the education level. In addition, we compared the proposed indexes to other indexes measuring the level of e-readiness of a country such as the E-Government Development Index (EGDI) its Online Service Index (OSI) component, the Networked Readiness Index (NRI) and its Government usage component (GU). The ranking comparison found that EGUI+ is correlated with the four indexes mentioned at 0.05 significance level, as the majority of countries were ranked in similar positions. The outcomes contribute to gaining understanding about the factors influencing the use of e-Government in Europe and the different adoption levels.

Speaker recognition: an enhanced approach to identify singer voice using neural network

Communication with people is the most common phenomena of human. Mostly they can recognize the voice of their known one. Even the same thing is seen while recognizing a voice in the music. If the voice of the artist is known, then the recognition will be the easier one, but if the voice is not very familiar to the listener, it will be a tough job to identify the voice within music. Thus, singer recognition is one of the demanding areas of research by the implication of eligible algorithms in the domain of audio signal processing. There are different approaches that can be made for fulfilling the objective by attaining the goal of truncating the voice frequency range from the audio signal or it may be the detection of the peaks of the voice within that music. As music is polyphonic, so, the essential analysis is required to check for the frequency components and thereby detecting the peaks of the voice signal which can be an easier approach for such detection. In this paper, some songs are taken into consideration to create the training data and through which the neural network is trained. With that training data, a separate set of data is prepared which is used for testing. Apart from the application of the supervised learning procedure, with the implication of hyper parameter tuning, the efficiency is observed for the detection of the new and unknown signer to be detected. Essentially, the neural network works in this field fairly with about 99.29% accuracy and thus the detection is made with a satisfactory level.

Pre-Synaptic Pool Modification (PSPM): A supervised learning procedure for recurrent spiking neural networks.

Learning synaptic weights of spiking neural network (SNN) models that can reproduce target spike trains from provided neural firing data is a central problem in computational neuroscience and spike-based computing. The discovery of the optimal weight values can be posed as a supervised learning task wherein the weights of the model network are chosen to maximize the similarity between the target spike trains and the model outputs. It is still largely unknown whether optimizing spike train similarity of highly recurrent SNNs produces weight matrices similar to those of the ground truth model. To this end, we propose flexible heuristic supervised learning rules, termed Pre-Synaptic Pool Modification (PSPM), that rely on stochastic weight updates in order to produce spikes within a short window of the desired times and eliminate spikes outside of this window. PSPM improves spike train similarity for all-to-all SNNs and makes no assumption about the post-synaptic potential of the neurons or the structure of the network since no gradients are required. We test whether optimizing for spike train similarity entails the discovery of accurate weights and explore the relative contributions of local and homeostatic weight updates. Although PSPM improves similarity between spike trains, the learned weights often differ from the weights of the ground truth model, implying that connectome inference from spike data may require additional constraints on connectivity statistics. We also find that spike train similarity is sensitive to local updates, but other measures of network activity such as avalanche distributions, can be learned through synaptic homeostasis.

On the Robust Estimation of Small Failure Probabilities for Strong Nonlinear Models

Abstract Structural reliability methods are nowadays a cornerstone for the design of robustly performing structures, thanks to advancements in modeling and simulation tools. Monte Carlo-based simulation tools have been shown to provide the necessary accuracy and flexibility. While standard Monte Carlo estimation of the probability of failure is not hindered in its applicability by approximations or limiting assumptions, it becomes computationally unfeasible when small failure probability needs to be estimated, especially when the underlying numerical model evaluation is time consuming. In this case, variance reduction techniques are commonly employed, allowing for the estimation of small failure probabilities with a reduced number of samples and model calls. As a competing approach to variance reduction techniques, surrogate models can be used to substitute the computationally expensive model and performance function with an easy to evaluate numerical function calibrated through a supervised learning procedure. Both these tools provide accurate results for structural application. However, particular care should be taken into account when the reliability problems deal with high-dimensional or strongly nonlinear structural performances since the accuracy of the estimate is largely dependent on choices made during the surrogate modeling process. In this work, we compare the performance of the most recent state-of-the-art advance Monte Carlo techniques and surrogate models when applied to strongly nonlinear performance functions. This will provide the analysts with an insight to the issues that could arise in these challenging problems and help to decide with confidence on which tool to select in order to achieve accurate estimation of the failure probabilities within feasible times with their available computational capabilities.

Automatic cephalometric landmarks detection on frontal faces: An approach based on supervised learning techniques

Facial landmarks are employed in many research areas, including facial recognition, craniofacial identification, age and sex estimation being the most important. In forensics, the focus is on the analysis of a particular set of facial landmarks, defined as cephalometric landmarks. Previous studies demonstrated that the descriptive adequacy of these anatomical references for indirect application (photo-anthropometric description) increased the marking precision of these points, contributing to greater reliability of these analyses. Nevertheless, most are performed manually and all are subject to bias on the part of expert examiners. Therefore, the purpose of this work was to develop and validate automatic techniques for detection of cephalometric landmarks from digital images of frontal facial images in forensics. The presented approach uses a combination of computer vision and image processing techniques within supervised learning procedures. The proposed methodology obtains similar precision to a group of human manual cephalometric reference markers and results that are more accurate than other state-of-the-art facial landmark detection frameworks. It achieves a normalized mean distance (in pixels) error of 0.014, similar to the mean inter-expert dispersion (0.009) and clearly better than other automatic approaches that were analyzed during the course of this study (0.026 and 0.101).

Comparison of Unsupervised Learning Methods for Natural Image Processing

For computer vision based appraoches such as image classification (Krizhevsky et al. 2012), object detection (Ren et al. 2015) or pixel-wise weed classification (Milioto et al. 2017) machine learning is used for both feature extraction and processing (e.g. classification or regression). Historically, feature extraction (e.g. PCA; Ch. 12.1. in Bishop 2006) and processing were sequential and independent tasks (Wöber et al. 2013). Since the rise of convolutional neuronal networks (LeCun et al. 1989), a deep machine learning approach optimized for images, in 2012 (Krizhevsky et al. 2012), feature extraction for image analysis became an automated procedure. A convolutional neuronal net uses a deep architecture of artificial neurons (Goodfellow 2016) for both feature extraction and processing. Based on prior information such as image classes and supervised learning procedures, parameters of the neuronal nets are adjusted. This is known as the learning process. Simultaneously, geometric morphometrics (Tibihika et al. 2018, Cadrin and Friedland 1999) are used in biodiversity research for association analysis. Those approaches use deterministic two-dimensional locations on digital images (landmarks; Mitteroecker et al. 2013), where each position corresponds to biologically relevant regions of interest. Since this methodology is based on scientific results and compresses image content into deterministic landmarks, no uncertainty regarding those landmark positions is taken into account, which leads to information loss (Pearl 1988). Both, the reduction of this loss and novel knowledge detection, can be done using machine learning. Supervised learning methods (e.g., neuronal nets or support vector machines (Ch. 5 and 6. in Bishop 2006)) map data on prior information (e.g. labels). This increases the performance of classification or regression but affects the latent representation of the data itself. Unsupervised learning (e.g. latent variable models) uses assumptions concerning data structures to extract latent representations without prior information. Those representations does not have to be useful for data processing such as classification and due to that, the use of supervised and unsupervised machine learning and combinations of both, needs to be chosen carefully, according to the application and data. In this work, we discuss unsupervised learning algorithms in terms of explainability, performance and theoretical restrictions in context of known deep learning restrictions (Marcus 2018, Szegedy et al. 2014, Su et al. 2017). We analyse extracted features based on multiple image datasets and discuss shortcomings and performance for processing (e.g. reconstruction error or complexity measurement (Pincus 1997)) using the principal component analysis (Wöber et al. 2013), independent component analysis (Stone 2004), deep neuronal nets (auto encoders; Ch. 14 in Goodfellow 2016) and Gaussian process latent variable models (Titsias and Lawrence 2010, Lawrence 2005).

Abstract 1396: Detection and local histological staging of prostate cancer foci in H&E whole slide images using convolutional neural networks

Abstract A pathologist’s histological diagnosis is the gold standard of prostate cancer diagnostic measures. In addition to cancer grade, the stage of cancer determines the follow-up and possible adjuvant therapies after surgery. Not only glandular pattern recognition but also the assessment of three-dimensional extent and size of cancer, in relation to the whole organ, are parts of the subjective diagnosis, which may vary among pathologists. Furthermore, evaluation of lymph nodes for any possible metastases is time-consuming and missing cancer in lymph nodes can lead to undertreatment of the patient. For the abovementioned reasons, our team is developing an algorithm that calculates the amount of cancer tissue objectively, assisting the pathologist in the diagnostic procedure. In addition, we are creating an assistant for cancer detection in lymph nodes, which are removed for histological evaluation. All in all, we aim to create a tool that will help pathologists to a better, faster, more secure and more accurate diagnosis. Our material consists of full sets of scanned whole slide images from 302 prostates that were retrieved by radical prostatectomy. After supervised learning procedure, convolutional neural networks were employed for the classification of cancerous and non-cancerous regions in the images. A tiling based approach was used in which a slide was divided into square shaped small tiles. Millions of cancerous and non-cancerous tiles were sampled from the dataset for training and validation. During the convolutional neural network training, several different tile sizes were used, i.e., 256x256, 512x512, 1024x1024 pixels. Four different types of architectures were fine-tuned and trained for the task of tile-wise binary classification, namely InceptionV3, Xception, ResNet50 and a custom convolutional neural network architecture. In our preliminary assays for cancer detection, in both pixel-wise and tile-wise evaluation, InceptionV3 performed outstandingly well with an AUC score of 0.97 and 0.951, respectively. In conclusion, our algorithm has developed very well thus far with an accuracy in cancer detection of 97%. It is not only a versatile assisting tool, aiding pathologists to a more objective, standardized and accurate diagnosis, but also serves as a second opinion in difficult and challenging diagnostic cases. Citation Format: Carolin Stürenberg, Umair Khan, Kevin Sandeman, Oguzhan Gencoglu, Adrian Malen, Andrew Erickson, Timo Heikkinen, Antti Rannikko, Tuomas Mirtti. Detection and local histological staging of prostate cancer foci in H&E whole slide images using convolutional neural networks [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1396.

Efficient Properties-Based Learning for Mismatch Removal

Mismatch removal is a critical step of feature matching, which is a prerequisite of many vision-based tasks. This paper aims to develop a general and robust method for mismatch removal. To this end, we propose an efficient learning-based mismatch removal method that can significantly improve outlier identification in terms of both accuracy and efficiency. The key idea of our approach is to use a set of properties to describe the putative matches and feed the match representations to a supervised learning procedure learning a binary classifier for mismatch removal. The efficient properties mainly include three aspects: consistency of neighborhood elements, weighted consistency of neighborhood topology, and the stability of correspondence. Different from existing consistency of neighborhood topology, we adopt a weighted strategy to emphasize the effect of different properties with respect to the identified correspondence. The match representations combine the spatial positions of the correspondences with their descriptor reliabilities, which can effectively enlarge the distributions between outliers and inliers. To handle large proportions of outliers, we design a simple strategy to obtain a subset with high ratio inliers guiding the match representations construction process. This strategy can also boost the number of true correspondences without sacrificing the accuracy. Extensive experiments demonstrate our superiority over the state-of-the-art methods.

Data-driven supervised learning of a viral protease specificity landscape from deep sequencing and molecular simulations

Biophysical interactions between proteins and peptides are key determinants of molecular recognition specificity landscapes. However, an understanding of how molecular structure and residue-level energetics at protein-peptide interfaces shape these landscapes remains elusive. We combine information from yeast-based library screening, next-generation sequencing, and structure-based modeling in a supervised machine learning approach to report the comprehensive sequence-energetics-function mapping of the specificity landscape of the hepatitis C virus (HCV) NS3/4A protease, whose function-site-specific cleavages of the viral polyprotein-is a key determinant of viral fitness. We screened a library of substrates in which five residue positions were randomized and measured cleavability of ∼30,000 substrates (∼1% of the library) using yeast display and fluorescence-activated cell sorting followed by deep sequencing. Structure-based models of a subset of experimentally derived sequences were used in a supervised learning procedure to train a support vector machine to predict the cleavability of 3.2 million substrate variants by the HCV protease. The resulting landscape allows identification of previously unidentified HCV protease substrates, and graph-theoretic analyses reveal extensive clustering of cleavable and uncleavable motifs in sequence space. Specificity landscapes of known drug-resistant variants are similarly clustered. The described approach should enable the elucidation and redesign of specificity landscapes of a wide variety of proteases, including human-origin enzymes. Our results also suggest a possible role for residue-level energetics in shaping plateau-like functional landscapes predicted from viral quasispecies theory.

Współczesne zmiany narzędzi badań statystycznych

Many changes are observed in statistical tools for research in the field of analysis and the forecasting of socio-economic processes. The starting point of the considerations carried out is a classic scheme of statistical investigations in the economic sciences. Particular attention is paid to its limitations. Modern methods of data analysis, based on artificial intelligence, can help eliminate the limitations of the classical statistical investigations. These methods can be counted among supervised learning procedures. The paper next goes on to discuss the basic methods of data classification, including LDA and logit. Supervised learning methods that may have wider application in socio-economic studies are then presented. These include: the Naive Bayes Classifier, Bayesian Networks, k-nearest neighbours, vector support machines, kernel classifiers, artificial neural networks, decision trees, and a multi-model approach (random forests, bagging, boosting). However, these methods are also subject to certain restrictions. The article is an overview and contains references to works in which supervised learning methods have been applied in socio-economic studies.