Supervised Machine Learning Methods Research Articles

Identifying Personal Narratives in Chinese Weblog Posts

Automated text classification technologies have enabled researchers to amass enormous collections of personal narratives posted to English-language weblogs. In this paper, we explore analogous approaches to identify personal narratives in Chinese weblog posts as a precursor to the future empirical studies of cross-cultural differences in narrative structure. We describe the collection of over half a million posts from a popular Chinese weblog hosting service, and the manual annotation of story and nonstory content in sampled posts. Using supervised machine learning methods, we developed an automated text classifier for personal narratives in Chinese posts, achieving classification accuracy comparable to previous work in English. Using this classifier, we automatically identify over sixty-four thousand personal narratives for use in future cross-cultural analyses and Chinese-language applications of narrative corpora.

Bootstrap robust prescriptive analytics

We address the problem of prescribing an optimal decision in a framework where the cost function depends on uncertain problem parameters that need to be learned from data. Earlier work proposed prescriptive formulations based on supervised machine learning methods. These prescriptive methods can factor in contextual information on a potentially large number of covariates to take context specific actions which are superior to any static decision. When working with noisy or corrupt data, however, such nominal prescriptive methods can be prone to adverse overfitting phenomena and fail to generalize on out-of-sample data. In this paper we combine ideas from robust optimization and the statistical bootstrap to propose novel prescriptive methods which safeguard against overfitting. We show indeed that a particular entropic robust counterpart to such nominal formulations guarantees good performance on synthetic bootstrap data. As bootstrap data is often a sensible proxy to actual out-of-sample data, our robust counterpart can be interpreted to directly encourage good out-of-sample performance. The associated robust prescriptive methods furthermore reduce to convenient tractable convex optimization problems in the context of local learning methods such as nearest neighbors and Nadaraya–Watson learning. We illustrate our data-driven decision-making framework and our novel robustness notion on a small newsvendor problem.

Transparent Sequential Learning for Statistical Process Control of Serially Correlated Data

Machine learning methods have been widely used in different applications, including process control and monitoring. For handling statistical process control (SPC) problems, conventional supervised machine learning methods (e.g., artificial neural networks and support vector machines) would have some difficulties. For instance, a training dataset containing both in-control and out-of-control (OC) process observations is required by a supervised machine learning method, but it is rarely available in SPC applications. Furthermore, many machine learning methods work like black boxes. It is often difficult to interpret their learning mechanisms and the resulting decision rules in the context of an application. In the SPC literature, there have been some existing discussions on how to handle the lack of OC observations in the training data, using the one-class classification, artificial contrast, real-time contrast, and some other novel ideas. However, these approaches have their own limitations to handle SPC problems. In this article, we extend the self-starting process monitoring idea that has been employed widely in modern SPC research to a general learning framework for monitoring processes with serially correlated data. Under the new framework, process characteristics to learn are well specified in advance, and process learning is sequential in the sense that the learned process characteristics keep being updated during process monitoring. The learned process characteristics are then incorporated into a control chart for detecting process distributional shift based on all available data by the current observation time. Numerical studies show that process monitoring based on the new learning framework is more reliable and effective than some representative existing machine learning SPC approaches.

An Efficient Heart Disease Prediction System based on Supervised Machine Learning Methods

Heart illness is the most frequent disease that can lead to grave situations in a human life. Every year more number of people gets affected by heart disease around the world. Prediction and finding of heart disease has always been a critical and difficult task for the medical field. To overcome the issues, Machine Learning (ML) methods played a vital role to predict and detect heart disease very accurately and quickly. In this paper, an innovative approach is proposed to predict heart disease using different ML methods. The proposed method consists of three stages. First to preprocess the heart disease Dataset, which consists of checking the missing values, data scaling and calculate the correlation between the predictors. In the second stage apply the supervised classification methods such as Logistic Regression (LR), KNN, Random Forest (RF) and AdaBoost to train the model. classification metrics are used to evaluate the model and choose the best classifier for heart disease prediction. Also comparison can be done using the metrics of each classifier. As a result of the experiment, it is concluded that the RF classifier has attained more accuracy of 95.08% , ROC-AUC score of 0.95.

Analysis and visualization of COVID-19 discourse on Twitter using data science: a case study of the USA, the UK and India

PurposeThis paper aims to understand, examine and interpret the main concerns and emotions of the people regarding COVID-19 pandemic in the UK, the USA and India using Data Science measures.Design/methodology/approachThis study implements unsupervised and supervised machine learning methods, i.e. topic modeling and sentiment analysis on Twitter data for extracting the topics of discussion and calculating public sentiment.FindingsGovernments and policymakers remained the focus of public discussion on Twitter during the first three months of the pandemic. Overall, public sentiment toward the pandemic remained neutral except for the USA.Originality/valueThis paper proposes a Data Science-based approach to better understand the public topics of concern during the COVID-19 pandemic.

Machine Learning Methods to Identify Missed Cases of Bladder Cancer in Population-Based Registries.

PURPOSEPopulation-based cancer incidence rates of bladder cancer may be underestimated. Accurate estimates are needed for understanding the burden of bladder cancer in the United States. We developed and evaluated the feasibility of a machine learning–based classifier to identify bladder cancer cases missed by cancer registries, and estimated the rate of bladder cancer cases potentially missed.METHODSData were from population-based cohort of 37,940 bladder cancer cases 65 years of age and older in the SEER cancer registries linked with Medicare claims (2007-2013). Cases with other urologic cancers, abdominal cancers, and unrelated cancers were included as control groups. A cohort of cancer-free controls was also selected using the Medicare 5% random sample. We used five supervised machine learning methods: classification and regression trees, random forest, logic regression, support vector machines, and logistic regression, for predicting bladder cancer.RESULTSRegistry linkages yielded 37,940 bladder cancer cases and 766,303 cancer-free controls. Using health insurance claims, classification and regression trees distinguished bladder cancer cases from noncancer controls with very high accuracy (95%). Bacille Calmette-Guerin, cystectomy, and mitomycin were the most important predictors for identifying bladder cancer. From 2007 to 2013, we estimated that up to 3,300 bladder cancer cases in the United States may have been missed by the SEER registries. This would result in an average of 3.5% increase in the reported incidence rate.CONCLUSIONSEER cancer registries may potentially miss bladder cancer cases during routine reporting. These missed cases can be identified leveraging Medicare claims and data analytics, leading to more accurate estimates of bladder cancer incidence.

Effective and practical neural ranking

Supervised machine learning methods that use neural networks ("deep learning") have yielded substantial improvements to a multitude of Natural Language Processing (NLP) tasks in the past decade. Improvements to Information Retrieval (IR) tasks, such as ad-hoc search, lagged behind those in similar NLP tasks, despite considerable community efforts. Although there are several contributing factors, I argue in this dissertation that early attempts were not more successful because they did not properly consider the unique characteristics of IR tasks when designing and training ranking models. I first demonstrate this by showing how large-scale datasets containing weak relevance labels can successfully replace training on in-domain collections. This technique improves the variety of queries encountered when training and helps mitigate concerns of over-fitting particular test collections. I then show that dataset statistics available in specific IR tasks can be easily incorporated into neural ranking models alongside the textual features, resulting in more effective ranking models. I also demonstrate that contextualized representations, particularly those from transformer-based language models, considerably improve neural ad-hoc ranking performance. I find that this approach is neither limited to the task of ad-hoc ranking (as demonstrated by ranking clinical reports) nor English content (as shown by training effective cross-lingual neural rankers). These efforts demonstrate that neural approaches can be effective for ranking tasks. However, I observe that these techniques are impractical due to their high query-time computational costs. To overcome this, I study approaches for offloading computational cost to index-time, substantially reducing query-time latency. These techniques make neural methods practical for ranking tasks. Finally, I take a deep dive into better understanding the linguistic biases of the methods I propose compared to contemporary and traditional approaches. The findings from this analysis highlight potential pitfalls of recent methods and provide a way to measure progress in this area going forward.

An algorithm for detecting leaks of insider information of financial markets in investment consulting

The paper focuses on revealing insider information leaks of financial markets during investment consulting. An original dataset was created, containing the records of the conversations between consultants and clients, presented in the formof dialogs in text format. The applicability of machine learning methods for automating the detection of leaks arising in a conversation between a consultant and a client has been studied. The authors examined the applicability of the following supervised machine learning methods for constructing and training a classifier: probabilistic (Naïve Bayes classifier), metric (k-nearest neighbors algorithm), logical (random forest), linear (support vector machine), and methods based on artificial neural networks. The paper considers various approaches to the construction of a natural language text model, such as tokenization (bag of words, word n-grams: bigrams and trigrams) and vectorization (one-hot encoding). The proposed algorithm for detecting financial markets insider information leaks is based on the use of support vector machine (SVM) and tokenization by bigrams. The obtained results demonstrate that SVM and bigram tokenization provide the highest leakage detection accuracy. The research results can be used in cybersecurity tools development, as well as for the further elaboration of natural language processing methods dealing with information security problems.

Supervised Machine Learning Methods and Hyperspectral Imaging Techniques Jointly Applied for Brain Cancer Classification.

Hyperspectral imaging techniques (HSI) do not require contact with patients and are non-ionizing as well as non-invasive. As a consequence, they have been extensively applied in the medical field. HSI is being combined with machine learning (ML) processes to obtain models to assist in diagnosis. In particular, the combination of these techniques has proven to be a reliable aid in the differentiation of healthy and tumor tissue during brain tumor surgery. ML algorithms such as support vector machine (SVM), random forest (RF) and convolutional neural networks (CNN) are used to make predictions and provide in-vivo visualizations that may assist neurosurgeons in being more precise, hence reducing damages to healthy tissue. In this work, thirteen in-vivo hyperspectral images from twelve different patients with high-grade gliomas (grade III and IV) have been selected to train SVM, RF and CNN classifiers. Five different classes have been defined during the experiments: healthy tissue, tumor, venous blood vessel, arterial blood vessel and dura mater. Overall accuracy () results vary from 60% to 95% depending on the training conditions. Finally, as far as the contribution of each band to the is concerned, the results obtained in this work are 3.81 times greater than those reported in the literature.

Semi‐supervised deep autoencoder for seismic facies classification

ABSTRACTFacies boundaries are critical for flow performance in a reservoir and are significant for lithofacies identification in well interpretation and reservoir prediction. Facies identification based on supervised machine learning methods usually requires a large amount of labelled data, which are sometimes difficult to obtain. Here, we introduce the deep autoencoder to learn the hidden features and conduct facies classification from elastic attributes. Both labelled and unlabelled data are involved in the training process. Then, we develop a semi‐supervised deep autoencoder by taking the mean of intra‐class and the whole population of facies into account to construct a classification regularization term, thereby improving the classification accuracy and reducing the uncertainty. The new method inherits the profits of deep autoencoder and absorbs the information provided by labelled data. The proposed method performs well and produces promising results when it is used to address problems of reservoir prediction and facies identification. The new method is evaluated on both well and seismic data and compared with the conventional deep autoencoder method, which demonstrates its feasibility and superiority with respect to classification accuracy.

Zero Initialised Unsupervised Active Learning by Optimally Balanced Entropy-Based Sampling for Imbalanced Problems

ABSTRACT Given the challenge of gathering labelled training data for machine learning tasks, active learning has become popular. This paper focuses on the beginning of unsupervised active learning, where there are no labelled data at all. The aim of this zero initialised unsupervised active learning is to select the most informative examples – even from an imbalanced dataset – to be labelled manually. Our solution with proposed selection strategy, called Optimally Balanced Entropy-Based Sampling (OBEBS) reaches a balanced training set at each step to avoid imbalanced problems. Two theorems of the optimal solution for selection strategy are also presented and proved in the paper. At the beginning of the active learning, there is not enough information for supervised machine learning method, thus our selection strategy is based on unsupervised learning (clustering). The cluster membership likelihoods of the items are essential for the algorithm to connect the clusters and the classes, i.e., to find assignment between them. For the best assignment, the Hungarian algorithm is used, and single, multi, and adaptive assignment variants of OBEBS method are developed. Based on generated and real images datasets of handwritten digits, the experimental results show that our method surpasses the state-of-the-art methods.

Comparative Analysis of Multiple Neurodegenerative Diseases Based on Advanced Epigenetic Aging Brain.

Background: Neurodegenerative Diseases (NDs) are age-dependent and include Alzheimer’s disease (AD), Parkinson’s disease (PD), progressive supranuclear palsy (PSP), frontotemporal dementia (FTD), and so on. There have been numerous studies showing that accelerated aging is closely related (even the driver of) ND, thus promoting imbalances in cellular homeostasis. However, the mechanisms of how different ND types are related/triggered by advanced aging are still unclear. Therefore, there is an urgent need to explore the potential markers/mechanisms of different ND types based on aging acceleration at a system level.Methods: AD, PD, PSP, FTD, and aging markers were identified by supervised machine learning methods. The aging acceleration differential networks were constructed based on the aging score. Both the enrichment analysis and sensitivity analysis were carried out to investigate both common and specific mechanisms among different ND types in the context of aging acceleration.Results: The extracellular fluid, cellular metabolisms, and inflammatory response were identified as the common driving factors of cellular homeostasis imbalances during the accelerated aging process. In addition, Ca ion imbalance, abnormal protein depositions, DNA damage, and cytoplasmic DNA in macrophages were also revealed to be special mechanisms that further promote AD, PD, PSP, and FTD, respectively.Conclusion: The accelerated epigenetic aging mechanisms of different ND types were integrated and compared through our computational pipeline.

Technology opportunity discovery of proton exchange membrane fuel cells based on generative topographic mapping

Technology opportunity discovery is a key factor in technological innovation that is closely related to the development of a country and has a great impact on promoting social progress. To find the unexplored areas of technology and present the detailed direction of technology development, this paper proposes a systematic approach to patent text data. It includes four stages. First, patent text data are collected and preprocessed by natural language process methods. Second, text mining is executed by the means of supervised machine learning methods. It is clustered with the k-mean++ algorithm after text representation as keywords vectors. Third, patents are visualized in two-dimensional space by the generative topographic mapping. Different from other work, Principal components analysis is used to transform complex multi-dimensional keyword vectors in two-dimensional space. Last, take proton exchange membrane fuel cells as an example. Discuss the meaning of each patent vacancy which be interpreted by its inverse mapping onto the original keyword vector and approved by experts. This approach not only saves time to identify patent vacancies but also increases objectivity and reliability. To some extent, it can help enterprises and researchers to identify the research and development strategy that focuses on innovation in the future.

Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task.

A three-dimensional (3D) structure extraction technique viewed from a two-dimensional image is essential for the development of a computer-aided diagnosis (CAD) system for colonoscopy. However, a straightforward application of existing depth-estimation methods to colonoscopic images is impossible or inappropriate due to several limitations of colonoscopes. In particular, the absence of ground-truth depth for colonoscopic images hinders the application of supervised machine learning methods. To circumvent these difficulties, we developed an unsupervised and accurate depth-estimation method. We propose a novel unsupervised depth-estimation method by introducing a Lambertian-reflection model as an auxiliary task to domain translation between real and virtual colonoscopic images. This auxiliary task contributes to accurate depth estimation by maintaining the Lambertian-reflection assumption. In our experiments, we qualitatively evaluate the proposed method by comparing it with state-of-the-art unsupervised methods. Furthermore, we present two quantitative evaluations of the proposed method using a measuring device, as well as a new 3D reconstruction technique and measured polyp sizes. Our proposed method achieved accurate depth estimation with an average estimation error of less than 1 mm for regions close to the colonoscope in both of two types of quantitative evaluations. Qualitative evaluation showed that the introduced auxiliary task reduces the effects of specular reflections and colon wall textures on depth estimation and our proposed method achieved smooth depth estimation without noise, thus validating the proposed method. We developed an accurate depth-estimation method with a new type of unsupervised domain translation with the auxiliary task. This method is useful for analysis of colonoscopic images and for the development of a CAD system since it can extract accurate 3D information.

MarkIt: A Collaborative Artificial Intelligence Annotation Platform Leveraging Blockchain For Medical Imaging Research.

Current research on medical image processing relies heavily on the amount and quality of input data. Specifically, supervised machine learning methods require well-annotated datasets. A lack of annotation tools limits the potential to achieve high-volume processing and scaled systems with a proper reward mechanism. We developed MarkIt, a web-based tool, for collaborative annotation of medical imaging data with artificial intelligence and blockchain technologies. Our platform handles both Digital Imaging and Communications in Medicine (DICOM) and non-DICOM images, and allows users to annotate them for classification and object detection tasks in an efficient manner. MarkIt can accelerate the annotation process and keep track of user activities to calculate a fair reward. A proof-of-concept experiment was conducted with three fellowship-trained radiologists, each of whom annotated 1,000 chest X-ray studies for multi-label classification. We calculated the inter-rater agreement and estimated the value of the dataset to distribute the reward for annotators using a crypto currency. We hypothesize that MarkIt allows the typically arduous annotation task to become more efficient. In addition, MarkIt can serve as a platform to evaluate the value of data and trade the annotation results in a more scalable manner in the future. The platform is publicly available for testing on https://markit.mgh.harvard.edu.

Automatic inference of demographic parameters using generative adversarial networks.

Population genetics relies heavily on simulated data for validation, inference and intuition. In particular, since the evolutionary ‘ground truth’ for real data is always limited, simulated data are crucial for training supervised machine learning methods. Simulation software can accurately model evolutionary processes but requires many hand‐selected input parameters. As a result, simulated data often fail to mirror the properties of real genetic data, which limits the scope of methods that rely on it. Here, we develop a novel approach to estimating parameters in population genetic models that automatically adapts to data from any population. Our method, pg‐gan, is based on a generative adversarial network that gradually learns to generate realistic synthetic data. We demonstrate that our method is able to recover input parameters in a simulated isolation‐with‐migration model. We then apply our method to human data from the 1000 Genomes Project and show that we can accurately recapitulate the features of real data.

Multispectral aerial imagery-based 3D digitisation, segmentation and annotation of large scale urban areas of significant cultural value

• Urban areas 3D digitisation using multispectral aerial imagery • Experimental low-cost multispectral camera for drone development • Supervised machine learning methods for 3D mesh segmentation using multispectral data • Performance evaluation of the proposed methods on a large scale case study • Web-based visualisation and annotation tool for segmentation fine-tuning Disaster risk management of movable and immovable cultural heritage is a highly significant research topic. In this work, we present a pipeline for 3D digitisation, segmentation and annotation of large scale urban areas in order to produce data that can be exploited in disaster management simulators (e.g fire spreading, crowd movement, firefighting training, evacuation planning, etc.). We have selected the old town of Xanthi (Greece) as a challenging case study. We developed a custom multispectral camera to be carried by a commercial drone. Using the structure from motion / multiview stereo (SFM/MVS) approach, we produced a 3D model of the urban area covering 0.5 k m 2 that is followed by a multilayer texture map which carries information from visible and near-infrared regions of the electromagnetic spectrum. We developed a set of machine learning approaches based on logistic regression, support vector machines and artificial neural networks that allow 3D model segmentation by exploiting not only morphological and structural features but also the multispectral behaviour of different material surfaces. We objectively evaluate the performance of the proposed segmentation approaches on six significant material-based classes (cobbled-roads granite kilns, building walls, ceramic roof-tiles, low-vegetation, high-vegetation and metal surfaces) that are used in simulating fire propagation and crowd movement. The experiments revealed that the segmentation accuracy can be enhanced by taking into consideration surface material multispectral properties as well as morphological features. A Web-based multi-user annotation tool complements our proposed pipeline by enabling further 3D model segmentation, fine tuning and semantics annotation (e.g. usage-based building classification and evacuation priorities, escape paths and gathering points).

A supervised machine learning method to detect anomalous real-time broiler breeder body weight data recorded by a precision feeding system

A precision feeding (PF) system is an intelligent computer-controlled feeding system that can be used to feed individual broilers, breeders or layers automatically based on measuring real-time body weight (BW). Vast amounts of real-time BW data can be generated every day when birds visit a PF station. However, anomalous observations occurred in real-time BW observations, which were caused by multiple birds entering the station at the same time, upward or downward variation in scale measurement in the recorded data due to the movement of the bird, or a misread for radio frequency identification tag. Known anomalous data should be removed because they have a negative impact on the interpretation of the data. Manually cleaning the anomalies is accurate, but it is time-consuming and labor-intensive. Statistical methods and unsupervised machine learning methods are effective in detecting anomalies to some extent because they just check data distribution. The current study reported a supervised machine learning method to detect anomalies in real-time BW recorded by the PF system. Real-time BW data of 5 broiler breeders from day 15 to 306 were checked and the anomalies were manually labeled. Variables regarding the statistical distribution of data and features regarding the feeding activity recorded by the PF system in each day were extracted from the dataset. Among the 4 machine learning algorithms including k-nearest neighbor (KNN), random forest classifier (RF), support vector machine (SVM), and artificial neural network (ANN), RF produced the highest F1 score (0.9712) and area under the precision-recall curve (0.9948). Compared with 4 other common anomaly detection methods including Z-scores, interquartile range (IQR), density-based spatial clustering of applications with noise (DBSCAN), and local outlier factor (LOF), RF had a higher average F1 score (0.9448), which indicated that RF was a more effective anomaly detection algorithm for this type of data.

A Morphological Classification Model to Identify Unresolved PanSTARRS1 Sources. II. Update to the PS1 Point Source Catalog

We present an update to the PanSTARRS-1 Point Source Catalog (PS1 PSC), which provides morphological classifications of PS1 sources. The original PS1 PSC adopted stringent detection criteria that excluded hundreds of millions of PS1 sources from the PSC. Here, we adapt the supervised machine learning methods used to create the PS1 PSC and apply them to different photometric measurements that are more widely available, allowing us to add ∼144 million new classifications while expanding the the total number of sources in PS1 PSC by ∼10%. We find that the new methodology, which utilizes PS1 forced photometry, performs ∼6%–8% worse than the original method. This slight degradation in performance is offset by the overall increase in the size of the catalog. The PS1 PSC is used by time-domain surveys to filter transient alert streams by removing candidates coincident with point sources that are likely to be Galactic in origin. The addition of ∼144 million new classifications to the PS1 PSC will improve the efficiency with which transients are discovered.

Predicting Quality of Castings via Supervised Learning Method

The process input data which materials processing operations can collect for each unit of production is extensive. Large datasets have long been difficult to work with as computing power to execute analysis in a timely fashion was unavailable. Further, the great velocity at which the data is generated makes near real-time decision-making unwieldy without a new set of tools with which to do the work. When troubleshooting by a small dataset, such as the last few hours of production, observations made on the measured parameters can be misleading. Machine learning is opening doors to high-dimensional data analysis in material processing. In this work, high-pressure die-casting (HPDC) is explored as an exemplar of high-volume materials processing. HPDC process summary data from a full year of production data covering over 950,000 machine cycles is analyzed via supervised machine learning methods to successfully model the prediction of good parts and process scrap as determined by the die-casting machine. Additionally, the prediction of ultimate tensile strength via a classification method of extracted tensile bars is performed and the important features identified are discussed. Supervised learning is found to be a useful tool for materials processing applications.