Current Machine Learning Models Research Articles

Calibration of Sentinel-2 Surface Reflectance for Water Quality Modelling in Binh Dinh’s Coastal Zone of Vietnam

Coastal zones are critically important ecosystems that are closely tied to human activities, such as tourism, urbanization, transport, and aquaculture. However, managing and monitoring sea water in the coastal areas is often challenging due to the diversity of the pollution sources. Traditional approaches of onsite measurement and surveys have limitations in terms of cost, efficiency and productivity compared with modern remote sensing methods, particularly for larger and longer observations. Optical remote sensing imagery has been proven to be a good data source for water quality assessment in general and for seawater studies in particular with the use of advanced techniques of data processing such as machine learning (ML) algorithms. However, optical remote sensing data also have their own disadvantages as they are much affected by climatic conditions, atmospheric gas and particles as a source of noise in the data. This noise could be reduced, but it is still unavoidable. This study aims to model seawater quality parameters (total suspended solids (TSS), chlorophyll-a (chla), chemical oxygen demand (COD), and dissolved oxygen (DO)) along a 134 km sea coastal area of the Binh Dinh province by applying the current robust machine learning models of decision tree (DT), random forest (RF), gradient boosting regression (GBR), and Ada boost regression (ABR) using Sentinel-2 imagery. To reduce the atmospheric effects, we conducted onsite measurements of sea surface reflectance (SSR) using the German RAMSES-TriOS instrument for calibration of the Sentinel-2 level 2A data before inputting them to the ML models. Our modeling results showed an improvement of the model accuracy using calibrated SSR compared with the original Sentinel-2 level 2A SSR data. The RF predicted the most accurate seawater quality parameters compared with in situ field-measured data (mean R2 = 0.59 using original Sentinel-2 level 2A SSR and R2 = 0.70 using calibrated SSR). The chla was the most precise estimate (R2 = 0.74 when modelled by the RF model) flowing by DO, COD and TSS. In terms of seawater quality estimation, this accuracy is at a good level. The results of the seawater quality distributions were strongly correlated with coastal features where higher values of TSS, chla, COD, and DO are near the river mouths and urban and tourist areas. These spatial water quality data could be extremely helpful for local governments to make decisions when the modelling is continuously conducted (using big data processing), and it is highly recommended for more applications.

Strain data augmentation enables machine learning of inorganic crystal geometry optimization

Machine-learning (ML) models offer the potential to rapidly evaluate the vast inorganic crystalline materials space to efficiently find materials with properties that meet the challenges of our time. Current ML models require optimized equilibrium structures to attain accurate predictions of formation energies. However, equilibrium structures are generally not known for new materials and must be obtained through computationally expensive optimization, bottlenecking ML-based material screening. A computationally efficient structure optimizer is therefore highly desirable. In this work, we present an ML model capable of predicting the crystal energy response to global strain by using available elasticity data to augment the dataset. The addition of global strains improves our model's understanding of local strains too, significantly improving the accuracy of energy predictions on distorted structures. This allows us to construct an ML-based geometry optimizer, which we used for improving the predictions of formation energy for structures with perturbed atomic positions.

Deep learning framework for congestion detection at public places via learning from synthetic data

Congestion in public places is one of the major problems in public transportation systems and causes a high level of discomfort for the commuters. Traditionally, overcrowding is detected by manually monitoring and analyzing the video streams from the surveillance cameras, which might lead to errors due to limited human activity. On the other hand, current machine learning models for automatic congestion detection require a massive amount of labeled data to train the network. These models suffer from the over-fitting problem and cannot be generalized to novel scenes. First, we propose a novel synthetic dataset for congestion detection in public places to address these problems. Secondly, we propose a Bidirectional Long-short-term-memory (Bi-LSTM) model that exploits synthetic datasets to boost the performance of congestion detection in the wild. We adopt a domain adaptation strategy to bridge the gap between the real and synthetic data by pre-train the model on the synthetic dataset and then fine-tuning the model on real data. From experiment results, we observe that the proposed framework achieves a significant performance boost on the real datasets after training on the synthetic dataset.

This looks More Like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation

Current machine learning models have shown high efficiency in solving a wide variety of real-world problems. However, their black box character poses a major challenge for the comprehensibility and traceability of the underlying decision-making strategies. As a remedy, numerous post-hoc and self-explanation methods have been developed to interpret the models’ behavior. Those methods, in addition, enable the identification of artifacts that, inherent in the training data, can be erroneously learned by the model as class-relevant features. In this work, we provide a detailed case study of a representative for the state-of-the-art self-explaining network, ProtoPNet, in the presence of a spectrum of artifacts. Accordingly, we identify the main drawbacks of ProtoPNet, especially its coarse and spatially imprecise explanations. We address these limitations by introducing Prototypical Relevance Propagation (PRP), a novel method for generating more precise model-aware explanations. Furthermore, in order to obtain a clean, artifact-free dataset, we propose to use multi-view clustering strategies for segregating the artifact images using the PRP explanations, thereby suppressing the potential artifact learning in the models.

Investigation of a Sparse Autoencoder-Based Feature Transfer Learning Framework for Hydrogen Monitoring Using Microfluidic Olfaction Detectors.

Alternative fuel sources, such as hydrogen-enriched natural gas (HENG), are highly sought after by governments globally for lowering carbon emissions. Consequently, the recognition of hydrogen as a valuable zero-emission energy carrier has increased, resulting in many countries attempting to enrich natural gas with hydrogen; however, there are rising concerns over the safe use, storage, and transport of H2 due to its characteristics such as flammability, combustion, and explosivity at low concentrations (4 vol%), requiring highly sensitive and selective sensors for safety monitoring. Microfluidic-based metal–oxide–semiconducting (MOS) gas sensors are strong tools for detecting lower levels of natural gas elements; however, their working mechanism results in a lack of real-time analysis techniques to identify the exact concentration of the present gases. Current advanced machine learning models, such as deep learning, require large datasets for training. Moreover, such models perform poorly in data distribution shifts such as instrumental variation. To address this problem, we proposed a Sparse Autoencoder-based Transfer Learning (SAE-TL) framework for estimating the hydrogen gas concentration in HENG mixtures using limited datasets from a 3D printed microfluidic detector coupled with two commercial MOS sensors. Our framework detects concentrations of simulated HENG based on time-series data collected from a cost-effective microfluidic-based detector. This modular gas detector houses metal–oxide–semiconducting (MOS) gas sensors in a microchannel with coated walls, which provides selectivity based on the diffusion pace of different gases. We achieve a dominant performance with the SAE-TL framework compared to typical ML models (94% R-squared). The framework is implementable in real-world applications for fast adaptation of the predictive models to new types of MOS sensor responses.

High-efficient low-cost characterization of composite material properties using domain-knowledge-guided self-supervised learning

Modern AI-assisted approaches have revolutionized our abilities to better understand the properties of concrete and composite materials. However, current machine learning models such as supervised learning (SL) models usually require a large amount of training data to feed the model. Here, we introduce self-supervised learning (SSL) to address the issue of lacking labeled data in concrete material characterization. We propose a generalized SSL-based framework with domain knowledge and demonstrate its robustness to predict the properties of a commonly-used composite material (concrete) with the fewest data possible. Our numerical results show that the performance of the proposed SSL model can match the commonly-used supervised learning model with only 5 % of data, and the SSL model is also proven with ease of implementation. Our study paves the way to expand further the usability of machine learning tools for composite material fields and the broader material science community.

Intelligent Hybrid Deep Learning Model for Breast Cancer Detection

Breast cancer (BC) is a type of tumor that develops in the breast cells and is one of the most common cancers in women. Women are also at risk from BC, the second most life-threatening disease after lung cancer. The early diagnosis and classification of BC are very important. Furthermore, manual detection is time-consuming, laborious work, and, possibility of pathologist errors, and incorrect classification. To address the above highlighted issues, this paper presents a hybrid deep learning (CNN-GRU) model for the automatic detection of BC-IDC (+,−) using whole slide images (WSIs) of the well-known PCam Kaggle dataset. In this research, the proposed model used different layers of architectures of CNNs and GRU to detect breast IDC (+,−) cancer. The validation tests for quantitative results were carried out using each performance measure (accuracy (Acc), precision (Prec), sensitivity (Sens), specificity (Spec), AUC and F1-Score. The proposed model shows the best performance measures (accuracy 86.21%, precision 85.50%, sensitivity 85.60%, specificity 84.71%, F1-score 88%, while AUC 0.89 which overcomes the pathologist’s error and miss classification problem. Additionally, the efficiency of the proposed hybrid model was tested and compared with CNN-BiLSTM, CNN-LSTM, and current machine learning and deep learning (ML/DL) models, which indicated that the proposed hybrid model is more robust than recent ML/DL approaches.

More than meets the AI: The possibilities and limits of machine learning in olfaction.

Can machine learning crack the code in the nose? Over the past decade, studies tried to solve the relation between chemical structure and sensory quality with Big Data. These studies advanced computational models of the olfactory stimulus, utilizing artificial intelligence to mine for clear correlations between chemistry and psychophysics. Computational perspectives promised to solve the mystery of olfaction with more data and better data processing tools. None of them succeeded, however, and it matters as to why this is the case. This article argues that we should be deeply skeptical about the trend to black-box the sensory system’s biology in our theories of perception. Instead, we need to ground both stimulus models and psychophysical data on real causal-mechanistic explanations of the olfactory system. The central question is: Would knowledge of biology lead to a better understanding of the stimulus in odor coding than the one utilized in current machine learning models? That is indeed the case. Recent studies about receptor behavior have revealed that the olfactory system operates by principles not captured in current stimulus-response models. This may require a fundamental revision of computational approaches to olfaction, including its psychological effects. To analyze the different research programs in olfaction, we draw on Lloyd’s “Logic of Research Questions,” a philosophical framework which assists scientists in explicating the reasoning, conceptual commitments, and problems of a modeling approach in question.

Multi-Scale Deep Residual Shrinkage Network for Atrial Fibrillation Recognition

In this paper, a novel multi-scale deep residual shrinkage network (MS-DRSN) is proposed for signal denoising and atrial fibrillation (AF) recognition. Signal denoising is done by multi-scale threshold denoising module (MS-TDM), which consists of two parts: threshold acquisition and threshold denoising. The thresholds are automatically obtained through the global attention module constructed by the neural network. Threshold denoising chooses Garrote as the threshold function, which combines the advantages of soft and hard thresholding. The multi-scale features consist of global attention module and local attention module, and then the multi-scale features are denoised using the acquired thresholds and threshold functions, and the AF recognition task is finally completed in the Softmax layer after the superposition of multiple MS-TDMs. An adaptive synthetic sampling (ADASYN) algorithm is also used to oversample the dataset and achieve data category balancing by generating new samples, which improves the accuracy of AF recognition and alleviates the overfitting of the neural network. This method was experimented and validated on the PhysioNet2017 dataset. The experimental results show that the approach achieves an accuracy of 0.894 and an [Formula: see text] score of 0.881, which is better than current machine learning and deep learning models.

A Machine Learning Approach to Identify the Importance of Novel Features for CRISPR/Cas9 Activity Prediction.

The reprogrammable CRISPR/Cas9 genome editing tool’s growing popularity is hindered by unwanted off-target effects. Efforts have been directed toward designing efficient guide RNAs as well as identifying potential off-target threats, yet factors that determine efficiency and off-target activity remain obscure. Based on sequence features, previous machine learning models performed poorly on new datasets, thus there is a need for the incorporation of novel features. The binding energy estimation of the gRNA-DNA hybrid as well as the Cas9-gRNA-DNA hybrid allowed generating better performing machine learning models for the prediction of Cas9 activity. The analysis of feature contribution towards the model output on a limited dataset indicated that energy features played a determining role along with the sequence features. The binding energy features proved essential for the prediction of on-target activity and off-target sites. The plateau, in the performance on unseen datasets, of current machine learning models could be overcome by incorporating novel features, such as binding energy, among others. The models are provided on GitHub (GitHub Inc., San Francisco, CA, USA).

Improving Human Decision-Making with Machine Learning

A key aspect of human intelligence is their ability to convey their knowledge to others in succinct forms. However, despite their predictive power, current machine learning models are largely blackboxes, making it difficult for humans to extract useful insights. Focusing on sequential decision-making, we design a novel machine learning algorithm that conveys its insights to humans in the form of interpretable

Temporal quality degradation in AI models

As AI models continue to advance into many real-life applications, their ability to maintain reliable quality over time becomes increasingly important. The principal challenge in this task stems from the very nature of current machine learning models, dependent on the data as it was at the time of training. In this study, we present the first analysis of AI “aging”: the complex, multifaceted phenomenon of AI model quality degradation as more time passes since the last model training cycle. Using datasets from four different industries (healthcare operations, transportation, finance, and weather) and four standard machine learning models, we identify and describe the main temporal degradation patterns. We also demonstrate the principal differences between temporal model degradation and related concepts that have been explored previously, such as data concept drift and continuous learning. Finally, we indicate potential causes of temporal degradation, and suggest approaches to detecting aging and reducing its impact.

Data as Expression

ABSTRACT In machine learning literature, the concept of expression is seldom addressed as a general term in relation to information construction and data. The term more often carries a narrower meaning and refers to human bodily, emotional, or artistic dimensions that are recorded to train a model. However, this paper discusses a view in which expression is understood as any human sensory realm that can bring incidents explicit through the act of “pressing out,” as the early etymology of expression indicates. Also, all the sensory data that are used to train a machine learning model and the data the trained model gives as an output are considered meaningful through their expressive mediality—a form that is actively produced and can therefore be subjected to critical phenomenological analysis. This paper contrasts nonverbal expressions with categorical or linguistic expressions and asks, “What do eye movements express when they are used in training a machine learning model? What kind of expression arises in linguistic models? What could be considered aesthetic data (and what would it express)?” For philosopher John Dewey, instinctive reactions in human behavior that for example exhibit mere discharge of an emotion should be separated from purposeful expressions. However, in machine learning, the two Deweyan positions (instinctive and intentional) collapse, as artificial expressions are purely simulations of learned logic. Therefore, the phenomenological question of this paper is traced back to the original stimulus and its human annotators. In this paper, philosopher Don Ihde’s experimental phenomenology explains how appearances can be attended to without subsuming them under any assumptions, whereas art theory—arising especially from philosopher Dieter Mersch’s thinking—provides an understanding of the mediality of expression. This paper introduces examples from machine learning that are not generally considered expressive but are used for regular tasks, such as object detection, and provides an alternative approach from aesthetics and artistic research that understands these modalities as expressive. If data are understood as expressive, it can be critically assessed how current machine learning models constitute knowledge.

Probabilistic Traffic Motion Labeling for Multi-Modal Vehicle Route Prediction.

The prediction of the motion of traffic participants is a crucial aspect for the research and development of Automated Driving Systems (ADSs). Recent approaches are based on multi-modal motion prediction, which requires the assignment of a probability score to each of the multiple predicted motion hypotheses. However, there is a lack of ground truth for this probability score in the existing datasets. This implies that current Machine Learning (ML) models evaluate the multiple predictions by comparing them with the single real trajectory labeled in the dataset. In this work, a novel data-based method named Probabilistic Traffic Motion Labeling (PROMOTING) is introduced in order to (a) generate probable future routes and (b) estimate their probabilities. PROMOTING is presented with the focus on urban intersections. The generation of probable future routes is (a) based on a real traffic dataset and consists of two steps: first, a clustering of intersections with similar road topology, and second, a clustering of similar routes that are driven in each cluster from the first step. The estimation of the route probabilities is (b) based on a frequentist approach that considers how traffic participants will move in the future given their motion history. PROMOTING is evaluated with the publicly available Lyft database. The results show that PROMOTING is an appropriate approach to estimate the probabilities of the future motion of traffic participants in urban intersections. In this regard, PROMOTING can be used as a labeling approach for the generation of a labeled dataset that provides a probability score for probable future routes. Such a labeled dataset currently does not exist and would be highly valuable for ML approaches with the task of multi-modal motion prediction. The code is made open source.

Efficacy of Deep Learning Models in the Prognosis Task

Deep learning model is an improved version of machine learning technology which brings out the efficiency much better on multivariate data. Deep Leaning models are performed actively and powerfully than current machine learning models. It triggers computational model to learn automatically from the given data at multiple levels. The reputation of deep learning is progressed on today’s large volume of data set and the advancement of hardware that provides potential computing systems. This article analyzes various deep learning models based on the architecture, activation functions, merits, and demerits in a novel way.

Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems

Visual representation of user and item relations is an important issue in recommender systems. This is a big data task that helps to understand the underlying structure of the information, and it can be used by company managers and technical staff. Current collaborative filtering machine learning models are designed to improve prediction accuracy, not to provide suitable visual representations of data. This paper proposes a deep learning model specifically designed to display the existing relations among users, items, and both users and items. Making use of representative datasets, we show that by setting small embedding sizes of users and items, the recommender system accuracy remains nearly unchanged; it opens the door to the use of bidimensional and three-dimensional representations of users and items. The proposed neural model incorporates variational embedding stages to “unpack” (extend) embedding representations, which facilitates identifying individual samples. It also replaces the join layers in current models with a Lambda Euclidean layer that better catches the space representation of samples. The results show numerical and visual improvements when the proposed model is used compared to the baselines. The proposed model can be used to explain recommendations and to represent demographic features (gender, age, etc.) of samples.

Breast and Lung Anticancer Peptides Classification Using N-Grams and Ensemble Learning Techniques

Anticancer peptides (ACPs) are short protein sequences; they perform functions like some hormones and enzymes inside the body. The role of any protein or peptide is related to its structure and the sequence of amino acids that make up it. There are 20 types of amino acids in humans, and each of them has a particular characteristic according to its chemical structure. Current machine and deep learning models have been used to classify ACPs problems. However, these models have neglected Amino Acid Repeats (AARs) that play an essential role in the function and structure of peptides. Therefore, in this paper, ACPs offer a promising route for novel anticancer peptides by extracting AARs based on N-Grams and k-mers using two peptides’ datasets. These datasets pointed to breast and lung cancer cells assembled and curated manually from the Cancer Peptide and Protein Database (CancerPPD). Every dataset consists of a sequence of peptides and their synthesis and anticancer activity on breast and lung cancer cell lines. Five different feature selection methods were used in this paper to improve classification performance and reduce the experimental costs. After that, ACPs were classified using four classifiers, namely AdaBoost, Random Forest Tree (RFT), Multi-class Support Vector Machine (SVM), and Multi-Layer Perceptron (MLP). These classifiers were evaluated by applying five well-known evaluation metrics. Experimental results showed that the breast and lung ACPs classification process provided an accurate performance that reached 89.25% and 92.56%, respectively. In terms of AUC, it reached 95.35% and 96.92% for both breast and lung ACPs, respectively. The proposed classifiers performed competently somewhat equally in AUC, accuracy, precision, F-measures, and recall, except for Multi-class SVM-based feature selection, which showed superior performance. As a result, this paper significantly improved the predictive performance that can effectively distinguish ACPs as virtual inactive, experimental inactive, moderately active, and very active.

Machine learning methods to predict presence of residual cancer following hysterectomy

Surgical management for gynecologic malignancies often involves hysterectomy, often constituting the most common gynecologic surgery worldwide. Despite maximal surgical and medical care, gynecologic malignancies have a high rate of recurrence following surgery. Current machine learning models use advanced pathology data that is often inaccessible within low-resource settings and are specific to singular cancer types. There is currently a need for machine learning models to predict non-clinically evident residual disease using only clinically available health data. Here we developed and tested multiple machine learning models to assess the risk of residual disease post-hysterectomy based on clinical and operative parameters. Data from 3656 hysterectomy patients from the NSQIP dataset over 14 years were used to develop models with a training set of 2925 patients and a validation set of 731 patients. Our models revealed the top postoperative predictors of residual disease were the initial presence of gross abdominal disease on the diaphragm, disease located on the bowel mesentery, located on the bowel serosa, and disease located within the adjacent pelvis prior to resection. There were no statistically significant differences in performances of the top three models. Extreme gradient Boosting, Random Forest, and Logistic Regression models had comparable AUC ROC (0.90) and accuracy metrics (87–88%). Using these models, physicians can identify gynecologic cancer patients post-hysterectomy that may benefit from additional treatment. For patients at high risk for disease recurrence despite adequate surgical intervention, machine learning models may lay the basis for potential prospective trials with prophylactic/adjuvant therapy for non-clinically evident residual disease, particularly in under-resourced settings.

A convolution neural network for forest leaf chlorophyll and carotenoid estimation using hyperspectral reflectance

Forest leaf chlorophyll (Cab) and carotenoid (Cxc) are key functional indicators for the state of the forest ecosystem. Current machine learning models based on hyperspectral reflectance are widely applied to estimate leaf Cab and Cxc contents at leaf scale. However, these models have certain accuracy for non-independent datasets but have poor generalization for independent datasets when they are used to estimate leaf Cab and Cxc contents. This fact limits that hyperspectral remote sensing completely replaces destructive measurements for leaf Cab and Cxc contents. Thus, the development of an estimation model with high accuracy and satisfactory generalization is necessary. Convolutional neural networks (CNNs) have certain accuracy and generalization in many domains, and have the potential to solve above-mentioned problem. Therefore, this study developed a CNN using one-dimensional hyperspectral reflectance, which aimed to improve the model's accuracy and generalization in leaf Cab and Cxc content estimation at leaf scale. The proposed CNN was developed by three steps. First, in consideration of the correlation between leaf Cab and Cxc contents in natural leaves, 2500 physical data with leaf reflectance and corresponding Cab and Cxc contents were generated by leaf radiative transfer model and multivariable gaussian distribution function. Then, the proposed CNN was built by five strategies based on the architecture of the AlexNet. Finally, five-fold cross validation was performed with 70% of the physical data to determine the best strategy to develop the proposed CNN. These were executed to ensure the proposed CNN with the maximum accuracy and generalization. In addition, the accuracy and generalization of the proposed CNN were tested using a non-independent dataset and an independent dataset, respectively. The proposed CNN was also compared with back propagation neural network (BPNN), support vector regression (SVR) and gaussian process regression (GPR). Results showed that the best CNN could be developed with one input, five convolutional, three max-pooling and three fully-connected layers. Comprehensively considering the model's accuracy and generalization, the proposed CNN was the best model for leaf Cab and Cxc content estimation compared with BPNN, SVR and GPR. This study provides a development strategy of CNN estimation model using one-dimensional hyperspectral reflectance at leaf scale. The proposed CNN could further promote the practical application of hyperspectral remote sensing in leaf Cab and Cxc content estimation.

Cost-Efficient and Skew-Aware Data Scheduling for Incremental Learning in 5G Networks

To facilitate the emerging applications in 5G networks, mobile network operators will provide many network functions in terms of control and prediction. Recently, they have recognized the power of machine learning (ML) and started to explore its potential to facilitate those network functions. Nevertheless, the current ML models for network functions are often derived in an offline manner, which is inefficient due to the excessive overhead for transmitting a huge volume of dataset to remote ML training clouds and failing to provide the incremental learning capability for the continuous model updating. As an alternative solution, we propose <i>Cocktail</i>, an incremental learning framework within a reference 5G network architecture. To achieve cost efficiency while increasing trained model accuracy, an efficient online data scheduling policy is essential. To this end, we formulate an online data scheduling problem to optimize the framework cost while alleviating the data skew issue caused by the capacity heterogeneity of training workers from the long-term perspective. We exploit the stochastic gradient descent to devise an online asymptotically optimal algorithm, including two optimal policies based on novel graph constructions for skew-aware data collection and data training. Small-scale testbed and large-scale simulations validate the superior performance of our proposed framework.