Class Predictions Research Articles

An evolving ensemble model of multi-stream convolutional neural networks for human action recognition in still images

Still image human action recognition (HAR) is a challenging problem owing to limited sources of information and large intra-class and small inter-class variations which requires highly discriminative features. Transfer learning offers the necessary capabilities in producing such features by preserving prior knowledge while learning new representations. However, optimally identifying dynamic numbers of re-trainable layers in the transfer learning process poses a challenge. In this study, we aim to automate the process of optimal configuration identification. Specifically, we propose a novel particle swarm optimisation (PSO) variant, denoted as EnvPSO, for optimal hyper-parameter selection in the transfer learning process with respect to HAR tasks with still images. It incorporates Gaussian fitness surface prediction and exponential search coefficients to overcome stagnation. It optimises the learning rate, batch size, and number of re-trained layers of a pre-trained convolutional neural network (CNN). To overcome bias of single optimised networks, an ensemble model with three optimised CNN streams is introduced. The first and second streams employ raw images and segmentation masks yielded by mask R-CNN as inputs, while the third stream fuses a pair of networks with raw image and saliency maps as inputs, respectively. The final prediction results are obtained by computing the average of class predictions from all three streams. By leveraging differences between learned representations within optimised streams, our ensemble model outperforms counterparts devised by PSO and other state-of-the-art methods for HAR. In addition, evaluated using diverse artificial landscape functions, EnvPSO performs better than other search methods with statistically significant difference in performance.

A discrete salp swarm algorithm with weights and Lévy flights: application for Parkinson’s disease detection

<span>A new hybrid algorithm named discrete salp swarm algorithm that integrates effectiveness of weights, Lévy flights, and an excellent classifier, support vector machine (SVM), has been proposed to predict Parkinson's disease. In the proposed algorithm, salp swarm algorithm (SSA) is used as a feature selection tool, which targets to reduce the noise in features of the speech PD dataset to improve the SVM classifier's prediction accuracy. The efficacy and usefulness of the proposed discrete salp swarm algorithm with Lévy flights have been meticulously assessed against the speech PD dataset in terms of G-mean, accuracy, F-measure, specificity, sensitivity, and precision measures. DWLSSA has achieved values of the measures, 97.76%, 98.75%, 98.77%, 97.37%, 98.15%, and 99.39% respectively. Comparison of DWLSSA with other nature inspired algorithms applied to predict Parkinson’s shows that the proposed DWLSSA performs better. It can be also said that DWLSSA can be an alternative for solving the NP-hard problems.</span>

An Intrinsic Structured Graph Alignment Module With Modality-Invariant Representations for NIR-VIS Face Recognition

Most existing near-infrared to visible (NIR-VIS) face recognition (FR) methods rely on global feature representations to reduce cross-modality discrepancies, but ignore the structural relationships between local features, e.g., the relative positions of eyes, nose, and mouth. Precise alignment of these local features can enhance the learning of modality-invariant face representations, thereby improving the performance of NIR-VIS FR. Therefore, in this letter, we propose an intrinsic structured graph alignment (ISGA) module that aims to obtain a graph-level local feature alignment across modalities. To this end, we first construct an intrinsic structure graph to model the inherent structural relationships of local features, and then enhance the discriminative feature representation by aligning the graphs between modalities. To jointly encourage cross-modality class consistency between semantics and structural relationships, a cross-modality class distribution (CMCD) loss is proposed to add an identity-preserving constraint to each class distribution in the embedding space between two modalities. To solve the resulting problem of suppressed class divisibility, we maximize the mutual information between inputs and class predictions. Extensive experiments on challenging NIR-VIS datasets indicate that our approach outperforms the state-of-the-arts. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JianYu777/ISGA-CMCD</uri> .

Attention-based Long Short Term Memory Model for DNA Damage Prediction in Mammalian Cells

The understanding of DNA damage intensity – concentration-level is critical for biological and biomedical research, such as cellular homeostasis, tumor suppression, immunity, and gametogenesis. Therefore, recognizing and quantifying DNA damage intensity levels is a substantial issue, which requires further robust and effective approaches. DNA damage has several intensity levels. These levels of DNA damage in malignant cells and in other unhealthy cells are significant in the assessment of lesion stages located in normal cells. There is a need to get more insight from the available biological data to predict, explore and classify DNA damage intensity levels. Herein, the development process relied on the available biological dataset related to DNA damage signaling pathways, which plays a crucial role in DNA damage in the mammalian cell system. The biological dataset that was used in the proposed model consists of 15000 records intensity – concentration-level for a set of five proteins which regulate DNA damage. This research paper proposes an innovative deep learning model, which consists of an attention-based long short term-memory (AT-LSTM) model for DNA damage multi class predictions. The proposed model splits the prediction procedure into dual stages. For the first stage, we adopt the related feature sequences which are inserted as input to the LSTM neural network. In the next stage, the attention feature is applied efficiently to adopt the related feature sequences which are inserted as input to the softmax layer for prediction in the following frame. Our developed framework not only solves the long-term dependence problem of prediction effectively, but also enhances the interpretability of the prediction methods that was established on the neural network. We conducted a novel proposed model on big and complex biological datasets to perform prediction and multi classification tasks. Indeed, the (AT-LSTM) model has the ability to predict and classify the DNA damage in several classes: No-Damage, Low-damage, Medium-damage, High-damage, and Excess-damage. The experimental results show that our framework for DNA damage intensity level can be considered as state of the art for the biological DNA damage prediction domain.

A study of machine learning algorithms to measure the feature importance in class-imbalance data of food insecurity cases in Indonesia

The development of various machine learning algorithms on supervised models has become one of the issues in selecting a suitable algorithm. The black box of machine learning requires a technique that can be used to interpret the feature importance using the SHAP in order to obtain predictors. The class-imbalance problem in real cases is another challenge in improving the performance of minority class predictions. This study uses a food insecurity dataset, one of the SDG's important indicators to study to achieve zero hunger. The machine learning algorithms studied consisted of Random Forest, XGBoost, SVM, and NN. Meanwhile, the study of the effect of class-imbalance used three treatments: without handling, SMOTE-N, and ADASYN-N. Twelve models are built based on a combination of four algorithms and three treatments to study the performance models and their feature importance. The SMOTE-N and ADASYN-N were able to increase the sensitivity value up to 0.48 units higher when compared to without handling data. The agreement level on without handling data has a low value, indicated by the 0.736 ICC value, while on SMOTE-N and ADASYN-N, it is higher, indicated by the 0.925 and 0.919 ICC values, respectively. This study dataset is more suitable for using SMOTE-N. It is based on the higher ICC and superior AUC performance. The relatively high ICC value indicates that the use of machine learning algorithms does not influence the agreement level on the feature importance score. Therefore, the choice of a machine learning algorithm can refer to a measure of its performance. Random Forest produced the best performance (AUC and sensitivity). Therefore, the Random Forest SMOTE-N is the best model in this study. It produces food insecurity household characteristics with household conditions having poor water, a small house size, low household head education, few/no savers, and cement or tile flooring.

The research of Wireless Edge Computing Gateway with Anomaly Detection

To the superiority of low cost and convenience, large-scale Internet of things has been deployed for various monitoring pervasively. Typically cloud platform is utilized as a remote data and control center. However, it is a challenge to the bandwidth and real-time feedback to the server because of mass data with uploading and processing. In this paper, to cope with the above problems, we design a gateway based on edge computing. The gateway not only retains the function of connecting multiple sensors and the interaction between devices and cloud, but also can provide real-time local services for terminal devices. It also can drop abnormal data through sensors’ correlation analysis and reduce the amount of data upload by diverting the computing tasks of a server. In this paper, an anomaly detection method based on the bag of words model is introduced. At first, the gateway makes a bag of words representation by the data collected from each sensor, including normal and abnormal data. Then it aggregates these class predictions using the bag of words to calculate the final prediction. Through the simulation analysis based on the data such as humidity, temperature, and lighting monitored by sensors in agricultural greenhouses, the results show that the bag of words model can detect abnormal sensor data.

Boosting Out-of-Distribution Image Detection With Epistemic Uncertainty

Modern deep neural networks are known to generate over-confident class predictions even for unseen samples. However, safety-critical applications are required to understand examples that differ from the training distribution. For example, an autonomous vehicle must return a instant refusal feedback when encountering an unexpected situation. The voice secretary should re-ask the user for a command that was not understood to prevent malfunction. In this paper, we propose an out-of-distribution sample detection algorithm using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Uncertainty-based Additive Fast Gradient Sign Method</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UA-FGSM</i> ), which uses Monte Carlo (MC) dropout during backpropagation. The proposed uncertainty-based method forces in-distribution sample predictions to be more over-confident and out-of-distribution sample predictions to be less over-confident in the pre-trained model. This boosts the discrimination between the in-distribution and out-of-distribution samples. In addition, we further boost this difference by continuously accumulating uncertainty-based gradients. Our method uses the inherent epistemic uncertainty of the pre-trained model. Therefore, the proposed algorithm does not require knowledge of the domain of the in-distribution dataset and works by simple pre-processing of the already trained model without any re-training. We demonstrate its effectiveness using diverse network architectures on various popular image datasets and noisy settings.

Optimization of software defects prediction in imbalanced class using a combination of resampling methods with support vector machine and logistic regression

The main problem in producing high accuracy software defect prediction is if the data set has an imbalance class and dichotomous characteristics. The imbalanced class problem can be solved using a data level approach, such as resampling methods. While the problem of software defects predicting if the data set has dichotomous characteristics can be approached using the classification method. This study aimed to analyze the performance of the proposed software defect prediction method to identify the best combination of resampling methods with the appropriate classification method to provide the highest accuracy. The combination of the proposed methods first is the resampling process using oversampling, under-sampling, or hybrid methods. The second process uses the classification method, namely the Support Vector Machine (SVM) algorithm and the Logistic Regression (LR) algorithm. The proposed, tested model uses five NASA MDP data sets with the same number attributes of 37. Based on the t-test, the &lt; = 0.0344 &lt; 0.05 and the &gt; = 3.1524 &gt; 2.7765 which indicates that the combination of the proposed methods is suitable for classifying imbalanced class. The performance of the classification algorithm has also improved with the use of the resampling process. The average increase in AUC values using the resampling in the SVM algorithm is 17.19%, and the LR algorithm is at 7.26% compared to without the resampling process. Combining the three resampling methods with the SVM algorithm and the LR algorithm shows that the best combining method is the oversampling method with the SVM algorithm to software defects prediction in imbalanced class with an average accuracy value of 84.02% and AUC 91.65%.

Deep Learning Applied to SEM Images for Supporting Marine Coralline Algae Classification

The classification of coralline algae commonly relies on the morphology of cells and reproductive structures, along with thallus organization, observed through Scanning Electron Microscopy (SEM). Nevertheless, species identification based on morphology often leads to uncertainty, due to their general plasticity. Evolutionary and environmental studies featured coralline algae for their ecological significance in both recent and past Oceans and need to rely on robust taxonomy. Research efforts towards new putative diagnostic tools have recently been focused on cell wall ultrastructure. In this work, we explored a new classification tool for coralline algae, using fine-tuning pretrained Convolutional Neural Networks (CNNs) on SEM images paired to morphological categories, including cell wall ultrastructure. We considered four common Mediterranean species, classified at genus and at the species level (Lithothamnion corallioides, Mesophyllum philippii, Lithophyllum racemus, Lithophyllum pseudoracemus). Our model produced promising results in terms of image classification accuracy given the constraint of a limited dataset and was tested for the identification of two ambiguous samples referred to as L. cf. racemus. Overall, explanatory image analyses suggest a high diagnostic value of calcification patterns, which significantly contributed to class predictions. Thus, CNNs proved to be a valid support to the morphological approach to taxonomy in coralline algae.

MACE: Model Agnostic Concept Extractor for Explaining Image Classification Networks

Deep convolutional networks have been quite successful at various image classification tasks. The current methods to explain the predictions of a pretrained model rely on gradient information, often resulting in saliency maps that focus on the foreground object as a whole. However, humans typically reason by dissecting an image and pointing out the presence of smaller concepts. The final output is often an aggregation of the presence or absence of these smaller concepts. In this work, we propose MACE: a model agnostic concept extractor, which can explain the working of a convolutional network through smaller concepts. The MACE framework dissects the feature maps generated by a convolution network for an image to extract concept-based prototypical explanations. Furthermore, it estimates the relevance of the extracted concepts to the pretrained model’s predictions, a critical aspect for explaining the individual class predictions, missing in existing approaches. We validate our framework using VGG16 and ResNet50 CNN architectures and datasets like Animals With Attributes 2 (AWA2) and Places365. Our experiments demonstrate that the concepts extracted by the MACE framework increase the human interpretability of the explanations and are faithful to the underlying pretrained black-box model.

Towards computationally efficient prediction of molecular signatures from routine histology images

Towards computationally efficient prediction of molecular signatures from routine histology images

Deep learning detects acute myeloid leukemia and predicts NPM1 mutation status from bone marrow smears

The evaluation of bone marrow morphology by experienced hematopathologists is essential in the diagnosis of acute myeloid leukemia (AML); however, it suffers from a lack of standardization and inter-observer variability. Deep learning (DL) can process medical image data and provides data-driven class predictions. Here, we apply a multi-step DL approach to automatically segment cells from bone marrow images, distinguish between AML samples and healthy controls with an area under the receiver operating characteristic (AUROC) of 0.9699, and predict the mutation status of Nucleophosmin 1 (NPM1)—one of the most common mutations in AML—with an AUROC of 0.92 using only image data from bone marrow smears. Utilizing occlusion sensitivity maps, we observed so far unreported morphologic cell features such as a pattern of condensed chromatin and perinuclear lightening zones in myeloblasts of NPM1-mutated AML and prominent nucleoli in wild-type NPM1 AML enabling the DL model to provide accurate class predictions.

Deep Feature Fusion Based Dual Branch Network for X-ray Security Inspection Image Classification

Automatic computer security inspection of X-ray scanned images has an irresistible trend in modern life. Aiming to address the inconvenience of recognizing small-sized prohibited item objects, and the potential class imbalance within multi-label object classification of X-ray scanned images, this paper proposes a deep feature fusion model-based dual branch network architecture. Firstly, deep feature fusion is a method to fuse features extracted from several model layers. Specifically, it operates these features by upsampling and dimension reduction to match identical sizes, then fuses them by element-wise sum. In addition, this paper introduces focal loss to handle class imbalance. For balancing importance on samples of minority and majority class, it assigns weights to class predictions. Additionally, for distinguishing difficult samples from easy samples, it introduces modulating factor. Dual branch network adopts the two components above and integrates them in final loss calculation through the weighted sum. Experimental results illustrate that the proposed method outperforms baseline and state-of-art by a large margin on various positive/negative ratios of datasets. These demonstrate the competitivity of the proposed method in classification performance and its potential application under actual circumstances.

An analytic framework using deep learning for prediction of traffic accident injury severity based on contributing factors

Vulnerable road users (VRUs) are exposed to the highest risk in the road traffic environment. Analyzing contributing factors that affect injury severity facilitates injury severity prediction and further application in developing countermeasures to guarantee VRUs safety. Recently, machine learning approaches have been introduced, in which analyses tend to be one-sided and may ignore important information. To solve this problem, this paper proposes a comprehensive analytic framework that employs a deep learning model referred to as the stacked sparse autoencoder (SSAE) to predict the injury severity of traffic accidents based on contributing factors. The essential idea of the method is to integrate various analyses into an analytical framework that performs corresponding data processing and analysis by different machine learning approaches. In the proposed method, first, we utilize a machine learning approach (i.e., Catboost) to analyze the importance and dependence of the contributing factors to injury severity and remove low correlation factors; second, according to the geographical information, we classify the data into different classes by utilizing a machine learning approach (i.e., k-means clustering); third, by employing high correlation factors, we employ an SSAE-based deep learning model to perform injury severity prediction in each data class. By experiments with a real-world traffic accident dataset, we demonstrated the effectiveness and applicability of the framework. Specifically, (1) the importance and dependence of contributing factors were obtained by CatBoost and the Shapley value, and (2) the SSAE-based deep learning model achieved the best performance compared to other baseline models. The proposed analytic framework can also be utilized for other accident data for severity or other risk indicator analyses involving VRUs safety.

An operational Sentinel-2 based monitoring system for the management and control of direct aids to the farmers in the context of the Common Agricultural Policy (CAP): A case study in mainland Portugal

The European Union (EU) member states are expected to develop new procedures, based on automatic earth observation data analysis, for the management and control of direct aids to the farmers, as part of the Common Agricultural Policy (CAP) reform of 2020. Here, we propose an operational monitoring system based on Sentinel-2 surface reflectance (SR) data and machine learning (ML) algorithms, consisting of a hierarchical approach triggering 3 color-coded warning alerts to distinguish among compliant (green), non-compliant (red), and inconclusive (yellow) parcels comparatively to the farmer’s declaration. A Random Forest (RF) model is applied to a 5-day interpolated SR time series to generate a preliminary crop map, where all the parcels whose predicted and declared crop type match are flagged as compliant. Next, a refinement procedure is adopted to improve the discrimination between temporary and permanent crops. At this stage, VI temporal metrics and texture are used as input to a Support Vector Machine (SVM) classifier trained using only the previous compliant parcels. Through a set of decision rules, SVM crop class predictions are flagged as compliant, non-compliant and inconclusive. The system was tested for a significant area in mainland Portugal, using the 2019 Land Parcel Information System (LPIS) data. The system returned 96.5%, 2.5% and 1% of the parcels as compliant, inconclusive, and non-compliant, respectively. Comparison with field inspections for the subsidy control of 2019 revealed that only 1.1% of the correct declarations were classified by the system as non-compliant (5% as admissible value), while less than 5% of the real non-compliant declarations passed through the system (10–20%).

Database, Features, and Machine Learning Model to Identify Thermally Driven Metal–Insulator Transition Compounds

Metal-insulator transition (MIT) compounds are materials that may exhibit insulating or metallic behavior, depending on the physical conditions, and are of immense fundamental interest owing to their potential applications in emerging microelectronics. There is a dearth of thermally-driven MIT materials, however, which makes delineating these compounds from those that are exclusively insulating or metallic challenging. Here we report a material database comprising temperature-controlled MITs (and metals and insulators with similar chemical composition and stoichiometries to the MIT compounds) from high quality experimental literature, built through a combination of materials-domain knowledge and natural language processing. We featurize the dataset using compositional, structural, and energetic descriptors, including two MIT relevant energy scales, an estimated Hubbard interaction and the charge transfer energy, as well as the structure-bond-stress metric referred to as the global-instability index (GII). We then perform supervised classification, constructing three electronic-state classifiers: metal vs non-metal (M), insulator vs non-insulator (I), and MIT vs non-MIT (T). We identify two important descriptors that separate metals, insulators, and MIT materials in a 2D feature space: the average deviation of the covalent radius and the range of the Mendeleev number. We further elaborate on other important features (GII and Ewald energy), and examine how they affect classification of binary vanadium and titanium oxides. We discuss the relationship of these atomic features to the physical interactions underlying MITs in the rare-earth nickelate family. Last, we implement an online version of the classifiers, enabling quick probabilistic class predictions by uploading a crystallographic structure file.

Artificial intelligence-assisted colonic endocytoscopy for cancer recognition: a multicenter study.

Background and study aims Large adenomas are sometimes misidentified as cancers during colonoscopy and are surgically removed. To address this overtreatment, we developed an artificial intelligence (AI) tool that identified cancerous pathology in vivo with high specificity. We evaluated our AI tool under the supervision of a government agency to obtain regulatory approval. Patients and methods The AI tool outputted three pathological class predictions (cancer, adenoma, or non-neoplastic) for endocytoscopic images obtained at 520-fold magnification and previously trained on 68,082 images from six academic centers. A validation test was developed, employing 500 endocytoscopic images taken from various parts of randomly selected 50 large (≥ 20 mm) colorectal lesions (10 images per lesion). An expert board labelled each of the 500 images with a histopathological diagnosis, which was made using endoscopic and histopathological images. The validation test was performed using the AI tool under a controlled environment. The primary outcome measure was the specificity in identifying cancer. Results The validation test consisted of 30 cancers, 15 adenomas, and five non-neoplastic lesions. The AI tool could analyze 83.6 % of the images (418/500): 231 cancers, 152 adenomas, and 35 non-neoplastic lesions. Among the analyzable images, the AI tool identified the three pathological classes with an overall accuracy of 91.9 % (384/418, 95 % confidence interval [CI]: 88.8 %–94.3 %). Its sensitivity and specificity for differentiating cancer was 91.8 % (212/231, 95 % CI: 87.5 %–95.0 %) and 97.3 % (182/187, 95 % CI: 93.9 %–99.1 %), respectively. Conclusions The newly developed AI system designed for endocytoscopy showed excellent specificity in identifying colorectal cancer.

An automated EHR-based tool to facilitate patient identification for biomarker-driven trials.

1539 Background: Clinical trial eligibility increasingly requires information found in NGS tests; lack of structured NGS results hinders the automation of trial matching for this criterion, which may be a deterrent to open biomarker-driven trials in certain sites. We developed a machine learning tool that infers the presence of NGS results in the EHR, facilitating clinical trial matching. Methods: The Flatiron Health EHR-derived database contains patient-level pathology and genetic counseling reports from community oncology practices. An internal team of clinical experts reviewed a random sample of patients across this network to generate labels of whether each patient had been NGS tested. A supervised ML model was trained by scanning documents in the EHR and extracting n-gram features from text snippets surrounding relevant keywords (i.e. 'Lung biomarker', 'Biomarker negative'). Through k-fold cross-validation and l2-regularization, we found that a logistic regression was able to classify patients' NGS testing status. The model's offline performance on a 20% hold-out test set was measured with standard classification metrics: sensitivity, specificity, positive predictive value (PPV) and NPV. In an online setting, we integrated the tool into Flatiron's clinical trial matching software OncoTrials by including in each patient's profile an indicator of "likely NGS tested" or "unlikely NGS tested" based on the classifier's prediction. For patients inferred as tested, the model linked users to a test report view in the EHR. In this online setting, we measured sensitivity and specificity of the model after user review in two community oncology practices. Results: This NGS testing status inference model was characterized using a test sample of 15,175 patients. The model sensitivity and specificity (95%CI) were 91.3% (90.2, 92.3) and 96.2% (95.8, 96.5), respectively; PPV was 84.5% (83.2, 85.8) and NPV was 98.0% (97.7, 98.2). In the validation sample (N = 200 originated from 2 distinct care sites), users identified NGS testing status with a sensitivity of 95.2% (88.3%, 98.7%). Conclusions: This machine learning model facilitates the screening for potential patient enrollment in biomarker-driven trials by automatically surfacing patients with NGS test results at high sensitivity and specificity into a trial matching application to identify candidates. This tool could mitigate a key barrier for participation in biomarker-driven trials for community clinics.

REFINE: Prediction Fusion Network for Panoptic Segmentation

Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results. Prediction fusion is therefore important to achieve accurate panoptic segmentation. In this paper, we present REFINE, pREdiction FusIon NEtwork for panoptic segmentation, to achieve high-quality panoptic segmentation by improving cross-task prediction fusion, and within-task prediction fusion. Our single-model ResNeXt-101 with DCN achieves PQ=51.5 on the COCO dataset, surpassing state-of-the-art performance by a convincing margin and is comparable with ensembled models. Our smaller model with a ResNet-50 backbone achieves PQ=44.9, which is comparable with state-of-the-art methods with larger backbones.

Semi-Supervised Learning with Variational Bayesian Inference and Maximum Uncertainty Regularization

We propose two generic methods for improving semi-supervised learning (SSL). The first integrates weight perturbation (WP) into existing “consistency regularization” (CR) based methods. We implement WP by leveraging variational Bayesian inference (VBI). The second method proposes a novel consistency loss called “maximum uncertainty regularization” (MUR). While most consistency losses act on perturbations in the vicinity of each data point, MUR actively searches for “virtual” points situated beyond this region that cause the most uncertain class predictions. This allows MUR to impose smoothness on a wider area in the input-output manifold. Our experiments show clear improvements in classification errors of various CR based methods when they are combined with VBI or MUR or both.