Multiple Evaluation Metrics Research Articles

A Review on Medical Image Super Resolution with Application of Deep Learning

Super resolution problems are often discussed in medical imaging. The spatial resolution of medical images is insufficient due to limitations such as image acquisition time, low radiation dose or hardware limitations. Various super-resolution methods have been proposed to solve these problems, such as optimization or learning-based approaches. Recently, deep learning methodologies have become a thriving technology and are evolving at an exponential rate. We believe we need to write a review to illustrate the current state of deep learning in super-resolution medical imaging. In this article, we provide an overview of image resolution and the deep learning introduced in super resolution. This document describes super resolution for single images versus super resolution for multiple images, evaluation metrics and loss functions.

Multi-task learning with Multi-view Weighted Fusion Attention for artery-specific calcification analysis

In general, artery-specific calcification analysis comprises the simultaneous calcification segmentation and quantification tasks. It can help provide a thorough assessment for calcification of different coronary arteries, and further allow for an efficient and rapid diagnosis of cardiovascular diseases (CVD). However, as a high-dimensional multi-type estimation problem, artery-specific calcification analysis has not been profoundly investigated due to the intractability of obtaining discriminative feature representations. In this work, we propose a Multi-task learning network with Multi-view Weighted Fusion Attention (MMWFAnet) to solve this challenging problem. The MMWFAnet first employs a Multi-view Weighted Fusion Attention (MWFA) module to extract discriminative feature representations by enhancing the collaboration of multiple views. Specifically, MWFA weights these views to improve multi-view learning for calcification features. Based on the fusion of these multiple views, the proposed approach takes advantage of multi-task learning to obtain accurate segmentation and quantification of artery-specific calcification simultaneously. We perform experimental studies on 676 non-contrast Computed Tomography scans, achieving state-of-the-art performance in terms of multiple evaluation metrics. These compelling results evince that the proposed MMWFAnet is capable of improving the effectivity and efficiency of clinical CVD diagnosis.

Research on Classification Model based on Neighborhood Rough Set and Evidence Theory

The forward greedy numerical attribute reduction algorithm based on the neighborhood rough set is used to reduce the continuous numerical evaluation metrics of the health condition of complex equipment. This eliminates the risk of data loss and the additional processing time, due to avoiding discretization of continuous numerical evaluation metrics. Furthermore, the reduced evaluation decision table is processed to construct the basic probability assignment function (BBAS). Finally, multiple evaluation metrics are fused by Dempster’s rule of combination to get the health condition grade, and the relationship between evaluation metrics and health condition grade is mined further. The theoretical analysis and experimental results show that the proposed model is effective and efficient for classification.

L, r-Stitch Unit: Encoder-Decoder-CNN Based Image-Mosaicing Mechanism for Stitching Non-Homogeneous Image Sequences

Image-stitching (or) mosaicing is considered an active research-topic with numerous use-cases in computer-vision, AR/VR, computer-graphics domains, but maintaining homogeneity among the input image sequences during the stitching/mosaicing process is considered as a primary-limitation & major-disadvantage. To tackle these limitations, this article has introduced a robust and reliable image stitching methodology (l,r-Stitch Unit), which considers multiple non-homogeneous image sequences as input to generate a reliable panoramically stitched wide view as the final output. The l,r-Stitch Unit further consists of a pre-processing, post-processing sub-modules & a l,r-PanoED-network, where each sub-module is a robust ensemble of several deep-learning, computer-vision & image-handling techniques. This article has also introduced a novel convolutional-encoder-decoder deep-neural-network (l,r-PanoED-network) with a unique split-encoding-network methodology, to stitch non-coherent input left, right stereo image pairs. The encoder-network of the proposed l,r-PanoED extracts semantically rich deep-feature-maps from the input to stitch/map them into a wide-panoramic domain, the feature-extraction & feature-mapping operations are performed simultaneously in the l,r-PanoED's encoder-network based on the split-encoding-network methodology. The decoder-network of l,r-PanoED adaptively reconstructs the output panoramic-view from the encoder networks' bottle-neck feature-maps. The proposed l,r-Stitch Unit has been rigorously benchmarked with alternative image-stitching methodologies on our custom-built traffic dataset and several other public-datasets. Multiple evaluation metrics (SSIM, PSNR, MSE, L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">α,β,γ</sub> , FM-rate, Average-latency-time) & wild-Conditions (rotational/color/intensity variances, noise, etc) were considered during the benchmarking analysis, and based on the results, our proposed method has outperformed among other image-stitching methodologies and has proved to be effective even in wild non-homogeneous inputs.

Retrieval-enhanced adversarial training with dynamic memory-augmented attention for image paragraph captioning

Existing image paragraph captioning methods generate long paragraph captions solely from input images, relying on insufficient information. In this paper, we propose a retrieval-enhanced adversarial training with dynamic memory-augmented attention for image paragraph captioning (RAMP), which makes full use of the R-best retrieved candidate captions to enhance the image paragraph captioning via adversarial training. Concretely, RAMP treats the retrieved captions as reference captions to augment the discriminator during adversarial training, encouraging the image captioning model (generator) to incorporate informative content in retrieved captions into the generated caption. In addition, a retrieval-enhanced dynamic memory-augmented attention network is devised to keep track of the coverage information and attention history along with the update-chain of the decoder state, and therefore avoiding generating repetitive or incomplete image descriptions. Finally, a copying mechanism is applied to select words from the retrieved candidate captions, which are then put into the proper positions of the target caption so as to improve the fluency and informativeness of the generated caption. Extensive experiments on a benchmark dataset (i.e., Stanford) demonstrate that the proposed RAMP model significantly outperforms the state-of-the-art methods across multiple evaluation metrics. For reproducibility, we submit the code and data at https://github.com/anonymous-caption/RAMP.

PharmKG: a dedicated knowledge graph benchmark for bomedical data mining.

Biomedical knowledge graphs (KGs), which can help with the understanding of complex biological systems and pathologies, have begun to play a critical role in medical practice and research. However, challenges remain in their embedding and use due to their complex nature and the specific demands of their construction. Existing studies often suffer from problems such as sparse and noisy datasets, insufficient modeling methods and non-uniform evaluation metrics. In this work, we established a comprehensive KG system for the biomedical field in an attempt to bridge the gap. Here, we introduced PharmKG, a multi-relational, attributed biomedical KG, composed of more than 500000 individual interconnections between genes, drugs and diseases, with 29 relation types over a vocabulary of ~8000 disambiguated entities. Each entity in PharmKG is attached with heterogeneous, domain-specific information obtained from multi-omics data, i.e. gene expression, chemical structure and disease word embedding, while preserving the semantic and biomedical features. For baselines, we offered nine state-of-the-art KG embedding (KGE) approaches and a new biological, intuitive, graph neural network-based KGE method that uses a combination of both global network structure and heterogeneous domain features. Based on the proposed benchmark, we conducted extensive experiments to assess these KGE models using multiple evaluation metrics. Finally, we discussed our observations across various downstream biological tasks and provide insights and guidelines for how to use a KG in biomedicine. We hope that the unprecedented quality and diversity of PharmKG will lead to advances in biomedical KG construction, embedding and application.

LPX: Overlapping community detection based on X‐means and label propagation algorithm in attributed networks

AbstractTraditional community detection methods in attributed networks (eg, social network) usually disregard abundant node attribute information and only focus on structural information of a graph. Existing community detection methods in attributed networks are mostly applied in the detection of nonoverlapping communities and cannot be directly used to detect the overlapping structures. This article proposes an overlapping community detection algorithm in attributed networks. First, we employ the modified X‐means algorithm to cluster attributes to form different themes. Second, we employ the label propagation algorithm (LPA), which is based on neighborhood network conductance for priority and the rule of theme weight, to detect communities in each theme. Finally, we perform redundant processing to form the final community division. The proposed algorithm improves the X‐means algorithm to avoid the effects of outliers. Problems of LPA such as instability of division and adjacent communities being easily merged can be corrected by prioritizing the node neighborhood network conductance. As the community is detected in the attribute subspace, the algorithm can find overlapping communities. Experimental results on real‐attributed and synthetic‐attributed networks show that the performance of the proposed algorithm is excellent with multiple evaluation metrics.

A Preprocessing Strategy for Denoising of Speech Data Based on Speech Segment Detection

In this paper, we propose a preprocessing strategy for denoising of speech data based on speech segment detection. A design of computationally efficient speech denoising is necessary to develop a scalable method for large-scale data sets. Furthermore, it becomes more important as the deep learning-based methods have been developed because they require significant costs while showing high performance in general. The basic idea of the proposed method is using the speech segment detection so as to exclude non-speech segments before denoising. The speech segmentation detection can exclude non-speech segments with a negligible cost, which will be removed in denoising process with a much higher cost, while maintaining the accuracy of denoising. First, we devise a framework to choose the best preprocessing method for denoising based on the speech segment detection for a target environment. For this, we speculate the environments for denoising using different levels of signal-to-noise ratio (SNR) and multiple evaluation metrics. The framework finds the best speech segment detection method tailored to a target environment according to the performance evaluation of speech segment detection methods. Next, we investigate the accuracy of the speech segment detection methods extensively. We conduct the performance evaluation of five speech segment detection methods with different levels of SNRs and evaluation metrics. Especially, we show that we can adjust the accuracy between the precision and recall of each method by controlling a parameter. Finally, we incorporate the best speech segment detection method for a target environment into a denoising process. Through extensive experiments, we show that the accuracy of the proposed scheme is comparable to or even better than that of Wavenet-based denoising, which is one of recent advanced denoising methods based on deep neural networks, in terms of multiple evaluation metrics of denoising, i.e., SNR, STOI, and PESQ, while it can reduce the denoising time of the Wavenet-based denoising by approximately 40–50% according to the used speech segment detection method.

NDMF: Neighborhood-Integrated Deep Matrix Factorization for Service QoS Prediction

Quality of service (QoS) has been mostly applied to represent non-functional properties of Web services and differentiate those with the same functionality. How to accurately predict service QoS has become a key research topic. Researchers have employed neighborhood information into matrix factorization (MF) for service QoS prediction in recent years. However, they are restricted to traditional matrix factorization that may incur a couple of limitations. 1) Conventional MF for QoS prediction linearly combines the multiplication of the latent feature representation of users and services through inner product, failing to fully capture the implicit features of user and service. 2) Most of approaches integrate user or service neighborhood as heuristics into MF model, where either location context or historical invocation records are used to calculate similar users or services. Nevertheless, combining both of them together in a collaborative way is ignored for neighborhood selection that has yet to be properly explored. To deal with the challenges, we propose a novel approach for service QoS prediction called ${N}$ eighborhood-integrated ${D}$ eep ${M}$ atrix ${F}$ actorization (NDMF), which integrates user neighborhood selected by a collaborative way into an enhanced matrix factorization model via deep neural network (DNN). We implement a prototype system and conduct extensive experiments on public and real-world large Web service dataset with almost 2,000,000 service invocations called WS-DREAM which is widely used in service QoS prediction. The experimental results demonstrate that our proposed approach significantly outperforms state-of-the-art ones in terms of multiple evaluation metrics.

DeepWSC: Clustering Web Services via Integrating Service Composability into Deep Semantic Features

With an growing number of web services available on the Internet, an increasing burden is imposed on the use and management of service repository. Service clustering has been employed to facilitate a wide range of service-oriented tasks, such as service discovery, selection, composition and recommendation. Conventional approaches have been proposed to cluster web services by using explicit features, including syntactic features contained in service descriptions or semantic features extracted by probabilistic topic models. However, service implicit features are ignored and have yet to be properly explored and leveraged. To this end, we propose a novel heuristics-based framework DeepWSC for web service clustering. It integrates deep semantic features extracted from service descriptions by an improved recurrent convolutional neural network and service composability features obtained from service invocation relationships by a signed graph convolutional network, to jointly generate integrated implicit features for web service clustering. Extensive experiments are conducted on 8,459 real-world web services. The experiment results demonstrate that DeepWSC outperforms state-of-the-art approaches for web service clustering in terms of multiple evaluation metrics.

Development of a novel prognostic score combining clinicopathologic variables, gene expression, and mutation profiles for lung adenocarcinoma

BackgroundIntegrating phenotypic and genotypic information to improve prognostic prediction is under active investigation for lung adenocarcinoma (LUAD). In this study, we developed a new prognostic model for event-free survival (EFS) and recurrence-free survival (RFS) based on the combination of clinicopathologic variables, gene expression, and mutation data.MethodsWe enrolled a total of 408 patients from the Cancer Genome Atlas Lung Adenocarcinoma (TCGA-LUAD) project for the study. We pre-selected gene expression or mutation features and constructed 14 different input feature sets for predictive model development. We assessed model performance with multiple evaluation metrics including the distribution of C-index on testing dataset, risk score significance, and time-dependent AUC under competing risks scenario. We stratified patients into higher- and lower-risk subgroups by the final risk score and further investigated underlying immune phenotyping variations associated with the differential risk.ResultsThe model integrating all three types of data achieved the best prediction performance. The resultant risk score provided a higher-resolution risk stratification than other models within pathologically defined subgroups. The score could account for extra EFS-related variations that were not captured by clinicopathologic scores. Being validated for RFS prediction under a competing risks modeling framework, the score achieved a significantly higher time-dependent AUC as compared to that of the conventional clinicopathologic variables-based model (0.772 vs. 0.646, p value < 0.001). The higher-risk patients were characterized with transcriptional aberrations of multiple immune-related genes, and a significant depletion of mast cells and natural killer cells.ConclusionsWe developed a novel prognostic risk score with improved prediction accuracy, using clinicopathologic variables, gene expression and mutation profiles as input, for LUAD. Such score was a significant predictor of both EFS and RFS.Trial registrationThis study was based on public open data from TCGA and hence the study objects were retrospectively registered.

An empirical comparison of commercial and open‐source web vulnerability scanners

SummaryWeb vulnerability scanners (WVSs) are tools that can detect security vulnerabilities in web services. Although both commercial and open‐source WVSs exist, their vulnerability detection capability and performance vary. In this article, we report on a comparative study to determine the vulnerability detection capabilities of eight WVSs (both open and commercial) using two vulnerable web applications: WebGoat and Damn vulnerable web application. The eight WVSs studied were: Acunetix; HP WebInspect; IBM AppScan; OWASP ZAP; Skipfish; Arachni; Vega; and Iron WASP. The performance was evaluated using multiple evaluation metrics: precision; recall; Youden index; OWASP web benchmark evaluation; and the web application security scanner evaluation criteria. The experimental results show that, while the commercial scanners are effective in detecting security vulnerabilities, some open‐source scanners (such as ZAP and Skipfish) can also be effective. In summary, this study recommends improving the vulnerability detection capabilities of both the open‐source and commercial scanners to enhance code coverage and the detection rate, and to reduce the number of false‐positives.

Combining texture features of whole slide images improves prognostic prediction of recurrence-free survival for cutaneous melanoma patients

BackgroundAccurate prediction of recurrence-free survival (RFS) is important for the prognosis of cutaneous melanoma patients. The image-based pathological examination remains as the gold standard for diagnosis. It is of clinical interest to account for computer-aided processing of pathology image when performing prognostic analysis.MethodsWe enrolled in this study a total of 152 patients from TCGA-SKCM (The Cancer Genome Atlas Skin Cutaneous Melanoma project) with complete information in recurrence-related survival time, baseline variables (clinicopathologic variables, mutation status of BRAF and NRAS genes), gene expression data, and whole slide image (WSI) features. We preprocessed WSI to segment global or nucleus areas, and extracted 3 types of texture features from each region. We performed cross validation and used multiple evaluation metrics including C-index and time-dependent AUC to determine the best model of predicting recurrence events. We further performed differential gene expression analysis between the higher and lower-risk groups within AJCC pathologic tumor stage III patients to explore the underlying molecular mechanisms driving risk stratification.ResultsThe model combining baseline variables and WSI features had the best performance among models with any other types of data integration. The prognostic risk score generated by this model could provide a higher-resolution risk stratification within pathologically defined subgroups. We found the selected image features captured important immune-related variations, such as the aberration of expression in T cell activation and proliferation gene sets, and therefore contributed to the improved prediction.ConclusionsOur study provided a prognostic model based on the combination of baseline variables and computer-processed WSI features. This model provided more accurate prediction than models based on other types of data combination in recurrence-free survival analysis.Trial registrationThis study was based on public open data from TCGA and hence the study objects were retrospectively registered.

Amazon Employees Resources Access Data Extraction via Clonal Selection Algorithm and Logic Mining Approach

Amazon.com Inc. seeks alternative ways to improve manual transactions system of granting employees resources access in the field of data science. The work constructs a modified Artificial Neural Network (ANN) by incorporating a Discrete Hopfield Neural Network (DHNN) and Clonal Selection Algorithm (CSA) with 3-Satisfiability (3-SAT) logic to initiate an Artificial Intelligence (AI) model that executes optimization tasks for industrial data. The selection of 3-SAT logic is vital in data mining to represent entries of Amazon Employees Resources Access (AERA) via information theory. The proposed model employs CSA to improve the learning phase of DHNN by capitalizing features of CSA such as hypermutation and cloning process. This resulting the formation of the proposed model, as an alternative machine learning model to identify factors that should be prioritized in the approval of employees resources applications. Subsequently, reverse analysis method (SATRA) is integrated into our proposed model to extract the relationship of AERA entries based on logical representation. The study will be presented by implementing simulated, benchmark and AERA data sets with multiple performance evaluation metrics. Based on the findings, the proposed model outperformed the other existing methods in AERA data extraction.

An Information Maximization Multi-task Clustering Method for egocentric temporal segmentation

With the widespread application of vision-based wearable devices, temporal segmentation helps people search for and localize all occurrences quickly in egocentric videos. In the same scenario, the activities are similar to each other, e.g., people staying at home typically cook, clean and watch TV. These relations among videos of different individuals are regarded as auxiliary information to improve task performance. Inspired by this, we propose an Information Maximization Multi-task Clustering (IMMC) algorithm for egocentric temporal segmentation. The algorithm mainly includes two parts: (1) within-task clustering: clustering on each task based on an information maximization approach, and (2) cross-task information transferring: a novel strategy is presented to transfer correlation information between tasks, which balances the correlation among clusters in different tasks to improve the performance of the individual task. A draw-merge method is designed to address the optimization problem. Experiments are performed on three publicly available first-person vision data sets and a new data set we construct (Outdoor data set). The results show that IMMC consistently outperforms the state-of-the-art clustering methods in multiple evaluation metrics. Moreover, it achieves relatively good performance on runtime cost and convergence.

Contextual encoder–decoder network for visual saliency prediction

Predicting salient regions in natural images requires the detection of objects that are present in a scene. To develop robust representations for this challenging task, high-level visual features at multiple spatial scales must be extracted and augmented with contextual information. However, existing models aimed at explaining human fixation maps do not incorporate such a mechanism explicitly. Here we propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. The architecture forms an encoder–decoder structure and includes a module with multiple convolutional layers at different dilation rates to capture multi-scale features in parallel. Moreover, we combine the resulting representations with global scene information for accurately predicting visual saliency. Our model achieves competitive and consistent results across multiple evaluation metrics on two public saliency benchmarks and we demonstrate the effectiveness of the suggested approach on five datasets and selected examples. Compared to state of the art approaches, the network is based on a lightweight image classification backbone and hence presents a suitable choice for applications with limited computational resources, such as (virtual) robotic systems, to estimate human fixations across complex natural scenes. Our TensorFlow implementation is openly available at https://github.com/alexanderkroner/saliency.

Empirical evaluation and study of text stemming algorithms

Text stemming is one of the basic preprocessing step for Natural Language Processing applications which is used to transform different word forms into a standard root form. For Arabic script based languages, adequate analysis of text by stemmers is a challenging task due to large number of ambigious structures of the language. In literature, multiple performance evaluation metrics exist for stemmers, each describing the performance from particular aspect. In this work, we review and analyze the text stemming evaluation methods in order to devise criteria for better measurement of stemmer performance. Role of different aspects of stemmer performance measurement like main features, merits and shortcomings are discussed using a resource scarce language i.e. Urdu. Through our experiments we conclude that the current evaluation metrics can only measure an average conflation of words regardless of the correctness of the stem. Moreover, some evaluation metrics favor some type of languages only. None of the existing evaluation metrics can perfectly measure the stemmer performance for all kind of languages. This study will help researchers to evaluate their stemmer using right methods.

Multi-label feature selection with shared common mode

Multi-label feature selection plays an indispensable role in multi-label learning, which eliminates irrelevant and redundant features while retaining relevant features. Most of existing multi-label feature selection methods employ two strategies to construct feature selection models: extracting label correlations to guide feature selection process and maintaining the consistency between the feature matrix and the reduced low-dimensional feature matrix. However, the data information is described by two data matrices: the feature matrix and the label matrix. Previous methods devote attention to either of the two data matrices. To address this issue, we propose a novel feature selection method named Feature Selection considering Shared Common Mode between features and labels (SCMFS). First, we utilize Coupled Matrix Factorization (CMF) to extract the shared common mode between the feature matrix and the label matrix, considering the comprehensive data information in the two matrices. Additionally, Non-negative Matrix Factorization (NMF) is adopted to enhance the interpretability for feature selection. Extensive experiments are implemented on fifteen real-world benchmark data sets for multiple evaluation metrics, the experimental results demonstrate the classification superiority of the proposed method.

LMA: label-based multi-head attentive model for long-tail web service classification

With the rapid growth of web services, service classification is widely used to facilitate service discovery, selection, composition and recommendation. Although there is much research in service classification, work rarely focuses on the long-tail problem to improve the accuracy of those categories which have fewer services. In this paper, we propose a novel label-based attentive model LMA with the multi-head structure for long-tail service classification. It can learn the various word-label subspace attention with a multi-head mechanism, and concatenate them to get the high-level feature of services. To demonstrate the effectiveness of LMA, extensive experiments are conducted on 14,616 real-world services with 80 categories crawled from the service repository ProgrammableWeb. The results prove that the LMA outperforms state-of-the-art approaches for long-tail service classification in terms of multiple evaluation metrics.

Breast cancer classification using digital biopsy histopathology images through transfer learning

Breast cancer (BC) infection, which is peculiar to women, brings about the high rate of deaths among women in every part of the world. The early investigation of BC has minimized the severe effects of cancer as compared to the last stage diagnosis. Doctors for diagnostic tests usually suggest the medical imaging modalities like mammograms or biopsy histopathology (Hp) images. However, Hp image analysis gives doctors more confidence to diagnose BC as compared to mammograms. Many studies used Hp images to develop BC classification models to assist doctors in early BC diagnosis. However, these models lack better and reliable results in terms of reporting multiple performance evaluation metrics. Therefore, the goal of this study is to create a reliable, more accurate model that consumes minimum resources by using transfer learning based convolution neural network model. The proposed model uses the trained model after fine tuning, hence requires less number of images and can show better results on minimum resources. BreakHis dataset, which is available publicly has been employed in overall experiments in this research. BreakHis dataset is separated into training, testing, and validation for the experimentation. In addition, the dataset for training was augmented followed by stain normalization. By using the concept of transfer learning (TL), AleNext was retained after fine-tuning the last layer for binary classification like benign and malignant. Afterward, preprocessed images are fed into the TL based model for training. The model training was performed many times by changing the hyper-parameters randomly until the minimum validation loss was achieved. Now the trained model was used for feature extraction. The extracted features were further evaluated by using six ML classifiers (i.e. softmax, Decision tree, Naïve Bayes, Linear discriminant analysis, Support vector machine, k-nearest neighbor) through five performance measures such as precision, F-measure, accuracy, specificity, and sensitivity for experimental evaluation. The softmax has outperformed among all classifiers. Furthermore, to reduce the wrong prediction, a misclassification reducing (MR) algorithm was developed. After using the MR algorithm the proposed model produced better and reliable results. The observed accuracy, specificity, sensitivity, precision and F measure are 81.25%, 77.47%, 82.49%, 91.70%, and 86.80% respectively. These results show that the proposed TL based model along with misclassification reduction algorithm produced comparable results to the current baseline models. Hence, the expected model could serve as a second opinion for BC classification in any healthcare center.