Proposed Confidence Measure Research Articles

Predicting drug-target interactions by measuring confidence with consistent causal neighborhood interventions

Predicting drug-target interactions (DTI) is a crucial stage in drug discovery and development. Understanding the interaction between drugs and targets is essential for pinpointing the specific relationship between drug molecules and targets, akin to solving a link prediction problem using information technology. While knowledge graph (KG) and knowledge graph embedding (KGE) methods have been rapid advancements and demonstrated impressive performance in drug discovery, they often lack authenticity and accuracy in identifying DTI. This leads to increased misjudgment rates and reduced efficiency in drug development. To address these challenges, our focus lies in refining the accuracy of DTI prediction models through KGE, with a specific emphasis on causal intervention confidence measures (CI). These measures aim to assess triplet scores, enhancing the precision of the predictions. Comparative experiments conducted on three datasets and utilizing 9 KGE models reveal that our proposed confidence measure approach via causal intervention, significantly improves the accuracy of DTI link prediction compared to traditional approaches. Furthermore, our experimental analysis delves deeper into the embedding of intervention values, offering valuable insights for guiding the design and development of subsequent drug development experiments. As a result, our predicted outcomes serve as valuable guidance in the pursuit of more efficient drug development processes.

A Robustness-Based Confidence Measure for Hybrid System Falsification

Verification of hybrid systems is very challenging, if not impossible, due to their continuous dynamics that leads to infinite state space. As a countermeasure, falsification is usually applied to show that a specification does not hold, by searching for a falsifying input as a counterexample that refutes the specification. A falsification algorithm exploits the quantitative robust semantics of temporal specifications, which provides a numerical robustness that tells how robustly a specification holds or not, and uses it as a guide to explore the input space towards the direction of robustness descent—once negative robustness is observed, it indicates that a falsifying input is found. However, if a falsification algorithm does not return any falsifying input, a user is not sure whether the specification does indeed hold, or there exist counterexamples that the algorithm did not manage to reach. In this case, a measurement on how likely there indeed exists no counterexample in the input space is necessary for better understanding the safety of the system and deciding whether more budget should be allocated for the falsification. To this end, we propose a confidence measure that assesses the likelihood that the system is not falsifiable, i.e., how confident a user should be that a specification holds, given the fact that an algorithm has sampled a set of inputs but did not find any falsifying one. The confidence measure is defined in terms of a coverage criterion of the input space that assesses to which extent the whole input space is explored and a local area is exploited where low robustness is observed. Experiments on commonly-used falsification benchmarks show that our proposed confidence measure is reasonable and can distinguish different specifications.

Real-time hand rubbing quality estimation using deep learning enhanced by separation index and feature-based confidence metric

Hand hygiene plays a crucial role in healthcare environments which can cease infections and diseases from spreading. It is also regarded as the second most effective way to control the transmission of COVID-19. The World Health Organization (WHO) recommends a 12-step guideline for alcohol-based hand rubbing. Compliance with this guideline is vital in order to clean the hands thoroughly. Hence, an automated system can help to improve the quality of this procedure. In this study, a large-scale and diverse dataset for both real and fake hand rubbing motions is collected as the first stage of building a reliable hand hygiene system. In the next stage, various pre-trained networks were analyzed and compared using a swift version of the Separation Index (SI) method. The proposed Swift SI method facilitates choosing the best pre-trained network without fine-tuning them on the whole dataset. Accordingly, the Inception-ResNet architecture achieved the highest SI among Inception, ResNet, Xception, and MobileNet networks. Fine-tuning the Inception-ResNet model led to an accuracy of 98% on the test dataset, which is the highest score in the literature. Therefore, from the proposed approach, a lightweight version of this model with fewer layers but almost the same accuracy is produced and examined. In the final stage, a novel metric, called Feature-Based Confidence (FBC), is devised for estimating the confidence of models in prediction. The proposed confidence measure is able to profoundly differentiate models with similar accuracy and determine the superior one. Based on the metrics results, the Inception-ResNet model is about 2x slower but 5% more confident than its lightweight version. Putting all together, by addressing the real-time application concerns, a Deep Learning based method is offered to qualify the hand rubbing process. The model is also employed in a commercial machine, called DeepHARTS, to estimate the quality of the hand rubbing procedure in different organizations and healthcare environments.

Towards efficient real-time traffic classifier: A confidence measure with ensemble Deep Learning

Traffic classification is a key network function for managing both Quality of Service (QoS) and security. While some traffic classification applications (e.g. QoS based path allocation) can tolerate delays, other applications (e.g. attack detection) are time critical. In this context, early traffic classification has been proposed based on the first few packets of flows. However, the choice of the number of packets to inspect is method dependent and based on empirical assessment without considering the information carried by these packets (features). In this paper, we aim at identifying the sufficient number of packets, N, that guarantees high classification accuracy while optimizing the response time, based on both empirical classification results and information theory. We propose a confidence measure based on the variations in the model training accuracy and the average mutual information among the packets’ features and the label vector. This measure is then used to define the value of N, which optimizes the trade-off between the time overhead and the classification accuracy. In addition, we propose an ensemble Deep Learning (DL)-based classifier model to enhance the classification accuracy by training successive DL models based on the traffic stream. The proposed ensemble method output is based on the average of the individual classifiers predictions. The experimental results show that when using the proposed confidence measure, we can achieve good classification accuracy at early phase of the flow. In addition, using the proposed ensemble method presents enhancement in the early classification accuracy. Consequently, combining the ensemble method with the confidence measure criteria allows for striking a good balance between high accuracy and fast response time.

Iterative confidence relabeling with deep ConvNets for organ segmentation with partial labels.

Training deep ConvNets requires large labeled datasets. However, collecting pixel-level labels for medical image segmentation is very expensive and requires a high level of expertise. In addition, most existing segmentation masks provided by clinical experts focus on specific anatomical structures. In this paper, we propose a method dedicated to handle such partially labeled medical image datasets. We propose a strategy to identify pixels for which labels are correct, and to train Fully Convolutional Neural Networks with a multi-label loss adapted to this context. In addition, we introduce an iterative confidence self-training approach inspired by curriculum learning to relabel missing pixel labels, which relies on selecting the most confident prediction with a specifically designed confidence network that learns an uncertainty measure which is leveraged in our relabeling process. Our approach, INERRANT for Iterative coNfidencE Relabeling of paRtial ANnoTations, is thoroughly evaluated on two public datasets (TCAI and LITS), and one internal dataset with seven abdominal organ classes. We show that INERRANT robustly deals with partial labels, performing similarly to a model trained on all labels even for large missing label proportions. We also highlight the importance of our iterative learning scheme and the proposed confidence measure for optimal performance. Finally we show a practical use case where a limited number of completely labeled data are enriched by publicly available but partially labeled data.

Learning and Selecting Confidence Measures for Robust Stereo Matching.

We present a robust approach for computing disparity maps with a supervised learning-based confidence prediction. This approach takes into consideration following features. First, we analyze the characteristics of various confidence measures in the random forest framework to select effective confidence measures depending on the characteristics of the training data and matching strategies, such as similarity measures and parameters. We then train a random forest using the selected confidence measures to improve the efficiency of confidence prediction and to build a better prediction model. Second, we present a confidence-based matching cost modulation scheme, based on predicted confidence values, to improve the robustness and accuracy of the (semi-) global stereo matching algorithms. Finally, we apply the proposed modulation scheme to popularly used algorithms to make them robust against unexpected difficulties that could occur in an uncontrolled environment using challenging outdoor datasets. The proposed confidence measure selection and cost modulation schemes are experimentally verified from various perspectives using the KITTI and Middlebury datasets.

Reinforced confidence in self-training for a semi-supervised medical data classification

The implication of semi-supervised method has become crucial to automate tasks that require the manual human expertise for data labelling, the advantage of this method resides in the fact that they require a low amount of labelled information. In this work, we are particularly interested in self-training paradigm. These techniques use the same principle as those of supervised techniques, but with a confidence measure that allows only a selection of the most confident samples. We propose a novel self-training algorithm named reinforced confidence in self-training (R-COSET) based on an iterative process. In each iteration the learned hypothesis can be improved by confidence data, where the proposed confidence measure is reinforced by two confidence levels in order to increase the robustness of the self-training process. Experiments show that the introduction of the second level of the neighbourhood graph in confidence measure is beneficial and that R-COSET can effectively improve classification performance.

Automatic face recognition system based on the SIFT features

The main goal of this paper is to propose and implement an experimental fully automatic face recognition system which will be used to annotate photographs during insertion into a database. Its main strength is to successfully process photos of a great number of different individuals taken in a totally uncontrolled environment. The system is available for research purposes for free. It uses our previously proposed SIFT based Kepenekci approach for the face recognition, because it outperforms a number of efficient face recognition approaches on three large standard corpora (namely FERET, AR and LFW). The next goal is proposing a new corpus creation algorithm that extracts the faces from the database and creates a facial corpus. We show that this algorithm is beneficial in a preprocessing step of our system in order to create good quality face models. We further compare the performance of our SIFT based Kepenekci approach with the original Kepenekci method on the created corpus. This comparison proves that our approach significantly outperforms the original one. The last goal is to propose two novel supervised confidence measure methods based on a posterior class probability and a multi-layer perceptron to identify incorrectly recognized faces. These faces are then removed from the recognition results. We experimentally validated that the proposed confidence measures are very efficient and thus suitable for our task.

Confidence Measure Using Composite Features for Eye Detection in a Face Recognition System

We propose a new confidence measure to evaluate the eye detection results and combine two different eye detectors. The confidence for the results of eye detection is measured by the distances from the test sample and the positive samples, where the distance is calculated in the composite feature space. By using the proposed confidence measure, we construct a hybrid detector by combining two different detectors, which are complementary to each other. The experimental results show that the proposed detector provides more accurate eye detection results and consequently results in better face recognition rates compared to when using an individual eye detector.

A robust two‐stage face recognition system with localisation error compensation

In practical systems, face recognition under unconstrained conditions is a very challenging task, where their input images are first pre‐processed and initially aligned by a face detection algorithm. However, there are still some residual localisation errors after the initial alignment. If we do not take these errors into account, the recognition performance should be greatly degraded for most face recognition algorithms. Generally, when designing a practical face recognition system, we need to compromise the capability of residual error tolerance and the discriminating capability. Although it is feasible to apply an iterative alignment algorithm to fine‐tune alignment, it will increase the computation load significantly. In this study, we propose an adaptive two‐stage face recognition system consisting of two block‐based recognition stages with a relatively larger cell size (i.e. the size of local regions) in the first stage to provide sufficient tolerance for geometric errors followed by a smaller one in the second stage to accurately evaluate a most probable candidate subset, which is adaptively determined according to the proposed confidence measure. In addition, an iterative gradient‐based alignment algorithm is incorporated into the two‐stage system to refine the alignment such that the recognition performance can be improved and the computation load can be saved simultaneously.

Confidence Measure Based on Context Consistency Using Word Occurrence Probability and Topic Adaptation for Spoken Term Detection

In this paper, we propose a novel confidence measure to improve the performance of spoken term detection (STD). The proposed confidence measure is based on the context consistency between a hypothesized word and its context in a word lattice. The main contribution of this paper is to compute the context consistency by considering the uncertainty in the results of speech recognition and the effect of topic. To measure the uncertainty of the context, we employ the word occurrence probability, which is obtained through combining the overlapping hypotheses in a word posterior lattice. To handle the effect of topic, we propose a method of topic adaptation. The adaptation method firstly classifies the spoken document according to the topics and then computes the context consistency of the hypothesized word with the topic-specific measure of semantic similarity. Additionally, we apply the topic-specific measure of semantic similarity by two means, and they are performed respectively with the information of the top-1 topic and the mixture of all topics according to topic classification. The experiments conducted on the Hub-4NE Mandarin database show that both the occurrence probability of context word and the topic adaptation are effective for the confidence measure of STD. The proposed confidence measure performs better compared with the one ignoring the uncertainty of the context or the one using a non-topic method.

Probabilistic voxel mapping using an adaptive confidence measure of stereo matching

In this paper, we describe a probabilistic voxel mapping algorithm using an adaptive confidence measure of stereo matching. Most of the 3D mapping algorithms based on stereo matching usually generate a map formed by point cloud. There are many reconstruction errors. The reconstruction errors are due to stereo reconstruction error factors such as calibration errors, stereo matching errors, and triangulation errors. A point cloud map with reconstruction errors cannot accurately represent structures of environments and needs large memory capacity. To solve these problems, we focused on the confidence of stereo matching and probabilistic representation. For evaluation of stereo matching, we propose an adaptive confidence measure that is suitable for outdoor environments. The confidence of stereo matching can be reflected in the probability of restoring structures. For probabilistic representation, we propose a probabilistic voxel mapping algorithm. The proposed probabilistic voxel map is a more reliable representation of environments than the commonly used voxel map that just contains the occupancy information. We test the proposed confidence measure and probabilistic voxel mapping algorithm in outdoor environments.

On Dynamic Stream Weighting for Audio-Visual Speech Recognition

The integration of audio and visual information improves speech recognition performance, specially in the presence of noise. In these circumstances it is necessary to introduce audio and visual weights to control the contribution of each modality to the recognition task. We present a method to set the value of the weights associated to each stream according to their reliability for speech recognition, allowing them to change with time and adapt to different noise and working conditions. Our dynamic weights are derived from several measures of the stream reliability, some specific to speech processing and others inherent to any classification task, and take into account the special role of silence detection in the definition of audio and visual weights. In this paper, we propose a new confidence measure, compare it to existing ones, and point out the importance of the correct detection of silence utterances in the definition of the weighting system. Experimental results support our main contribution: the inclusion of a voice activity detector in the weighting scheme improves speech recognition over different system architectures and confidence measures, leading to an increase in performance more relevant than any difference between the proposed confidence measures.

Phonetic Distance Based Confidence Measure

This letter presents a novel confidence measure for the purpose of improving user performance in Spoken Document Retrieval (SDR). The proposed confidence measure is based on the phonetic distance between subword models, employing an anti-model which is determined to be discriminative to a target model using offline training data. As an advancement from our previous work, the proposed method employs separate phonetic similarity knowledge for vowels and consonants, resulting in more reliable performance over diverse SDR recorded speech conditions. A transcript reliability estimator is also presented, with evaluation as an application of the proposed confidence measure. Analysis on a variety of corpora including background noise, frequency band-restrictions, and a range of real-life conditions, shows that the proposed confidence measure is more reliable in detecting corrupted speech due to acoustic conditions or an unarticulated speaking style, providing a higher correlation to word error rate (WER). The proposed confidence measure is effective in increasing transcript reliability estimation performance with a 16.21% relative improvement.

Reliable block motion estimation through the confidence measure of error surface

The conventional coarse-to-fine search algorithms for block matching motion estimation reduce the set of possible displacements for locating the motion vector. Nearly all of these algorithms rely on the assumption: the MAD distortion function increases monotonically as the search location moves away from the global minimum. Obviously, this assumption essentially requires that the MAD error surface be unimodal over the search window. Unfortunately, this is usually not true in real-world video signals. In this paper, we formulate a criterion to check the confidence of unimodal error surface over the search window. The proposed confidence measure of error surface (CMES) would be a good measure for identifying whether the searching should be continued or not. It is found that this proposed measure is able to strengthen the conventional fast search algorithms for block matching motion estimation. Experimental results show that, as compared to the conventional approach, the new algorithm through the CMES is more robust, produces smaller motion compensation errors, and has simple computational complexity.