Labeling Errors Research Articles

Graph Confident Learning for Software Vulnerability Detection

Code vulnerability exposes millions of software to the possibility of being attacked, as evidence every year on increasing reports of security issues, such as information leaks, system compromise, and denial of service. Despite with many vulnerability detection models proposed so far, their effectiveness is still limited due to the ignorance of syntactic structural information analysis in source code and the improper handling of labeling errors. To address these issues, we propose the Graph Confident Learning for Software Vulnerability Detection (GCL4SVD) model, a machine learning model to detect software vulnerability in the development phase. It comprises two components: code graph embedding and graph confident learning denoising. To address the syntactic structural information analysis limitation, the code graph embedding component extracts the structure and semantic information of source code with a sliding window mechanism, and then encodes source code into a graph structure to capture the patterns and characteristics of code vulnerabilities. Additionally, the graph confident learning denoising component identifies labeling errors to improve the quality of training set. Experimental results show that GCL4SVD outperforms the state-of-the-art vulnerability detection models on four open source datasets by 3.7%, 3.3%, 2.5%, 0.8% in terms of Accuracy, respectively, and by 10.2%, 21.8%, 8.2%, 11.2% in terms of F1-score.

CUS3D: A New Comprehensive Urban-Scale Semantic-Segmentation 3D Benchmark Dataset

With the continuous advancement of the construction of smart cities, the availability of large-scale and semantically enriched datasets is essential for enhancing the machine’s ability to understand urban scenes. Mesh data have a distinct advantage over point cloud data for large-scale scenes, as they can provide inherent geometric topology information and consume less memory space. However, existing publicly available large-scale scene mesh datasets are limited in scale and semantic richness and do not cover a wide range of urban semantic information. The development of 3D semantic segmentation algorithms depends on the availability of datasets. Moreover, existing large-scale 3D datasets lack various types of official annotation data, which hinders the widespread applicability of benchmark applications and may cause label errors during data conversion. To address these issues, we present a comprehensive urban-scale semantic segmentation benchmark dataset. It is suitable for various research pursuits on semantic segmentation methodologies. This dataset contains finely annotated point cloud and mesh data types for 3D, as well as high-resolution original 2D images with detailed 2D semantic annotations. It is constructed from a 3D reconstruction of 10,840 UVA aerial images and spans a vast area of approximately 2.85 square kilometers that covers both urban and rural scenes. The dataset is composed of 152,298,756 3D points and 289,404,088 triangles. Each 3D point, triangular mesh, and the original 2D image in the dataset are carefully labeled with one of the ten semantic categories. Six typical 3D semantic segmentation methods were compared on the CUS3D dataset, with KPConv demonstrating the highest overall performance. The mIoU is 59.72%, OA is 89.42%, and mAcc is 97.88%. Furthermore, the experimental results on the impact of color information on semantic segmentation suggest that incorporating both coordinate and color features can enhance the performance of semantic segmentation. The current limitations of the CUS3D dataset, particularly in class imbalance, will be the primary target for future dataset enhancements.

MisDetect: Iterative Mislabel Detection using Early Loss

Supervised machine learning (ML) models trained on data with mislabeled instances often produce inaccurate results due to label errors. Traditional methods of detecting mislabeled instances rely on data proximity, where an instance is considered mislabeled if its label is inconsistent with its neighbors. However, it often performs poorly, because an instance does not always share the same label with its neighbors. ML-based methods instead utilize trained models to differentiate between mislabeled and clean instances. However, these methods struggle to achieve high accuracy, since the models may have already overfitted mislabeled instances. In this paper, we propose a novel framework, MisDetect, that detects mislabeled instances during model training. MisDetect leverages the early loss observation to iteratively identify and remove mislabeled instances. In this process, influence-based verification is applied to enhance the detection accuracy. Moreover, MisDetect automatically determines when the early loss is no longer effective in detecting mislabels such that the iterative detection process should terminate. Finally, for the training instances that MisDetect is still not certain about whether they are mislabeled or not, MisDetect automatically produces some pseudo labels to learn a binary classification model and leverages the generalization ability of the machine learning model to determine their status. Our experiments on 15 datasets show that MisDetect outperforms 10 baseline methods, demonstrating its effectiveness in detecting mislabeled instances.

Benchmarking Automated Machine Learning (AutoML) Frameworks for Object Detection

Automated Machine Learning (AutoML) is a subdomain of machine learning that seeks to expand the usability of traditional machine learning methods to non-expert users by automating various tasks which normally require manual configuration. Prior benchmarking studies on AutoML systems—whose aim is to compare and evaluate their capabilities—have mostly focused on tabular or structured data. In this study, we evaluate AutoML systems on the task of object detection by curating three commonly used object detection datasets (Open Images V7, Microsoft COCO 2017, and Pascal VOC2012) in order to benchmark three different AutoML frameworks—namely, Google’s Vertex AI, NVIDIA’s TAO, and AutoGluon. We reduced the datasets to only include images with a single object instance in order to understand the effect of class imbalance, as well as dataset and object size. We used the metrics of the average precision (AP) and mean average precision (mAP). Solely in terms of accuracy, our results indicate AutoGluon as the best-performing framework, with a mAP of 0.8901, 0.8972, and 0.8644 for the Pascal VOC2012, COCO 2017, and Open Images V7 datasets, respectively. NVIDIA TAO achieved a mAP of 0.8254, 0.8165, and 0.7754 for those same datasets, while Google’s VertexAI scored 0.855, 0.793, and 0.761. We found the dataset size had an inverse relationship to mAP across all the frameworks, and there was no relationship between class size or imbalance and accuracy. Furthermore, we discuss each framework’s relative benefits and drawbacks from the standpoint of ease of use. This study also points out the issues found as we examined the labels of a subset of each dataset. Labeling errors in the datasets appear to have a substantial negative effect on accuracy that is not resolved by larger datasets. Overall, this study provides a platform for future development and research on this nascent field of machine learning.

Supervised diagnostic classification of cognitive attributes using data augmentation.

Over recent decades, machine learning, an integral subfield of artificial intelligence, has revolutionized diverse sectors, enabling data-driven decisions with minimal human intervention. In particular, the field of educational assessment emerges as a promising area for machine learning applications, where students can be classified and diagnosed using their performance data. The objectives of Diagnostic Classification Models (DCMs), which provide a suite of methods for diagnosing students' cognitive states in relation to the mastery of necessary cognitive attributes for solving problems in a test, can be effectively addressed through machine learning techniques. However, the challenge lies in the latent nature of cognitive status, which makes it difficult to obtain labels for the training dataset. Consequently, the application of machine learning methods to DCMs often assumes smaller training sets with labels derived either from theoretical considerations or human experts. In this study, the authors propose a supervised diagnostic classification model with data augmentation (SDCM-DA). This method is designed to utilize the augmented data using a data generation model constructed by leveraging the probability of correct responses for each attribute mastery pattern derived from the expert-labeled dataset. To explore the benefits of data augmentation, a simulation study is carried out, contrasting it with classification methods that rely solely on the expert-labeled dataset for training. The findings reveal that utilizing data augmentation with the estimated probabilities of correct responses substantially enhances classification accuracy. This holds true even when the augmentation originates from a small labeled sample with occasional labeling errors, and when the tests contain lower-quality items that may inaccurately measure students' true cognitive status. Moreover, the study demonstrates that leveraging augmented data for learning can enable the successful classification of students, thereby eliminating the necessity for specifying an underlying response model.

Sample Management Errors in a Multispecialty Hospital-based Blood Bank

ABSTRACT Background and Objectives: Specimen labeling errors along with incorrect patient identification have been described as among the most complex and demanding occurrences in laboratory diagnostics. The study aimed to describe the rates of sample identification and labeling errors in the blood center of a multispecialty hospital and identify possible measures to reduce them. Methods: A retrospective document review and analysis of sample identification and labeling errors were conducted in a Blood Bank Laboratory in Saudi Arabia for 4 years. The quality assurance database from the laboratory information system was used to access and review all relevant information. The one-way analysis of variance was employed to check the statistical differences between the laboratory errors and other variables in the study. The significance level was set at P < 0.05. Results: Overall, 74,279 samples and laboratory requests were received, with over 3107 canceled requests and rejected samples. An error rate of 0.43% was due to incorrect identification and labeling. This accounts for 10.2% of the total canceled tests. The annual labeling error ranges from 0.26% to 0.73%. Primarily, the errors were due to incomplete data (0.37%) and were noted as the leading type of error in all the departments. More than half (51.3%) of the errors happened in type and screen tests, whereas a significant proportion was also noted in crossmatch red blood cells (31.9%). Conclusions: The rate of sample identification and labeling errors in this study was comparable to established data and was primarily due to incomplete data or labels. Identification and labeling errors are most common in type and screen testing and crossmatch. Carefully monitoring specimen labeling quality continually can lower specimen labeling errors and determine improvements.

Automatic Fingerprint Data Labeling Using WiFi Signal and Smartphone Camera for Indoor Positioning

WiFi fingerprinting has been one of the most practical approaches for implementing an indoor positioning system. However, the need to measure location labels for fingerprint data has hindered the deployment of WiFi fingerprint‐based positioning systems. To reduce the cost of location labeling, several methods for automatically labeling locations or bypassing location labeling have been proposed. However, these methods still need to measure location labels for partial fingerprint data, or the estimated location labels are inaccurate. In this paper, we propose Heimdall, an automatic fingerprint data labeling method using a smartphone camera and WiFi signal for an indoor positioning system. Since Heimdall utilizes camera geometry for accurate automatic labeling, it aims to find the location labels of unlabeled fingerprint data composed of WiFi received signal strengths (RSSs) and photos collected using smartphones in multiple reference points (RPs). Using structure from motion (SfM) with all photos in the unlabeled fingerprint dataset, the location and orientation of the cameras can be recovered on an arbitrary world coordinate system. However, the world coordinate system derived by SfM is warped. In other words, it is different from the real‐world coordinate system where we express the location labels since SfM has similarity ambiguity, including scale, rotation, and translation ambiguity. Heimdall finds the location labels by estimating and applying an unwarping transformation that removes those ambiguities. We use two types of geometric constraints for estimating the unwarping transformation. First, several geometric entities in the warped‐world coordinates and real‐world coordinates are extracted from the photos and prior information about the indoor layout. The unwarping transformation transforms the geometric entities in warped coordinates to those in real‐world coordinates. Second, we utilize the log‐distance pathloss model with RSSs to derive the constraint about distances from the location label to the access points. We construct the optimization problem subject to those constraints to estimate the unwarping transformation. To evaluate the performance of Heimdall, we performed several experiments. Heimdall yields a mean location labeling error of 0.12 m and a standard deviation of location labeling error of 0.06 m for the best case in the experiment. These results demonstrate that a fingerprint‐based indoor positioning system can use the location labels estimated by Heimdall without performance degradation.

항공보안 종사사의 조직문화가 임파워먼트, 신뢰와 조직유효성에 미치는 영향

The purpose of this study is to investigate how organizational culture, empowerment, trust, and organizational effectiveness are having cause-and-effect relationship each other in order for aviation security workers to effectively manage the institute.
 From July 2022 to September 2022,the survey was distributed and collected for 300 aviation security workers and analyzed through a cluster sampling method. Among the collected questionnaires, 272 copies were finally used for analysis, excluding labeling errors, omissions, and duplicates. For statistical analysis of the collected data, the STATA 14.0 program was used to test the research hypothesis.
 The results of regression analysis on the relationship among organizational culture, empowerment, trust, and organizational effectiveness are as follows. First, it was found that organizational culture had a positive (+) effect on organizational effectiveness. Second, it was found that organizational culture had a positive (+) effect on trust. Third, it was found that organizational culture had a positive (+) effect on empowerment. Fourth, empowerment was found to have a positive (+) effect on trust. Fifth, empowerment was found to have a positive (+) effect on organizational effectiveness Finally, trust was found to have a positive (+) effect on organizational effectiveness.

Functional Labeled Optimal Partitioning

Peak detection is a problem in sequential data analysis that involves differentiating regions with higher counts (peaks) from regions with lower counts (background noise). It is crucial to correctly predict areas that deviate from the background noise, in both the train and test sets of labels. Dynamic programming changepoint algorithms have been proposed to solve the peak detection problem by constraining the mean to alternatively increase and then decrease. The current constrained changepoint algorithms only create predictions on the test set, while completely ignoring the train set. Changepoint algorithms that are both accurate when fitting the train set, and make predictions on the test set, have been proposed but not in the context of peak detection models. We propose to resolve these issues by creating a new dynamic programming algorithm, FLOPART, that has zero train label errors, and is able to provide highly accurate predictions on the test set. We provide an empirical analysis that shows FLOPART has a similar time complexity while being more accurate than the existing algorithms in terms of train and test label errors.

Impact of technology interventions on technology sensitive dispensing errors

Abstract Introduction Technology has the potential to impact significantly on dispensing error rates. Technology systems are already widely established in UK hospitals such as automated dispensing systems (robots) and electronic prescribing systems. A single technology system cannot achieve error reduction across all dispensing error types, for example, using a robot can only have impact on picking (contents) errors and not labelling errors. Quantifying the technology sensitive errors after implementation of technology systems enables us to focus on the highest risk error types and combine the most useful risk reduction strategies1. Aim To quantify the technology sensitive error rate found in dispensed items after different technological interventions. Methods Prevented dispensing errors were collected at the Royal Cornwall Hospital in 2021 over four days. Errors were documented at the final checking stage of dispensing on a standardised data collection tool used as part of the hospital usual governance processes. Errors were categorised by error type and the technology interventions involved in the dispensing process; robot picked, robot picked and labelled, or entered by barcode. An error was defined as technology sensitive if that technology could have had a direct impact on that error type2. Data was analysed in Excel. Chi-square significance testing was undertaken on those cohorts that were sized appropriately. Ethics approval was not required as the study met the definition of a service evaluation. Results 2461 items were dispensed in the study period. The overall dispensing error rate for the period was 2.64% (N=65). Labelling errors represented 66.9% of errors whereas contents errors 33.1%. The overall difference in the error rate in items picked by the robot versus manually was not significant. However, when excluding errors that are not sensitive to the technology e.g., label direction errors, the results become significant with a reduction of 0.86% (manually picked) to 0.05% (robot picked). Items that were entered into the stock control system by barcode showed a technology sensitive error reduction of 0.31% to 0%. It was significant that items that were packed down were associated with more dispensing errors (4.37%) compared with those dispensed as original packs (1.14%). Due to small sample size, the reduction in technology sensitive error rate from 0.34%-0.2% when using robotic labelling with robotic picking could not be tested for significance. Discussion/Conclusion Translation of medication orders from a prescription to a stock control system is one of the highest risk areas for error substantiated by labelling errors contributing to 66.9% of the overall errors in this study. A study has demonstrated significant reduction in dispensing error rate using integrated prescribing systems and stock control systems1. This study and previous work found bar-coded data entry to be similarly effective3. Part pack, manual dispensing shows the greatest risk of dispensing errors and therefore technologies that can impact in this area would likely have great benefit. The study was limited by sample size in some of the technology cohorts. With the correct combination of technologies we aim to reduce errors at the point of dispensing to negligible levels.

베이지안 방식의 데이터 증강을 이용한 지도학습 기반 인지진단평가 연구

Objectives The purposes of this study were to propose and validate a method for applying supervised learning for cognitive diagnostic assessment by augmenting expert-labeled dataset using Bayesian parameter estimation techniques. Methods The method was evaluated with a simulation study and an application to real data. The simulation study was conducted to compare the diagnostic accuracy of neural networks trained with both ‘Bayesian-based aug-mented dataset’ and ‘expert-labeled dataset’ across four factors: ‘item quality’, ‘expert labeling error rate’, ‘size of the expert labeled dataset’, and ‘size of the augmented dataset’. Additionally, the real data consists of the binary responses of 936 middle-school students to 24 arithmetic problems on linear equations and linear inequalities. Results The performance of supervised learning trained on the ‘Bayesian-based augmented dataset’ showed significant improvements in all conditions compared to solely using the ‘expert-labeled dataset’. The effect of the labeling error rate and the size of the expert-labeled dataset was trivial. As the quality of the items increased and the size of the augmented dataset grew, accuracy improved, though beyond a certain size N=3,000 of the aug-mented dataset, the difference was negligible. For the real data on linear equations and inequalities, supervised learning trained on the ‘Bayesian-based augmented dataset’ showed enhanced accuracy over just on the ‘ex-pert-labeled dataset’. It is also evident that students understand solving linear equations better than solving linear inequalities. Conclusions When performing supervised learning based cognitive diagnosis with a small expert-labeled data-set, the diagnostic performance can be enhanced through Bayesian-based data augmentation method. Moreover, students tend to find understanding inequalities more challenging than equations, suggesting the need for opportunities to master cognitive attributes related to inequalities.

Geographical Information System Enhancement Using Active Learning-Enhanced Semantic Segmentation

Images captured by drones are increasingly used in various fields, including geographic information management. This study evaluates a procedure that incorporates active learning semantic segmentation for verifying the building registration ledger. Several semantic segmentation techniques were evaluated to extract building information, with ResNet identified as the most effective method for accurately recognizing building roofs. Using active learning, the training data were refined by removing instances with low similarity, leading to improved network performance of the model. The procedure was demonstrated to identify discrepancies between the building information system and the inferred label images, as well as to detect labeling errors on a training dataset. Through this research, the geographic information system dataset is enhanced with minimal human oversight, offering significant potential for urban planning and building detection advancements.

Transfer learning for hate speech detection in social media

Today, the internet is an integral part of our daily lives, enabling people to be more connected than ever before. However, this greater connectivity and access to information increase exposure to harmful content, such as cyber-bullying and cyber-hatred. Models based on machine learning and natural language offer a way to make online platforms safer by identifying hate speech in web text autonomously. However, the main difficulty is annotating a sufficiently large number of examples to train these models. This paper uses a transfer learning technique to leverage two independent datasets jointly and builds a single representation of hate speech. We build an interpretable two-dimensional visualization tool of the constructed hate speech representation—dubbed the Map of Hate—in which multiple datasets can be projected and comparatively analyzed. The hateful content is annotated differently across the two datasets (racist and sexist in one dataset, hateful and offensive in another). However, the common representation successfully projects the harmless class of both datasets into the same space and can be used to uncover labeling errors (false positives). We also show that the joint representation boosts prediction performances when only a limited amount of supervision is available. These methods and insights hold the potential for safer social media and reduce the need to expose human moderators and annotators to distressing online messaging.

Identification of enteral nutrition errors in a single-center quality-improvement audit.

Enteral nutrition (EN) therapy is a multistep process including evaluation, prescription, procurement, dispensing, labeling, administration, and monitoring. EN therapy is prone to human errors, but these are poorly defined in the literature. The purpose of this study was to audit EN administration practices to quantify errors of execution and identify which components of the EN order were labeled, administered, or documented incorrectly. On 2 nonconsecutive days, we identified all hospitalized patients with active EN orders and prospectively collected the following information: EN formula hanging/documented, formula hang time, infusion rate/documented rate (continuous EN), infused volume and documented schedule (intermittent EN), and EN modular documentation. Mismatches to the EN order were considered errors. We reviewed 1 month of hospital EN-related safety events for comparison. Of 1045 data points collected from 160 patients, we identified 275 errors of execution: 135 labeling errors and 140 administration errors. The most common were hang time >48 h (85%), wrong number of modulars documented (48%), and wrong infusion rate (19%). We found one reported safety event (wrong formula delivered but not infused). We identified a 15.9% error rate in EN order execution/documentation and 14% compliance with documentation of 48-h hang time. Errors (safety events) were grossly underreported. This highlighted several areas of opportunity to improve current EN use process, consistent with previous research on EN and oral nutrition supplement administration. Based on our findings, we plan to recommend implementation of EN barcoding at our institution, to model the familiar medication administration record.

Evaluation of intraoperative neuromonitoring (IONM) data with the Mainz IONM Quality Assurance and Analysis tool

Abstract Background Intraoperative neuromonitoring is widely used in thyroid and parathyroid surgery to prevent unilateral and especially bilateral recurrent nerve paresis. Reference values for amplitude and latency for the recurrent laryngeal nerve and vagus nerve have been published. However, data quality measures that exclude errors of the underlying intraoperative neuromonitoring (IONM) data (immanent software errors, false data labelling) before statistical analysis have not yet been implemented. Methods The authors developed an easy-to-use application (the Mainz IONM Quality Assurance and Analysis tool) using the programming language R. This tool allows visualization, automated and manual correction, and statistical analysis of complete raw data sets (electromyogram signals of all stimulations) from intermittent and continuous neuromonitoring in thyroid and parathyroid surgery. The Mainz IONM Quality Assurance and Analysis tool was used to evaluate IONM data generated and exported from ‘C2’ and ‘C2 Xplore’ neuromonitoring devices (inomed Medizintechnik GmbH) after surgery. For the first time, reference values for latency and amplitude were calculated based on ‘cleaned’ IONM data. Results Intraoperative neuromonitoring data files of 1935 patients consecutively operated on from June 2014 to May 2020 were included. Of 1921 readable files, 34 were excluded for missing data labelling. Automated plausibility checks revealed: less than 3 per cent device errors for electromyogram signal detection; 1138 files (approximately 60 per cent) contained potential labelling errors or inconsistencies necessitating manual review; and 915 files (48.5 per cent) were indeed erroneous. Mean(s.d.) reference onset latencies for the left vagus nerve, right vagus nerve, recurrent laryngeal nerve, and external branch of the superior laryngeal nerve were 6.8(1.1), 4.2(0.8), 2.5(1.1), and 2.1(0.5) ms, respectively. Conclusion Due to high error frequencies, IONM data should undergo in-depth review and multi-step cleaning processes before analysis to standardize scientific reporting. Device software calculates latencies differently; therefore reference values are device-specific (latency) and/or set-up-specific (amplitude). Novel C2-specific reference values for latency and amplitude deviate considerably from published values.

Prospective Audit of Blood Transfusion Practices among Nurses and Role of Education and Intervention at a Tertiary Care Hospital

Background and Objectives: Periodical internal audits evaluate the process flow of transfusion which arrests the spectrum of adverse events. The objective of the study was to provide evidence that blood is being ordered, handled, and administered according to transfusion guidelines and to highlight the deviations with the secondary objective of improving knowledge and practices in nursing officers. Methods: Nursing officers were audited on 50 patients each in P1 and P2 phase. Training of 243 nursing staff was done after P1 phase. Training comprised education and interventions over 45 days – 18 didactic lectures and 36 bedside classes. Nurses were audited both before and after the completion of the training. Results: One hundred and forty-five nurses had knowledge and practice score in between 11 and 15 in P1 which improved to 243 (100%) in P2. Transfusion order for one patient was not written in P1, whereas 100% compliance was seen in P2. Noncompliance in taking consent for transfusion improved from 5 to 1 in P2. Blood request error and sample labeling error reduced from 8% to 0% and 4% to 0%, respectively, in P2. Sample labeling was inappropriate in 2 patients in P1 which improved to nil in P2. Checking of vital signs check and intravenous cannula patency improved from 96% to 98%. Crosscheck of packed red blood cell nursing officers before transfusion reduced from 8% to 0% and from 30% to 2% among medical officer (P < 0.05). Delay in the initiation of transfusion, reduced from 7 to 2 patients. Conclusion: Timely audit of transfusion practices reduces mistakes and errors hence improved patient care.

Wind Turbine Blade Early Fault Detection With Faulty Label Unknown and Labeling Bias

In practical industrial applications, the need for a large number of accurately labeled training samples is a significant challenge for fault detection tasks. However, labeling all training samples is expensive and prone to labeling errors, especially for early fault detection of wind turbine blades. This paper proposes a labeling bias (LB) hypothesis. Assuming the labeler only needs to label parts of normal samples that are easy to judge, we design a probability ratio least-squares importance fitting (PRL-SIF) method based on variable homogeneity. Unlike other state-of-the-art positive unlabeled (PU) learning methods, PRL-SIF does not require knowledge of the class priors to achieve training. Furthermore, to better handle the multi-dimensional time-series data of wind turbines, we provide a data preprocessing method based on functional analysis to achieve time series feature extraction and dimensionality reduction. The effectiveness and robustness of the proposed method are verified on 23 real-world wind turbine datasets. Experimental results show that the proposed method can achieve nearly 90% accuracy while only labeling 20% of normal samples.

Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria

Although many fairness criteria have been proposed to ensure that machine learning algorithms do not exhibit or amplify our existing social biases, these algorithms are trained on datasets that can themselves be statistically biased. In this paper, we investigate the robustness of existing (demographic) fairness criteria when the algorithm is trained on biased data. We consider two forms of dataset bias: errors by prior decision makers in the labeling process, and errors in the measurement of the features of disadvantaged individuals. We analytically show that some constraints (such as Demographic Parity) can remain robust when facing certain statistical biases, while others (such as Equalized Odds) are significantly violated if trained on biased data. We provide numerical experiments based on three real-world datasets (the FICO, Adult, and German credit score datasets) supporting our analytical findings. While fairness criteria are primarily chosen under normative considerations in practice, our results show that naively applying a fairness constraint can lead to not only a loss in utility for the decision maker, but more severe unfairness when data bias exists. Thus, understanding how fairness criteria react to different forms of data bias presents a critical guideline for choosing among existing fairness criteria, or for proposing new criteria, when available datasets may be biased.

Deep labeller: automatic bounding box generation for synthetic violence detection datasets

Manually labelling datasets for training violence detection systems is time-consuming, expensive, and labor-intensive. Mind wandering, boredom, and short attention span can also cause labelling errors. Moreover, collecting and distributing sensitive images containing violence has ethical implications. Automation is the future for labelling sensitive image datasets. Deep labeller is a two-stage Deep Learning (DL) method that uses pre-trained DL object detection methods on MS-COCO for automatic labelling. The Deep Labeller method labels violent and nonviolent images in WVD and USI. In stage 1, WVD generates weak labels using synthetic images. In stage 2, the Deep labeller method is retrained on weak labels. USI dataset is used to test our method on real-world violence. Deep labeller generated weak and strong labels with an IoU of 0.80036 in stage 1 and 0.95 in stage 2 on the WVD. Automatically generated labels. To test our method’s generalisation power, violent and nonviolent image labels on USI dataset had a mean IoU of 0.7450.

Combining seeded region growing and k-nearest neighbours for the segmentation of routinely acquired spatio-temporal image data

PurposeThe acquisition conditions of medical imaging are often precisely defined, leading to a high homogeneity among different data sets. Nonetheless, outliers or artefacts still appear and need to be reliably detected to ensure a reliable diagnosis. Thus, the algorithms need to handle small sample sizes especially, when working with domain specific imaging modalities.MethodsIn this work, we suggest a pipeline for the detection and segmentation of light pollution in near-infrared fluorescence optical imaging (NIR-FOI), based on a small sample size. NIR-FOI produces spatio-temporal data with two spatial and one temporal dimension. To calculate a two-dimensional light pollution map for the entire image stack, we combine region growing and k-nearest neighbours (kNN), which classifies pixels into fore- and background by its entire temporal component. Thus, decision-making on reduced data is omitted.ResultsWe achieved a F_1 score of 0.99 for classifying a data set as light polluted or pollution-free. Additionally, we reached a total F_1 score of 0.90 for detecting regions of interest within the polluted data sets. Finally, an average Dice’s coefficient measuring the segmentation performance over all polluted data sets of 0.80 was accomplished.ConclusionsA Dice’s coefficient of 0.80 for the area segmentation does not seem perfect. However, there are two main factors, besides true prediction errors, lowering the score: Segmentation mistakes on small areas lead to a rapid decrease in the score and labelling errors due to complex data. However, in combination with the light-polluted data set and pollution area detection, these results can be considered successful and play a key role in our general goal: Exploiting NIR-FOI for the early detection of arthritis within hand joints.