Performance Of Various Metrics Research Articles

The inconclusive category, entropy, and forensic firearm identification

There has been extensive recent discussion of the difficulty in estimating meaningful error rates in forensic firearms examinations, and other areas of pattern evidence. The 2016 President’s Council of Advisors on Science and Technology (PCAST) report was clear in criticizing many forensic disciplines as lacking the types of studies that would provide error rate measurements seen in other scientific fields. However, there is a substantial lack of consensus on the approach to measuring an “error rate” for fields such as forensic firearm examination that include in the conclusion scale the “inconclusive” category, as occurs in the Association of Firearm and Tool Mark Examiners (AFTE) Range of Conclusions and many other such fields. Many authors appear to assume the error rate calculated in the binary decision model is the only appropriate way to report errors, but there have been attempts made to adapt the error rate from the binary decision model to scientific fields in which the inconclusive category is viewed as a meaningful outcome of the examination process. In this study we present three neural networks of differing complexity and performance trained to classify the outlines of ejector marks on cartridge cases fired from different firearm models, as a model system for examining the performance of various metrics of error in systems using the inconclusive category. We also discuss an entropy, or information, based method to assess the similarity of classifications to ground truth that is applicable to range of conclusion scales, even when the inconclusive category is used.

User perception for dynamic video resolution change using VVC

We define experiments that measure user perception when video resolution changes dynamically. Versatile Video Coding (VVC) standard was recently finalized and it includes a reference picture resampling (RPR) tool. VVC RPR supports changing spatial resolution in a coded video sequence on a per picture basis. VVC RPR defines the downsampling and upsampling filters to be used when changing resolution. This paper provides results from subjective evaluation when VVC RPR is used for part of the video sequence to dynamically change resolution. The experiments use different QP values (or bitrates), different RPR scale factors and different highest original spatial resolutions. The results compare how users perceive video coded using VVC RPR for some pictures compared to an anchor which does not use RPR. In addition to the subjective results, we also describe performance of various metrics including PSNR, VMAF and MS-SSIM. Our results can help choose the highest RPR scale factor that can be used to achieve/ maintain certain perceived quality when using RPR (for example for bitrate reduction). The study also confirms that MS-SSIM and VMAF match subjective test results more closely compared to PSNR.

Characterizing the performance of emergency medical transport time metrics in a residentially segregated community.

To derive and characterize the performance of various metrics of emergency transport time in assessing for sociodemographic disparities in the setting of residential segregation. Secondarily to characterize racial disparities in emergency transport time of suspected stroke patients in Austin, Texas. We used a novel dataset of 2518 unique entries with detailed spatial and temporal information on all suspected stroke transports conducted by a public emergency medical service in Central Texas between 2010 and 2018. We conducted one-way ANOVA tests with post-hoc pairwise t-tests to assess how mean hospital transport times varied by patient race. We also developed a spatially-independent metric of emergency transport urgency, the ratio of expected duration of self-transport to a hospital and the measured transport time by an ambulance. We calculated ambulance arrival and destination times using sequential temporospatial coordinates. We excluded any entries in which patient race was not recorded. We also excluded entries in which ambulances' routes did not pass within 100 m of either the patient's location or the documented hospital destination. We found that mean transport time to a hospital was 2.5 min shorter for black patients compared to white patients. However, white patients' transport times to a hospital were found to be, on average, 4.1 min shorter than expected compared to 3.4 min shorter than expected for black patients. One-way ANOVA testing for the spatially-independent index of emergency transport urgency was not statistically significant, indicating that average transport time did not vary significantly across racial groups when accounting for variations in transport distance. Using a novel transport urgency index, we demonstrate that these findings represent race-based variation in spatial distributions rather than racial bias in emergency medical transport. These results highlight the importance of closely examining spatial distributions when utilizing temporospatial data to investigate geographically-dependent research questions.

AI-Inspired Non-Terrestrial Networks for IIoT: Review on Enabling Technologies and Applications

During the last few years, various Industrial Internet of Things (IIoT) applications have emerged with numerous network elements interconnected using wired and wireless communication technologies and equipped with strategically placed sensors and actuators. This paper justifies why non-terrestrial networks (NTNs) will bring the IIoT vision closer to reality by providing improved data acquisition and massive connectivity to sensor fields in large and remote areas. NTNs are engineered to utilize satellites, airships, and aircrafts, which can be employed to extend the radio coverage and provide remote monitoring and sensing services. Additionally, this paper describes indicative delay-tolerant massive IIoT and delay-sensitive mission-critical IIoT applications spanning a large number of vertical markets with diverse and stringent requirements. As the heterogeneous nature of NTNs and the complex and dynamic communications scenarios lead to uncertainty and a high degree of variability, conventional wireless communication technologies cannot sufficiently support ultra-reliable and low-latency communications (URLLC) and offer ubiquitous and uninterrupted interconnectivity. In this regard, this paper sheds light on the potential role of artificial intelligence (AI) techniques, including machine learning (ML) and deep learning (DL), in the provision of challenging NTN-based IIoT services and provides a thorough review of the relevant research works. By adding intelligence and facilitating the decision-making and prediction procedures, the NTNs can effectively adapt to their surrounding environment, thus enhancing the performance of various metrics with significantly lower complexity compared to typical optimization methods.

MSstatsQC 2.0: R/Bioconductor Package for Statistical Quality Control of Mass Spectrometry-Based Proteomics Experiments.

MSstatsQC is an R/Bioconductor package for statistical monitoring of longitudinal system suitability and quality control in mass spectrometry-based proteomics. MSstatsQC was initially designed for targeted selected reaction monitoring experiments. This paper presents an extension, MSstatsQC 2.0, that supports experiments with global data-dependent and data-independent acquisition. The extension implements data processing and analyses that are specific to these acquisition types. It relies on state-of-the-art methods of statistical process control to detect deviations from optimal performance of various metrics (such as intensity and retention time of chromatographic peaks) and to summarize the results across multiple metrics and analytes. Additionally, the web-based graphical user interface MSstatsQCgui, implemented as a separate R/Bioconductor package, provides a user-friendly way to visualize and report the results from MSstatsQC 2.0.

Metrics including time-varying loudness models to assess the impact of sonic booms and other transient sounds

Aircraft manufacturers have proposed that it is possible to build supersonic business jets that create low amplitude sonic booms that may be less objectionable to the public than those that led to the ban of civil supersonic flight over the US. Not only do the proposed sonic boom signatures differ from traditional sonic booms in amplitude, they also differ in shape. Due to the differences between these �low� booms and traditional booms, there has been increased interest in which metrics should be used to quantify the impact of these sounds. A semantic differential experiment, conducted over high quality earphones, was designed to determine the number of independent attributes people perceived and how these attributes influence acceptability or annoyance and to investigate what metrics were correlated to these factors. The work described here is focused on the performance of various metrics in predicting participants� ratings of impulsive sounds. It was found that statistics of and calculations derived from the output of Moore and Glasberg�s time-varying loudness model were the best predictors of participant-rated loudness, startle and annoyance. Performance of other commonly used metrics is also discussed.