Performance Evaluation Of Detection Research Articles

Monte Carlo model of a prototype flat-panel detector for multi-energy applications in radiotherapy.

The incorporation of multi-energy capabilities into radiotherapy flat-panel detectors offers advantages including enhanced soft tissue visualization by reduction of signal from overlapping anatomy such as bone in 2D image projections; creation of virtual monoenergetic images for 3D contrast enhancement, metal artefact reduction and direct acquisition of relative electron density. A novel dual-layer on-board imager offering dual energy processing capabilities is being designed. As opposed to other dual-energy implementation techniques which require separate acquisition with two different x-ray spectra, the dual-layer detector design enables simultaneous acquisition of high and low energy images with a single exposure. A computational framework is required to optimize the design parameters and evaluate detector performance for specific clinical applications. In this study, we report on the development of a Monte Carlo (MC) model of the imager including model validation. The stack-up of the dual-layer imager (DLI) was implemented in GEANT4 Application for Tomographic Emission (GATE). The DLI model has an active area of 43×43 cm2 , with top and bottom Cesium Iodide (CsI) scintillators of 600 and 800μm thickness, respectively. Measurement of spatial resolution and imaging of dedicated multi-material dual-energy (DE) phantoms were used to validate the model. The modulation transfer function (MTF) of the detector was calculated for a 120 kVp x-ray spectrum using a 0.5mm thick tantalum edge rotated by 2.5o . For imaging validation, the DE phantom was imaged using a 140 kVp x-ray spectrum. For both validation simulations, corresponding measurements were done using an initial prototype of the imager. Agreement between simulations and measurement was assessed using normalized root mean square error (NRMSE) and 1D profile difference for the MTF and phantom images respectively. Further comparison between measurement and simulation was made using virtual monoenergetic images (VMIs) generated from basis material images derived using precomputed look-up tables. The MTF of the bottom layer of the dual-layer model shows values decreasing more quickly with spatial frequency, compared to the top layer, due to the thicker bottom scintillator thickness and scatter from the top layer. A comparison with measurement shows NRMSE of 0.013 and 0.015 as well as identical MTF50 of 0.8 mm1 and 1.0 mm1 for the top and bottom layer respectively. For the DE imaging of the DE-phantom, although a maximum deviation of 3.3% is observed for the 10mm aluminum and Teflon inserts at the top layer, the agreement for all other inserts is less than 2.2% of the measured value at both layers. Material decomposition of simulated scatter-free DE images gives an average accuracy in PMMA and aluminum composition of 4.9% and 10.3% for 11-30mm PMMA and 1-10mm aluminum objects respectively. A comparison of decomposed values using scatter containing measured and simulated DE images shows good agreement within statistical uncertainty. Validation using both MTF and phantom imaging shows good agreement between simulation and measurements. With the present configuration of the digital prototype, the model can generate material decomposed images and virtual monoenergetic images.

Building Footprint Extraction from LiDAR Data Using SVM Classification

Automatic detection of buildings from very high resolution (VHR) satellite images is a current research hotspot in remote sensing. In this work presents a novel approach to detect the buildings by atomization of the training area collecting stage for supervised classification. The classification method for the analysis of satellite image based on Normalized Difference Vegetation Index (NDVI). The method employs the multi-spectral remote sensing data technique to find spectral signature of different objects such as vegetation index, land cover classification, concrete structure, road structure, rural and urban areas, rocky areas and remaining areas presented in the image. In the experiments, the proposed SVM framework outperformed for the three (Linear SVM, Cubic SVM, Medium Gaussian SVM) methods tested. SVM is trained through supervised learning to classify each location in the image as “buildings” or “Non-buildings.” Finally, SVM algorithm to find the accuracy value, training time and receiver operating characteristic curves to evaluate detection performance, and compare the proposed algorithm with several existing methods.

Pre-whitened matched filter and convolutional neural network based model observer performance for mass lesion detection in non-contrast breast CT.

Mathematical model observers have been shown to reasonably predict human observer performance and are useful when human observer studies are infeasible. Recently, convolutional neural networks (CNNs) have also been used as substitutes for human observers, and studies have shown their utility as an optimal observer. In this study, a CNN model observer is compared to the pre-whitened matched filter (PWMF) model observer in detecting simulated mass lesions inserted into 253 acquired breast computed tomography (bCT) images from patients imaged at our institution. To compare CNN and PWMF model observers for detecting signal-known-exactly (SKE) location-known-exactly (LKE) simulated lesions in bCT images with real anatomical backgrounds, and to use these model observers collectively to optimize parameters and understand trends in performance with breast CT. Spherical lesions with different diameters (1, 3, 5, 9mm) were mathematically inserted into reconstructed patient bCT image data sets to mimic 3D mass lesions in the breast. 2D images were generated by extracting the center slice along the axial dimension or by slice averaging across adjacent slices to model thicker sections (0.4, 1.2, 2.0, 6.0, 12.4, 20.4mm). The role of breast density was retrospectively studied using the range of breast densities intrinsic to the patient bCT data sets. In addition, mass lesions were mathematically inserted into Gaussian images matched to the mean and noise power spectrum of the bCT images to better understand the performance of the CNN in the context of a known ideal observer (the PWMF). The simulated Gaussian and bCT images were divided into training and testing data sets. Each training data set consisted of 91 600 images, and each testing data set consisted of 96 000 images. A CNN and PWMF was trained on the Gaussian training images, and a different CNN and PWMF was trained on the bCT training images. The trained model observers were tested, and receiver operating characteristic (ROC) curve analysis was used to evaluate detection performance. The area under the ROC curve (AUC) was the primary performance metric used to compare the model observers. In the Gaussian background, the CNN performed essentially identically to the PWMF across lesion sizes and section thicknesses. In the bCT background, the CNN outperformed the PWMF across lesion size, breast density, and most section thicknesses. These findings suggest that there are higher-order features in bCT images that are harnessed by the CNN observer but are inaccessible to the PWMF. The CNN performed equivalently to the ideal observer in Gaussian textures. In bCT background, the CNN captures more diagnostic information than the PWMF and may be a more pertinent observer when conducting optimal performance studies in breast CT images.

YOLOv5 Drone Detection Using Multimodal Data Registered by the Vicon System.

This work is focused on the preliminary stage of the 3D drone tracking challenge, namely the precise detection of drones on images obtained from a synchronized multi-camera system. The YOLOv5 deep network with different input resolutions is trained and tested on the basis of real, multimodal data containing synchronized video sequences and precise motion capture data as a ground truth reference. The bounding boxes are determined based on the 3D position and orientation of an asymmetric cross attached to the top of the tracked object with known translation to the object's center. The arms of the cross are identified by the markers registered by motion capture acquisition. Besides the classical mean average precision (mAP), a measure more adequate in the evaluation of detection performance in 3D tracking is proposed, namely the average distance between the centroids of matched references and detected drones, including false positive and false negative ratios. Moreover, the videos generated in the AirSim simulation platform were taken into account in both the training and testing stages.

Ohun: An R package for diagnosing and optimizing automatic sound event detection

Abstract Animal acoustic signals are widely used in diverse research areas due to the relative ease with which sounds can be registered across a wide range of taxonomic groups and research settings. However, bioacoustics research can quickly generate large data sets, which might prove challenging to analyse promptly. Although many tools are available for the automated detection of sounds, choosing the right approach can be difficult and only a few tools provide a framework for evaluating detection performance. Here, we present ohun, an R package intended to facilitate automated sound event detection. ohun provides functions to diagnose and optimize detection routines, compare performance among different detection approaches and evaluate the accuracy in inferring the temporal location of events. The package uses reference annotations containing the time position of target sounds in a training data set to evaluate detection routine performance using common signal detection theory indices. This can be done both with routine outputs imported from other software and detections run within the package. The package also provides functions to organize acoustic data sets in a format amenable to detection analyses. In addition, ohun includes energy‐based and template‐based detection methods, two commonly used automatic approaches in bioacoustics research. We show how ohun can be used to automatically detect vocal signals with case studies of adult male zebra finch Taenopygia gutata songs and Spix's disc‐winged bat Thyroptera tricolor ultrasonic social calls. We also include examples of how to evaluate the detection performance of ohun and external software. Finally, we provide some general suggestions to improve detection performance.

Analysis and performance evaluation of Infected shoot detection through Deep learning Supervised methods

Analysis and performance evaluation of Infected shoot detection through Deep learning Supervised methods

Approaches to characterizing oscillatory burst detection algorithms for electrophysiological recordings

BackgroundCognitive processes are associated with fast oscillations of the local field potential and electroencephalogram. There is a growing interest in targeting them because these are disrupted by aging and disease. This has proven challenging because they often occur as short-lasting bursts. Moreover, they are obscured by broad-band aperiodic activity reflecting other neural processes. These attributes have made it exceedingly difficult to develop analytical tools for estimating the reliability of detection methods. New methodTo address this challenge, we developed an open-source toolkit with four processing steps, that can be tailored to specific brain states and individuals. First, the power spectrum is decomposed into periodic and aperiodic components, each of whose properties are estimated. Second, the properties of the transient oscillatory bursts that contribute to the periodic component are derived and optimized to account for contamination from the aperiodic component. Third, using the burst properties and aperiodic power spectrum, surrogate neural signals are synthesized that match the observed signal’s spectrotemporal properties. Lastly, oscillatory burst detection algorithms run on the surrogate signals are subjected to a receiver operating characteristic analysis, providing insight into their performance. ResultsThe characterization algorithm extracted features of oscillatory bursts across multiple frequency bands and brain regions, allowing for recording-specific evaluation of detection performance. For our dataset, the optimal detection threshold for gamma bursts was found to be lower than the one commonly used. Comparison with existing methodsExisting methods characterize the power spectrum, while ours evaluates the detection of oscillatory bursts. ConclusionsThis pipeline facilitates the evaluation of thresholds for detection algorithms from individual recordings.

A comprehensive dataset of annotated brain metastasis MR images with clinical and radiomic data

Brain metastasis (BM) is one of the main complications of many cancers, and the most frequent malignancy of the central nervous system. Imaging studies of BMs are routinely used for diagnosis of disease, treatment planning and follow-up. Artificial Intelligence (AI) has great potential to provide automated tools to assist in the management of disease. However, AI methods require large datasets for training and validation, and to date there have been just one publicly available imaging dataset of 156 BMs. This paper publishes 637 high-resolution imaging studies of 75 patients harboring 260 BM lesions, and their respective clinical data. It also includes semi-automatic segmentations of 593 BMs, including pre- and post-treatment T1-weighted cases, and a set of morphological and radiomic features for the cases segmented. This data-sharing initiative is expected to enable research into and performance evaluation of automatic BM detection, lesion segmentation, disease status evaluation and treatment planning methods for BMs, as well as the development and validation of predictive and prognostic tools with clinical applicability.

Performance Evaluation of MEMS-Based Automotive LiDAR Sensor and Its Simulation Model as per ASTM E3125-17 Standard

Measurement performance evaluation of real and virtual automotive light detection and ranging (LiDAR) sensors is an active area of research. However, no commonly accepted automotive standards, metrics, or criteria exist to evaluate their measurement performance. ASTM International released the ASTM E3125-17 standard for the operational performance evaluation of 3D imaging systems commonly referred to as terrestrial laser scanners (TLS). This standard defines the specifications and static test procedures to evaluate the 3D imaging and point-to-point distance measurement performance of TLS. In this work, we have assessed the 3D imaging and point-to-point distance estimation performance of a commercial micro-electro-mechanical system (MEMS)-based automotive LiDAR sensor and its simulation model according to the test procedures defined in this standard. The static tests were performed in a laboratory environment. In addition, a subset of static tests was also performed at the proving ground in natural environmental conditions to determine the 3D imaging and point-to-point distance measurement performance of the real LiDAR sensor. In addition, real scenarios and environmental conditions were replicated in the virtual environment of a commercial software to verify the LiDAR model's working performance. The evaluation results show that the LiDAR sensor and its simulation model under analysis pass all the tests specified in the ASTM E3125-17 standard. This standard helps to understand whether sensor measurement errors are due to internal or external influences. We have also shown that the 3D imaging and point-to-point distance estimation performance of LiDAR sensors significantly impacts the working performance of the object recognition algorithm. That is why this standard can be beneficial in validating automotive real and virtual LiDAR sensors, at least in the early stage of development. Furthermore, the simulation and real measurements show good agreement on the point cloud and object recognition levels.

0.2-4.0 THz broadband terahertz detector based on antenna-coupled AlGaN/GaN HEMTs arrayed in a bow-tie pattern.

To satisfy the demand for broadband and high-sensitivity terahertz detectors, we designed and verified a broadband terahertz detector built with antenna-coupled AlGaN/GaN high-electron-mobility transistors (HEMTs). Eighteen pairs of dipole antennas with different center frequency from 0.24 to 7.4 THz are arrayed into a bow-tie pattern. The corresponding eighteen transistors have common a source and a drain but different gated channels coupled by the corresponding antennas. The photocurrents generated by each gated channel are combined in the drain as the output port. With incoherent terahertz radiation from a hot blackbody in a Fourier-transform spectrometer (FTS), the detector exhibits a continuous response spectrum from 0.2 to 2.0 THz at 298 K and from 0.2 to 4.0 THz at 77 K, respectively. The results agree well with simulations taking into account the silicon lens, antenna and blackbody radiation law. The sensitivity is characterized under coherent terahertz irradiation, the average noise-equivalent power (NEP) is about 188 p W/H z at 298 K and 19 p W/H z at 77 K from 0.2 to 1.1 THz, respectively. A maximum optical responsivity of 0.56 A/W and a minimum NEP of 7.0 p W/H z at 0.74 THz are achieved at 77 K. The blackbody response spectrum is divided by the blackbody radiation intensity to obtain a performance spectrum, which is calibrated by measuring coherence performance from 0.2 to 1.1 THz to evaluate detector performance at frequencies above 1.1 THz. At 298 K, the NEP is about 1.7 n W/H z at 2.0 THz. At 77 K, the NEP is about 3 n W/H z at 4.0 THz. For further improvements in sensitivity and bandwidth, high-bandwidth coupling components, smaller series resistance, smaller gate lengths and high-mobility materials need to be considered.

Performance evaluation of plant leaf disease detection using deep learning models

Plant diseases have a serious impact on production, and hence they must be detected and recognised at early stages. Smart firming using deep learning can automatically identify infected crops. We provide extremely effective convolution neural network (CNN) designs for the identification of leaf diseases in this research strategy. For the training and testing phases of this study, a database of potato leaves is produced. To classify the disease from the input photos of the supported training dataset, we employed CNN to extract its characteristics. 1700 photos of potato leaves were used for model training, and then about 600 images were used for testing. To identify citrus diseases, Convolutional Neural Networks, Deep Learning, base learning, and transfer learning were applied. Results from training, testing, and experiments indicate that the suggested architecture has outperformed other current models in terms of ResNet model accuracy, achieving a score of 99.62%.

Performance evaluation of keyword detection for the chatbot model

Performance evaluation of keyword detection for the chatbot model

Performance evaluation of CNN-based crack detection for electrical discharge machined steel surfaces

Electrical discharge machining (EDM) is a non-traditional machining technique that is frequently employed on hard materials. In today's industrial practice, it has been the most prevalent non-traditional material removal procedure. It allows you to process challenging materials and construct complicated forms with excellent accuracy. In the context of image pre-processing approaches to discover defective or non-defective machining pieces through EDM, the Centre of attention of this article is utilizing convolutional neural networks (CNN) to create a machinery piece defective detection approach. A total of 180 datasets of varying sizes produced from public datasets, the proposed CNN model is assessed and compared to pre-trained networks, namely the VGG-16, VGG-19, ResNet-50, and ResNet-101 models. The assessment took into crack detection outcomes, and classification parameters such as accuracy, precision, recall, and F1-score. Also, we have given the receiver operating characteristic (ROC) curve, precision–recall curve, and confusion matrices for each model for the required classification to predict the defective cracks, and also, we included the histograms to find the probabilities of defective and non-defective cracks. The suggested model can discriminate between pictures that are cracked and those that are not. According to the findings, VGG-16 has the best accuracy out of all of these models. In comparison to prior conventional procedures, the proposed EDM crack defective detecting methodology gives great accuracy.VGG-16 attains 93% accuracy, 93% of F1-score, 94% of precision, 93% of recall, 93% of specificity, as well as 97% of AUC, testing findings suggest that our strategy is capable of incredible performances.

Detection model and performance evaluation for the infrared search and tracking system.

To effectively evaluate the detection capability of the infrared search and tracking system (IRST), the mathematical models of target infrared (IR) radiation intensity envelope, atmospheric transmittance, and signal-to-noise ratio (SNR) operating range are improved, respectively. Based on the above three models, the IRST performance evaluation simulation system is designed to analyze the optimal detection point and effective detection area under different conditions (such as target speed, detection angle, detection probability, false alarm probability, and operating range). Meanwhile, in order to verify the validity of the model, the range evaluation method of calibration and cross-validation is proposed. And taking the cross-validation method as the baseline, the error of the optimized mathematical model of IRST in this paper is within 10%. The research results are of reference significance for IRST operating range evaluation, design, and use.

Research on the Health Detection and Seismic Performance Evaluation of High-Rise Buildings

Research background: With the rapid development of civil engineering, the traditional low rise building model can no longer meet the needs of urbanization, and high-rise building structure has become the future development direction. In recent years, earthquake disasters occur frequently in China, which has caused huge losses to high-rise buildings. Therefore, we must strengthen the relevant research of high-rise building structures, which can make the high-rise building structural columns meet the requirements of safety performance. Therefore, the health detection and seismic performance evaluation of high-rise buildings will be of great significance. Research method: Through static elastic-plastic evaluation, we can calculate the dissipated strain energy of the structural column, which will further obtain the damage index of the structural column and obtain relevant data. Through this calculation method, we can complete the seismic performance evaluation of high-rise buildings, which is a calculation mode different from the traditional finite element software analysis. Through static elastic-plastic analysis, the seismic performance evaluation of high-rise buildings will have higher accuracy, which will build a damage index. Therefore, this paper uses a variety of research methods, mainly literature survey and empirical analysis. Through literature analysis, this paper can fully and correctly understand the methods of health detection and seismic performance evaluation of high-rise building structures at home and abroad, which will further clarify the current research level at home and abroad. Through empirical research, this paper analyzes specific examples, which can obtain more accurate results. Conclusion: In this paper, the seismic performance of high-rise buildings is comprehensively evaluated according to the damage index value of building structural columns. The simulation results show that more ideal seismic evaluation results can be obtained.

Chaotic Behavior Analysis and Multifractal Feature Learning of Specular Reflection Ionosphere Clutter for Target Detection

Because ionosphere condition is dynamically varying in the Earth’s middle-latitude, the long-range detection performance of operating high frequency surface wave radar (HFSWR) systems is severely restricted by the presence of ionosphere clutter. The development and performance evaluation of target detection under the F-layer specular reflection ionosphere clutter with concentrated power (Fs-concentrated clutter) will be described in this paper. First, this article pays attention to the variable chaotic behavior of Fs-concentrated clutter by using amplitude and phase echo data. Then, this article investigates the multifractal property of the power spectrum between the Fs-concentrated clutter and mixed echo component. The various fractal characteristics of different types of echo component and Doppler scale are analyzed. Next, the singularity intensity correlation function width, the accumulation area of multifractal correlation spectrum and asymmetry parameter are extracted as the learning features. These multifractal feature parameters are fed into the improved AlexNet and Resnet networks for distinguishing the types of echo component. Experimental results using real HFSWR echoes datasets have demonstrated that this proposed method can increase the detection probability by about 20% than the state-of-art detection methods and classical target detection methods under various signal-clutter ratio (SCR) background.

FT-NIR online detection performance improvement and evaluation for 2,6-dimethylphenol purity

In this paper, Fourier transform near-infrared (FT-NIR) spectroscopy was used to online detect the 2,6-dimethylphenol(2,6-DMP) purity of different rectifying columns in the distillation separation process. Considering the similarity of spectral data between different columns, transfer learning could be introduced to improve the online detection performance of 2,6-DMP purity by reusing the spectral data from other columns. Furthermore, in order to analyze the effect of the quantity and quality of the FT-NIR spectra on the detection accuracy, the influence function was used to construct the quantitative relationship between the detection accuracy and the spectral information to be transferred. The effectiveness of the approach was analyzed on the FT-NIR spectral datasets from the distillation purification process in the 2,6-DMP monomer separation section. By transferring more favorable spectral data to avoid negative transfer, the detection accuracy will be obviously increased.

Corner Detection via Scale-Space Behavior-Guided Trajectory Tracing

Existing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">curvature scale-space</i> (CSS) methods detect corners by tracing the CSS trajectories from a determined high scale toward the lowest one. For those images with sophisticated details, such approach could often yield unsatisfactory corner detection results; i.e., miss-detected <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">true</i> corners (false negatives) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">round</i> corners (false positives). In this letter, these two fundamental problems are investigated. To tackle them, a novel CSS-based corner detector is proposed by incorporating our mathematically derived scale-space properties of the planar curves and corner points into the developed trajectory tracing algorithm, called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scale-space behavior-guided trajectory tracing</i> (SBTT). In view of lacking a benchmark dataset with the ground truth, another contribution from our work is on the establishment of an augmented test image dataset, containing 147 test images with manually-labelled ground truth and their augmented images up to 62,328 images in total. Based on the ground truth, four commonly-used metrics are exploited to conduct corner detection performance evaluation. The obtained simulation results show that our proposed corner detector yields the highest F-score, when compared with that of nine state-of-the-art methods.

Track-before-Detect Algorithm for Underwater Diver Based on Knowledge-Aided Particle Filter.

This work studies the underwater detection and tracking of diver targets under a low signal-to-reverberation ratio (SRR) in active sonar systems. In particular, a particle filter track-before-detect based on a knowledge-aided (KA-PF-TBD) algorithm is proposed. Specifically, the original echo data is directly used as the input of the algorithm, which avoids the information loss caused by threshold detection. Considering the prior motion knowledge of the underwater diver target, we established a multi-directional motion model as the state transition model. An efficient method for calculating the statistical characteristics of echo data about the extended target is proposed based on the non-parametric kernel density estimation theory. The multi-directional movement model set and the statistical characteristics of the echo data are used as the knowledge-aided information of the particle filter process: this is used to calculate the particle weight with the sub-area instead of the whole area, and then the particles with the highest weight are used to estimate the target state. Finally, the effectiveness of the proposed algorithm is proved by simulation and sea-level experimental data analysis through joint evaluation of detection and tracking performance.

Development and performance evaluation of a deep learning lung nodule detection system

BackgroundLung cancer is the leading cause of cancer-related deaths throughout the world. Chest computed tomography (CT) is now widely used in the screening and diagnosis of lung cancer due to its effectiveness. Radiologists must identify each small nodule shadow from 3D volume images, which is very burdensome and often results in missed nodules. To address these challenges, we developed a computer-aided detection (CAD) system that automatically detects lung nodules in CT images.MethodsA total of 1997 chest CT scans were collected for algorithm development. The algorithm was designed using deep learning technology. In addition to evaluating detection performance on various public datasets, its robustness to changes in radiation dose was assessed by a phantom study. To investigate the clinical usefulness of the CAD system, a reader study was conducted with 10 doctors, including inexperienced and expert readers. This study investigated whether the use of the CAD as a second reader could prevent nodular lesions in lungs that require follow-up examinations from being overlooked. Analysis was performed using the Jackknife Free-Response Receiver-Operating Characteristic (JAFROC).ResultsThe CAD system achieved sensitivity of 0.98/0.96 at 3.1/7.25 false positives per case on two public datasets. Sensitivity did not change within the range of practical doses for a study using a phantom. A second reader study showed that the use of this system significantly improved the detection ability of nodules that could be picked up clinically (p = 0.026).ConclusionsWe developed a deep learning-based CAD system that is robust to imaging conditions. Using this system as a second reader increased detection performance.