Detection Metrics Research Articles

Augmenting Affective Dependency Graph via Iterative Incongruity Graph Learning for Sarcasm Detection

Recently, progress has been made towards improving automatic sarcasm detection in computer science. Among existing models, manually constructing static graphs for texts and then using graph neural networks (GNNs) is one of the most effective approaches for drawing long-range incongruity patterns. However, the manually constructed graph structure might be prone to errors (e.g., noisy or incomplete) and not optimal for the sarcasm detection task. Errors produced during the graph construction step cannot be remedied and may accrue to the following stages, resulting in poor performance. To surmount the above limitations, we explore a novel Iterative Augmenting Affective Graph and Dependency Graph (IAAD) framework to jointly and iteratively learn the incongruity graph structure. IAAD can alternatively update the incongruity graph structure and node representation until the learning graph structure is optimal for the metrics of sarcasm detection. More concretely, we begin with deriving an affective and a dependency graph for each instance, then an iterative incongruity graph learning module is employed to augment affective and dependency graphs for obtaining the optimal inconsistent semantic graph with the goal of optimizing the graph for the sarcasm detection task. Extensive experiments on three datasets demonstrate that the proposed model outperforms state-of-the-art baselines for sarcasm detection with significant margins.

Learning to detect 3D planes from a single plenoptic image.

In this paper, we propose a deep neural network to detect 3D planar surfaces from single plenoptic images captured by a Lytro Illum camera. Different from learning methods based on a single RGB image, we train a network to exploit the light distribution information both in spatial and angular dimensions from multi-sub-aperture images. The features from each sub-aperture image are extracted by using parameter-sharing convolutional layers and then fused to jointly infer the parameters of planes, depths, and segmentation masks. The experiments demonstrate that our approach outperforms the existing state-of-the-art methods with significant margins in the plane detection metrics.

Detecting common coccinellids found in sorghum using deep learning models

Increased global production of sorghum has the potential to meet many of the demands of a growing human population. Developing automation technologies for field scouting is crucial for long-term and low-cost production. Since 2013, sugarcane aphid (SCA) Melanaphis sacchari (Zehntner) has become an important economic pest causing significant yield loss across the sorghum production region in the United States. Adequate management of SCA depends on costly field scouting to determine pest presence and economic threshold levels to spray insecticides. However, with the impact of insecticides on natural enemies, there is an urgent need to develop automated-detection technologies for their conservation. Natural enemies play a crucial role in the management of SCA populations. These insects, primary coccinellids, prey on SCA and help to reduce unnecessary insecticide applications. Although these insects help regulate SCA populations, the detection and classification of these insects is time-consuming and inefficient in lower value crops like sorghum during field scouting. Advanced deep learning software provides a means to perform laborious automatic agricultural tasks, including detection and classification of insects. However, deep learning models for coccinellids in sorghum have not been developed. Therefore, our objective was to develop and train machine learning models to detect coccinellids commonly found in sorghum and classify them according to their genera, species, and subfamily level. We trained a two-stage object detection model, specifically, Faster Region-based Convolutional Neural Network (Faster R-CNN) with the Feature Pyramid Network (FPN) and also one-stage detection models in the YOLO (You Only Look Once) family (YOLOv5 and YOLOv7) to detect and classify seven coccinellids commonly found in sorghum (i.e., Coccinella septempunctata, Coleomegilla maculata, Cycloneda sanguinea, Harmonia axyridis, Hippodamia convergens, Olla v-nigrum, Scymninae). We used images extracted from the iNaturalist project to perform training and evaluation of the Faster R-CNN-FPN and YOLOv5 and YOLOv7 models. iNaturalist is an imagery web server used to publish citizen’s observations of images pertaining to living organisms. Experimental evaluation using standard object detection metrics, such as average precision (AP), AP@0.50, etc., has shown that the YOLOv7 model performs the best on the coccinellid images with an AP@0.50 as high as 97.3, and AP as high as 74.6. Our research contributes automated deep learning software to the area of integrated pest management, making it easier to detect natural enemies in sorghum.

Lorentz group equivariant autoencoders

There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group textrm{SO}^+(3,1), with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

Digital holography for real-time non-invasive monitoring of larval fish at power plant intakes

Effective evaluation of technological and operational approaches to reduce entrainment of marine organisms at cooling water intake structures (CWIS) requires accurate organism-sensing systems. Current detection methods lead to large temporal data gaps, require tedious manual analysis, and are fatal to organisms. Here, we describe integrating deep learning with a non-lethal, non-intrusive imaging method—digital holography—to rapidly detect fish larvae. Laboratory experiments demonstrated that the instrument could successfully image fish larvae at flow rates exceeding ranges seen in CWIS. Holograms of two fish larvae species, in the presence of bubbles and detritus, were collected to build a large database for training a lightweight convolutional neural network. The model achieves 97% extraction accuracy in quantifying larvae, and distinguishing them from other particles, including detritus and bubbles, when applied to a dataset of manually classified images, exceeding previous metrics for non-lethal, accurate, and real-time detection. These results demonstrate the potential of in situ holographic imaging for monitoring endangered larval fish species at power plant intake structures, and for high-fidelity, real-time applications in monitoring aquatic ichthyoplankton.

Sources of performance variability in deep learning-based polyp detection

PurposeValidation metrics are a key prerequisite for the reliable tracking of scientific progress and for deciding on the potential clinical translation of methods. While recent initiatives aim to develop comprehensive theoretical frameworks for understanding metric-related pitfalls in image analysis problems, there is a lack of experimental evidence on the concrete effects of common and rare pitfalls on specific applications. We address this gap in the literature in the context of colon cancer screening.MethodsOur contribution is twofold. Firstly, we present the winning solution of the Endoscopy Computer Vision Challenge on colon cancer detection, conducted in conjunction with the IEEE International Symposium on Biomedical Imaging 2022. Secondly, we demonstrate the sensitivity of commonly used metrics to a range of hyperparameters as well as the consequences of poor metric choices.ResultsBased on comprehensive validation studies performed with patient data from six clinical centers, we found all commonly applied object detection metrics to be subject to high inter-center variability. Furthermore, our results clearly demonstrate that the adaptation of standard hyperparameters used in the computer vision community does not generally lead to the clinically most plausible results. Finally, we present localization criteria that correspond well to clinical relevance.ConclusionWe conclude from our study that (1) performance results in polyp detection are highly sensitive to various design choices, (2) common metric configurations do not reflect the clinical need and rely on suboptimal hyperparameters and (3) comparison of performance across datasets can be largely misleading. Our work could be a first step towards reconsidering common validation strategies in deep learning-based colonoscopy and beyond.

Gradient Structure Information-Guided Attention Generative Adversarial Networks for Remote Sensing Image Generation

A rich and effective dataset is an important foundation for the development of AI algorithms, and the quantity and quality of the dataset determine the upper limit level of the algorithms. For aerospace remote sensing datasets, due to the high cost of data collection and susceptibility to meteorological and airway conditions, the existing datasets have two problems: firstly, the number of datasets is obviously insufficient, and, secondly, there is large unevenness between different categories in datasets. One of the effective solutions is to use neural networks to generate fake data by learning from real data, but existing methods still find difficulty in generating remote sensing sample images with good texture detail and geometric distortion. To address the shortcomings of existing image generation algorithms, this paper proposes a gradient structure information-guided attention generative adversarial network (SGA-GAN) for remote sensing image generation, which contains two innovative initiatives: on the one hand, a learnable gradient structure information extraction branch network can be added to the generator network to obtain complex structural information in the sample image, thus alleviating the distortion of the sample geometric structure in remote sensing image generation; on the other hand, a multidimensional self-attention feature selection module is proposed to further improve the quality of the generated remote sensing images by connecting cross-attentive modules as well as spatial and channel attention modules in series to guide the generator to better utilize global information. The algorithm proposed in this paper outperformed other methods, such as StyleGAN-XL and FastGAN, in both the qualitative and quantitative evaluation, whereby the FID on the DOTA dataset decreased by 23.927 and the IS was improved by 2.351. The comparison experiments show that the method proposed in this paper can generate more realistic sample images, and images generated by this method can improve object detection metrics by increasing the number of single-category datasets and the number of targets in fewer categories in multi-category datasets, which means it can be effectively used in the field of intelligent processing of remote sensing images.

Removing Human Bottlenecks in Bird Classification Using Camera Trap Images and Deep Learning

Birds are important indicators for monitoring both biodiversity and habitat health; they also play a crucial role in ecosystem management. Declines in bird populations can result in reduced ecosystem services, including seed dispersal, pollination and pest control. Accurate and long-term monitoring of birds to identify species of concern while measuring the success of conservation interventions is essential for ecologists. However, monitoring is time-consuming, costly and often difficult to manage over long durations and at meaningfully large spatial scales. Technology such as camera traps, acoustic monitors and drones provide methods for non-invasive monitoring. There are two main problems with using camera traps for monitoring: (a) cameras generate many images, making it difficult to process and analyse the data in a timely manner; and (b) the high proportion of false positives hinders the processing and analysis for reporting. In this paper, we outline an approach for overcoming these issues by utilising deep learning for real-time classification of bird species and automated removal of false positives in camera trap data. Images are classified in real-time using a Faster-RCNN architecture. Images are transmitted over 3/4G cameras and processed using Graphical Processing Units (GPUs) to provide conservationists with key detection metrics, thereby removing the requirement for manual observations. Our models achieved an average sensitivity of 88.79%, a specificity of 98.16% and accuracy of 96.71%. This demonstrates the effectiveness of using deep learning for automatic bird monitoring.

A System for the Automatic Detection and Evaluation of the Runway Surface Cracks Obtained by Unmanned Aerial Vehicle Imagery Using Deep Convolutional Neural Networks

The timely detection and recognizing of distress on an airport pavement is crucial for safe air traffic. For this purpose, a physical inspection of the airport maneuvering areas is regularly carried out, which might be time-consuming due to its size. One of the modern approaches to speeding up this process is unmanned aerial vehicle imagery followed by an automatic evaluation. This study explores the automatic detection of the transverse crack, its dimension measurement, and position determination within the slab on the concrete runway. The aerial image data were obtained from flights at the given altitude above the runway and processed using commercial multi-view reconstruction software to create a dataset for the training, verification, and testing of a YOLOv2 object detector. Once the crack was detected, the main features were obtained by image segmentation and morphological operations. The YOLOv2 detector was tuned with 3279 images until the detection metrics (average precision AP = 0.89) reached sufficient value for real deployment. The detected cracks were further processed to determine their position within the concrete slab, and their dimensions, i.e., length and width, were measured. The automated crack detection and evaluation system developed in this study was successfully verified on the experimental section of the runway as an example of practical application. It was proven that unmanned aerial vehicle imagery is efficient over broad areas and produces impressive results with the combination of artificial intelligence.

Longitudinal fNIRS and EEG metrics of habituation and novelty detection are correlated in 1–18-month-old infants

IntroductionHabituation and novelty detection are two fundamental and widely studied neurocognitive processes. Whilst neural responses to repetitive and novel sensory input have been well-documented across a range of neuroimaging modalities, it is not yet fully understood how well these different modalities are able to describe consistent neural response patterns. This is particularly true for infants and young children, as different assessment modalities might show differential sensitivity to underlying neural processes across age. Thus far, many neurodevelopmental studies are limited in either sample size, longitudinal scope or breadth of measures employed, impeding investigations of how well common developmental trends can be captured via different methods. MethodThis study assessed habituation and novelty detection in N = 204 infants using EEG and fNIRS measured in two separate paradigms, but within the same study visit, at 1, 5 and 18 months of age in an infant cohort in rural Gambia. EEG was acquired during an auditory oddball paradigm during which infants were presented with Frequent, Infrequent and Trial Unique sounds. In the fNIRS paradigm, infants were familiarised to a sentence of infant-directed speech, novelty detection was assessed via a change in speaker. Indices for habituation and novelty detection were extracted for both EEG and NIRS ResultsWe found evidence for weak to medium positive correlations between responses on the fNIRS and the EEG paradigms for indices of both habituation and novelty detection at most age points. Habituation indices correlated across modalities at 1 month and 5 months but not 18 months of age, and novelty responses were significantly correlated at 5 months and 18 months, but not at 1 month. Infants who showed robust habituation responses also showed robust novelty responses across both assessment modalities. DiscussionThis study is the first to examine concurrent correlations across two neuroimaging modalities across several longitudinal age points. Examining habituation and novelty detection, we show that despite the use of two different testing modalities, stimuli and timescale, it is possible to extract common neural metrics across a wide age range in infants. We suggest that these positive correlations might be strongest at times of greatest developmental change.

Linear Programming Models for Leak Detection and Localization in Water Distribution Networks

This study examined the benefits and limitations of formulating linear programming models for water distribution network leak detection and localization considering sequences of demands. First, the consistency of sensitivity matrixes for different demand magnitudes and spatially varying nodal demands was investigated to determine if different sensitivity matrixes are needed for each time step and to determine the demands to use for computing the gradient matrixes. Then several unconstrained and constrained models were developed and their detection and localization performance was evaluated for a network in Austin, Texas. To constrain the failure location in time and space, binary (zero–one) integer linear programming (BILP) was used. A set of threshold-based detection rules was applied to test for the anomalies in the time series, and they were compared for a range of realistic leak sizes. Based on the numerical results, the best optimization model was identified considering multiple detection metrics (detection effectiveness, efficiency, and localization). The best BILP model that constrains leak locations spatially and temporally outperformed the unconstrained model in detecting and locating relatively small leaks.

Evaluation metrics systematization for 2D human poses analysis models

This paper describes the systematization of evaluation metrics for 2D human pose analysis models. Some of the most popular tasks solved using machine learning (ML) methods are detection, tracking and recognition of human actions for various practical applications. There are a lot of different metrics that allow evaluating the model from one point or another. To evaluate a specific task, a certain set of metrics is used. However, as literature analysis shows, the vast number of metric definitions, as well as the use of different terms and multiple representations of the same ideas, causes problems of interpretation and comparison of different ML models and methods in detecting, tracking, and recognizing human actions. The purpose of this work is to analyze the metrics for evaluating methods for processing 2D human poses in video in order to facilitate the informed choice of the metrics. To improve the objectivity of evaluating the results of empirical studies of existing and newly developed methods and models for detecting, tracking, and recognizing human actions, a systematization of existing metrics into subgroups was proposed, depending on what task they evaluate. Four classes of evaluation metrics were introduced: classification metrics, key point’s detection, object tracking, and general metrics. Classification metrics are based on quality evaluation and matching values from predicted bounding boxes with ground truths. Key point’s detection metrics are oriented on the quality of found joints of the human body skeleton. Tracking metrics evaluate the object detection on each frame and the correctness of determining its trajectory. General metrics are not specifically related to any of the human 2D pose analysis tasks. The prototype of the application based on suggested metrics systematization, the purpose of which is to help data scientists in formalizing the choice of metrics for evaluating models depending on the ML problem being solved and the application area was developed. To evaluate and demonstrate the metrics, that were suggested in this application, Faster R-CNN, SSD and YOLOv3 object detection models were analyzed and compared in scope of 2D human pose analysis application area. The results of the analysis showed that Faster R-CNN and YOLOv3 have the most accurate responses, although they have the disadvantage of a high False positive rate. The implementation also showed that metrics that based on True negative values are uninformative in scope of working with bounding boxes, because of the specific of application area and inability to calculate True negatives on the image data.

An IoT-Fuzzy-Based Jamming Detection and Recovery System in Wireless Video Surveillance System

Wireless video surveillance system is one of the cyber-physical security systems kinds, which transmits the signal of IP cameras through a wireless medium using a radio band. WVSSs are widely deployed with large systems for use in strategic places such as city centers, public transportation, public roads, airports, and play a significant role in critical infrastructure protection. WVSSs are vulnerable to jamming attacks creating an unwanted denial of service. Hence, it is essential to secure this system from jamming attacks. In this paper, three models of IoT-fuzzy inference system-based jamming detection system are proposed for detecting and countermeasure the presence of jamming by computing two jamming detection metrics; PDR and PLR, and based on the result, the system countermeasures this attack by storing the video feed locally in the subsystem nodes. FIS models are based on Mamdani, Tsukamoto, and Sugeno fuzzy logic which optimizes the jamming detection metrics for detecting the jamming attack. The efficiency of these proposed models is compared in detecting jamming signals. The experimental results show that the proposed Tsukamoto model detects jamming attacks with high accuracy and efficiency. Finally, the proposed IoT-Tsukamoto-based model was compared with the existing systems and proved to be superior to them in terms of central processing complexity, accuracy, and countermeasure for this attack.

Robust estimation‐based multipath detection for vector tracking loop

AbstractMultipath interference in cities has always been one of the main problems leading to abnormal positioning results of a global navigation satellite system. Vector tracking is superior to conventional scalar tracking in the ability of mitigating multipath interference. However, due to the special sharing structure, fault propagation which easily occurs in the vector tracking loop leads to false alarms of normal satellite signals during multipath detection. To solve this problem, a fault‐tolerant algorithm based on robust estimation is applied to suppress fault propagation. Meanwhile, in order to improve the ability of detecting multiple short delayed multipath signals, chi‐square detection is proposed to be used before applying the fault‐tolerant algorithm. By taking advantage of the characteristic that the occurrence of multipath signals will cause significant deviation of code phase errors, the mean value of code phase errors is used as the detection metric. Both simulation test and field experiment are carried out to verify the detection performance. Experimental results show that the proposed algorithm can effectively prevent fault propagation and solve the problem of false positives of normal satellite signals. As a result, not only multipath signals are detected more accurately, but also more accurate navigation solution is obtained.

Fight Hardware with Hardware: Systemwide Detection and Mitigation of Side-channel Attacks Using Performance Counters

We present a kernel-level infrastructure that allows systemwide detection of malicious applications attempting to exploit cache-based side-channel attacks to break the process confinement enforced by standard operating systems. This infrastructure relies on hardware performance counters to collect information at runtime from all applications running on the machine. High-level detection metrics are derived from these measurements to maximize the likelihood of promptly detecting a malicious application. Our experimental assessment shows that we can catch a large family of side-channel attacks with a significantly reduced overhead. We also discuss countermeasures that can be enacted once a process is suspected of carrying out a side-channel attack to increase the overall tradeoff between the system’s security level and the delivered performance under non-suspected process executions.

An Early Minor-Fault Diagnosis Method for Lithium-Ion Battery Packs Based on Unsupervised Learning

Dear Editor, Any fault of a battery system that is not handled timely can cause catastrophic consequences. Therefore, it is significant to diagnose battery faults early and accurately. Due to the complex nonlinear features and inconsistency of lithium batteries, traditional fault diagnosis methods usually fail to detect battery minor faults in the early stages. Therefore, this letter proposes a real-time unsupervised learning diagnosis approach for early battery faults based on improved principal component analysis. The technique rotates the battery pack voltage sequence into a new coordinate space through linear combination, while the detection metrics of square prediction errors and modified contribution plots are employed to achieve minor fault traceability. In addition, the training sample relies on the voltage sequence of the battery health state instead of the fault data, which is difficult to collect. Moreover, this approach can not only locate the battery cell where the fault occurs but also diagnose battery open-circuit and short-circuit faults as well as the occurrence and duration of the fault in real-time. Furthermore, the feasibility and stability of the proposed method are verified by applying different experimental data. In summary, the presented approach provides an easy-to-implement option that does not require accurate mathematical modeling, expert understanding, and complex computational processes.

A Resource-Efficient Green Paradigm For Crowdsensing Based Spectrum Detection In Internet of Things Networks

Crowdsensing-based spectrum detection (CSD) is promising to enable full-coverage radio resource availability for the increasingly connected machines in the Internet of Things (IoT) networks. The current CSD scheme consumes a lot of energy and network resources for local sensing, processing, and distributed data reporting for each crowdsensing device. Furthermore, when the amount of reported data is large, the data fusion implemented at the requestor can easily cause high latency. For improving efficiencies in both energy and network resources, this paper proposes a green CSD (GCSD) paradigm. The ambient backscatter (AmB) is used to enable a battery-free mode of operation in which the received spectrum data is reported directly through backscattering without local processing. The energy for backscattering can be provided by ambient radio frequency (RF) sources. Then, relying on air computation (AirComp), the data fusion can be implemented during the backscattering process and over the air by utilizing the summation property of wireless channel. This paper illustrates the model and the implementation process of the GCSD paradigm. Closed-form expressions of detection metrics are derived for the proposed GCSD. Simulation results verify the correctness of the theoretical derivation and demonstrate the green properties of the GCSD paradigm.

A novel CSoDoA based passive islanding detection technique for multi‐DER AC microgrid control

AbstractMicro‐grid islanding methods are gaining importance as the presence of renewable energy resources is becoming increasingly popular in power distribution. Among all the islanding methods, passive methods are easy to implement and applicable to all types of DERs provided the NDZ is minimized. This article presents a novel passive islanding detection method based on the CSoDoA of ellipse formed by the successive phase voltages to address the NDZ problem. The peak magnitude and phase angle difference of voltages are the parameters to be estimated to compute the area of the ellipse. State‐of‐the‐art passive methods consider the effect of either magnitude or phase angle or frequency as an islanding detection metric. The CSoDoA metric combines the effect of both magnitude and phase angle thereby reducing the NDZ. The proposed method is validated on both synchronous and inverter‐interfaced DERs. The test systems considered are both single‐DER based MG and modified IEEE 13 bus multi‐DER MG. The CSoDoA algorithm is implemented on a Raspberry‐Pi controller and validated using HIL workbench with real‐time simulator OP‐4510. Various islanding and non‐islanding events are simulated, tested and it is evident from the results that it has a very small NDZ.

An Intelligent Anomaly Detection Approach for Accurate and Reliable Weather Forecasting at IoT Edges: A Case Study.

Industrialization and rapid urbanization in almost every country adversely affect many of our environmental values, such as our core ecosystem, regional climate differences and global diversity. The difficulties we encounter as a result of the rapid change we experience cause us to encounter many problems in our daily lives. The background of these problems is rapid digitalization and the lack of sufficient infrastructure to process and analyze very large volumes of data. Inaccurate, incomplete or irrelevant data produced in the IoT detection layer causes weather forecast reports to drift away from the concepts of accuracy and reliability, and as a result, activities based on weather forecasting are disrupted. A sophisticated and difficult talent, weather forecasting needs the observation and processing of enormous volumes of data. In addition, rapid urbanization, abrupt climate changes and mass digitization make it more difficult for the forecasts to be accurate and reliable. Increasing data density and rapid urbanization and digitalization make it difficult for the forecasts to be accurate and reliable. This situation prevents people from taking precautions against bad weather conditions in cities and rural areas and turns into a vital problem. In this study, an intelligent anomaly detection approach is presented to minimize the weather forecasting problems that arise as a result of rapid urbanization and mass digitalization. The proposed solutions cover data processing at the edge of the IoT and include filtering out the missing, unnecessary or anomaly data that prevent the predictions from being more accurate and reliable from the data obtained through the sensors. Anomaly detection metrics of five different machine learning (ML) algorithms, including support vector classifier (SVC), Adaboost, logistic regression (LR), naive Bayes (NB) and random forest (RF), were also compared in the study. These algorithms were used to create a data stream using the time, temperature, pressure, humidity and other sensor-generated information.

Multi-modal pedestrian detection with misalignment based on modal-wise regression and multi-modal IoU

Multi-modal pedestrian detection, which integrates visible and thermal sensors, has been developed to overcome many limitations of visible-modal pedestrian detection, such as poor illumination, cluttered background, and occlusion. By adopting the combination of multiple modalities, we can efficiently detect pedestrians even with poor visibility. Nevertheless, the critical assumption of multi-modal pedestrian detection is that multi-modal images are perfectly aligned. In general, however, this assumption often becomes invalid in real-world situations. Viewpoints of the different modal sensors are usually different. Then, the positions of pedestrians on the different modal images have disparities. We proposed a multi-modal faster-RCNN specifically designed to handle misalignment between two modalities. The faster-RCNN consists of a region proposal network (RPN) and a detector. We introduce position regressors for both modalities in the RPN and the detector. Intersection over union (IoU) is one of the useful metrics for object detection but is defined only for a single-modal image. We extend it into multi-modal IoU to evaluate the preciseness of both modalities. Our experimental results with the proposed evaluation metrics demonstrate that the proposed method has comparable performance with state-of-the-art methods and outperforms them for data with significant misalignment.