Intersection Over Union Research Articles

Fast Semantic Segmentation of Ultra-High-Resolution Remote Sensing Images via Score Map and Fast Transformer-Based Fusion

For ultra-high-resolution (UHR) image semantic segmentation, striking a balance between computational efficiency and storage space is a crucial research direction. This paper proposes a Feature Fusion Network (EFFNet) to improve UHR image semantic segmentation performance. EFFNet designs a score map that can be embedded into the network for training purposes, enabling the selection of the most valuable features to reduce storage consumption, accelerate speed, and enhance accuracy. In the fusion stage, we improve upon previous redundant multiple feature fusion methods by utilizing a transformer structure for one-time fusion. Additionally, our combination of the transformer structure and multibranch structure allows it to be employed for feature fusion, significantly improving accuracy while ensuring calculations remain within an acceptable range. We evaluated EFFNet on the ISPRS two-dimensional semantic labeling Vaihingen and Potsdam datasets, demonstrating that its architecture offers an exceptionally effective solution with outstanding semantic segmentation precision and optimized inference speed. EFFNet substantially enhances critical performance metrics such as Intersection over Union (IoU), overall accuracy, and F1-score, highlighting its superiority as an architectural innovation in ultra-high-resolution remote sensing image semantic segmentation.

Multi-label deep learning for comprehensive optic nerve head segmentation through data of fundus images

Early diagnosis and continuous monitoring of patients with eye diseases are critical in computer-aided detection (CAD) techniques. Semantic segmentation, a key component in computer vision, enables pixel-level classification and provides detailed information about objects within images. In this study, we present three U-Net models designed for multi-class semantic segmentation, leveraging the U-Net architecture with transfer learning. To generate ground truth for the HRF dataset, we combine two U-Net models, namely MSU-Net and BU-Net, to predict probability maps for the optic disc and cup regions. Binary masks are then derived from these probability maps to extract the optic disc and cup regions from retinal images. The dataset used in this study includes pre-existing blood vessels and manually annotated peripapillary atrophy zones (alpha and beta) provided by expert ophthalmologists. This comprehensive dataset, integrating existing blood vessels and expert-marked peripapillary atrophy zones, fulfills the study's objectives. The effectiveness of the proposed approach is validated by training nine pre-trained models on the HRF dataset comprising 45 retinal images, successfully segmenting the optic disc, cup, blood vessels, and peripapillary atrophy zones (alpha and beta). The results demonstrate 87.7 % pixel accuracy, 87 % Intersection over Union (IoU), 86.9 % F1 Score, 85 % mean IoU (mIoU), and 15 % model loss, significantly contributing to the early diagnosis and monitoring of glaucoma and optic nerve disorders.

Event-driven nearshore and shoreline coastline detection on SpiNNaker neuromorphic hardware

Abstract Coastline detection is vital for coastal management, involving frequent observation and assessment to understand coastal dynamics and inform decisions on environmental protection. 
Continuous streaming of high-resolution images demands robust data processing and storage solutions to manage large datasets efficiently, posing challenges that require innovative solutions for real-time analysis and meaningful insights extraction.
This work leverages low-latency event-based vision sensors coupled with neuromorphic hardware in an attempt to decrease a two-fold challenge, reducing the computational burden to ~0.375 mW whilst obtaining a coastline detection map in as little as 20 ms.
The proposed Spiking Neural Network (SNN) runs on the SpiNNaker neuromorphic platform using a total of 18040 neurons reaching 98.33% accuracy.
The model has been characterised and evaluated by computing the accuracy of Intersection over Union (IoU) scores over the ground truth of a real-world coastline dataset across different time windows. The system's robustness was further assessed by evaluating its ability to avoid coastline detection in non-coastline profiles and funny shapes, achieving a success rate of 97.3%.

CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation.

Some cognitive research has discovered that humans accomplish event segmentation as a side effect of event anticipation. Inspired by this discovery, we propose a simple yet effective end-to-end self-supervised learning framework for event segmentation/boundary detection. Unlike the mainstream clustering-based methods, our framework exploits a transformer-based feature reconstruction scheme to detect event boundaries by reconstruction errors. This is consistent with the fact that humans spot new events by leveraging the deviation between their prediction and what is perceived. Thanks to their heterogeneity in semantics, the frames at boundaries are difficult to be reconstructed (generally with large reconstruction errors), which is favorable for event boundary detection. In addition, since the reconstruction occurs on the semantic feature level instead of the pixel level, we develop a temporal contrastive feature embedding (TCFE) module to learn the semantic visual representation for frame feature reconstruction (FFR). This procedure is like humans building up experiences with "long-term memory." The goal of our work is to segment generic events rather than localize some specific ones. We focus on achieving accurate event boundaries. As a result, we adopt the F1 score (Precision/Recall) as our primary evaluation metric for a fair comparison with previous approaches. Meanwhile, we also calculate the conventional frame-based mean over frames (MoF) and intersection over union (IoU) metric. We thoroughly benchmark our work on four publicly available datasets and demonstrate much better results. The source code is available at https://github.com/wang3702/CoSeg.

Cross‐Layer Connection SegFormer Attention U‐Net for Efficient TRUS Image Segmentation

ABSTRACTAccurately and rapidly segmenting the prostate in transrectal ultrasound (TRUS) images remains challenging due to the complex semantic information in ultrasound images. The paper discusses a cross‐layer connection with SegFormer attention U‐Net for efficient TRUS image segmentation. The SegFormer framework is enhanced by reducing model parameters and complexity without sacrificing accuracy. We introduce layer‐skipping connections for precise positioning and combine local context with global dependency for superior feature recognition. The decoder is improved with Multi‐layer Perceptual Convolutional Block Attention Module (MCBAM) for better upsampling and reduced information loss, leading to increased accuracy. The experimental results show that compared with classic or popular deep learning methods, this method has better segmentation performance, with the dice similarity coefficient (DSC) of 97.55% and the intersection over union (IoU) of 95.23%. This approach balances encoder efficiency, multi‐layer information flow, and parameter reduction.

Coal-rock interface real-time recognition based on the improved YOLO detection and bilateral segmentation network

To improve the accuracy and efficiency of coal-rock interface recognition, this study proposes a model built on the real-time detection algorithm, you only look once (YOLO), and the lightweight bilateral segmentation network. Simultaneously, the regional similarity transformation function and dragonfly algorithm are introduced to enhance the quality of coal-rock images. The comparison with three other models demonstrates the superior edge inference performance of the proposed model, achieving a mean Average Precision (mAP) of 90.2 at the Intersection over Union (IoU) threshold of 0.50 (mAP50) and 81.4 across a range of IoU thresholds from 0.50 to 0.95 (mAP[50,95]). Furthermore, to maintain high accuracy and real-time recognition capabilities, the proposed model is optimized using the open visual inference and neural network optimization toolkit, resulting in a 144.97% increase in the mean frames per second. Experimental results on four actual coal faces confirm the efficacy of the proposed model, showing a better balance between accuracy and efficiency in coal-rock image recognition, which supports further advancements in coal mining intelligence.

Instance segmentation of cells and nuclei from multi-organ cross-protocol microscopic images.

Light microscopy is a widely used technique in cell biology due to its satisfactory resolution for cellular structure analysis, prevalent availability of fluorescent probes for staining, and compatibility for the dynamic analysis of live cells. However, the segmentation of cells and nuclei from microscopic images is not a straightforward process because it has several challenges such as high variation in morphology and shape, the presence of noise and diverse contrast in backgrounds, clustering or overlapping nature of cells. Dealing with these challenges and facilitating more reliable analysis necessitates the implementation of computer-aided methods that leverage image processing techniques and deep learning algorithms. The major goal of this study is to propose a model, for instance segmentation of cells and nuclei, applying the most cutting-edge deep learning techniques. A fine-tuned You Only Look at Once version 9 extended (YOLOv9-E) model is initially applied as a prompt generator to generate bounding box prompts. Using the generated prompts, a pre-trained segment anything model (SAM) is subsequently applied through zero-short inferencing to produce raw segmentation masks. These segmentation masks are then refined using non-max suppression and simple image processing methods such as image addition and morphological processing. The proposed method is developed and evaluated using an open-sourced dataset called Expert Visual Cell Annotation (EVICAN), which is relatively large and contains 4,738 microscopy images extracted from cross organs using different protocols. Based on the evaluation results on three different levels of EVICAN test sets, the proposed method demonstrates noticeable performances showing average mAP50 [mean average precision at intersection over union (IoU) =0.50] scores of 96.25, 95.05, and 94.18 for cell segmentation, and 68.04, 54.66, and 38.29 for nucleus segmentation on easy, medium, and difficult test sets, respectively. Our proposed method for instance segmentation of cells and nuclei provided favorable performance compared to the existing methods in literature, indicating its potential utility as an assistive tool for cell culture experts, facilitating prompt and reliable analysis.

Intelligent system based on multiple networks for accurate ovarian tumor semantic segmentation

Ovarian tumors, especially malignant ones, represent a global concern, with increased prevalence in recent years. More accurate medical support systems are urgently needed to support medical staff in obtaining an efficient ovarian tumors diagnosis since detection in early stages could lead to immediately applying appropriate treatment, and implicitly improving the survival rate. The current paper aims to demonstrate that more accurate systems could be designed by combining different convolutional neural networks using different custom combination approaches and by selecting the appropriate networks to be involved in the ensemble model to achieve the best performance metrics. It is essential to understand if combining all experimented networks or only the best-performing ones could always lead to the most effective results or not. The current paper is structured in three main phases. The first step is to propose the individual networks involved in the experiments. Five DeepLab-V3+ networks with different encoders (ResNet-18, ResNet-50, MobileNet-V2, InceptionResNet-V2, and Xception) were used. In the second step, the paper proposes a custom algorithm to combine multiple individual semantic segmentation networks, while the last step describes the iterative selection approach for selecting all individual networks to be combined so that the most accurate ensemble is obtained. The system performing semantic segmentation for different types of ovarian tumors, covering both benign and malignant ones, achieved 91.18 % Intersection over union (IoU), thus overperforming all individual networks. The proposed method could be extended so that more powerful deep learning models could be used.

276. AI NAVIGATION SURGERY FOR ESOPHAGEAL CANCER

Abstract With recent technological innovations, artificial intelligence (AI) has begun to be used not only in endoscopic diagnosis and pathology diagnosis but also in the field of surgery. AI enables objective and consistent recognition and evaluation, making it suitable for surgical assistance in surgical operations as well as for the evaluation of surgical techniques and learning curves. First, an AI-assisted surgical procedure recognition system for robotic esophagectomy was developed. By focusing on surgical procedures, the system automatically recognizes surgical procedures with 84% accuracy using AI. In particular, more than 90% of accuracy was obtained in the phase for Left RLN lymph node dissection and Post-dissection to completion of surgery. In addition, as intraoperative navigation, a system for automatic recognition of critical organs such as the recurrent laryngeal nerve, trachea, and subclavian artery was developed, and its recognition accuracy is gradually improving, enabling organ recognition in real time. Currently, with data from more than 100 cases, the system is able to predict Intersection over Union (IoU) with an accuracy of 0.3-0.4. Similar developments are underway in other areas, such as automatic recognition of the surgical process in robotic distal gastrectomy, and a surgical difficulty prediction system is being built using this technology. This system is expected to make it possible to automatically determine the degree of difficulty of surgery during the operation, leading to the decision to change the surgical team during the operation and the prediction of the operation completion time. These technologies can be useful not only for intraoperative navigation, but also surgical education and postoperative care. Using them, young surgeons have the opportunity to dive into authentic scenarios and acquire practical experience safely, bypassing the dangers inherent in real-life surgeries. Through the use of AI-powered interactive simulations, they can deeply grasp surgical methods, establishing the groundwork of crucial practical abilities needed for performing successful operations. In the future, we intend to verify the external validity of these systems using data from multiple facilities, and further work on the development of a surgical support system that transcends fields, with the aim of obtaining regulatory approval as a medical device. In addition to the potential clinical applications of the systems under development, we would like to discuss the current status and future of AI-based surgical support systems in the surgical field.

Double-Exposure Algorithm: A Powerful Approach to Address the Accuracy Issues of Fractional Vegetation Extraction under Shadow Conditions

When recording the vegetation distribution with a camera, shadows can form due to factors like camera angle and direct sunlight. These shadows result in the loss of pixel information and texture details, significantly reducing the accuracy of fractional vegetation coverage (FVC) extraction. To address this issue, this study proposes an efficient double-exposure algorithm. The method reconstructs the pixel information in shadow areas by fusing normal-exposure and overexposed images. This approach overcomes the limitations of the camera’s dynamic range in capturing pixel information in shadowed regions. The study evaluates images with five levels of overexposure combined with five vegetation extraction indices. The aim is to determine the best-performing double-exposure combination under shadow conditions and the most suitable vegetation index. Experimental results reveal that the R² value between the best vegetation index and the FVC calculated from the fused double-exposure images and the ground truth FVC increases from 0.750 to 0.969. The root mean square error (RMSE) reduces from 0.146 to 0.046, and the intersection over union (IOU) increases from 0.856 to 0.943. These results demonstrate the excellent vegetation extraction capability of the double-exposure algorithm under shadow conditions, offering a straightforward and effective solution to low accuracy of FVC in shadowed areas.

Cell nuclei image segmentation using U-Net and DeepLabV3+ with transfer learning and regularization

Semantic nuclei segmentation is a challenging area of computer vision. Accurate nuclei segmentation can help medics in diagnosing many diseases. Automatic nuclei segmentation can help medics in diagnosing many diseases such as cancer by providing automatic tissue analysis. Deep learning algorithms allow automatic feature extraction from medical images, however, hematoxylin and eosin (H&E) stained images are challenging due to variability in staining and textures. Using pre-trained models in deep learning speeds up development and improves their performance. This paper compares Deeplabv3+ and U-Net deep learning methods with the pre-trained models ResNet-50 and EfficientNetB4 embedded in their architecture. In addition, different regularization and dropout parameters are applied to prevent overtraining. The experiment was conducted on the PanNuke dataset consisting of nearly 8,000 histological images and annotated nuclei. As a result, the ResNet50-based DeepLabV3+ model with L2 regularization of 0.02 and dropout of 0.7 showed efficiency with dice coefficient (DCS) of 0.8356, intersection over union (IOU) of 0.7280, and loss of 0.3212 on the test set.

Classification and segmentation of kidney MRI images for chronic kidney disease detection

Chronic Kidney Disease (CKD) is a common ailment with significant public health implications, underscoring the critical importance of early detection and diagnosis for effective management. Our research focuses on utilizing deep learning models to accurately classify and segment kidney MRI images, aiding in the timely detection and diagnosis of CKD. Utilizing the Zenodo dataset comprising 1299 kidney MRI images from 100 T2-weighted abdomen MRI scans, our investigation reveals that Vision Transformers (ViT) based model EfficientNet_b1 exhibited superior performance compared to other models. It achieved an accuracy of 94.38% in distinguishing between healthy and CKD cases in kidney MRI image classification. Moreover, the pretrained models Densenet201 and VGG19, as well as the ViT-based models EfficientNet_b1 and tf_mobilenetv3_large_075, demonstrate remarkable sensitivity and accuracy while compared to identifying kidney MRI images which highlights the effectiveness of deep learning architectures in identifying diseases. Additionally, our segmentation analysis reveals that our proposed network, Resnet18-Self-ONN-UNet++, adeptly delineates kidney structures. The network shows remarkable performance in medical diagnosis and treatment planning, with an Intersection over Union (IoU) of 82.34% and DS of 91.57% when integrated with the Contrast Limited Adaptive Histogram Equalization (CLAHE) preprocessing technique and Simultaneous truth and performance level estimation (STAPLE) as post processing technique. In summary, deep learning-driven kidney MRI image classification and segmentation enhance medical imaging analysis, potentially aiding in early CKD diagnosis and treatment, thereby addressing critical public health concerns, and enhancing patient outcomes.

An algorithm for building contour inference fitting based on multiple contour point classification processes

Extracting buildings from True Digital Ortho Maps often encounters occlusions and misidentifications, making it challenging to obtain complete, regular, and accurate building contours. To address this issue, we developed a building recognition process based on the Segment Anything Model, and proposed a novel regularization algorithm for building contour inference and fitting, which quantifies the confidence levels of contour points to accurately fit building contours from data containing substantial noise, and reformulates the fitting problem as progressive node classification tasks consisting of contour simplification, iterative regularization, and rationality assessment. In experimental evaluations, the proposed contour fitting algorithm achieved 97.99 % Intersection over Union (IoU), 95.39 % consistency with the standard contour edge count, and 88.06 % of cases with Hausdorff distances less than or equal to 15 pixels (30 cm), significantly outperforming comparative methods. Notably, it was the only contour regularization algorithm that improved IoU (1.03 %) compared to the original contours. The experimental results demonstrate that the proposed algorithm effectively suppresses noise and infers incomplete building contours, producing accurate and regular contours comparable to manual delineation. It is particularly suitable for buildings with near-orthogonal structures, exhibiting significant practical applicability and generalization potential.

RA-XTNet: A Novel CNN Model to Predict Rheumatoid Arthritis from Hand Radiographs and Thermal Images: A Comparison with CNN Transformer and Quantum Computing.

The aim and objective of the research are to develop an automated diagnosis system for the prediction of rheumatoid arthritis (RA) based on artificial intelligence (AI) and quantum computing for hand radiographs and thermal images. The hand radiographs and thermal images were segmented using a UNet++ model and color-based k-means clustering technique, respectively. The attributes from the segmented regions were generated using the Speeded-Up Robust Features (SURF) feature extractor and classification was performed using k-star and Hoeffding classifiers. For the ground truth and the predicted test image, the study utilizing UNet++ segmentation achieved a pixel-wise accuracy of 98.75%, an intersection over union (IoU) of 0.87, and a dice coefficient of 0.86, indicating a high level of similarity. The custom RA-X-ray thermal imaging (XTNet) surpassed all the models for the detection of RA with a classification accuracy of 90% and 93% for X-ray and thermal imaging modalities, respectively. Furthermore, the study employed quantum support vector machine (QSVM) as a quantum computing approach which yielded an accuracy of 93.75% and 87.5% for the detection of RA from hand X-ray and thermal images. In addition, vision transformer (ViT) was employed to classify RA which obtained an accuracy of 80% for hand X-rays and 90% for thermal images. Thus, depending on the performance measures, the RA-XTNet model can be used as an effective automated diagnostic method to diagnose RA accurately and rapidly in hand radiographs and thermal images.

In-line semantic segmentation of kimchi cabbage deterioration using YOLOv8n and DeepLabv3+

To ensure the safety of kimchi cabbage (KC), the effective identification of deterioration is required. In this study, we aimed to develop and validate a semantic segmentation pipeline to detect and segment the mild and severe deterioration of KC on moving industrial conveyor belts. The pipeline integrates the YOLOv8n and DeepLabv3+ models, image processing and semantic labelling models, respectively. The YOLOv8n model was employed for initial detection, achieving an F1 score (a measure of precision and recall) of 0.988 and an AP50 (average precision at 50 % intersection over union (IoU) with the ground truth) of 0.994. For detailed segmentation, the DeepLabv3+ model with Resnet101 and Mobilenetv3 backbones was used, and the performance of these models was evaluated based on F1 scores. The YOLOv8n model demonstrated high accuracy in detection. Further, the DeepLabv3+ model with the Resnet101 backbone achieved F1 scores of 0.772 and 0.851, and IoUs of 0.628 and 0.741 for mild and severe deterioration, respectively. Notably, the DeepLabv3+ model with Mobilenetv3 backbone excelled in identifying severe deterioration, indicating its potential for generating rapid alerts in edge devices. Crucially, the developed methodology enables the real-time, in-line monitoring of KC deterioration and is adaptable for future edge device applications. The results of this study highlight the potential of accessible technologies in advancing agricultural quality control, setting a benchmark for future research to expand data sources and enhance detection capabilities.

Breast lesions segmentation from ultrasound images using DeepLabV3 + model with channel and spatial attention mechanism

Segmenting breast tumors in ultrasound images is pivotal for computer-aided diagnosis (CAD) systems aimed at detecting breast cancer. This research contributes directly to the Sustainable Development Goal (SDG), good health and well-being by leveraging innovative deep learning technique to boost the accuracy of breast tumor segmentation. Achieving accurate segmentation is crucial for precise tumor size, shape, and location determination, which in turn facilitates subsequent tumor quantification and classification. However, this task presents challenges, particularly when dealing with small tumors. The complexity arises from factors such as speckle noise, variations in tumor shapes and sizes across patients, and the presence of tumor-like regions in the images. Although deep learning-based methods have shown remarkable success in biomedical image analysis, the current state-of-the-art approaches still struggle to achieve satisfactory performance in segmenting small breast tumors. In this paper, BUS (Breast Ultrasound) dataset was initially applied on state of-the art DeepLabV3 + architecture for the segmentation of breast lesions in ultrasound images. After that to focus on more informative features, a modified DeepLabV3 + has been proposed by integrating a CBAM module in encoder as well as decoder. At the end, comparative analysis of state-of-art DeepLabV3 + model and proposed modified DeepLabV3 + has also been presented in terms of performance metrics like Dice Coefficient, Intersection over Union, Precision, Recall and Specificity. The proposed model demonstrates outstanding performance across different evaluation metrics achieving precision, recall, specificity, dice coefficient and Intersection over Union (IoU) values of 0.974, 0.933, 0.997, 0.951, and 0.933. respectively.

Automated Analysis Of Diabetic Vasculopathy Using Semantic Segmentation Of Thermal Images Of Peroneal Vessel

Introduction: Diabetic vascular disease is one of most serious health problems in diabetic patients, it causes the development of severe complications including delayed wound healing and increased susceptibility to infections. Methods: To provide accurate and are non-invasive diagnosis, current work emphasizes on Diabetic Vasculopathy (DV) that is analysed with thermoregulation images through Semantic Segmentation (SS). A novel methodology was adapted, combining thermoregulation imaging with SS using the U-Net++ model to investigate temperature distributions at the skin level. This work introduces a novel method that utilizes MobileNetV2 as the encoder for fast Feature Extraction (FE). Results: The results from the suggested model, achieves a segmentation accuracy of 95%, which is significantly more compared to that of DeepLabV3+ and PSPNet models. A mean and Intersection over Union (IoU) of 85% and 87% was reported by the suggested frameworks throughout the training and validation phases. Conclusion: Classifying normal and abnormal regions can be done via the outcomes, as it offers the great visibility in the thermal image for clinicians by detecting the non-thermal regions

Multi-Task Environmental Perception Methods for Autonomous Driving.

In autonomous driving, environmental perception technology often encounters challenges such as false positives, missed detections, and low accuracy, particularly in detecting small objects and complex scenarios. Existing algorithms frequently suffer from issues like feature redundancy, insufficient contextual interaction, and inadequate information fusion, making it difficult to perform multi-task detection and segmentation efficiently. To address these challenges, this paper proposes an end-to-end multi-task environmental perception model named YOLO-Mg, designed to simultaneously perform traffic object detection, lane line detection, and drivable area segmentation. First, a multi-stage gated aggregation network (MogaNet) is employed during the feature extraction process to enhance contextual interaction by improving diversity in the channel dimension, thereby compensating for the limitations of feed-forward neural networks in contextual understanding. Second, to further improve the model's accuracy in detecting objects of various scales, a restructured weighted bidirectional feature pyramid network (BiFPN) is introduced, optimizing cross-level information fusion and enabling the model to handle object detection at different scales more accurately. Finally, the model is equipped with one detection head and two segmentation heads to achieve efficient multi-task environmental perception, ensuring the simultaneous execution of multiple tasks. The experimental results on the BDD100K dataset demonstrate that the model achieves a mean average precision (mAP50) of 81.4% in object detection, an Intersection over Union (IoU) of 28.9% in lane detection, and a mean Intersection over Union (mIoU) of 92.6% in drivable area segmentation. The tests conducted in real-world scenarios show that the model performs effectively, significantly enhancing environmental perception in autonomous driving and laying a solid foundation for safer and more reliable autonomous driving systems.

Landslide Recognition Based on Machine Learning Considering Terrain Feature Fusion

Landslides are one of the major disasters that exist worldwide, posing a serious threat to human life and property safety. Rapid and accurate detection and mapping of landslides are crucial for risk assessment and humanitarian assistance in affected areas. To achieve this goal, this study proposes a landslide recognition method based on machine learning (ML) and terrain feature fusion. Taking the Dawan River Basin in Detuo Township and Tianwan Yi Ethnic Township as the research area, firstly, landslide-related data were compiled, including a landslide inventory based on field surveys, satellite images, historical data, high-resolution remote sensing images, and terrain data. Then, different training datasets for landslide recognition are constructed, including full feature datasets that fusion terrain features and remote sensing features and datasets that only contain remote sensing features. At the same time, different ratios of landslide to non-landslide (or positive/negative, P/N) samples are set in the training data. Subsequently, five ML algorithms, including Extreme Gradient Boost (XGBoost), Adaptive Boost (AdaBoost), Light Gradient Boost (LightGBM), Random Forest (RF), and Convolutional Neural Network (CNN), were used to train each training dataset, and landslide recognition was performed on the validation area. Finally, accuracy (A), precision (P), recall (R), F1 score (F1), and intersection over union (IOU) were selected to evaluate the landslide recognition ability of different models. The research results indicate that selecting ML models suitable for the study area and the ratio of the P/N samples can improve the A, R, F1, and IOU of landslide identification results, resulting in more accurate and reasonable landslide identification results; Fusion terrain features can make the model recognize landslides more comprehensively and align better with the actual conditions. The best-performing model in the study is LightGBM. When the input data includes all features and the P/N sample ratio is optimal, the A, P, R, F1, and IOU of landslide recognition results for this model are 97.47%, 85.40%, 76.95%, 80.95%, and 71.28%, respectively. Compared to the landslide recognition results using only remote sensing features, this model shows improvements of 4.51%, 35.66%, 5.41%, 22.27%, and 29.16% in A, P, R, F1, and IOU, respectively. This study serves as a valuable reference for the precise and comprehensive identification of landslide areas.

Unveiling the human-like similarities of automatic facial expression recognition: An empirical exploration through explainable ai

AbstractFacial expression recognition is vital for human behavior analysis, and deep learning has enabled models that can outperform humans. However, it is unclear how closely they mimic human processing. This study aims to explore the similarity between deep neural networks and human perception by comparing twelve different networks, including both general object classifiers and FER-specific models. We employ an innovative global explainable AI method to generate heatmaps, revealing crucial facial regions for the twelve networks trained on six facial expressions. We assess these results both quantitatively and qualitatively, comparing them to ground truth masks based on Friesen and Ekman’s description and among them. We use Intersection over Union (IoU) and normalized correlation coefficients for comparisons. We generate 72 heatmaps to highlight critical regions for each expression and architecture. Qualitatively, models with pre-trained weights show more similarity in heatmaps compared to those without pre-training. Specifically, eye and nose areas influence certain facial expressions, while the mouth is consistently important across all models and expressions. Quantitatively, we find low average IoU values (avg. 0.2702) across all expressions and architectures. The best-performing architecture averages 0.3269, while the worst-performing one averages 0.2066. Dendrograms, built with the normalized correlation coefficient, reveal two main clusters for most expressions: models with pre-training and models without pre-training. Findings suggest limited alignment between human and AI facial expression recognition, with network architectures influencing the similarity, as similar architectures prioritize similar facial regions.