Intersection Over Union Research Articles

Multi-type stroke lesion segmentation: comparison of single-stage and hierarchical approach.

Stroke, a major cause of death and disability worldwide, can be haemorrhagic or ischaemic depending on the type of bleeding in the brain. Rapid and accurate identification of stroke type and lesion segmentation is critical for timely and effective treatment. However, existing research primarily focuses on segmenting a single stroke type, potentially limiting their clinical applicability. This study addresses this gap by exploring multi-type stroke lesion segmentation using deep learning methods. Specifically, we investigate two distinct approaches: a single-stage approach that directly segments all tissue types in one model and a hierarchical approach that first classifies stroke types and then utilises specialised segmentation models for each subtype. Recognising the importance of accurate stroke classification for the hierarchical approach, we evaluate ResNet, ResNeXt and ViT networks, incorporating focal loss and oversampling techniques to mitigate the impact of class imbalance. We further explore the performance of U-Net, U-Net++ and DeepLabV3 models for segmentation within each approach. We use a comprehensive dataset of 6650 images provided by the Ministry of Health of the Republic of Türkiye. This dataset includes 1130 ischaemic strokes, 1093 haemorrhagic strokes and 4427 non-stroke cases. In our comparative experiments, we achieve an AUC score of 0.996 when classifying stroke and non-stroke slices. For lesion segmentation task, while the performance of different architectures is comparable, the hierarchical training approach outperforms the single-stage approach in terms of intersection over union (IoU). The performance of the U-Net model increased significantly from an IoU of 0.788 to 0.875 when the hierarchical approach is used. This comparative analysis aims to identify the most effective approach and deep learning model for multi-type stroke lesion segmentation in brain CT scans, potentially leading to improved clinical decision-making, treatment efficiency and outcomes.

Camera-view supervision for bird's-eye-view semantic segmentation

Bird's-eye-view Semantic Segmentation (BEVSS) is a powerful and crucial component of planning and control systems in many autonomous vehicles. Current methods rely on end-to-end learning to train models, leading to indirectly supervised and inaccurate camera-to-BEV projections. We propose a novel method of supervising feature extraction with camera-view depth and segmentation information, which improves the quality of feature extraction and projection in the BEVSS pipeline. Our model, evaluated on the nuScenes dataset, shows a 3.8% improvement in Intersection-over-Union (IoU) for vehicle segmentation and a 30-fold reduction in depth error compared to baselines, while maintaining competitive inference times of 32 FPS. This method offers more accurate and reliable BEVSS for real-time autonomous driving systems. The codes and implementation details and code can be found at https://github.com/bluffish/sucam.

An enhanced YOLOv8n object detector for synthetic diamond quality evaluation.

To address the need for automated sorting of synthetic diamonds based on quality in manufacturing enterprises, this study developed a dedicated dataset and an enhanced YOLOv8n model for synthetic diamonds detection and quality evaluation, named YOLOv8n-adamas. We redesigned the backbone network to improve feature extraction capabilities and introduced a dynamic detection head based on attention mechanisms to further enhance model performance. Experimental results show that on synthetic diamonds dataset, YOLOv8n-adamas achieved a 4.0% improvement in precision (P), a 2.7% increase in recall (R), and improvements of 1.5% and 1.3% in mean average precisions at 50% and 95% Intersection over Union (IoU) thresholds (mAP50 and mAP95) compared to YOLOv8. Furthermore, YOLOv8n-adamas also outperforms other commonly used, high-performing models in various metrics on this dataset, offering effective technical support for the automated quality-based sorting of synthetic diamonds.

In-Situ visual reconstruction of strut profiles in pulsed wire arc additive manufacturing of lattice structures

ABSTRACT The wire arc additive manufacturing (WAAM) process often results in geometric inaccuracies in the produced struts, highlighting the need for in-situ monitoring. Industrial cameras struggle to capture accurate images due to interference from arc light and metal radiation. This study introduces a real-time visual reconstruction method that leverages regions of interest (ROI) to enhance profile accuracy. Initially, the melt pool is analyzed and segmented from a series of frames to establish an initial strut profile. This profile then serves as the ROI for segmenting the solidified weld bead regions, yielding a more precise strut profile. Comparative analysis shows that the proposed method achieves a high Intersection Over Union (IOU) score of 0.911 and an Average Symmetric Surface Distance (ASD) error of 0.355 mm, demonstrating both accuracy and robustness. Additionally, the inclination angle and diameter are extracted from the reconstructed profiles.

Enhancing detection of various pancreatic lesions on endoscopic ultrasound through artificial intelligence: a basis for computer-aided detection systems.

Endoscopic ultrasound (EUS) is the most sensitive method for evaluation of pancreatic lesions but is limited by significant operator dependency. Artificial intelligence (AI), in the form of computer-aided detection (CADe) systems, has shown potential in increasing accuracy and bridging operator dependency in several endoscopic domains. However, the complexity of integrating AI into EUS is far more challenging. This aims to develop and test the basis for a CADe system for real-time detection and segmentation of all pancreatic lesions. In this single-center study EUS studies of pancreatic findings were included. Lesions were outlined by two expert (>5years performing EUS) endoscopists, and the two leading types of models were benchmarked. The models' performance was evaluated through per-pixel intersection over union (IoU). A total of 1497 EUS images from 165 patients were evaluated. The dataset included malignancies, neuroendocrine tumors, benign cysts, chronic and acute pancreatitis, normal fatty pancreas, and benign lesions. The best model demonstrated detection and segmentation on the test set with a mean IoU of 0.73, achieving a PPV, NPV, total accuracy, and ROC of 0.82, 0.96, 0.95, and 0.95, respectively. The algorithm is adaptable for real-time processing. We developed and tested deep learning models for real-time detection and segmentation of pancreatic lesions during EUS with promising results. This constitutes the basis for a CADe system for EUS, which could be valuable in future detection and evaluation of pancreatic lesions. Further studies for validation and generalization are underway.

Three-Dimensional Instance Segmentation Using the Generalized Hough Transform and the Adaptive n-Shifted Shuffle Attention

The progress of 3D instance segmentation techniques has made it essential for several applications, such as augmented reality, autonomous driving, and robotics. Traditional methods usually have challenges with complex indoor scenes made of multiple objects with different occlusions and orientations. In this work, the authors present an innovative model that integrates a new adaptive n-shifted shuffle (ANSS) attention mechanism with the Generalized Hough Transform (GHT) for robust 3D instance segmentation of indoor scenes. The proposed technique leverages the n-shifted sigmoid activation function, which improves the adaptive shuffle attention mechanism, permitting the network to dynamically focus on relevant features across various regions. A learnable shuffling pattern is produced through the proposed ANSS attention mechanism to spatially rearrange the relevant features, thus augmenting the model’s ability to capture the object boundaries and their fine-grained details. The integration of GHT furnishes a vigorous framework to localize and detect objects in the 3D space, even when heavy noise and partial occlusions are present. The authors evaluate the proposed method on the challenging Stanford 3D Indoor Spaces Dataset (S3DIS), where it establishes its superiority over existing methods. The proposed approach achieves state-of-the-art performance in both mean Intersection over Union (IoU) and overall accuracy, showcasing its potential for practical deployment in real-world scenarios. These results illustrate that the integration of the ANSS and the GHT yields a robust solution for 3D instance segmentation tasks.

Deep Learning in Endoscopic Ultrasound: A Breakthrough in Detecting Distal Cholangiocarcinoma

Introduction: Cholangiocarcinoma (CCA) is a highly lethal malignancy originating in the bile ducts, often diagnosed late with poor prognosis. Differentiating benign from malignant biliary tumors remains challenging, necessitating advanced diagnostic techniques. Objective: This study aims to enhance the diagnostic accuracy of endoscopic ultrasound (EUS) for distal cholangiocarcinoma (dCCA) using advanced convolutional neural networks (CCNs) for the classification and segmentation of EUS images, specifically targeting dCCAs, the pancreas, and the bile duct. Materials and Methods: In this retrospective study, EUS images from patients diagnosed with dCCA via biopsy and an EUS-identified bile duct tumor were evaluated. A custom CNN was developed for classification, trained on 156 EUS images. To enhance the model’s robustness, image augmentation techniques were applied, generating a total of 1248 images. For tumor and organ segmentation, the DeepLabv3+ network with ResNet50 architecture was utilized, employing Tversky loss to manage unbalanced classes. Performance evaluation included metrics such as accuracy, sensitivity, specificity, and Intersection over Union (IoU). These methods were implemented in collaboration with the ADAPTED Research Group at the Technical University of Cluj-Napoca. Results: The classification model achieved a high accuracy of 97.82%, with precision and specificity both at 100% and sensitivity at 94.44%. The segmentation models for the pancreas and bile duct demonstrated global accuracies of 84% and 90%, respectively, with robust IoU scores indicating good overlap between predicted and actual contours. The application performed better than the UNet model, particularly in generalization and boundary delineation. Conclusions: This study demonstrates the significant potential of AI in EUS imaging for dCCA, presenting a robust tool that enhances diagnostic accuracy and efficiency. The developed MATLAB application serves as a valuable aid for medical professionals, facilitating informed decision-making and improving patient outcomes in the diagnosis of cholangiocarcinoma and related pathologies.

Automated Vertebral Bone Quality Determination from T1-Weighted Lumbar Spine MRI Data Using a Hybrid Convolutional Neural Network–Transformer Neural Network

Vertebral bone quality (VBQ) is a promising new method that can improve screening for osteoporosis. The drawback of the current method is that it requires manual determination of the regions of interest (ROIs) of vertebrae and cerebrospinal fluid (CSF) by a radiologist. In this work, an automatic method for determining the VBQ is proposed, in which the ROIs are obtained using a trained neural network model. A large, publicly available dataset of sagittal lumbar spine MRI images with ground truth segmentations was used to train a BRAU-Net++ hybrid CNN–transformer neural network. The performance of the trained model was evaluated using the dice similarity coefficient (DSC), accuracy, precision, recall and intersection-over-union (IoU) metrics. The trained model performed similarly to state-of-the-art lumbar spine segmentation models, with an average DSC value of 0.914 ± 0.007 for the vertebrae and 0.902 for the spinal canal. Four different methods of VBQ determination with automatic segmentation are presented and compared with one-way ANOVA. These methods use different algorithms for CSF extraction from the segmentation of the spinal canal using T1- and T2-weighted image data and applying erosion to the vertebral ROI to avoid a sharp change in SI at the edge of the vertebral body.

Building Extraction from Unmanned Aerial Vehicle (UAV) Data in a Landslide-Affected Scattered Mountainous Area Based on Res-Unet

Building extraction in landslide-affected scattered mountainous areas is essential for sustainable development, as it improves disaster risk management, fosters sustainable land use, safeguards the environment, and bolsters socio-economic advancement; however, this process entails considerable challenges. This study proposes a Res-Unet-based model to extract landslide-affected buildings from unmanned aerial vehicle (UAV) data in scattered mountain regions, leveraging the feature extraction capabilities of ResNet and the precise localization abilities of U-Net. A landslide-affected, scattered mountainous region within the Three Gorges Reservoir area was selected as a case study to validate the model’s performance. Experimental results indicate that Res-Unet displays high accuracy and robustness in building recognition, attaining accuracy (ACC), intersection-over-union (IOU), and F1-score values of 0.9849, 0.9785, and 0.9892, respectively. This enhancement can be attributed to the combined model, which amalgamates the skip connections, the symmetric architecture of U-Net, and the residual blocks of ResNet. This integration preserves low-level detail during recovery at higher levels, facilitating the extraction of multi-scale features while also mitigating the vanishing gradient problem prevalent in deep network training through the residual block structure, thus enabling the extraction of more complex features. The proposed Res-Unet approach shows significant potential for the accurate recognition and extraction of buildings in complex terrains through the efficient processing of remote sensing images.

Integrating Super-Resolution with Deep Learning for Enhanced Periodontal Bone Loss Segmentation in Panoramic Radiographs

Periodontal disease is a widespread global health concern that necessitates an accurate diagnosis for effective treatment. Traditional diagnostic methods based on panoramic radiographs are often limited by subjective evaluation and low-resolution imaging, leading to suboptimal precision. This study presents an approach that integrates Super-Resolution Generative Adversarial Networks (SRGANs) with deep learning-based segmentation models to enhance the segmentation of periodontal bone loss (PBL) areas on panoramic radiographs. By transforming low-resolution images into high-resolution versions, the proposed method reveals critical anatomical details that are essential for precise diagnostics. The effectiveness of this approach was validated using datasets from the Chungbuk National University Hospital and the Kaggle data portal, demonstrating significant improvements in both image resolution and segmentation accuracy. The SRGAN model, evaluated using the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) metrics, achieved a PSNR of 30.10 dB and an SSIM of 0.878, indicating high fidelity in image reconstruction. When applied to semantic segmentation using a U-Net architecture, the enhanced images resulted in a dice similarity coefficient (DSC) of 0.91 and an intersection over union (IoU) of 84.9%, compared with 0.72 DSC and 65.4% IoU for native low-resolution images. These results underscore the potential of SRGAN-enhanced imaging to improve PBL area segmentation and suggest broader applications in medical imaging, where enhanced image clarity is crucial for diagnostic accuracy. This study also highlights the importance of further research to expand the dataset diversity and incorporate clinical validation to fully realize the benefits of super-resolution techniques in medical diagnostics.

Pixels relationship analysis for extracting building footprints

Abstract. Currently, remote sensing research is focused on developing an automated algorithm that can compete with empirical methods for mapping the contours of individual buildings. Despite facing numerous challenges related to suboptimal imaging conditions, diverse building architecture, and complex backgrounds, creating such an algorithm is essential for monitoring urban and natural areas, generating 3D city models, managing disasters, and estimating population density. Obtaining the polygonal boundary of a building and extracting a vectorized building mask as output for direct use is one of the current challenges in drawing building outlines. This work provides a comprehensive workflow for building extraction and improves their predicted area by regularization the boundaries of buildings. First, a convolutional neural network is used to train the binary semantic segmentation model and then regularization and vectorization processes are performed. The main difference from existing methods is a new regularization method based on compiling a neighborhood matrix for each point belonging to the “building” class. According to the experimental results, the algorithm shows high efficiency: IOU (intersection over union) = 91.2%, AP (average precision) = 64.1% and AR (average recall) = 75.1%, comparable to leading building boundary extraction methods such as PolyWorld.

The Segment Anything foundation model achieves favorable brain tumor auto-segmentation accuracy in MRI to support radiotherapy treatment planning.

Promptable foundation auto-segmentation models like Segment Anything (SA, Meta AI, New York, USA) represent anovel class of universal deep learning auto-segmentation models that could be employed for interactive tumor auto-contouring in RT treatment planning. Segment Anything was evaluated in an interactive point-to-mask auto-segmentation task for glioma brain tumor auto-contouring in 16,744 transverse slices from 369 MRI datasets (BraTS 2020 dataset). Up to nine interactive point prompts were automatically placed per slice. Tumor boundaries were auto-segmented on contrast-enhanced T1w sequences. Out of the three auto-contours predicted by SA, accuracy was evaluated for the contour with the highest calculated IoU (Intersection over Union, "oracle mask," simulating interactive model use with selection of the best tumor contour) and for the tumor contour with the highest model confidence ("suggested mask"). Mean best IoU (mbIoU) using the best predicted tumor contour (oracle mask) in full MRI slices was 0.762 (IQR 0.713-0.917). The best 2D mask was achieved after amean of 6.6 interactive point prompts (IQR 5-9). Segmentation accuracy was significantly better for high- compared to low-grade glioma cases (mbIoU 0.789 vs. 0.668). Accuracy was worse using the suggested mask (0.572). Stacking best tumor segmentations from transverse MRI slices, mean 3D Dice score for tumor auto-contouring was 0.872, which was improved to 0.919 by combining axial, sagittal, and coronal contours. The Segment Anything foundation segmentation model can achieve high accuracy for glioma brain tumor segmentation in MRI datasets. The results suggest that foundation segmentation models could facilitate RT treatment planning when properly integrated in aclinical application.

An Improved UNet-Based Path Recognition Method in Low-Light Environments

The fruit industry is a significant economic sector in China, with modern orchards gradually transitioning to trellis orchards. For mechanized orchard operations, automatic vehicle navigation is essential. However, in trellis orchards, the shading from trees results in low average light intensity and large variations in lighting, posing challenges for path navigation. To address this, a path navigation algorithm for trellis orchards is proposed based on the UNet-CBAM model. The network structures of UNet, FCN, and SegNet are compared to identify and select the optimal structure for further improvement. Among the three attention mechanisms of channel attention, spatial attention, and combined attention, the most effective mechanism is identified. The optimal attention mechanism is incorporated into the optimized network to enhance the model’s ability to detect path edges and improve detection performance. To validate the effectiveness and generalizability of the model, a total of 400 images were collected under varying lighting intensities. The experimental results show that this method achieves an accuracy of 97.63%, a recall of 93.94%, and an Intersection over Union (IoU) of 92.19%. These results significantly enhance path recognition accuracy in trellis orchards, particularly under low light under conditions. Compared to the FCN and SegNet algorithms, this method provides higher detection accuracy and offers a new theoretical foundation and research approach for path recognition in low-light environments.

Autonomous Extraction Technology for Aquaculture Ponds in Complex Geological Environments Based on Multispectral Feature Fusion of Medium-Resolution Remote Sensing Imagery

Coastal aquaculture plays a crucial role in global food security and the economic development of coastal regions, but it also causes environmental degradation in coastal ecosystems. Therefore, the automation, accurate extraction, and monitoring of coastal aquaculture areas are crucial for the scientific management of coastal ecological zones. This study proposes a novel deep learning- and attention-based median adaptive fusion U-Net (MAFU-Net) procedure aimed at precisely extracting individually separable aquaculture ponds (ISAPs) from medium-resolution remote sensing imagery. Initially, this study analyzes the spectral differences between aquaculture ponds and interfering objects such as saltwater fields in four typical aquaculture areas along the coast of Liaoning Province, China. It innovatively introduces a difference index for saltwater field aquaculture zones (DIAS) and integrates this index as a new band into remote sensing imagery to increase the expressiveness of features. A median augmented adaptive fusion module (MEA-FM), which adaptively selects channel receptive fields at various scales, integrates the information between channels, and captures multiscale spatial information to achieve improved extraction accuracy, is subsequently designed. Experimental and comparative results reveal that the proposed MAFU-Net method achieves an F1 score of 90.67% and an intersection over union (IoU) of 83.93% on the CHN-LN4-ISAPS-9 dataset, outperforming advanced methods such as U-Net, DeepLabV3+, SegNet, PSPNet, SKNet, UPS-Net, and SegFormer. This study’s results provide accurate data support for the scientific management of aquaculture areas, and the proposed MAFU-Net method provides an effective method for semantic segmentation tasks based on medium-resolution remote sensing images.

Differences in regions of interest to identify deeply invasive colorectal cancers: Computer-aided diagnosis vs expert endoscopists.

Background and study aims Diagnostic performance of a computer-aided diagnosis (CAD) system for deep submucosally invasive (T1b) colorectal cancer was excellent, but the "regions of interest" (ROI) within images are not obvious. Class activation mapping (CAM) enables identification of the ROI that CAD utilizes for diagnosis. The purpose of this study was a quantitative investigation of the difference between CAD and endoscopists. Patients and methods Endoscopic images collected for validation of a previous study were used, including histologically proven T1b colorectal cancers (n = 82; morphology: flat 36, polypoid 46; median maximum diameter 20 mm, interquartile range 15-25 mm; histological subtype: papillary 5, well 51, moderate 24, poor 2; location: proximal colon 26, distal colon 27, rectum 29). Application of CAM was limited to one white light endoscopic image (per lesion) to demonstrate findings of T1b cancers. The CAM images were generated from the weights of the previously fine-tuned ResNet50. Two expert endoscopists depicted the ROI in identical images. Concordance of the ROI was rated by intersection over union (IoU) analysis. Results Pixel counts of ROIs were significantly lower using 165K[x103] [108K-227K] than by endoscopists (300K [208K-440K]; P < 0.0001) and median [interquartile] of the IoU was 0.198 [0.024-0.349]. IoU was significantly higher in correctly identified lesions (n = 54, 0.213 [0.116-0.364]) than incorrect ones (n=28, 0.070 [0.000-0.2750, P = 0.033). Concusions IoU was larger in correctly diagnosed T1b colorectal cancers. Optimal annotation of the ROI may be the key to improving diagnostic sensitivity of CAD for T1b colorectal cancers.

CSGO: A Deep Learning Pipeline for Whole-Cell Segmentation in Hematoxylin and Eosin Stained Tissues

Accurate whole-cell segmentation is essential in various biomedical applications, particularly in studying the tumor microenvironment (TME). Despite advancements in machine learning for nuclei segmentation in hematoxylin and eosin (H&E) stained images, there remains a need for effective whole-cell segmentation methods. This study aims to develop a deep learning-based pipeline to automatically segment cells in H&E-stained tissues, thereby advancing the capabilities of pathological image analysis. The Cell Segmentation with Globally Optimized boundaries (CSGO) framework integrates nuclei and membrane segmentation algorithms, followed by post-processing using an energy-based watershed method. Specifically, we employed the You Only Look Once (Yolo) object detection algorithm for nuclei segmentation and U-Net for membrane segmentation. The membrane detection model was trained on a dataset of 7 hepatocellular carcinomas and 11 normal liver tissue patches. The cell segmentation performance was extensively evaluated on five external datasets, including liver, lung, and oral disease cases. CSGO demonstrated superior performance over the state-of-the-art method Cellpose, achieving higher F1 scores ranging from 0.37 to 0.53 at an intersection over union (IoU) threshold of 0.5 in four of the five external datasets, compared to that of Cellpose from 0.21 to 0.36. These results underscore the robustness and accuracy of our approach in various tissue types. A web-based application is available at https://ai.swmed.edu/projects/csgo, providing a user-friendly platform for researchers to apply our method to their own datasets. Our method exhibits remarkable versatility in whole-cell segmentation across diverse cancer subtypes, serving as an accurate and reliable tool to facilitate TME studies. The advancements presented in this study have the potential to significantly enhance the precision and efficiency of pathological image analysis, contributing to better understanding and treatment of cancer.

Bridging Human and Machine Intelligence: Reverse-Engineering Radiologist Intentions for Clinical Trust and Adoption

In the rapidly evolving landscape of medical imaging, the integration of artificial intelligence (AI) with clinical expertise offers unprecedented opportunities to enhance diagnostic precision and accuracy. Yet, the "black box" nature of AI models often limits their integration into clinical practice, where transparency and interpretability are important. This paper presents a novel system leveraging the Large Multimodal Model (LMM) to bridge the gap between AI predictions and the cognitive processes of radiologists. This system consists of two core modules, Temporally Grounded Intention Detection (TGID) and Region Extraction (RE). The TGID module predicts the radiologist's intentions by analyzing eye gaze fixation heatmap videos and corresponding radiology reports. Additionally, the RE module extracts regions of interest that align with these intentions, mirroring the radiologist’s diagnostic focus. This approach introduces a new task, radiologist intention detection, and is the first application of Dense Video Captioning (DVC) in the medical domain. By making AI systems more interpretable and aligned with radiologist’s cognitive processes, this proposed system aims to enhance trust, improve diagnostic accuracy, and support medical education. Additionally, it holds the potential for automated error correction, guiding junior radiologists, and fostering more effective training and feedback mechanisms. This work sets a precedent for future research in AI-driven healthcare, offering a pathway towards transparent, trustworthy, and human-centered AI systems. We evaluated this model using NLG(Natural Language Generation), time-related, and vision-based metrics, demonstrating superior performance in generating temporally grounded intentions on REFLACX and EGD-CXR datasets. This model also demonstrated strong predictive accuracy in overlap scores for medical abnormalities and effective region extraction with high IoU(Intersection over Union), especially in complex cases like cardiomegaly and edema. These results highlight the system's potential to enhance diagnostic accuracy and support continuous learning in radiology. We are also releasing the source code for our project, available here.

A Heatmap-Supplemented R-CNN Trained Using an Inflated IoU for Small Object Detection

Object detection architectures struggle to detect small objects across applications including remote sensing and autonomous vehicles. Specifically, for unmanned aerial vehicles, poor detection of small objects directly limits this technology’s applicability. Objects both appear smaller than they are in large-scale images captured in aerial imagery and are represented by reduced information in high-altitude imagery. This paper presents a new architecture, CR-CNN, which predicts independent regions of interest from two unique prediction branches within the first stage of the network: a conventional R-CNN convolutional backbone and an hourglass backbone. Utilizing two independent sources within the first stage, our approach leads to an increase in successful predictions of regions that contain smaller objects. Anchor-based methods such as R-CNNs also utilize less than half the number of small objects compared to larger ones during training due to the poor intersection over union (IoU) scores between the generated anchors and the groundtruth—further reducing their performance on small objects. Therefore, we also propose artificially inflating the IoU of smaller objects during training using a simple, size-based Gaussian multiplier—leading to an increase in the quantity of small objects seen per training cycle based on an increase in the number of anchor–object pairs during training. This architecture and training strategy led to improved detection overall on two challenging aerial-based datasets heavily composed of small objects while predicting fewer false positives compared to Mask R-CNN. These results suggest that while new and unique architectures will continue to play a part in advancing the field of object detection, the training methodologies and strategies used will also play a valuable role.

Barrier-free tomato fruit selection and location based on optimized semantic segmentation and obstacle perception algorithm.

With the advancement of computer vision technology, vision-based target perception has emerged as a predominant approach for harvesting robots to identify and locate fruits. However, little attention has been paid to the fact that fruits may be obscured by stems or other objects. In order to improve the vision detection ability of fruit harvesting robot, a fruit target selection and location approach considering obstacle perception was proposed. To enrich the dataset for tomato harvesting, synthetic data were generated by rendering a 3D simulated model of the tomato greenhouse environment, and automatically producing corresponding pixel-level semantic segmentation labels. An attention-based spatial-relationship feature extraction module (SFM) with lower computational complexity was designed to enhance the ability of semantic segmentation network DeepLab v3+ in accurately segmenting linear-structured obstructions such as stems and wires. An adaptive K-means clustering method was developed to distinguish individual instances of fruits. Furthermore, a barrier-free fruit selection algorithm that integrates information of obstacles and fruit instances was proposed to identify the closest and largest non-occluded fruit as the optimal picking target. The improved semantic segmentation network exhibited enhanced performance, achieving an accuracy of 96.75%. Notably, the Intersection-over-Union (IoU) of wire and stem classes was improved by 5.0% and 2.3%, respectively. Our target selection method demonstrated accurate identification of obstacle types (96.15%) and effectively excluding fruits obstructed by strongly resistant objects (86.67%). Compared to the fruit detection method without visual obstacle avoidance (Yolo v5), our approach exhibited an 18.9% increase in selection precision and a 1.3% reduction in location error. The improved semantic segmentation algorithm significantly increased the segmentation accuracy of linear-structured obstacles, and the obstacle perception algorithm effectively avoided occluded fruits. The proposed method demonstrated an appreciable ability in precisely selecting and locating barrier-free fruits within non-structural environments, especially avoiding fruits obscured by stems or wires. This approach provides a more reliable and practical solution for fruit selection and localization for harvesting robots, while also being applicable to other fruits and vegetables such as sweet peppers and kiwis.

Real-time crop row detection using computer vision- application in agricultural robots.

The goal of achieving autonomous navigation for agricultural robots poses significant challenges, mostly arising from the substantial natural variations in crop row images as a result of weather conditions and the growth stages of crops. The processing of the detection algorithm also must be significantly low for real-time applications. In order to address the aforementioned requirements, we propose a crop row detection algorithm that has the following features: Firstly, a projective transformation is applied to transform the camera view and a color-based segmentation is employed to distinguish crop and weed from the background. Secondly, a clustering algorithm is used to differentiate between the crop and weed pixels. Lastly, a robust line-fitting approach is implemented to detect crop rows. The proposed algorithm is evaluated throughout a diverse range of scenarios, and its efficacy is assessed in comparison to four distinct existing solutions. The algorithm achieves an overall intersection over union (IOU) of 0.73 and exhibits robustness in challenging scenarios with high weed growth. The experiments conducted on real-time video featuring challenging scenarios show that our proposed algorithm exhibits a detection accuracy of over 90% and is a viable option for real-time implementation. With the high accuracy and low inference time, the proposed methodology offers a viable solution for autonomous navigation of agricultural robots in a crop field without damaging the crop and thus can serve as a foundation for future research.