Neural Networks For Object Detection Research Articles

Semantic Segmentation of the Prostate Based on Onefold and Joint Multimodal Medical Images Using YOLOv4 and U-Net

Magnetic Resonance Imaging is increasing in importance in prostate cancer diagnosis due to the high accuracy and quality of the examination procedure. However, this process requires a time-consuming analysis of the results. Currently, machine vision is widely used in many areas. It enables automation and support in radiological studies. Successful detection of primary prostate tumors depends on the effective segmentation of the prostate itself. At times, a CT scan may be performed; alternatively, MRI may be the selected option. The data always reach a bottleneck stage. This paper presents the effective training of deep learning models to segment the prostate based on onefold and multimodal medical images. This approach supports the computer-aided diagnosis (CAD) system for radiologists as the first step in cancer exams. A comparison of two approaches designed for prostate segmentation is described. The first combines YOLOv4, the object detection neural network, and U-Net for a semantic segmentation based on onefold modality MRI images. The second presents the same method trained on multimodal images—a CT and MRI mixed dataset. The learning process was carried out in a cloud environment using GPU cards. The experiments are based on data from 120 patients who have undergone MRI and CT examinations. Several metrics evaluated the trained models. In the prostate semantic segmentation process, better results were achieved by mixed MRI with CT datasets. The best model achieved the value of 0.9685 for the Sørensen–Dice coefficient for the threshold value of 0.6.

Spiking Neural Networks for Object Detection Based on Integrating Neuronal Variants and Self-Attention Mechanisms

Thanks to their event-driven asynchronous computing capabilities and low power consumption advantages, spiking neural networks (SNNs) show significant potential for computer vision tasks, especially in object detection. However, effective training methods and optimization mechanisms for SNNs remain underexplored. This study proposes two high accuracy SNNs for object detection, AMS_YOLO and AMSpiking_VGG, integrating neuronal variants and attention mechanisms. To enhance these proposed networks, we explore the impact of incorporating different neuronal variants.The results show that the optimization in the SNN’s structure with neuronal variants outperforms that in the attention mechanism for object detection. Compared to the state-of-the-art in the current SNNs, AMS_YOLO improved by 6.7% in accuracy on the static dataset COCO2017, and AMS_Spiking has improved by 11.4% on the dynamic dataset GEN1.

A deep learning based assisted analysis approach for Sjogren's syndrome pathology images.

Diagnosing Sjogren's syndrome requires considerable time and effort from physicians, primarily because it necessitates rigorously establishing the presence lymphatic infiltration in the pathological tissue of the labial gland. The aim of this study is to use deep learning techniques to overcome these limitations and improve diagnostic accuracy and efficiency in pathology. We develop an auxiliary diagnostic system for Sjogren's syndrome. The system incorporates the state-of-the-art object detection neural network, YOLOv8, and enables the precise identification and flagging of suspicious lesions. We design the multi-dimensional attention module and S-MPDIoU loss function to improve the detection performance of YOLOv8. By extracting features from multiple dimensions of the feature map, the utilization of the multi-dimensional attention mechanism enhances the feature interaction across disparate positions, enabling the network to proficiently learn and retain salient cell features. S-MPDIoU introduces an angle penalty term that efficiently minimizes the diagonal distance between predicted and ground truth boxes. Additionally, it incorporates a flexible scale factor tailored to different size feature maps, which balances the issue of sudden gradient decrease during high overlap, thereby accelerating the overall convergence rate. To verify the effectiveness of our methods, we create a dataset of lymphocytes using labial gland biopsy pathology images collected from YanTaiShan hospital and trained the model with this dataset. The proposed model is assessed using standard metrics like precision, recall, mAP. The improved model achieves an increase in recall by 9.1%, mAP.5 by 3.2%, and mAP.95 by 2%. The study demonstrated deep learning's potential to analysis pathology images, offering a reference framework for the application of deep learning technology in the medical domain.

LRSDet: lightweight remote sensing target detection network with local-global information awareness and rapid sample assignment

Abstract With the rapid development of aerospace and unmanned aerial vehicles, using neural networks for object detection in optical remote sensing images (O-RSI) has encountered heightened challenges. The optical remote sensing images have the characteristics of complex geometric scenes, dense groups of objects, and significant multi-scale variations of objects; researchers need to use more complex models to achieve higher accuracy. However, this complexity also brings challenges to the application of lightweight scenes. Therefore, to cope with the trade-off challenge between model complexity and detection accuracy, we propose a lightweight network model LRSDet in this study. The model integrates local and global information processing mechanisms and introduces a fast positive sample assignment strategy to adapt to resource-constrained embedded and mobile platforms. By constructing a lightweight feature extraction network and a lightweight path aggregation network and incorporating the ESM-Attention module, the feature extraction capability of the model in complex remote sensing scenarios is significantly improved. In addition, the application of a dynamic soft threshold strategy further optimizes the positive sample selection process and improves the detection efficiency of the model. Experimental on the O-RSI datasets DIOR, NWPU VHR-10, and RSOD, while analyzing model real-time performance on aerial video and embedded devices, outperforming other state-of-the-art methods.

A novel edge intelligence-based solution for safer footpath navigation of visually impaired using computer vision

Navigating through a tactile paved footpath surrounded by various sizes of static and dynamic obstacles is one of the biggest impediments visually impaired people face, especially in Dhaka, Bangladesh. This problem is important to address, considering the number of accidents in such densely populated footpaths. We propose a novel deep-edge solution using Computer Vision to make people aware of the obstacles in the vicinity and reduce the necessity of a walking cane. This study introduces a diverse novel tactile footpath dataset of Dhaka covering different city areas. Additionally, existing state-of-the-art deep neural networks for object detection have been fine-tuned and investigated using this dataset. A heuristic-based breadth-first navigation algorithm (HBFN) is developed to provide navigation directions that are safe and obstacle-free, which is then deployed in a smartphone application that automatically captures images of the footpath ahead to provide real-time navigation guidance delivered by speech. The findings from this study demonstrate the effectiveness of the object detection model, YOLOv8s, which outperformed other benchmark models on this dataset, achieving a high mAP of 0.974 and an F1 score of 0.934. The model’s performance is analyzed after quantization, reducing its size by 49.53% while retaining 98.97% of the original mAP.

A Dynamic Visual SLAM System Incorporating Object Tracking for UAVs

The capability of unmanned aerial vehicles (UAVs) to capture and utilize dynamic object information assumes critical significance for decision making and scene understanding. This paper presents a method for UAV relative positioning and target tracking based on a visual simultaneousocalization and mapping (SLAM) framework. By integrating an object detection neural network into the SLAM framework, this method can detect moving objects and effectively reconstruct the 3D map of the environment from image sequences. For multiple object tracking tasks, we combine the region matching of semantic detection boxes and the point matching of the optical flow method to perform dynamic object association. This joint association strategy can prevent trackingoss due to the small proportion of the object in the whole image sequence. To address the problem ofacking scale information in the visual SLAM system, we recover the altitude data based on a RANSAC-based plane estimation approach. The proposed method is tested on both the self-created UAV dataset and the KITTI dataset to evaluate its performance. The results demonstrate the robustness and effectiveness of the solution in facilitating UAV flights.

An Enhanced Single-Stage Neural Network for Object Detection in Transmission Line Inspection

An Enhanced Single-Stage Neural Network for Object Detection in Transmission Line Inspection

Real Time Object Detection using TensorFlow

In recent years, deep learning has been used in image classification, object tracking, action recognition and scene labeling. Traditionally, Image Processing techniques were used to solve any Computer Vision problems occurred in an artificial intelligence system. However, in real-time identification, image processing cannot be used. This is where Deep Learning concepts are applied. We built a simple Convolutional Neural Network for object detection. The model is trained and multiple test cases are implemented in the TensorFlow environment so as to obtain accurate results

Toward robust and high-throughput detection of seed defects in X-ray images via deep learning

BackgroundThe detection of internal defects in seeds via non-destructive imaging techniques is a topic of high interest to optimize the quality of seed lots. In this context, X-ray imaging is especially suited. Recent studies have shown the feasibility of defect detection via deep learning models in 3D tomography images. We demonstrate the possibility of performing such deep learning-based analysis on 2D X-ray radiography for a faster yet robust method via the X-Robustifier pipeline proposed in this article.Results2D X-ray images of both defective and defect-free seeds were acquired. A deep learning model based on state-of-the-art object detection neural networks is proposed. Specific data augmentation techniques are introduced to compensate for the low ratio of defects and increase the robustness to variation of the physical parameters of the X-ray imaging systems. The seed defects were accurately detected (F1-score >90%), surpassing human performance in computation time and error rates. The robustness of these models against the principal distortions commonly found in actual agro-industrial conditions is demonstrated, in particular, the robustness to physical noise, dimensionality reduction and the presence of seed coating.ConclusionThis work provides a full pipeline to automatically detect common defects in seeds via 2D X-ray imaging. The method is illustrated on sugar beet and faba bean and could be efficiently extended to other species via the proposed generic X-ray data processing approach (X-Robustifier). Beyond a simple proof of feasibility, this constitutes important results toward the effective use in the routine of deep learning-based automatic detection of seed defects.

MRMNet: Multi-scale residual multi-branch neural network for object detection

Neural networks in object detection methods are designed to handle the challenging task of multi-scale object detection in computer vision. However, small objects occupy small blocks of pixels, and feature information can easily be overwhelmed, which further leads to certain shortcomings and deficiencies in the network’s multi-scale representation abilities. To alleviate this problem, we propose a novel multi-scale residual multi-branch neural network (MRMNet) for object detection. Particularly, we first propose a multi-scale extension (MSE) module to structurally construct optimal solutions to enhance feature extraction and alleviate information loss, thus effectively improving multi-scale object detection performance. Furthermore, we propose a contextual feature refinement (CFR) module, which draws on the idea of residuals to introduce a multi-residual structure and the feature of partial convolution, which can effectively alleviate the problem of small objects being overwhelmed by conflicting semantic information while avoiding the degradation problem of deep feature networks in some sense. Finally, by combining our own MSE module and integrating it into the existing Modified CSP v6 backbone, we propose a new backbone network, the multi-scale residual (MSRes) network. Experimental results show that our MRMNet and MSRes backbone achieve a competitive detection performance of 43.1% AP on the MS-COCO 2017 dataset.

PODB: A learning-based polarimetric object detection benchmark for road scenes in adverse weather conditions

Due to its insensitivity to light intensity and the capability to capture multidimensional information, polarimetric imaging technology has been proven to have advantages over traditional intensity-based imaging techniques for object detection tasks in adverse environmental conditions, particularly in road traffic scenarios. Recently, with the rapid development of artificial intelligence technology, deep learning (DL)-powered object detection techniques can further enhance recognition accuracy and algorithm robustness. This improvement is made possible by the ability of DL technology to leverage large datasets and extract deeper levels of target-specific features. However, constructing large-scale polarimetric datasets poses challenges as obtaining polarimetric information requires multiple captures of intensity images. In other words, the workload is several times higher compared to traditional imaging techniques. To address the current scarcity of polarimetric datasets and evaluate the practical performance of various algorithms on polarimetric datasets, this paper proposes a Polarimetric Object Detection Benchmark (PODB) dataset. The PODB provides an integrated quality evaluation framework for DL-based object detection algorithms in complex road scenes by incorporating polarimetric imaging. Besides, we conducted extensive object detection experiments using the PODB, which allowed for a comprehensive validation and performance evaluation of effective benchmark algorithms. Furthermore, a physics-based multi-scale image fusion cascaded object detection neural network model is proposed. By combining the multidimensional information provided by polarized images with an adaptive learning multi-decision object detection neural network model, the recognition accuracy of complex road scenes in adverse weather conditions has been improved by approximately 10%. Additionally, we anticipate that PODB will serve as an effective platform for evaluating and comparing the performance of object detection algorithms, as well as providing researchers with a baseline for future studies in developing new DL-based methods.

Image-based profiling and deep learning reveal morphological heterogeneity of colorectal cancer organoids

Patient-derived organoids have proven to be a highly relevant model for evaluating of disease mechanisms and drug efficacies, as they closely recapitulate in vivo physiology. Colorectal cancer organoids, specifically, exhibit a diverse range of morphologies, which have been analyzed with image-based profiling. However, the relationship between morphological subtypes and functional parameters of the organoids remains underexplored. Here, we identified two distinct morphological subtypes (“cystic” and “solid”) across 31360 bright field images using image-based profiling, which correlated differently with viability and apoptosis level of colorectal cancer organoids. Leveraging object detection neural networks, we were able to categorize single organoids achieving higher viability scores as “cystic” than “solid” subtype. Furthermore, a deep generative model was proposed to predict apoptosis intensity based on a apoptosis-featured dataset encompassing over 17000 bright field and matched fluorescent images. Notably, a significant correlation of 0.91 between the predicted value and ground truth was achived, underscoring the feasibility of this generative model as a potential means for assessing organoid functional parameters. The underlying cellular heterogeneity of the organoids, i.e., conserved colonic cell types and rare immune components, was also verified with scRNA sequencing, implying a compromised tumor microenvironment. Additionally, the “cystic” subtype was identified as a relapse phenotype featuring intestinal stem cell signatures, suggesting that this visually discernible relapse phenotype shows potential as a novel biomarker for colorectal cancer diagnosis and prognosis. In summary, our findings demonstrate that the morphological heterogeneity of colorectal cancer organoids explicitly recapitulate the association of phenotypic features and exogenous perturbations through the image-based profiling, providing new insights into disease mechanisms.

Object-Oriented and Visual-Based Localization in Urban Environments.

In visual-based localization, prior research falls short in addressing challenges for the Internet of Things with limited computational resources. The dominant state-of-the-art models are based on separate feature extractors and descriptors without consideration of the constraints of small hardware, the issue of inconsistent image scale, or the presence of multi-objects. We introduce "OOPose", a real-time object-oriented pose estimation framework that leverages dense features from off-the-shelf object detection neural networks. It balances between pixel-matching accuracy and processing speed, enhancing overall performance. When input images share a comparable set of features, their matching accuracy is substantially heightened, while the reduction in image size facilitates faster processing but may compromise accuracy. OOPose resizes both the original library and cropped query object images to a width of 416 pixels. This adjustment results in a 2.4-fold improvement in pose accuracy and an 8.6-fold increase in processing speed. Moreover, OOPose eliminates the need for traditional sparse point extraction and description processes by capitalizing on dense network backbone features and selecting the detected query objects and sources of object library images, ensuring not only 1.3 times more accurate results but also three times greater stability compared to real-time sparse ORB matching algorithms. Beyond enhancements, we demonstrated the feasibility of OOPose in an autonomous mobile robot, enabling self-localization with a single camera at 10 FPS on a single CPU. It proves the cost-effectiveness and real-world applicability of OOPose for small embedded devices, setting the stage for potential markets and providing end-users with distinct advantages.

AraDQ: an automated digital phenotyping software for quantifying disease symptoms of flood-inoculated Arabidopsis seedlings

BackgroundPlant scientists have largely relied on pathogen growth assays and/or transcript analysis of stress-responsive genes for quantification of disease severity and susceptibility. These methods are destructive to plants, labor-intensive, and time-consuming, thereby limiting their application in real-time, large-scale studies. Image-based plant phenotyping is an alternative approach that enables automated measurement of various symptoms. However, most of the currently available plant image analysis tools require specific hardware platform and vendor specific software packages, and thus, are not suited for researchers who are not primarily focused on plant phenotyping. In this study, we aimed to develop a digital phenotyping tool to enhance the speed, accuracy, and reliability of disease quantification in Arabidopsis.ResultsHere, we present the Arabidopsis Disease Quantification (AraDQ) image analysis tool for examination of flood-inoculated Arabidopsis seedlings grown on plates containing plant growth media. It is a cross-platform application program with a user-friendly graphical interface that contains highly accurate deep neural networks for object detection and segmentation. The only prerequisite is that the input image should contain a fixed-sized 24-color balance card placed next to the objects of interest on a white background to ensure reliable and reproducible results, regardless of the image acquisition method. The image processing pipeline automatically calculates 10 different colors and morphological parameters for individual seedlings in the given image, and disease-associated phenotypic changes can be easily assessed by comparing plant images captured before and after infection. We conducted two case studies involving bacterial and plant mutants with reduced virulence and disease resistance capabilities, respectively, and thereby demonstrated that AraDQ can capture subtle changes in plant color and morphology with a high level of sensitivity.ConclusionsAraDQ offers a simple, fast, and accurate approach for image-based quantification of plant disease symptoms using various parameters. Its fully automated pipeline neither requires prior image processing nor costly hardware setups, allowing easy implementation of the software by researchers interested in digital phenotyping of diseased plants.

Learning Point Processes and Convolutional Neural Networks for Object Detection in Satellite Images

Convolutional neural networks (CNN) have shown great results for object-detection tasks by learning texture and pattern-extraction filters. However, object-level interactions are harder to grasp without increasing the complexity of the architectures. On the other hand, Point Process models propose to solve the detection of the configuration of objects as a whole, allowing the factoring in of the image data and the objects’ prior interactions. In this paper, we propose combining the information extracted by a CNN with priors on objects within a Markov Marked Point Process framework. We also propose a method to learn the parameters of this Energy-Based Model. We apply this model to the detection of small vehicles in optical satellite imagery, where the image information needs to be complemented with object interaction priors because of noise and small object sizes.

MS-YOLO: integration-based multi-subnets neural network for object detection in aerial images

MS-YOLO: integration-based multi-subnets neural network for object detection in aerial images

PMONN: an optical neural network for photonic integrated circuits based on micro-resonator.

We propose an improved optical neural network (ONN) circuit architecture based on conventional micro-resonator ONNs, called the Phase-based Micro-resonator Optical Neural Network (PMONN). PMONN's core architecture features a Convolutions and Batch Normalization (CB) unit, comprising a phase-based (PB) convolutional layer, a Depth-Point-Wise (DPW) convolutional layer, and a reconstructed Batch Normalization (RBN) layer. The PB convolution kernel uses modulable phase shifts of Add-drop MRRs as learnable parameters and their optical transfer function as convolution weights. The DPW convolution kernel amplifies PB convolution weights by learning the amplification factors. To address the internal covariate shift during training, the RBN layer normalizes DPW outputs by reconstructing the BN layer of the electronic neural network, which is then merged with the DPW layer in the test stage. We employ the tunable DAs in the architecture to implement the merged layer. PMONN achieves 99.15% and 91.83% accuracy on MNIST and Fashion-MNIST datasets, respectively. This work presents a method for implementing an optical neural network on the improved architecture based on MRRs and increases the flexibility and reusability of the architecture. PMONN has potential applications as the backbone for future optical object detection neural networks.

Performance evaluation of three versions of a convolutional neural network for object detection and segmentation using a multiclass and reduced panoramic radiograph dataset

ObjectivesTo evaluate the diagnostic performance of three versions of a deep-learning convolutional neural network in terms of object detection and segmentation using a multiclass panoramic radiograph dataset. MethodsA total of 600 orthopantomographies were randomly selected for this study and manually annotated by a single operator using an image annotation tool (COCO Annotator v.11.0.1) to establish ground truth. The annotation classes included teeth, maxilla, mandible, inferior alveolar nerve, dento- and implant-supported crowns/pontics, endodontic treatment, resin-based restorations, metallic restorations, and implants. The dataset was then divided into training, validation, and testing subsets, which were used to train versions 5, 7, and 8 of You Only Look Once (YOLO) Neural Network. Results were stored, and a posterior performance analysis was carried out by calculating the precision (P), recall (R), F1 Score, Intersection over Union (IoU), and mean average precision (mAP) at 0.5 and 0.5–0.95 thresholds. The confusion matrix and recall precision graphs were also sketched. ResultsYOLOv5s showed an improvement in object detection results with an average R = 0.634, P = 0.781, mAP0.5 = 0.631, and mAP0.5–0.95 = 0.392. YOLOv7m achieved the best object detection results with average R = 0.793, P = 0.779, mAP0.5 = 0.740, and mAP0.5–0.95 = 0,481. For object segmentation, YOLOv8m obtained the best average results (R = 0.589, P = 0.755, mAP0.5 = 0.591, and mAP0.5–0.95 = 0.272). ConclusionsYOLOv7m was better suited for object detection, while YOLOv8m demonstrated superior performance in object segmentation. The most frequent error in object detection was related to background classification. Conversely, in object segmentation, there is a tendency to misclassify True Positives across different dental treatment categories. Clinical SignificanceGeneral diagnostic and treatment decisions based on panoramic radiographs can be enhanced using new artificial intelligence-based tools. Nevertheless, the reliability of these neural networks should be subjected to training and validation to ensure their generalizability.

Hair cluster detection model based on dermoscopic images

Introduction: Hair loss has always bothered many people, with numerous individuals potentially facing the issue of sparse hair.Methods: Due to a scarcity of accurate research on detecting sparse hair, this paper proposes a sparse hair cluster detection model based on improved object detection neural network and medical images of sparse hair under dermatoscope to optimize the evaluation of treatment outcomes for hair loss patients. A new Multi-Level Feature Fusion Module is designed to extract and fuse features at different levels. Additionally, a new Channel-Space Dual Attention Module is proposed to consider both channel and spatial dimensions simultaneously, thereby further enhancing the model’s representational capacity and the precision of sparse hair cluster detection.Results: After testing on self-annotated data, the proposed method is proven capable of accurately identifying and counting sparse hair clusters, surpassing existing methods in terms of accuracy and efficiency.Discussion: Therefore, it can work as an effective tool for early detection and treatment of sparse hair, and offer greater convenience for medical professionals in diagnosis and treatment.

Train Distance Estimation for Virtual Coupling Based on Monocular Vision.

By precisely controlling the distance between two train sets, virtual coupling (VC) enables flexible coupling and decoupling in urban rail transit. However, relying on train-to-train communication for obtaining the train distance can pose a safety risk in case of communication malfunctions. In this paper, a distance-estimation framework based on monocular vision is proposed. First, key structure features of the target train are extracted by an object-detection neural network, whose strategies include an additional detection head in the feature pyramid, labeling of object neighbor areas, and semantic filtering, which are utilized to improve the detection performance for small objects. Then, an optimization process based on multiple key structure features is implemented to estimate the distance between the two train sets in VC. For the validation and evaluation of the proposed framework, experiments were implemented on Beijing Subway Line 11. The results show that for train sets with distances between 20 m and 100 m, the proposed framework can achieve a distance estimation with an absolute error that is lower than 1 m and a relative error that is lower than 1.5%, which can be a reliable backup for communication-based VC operations.