Object Extraction Research Articles

Rescuing the knee: arthroscopic extraction of a long-neglected glass fragment: a case report

The knee joint frequently encounters penetrating injuries and has ample space to harbor foreign objects. While foreign bodies in the knee joint are unusual, especially those not originating from surgical procedures, extensive literature exists on the extraction of loose bodies and metallic objects from this articulation. However, documentation on uncommon intruders remains scarce. This case study delves into the experience of a 22-year-old male who suffered persistent pain due to a neglected foreign object in his knee joint. After its removal, a complete resolution of symptoms was achieved. It highlights the importance of prompt arthroscopic intervention for optimal outcomes. Here we present a unique case involving a 22-year-old male who underwent arthroscopic retrieval of a neglected glass fragment from his knee joint. The patient sought medical attention five months after the initial injury due to sudden and severe pain onset. Arthroscopy stands as the favored minimally invasive method for extracting foreign bodies from the knee joint, especially from its deeper and less accessible regions. Utilizing pre-operative radiological imaging enhances understanding of the anatomical placement, facilitates surgical orientation, and reduces operation time. A keen sense of suspicion and thorough history-taking are crucial in achieving an accurate diagnosis.

Application of Visual Transformer in Low-resolution Thermal Infrared Image Recognition

Abstract Addressing the challenges of inadequate accuracy and limited robustness exhibited by current lightweight object detection networks specifically tailored for low-resolution thermal infrared face detection scenarios, this paper delves into developing an ultra-lightweight thermal infrared face detection algorithm that leverages visual attention mechanisms. To ascertain the optimal neural network complexity, a series of comparative experiments are meticulously conducted. With Yolo-FastestDet serving as the benchmark, this study endeavors to compress the backbone network, striking a delicate balance between network depth and detection speed. Additionally, to bolster the network’s capacity for profound feature extraction and precise discrimination of target edges and small objects, a Transformer-Encoder-based visual attention module is seamlessly integrated. Consequently, a lightweight face detection algorithm, enriched with attention mechanisms, is formulated. Furthermore, to mitigate the scarcity of low-resolution infrared face image data, a self-constructed visible-infrared face dataset is employed for training and evaluation purposes. The experimental outcomes reveal that the proposed algorithm attains an impressive mAP@0.5 score of 0.953 on the test dataset while satisfying the stringent real-time detection criterion of 30 frames per second (FPS) when deployed on an embedded Raspberry Pi CPU.

Intelligent extraction of salient objects from fuzzy distorted images based on multi-scale convolution

ABSTRACT To solve the problem of how to obtain important information from fuzzy distorted images, an intelligent method for extracting salient objects from fuzzy distorted images based on multi-scale convolution is proposed. A multi-scale deformation feature convolution network is established to extract the salient features of fuzzy distorted images. Among them, the multi-scale convolution network VGG-16 extracts the shallow features of the fuzzy distortion image through the convolution layer operation, uses the multi-scale convolution kernel to extract the fuzzy distortion image in parallel, and introduces the self-attention mechanism to make the extracted features more relevant, Input the fused features into the proposed area extraction network, Through the target area detection network, the proposed target area of the fuzzy distortion image is classified and position regression is performed to obtain the final salient target of the fuzzy distortion image. The experimental results show that this method can accurately recognize the contour of salient objects, and extract salient objects from fuzzy distorted images; The extracted salient objects have high accuracy, low mean square error and similarity of 92.9% with the original image; For the salient target extraction in various scenarios, it shows high advantages.

Taking relations as known conditions: A tagging based method for relational triple extraction

Relational triple extraction refers to extracting entities and relations from natural texts, which is a crucial task in the construction of knowledge graph. Recently, tagging based methods have received increasing attention because of their simple and effective structural form. Among them, the two-step extraction method is easy to cause the problem of category imbalance. To address this issue, we propose a novel two-step extraction method, which first extracts subjects, generates a fixed-size embedding for each relation, and then regards these relations as known conditions to extract the objects directly with the identified subjects. In order to eliminate the influence of irrelevant relations when predicting objects, we use a relation-special attention mechanism and a gate unit to select appropriate relations. In addition, most current models do not account for two-way interaction between tasks, so we design a feature interactive network to achieve bidirectional interaction between subject and object extraction tasks and enhance their connection. Experimental results on NYT, WebNLG, NYT⋆ and WebNLG⋆ datasets show that our model is competitive among joint extraction models.

Small object change detection in UAV imagery via a Siamese network enhanced with temporal mutual attention and contextual features: A case study concerning solar water heaters

Small object change detection (SOCD) based on high-spatial resolution (HSR) images is of significant practical value in applications such as the investigation of illegal urban construction, but little research is currently available. This study proposes an SOCD model called TMACNet based on a multitask network architecture. The model modifies the YOLOv8 network into a Siamese network and adds structures, including a feature difference branch (FDB), temporal mutual attention layer (TMAL) and contextual attention module (CAM), to merge differential and contextual features from different phases for the accurate extraction and analysis of small objects and their changes. To verify the proposed method, an SOCD dataset called YZDS is created based on unmanned aerial vehicle (UAV) images of small-scale solar water heaters on rooftops. The experimental results show that TMACNet exhibits strong resistance to image registration errors and building height displacement and prevents error propagation from object detection to change detection originating from overlay-based change detection. TMACNet also provides an enhanced approach to small object detection from the perspective of multitemporal information fusion. In the change detection task, TMACNet exhibits notable F1 improvements exceeding 5.96% in comparison with alternative change detection methods. In the object detection task, TMACNet outperforms the single-temporal object detection models, increasing accuracy with an approximately 1–3% improvement in the AP metric while simplifying the technical process.

Multi-granularity spatiotemporal object modelling of waterborne traffic elements

The electronic navigational charts are crucial carriers for representing the multi-source heterogeneous data of Waterborne Traffic Elements (WTEs). However, their layer-based modelling method has some shortcomings in expressing the multi-granularity features, complex relationships, and dynamic evolution of elements. This paper proposes an objectification modelling method for WTEs based on the concept of multi-granularity spatiotemporal object modelling. A classification system for waterborne traffic objects is developed based on the relevance of behavior to elements; combining characteristics of waterborne traffic, a data model for waterborne traffic objects is constructed from eight aspects: spatiotemporal reference, spatiotemporal position, spatial form, basic information, attributes, behavioral ability, structure, and associative relationships. An object extraction function is also established, extracting object attributes and relationships between objects according to different element classes. Taking the Jiashan section of the Hangzhou-Shanghai Line in Zhejiang Province as the experimental subject, the multi-granularity spatiotemporal characteristics, dynamic evolution, and relationship expression of channel class objects are tested. The experimental results show that the proposed method provides the theoretical basis and data organization mode for the multi-granularity expression of WTEs.

Social and Environmental Impact of a Plant Disease Analysis Method Based on Object Extraction

Social and Environmental Impact of a Plant Disease Analysis Method Based on Object Extraction

Ship Lock Extraction from High-Resolution Remote Sensing Images Based on Fuzzy Theory and Prior Knowledge

As crucial water conservancy projects, ship locks play a key role in flood control, shipping, water resource allocation, and promoting regional economic development, making them an indispensable part of the modern water transportation system. Utilizing satellite remote sensing for lock extraction can significantly reduce manual workload and costs, assist in the daily dynamic maintenance of lock hubs, and provide more comprehensive data support for the construction and management of water transport infrastructure. In this context, this paper proposes a new method for ship lock object extraction. Leveraging fuzzy theory and prior knowledge of locks, the extraction of lock objects is achieved from Gaofen-1 (GF-1) high-resolution remote sensing images. The experimental results demonstrate that the proposed algorithm can effectively extract small lock objects in remote sensing images, achieving an average extraction accuracy of 80.9% in the study area.

Unsupervised Object Cosegmentation Method Devoted to Image Classification

Rich heterogeneous data provided by social networks can be very big, which imposes considerable challenges for object extraction and image classification. Therefore, the objective of this work is to propose an unsupervised object cosegmentation method that could be notably efficient to improve image classification performance. The main goal of cosegmentation is to extract the salient common objects within each image. To this end, we propose to minimize an energy function based on the Markov Random Field using the saliency detection, while considering linear dependence of generated foreground histograms of the input image collection. In fact, the saliency detection is processed in two steps. In each image, we detect salient objects, by considering appearance similarity and spatial distributions of image pixels. Then, fuzzy quantification is used to correct the belonging of pixels to the foreground objects. Finally, an iterative optimization permits to enhance the final segmentation results. The proposed method has been validated as a preprocessing step for image classification. Indeed, to enhance cosegmentation-based classification performance, we have applied a semi-supervised object classification based on ensemble projection. Qualitative and quantitative evaluations of the proposed cosegmentation and classification techniques on the iCoseg, CDS and Oxford Flowers 17 datasets demonstrate the effectiveness of the proposed framework.

EDASNet: efficient dynamic adaptive-scale network for infrared pedestrian detection

Abstract Infrared images are widely utilized due to their exceptional anti-interference capabilities. However, challenges such as low resolution and an absence of detailed texture can impede the effective recognition of multi-scale target information, particularly for small targets. To address these issues, we introduce a multi-scale detection framework named efficient dynamic adaptive-scale network (EDASNet), which focuses on enhancing the feature extraction of small objects while ensuring efficient detection of multi-scale. Firstly, we design a lightweight dynamic enhance network as the backbone for feature extraction. It mainly includes a lightweight adaptive-weight downsampling module and a dynamic enhancement convolution module. In addition, a multi-scale aggregation feature pyramid network is proposed, which improves the perception effect of small objects through a multi-scale convolution module. Then, the Repulsion Loss term was introduced based on CIOU to effectively solve the missed detection problem caused by target overlap. Finally, the dynamic head was used as the network detection head, and through the superposition of dynamic convolution and multiple attention, the network was able to accurately realize multi-scale object detection. Comprehensive experiments show that EDASNet outperforms existing efficient models and achieves a good trade-off between speed and accuracy.

Impact of Tile Size and Tile Overlap on the Prediction Performance of Convolutional Neural Networks Trained for Road Classification

Popular geo-computer vision works make use of aerial imagery, with sizes ranging from 64 × 64 to 1024 × 1024 pixels without any overlap, although the learning process of deep learning models can be affected by the reduced semantic context or the lack of information near the image boundaries. In this work, the impact of three tile sizes (256 × 256, 512 × 512, and 1024 × 1024 pixels) and two overlap levels (no overlap and 12.5% overlap) on the performance of road classification models was statistically evaluated. For this, two convolutional neural networks used in various tasks of geospatial object extraction were trained (using the same hyperparameters) on a large dataset (containing aerial image data covering 8650 km2 of the Spanish territory that was labelled with binary road information) under twelve different scenarios, with each scenario featuring a different combination of tile size and overlap. To assess their generalisation capacity, the performance of all resulting models was evaluated on data from novel areas covering approximately 825 km2. The performance metrics obtained were analysed using appropriate descriptive and inferential statistical techniques to evaluate the impact of distinct levels of the fixed factors (tile size, tile overlap, and neural network architecture) on them. Statistical tests were applied to study the main and interaction effects of the fixed factors on the performance. A significance level of 0.05 was applied to all the null hypothesis tests. The results were highly significant for the main effects (p-values lower than 0.001), while the two-way and three-way interaction effects among them had different levels of significance. The results indicate that the training of road classification models on images with a higher tile size (more semantic context) and a higher amount of tile overlap (additional border context and continuity) significantly impacts their performance. The best model was trained on a dataset featuring tiles with a size of 1024 × 1024 pixels and a 12.5% overlap, and achieved a loss value of 0.0984, an F1 score of 0.8728, and an ROC-AUC score of 0.9766, together with an error rate of 3.5% on the test set.

Visual primitives as words: Alignment and interaction for compositional zero-shot learning

Compositional Zero-Shot Learning (CZSL) aims to recognize seen and unseen attribute-object compositions. Recently, some researchers apply vision-language models to CZSL task. However, they only roughly match the image embedding and composition embedding on the image level, which can be a barrier to further improvement. With observation and analysis, we believe that a visual primitive is worth a word. To make full use of visual primitives to achieve fine-grained alignment and bridging modal gap, we propose VisPrompt for interacting visual primitives with sub-concepts in a prompt. Specifically, VisPrompt aligns the visual primitives (i.e., visual attribute and visual object) with the sub-concepts (i.e., text attribute and text object) at a fine-grained level. It consists of two steps: (1) First, we extract the visual attribute embedding by an attribute extraction module, and the visual object embedding by an object extraction module; (2) Second, we design an attribute-wise prompt, an object-wise prompt, and a visual reconstructed prompt to be encoded, where a visual primitive plays the role of corresponding sub-concept to interact. Therefore, our model is capable of applying fine-grained alignment and bridging the gap between vision and text. Sufficient experiments on widely-used MIT-States, UT-Zappos, CGQA, and VAW-CZSL datasets show that our VisPromt achieves state-of-the-art on the core metric AUC.

Method of Identification of Combat Vehicles Based on Yolo

A method for recognizing contours of objects in a video data stream is proposed. Data will be uploaded using a video camera in real time and object recognition will be performed. We will use the YOLO network – a method of identifying and recognizing objects in real time. Recognized objects will be recorded in a video sequence showing the contours of the objects. The approach proposed in the project reasonably synthesizes methods of artificial intelligence, theories of computer vision on the one hand, and pattern recognition on the other; it makes it possible to obtain control influences and mathematical functions for decision-making at every moment of time with the possibility of analyzing the influence of external factors and forecasting the flow of processes, and refers to the fundamental problems of mathematical modeling of real processes. The installation of the neural network is shown in detail. The characteristics of the neural network are shown and its capabilities are substantiated. Approaches to computer vision for object extraction are shown. Well-known methods are methods of expanding areas, methods based on clustering, contour selection, and methods using a histogram. The work envisages building a system for rapid identification of combat vehicles based on the latest image filtering methods developed using deep learning methods. The time spent on identifying the machine will be 10 –20 % shorter, thanks to the developed new information technology for detecting objects in conditions of rapidly changing information.

Semantic-embedded similarity prototype for scene recognition

Due to the high inter-class similarity caused by the complex composition and the co-existing objects across scenes, numerous studies have explored object semantic knowledge within scenes to improve scene recognition. However, a resulting challenge emerges as object information extraction techniques require heavy computational costs, thereby burdening the network considerably. This limitation often renders object-assisted approaches incompatible with edge devices in practical deployment. In contrast, this paper proposes a semantic knowledge-based similarity prototype, which can help the scene recognition network achieve superior accuracy without increasing the computational cost in practice. It is simple and can be plug-and-played into existing pipelines. More specifically, a statistical strategy is introduced to depict semantic knowledge in scenes as class-level semantic representations. These representations are used to explore correlations between scene classes, ultimately constructing a similarity prototype. Furthermore, we propose to leverage the similarity prototype to support network training from the perspective of Gradient Label Softening and Batch-level Contrastive Loss, respectively. Comprehensive evaluations on multiple benchmarks show that our similarity prototype enhances the performance of existing networks, all while avoiding any additional computational burden in practical deployments. Code and the statistical similarity prototype will be available at https://github.com/ChuanxinSong/SimilarityPrototype.

X-Detect: explainable adversarial patch detection for object detectors in retail

Object detection models, which are widely used in various domains (such as retail), have been shown to be vulnerable to adversarial attacks. Existing methods for detecting adversarial attacks on object detectors have had difficulty detecting new real-life attacks. We present X-Detect, a novel adversarial patch detector that can: (1) detect adversarial samples in real time, allowing the defender to take preventive action; (2) provide explanations for the alerts raised to support the defender’s decision-making process, and (3) handle unfamiliar threats in the form of new attacks. Given a new scene, X-Detect uses an ensemble of explainable-by-design detectors that utilize object extraction, scene manipulation, and feature transformation techniques to determine whether an alert needs to be raised. X-Detect was evaluated in both the physical and digital space using five different attack scenarios (including adaptive attacks) and the benchmark COCO dataset and our new Superstore dataset. The physical evaluation was performed using a smart shopping cart setup in real-world settings and included 17 adversarial patch attacks recorded in 1700 adversarial videos. The results showed that X-Detect outperforms the state-of-the-art methods in distinguishing between benign and adversarial scenes for all attack scenarios while maintaining a 0% FPR (no false alarms) and providing actionable explanations for the alerts raised. A demo is available.

Improving real-time apple fruit detection: Multi-modal data and depth fusion with non-targeted background removal

In automated fruit detection, RGB-Depth (RGB-D) images aid the detection model with additional depth information to enhance detection accuracy. However, outdoor depth images are usually of low quality, which limits the quality of depth data. In this study, an approach/technique for real-time apple fruit detection in a high-density orchard environment by using multi-modal data is presented. Non-targeted background removal using the depth fusion (NBR-DF) method was developed to reduce the high noise condition of depth images. The noise occurred due to the uncontrolled lighting condition and holes with incomplete depth information in the depth images. NBR-DF technique follows three primary steps: pre-processing of depth images (point cloud generation), target object extraction, and background removal. The NBR-DF method serves as a pipeline to pre-process multi-modal data to enhance features of depth images by filling holes to eliminate noise generated by depth holes. Further, the NBR-DF implemented with the YOLOv5 enhances the detection accuracy in dense orchard conditions by using multi-modal information as input. An attention-based depth fusion module that adaptively fuses the multi-modal features was developed. The integration of the depth-attention matrix involved pooling operations and sigmoid normalization, both of which are efficient methods for summarizing and normalizing depth information. The fusion module improves the identification of multiscale objects and strengthens the network's resistance to noise. The network then detects the fruit position using multiscale information from the RGB-D images in highly complex orchard environments. The detection results were compared and validated with other methods using different input modals and fusion strategies. The results showed that the detection accuracy using the NBR-DF approach achieved an average precision rate of 0.964 in real time. The performance comparison with other state-of-the-art methods and the model generalization study also establish that the present advanced depth-fusion attention mechanism and effective preprocessing steps in NBR-DF-YOLOv5 significantly surpass those in performance. In conclusion, the developed NBR-DF technique showed the potential to improve real-time apple fruit detection using multi-modal information.

A universal method for solar filament detection from Hαobservations using semi-supervised deep learning

Filaments are omnipresent features in the solar atmosphere. Their location, properties, and time evolution can provide important information about changes in solar activity and assist in the operational space weather forecast. Therefore, filaments have to be identified in full-disk images and their properties extracted from these images, but manual extraction is tedious and too time-consuming, and extraction with morphological image processing tools produces a large number of false positive detections. Automatic object detection, segmentation, and extraction in a reliable manner would allow for the processing of more data in a shorter time frame. The Chromospheric Telescope (ChroTel; Tenerife, Spain), the Global Oscillation Network Group (GONG), and the Kanzelhöhe Observatory for Solar and Environmental Research (KSO; Austria) provide regular full-disk observations of the Sun in the core of the chromospheric Hαabsorption line. In this paper, we present a deep learning method that provides reliable extractions of solar filaments from Hαfiltergrams. First, we trained the object detection algorithm YOLOv5 with labeled filament data of ChroTel Hαfiltergrams. We used the trained model to obtain bounding boxes from the full GONG archive. In a second step, we applied a semi-supervised training approach where we used the bounding boxes of filaments to train the algorithm on a pixel-wise classification of solar filaments with u-net. We made use of the increased data set size, which avoids overfitting of spurious artifacts from the generated training masks. Filaments were predicted with an accuracy of 92%. With the resulting filament segmentations, physical parameters such as the area or tilt angle could be easily determined and studied. We demonstrated this in an example where we determined the rush-to-the pole for Solar Cycle 24 from the segmented GONG images. In a last step, we applied the filament detection to Hαobservations from KSO and demonstrated the general applicability of our method to Hαfiltergrams.

Conversion of an Entire Image into a 3D Scene with Object Detection and Single 3D Models

Abstract: The paper presents a framework for the extraction of objects from 2D images using object detection techniques in open CV and then generating 3D models of the segmented objects. To achieve precise object detection, the suggested methodology blends deep learning and computer vision methods. The 3D models that are produced have fine-grained geometric and textural features, which improves the object reconstruction's accuracy. The usefulness and robustness of our technique in various settings are demonstrated by experimental findings on a variety of datasets. For use in robotics, virtual reality, and the preservation of cultural heritage, where precise 3D models of actual objects are essential, this research shows great promise.

Digitization of Astronomical Photographic Plates of China and Astrometric Measurement of Single-exposure Plates

From the mid-19th century to the end of the 20th century, photographic plates served as the primary detectors for astronomical observations. Astronomical photographic observations in China began in 1901, and over a century, a total of approximately 30,000 astronomical photographic plates were captured. These historical plates play an irreplaceable role in conducting long-term, time-domain astronomical research. To preserve and explore these valuable original astronomical observational data, Shanghai Astronomical Observatory has organized the transportation of plates, taken during nighttime observations from various stations across the country, to the Sheshan Plate Archive for centralized preservation. For the first time, plate information statistics were calculated. On this basis, the plates were cleaned and digitally scanned, and finally digitized images were acquired for 29,314 plates. In this study, using Gaia DR2 as the reference star catalog, astrometric processing was carried out successfully on 15,696 single-exposure plates, including object extraction, stellar identification, and plate model computation. As a result, for long focal length telescopes, such as the 40 cm double-tube refractor telescope, the 1.56 m reflector telescope at Shanghai Astronomical Observatory, and the 1m reflecting telescope at Yunnan Astronomical Observatory, the astrometric accuracy obtained for their plates is approximately 0.″1–0.″3. The distribution of astrometric accuracy for medium and short focal length telescopes ranges from 0.″3 to 1.″0. The relevant data of this batch of plates, including digitized images and a stellar catalog of the plates, are archived and released by the National Astronomical Data Center. Users can access and download plate data based on keywords such as station, telescope, observation year, and observed celestial coordinates.

Proposing Smart System for Detecting and Monitoring Vehicle Using Multiobject Multicamera Tracking

Nowadays, the strong development of the economy and society has driven the increase in traffic participation, making traffic management increasingly difficult. To effectively address this issue, AI applications are being applied to improve urban traffic management and operations. Therefore, we propose a smart system to detect and monitor vehicles across multiple surveillance cameras. Our system leverages data collected from traffic surveillance cameras and harnesses the power of deep learning technology to detect and track vehicles smoothly. To achieve this, we use the YOLO model for detection in conjunction with the DeepSORT algorithm for precise vehicle tracking on each camera. Furthermore, our system uses a ResNet backbone model for feature extraction of objects within each camera’s frame. It utilizes cosine distance to identify similar objects in other cameras, facilitating multicamera tracking. To ensure optimal performance, our system is implemented using the NVIDIA DeepStream SDK, enabling it to achieve an impressive speed of 21 fps on each camera and an average of precision approximately 85% for three modules. The results of our study affirm the system’s suitability and its potential for practical applications in the field of urban traffic management.