Object Pixels Research Articles

Small object intelligent Detection method based on Adaptive Cascading Context

With the technology advances, deep learning-based object detection has made unprecedented progress. However, the small spatial ratio of object pixels affects the effective extraction of deep details features, resulting in poor detection results in small object detection. To improve the accuracy of small object detection, an adaptive Cascading Context small (ACC) object detection method is proposed based on YOLOv5. Firstly, a separate shallow layer feature was proposed to obtain more detailed information beneficial to small object detection. Secondly, an adaptive cascade method is proposed to fuse the output features of the three layers of the pyramid to adaptively filter negative semantic information, while fusing with shallow features to solve the problem of low classification accuracy caused by insufficient semantic information of shallow features. Finally, an adaptive context model is proposed to use a deformable convolution to obtain spatial context features of shallow small objects, associating the targets with the background, thereby improving the accuracy of small object detection. The experimental results show that the detection accuracy of the proposed method has been improved by 6.12%, 3.35%, 3.33%, and 5.2%, respectively, compared with the source code on the PASCAL VOC, NWPU VHR-10, KITTI, and RSOD datasets, which fully demonstrate the effectiveness of our method in small object detection.

Real-time safety distance detection and warning for elevator inspection based on adaptive monocular depth estimation

During elevator inspection, workers are prone to collisions with hazards in narrow elevator machine rooms. Because of the numerous elevator machine rooms with diverse scenes in cities, it is difficult to effectively monitor the distance between workers and hazards. We propose a real-time safety distance detection and warning system consisting of four parts: object detection, depth estimation, safety distance detection, and safety warning. By introducing two adaptive error correction terms, the proposed adaptive monocular depth estimation algorithm effectively reduces the distortion of object pixel size with changes in depth and shooting angle. Then the calculation formula has been improved to accurately measure the distance between objects. The system has been successfully applied to safety supervision at the elevator inspection site in Nanchang City, Jiangxi Province, China. The results, with an average relative error of 3.34% and 84 frames detected per second, show the system exhibits excellent accuracy and real-time performance.

Combining transformer global and local feature extraction for object detection

Convolutional neural network (CNN)-based object detectors perform excellently but lack global feature extraction and cannot establish global dependencies between object pixels. Although the Transformer is able to compensate for this, it does not incorporate the advantages of convolution, which results in insufficient information being obtained about the details of local features, as well as slow speed and large computational parameters. In addition, Feature Pyramid Network (FPN) lacks information interaction across layers, which can reduce the acquisition of feature context information. To solve the above problems, this paper proposes a CNN-based anchor-free object detector that combines transformer global and local feature extraction (GLFT) to enhance the extraction of semantic information from images. First, the segmented channel extraction feature attention (SCEFA) module was designed to improve the extraction of local multiscale channel features from the model and enhance the discrimination of pixels in the object region. Second, the aggregated feature hybrid transformer (AFHTrans) module combined with convolution is designed to enhance the extraction of global and local feature information from the model and to establish the dependency of the pixels of distant objects. This approach compensates for the shortcomings of the FPN by means of multilayer information aggregation transmission. Compared with a transformer, these methods have obvious advantages. Finally, the feature extraction head (FE-Head) was designed to extract full-text information based on the features of different tasks. An accuracy of 47.0% and 82.76% was achieved on the COCO2017 and PASCAL VOC2007 + 2012 datasets, respectively, and the experimental results validate the effectiveness of our method.

Automatic detection method for tobacco beetles combining multi-scale global residual feature pyramid network and dual-path deformable attention

Aiming at the problems of identifying storage pest tobacco pest beetles from images that have few object pixels and considerable image noise, and therefore suffer from lack of information and identifiable features, this paper proposes an automatic monitoring method of tobacco beetle based on Multi-scale Global residual Feature Pyramid Network and Dual-path Deformable Attention (MGrFPN-DDrGAM). Firstly, a Multi-scale Global residual Feature Pyramid Network (MGrFPN) is constructed to obtain rich high-level semantic features and more complete information on low-level features to reduce missed detection; Then, a Dual-path Deformable receptive field Guided Attention Module (DDrGAM) is designed to establish long-range channel dependence, guide the effective fusion of features and improve the localization accuracy of tobacco beetles by fitting the spatial geometric deformation features of and capturing the spatial information of feature maps with different scales to enrich the feature information in the channel and spatial. Finally, to simulate a real scene, a multi-scene tobacco beetle dataset is created. The dataset includes 28,080 images and manually labeled tobacco beetle objects. The experimental results show that under the framework of the Faster R-CNN algorithm, the detection precision and recall rate of this method can reach 91.4% and 98.4% when the intersection ratio (IoU) is 0.5. Compared with Faster R-CNN and FPN, when the intersection ratio (IoU) is 0.7, the detection precision is improved by 32.9% and 6.9%, respectively. The proposed method is superior to the current mainstream methods.

Speckle noise suppression method in a holographic display based on pixel processing.

In this paper, we propose a method to suppress the speckle noise in a holographic display based on pixel processing. Through the separation of object pixels in space, the recorded object is divided into multiple object point groups. The complex amplitude of the light field for each object point group is recorded as a sub-computer-generated hologram (sub-CGH). The phase of each pixel on a sub-CGH is optimized to generate the final sub-CGH. Therefore, the pixels of the recorded object and sub-CGH are processed. In the reconstruction process, the final sub-CGHs are loaded on the spatial light modulator sequentially. The speckle noise of the reconstructed image is suppressed by reducing the algorithm error and the overlapping area of adjacent image points. The experimental results prove the feasibility of the proposed method.

A Vision Enhancement and Feature Fusion Multiscale Detection Network

In the field of object detection, there is often a high level of occlusion in real scenes, which can very easily interfere with the accuracy of the detector. Currently, most detectors use a convolutional neural network (CNN) as a backbone network, but the robustness of CNNs for detection under cover is poor, and the absence of object pixels makes conventional convolution ineffective in extracting features, leading to a decrease in detection accuracy. To address these two problems, we propose VFN (A Vision Enhancement and Feature Fusion Multiscale Detection Network), which first builds a multiscale backbone network using different stages of the Swin Transformer, and then utilizes a vision enhancement module using dilated convolution to enhance the vision of feature points at different scales and address the problem of missing pixels. Finally, the feature guidance module enables features at each scale to be enhanced by fusing with each other. The total accuracy demonstrated by VFN on both the PASCAL VOC dataset and the CrowdHuman dataset is better than that of other methods, and its ability to find occluded objects is also better, demonstrating the effectiveness of our method.The code is available at https://github.com/qcw666/vfn.

Pixel-Centric Context Perception Network for Camouflaged Object Detection.

Camouflaged object detection (COD) aims to identify object pixels visually embedded in the background environment. Existing deep learning methods fail to utilize the context information around different pixels adequately and efficiently. In order to solve this problem, a novel pixel-centric context perception network (PCPNet) is proposed, the core of which is to customize the personalized context of each pixel based on the automatic estimation of its surroundings. Specifically, PCPNet first employs an elegant encoder equipped with the designed vital component generation (VCG) module to obtain a set of compact features rich in low-level spatial and high-level semantic information across multiple subspaces. Then, we present a parameter-free pixel importance estimation (PIE) function based on multiwindow information fusion. Object pixels with complex backgrounds will be assigned with higher PIE values. Subsequently, PIE is utilized to regularize the optimization loss. In this way, the network can pay more attention to those pixels with higher PIE values in the decoding stage. Finally, a local continuity refinement module (LCRM) is used to refine the detection results. Extensive experiments on four COD benchmarks, five salient object detection (SOD) benchmarks, and five polyp segmentation benchmarks demonstrate the superiority of PCPNet with respect to other state-of-the-art methods.

VoxAR: Adaptive Visualization of Volume Rendered Objects in Optical See-Through Augmented Reality.

We present VoxAR, a method to facilitate an effective visualization of volume-rendered objects in optical see-through head-mounted displays (OST-HMDs). The potential of augmented reality (AR) to integrate digital information into the physical world provides new opportunities for visualizing and interpreting scientific data. However, a limitation of OST-HMD technology is that rendered pixels of a virtual object can interfere with the colors of the real-world, making it challenging to perceive the augmented virtual information accurately. We address this challenge in a two-step approach. First, VoxAR determines an appropriate placement of the volume-rendered object in the real-world scene by evaluating a set of spatial and environmental objectives, managed as user-selected preferences and pre-defined constraints. We achieve a real-time solution by implementing the objectives using a GPU shader language. Next, VoxAR adjusts the colors of the input transfer function (TF) based on the real-world placement region. Specifically, we introduce a novel optimization method that adjusts the TF colors such that the resulting volume-rendered pixels are discernible against the background and the TF maintains the perceptual mapping between the colors and data intensity values. Finally, we present an assessment of our approach through objective evaluations and subjective user studies.

Negative Deterministic Information-Based Multiple Instance Learning for Weakly Supervised Object Detection and Segmentation.

Weakly supervised object detection (WSOD) and semantic segmentation with image-level annotations have attracted extensive attention due to their high label efficiency. Multiple instance learning (MIL) offers a feasible solution for the two tasks by treating each image as a bag with a series of instances (object regions or pixels) and identifying foreground instances that contribute to bag classification. However, conventional MIL paradigms often suffer from issues, e.g., discriminative instance domination and missing instances. In this article, we observe that negative instances usually contain valuable deterministic information, which is the key to solving the two issues. Motivated by this, we propose a novel MIL paradigm based on negative deterministic information (NDI), termed NDI-MIL, which is based on two core designs with a progressive relation: NDI collection and negative contrastive learning (NCL). In NDI collection, we identify and distill NDI from negative instances online by a dynamic feature bank. The collected NDI is then utilized in a NCL mechanism to locate and punish those discriminative regions, by which the discriminative instance domination and missing instances issues are effectively addressed, leading to improved object-and pixel-level localization accuracy and completeness. In addition, we design an NDI-guided instance selection (NGIS) strategy to further enhance the systematic performance. Experimental results on several public benchmarks, including PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO, show that our method achieves satisfactory performance. The code is available at: https://github.com/GC-WSL/NDI.

Structured Knowledge Distillation for Accurate and Efficient Object Detection.

Knowledge distillation, which aims to transfer the knowledge learned by a cumbersome teacher model to a lightweight student model, has become one of the most popular and effective techniques in computer vision. However, many previous knowledge distillation methods are designed for image classification and fail in more challenging tasks such as object detection. In this paper, we first suggest that the failure of knowledge distillation on object detection is mainly caused by two reasons: (1) the imbalance between pixels of foreground and background and (2) lack of knowledge distillation on the relation among different pixels. Then, we propose a structured knowledge distillation scheme, including attention-guided distillation and non-local distillation to address the two issues, respectively. Attention-guided distillation is proposed to find the crucial pixels of foreground objects with an attention mechanism and then make the students take more effort to learn their features. Non-local distillation is proposed to enable students to learn not only the feature of an individual pixel but also the relation between different pixels captured by non-local modules. Experimental results have demonstrated the effectiveness of our method on thirteen kinds of object detection models with twelve comparison methods for both object detection and instance segmentation. For instance, Faster RCNN with our distillation achieves 43.9 mAP on MS COCO2017, which is 4.1 higher than the baseline. Additionally, we show that our method is also beneficial to the robustness and domain generalization ability of detectors. Codes and model weights have been released on GitHub†.

OSC-CO2: coattention and cosegmentation framework for plant state change with multiple features.

Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object's pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%.

Vision and Tactile Robotic System to Grasp Litter in Outdoor Environments

The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another has been designed to detect slippage in order to prevent the objects grasped from falling. Our proposal was successfully tested by carrying out extensive experimentation with different objects varying in size, texture, geometry and materials in different outdoor environments (a tiled pavement, a surface of stone/soil, and grass). Our system achieved an average score of 94% for the detection and Collection Success Rate (CSR) as regards its overall performance, and of 80% for the collection of items of litter at the first attempt.

Coarse2Fine: Local Consistency Aware Re-prediction for Weakly Supervised Object Localization

Weakly supervised object localization aims to localize objects of interest by using only image-level labels. Existing methods generally segment activation map by threshold to obtain mask and generate bounding box. However, the activation map is locally inconsistent, i.e., similar neighboring pixels of the same object are not equally activated, which leads to the blurred boundary issue: the localization result is sensitive to the threshold, and the mask obtained directly from the activation map loses the fine contours of the object, making it difficult to obtain a tight bounding box. In this paper, we introduce the Local Consistency Aware Re-prediction (LCAR) framework, which aims to recover the complete fine object mask from locally inconsistent activation map and hence obtain a tight bounding box. To this end, we propose the self-guided re-prediction module (SGRM), which employs a novel superpixel aggregation network to replace the post-processing of threshold segmentation. In order to derive more reliable pseudo label from the activation map to supervise the SGRM, we further design an affinity refinement module (ARM) that utilizes the original image feature to better align the activation map with the image appearance, and design a self-distillation CAM (SD-CAM) to alleviate the locator dependence on saliency. Experiments demonstrate that our LCAR outperforms the state-of-the-art on both the CUB-200-2011 and ILSVRC datasets, achieving 95.89% and 70.72% of GT-Know localization accuracy, respectively.

An automated system for polymer wear debris analysis in total disc arthroplasty using convolution neural network.

Introduction: Polymer wear debris is one of the major concerns in total joint replacements due to wear-induced biological reactions which can lead to osteolysis and joint failure. The wear-induced biological reactions depend on the wear volume, shape and size of the wear debris and their volumetric concentration. The study of wear particles is crucial in analysing the failure modes of the total joint replacements to ensure improved designs and materials are introduced for the next generation of devices. Existing methods of wear debris analysis follow a traditional approach of computer-aided manual identification and segmentation of wear debris which encounters problems such as significant manual effort, time consumption, low accuracy due to user errors and biases, and overall lack of insight into the wear regime. Methods: This study proposes an automatic particle segmentation algorithm using adaptive thresholding followed by classification using Convolution Neural Network (CNN) to classify ultra-high molecular weight polyethylene polymer wear debris generated from total disc replacements tested in a spine simulator. A CNN takes object pixels as numeric input and uses convolution operations to create feature maps which are used to classify objects. Results: Classification accuracies of up to 96.49% were achieved for the identification of wear particles. Particle characteristics such as shape, size and area were estimated to generate size and volumetric distribution graphs. Discussion: The use of computer algorithms and CNN facilitates the analysis of a wider range of wear debris with complex characteristics with significantly fewer resources which results in robust size and volume distribution graphs for the estimation of the osteolytic potential of devices using functional biological activity estimates.

Photon-noise: is a single-pixel camera better than point scanning? A signal-to-noise ratio analysis for Hadamard and Cosine positive modulation

In a single-pixel camera, an unknown object is sequentially illuminated by intensity patterns. The total reflected or transmitted intensity is summed in a single-pixel detector from which the object is computationally reconstructed. In the situation where the measurements are limited by photon-noise, it is questionable whether a single-pixel camera performs better or worse than simply scanning the object with a focused intensity spot—a modality known as point raster scanning and employed in many laser scanning systems. Here, we solve this general question and report that positive intensity modulation based on Hadamard or Cosine patterns does not necessarily improve the single-to-noise ratio (SNR) of single-pixel cameras, as compared to point raster scanning (RS). Instead, we show that the SNR is only improved on object pixels at least k times brighter than the object mean signal , where k is a constant that depends on the modulation scheme (modulation matrix, number of detectors, etc). The constant k is derived for several widespread cases and has important consequences on the choice of the optical deign. This fundamental property is demonstrated theoretically, numerically, and is experimentally confirmed in the spatial domain (widefield fluorescence imaging) and in the spectral domain (spontaneous Raman spectral measurements). Finally, we provide user-oriented guidelines that help decide when and how multiplexing under photon-noise should be used instead of point RS.

Steganographic Method in Selected Areas of the Stego-Carrier in the Spatial Domain

The main goal of this paper is the proposal of a key-based steganographic system in which the ratio of capacity and image quality metrics that represents the stego object while reducing the detectability of hidden content was improved. The main contribution of the proposed steganographic system is a new algorithm for selecting stego areas. The area selection algorithm is based on clustering the pixels of the cover object into a predetermined number of clusters. The goal of this selection of areas (clusters) is to group as many homogeneous parts of the image as possible in order to cover these areas with as few rectangular shapes as possible. Since the data on the defined rectangles represent the key of the system, the capacity of the additional secret channel is minimized in this way. On the obtained stego-carriers, an embedding of test random content is performed in order to estimate its detectability. By combining the proposed area selection method with the Minimal Decimal Difference steganographic method, a system was created with an optimal trade-off between detectability of secret content, quality and capacity of the carrier, and the length of the stego-key. Finally, a comparison of the obtained results with relevant adaptive steganographic methods is presented. The proposed concept obtains its verification in one practical system for secure file transfer of controlled cryptographic strength.

POSEA: A novel algorithm to evaluate the performance of multi-object instance image segmentation.

Many techniques and software packages have been developed to segment individual cells within microscopy images, necessitating a robust method to evaluate images segmented into a large number of unique objects. Currently, segmented images are often compared with ground-truth images at a pixel level; however, this standard pixel-level approach fails to compute errors due to pixels incorrectly assigned to adjacent objects. Here, we define a per-object segmentation evaluation algorithm (POSEA) that calculates segmentation accuracy metrics for each segmented object relative to a ground truth segmented image. To demonstrate the performance of POSEA, precision, recall, and f-measure metrics are computed and compared with the standard pixel-level evaluation for simulated images and segmented fluorescence microscopy images of three different cell samples. POSEA yields lower accuracy metrics than the standard pixel-level evaluation due to correct accounting of misclassified pixels of adjacent objects. Therefore, POSEA provides accurate evaluation metrics for objects with pixels incorrectly assigned to adjacent objects and is robust for use across a variety of applications that require evaluation of the segmentation of unique adjacent objects.

3D Street Object Detection from Monocular Images Using Deep Learning and Depth Information

In this study, we present a three-dimensional (3D) object detection algorithm based on monocular images by constructing an end-to-end network, that incorporates depth information. The entire network consists of three parts. The first part includes the basic object detection neural network as the main body, that uses the region proposal network to obtain the two-dimensional (2D) region proposal of the object. The second part is the depth estimation branch network, that obtains the depth information of the object pixels and calculates the corresponding 3D point cloud. In the last part, concatenated features obtained from the aforementioned two parts are fed into the fully-connected layers. Subsequently, 2D and 3D detection results are obtained. Compared with certain existing methods, the accuracy of the detection results is improved in this study.

ECFNet: A Siamese Network With Fewer FPs and Fewer FNs for Change Detection of Remote-Sensing Images

High-resolution remote-sensing image change detection plays an important role in areas such as land and resources investigation, natural disaster prediction, and military strategy research. Current change detection methods often focus on extracting more discriminative features, while ignoring the information loss and imbalance problems in the process of feature fusion, which results in weakness in small change objects and edge pixels of change objects. In this letter, a simple but efficient network architecture, extraction, comparison and fusion network (ECFNet), for change detection in remote-sensing images is proposed. By constraining the number of feature channels in the fusion process by the feature comparison module (FCM), ECFNet can better utilize the fine-grained information in the multiscale feature map for result prediction, which not only improves the detection performance of small objects, but also reduces the false detection around the edge pixels of change objects. Experiments on the deeply supervised image fusion network for change detection (DSIFN-CD) test set show that ECFNet achieves state-of-the-art results with a small amount of computation.

Unsupervised Face Detection in the Dark.

Low-light face detection is challenging but critical for real-world applications, such as nighttime autonomous driving and city surveillance. Current face detection models rely on extensive annotations and lack generality and flexibility. In this paper, we explore how to learn face detectors without low-light annotations. Fully exploiting existing normal light data, we propose adapting face detectors from normal light to low light. This task is difficult because the gap between brightness and darkness is too large and complicated at the object level and pixel level. Accordingly, the performance of current low-light enhancement or adaptation methods is unsatisfactory. To solve this problem, we propose a joint High-Low Adaptation (HLA) framework. We design bidirectional low-level adaptation and multitask high-level adaptation. For low-level, we enhance the dark images and degrade the normal-light images, making both domains move toward each other. For high-level, we combine context-based and contrastive learning to comprehensively close the features on different domains. Experiments show that our HLA-Face v2 model obtains superior low-light face detection performance even without the use of low-light annotations. Moreover, our adaptation scheme can be extended to a wide range of applications, such as improving supervised learning and generic object detection. Project publicly available at: https://daooshee.github.io/HLA-Face-v2-Website/.