Object Pixels Research Articles

Position-Aware Relation Learning for RGB-Thermal Salient Object Detection

Salient object detection (SOD) is an important task in computer vision that aims to identify visually conspicuous regions in images. RGB-Thermal SOD combines two spectra to achieve better segmentation results. However, most existing methods for RGB-T SOD use boundary maps to learn sharp boundaries, which lead to sub-optimal performance as they ignore the interactions between isolated boundary pixels and other confident pixels. To address this issue, we propose a novel position-aware relation learning network (PRLNet) for RGB-T SOD. PRLNet explores the distance and direction relationships between pixels by designing an auxiliary task and optimizing the feature structure to strengthen intra-class compactness and inter-class separation. Our method consists of two main components: A signed distance map auxiliary module (SDMAM), and a feature refinement approach with direction field (FRDF). SDMAM improves the encoder feature representation by considering the distance relationship between foreground-background pixels and boundaries, which increases the inter-class separation between foreground and background features. FRDF rectifies the features of boundary neighborhoods by exploiting the features inside salient objects. It utilizes the direction relationship of object pixels to enhance the intra-class compactness of salient features. In addition, we constitute a transformer-based decoder to decode multispectral feature representation. Experimental results on three public RGB-T SOD datasets demonstrate that our proposed method not only outperforms the state-of-the-art methods, but also can be integrated with different backbone networks in a plug-and-play manner. Ablation study and visualizations further prove the validity and interpretability of our method.

An Uncertainty-Adaptive Consistency Training Algorithm for Semi-Supervised Object Counting Networks

Object counting presents a significant challenge for applications demanding high levels of automation. In recently ears, semi-supervised counting methods have been explored to alleviate the burden of data annotation. Despite considerable progress, the state-of-the-art methods treat pixels with varying levels of difficulty equally during each training iteration, resulting in an inflexible consistency training that can lead to high supervision loss at the onset of training and potentially cause network collapse. To address this limitation, we propose an uncertainty-adaptive consistency training algorithm for semi-supervised object counting networks in this paper. Our approach leverages image colorization to construct colorization uncertainty, which can measure the difficulty of object pixels and background pixels. To ensure superior uncertainty estimation, we then enhance this colorization uncertainty by using density features to guide the learning of colorization. Furthermore, an uncertainty-adaptive consistency training (UACT) algorithm is introduced for the consistency training of density maps. By adaptively adjusting the uncertainty threshold, we prioritize harder pixels for selection, gradually incorporating easier ones into the consistency training. The effectiveness o f t he UACT is evaluated through extensive experiments on two datasets. Experimental results demonstrate that our method outperforms state-of-the-art semi-supervised counting methods.

Adaptive Feature Fusion for Small Object Detection

In order to alleviate the situation that small objects are prone to missed detection and false detection in natural scenes, this paper proposed a small object detection algorithm for adaptive feature fusion, referred to as MMF-YOLO. First, aiming at the problem that small object pixels are easy to lose, a multi-branch cross-scale feature fusion module with fusion factor was proposed, where each fusion path has an adaptive fusion factor, which can allow the network to independently adjust the importance of features according to the learned weights. Then, aiming at the problem that small objects are similar to background information and small objects overlap in complex scenes, the M-CBAM attention mechanism was proposed, which was added to the feature reinforcement extraction module to reduce feature redundancy. Finally, in light of the problem of small object size and large size span, the size of the object detection head was modified to adapt to the small object size. Experiments on the VisDrone2019 dataset showed that the mAP of the proposed algorithm could reach 42.23%, and the parameter quantity was only 29.33 MB, which is 9.13% ± 0.07% higher than the benchmark network mAP, and the network model was reduced by 5.22 MB.

Robust self-supervised monocular visual odometry based on prediction-update pose estimation network

Visual odometry aims at estimating the camera pose from video sequence, which is an important part of visual Simultaneous Localization and Mapping (SLAM). In this paper, we propose a novel prediction-update pose estimation network, PU-PoseNet, for self-supervised monocular visual odometry. It allows the network to use the effective information of the previous frame in estimating the current pose. The long-time pose consistency constraint-based motion weighted photometric loss is designed to make the network to pay more attention to the pixels of stationary objects and enhance the time consistency of estimation results. The depth image-based occlusion detection, the depth smoothness loss and auto-Masking are used to construct the depth consistency constraint loss term to reduce the influences of interferences such as occlusion. To further improve the robustness and the accuracy of the proposed method, both the depth consistency constraint and the variational auto-encoder are used for network training. For frame missing cases, a novel frame missing training strategy is used to make our method adapt frame missing cases. Extensive experiments on the KITTI dataset have validated the effectiveness of our proposed method.

ER-Pose: Learning edge representation for 6D pose estimation of texture-less objects

6D pose estimation from a single RGB image is a fundamental task in computer vision. We introduce a two-stage 6D pose estimation method for texture-less objects. Instead of utilizing the object mask in almost current monocular methods, we propose an edge representation for texture-less objects. An object is represented by the combination of visible edges corresponding to the object’s 6D pose, allowing the neural network to focus more on the object’s invariant global shape and structure, rather than indistinguishable local patches with image noise and similar texture. Given an RGB image, the proposed method predicts the direction and distance to a certain object keypoint from all object pixels within the range of object edge representation, establishes voting-based sparse 2D-3D correspondences, and solves the object pose with PnP algorithm. In the experiments, we found that directly replacing the object mask with the edge representation can bring a performance improvement in two current two-stage pipelines without any modification. Further evaluations on three different benchmark datasets containing symmetric and occluded objects show our method outperforms the state-of-the-art methods using RGB images only.

Self-Supervised Leaf Segmentation under Complex Lighting Conditions

As an essential prerequisite task in image-based plant phenotyping, leaf segmentation has garnered increasing attention in recent years. While self-supervised learning is emerging as an effective alternative to various computer vision tasks, its adaptation for image-based plant phenotyping remains rather unexplored. In this work, we present a self-supervised leaf segmentation framework consisting of a self-supervised semantic segmentation model, a color-based leaf segmentation algorithm, and a self-supervised color correction model. The self-supervised semantic segmentation model groups the semantically similar pixels by iteratively referring to the self-contained information, allowing the pixels of the same semantic object to be jointly considered by the color-based leaf segmentation algorithm for identifying the leaf regions. Additionally, we propose to use a self-supervised color correction model for images taken under complex illumination conditions. Experimental results on datasets of different plant species demonstrate the potential of the proposed self-supervised framework in achieving effective and generalizable leaf segmentation.

Long-Distance Vehicle Dynamic Detection and Positioning Based on Gm-APD Lidar and LIDAR-YOLO

Aiming at the problems of long-distance night dynamic object detection in military operations, such as difficult imaging of traditional equipment, a small number of object pixels, a large difference in object size at different distances, and poor real-time performance. A long-distance vehicle detection method based on the LIDAR-YOLO network for Gm-APD (Geiger mode avalanche photodiode) lidar image is proposed. This method is improved based on YOLOv5s, which can detect long-distance vehicles and accurately locate motion trajectories in real-time. Firstly, the intensity image I of the lidar is preprocessed and combined with the distance image R, the BC_IIR image is obtained, and the vehicle detection data set is established. Then, according to the obvious vehicle features in the BC_IIR image, the low-complexity backbone network LIDARNet is redesigned, and several attention mechanism modules are added. The network feature extraction ability is enhanced by comparing and selecting the CA attention mechanism module. In the network prediction layer, context-aware feature map fusion is added to improve the detection precision of vehicles at different distances. Next, a GIOU loss is introduced to improve the detection precision of occluded vehicles. Finally, the method can depict the trajectory of the vehicle in 3D space based on distance information. After testing, the network achieves a detection speed of 62.67 frames per second (FPS), recall rate (R) = 98.19%, precision rate (P) = 95.53%, and F1 is 0.9675. The results show that the algorithm has better performance than other methods.

SPCS: a spatial pyramid convolutional shuffle module for YOLO to detect occluded object

In crowded scenes, one of the most important issues is that heavily overlapped objects are hardly distinguished from each other since most of their pixels are shared and the visible pixels of the occluded objects, which are used to represent their features, are limited. In this paper, a spatial pyramid convolutional shuffle (SPCS) module is proposed to extract refined information from the limited visible pixels of the occluded objects and generate distinguishable representations for the heavily overlapped objects. We adopt four convolutional kernels with different sizes and dilation rates at each location in the pyramid features and adjacently recombine their fused outputs spatially using a pixel shuffle module. In this way, four distinguishable instance predictions corresponding different convolutional kernels can be produced for each location in the pyramid feature. In addition, multiple convolutional operations with different kernel sizes and dilation rates at the same location can generate refined information for the corresponding regions, which is helpful to extract features for the occluded objects from their limited visible pixels. Extensive experimental results demonstrate that SPCS module can effectively boost the performance in crowded human detection. YOLO detector with SPCS module achieves 94.11% AP, 41.75% MR, 97.75% Recall on CrowdHuman, 93.04% AP, and 98.45% Recall on WiderPerson, which are the best compared with previous state-of-the-art models.

MobileTrack: Siamese efficient mobile network for high‐speed UAV tracking

Recently, Siamese-based trackers have drawn amounts of attention in visual tracking field because of their excellent performance. However, visual object tracking on Unmanned Aerial Vehicles platform encounters difficulties under circumstances such as small objects and similar objects interference. Most existing tracking methods for aerial tracking adopt deeper networks or inefficient policies to promote performance, but most trackers can hardly meet real-time requirements on mobile platforms with limited computing resources. Thus, in this work, an efficient and lightweight siamese tracker (MobileTrack) is proposed for high-time Unmanned Aerial Vehicles tracking, realising the balance between performance and speed. Firstly, a lightweight convolutional network (D-MobileNet) is designed to enhance the characterisation ability of small objects. Secondly, an efficient object-aware module is proposed for local cross-channel information exchange, enhancing the feature information of the tracking object. Besides, an anchor-free region proposal network is introduced to predict the object pixel by pixel. Finally, deep and shallow feature information is fully utilised by cascading multiple anchor-free region proposal networks for accurate locating and robust tracking. Extensive experiments on the three Unmanned Aerial Vehicles benchmarks show that the proposed tracker achieves outstanding performance while keeping a beyond-real-time speed.

Convex optimization for additive noise reduction in quantitative complex object wave retrieval using compressive off-axis digital holographic imaging

Abstract Image denoising is one of the important problems in the research field of computer vision, artificial intelligence, 3D vision, and image processing, where the fundamental aim is to recover the original image features from a noisy contaminated image. The camera sensor additive noise present in the holographic recording process reduces the quality of the retrieved image. Even though various techniques have been developed to minimize the noise in digital holography, the noise reduction still remains a challenging task. This article presents a compressive sensing (CS) technique to minimize the additive noise in the digital holographic reconstruction process. We demonstrate the reduction of additive noise using complex wave retrieval method as a sensing matrix in the CS model. The proposed CS method to suppress the noise during the reconstruction process is illustrated using numerical simulations. Only 50% of the pixel measurements are considered in the noisy hologram, which is far less than the original complex object pixels. The impact of additive gaussian noise in the recording plane on the reconstruction accuracy of both intensity and phase distribution is analysed. The CS method denoises and estimates the complex object information accurately. The numerical simulation results have shown that the proposed CS method has effectively minimized the noise in the reconstructed image and has greatly improved the quality of both intensity and phase information.

Part-Based Obstacle Detection Using a Multiple Output Neural Network.

Detecting the objects surrounding a moving vehicle is essential for autonomous driving and for any kind of advanced driving assistance system; such a system can also be used for analyzing the surrounding traffic as the vehicle moves. The most popular techniques for object detection are based on image processing; in recent years, they have become increasingly focused on artificial intelligence. Systems using monocular vision are increasingly popular for driving assistance, as they do not require complex calibration and setup. The lack of three-dimensional data is compensated for by the efficient and accurate classification of the input image pixels. The detected objects are usually identified as cuboids in the 3D space, or as rectangles in the image space. Recently, instance segmentation techniques have been developed that are able to identify the freeform set of pixels that form an individual object, using complex convolutional neural networks (CNNs). This paper presents an alternative to these instance segmentation networks, combining much simpler semantic segmentation networks with light, geometrical post-processing techniques, to achieve instance segmentation results. The semantic segmentation network produces four semantic labels that identify the quarters of the individual objects: top left, top right, bottom left, and bottom right. These pixels are grouped into connected regions, based on their proximity and their position with respect to the whole object. Each quarter is used to generate a complete object hypothesis, which is then scored according to object pixel fitness. The individual homogeneous regions extracted from the labeled pixels are then assigned to the best-fitted rectangles, leading to complete and freeform identification of the pixels of individual objects. The accuracy is similar to instance segmentation-based methods but with reduced complexity in terms of trainable parameters, which leads to a reduced demand for computational resources.

Towards lifelong object recognition: A dataset and benchmark

Lifelong learning algorithms aim to enable robots to handle open-set and detrimental conditions, and yet there is a lack of adequate datasets with diverse factors for benchmarking. In this work, we constructed and released a lifelong learning robotic vision dataset, OpenLORIS-Object. This dataset was collected by RGB-D camera capturing dynamic environment in daily life scenarios with diverse factors, including illumination, occlusion, object pixel size and clutter, of quantified difficulty levels. To the best of our knowledge, this is an unique real-world dataset for robotic vision with independent and quantifiable environmental factors, which are currently unaccounted for in other lifelong learning datasets such as CORe50 and NICO. We tested 9 state-of-the-art algorithms with 4 evaluation metrics over the dataset in Domain Incremental Learning, Task Incremental Learning, and Class Incremental Learning scenarios. The results demonstrate that these existing algorithms are insufficient to handle lifelong learning task in dynamic environments. Our dataset and benchmarks are now publicly available at this website.22https://lifelong-robotic-vision.github.io/dataset/object

Semantic Modeling and Pixel Discrimination for Image Manipulation Detection

Image manipulation methods, such as the copy-move, splicing, and removal methods, have become increasingly mature and changed the common perception of “seeing is believing.” The credibility of digital media has been seriously damaged with the development of image manipulation methods. Most image manipulation detection methods detect traces of tampering pixel by pixel. As a result, the detected manipulation areas are separated, which results in insufficient consideration of content manipulation at the object level. In this paper, the detection of image manipulation areas based on forgery object detection and pixel discrimination is proposed. Specifically, the pixel-level detection branch resamples features and uses an LSTM to detect manipulations, such as resampling, rotation, and cropping. The goal of the forgery object detection branch, which is based on Faster R-CNN, is to extract the regions of interest and analyze the regions with high contrast as well as the forgery objects of the image. Furthermore, the fused heatmaps of the two branches are integrated with the object detection results. The noise in the heatmaps is shielded based on the forgery object information of the region proposal network. Experimental results on multiple standard forgery datasets have demonstrated the superiority of our proposed method compared with the state-of-the-art methods.

A video-based SlowFastMTB model for detection of small amounts of smoke from incipient forest fires

Abstract This paper proposes a video-based SlowFast model that combines the SlowFast deep learning model with a new boundary box annotation algorithm. The new algorithm, namely the MTB (i.e., the ratio of the number of Moving object pixels To the number of Bounding box pixels) algorithm, is devised to automatically annotate the bounding box that includes the smoke with fuzzy boundaries. The model parameters of the MTB algorithm are examined by multifactor analysis of variance. To demonstrate the validity of the proposed approach, a case study is provided that examines real video clips of incipient forest fires with small amounts of smoke. The performance of the proposed approach is compared with those of existing deep learning models, including convolutional neural network (CNN), faster region-based CNN (faster R-CNN), and SlowFast. It is demonstrated that the proposed approach achieves enhanced detection accuracy, while reducing false negative rates.

Automated Object Detection in Experimental Data Using Combination of Unsupervised and Supervised Methods.

Deep neural networks (DNN) have shown their success through computer vision tasks such as object detection, classification, and segmentation of image data including clinical and biological data. However, supervised DNNs require a large volume of labeled data to train and great effort to tune hyperparameters. The goal of this study is to segment cardiac images in movie data into objects of interest and a noisy background. This task is one of the essential tasks before statistical analysis of the images. Otherwise, the statistical values such as means, medians, and standard deviations can be erroneous. In this study, we show that the combination of unsupervised and supervised machine learning can automatize this process and find objects of interest accurately. We used the fact that typical clinical/biological data contain only limited kinds of objects. We solve this problem at the pixel level. For example, if there is only one object in an image, there are two types of pixels: object pixels and background pixels. We can expect object pixels and background pixels are quite different and they can be grouped using unsupervised clustering methods. In this study, we used the k-means clustering method. After finding object pixels and background pixels using unsupervised clustering methods, we used these pixels as training data for supervised learning. In this study, we used logistic regression and support vector machine. The combination of the unsupervised method and the supervised method can find objects of interest and segment images accurately without predefined thresholds or manually labeled data.

Tiny Vehicle Detection for Mid-to-High Altitude UAV Images Based on Visual Attention and Spatial-Temporal Information.

Mid-to-high altitude Unmanned Aerial Vehicle (UAV) imagery can provide important remote sensing information between satellite and low altitude platforms, and vehicle detection in mid-to-high altitude UAV images plays a crucial role in land monitoring and disaster relief. However, the high background complexity of images and limited pixels of objects challenge the performance of tiny vehicle detection. Traditional methods suffer from poor adaptation ability to complex backgrounds, while deep neural networks (DNNs) have inherent defects in feature extraction of tiny objects with finite pixels. To address the issue above, this paper puts forward a vehicle detection method combining the DNNs-based and traditional methods for mid-to-high altitude UAV images. We first employ the deep segmentation network to exploit the co-occurrence of the road and vehicles, then detect tiny vehicles based on visual attention mechanism with spatial-temporal constraint information. Experimental results show that the proposed method achieves effective detection of tiny vehicles in complex backgrounds. In addition, ablation experiments are performed to inspect the effectiveness of each component, and comparative experiments on tinier objects are carried out to prove the superior generalization performance of our method in detecting vehicles with a limited size of 5 × 5 pixels or less.

Grabcut Algorithm: A New Assessment of Burn Wound Area from Digital Images

Background: As far as high technology is concerned, digital images of burn patients are widely used for wound documentation and specialist consultation. Wound area can be approximated from the photographs to aid further treatment for the patients. However, objective estimation of the wound areas in the digital images is still problematic. Objective: To calculate sizes of burn wounds from digital images. Materials and Methods: A computer application, Burn Percentage Calculation (BPC), was developed based on image segmentation algorithm called Grabcut. This image processing tool extracted an object of interest out of the background by using differences of pixel color and contrast in the image. The probabilistic models of being foreground and background were created to distinguish object pixels from the rest of the input image. The BPC application effectively extracted patient’s body parts and burn wounds from the background. By mapping to Lund and Browder chart in the application, the wound area in a unit of percent total body surface area (%TBSA) was obtained. To validate the application, 10 plastic surgery residents were assigned to assess burn wound areas from 5 digital photographs using BPC application compared to conventional Lund and Browder chart. Results: Forty-one digital images of burn patients were processed by BPC application. The application clearly separated the burn wounds from the background, and calculated wound areas in a unit of %TBSA. Burn sizes calculated by BPC application were not significantly different from those using conventional Lund and Browder diagram. Conclusion: Grabcut algorithm is a powerful image segmentation tool which can sufficiently extract burn wounds from a 2D photograph. Our study demonstrated that sizes of burn wounds in the photographs were accurately calculated using an image segmentation-aided computer application. The resulting burn areas can guide individualized treatment for burned patients. Keywords: Burn wound area; Digital image; Grabcut algorithm; Image segmentation; TBSA

Reconstruction osseuse : évaluation subjective et analyse objective basées sur les radiographies numériques conventionnelles – analyse rétrospective

BackgroundVarious approaches have been used to evaluate callus stiffness in distraction osteogenesis. This assessment becomes even more critical as inadequate corticalization and premature removal of the frame lead to bending, refracture, and loss of length. This study aimed to verify the objective Pixel Value Ratio (PVR) and define an evaluation scheme to assess callus tissue development during internal and external bone lengthening. HypothesisThere are no differences between subjective parameters and objective evaluation for external and internal bone lengthening. Patients and methodsWe included 24 patients in this retrospective study, treated by distraction osteogenesis of the lower limb. Ten patients were treated with external ring fixators and 14 with intramedullary lengthening devices. The minimum distraction distance was 3cm. Callus tissue was evaluated with the PVR, using digital x-rays during and after treatment. We combined this method with subjective evaluation parameters, including the continuity, signal intensity, and the regeneration tissue's homogeneity, presented in the X-ray Evaluation System for Distraction Osteogenesis (XESDO). ResultsThe subjective evaluation showed an increase of continuity and signal-intensity between 4th postoperative weeks and complete healing with significant differences between the external and internal groups for homogeneity only after four weeks and for all parameters at 3C (p<0.05). The PVR showed an increase during consolidation with values between 0.7 and 1.0 for both groups with almost perfect interobserver agreement. Significant lower values were found in the anterior tibial cases. ConclusionPVR appears to be a practical tool for radiological evaluation of callus tissue development. In combination with the newly introduced XESDO and the surgeon's experience, these methods can be helpful for decisions during all phases of distraction osteogenesis. It is important for adapting alignment or distraction-rate, as various potential complications can be detected early. However, further studies are necessary to assess the image-based determination of callus tissue stability. Level of evidenceIII; cohort study.

Cluster-Based Memetic Approach of Image Alignment

The paper presents a new memetic, cluster-based methodology for image registration in case of geometric perturbation model involving translation, rotation and scaling. The methodology consists of two stages. First, using the sets of the object pixels belonging to the target image and to the sensed image respectively, the boundaries of the search space are computed. Next, the registration mechanism residing in a hybridization between a version of firefly population-based search procedure and the two membered evolutionary strategy computed on clustered data is applied. In addition, a procedure designed to deal with the premature convergence problem is embedded. The fitness to be maximized by the memetic algorithm is defined by the Dice coefficient, a function implemented to evaluate the similarity between pairs of binary images. The proposed methodology is applied on both binary and monochrome images. In case of monochrome images, a preprocessing step aiming the binarization of the inputs is considered before the registration. The quality of the proposed approach is measured in terms of accuracy and efficiency. The success rate based on Dice coefficient, normalized mutual information measures, and signal-to-noise ratio are used to establish the accuracy of the obtained algorithm, while the efficiency is evaluated by the run time function.

Bone reconstruction: Subjective evaluation and objective analysis based on conventional digital X-rays – a retrospective evaluation

BackgroundVarious approaches have been used to evaluate callus stiffness in distraction osteogenesis. This assessment becomes even more critical as inadequate corticalization and premature removal of the frame lead to bending, refracture, and loss of length. This study aimed to verify the objective Pixel Value Ratio (PVR) and define an evaluation scheme to assess callus tissue development during internal and external bone lengthening. HypothesisThere are no differences between subjective parameters and objective evaluation for external and internal bone lengthening. Patients and methodsWe included 24 patients in this retrospective study, treated by distraction osteogenesis of the lower limb. Ten patients were treated with external ring fixators and 14 with intramedullary lengthening devices. The minimum distraction distance was 3cm. Callus tissue was evaluated with the PVR, using digital X-rays during and after treatment. We combined this method with subjective evaluation parameters, including the continuity, signal intensity, and the regeneration tissue's homogeneity, presented in the X-ray Evaluation System for Distraction Osteogenesis (XESDO). ResultsThe subjective evaluation showed an increase of continuity and signal-intensity between 4th postoperative weeks and complete healing with significant differences between the external and internal groups for homogeneity only after four weeks and for all parameters at 3C (p<0.05). The PVR showed an increase during consolidation with values between 0.7 and 1.0 for both groups with almost perfect interobserver agreement. Significant lower values were found in the anterior tibial cases. ConclusionPVR appears to be a practical tool for radiological evaluation of callus tissue development. In combination with the newly introduced XESDO and the surgeon's experience, these methods can be helpful for decisions during all phases of distraction osteogenesis. It is important for adapting alignment or distraction-rate, as various potential complications can be detected early. However, further studies are necessary to assess the image-based determination of callus tissue stability. Level of evidenceIII; cohort study.