Object Regions Research Articles

Person Re-Identification With Visual Semantic Representation Mining and Reasoning

Person re-identification aims to match images of the same person in different scenarios. The challenge of this task is how to extract discriminative features from person images with complex background noise, severe occlusions and large pose variations. Recently, some studies have attempted to apply human semantic parsing or attention mechanisms to help capture human parts or important object regions. Despite the performance improvements, these methods either require the introduction of additional prior knowledge or ignore the connections between the human body parts. To solve the above problem, we propose a novel Visual Semantic Representation Mining and Reasoning (SRMR) block to capture more discriminative semantic features of person images. Specifically, to get rid of the restriction of the external model, we propose to directly mine the clustering relationship between each local feature in the global structure and obtain the discriminative region in the person image by voting. Then, to establish the relationship between person body parts, we utilize Graph Convolutional Networks (GCN) to effectively construct the correlation between body parts. Extensive ablation studies demonstrate that our SRMR block can significantly improve the feature representation power and achieve state-of-the-art performance on several popular benchmarks.

Blind Quality Assessment of Images Containing Objects of Interest.

To monitor objects of interest, such as wildlife and people, image-capturing devices are used to collect a large number of images with and without objects of interest. As we are recording valuable information about the behavior and activity of objects, the quality of images containing objects of interest should be better than that of images without objects of interest, even if the former exhibits more severe distortion than the latter. However, according to current methods, quality assessments produce the opposite results. In this study, we propose an end-to-end model, named DETR-IQA (detection transformer image quality assessment), which extends the capability to perform object detection and blind image quality assessment (IQA) simultaneously by adding IQA heads comprising simple multi-layer perceptrons at the top of the DETRs (detection transformers) decoder. Using IQA heads, DETR-IQA carried out blind IQAs based on the weighted fusion of the distortion degree of the region of objects of interest and the other regions of the image; the predicted quality score of images containing objects of interest was generally greater than that of images without objects of interest. Currently, the subjective quality score of all public datasets is in accordance with the distortion of images and does not consider objects of interest. We manually extracted the images in which the five predefined classes of objects were the main contents of the largest authentic distortion dataset, KonIQ-10k, which was used as the experimental dataset. The experimental results show that with slight degradation in object detection performance and simple IQA heads, the values of PLCC and SRCC were 0.785 and 0.727, respectively, and exceeded those of some deep learning-based IQA models that are specially designed for only performing IQA. With the negligible increase in the computation and complexity of object detection and without a decrease in inference speeds, DETR-IQA can perform object detection and IQA via multi-tasking and substantially reduce the workload.

Deep attention sampling hashing for efficient image retrieval

Hashing has received broad attention in large-scale image retrieval due to its appealing efficiency in computation and storage. Particularly, with the drawn of deep learning, much efforts have been directed towards using deep neural networks to learn feature representations and hash codes simultaneously, and the developed deep hashing methods have shown superior performance over conventional hashing methods. In this paper, we propose Deep Attention Sampling Hashing (DASH), a novel deep hashing method that yields high-quality hash codes to enable efficient image retrieval. Specifically, we employ two sub-networks in DASH, i.e., a master branch and a part branch, to capture global structure features and discriminative feature representations, respectively. Furthermore, we develop an Attention Sampler Module (ASM), which consists of an Object Region Extraction (ORE) block and an Informative Patch Generation (IPG) block, to yield richer informative image patches. The ORE block provides a well-designed multi-scale attentional fusion mechanism to highlight and extract the significant regions of images, and the IPG block employs a direction-specific shift mechanism to generate desired image patches with discriminative details. Both blocks could be seamlessly integrated into various convolutional neural network (CNN) architectures. Subsequently, we conduct knowledge distillation optimization to transfer the details learned by the part branch into the master branch to guide hash code learning. In addition, we design a Weibull quantization loss to minimize the information loss caused by binary quantization. The experimental results on three benchmark datasets demonstrate the effectiveness of the proposed DASH with respect to different evaluation metrics.

Clutter Effect on a Combination of Microwave Imaging and Target Identification Using Ground-Penetrating Radar

This paper presents an experimental investigation of the effect of clutter on the performance of microwave imaging and target identification using ground-penetrating radar (GPR) for buried objects. Signals collected by the GPR were formed to be B-scan and 3D images to carry out the microwave imaging of buried objects. Poles extracted using the short-time matrix pencil method from a late-time response of the received signal collected at a specific position were exploited to identify buried objects. The experiment was set up by constructing a large box filled with sand where an L-shaped metallic sheet was buried. The GPR was installed along with a 2-dimensional scanner above the box to image and identify the buried metallic sheet. In order to investigate the effect of the clutter on the performance of the GPR-based microwave imaging and target identification, small rocks were arranged on the sand surface, and their density was then varied. The experimental results have shown that clutter density significantly affects the sharpness of the B-scan and 3D images. The rocks result in an additional surface layer in the images. Moreover, the results have also shown that the clutter does not affect the performance of the identification of the buried object. From the results, the poles obtained from the GPR scanning inside and outside the buried object region differed. This confirms that the poles of the buried object should be different from those without the object. The poles of the buried object remain constant, although rocks on the sand surface exist since the clutter affects only the early-time response (not the late-time response). With varying clutter densities, the underlying poles were significantly changed, but this does not affect the identification of the buried object.

SiamC Transformer: Siamese coupling swin transformer Multi-Scale semantic segmentation network for vegetation extraction under shadow conditions

The accuracy of fractional vegetation cover (FVC) estimation has important significance in high-precision agricultural and ecological environment assessments. However, the presence ofshadowsunder natural light conditions can cause problems such as poor vegetation distinguishability and blurred boundaries during vegetation segmentation. This leads to missing or over-segmented vegetation inshadedareas, thus reducing the accuracy of FVC. Polarizationinformation can reflect the texture structure features, edgecharacteristics and surface state information, providing important supplementary information to light intensity information. To address the issue of shadows interfering with vegetation segmentation, a Siamese coupling swin transformer (SiamC Transformer) is proposed in this study. In this network, a dual-flow structure is used to simultaneously retrieve the feature information of vegetation degree of linear polarization (DoLP) images and light intensity images simultaneously to obtain multi-dimensional global semantic information of multi-scale vegetation. In the feature fusion stage, two fusion modules are proposed: an adaptive fusion module (AFM) and an adaptive fusion plus module (AFPM), which can fuse shallow spatial information, texture information and deep semantic information. The AFM and AFPM enable the network to better achieve object localization, region activation and edge sharpening, while improving the segmentation accuracy of small objects. The experimental results show that the mean intersection over union (mIoU) of this network is as high as 97.46% on vegetation datasets consisting of light intensity and polarization images, which is better than that of other algorithms. This network has higher accuracy and adaptability for shadow scenes and improves the accuracy of FVC calculation under shadow conditions.

LCW-Net: Low-light-image-based crop and weed segmentation network using attention module in two decoders

Crop segmentation using cameras is commonly used in large agricultural areas, but the time and duration of crop harvesting varies in large farms. Considering this situation, there is a need for low-light image-based segmentation of crop and weed images for late-time harvesting, but no prior research has considered this. As a first study on this topic, we propose a low-light image-based crop and weed segmentation network (LCW-Net) that uses an attention module in two decoders to perform only one step without restoration of low-light images. We also design a loss function to accurately segment regions of objects, crops, and weeds in low-light images to avoid training overfitting and balance the learning task for object, crop, and weed segmentation.There are no existing low-light public databases, and it is difficult to obtain ground truth segmentation information for self-collected database in low-light environments. Therefore, we experimented with converting two public databases, the crop and weed field image dataset (CWFID) and BoniRob dataset, into low-light datasets. The experimental results showed that the mean intersection of unions (mIoUs) of segmentation for crops and weeds were 0.8718 and 0.8693 for the BoniRob dataset, respectively, and 0.8337 and 0.8221 for the CWFID dataset, respectively, indicating that LCW-Net outperforms the state-of-the-art methods.

Research on the Rapid Recognition Method of Electric Bicycles in Elevators Based on Machine Vision

People are gradually coming around to the idea of living a low-carbon lifestyle and using green transportation, and given the severe urban traffic congestion, electric bicycle commuting has taken over as the preferred mode of short-distance transportation for many. Since batteries are used to power electric bicycles, there are no greenhouse gas emissions while they are in use, which is more in line with the requirement for sustainable development around the world. The public has been increasingly concerned about the safety issues brought on by electric bicycles as a result of the industry’s quick development and the rapid increase in the number of electric bicycles worldwide. The unsafe operation of the elevator and the safety of the building have been seriously compromised by the unauthorized admission of electric bicycles into the elevator. To meet the need for fast detection and identification of electric bicycles in elevators, we designed a modified YOLOv5-based identification approach in this study. We propose the use of the EIoU loss function to address the occlusion problem in electric bicycle recognition. By considering the interaction ratio and overlap loss of the target frames, we are able to enhance localization accuracy and reduce the missed detection rate of occluded targets. Additionally, we introduce the CBAM attention mechanism in both the backbone and head of YOLOv5 to improve the expressive power of feature maps. This allows the model to prioritize important regions of the target object, leading to improved detection accuracy. Furthermore, we utilize the CARAFE operator during upsampling instead of the nearest operator in the original model. This enables our model to recover details and side information more accurately, resulting in finer sampling results. The experimental results demonstrate that our improved model achieves an mAP of 86.35 percent, a recall of 81.8 percent, and an accuracy of 88.0 percent. When compared to the original model under the same conditions, our improved YOLOv5 model shows an average detection accuracy increase of 3.49 percent, a recall increase of 5.6 percent, and an accuracy increase of 3.5 percent. Tests in application scenarios demonstrate that after putting the model on the hardware platform Jeston TX2 NX, stable and effective identification of electric bicycles can be accomplished.

A Fusion Underwater Salient Object Detection Based on Multi-Scale Saliency and Spatial Optimization

Underwater images contain abundant information, but many challenges remain for underwater object detection tasks. Various salient object detection methods may encounter low detection precision, and the segmented map has an incomplete region of the target object. To deal with blurry underwater scenes and vague detection problems, a novel fusion underwater salient object detection algorithm (FUSOD) is proposed based on multi-scale saliency and spatial optimization. Firstly, an improved underwater color restoration was utilized to restore the color information for afterward color contrast saliency calculation. Secondly, a more accurate multi-scale fusion saliency map was obtained by fully considering both the global and local feature contrast information. Finally, the fusion saliency was optimized by the proposed spatial optimization method to enhance the spatial coherence. The proposed FUSOD algorithm may process turbid and complex underwater scenes and preserve a complete structure of the target object. Experimental results on the USOD dataset show that the proposed FUSOD algorithm can segment the salient object with a comparatively higher detection precision than the other traditional state-of-the-art algorithms. An ablation experiment showed that the proposed spatial optimization method increases the detection precision by 0.0325 scores in the F-Measure.

Background Instance-Based Copy-Paste Data Augmentation for Object Detection

In supervised deep learning object detection, the quantity of object information and annotation quality in a dataset affect model performance. To augment object detection datasets while maintaining contextual information between objects and backgrounds, we proposed a Background Instance-Based Copy-Paste (BIB-Copy-Paste) data augmentation model. We devised a method to generate background pseudo-labels for all object classes by calculating the similarity between object background features and image region features in Euclidean space. The background classifier, trained with these pseudo-labels, can guide copy-pasting to ensure contextual relevance. Several supervised object detectors were evaluated on the PASCAL VOC 2012 dataset, achieving a 1.1% average improvement in mean average precision. Ablation experiments with the BlitzNet object detector on the PASCAL VOC 2012 dataset showed an improvement of mAP by 1.19% using the proposed method, compared to a 0.18% improvement with random copy-paste. Images from the MS COCO dataset containing objects of the same classes as in PASCAL VOC 2012 were also selected for object pasting experiments. The contextual relevance of pasted objects demonstrated our model’s effectiveness and transferability between datasets with same class of objects.

Drivable Agricultural Road Region Detection Based on Pixel-Level Segmentation with Contextual Representation Augmentation

Drivable area detection is crucial for the autonomous navigation of agricultural robots. However, semi-structured agricultural roads are generally not marked with lanes and their boundaries are ambiguous, which impedes the accurate segmentation of drivable areas and consequently paralyzes the robots. This paper proposes a deep learning network model for realizing high-resolution segmentation of agricultural roads by leveraging contextual representations to augment road objectness. The backbone adopts HRNet to extract high-resolution road features in parallel at multiple scales. To strengthen the relationship between pixels and corresponding object regions, we use object-contextual representations (OCR) to augment the feature representations of pixels. Finally, a differentiable binarization (DB) decision head is used to perform threshold-adaptive segmentation for road boundaries. To quantify the performance of our method, we used an agricultural semi-structured road dataset and conducted experiments. The experimental results show that the mIoU reaches 97.85%, and the Boundary IoU achieves 90.88%. Both the segmentation accuracy and the boundary quality outperform the existing methods, which shows the tailored segmentation networks with contextual representations are beneficial to improving the detection accuracy of the semi-structured drivable areas in agricultural scene.

Unsupervised cycle-consistent adversarial attacks for visual object tracking

Adversarial attacks on visual object tracking have been gaining momentum in the quest to assess and enhance the robustness and security of object tracking models. Most adversarial attack strategies for object tracking rely on fully supervised attacks; that is, they necessitate the use of labels. However, it is not feasible to apply these labeled video object tracking techniques to all attacks. As such, this paper presents an unsupervised attack methodology against visual object tracking models. The approach employs the cycle consistency principle of object tracking models to maximize the inconsistency between forward and backward tracking, thereby providing effective countermeasures. Additionally, this paper introduces a contextual attack method, leveraging the information from the attack object’s region and its surrounding contextual regions. This strategy attacks the object region and its surrounding context regions simultaneously, aiming to decrease its response score to the attack. The proposed attack method is assessed across various types of deep learning-based object trackers. Experimental results on multiple benchmarks reveal that the suggested method yields competitive attack outcomes.

Swin-YOLO for Concealed Object Detection in Millimeter Wave Images

Concealed object detection in millimeter wave (MMW) images has gained significant attention in the realm of public safety, primarily due to its distinctive advantages of non-hazardous and non-contact operation. However, this undertaking confronts substantial challenges in practical applications, owing to the inherent limitations of low imaging resolution, small concealed object size, intricate environmental noise, and the need for real-time performance. In this study, we propose Swin-YOLO, an innovative single-stage detection model built upon transformer layers. Our approach encompasses several key contributions. Firstly, the integration of Local Perception Swin Transform Layers (LPST Layers) enhanced the network’s capability to acquire contextual information and local awareness. Secondly, we introduced a novel feature fusion layer and a specialized prediction head for detecting small targets, effectively leveraging the network’s shallow feature information. Lastly, a coordinate attention (CA) module was seamlessly incorporated between the neck network and the detection head, augmenting the network’s sensitivity towards critical regions of small objects. To validate the efficacy and feasibility of our proposed method, we created a new MMW dataset containing a large number of small concealed objects and conducted comprehensive experiments to evaluate the effectiveness of overall and partial improvements, as well as computational efficiency. The results demonstrated a remarkable 4.7% improvement in the mean Average Precision (mAP) for Swin-YOLO compared with the YOLOv5 baseline. Moreover, when compared with other enhanced transformer-based models, Swin-YOLO exhibited a superior accuracy and the fastest inference speed. The proposed model showcases enhanced performance and holds promise for advancing the capabilities of real-world applications in public safety domains.

Land Registration Derived from Customary Land According to Government Regulation Number 24 of 1997

The purpose of this study is to determine the procedure for registering land derived from customary land according to Government Regulation Number 24 of 1997. The results show that the procedure for registering land derived from customary land includes the activity of initial registration, which is the registration of land objects that have not been registered based on Government Regulation Number 10 of 1961, as amended by Government Regulation Number 24 of 1997. This registration is carried out systematically and sporadically with certain requirements that must be fulfilled by the applicant. These requirements include a letter of application, a photocopy of the applicant's identification card, a photocopy of the Land and Building Tax, written evidence of the original land accompanied by a certificate of physical possession and a certificate from the village head. Once all the required data is complete and meets the criteria, a certificate of rights over customary land is issued.

Autonomously Tuning Multilayer Thermal Cloak with Variable Thermal Conductivity Based on Thermal Triggered Dual Phase-Transition Metamaterial

Thermal cloaks offer the potential to conceal internal objects from detection or to prevent thermal shock by controlling external heat flow. However, most conventional natural materials lack the desired flexibility and versatility required for on-demand thermal manipulation. We propose a solution in the form of homogeneous multilayer thermodynamic cloaks. Through an ingenious design, these cloaks achieve exceptional and extreme parameters, enabling the distribution of multiple materials in space. We first investigate the effects of important design parameters on the thermal shielding effectiveness of conventional thermal cloaks. Subsequently, we introduce an autonomous tuning function for the thermodynamic cloak, accomplished by leveraging two phase transition materials as thermal conductive layers. Remarkably, this tuning function does not require any energy input. Finite element analysis results demonstrate a significant reduction in the temperature gradient inside the thermal cloak compared to the surrounding background. This reduction indicates the cloak’s remarkable ability to manipulate the spatial thermal field. Furthermore, the utilization of materials undergoing phase transition leads to an increase in thermal conductivity, enabling the cloak to achieve the opposite variation of the temperature field between the object region and the background. This means that, while the temperature gradient within the cloak decreases, the temperature gradient in the background increases. This work addresses a compelling and crucial challenge in the realm of thermal metamaterials, i.e., autonomous tuning of the thermal field without energy input. Such an achievement is currently unattainable with existing natural materials. This study establishes the groundwork for the application of thermal metamaterials in thermodynamic cloaks, with potential extensions into thermal energy harvesting, thermal camouflage, and thermoelectric conversion devices. By harnessing phonons, our findings provide an unprecedented and practical approach to flexibly implementing thermal cloaks and manipulating heat flow.

Online object tracking based interactive attention

By embedding Transformer into the Siamese tracking framework, some Transformer-based Siamese tracking network are proposed, such as TransT and SwinTrack. Nevertheless, the existing Transformer-based Siamese tracking networks do not fully utilize the object information of template branch, and their position encoding methods cannot accurately perceive the object position. Aiming at these problems, a novel Transformer-based Siamese tracking network, which includes feature extraction, feature fusion and prediction head, is proposed in this work. Firstly, an interactive attention calculation module for template branch and search branch is designed to enhance the feature extraction capability of the network for the object region and suppress the background interference. In addition, to address the problem that the existing Transformer-based feature fusion network is not sensitive enough to the object location region, a position encoding method that characterizes the relative distance is proposed to enhance the perception ability of the object location and reduce network parameters. Then, the contrastive loss is introduced to enhance the discriminative ability of the classification layer between foreground and background, and to effectively deal with the interference of similar objects in the background. Extensive experiments with state-of-the-art trackers are carried out on four challenging visual object tracking benchmarks: GOT-10k, LaSOText, TLP, and TrackingNet. Experimental results demonstrate the proposed method is more robust on multiple challenges and can achieve considerable performances with AO of 70.1% on GOT-10k, SUC score of 45.5% on LaSOT ext, 56.9% on TLP, and 79.7% on TrackingNet datasets. It runs faster (7.1G MACs) and occupies less memory (21M), making it more suitable for practical applications.

Steel Surface Defect Detection Based on SSAM-YOLO

The defect inspection of the steel surface is crucial to modern manufacturing and highly depends on inefficient manual work. The emergence of deep learning has prompted the development of automated defect detection methods, but the current methods perform badly in the detection of the crazing and rolled-in scale-two types of defects on steel surfaces. The difficulty in the detection of crazing and rolled-in scale is mainly due to the similarity between object regions and background regions. Based on this, the authors propose a supervised spatial-attention module (SSAM). It introduces a priori knowledge compared to the traditional spatial attention mechanism, which can enhance the supervision of relevant parameters in the attention mechanism module during network training. Finally, they introduced the SSAM to the YOLOv5 and got the SSAM-YOLO. The test result on the NEU-DET dataset shows that the proposed method has better detection accuracy, achieving improvements of 7.3% and 3.02% on the AP@0.5 for the crazing and rolled-in scale. The method also outperforms the comparative main stream algorithms for steel surface defect detection, verifying the effectiveness of our algorithm.

Collision-constrained deformable image registration framework for discontinuity management.

Topological changes like sliding motion, sources and sinks are a significant challenge in image registration. This work proposes the use of the alternating direction method of multipliers as a general framework for constraining the registration of separate objects with individual deformation fields from overlapping in image registration. This constraint is enforced by introducing a collision detection algorithm from the field of computer graphics which results in a robust divide and conquer optimization strategy using Free-Form Deformations. A series of experiments demonstrate that the proposed framework performs superior with regards to the combination of intersection prevention and image registration including synthetic examples containing complex displacement patterns. The results show compliance with the non-intersection constraints while simultaneously preventing a decrease in registration accuracy. Furthermore, the application of the proposed algorithm to the DIR-Lab data set demonstrates that the framework generalizes to real data by validating it on a lung registration problem.

Feature selection based on the self-calibration of binocular camera extrinsic parameters

The accuracy of feature-based vision algorithms, including the self-calibration of binocular camera extrinsic parameters used in autonomous driving environment perception techniques relies heavily on the quality of the features extracted from the images. This study investigates the influence of the depth distance between objects and the camera, the feature points in different object regions, and the feature points in dynamic object regions on the self-calibration of binocular camera extrinsic parameters. To achieve this, the study first filters out different types of objects in the image through semantic segmentation. Then, it identifies the areas of dynamic objects and extracts the feature points in the static object region for the self-calibration of binocular camera extrinsic parameters. By calculating the baseline error of the binocular camera and the row alignment error of the matching feature points, this study evaluates the influence of feature points in dynamic object regions, feature points in different object regions, and feature points at different distances on the self-calibration algorithm. The experimental results demonstrate that feature points at static objects close to the camera are beneficial for the self-calibration of extrinsic parameters of binocular camera.

Material Characterization of Mayapán’s Effigy Censers’ Sherds

Ceramic production from ancient Mesoamerican civilizations is related with cultural and technological evolution processes. Studying ritual objects also provides information on ancient traditions and allows researchers to determine the importance of certain materials employed in its manufacture. In this work, a set of 72 of Mayapán’s effigy censers’ sherds was analyzed in situ by using a combination of non-invasive, non-destructive spectroscopic and imaging techniques for material characterization; colorimetry established an initial classification of the pigments present in the objects, XRF provided elemental information, FORS allowed us to describe the molecular characteristics, and hyperspectral imaging established compositional contrasts or similitudes between large regions of the different objects. Pigments were characterized, allowing us to describe the materials used in the decorations of such ritual objects. The pottery matrix was also characterized, leading to a detailed description of the clays and mixtures of minerals employed in the construction of the effigy censers.

Learning class-agnostic masks with cross-task refinement for weakly supervised semantic segmentation

Weakly supervised semantic segmentation (WSSS) commonly relies on Class Activation Mapping (CAM) to produce pseudo semantic labels using image-level annotations. However, because CAM maps often form sparse object regions with poor boundaries, they cannot provide sufficient segmentation supervision. Because off-the-shelf saliency maps can provide rich object boundaries that can be leveraged to improve semantic segmentation, we propose to jointly learn semantic segmentation and class-agnostic masks by using image-level annotations and off-the-shelf saliency maps as supervision. We also propose a cross-task label refinement mechanism, which takes advantage of the learned class-agnostic masks and semantic segmentation masks, to refine the pseudo labels and provide more accurate supervision to both tasks. Moreover, we introduce a new normalization method for CAM to generate more complete class-specific localization maps. The improved CAM maps complement our learned class-agnostic masks, leading to high-quality pseudo semantic segmentation labels. Extensive experiments demonstrate the effectiveness of the proposed approach, with state-of-the-art WSSS results established on PASCAL VOC 2012 and MS COCO.