Appearance Variations Research Articles

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

Lightweight detection of small tools for safer construction

Construction sites present significant potential safety hazards to the workers, with hand tools being a major source of injuries. This paper presents a Lightweight approach for Small Tools Detection (LSTD) method using a deep neural network for real-time detection of small construction tools. LSTD utilizes a lightweight backbone with Dynamic Feature Extraction, Accurate Separated Head, and Integrated Feature Fusion, reducing parameters by 73% and computations by 28% versus YOLOv5 while achieving 87.3% mean Average Precision (mAP) on challenging construction site datasets. Additional modules enhance detection recall and robustness to appearance variation and scale changes. Extensive experiments demonstrate LSTD's superior performance in misty conditions and illumination changes. With high accuracy in a compact 2.87 M parameter network, LSTD brings ubiquitous worker safety monitoring via edge devices closer to reality. The proposed model marks a significant advancement in improving safety in high-risk construction environments.

Online discrimination-correction subnet for background suppression and dynamic template updating in Siamese visual tracking

The offline Siamese tracking methods have recently attracted much attention due to their impressive performance on speed and accuracy. However, they ignore the target and background information during tracking, thus lacking sufficient discrimination ability to handle complex background interference, and cannot robustly adapt to sharp appearance variations in challenging situations, such as scale variation, rotation, occlusion, and background clutter. To address the drawbacks above, we propose an online discrimination-correction subnet, which discriminates and corrects the tracking results obtained by the Siamese tracker through online learning, and further guides the template update. Firstly, a feature refinement module is designed to enhance the discriminative ability of features, which compresses and combines target features based on the importance of feature channels to obtain target-specific features. On this basis, an online discriminator module is proposed to learn the decision boundary between target and background through samples collected online to suppress background interference. In addition, a sample equilibrium strategy is proposed to alleviate the imbalance in the distribution of positive and negative samples during online training. Finally, to handle sharp target appearance variations, a robust template updating strategy is proposed to screen high-quality historical samples to generate dynamic templates. To evaluate the generalization ability and effectiveness of our method, we apply it to four classical Siamese trackers, namely SiamFC, SiamRPN++, SiamCAR, and TransT. Extensive experiments on seven challenging benchmarks demonstrate that our method significantly improves the baseline methods and achieves outstanding performance compared with state-of-the-art trackers.

Insight into the changes in active metabolite profiles of noni (Morinda citrifolia L.) fruit subjected to different drying treatments

Noni fruit is renowned for its abundance of bioactive compounds. Drying is an important method for processing functional products derived from noni. However, limited information exists on how drying methods affect the active metabolite profiles of noni fruit. This study investigated the impact of four common drying methods, including hot-air drying (HAD), vacuum freeze drying (VFD), microwave drying (MWD), and far infrared drying (FID), on the physicochemical indexes, bioactive components, and functional properties of dried noni fruit slices using targeted and untargeted metabonomics analysis. The results showed significant variations in appearance, water migration, and microstructure of dried noni fruit slices subjected to the four drying methods. VFD treatment yielded better dried noni fruit products when compared to other drying methods. The superiority of VFD treatment was due to its uniform stratification, reduced collapse, better retention of bioactive components and antioxidants, and higher enzyme inhibitory rates. These findings suggest that VFD method is ideal for obtaining premium bioactive profiles and maintaining the biological activity of noni fruit.

NeRF-Art: Text-Driven Neural Radiance Fields Stylization.

As a powerful representation of 3D scenes, the neural radiance field (NeRF) enables high-quality novel view synthesis from multi-view images. Stylizing NeRF, however, remains challenging, especially in simulating a text-guided style with both the appearance and the geometry altered simultaneously. In this paper, we present NeRF-Art, a text-guided NeRF stylization approach that manipulates the style of a pre-trained NeRF model with a simple text prompt. Unlike previous approaches that either lack sufficient geometry deformations and texture details or require meshes to guide the stylization, our method can shift a 3D scene to the target style characterized by desired geometry and appearance variations without any mesh guidance. This is achieved by introducing a novel global-local contrastive learning strategy, combined with the directional constraint to simultaneously control both the trajectory and the strength of the target style. Moreover, we adopt a weight regularization method to effectively suppress cloudy artifacts and geometry noises which arise easily when the density field is transformed during geometry stylization. Through extensive experiments on various styles, we demonstrate that our method is effective and robust regarding both single-view stylization quality and cross-view consistency.

Pest-ConFormer: A hybrid CNN-Transformer architecture for large-scale multi-class crop pest recognition

Crop pests are acknowledged as the major factors in reducing the yield and quality of agricultural production worldwide. It is an urgent necessity to recognize crop pests accurately to protect the crop in the early stage to reduce the loss for the agricultural economy. Due to the ecological characteristics of the crop pests and the complex background in fields, the crop pests show high inter-class similarity and significant intra-class variation in external morphology appearance, which makes current recognition methods suffer from low classification accuracy and poor generalization ability in complex natural environment recognition tasks. To tackle this problem, a hybrid convolutional neural network and transformer-based model, namely Pest-ConFormer, featured with multi-scale weakly supervised feature selection mechanisms is proposed, which has shown excellent multi-scale discriminative feature extraction in fine-grained visual classification (FGVC) tasks. This method employs a hybrid convolution-transformer encoder architecture pre-training in a self-supervised masked autoencoder manner as a backbone to learn pests’ highly discriminative features across various scales. Next, a dual-path feature aggregation structure with a top-down FPN-like feature pathway and a bottom-up PANet-like feature pathway based on attention mechanisms is designed to learn high-level global context information and low-level local detailed feature representation. Thirdly, a fine-grained classification module using weakly supervised learning is introduced to select the discriminative feature points in different pyramidal levels. Then, these feature points are fed into a graph convolutional network to accomplish classification. Several experiments are conducted on the large-scale multi-class IP102 benchmark dataset, and the proposed method achieves an accuracy of 77.81 % regarding crop pest recognition. The experimental results indicate that our approach outperforms other state-of-the-art methods by nearly 2 percent points, demonstrating that the proposed hybrid architecture with dual-path feature aggregation and fine-grained classification modules can be more effective in the crop pest recognition field than CNN-based methods and can be deployed in the practical natural environment. The source code will be available at https://github.com/mwfang/pestconformer.

Attentive multi-granularity perception network for person search

Person search is an extremely challenging task that seeks to identify individuals through joint person detection and person re-identification from uncropped real scene images. Previous studies primarily focus on learning rich features to enhance identification. However, arbitrary feature enhancement strategies may introduce unwanted background noise. Moreover, different scenarios usually exhibit varying pedestrian appearances or even intricate occlusions, leading to inconsistent/incomplete pedestrian features in different images. In this paper, we introduce a novel Attentive Multi-granularity Perception (AMP) module seamlessly integrated into our AMPN network. This module specifically addresses appearance variations and occlusions within a person's Region of Interest (RoI). The AMP module harnesses discriminative relationship features from various local regions, significantly enhancing identification accuracy. It comprises two principal components: the Pedestrian Perception Enhancement (PPE) block and the Background Interference Suppressor (BIS). The PPE block introduces a Spatial-wise Feature Mixer and a Channel-wise Feature Mixer, which effectively capture and refine discriminative relation features. Simultaneously, the BIS operates in parallel with the PPE block, enriching the discriminative relation features and enhancing the distinctiveness between the foreground and background. Our AMP module is plug-and-play and can integrate with other person search models. Extensive experiments validate our model's merits, achieving state-of-the-art performance on CUHK-SYSU and a 4.8% mAP gain over SeqNet on PRW at a desirable speed. Our code is accessible at https://github.com/zqx951102/AMPN.

Adaptive Feature Refinement and Weighted Similarity for Deep Loop Closure Detection in Appearance Variation

Adaptive Feature Refinement and Weighted Similarity for Deep Loop Closure Detection in Appearance Variation

Correlation filter based single object tracking: A review

In recent years, correlation filter-based (CF) tracking algorithms have gained momentum in the field of visual tracking. CF tracking algorithms have achieved compelling performance by addressing its limitations such as boundary effect and filter corruption during various tracking and target appearance variations. Many researchers have attempted to provide better efficiency and tracking results by extracting handcrafted and deep features either from vision sensors or specialized sensors in the CF tracking framework. Handcrafted features are integrated with deep features, thermal features, and depth features to prevent tracking failures during dense tracking challenges. To provide a detailed understanding of CF-based trackers, the tracking algorithms are categorized either as kernelized CF trackers or fusion-based CF trackers in this work. This is the first review of its kind, which categorizes various correlation-based tracking algorithms based on the key methodologies and the exploited features in the appearance model. Under each category, salient features, detailed overview, and current advancement are discussed and tabulated to provide future research directions. In addition, the performance of CF-based state-of-the-art is experimentally evaluated on multiple datasets namely, OTB100 and VOT2017 under tough tracking situations.

Self-Supervised Tracking via Target-Aware Data Synthesis.

While deep-learning-based tracking methods have achieved substantial progress, they entail large-scale and high-quality annotated data for sufficient training. To eliminate expensive and exhaustive annotation, we study self-supervised (SS) learning for visual tracking. In this work, we develop the crop-transform-paste operation, which is able to synthesize sufficient training data by simulating various appearance variations during tracking, including appearance variations of objects and background interference. Since the target state is known in all synthesized data, existing deep trackers can be trained in routine ways using the synthesized data without human annotation. The proposed target-aware data-synthesis method adapts existing tracking approaches within a SS learning framework without algorithmic changes. Thus, the proposed SS learning mechanism can be seamlessly integrated into existing tracking frameworks to perform training. Extensive experiments show that our method: 1) achieves favorable performance against supervised (Su) learning schemes under the cases with limited annotations; 2) helps deal with various tracking challenges such as object deformation, occlusion (OCC), or background clutter (BC) due to its manipulability; 3) performs favorably against the state-of-the-art unsupervised tracking methods; and 4) boosts the performance of various state-of-the-art Su learning frameworks, including SiamRPN++, DiMP, and TransT.

Body image and psychosocial outcomes in youth and young adults with differences of sex development: a multi-method study.

Differences of sex development (DSD) can affect the physical health, appearance, and psychosocial functioning of affected individuals, but little is known about how subjective appearance perceptions (body image) impact psychosocial outcomes. This study evaluated body image and its associations with psychosocial outcomes including quality of life, resilience, and psychosocial adjustment. This cross-sectional, multi-method study assessed body image and psychosocial outcomes including quality of life, adjustment, and resilience in 97 youth and young adults with DSD (mean age = 17 ± 3.7 years; 56% assigned female in infancy) using psychometrically sound instruments. A subsample (n = 40) completed qualitative interviews. Quantitative results indicated that overall, participants were satisfied with their physical appearance, although less so with their primary sex characteristics. Body image dissatisfaction was associated with poorer psychosocial adjustment, quality of life, and resilience. Qualitatively, youth and young adults reported a variety of perceptions, both positive and negative, related to their body image and the impact of living with a DSD condition. Themes identified included appearance management; effects of DSD on body image; diagnostic factors and features; attitudes about diagnosis; and treatment. Body image is significantly associated with psychosocial outcomes in youth and young adults with DSD, with qualitative findings highlighting both positive and negative body image experiences. Results have implications for clinical care including screening for appearance concerns, normalization of appearance variations, and intervention development to better support healthy body image and psychosocial functioning in youth and young adults with DSD.

Triangulated Irregular Network based Seamline Determination for Fast Image Stitching of Multiple UAV Images

Abstract. To maintain the efficiency of UAV (unmanned aerial vehicle) remote sensing, rapid image stitching is essential to make multiple UAV images into a seamless mosaic image. Relief displacement introduces variations in object appearance for each image, causing mismatch errors at mosaic seamlines. Traditional approaches involve orthorectifying images using DSMs (digital surface models). While these approaches allow for accurate image stitching, they do not cope with the advantages of UAVs due to their time consumption. In contrast, fast image stitching techniques that do not use orthorectification are well suited for UAV image processing. Related researches have attempted to optimize seamlines to eliminate the errors caused by relief displacement without the use of DSMs. We propose to utilize a TIN (triangular irregular network) of tiepoints to effectively eliminate errors caused by relief displacement while maintaining the fast speed of image stitching. In this study, a TIN is constructed based image tiepoints whose ground coordinates have been obtained through bundle adjustment. The edges of the TIN are used to generate seamlines for image stitching, and the facets of the TIN are used to select minimal images for image stitching and to optimize seamlines. Image stitching results of our proposed method had small error of 1–2 pixels and the processing time of less than 10 minutes for 97 UAV images. This study showed that the proposed method could stitch multiple images while maintaining stable quality using only geometric clues of a TIN of tiepoints.

Multimodal fine-grained grocery product recognition using image and OCR text

Automatic recognition of grocery products can be used to improve customer flow at checkouts and reduce labor costs and store losses. Product recognition is, however, a challenging task for machine learning-based solutions due to the large number of products and their variations in appearance. In this work, we tackle the challenge of fine-grained product recognition by first extracting a large dataset from a grocery store containing products that are only differentiable by subtle details. Then, we propose a multimodal product recognition approach that uses product images with extracted OCR text from packages to improve fine-grained recognition of grocery products. We evaluate several image and text models separately and then combine them using different multimodal models of varying complexities. The results show that image and textual information complement each other in multimodal models and enable a classifier with greater recognition performance than unimodal models, especially when the number of training samples is limited. Therefore, this approach is suitable for many different scenarios in which product recognition is used to further improve recognition performance. The dataset can be found at https://github.com/Tubbias/finegrainocr.

Preliminary evidence that eye appearance in parrots (Psittaciformes) co-varies with latitude and altitude

External eye appearance in avian taxa has been proposed to be driven by social and ecological functions. Recent research in primates suggests, instead, that, photoprotective functions are important drivers of external eye appearance. Using similar methods, we examined the variation in external eye appearance of 132 parrot species (Psittaciformes) in relation to their ecology and sociality. Breeding systems, flock size and sexual dimorphism, as well as species’ latitude and maximum living altitude, and estimated UV-B incidence in species’ ranges were used to explore the contribution of social and ecological factors in driving external eye appearance. We measured the hue and brightness of visible parts of the eye and the difference in measurements of brightness between adjacent parts of the eye. We found no link between social variables and our measurements. We did, however, find a negative association between the brightness of the inner part of the iris and latitude and altitude. Darker inner irises were more prevalent farther away from the equator and for those species living at higher altitudes. We found no link between UV-B and brightness measurements of the iris, or tissue surrounding the eye. We speculate that these results are consistent with an adaptation for visual functions. While preliminary, these results suggest that external eye appearance in parrots is influenced by ecological, but not social factors.

CAC-YOLOv8: real-time bearing defect detection based on channel attenuation and expanded receptive field strategy

Bearing defect detection plays a crucial role in the intelligent production of chemical transmission equipment, where timely identification and handling of defective bearings are essential. However, in practical large-scale industrial production, product surface defects are often complex, diverse, and exhibit significant variations in appearance, posing severe challenges to the discriminative ability and detection efficiency of bearing defect detection algorithms. This paper proposes a real-time bearing surface defect detection algorithm, CAC-YOLOv8, which designs the Channel Attenuation Network (CAN) and Compound Pooling Pyramid Spatial Pyramid Pooling Fast (CPPSPPF) structure. Specifically, the model introduces the Channel Attenuation Network to achieve parallel feature extraction, deep feature processing, and feature fusion under different channel numbers, capturing critical features related to bearing defects and thereby improving the inference speed. Subsequently, based on the concept of overlapped receptive fields, a CPPSPPF structure is constructed, utilizing multiple iterations of max-pooling operations with smaller pooling kernel sizes to prevent information loss while expanding the receptive field, thereby strengthening the capturing ability of features at different scales. The experimental results indicate that the proposed CAC-YOLOv8 bearing surface defect detection algorithm, compared to the YOLOv8 model, achieved a 0.3% improvement in mAP@0.5, reduced model size by 14.4%, and enhanced model inference speed by 33.3%. This enables the CAC-YOLOv8 model to significantly improve the real-time performance of bearing defect detection while maintaining high-precision detection. The performance in practical industrial detection demonstrates that the proposed approach has achieved outstanding results in both speed and accuracy.

Multi-granularity tooth analysis via YOLO-based object detection models for effective tooth detection and classification

Accurate detection and classification of teeth is the first step in dental disease diagnosis. However, the same class of tooth exhibits significant variations in surface appearance. Moreover, the complex geometrical structure poses challenges in learning discriminative features among the different tooth classes. Due to these complex features, tooth classification is one of the challenging research domains in deep learning. To address the aforementioned issues, the presented study proposes discriminative local feature extraction at different granular levels using YOLO models. However, this necessitates a granular intra-oral image dataset. To facilitate this requirement, a dataset at three granular levels (two, four, and seven teeth classes) is developed. YOLOv5, YOLOv6, and YOLOv7 models were trained using 2,790 images. The results indicate superior performance of YOLOv6 for two-class classification problems. The model generated a mean average precision (mAP) value of 94%. However, as the granularity level is increased, the performance of YOLO models decreases. For, four and seven-class classification problems, the highest mAP value of 87% and 79% was achieved by YOLOv5 respectively. The results indicate that different levels of granularity play an important role in tooth detection and classification. The YOLO’s performance gradually decreased as the granularity decreased especially at the finest granular level.

Enhanced-YOLOv8: A new small target detection model

Small target detection is a very difficult task and remains one of the most challenging problems in computer vision due to variations in object shape, appearance and position, as well as the effects of lighting and occlusion during imaging. To improve the accuracy of the small target detection results, we propose a new small target detection model, Enhanced-YOLOv8. Firstly, a small target detection level (STDL) is added to the original effective feature layer of YOLOv8, which not only provides richer semantic information but also get more accurate target localization and bounding box accuracy. When detecting small targets, the detection accuracy can be improved by more detailed information. Then, the fusion convolutional block attention module (FCBAM) is proposed by introducing the position attention module (PAM) based on the traditional CBAM. FCBAM not only can adaptively select and fuse the most representative features, but also can better capture the image important features at different positions in the image and enhance spatial detail perception. Finally, semantic fusion network (SFN) is proposed on the basis of residual network, which introduces semantic information of high-layer feature into low-layer feature. It can adaptively guide the fusion of high-layer feature and low-layer feature to reduce the loss of feature information. After experimental verification, the Enhanced-YOLOv8 proposed improves the accuracy of the detection results.

Traffic Sign Classification Using CNN

This study presents a novel approach for traffic sign classification leveraging Convolutional Neural Networks (CNNs). With the proliferation of autonomous vehicles and advanced driver assistance systems, accurate and efficient traffic sign recognition is imperative for safe and efficient navigation. The proposed CNN-based method utilizes deep learning techniques to automatically extract discriminative features from traffic sign images, enabling robust classification across diverse environmental conditions and variations in sign appearance. Experimental results demonstrate the effectiveness of the proposed approach. This research contributes to the advancement of intelligent transportation systems by providing a reliable and scalable solution for real-time traffic sign classification. Our findings highlight the potential of deep learning for robust and accurate traffic sign classification in real-world scenarios.

Convolutional MLP orthogonal fusion of multiscale features for visual place recognition

Visual place recognition (VPR) involves obtaining robust image descriptors to cope with differences in camera viewpoints and drastic external environment changes. Utilizing multiscale features improves the robustness of image descriptors; however, existing methods neither exploit the multiscale features generated during feature extraction nor consider the feature redundancy problem when fusing multiscale information when image descriptors are enhanced. We propose a novel encoding strategy—convolutional multilayer perceptron orthogonal fusion of multiscale features (ConvMLP-OFMS)—for VPR. A ConvMLP is used to obtain robust and generalized global image descriptors and the multiscale features generated during feature extraction are used to enhance the global descriptors to cope with changes in the environment and viewpoints. Additionally, an attention mechanism is used to eliminate noise and redundant information. Compared to traditional methods that use tensor splicing for feature fusion, we introduced matrix orthogonal decomposition to eliminate redundant information. Experiments demonstrated that the proposed architecture outperformed NetVLAD, CosPlace, ConvAP, and other methods. On the Pittsburgh and MSLS datasets, which contained significant viewpoint and illumination variations, our method achieved 92.5% and 86.5% Recall@1, respectively. We also achieved good performances—80.6% and 43.2%—on the SPED and NordLand datasets, respectively, which have more extreme illumination and appearance variations.

A Learnable Viewpoint Evolution Method for Accurate Pose Estimation of Complex Assembled Product

Balancing adaptability, reliability, and accuracy in vision technology has always been a major bottleneck limiting its application in appearance assurance for complex objects in high-end equipment production. Data-driven deep learning shows robustness to feature diversity but is limited by interpretability and accuracy. The traditional vision scheme is reliable and can achieve high accuracy, but its adaptability is insufficient. The deeper reason is the lack of appropriate architecture and integration strategies between the learning paradigm and empirical design. To this end, a learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex assembled products under free view is proposed. To alleviate the balance problem of exploration and optimization in estimation, shape-constrained virtual–real matching, evolvable feasible region, and specialized population migration and reproduction strategies are designed. Furthermore, a learnable evolution control mechanism is proposed, which integrates a guided model based on experience and is cyclic-trained with automatically generated effective trajectories to improve the evolution process. Compared to the 1.69°,55.67 mm of the state-of-the-art data-driven method and the 1.28°,77.67 mm of the classic strategy combination, the pose estimation error of complex assembled product in this study is 0.23°,23.71 mm, which proves the effectiveness of the proposed method. Meanwhile, through in-depth exploration, the robustness, parameter sensitivity, and adaptability to the virtual–real appearance variations are sequentially verified.