Target Instances Research Articles

SGW-YOLOv8n: An Improved YOLOv8n-Based Model for Apple Detection and Segmentation in Complex Orchard Environments

This study addresses the problem of detecting occluded apples in complex unstructured environments in orchards and proposes an apple detection and segmentation model based on improved YOLOv8n-SGW-YOLOv8n. The model improves apple detection and segmentation by combining the SPD-Conv convolution module, the GAM global attention mechanism, and the Wise-IoU loss function, which enhances the accuracy and robustness. The SPD-Conv module preserves fine-grained features in the image by converting spatial information into channel information, which is particularly suitable for small target detection. The GAM global attention mechanism enhances the recognition of occluded targets by strengthening the feature representation of channel and spatial dimensions. The Wise-IoU loss function further optimises the regression accuracy of the target frame. Finally, the pre-prepared dataset is used for model training and validation. The results show that the SGW-YOLOv8n model significantly improves relative to the original YOLOv8n in target detection and instance segmentation tasks, especially in occlusion scenes. The model improves the detection mAP to 75.9% and the segmentation mAP to 75.7% and maintains a processing speed of 44.37 FPS, which can meet the real-time requirements, providing effective technical support for the detection and segmentation of fruits in complex unstructured environments for fruit harvesting robots.

Comparative analysis of hybrid cognitive radar techniques for enhanced target detection and tracking: A performance evaluation perspective

This paper introduces an innovative hybrid algorithmic approach for cognitive radar systems, by integrating unique machine learning optimization techniques such as, YOLO (You Only Look Once), Mask R-CNN, and Recurrent Neural Networks (RNNs). Through extensive simulations, the integrated approach demonstrates notable enhancements in target detection, instance segmentation, and target tracking within radar systems. Leveraging deep learning models, the framework facilitates adaptive and intelligent processing of radar data, augmenting system performance in dynamic environments. By seamlessly integrating state-of-the-art techniques, the proposed framework showcases a comprehensive solution to the challenges faced by traditional radar systems. This research represents a significant stride forward in radar technology, promising transformative impacts across various domains, including surveillance, remote sensing, and autonomous navigation. As radar systems continue to evolve, the adoption of advanced deep learning techniques offers unprecedented opportunities for enhancing situational awareness and decision-making capabilities in complex operational scenarios.

Temporal relation transformer for robust visual tracking with dual-memory learning

Recently, transformer trackers mostly associate multiple reference images with the search area to adapt to the changing appearance of the target. However, they ignore the learned cross-relations between the target and surrounding, leading to difficulties in building coherent contextual models for specific target instances. This paper presents a Temporal Relation Transformer Tracker (TRTT) for robust visual tracking, providing a concise approach to modeling temporal relations by dual target memory learning. Specifically, a temporal relation transformer network generates paired memories based on static and dynamic templates, which are reinforced interactively. The memory contains implicit relation hints that capture the relations between the tracked object and its immediate surroundings. More importantly, to ensure consistency of target instance identities between frames, the relation hints from previous frames are transferred to the current frame for merging temporal contextual attention. Our method also incorporates mechanisms for reusing favorable cross-relations and instance-specific features, thereby overcoming background interference in complex spatio-temporal interactions through a sequential constraint. Furthermore, we design a memory token sparsification method that leverages the key points of the target to eliminate interferences and optimize attention calculations. Extensive experiments demonstrate that our method surpasses advanced trackers on 8 challenging benchmarks while maintaining real-time running speed.

A novel method for tomato stem diameter measurement based on improved YOLOv8-seg and RGB-D data

The automatic acquisition of crop information can promote the rapid development of precision agriculture. Tomatoes, as a representative greenhouse crop, can demonstrate their growth status and overall health could be exhibited by measuring the diameters of their main stems. Automated measurement of the diameters of the trunk stems can not only reduce labor costs, but also enhance efficiency, thereby facilitating the improvement of tomato cultivation and planting management. Therefore, this study proposes a novel method based on an instance segmentation algorithm combined with RGB-D data to measure the diameter of a tomato trunk. First, we utilize the improved YOLOv8-seg to acquire the masks and bounding boxes of the buds and stems. Namely, we replace the SPPF module with Soft-SPPF to enhance the model’s capability to extract multi-scale features. Additionally, we improve the neck layer using cross-stage and weighted feature fusion to enhance the feature fusion effect. Then, we design a method to calculate the diameter of the tomato trunk by utilizing the instance segmentation results and depth information. Specifically, we first obtain the measurement point and the ROI (Region of Interest) to be measured by identifying the intersection between the bounding box of the bud and the straight line fitted by the stem mask in an image. We then filter out irrelevant depth information using the mask within the ROI, optimize the coordinate values of the measurement point, and calculate the main stem diameter. The results demonstrate that the measurements obtained in this study have a RMSE (Root Mean Square Error) of 1.5 mm and a MAPE (Mean Absolute Percentage Error) of 12.37 % compared with the manual measurements. Compared with the method based on object detection, the direct acquisition of contour information for the target instances by the instance segmentation algorithm reduces the algorithmic complexity of the subsequent processing. The proposed method can offer precise and reliable information on the diameter of the main stem of a tomato in real-world scenarios. It can be extended to other types of crops in greenhouses.

Progressive Decision Boundary Shifting for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) is attracting more attention from researchers for boosting the task-specific generalization on target domain. It focuses on addressing the domain shift between the labeled source domain and the unlabeled target domain. Recent biclassifier-based UDA models perform category-level alignment to reduce domain shift, and meanwhile, self-training is used for improving the discriminability of target instances. However, the error accumulation problem of instances with high semantic uncertainty may cause discriminability degradation and category-level misalignment. To solve this issue, we design the progressive decision boundary shifting algorithm, where stable category information of target instances is explored for learning a discriminability structure on target domain. Specifically, we first model the semantic uncertainty of instances by progressively shifting decision boundaries of category. Then, we introduce the uncertainty decoupling in a contrastive manner, where the discriminative information is learned from the source domain for instance with low semantic uncertainty. Furthermore, we minimize the predictive entropy of instances with high semantic uncertainty to reduce their prediction confidence. Extensive experiments on three popular datasets show that our model outperforms the current state-of-the-art (SOTA) UDA methods.

Label distribution feature selection based on neighborhood rough set

SummaryIn label distribution learning (LDL), an instance is involved with many labels in different importance degrees, and the feature space of instances is accompanied with thousands of redundant and/or irrelevant features. Therefore, the main characteristic of feature selection in LDL is to evaluate the ability of each feature. Motivated by neighborhood rough set (NRS), which can be used to measure the dependency degree of feature via constructing neighborhood relations on feature space and label space, respectively, this article proposes a novel label distribution feature selection method. In this article, the neighborhood class of instance in label distribution space is defined, which is beneficial to recognize the logical class of target instance. Then, a new NRS model for LDL is proposed. Specially, the dependency degree of feature combining label weight is defined. Finally, a label distribution feature selection based on NRS is presented. Extensive experiments on 12 data sets show the effectiveness of the proposed algorithm.

TTIS-YOLO: a traffic target instance segmentation paradigm for complex road scenarios

Abstract The instance segmentation of traffic targets in complex road scenes is one of the most challenging tasks in autonomous driving. Unlike the bounding box localization for object detection and the category perception mask for semantic segmentation, instance segmentation requires accurate identification of each object under each category and more precise segmentation and positioning of these target objects. Although instance segmentation has apparent advantages, methods, for instance segmentation in complex road scenes, still need to be discovered. In this paper, we proposed an efficient instance segmentation method TTIS-YOLO(Traffic Target Instance Segmentation - YOLO) based on YOLOV5-7.0 for traffic object segmentation of complex road scenes. Our main work is as follows: to propose a MECSP(Multiscale Efficient Cross Stage Partial Network) module, which has fewer parameters, better cross-layer information exchange, and feature representation capabilities. Propose an Efficient Bidirectional Cross Scale Connection optimization method that enables the network to perform more detailed and efficient feature fusion without losing original information, refining the mask flow. WIoU Loss is used as the loss function of positioning and segmentation, and the positioning performance of the model is effectively improved through the strategy of dynamically allocating gradient gains. Experiments have shown that our proposed TTIS-YOLO outperforms baseline models and other mainstream instances segmentation algorithms such as Mask RCNN, YOLACT, SOLO, and SOLOV2 with the highest segmentation accuracy and fastest inference speed. Our proposed TTIS-YOLO-S achieves the best balance between segmentation accuracy and inference speed. Compared to the baseline model, the AP50 and recall values on the Cityscapes validation set increased by 1.7% and 0.9%, respectively, with a parameter reduction of 20.6%, inference speed of 78.1fps on GeForce RTX 3090Ti. Meanwhile, TTIS-YOLO-L achieved the highest segmentation accuracy, with an AP50 value of 27%, and the model parameter quantity decreased by 35.4% compared to the baseline model.

Class-agnostic counting and localization with feature augmentation and scale-adaptive aggregation

Compared to traditional class-specific counting, class-agnostic counting (CAC) enables the counting of arbitrary novel categories with only a few exemplars. The core issue of CAC lies in feature matching between the query image and exemplars. However, the scarce user-given exemplars make it difficult to adequately cover the diverse appearance of target objects, which poses challenges for feature matching. In this paper, we propose a Class-Agnostic Counting and Localization model (CACL) to enrich the exemplars by mining more inherent instances from the query feature. Specifically, CACL consists mainly of Self-Augment Feature Matching (SAFM) and Scale-Adaptive Spatial Aggregation (SASA). SAFM collects features of target instances frequently occurring in the query feature to augment the limited user-given exemplar, thus producing more comprehensive semantic features of the target category. To improve the robustness against scale variation, we design the SASA to enable the model to aggregate multiscale features adaptively. Compared to previous density map-based approaches, our method can locate the target objects which is more practical in real-world applications. Extensive experiments on FSC-147 suggest that our method achieves a competitive counting performance compared to state-of-the-art methods. Moreover, cross-dataset experiments demonstrate superior generalization in class-specific counting tasks.

Selective Random Walk for Transfer Learning in Heterogeneous Label Spaces.

Transfer learning has been widely used in different scenarios, especially in those lacking enough labeled data. However, most of the existing transfer learning methods are based on the assumption that the source and target domains should share the label space entirely or partially, which greatly limits their application scopes. In this article, a Selective Random Walk (SRW) method for transfer learning in heterogeneous label spaces is proposed to make full use of unlabeled auxiliary data, which acts as a bridge for knowledge transfer from the source domain to the target domain. The proposed SRW method can explicitly identify transfer sequences between source and target instances via auxiliary instances based on random walk techniques. Since not all of the transfer sequences generated by random walk are credible for the target task, the SRW method can learn to weight transfer sequences adaptively. Based on the weights of the transfer sequences, the SRW method leverages knowledge by forcing adjacent data points in the transfer sequence to be similar and making the target data point in the sequence represented by other data points in the same sequence. Experiments show that the SRW method outperforms state-of-the-art models in plenty of transfer learning tasks with heterogeneous label spaces constructed within and across several benchmark datasets.

VCAFusion: An infrared and visible image fusion network with visual perception and cross-scale attention

Infrared and visible image fusion methods aim to combine salient target instances and abundant texture details into fused images. However, due to the interference of harsh conditions, such as dense smoke, fog, and intense light, it is feasible for external interference information to be integrated into the fused image, which seriously affects the image quality. To this end, we propose an infrared and visible image fusion network with visual perception and cross-scale attention modules, termed VCAFusion, which integrates critical information from source images in harsh conditions more efficiently. Specifically, considering that the human eye can identify key information under adverse conditions, we design a visual perception module (VPM), guiding information integration from the perspective of human visual perception. In addition, we propose a cross-scale attention module (CSAM) based on shifted window cross-attention, which aims to capture the long-distance correlation between adjacent scale features, providing more accurate image information for image restoration. Experiments with a variety of datasets reveal that the VCAFusion can adaptively retain image information and further improve image generation ability in harsh conditions.

Meta-Learning With Distributional Similarity Preference for Few-Shot Fault Diagnosis Under Varying Working Conditions.

Few-shot fault diagnosis is a challenging problem for complex engineering systems due to the shortage of enough annotated failure samples. This problem is increased by varying working conditions that are commonly encountered in real-world systems. Meta-learning is a promising strategy to solve this point, open issues remain unresolved in practical applications, such as domain adaptation, domain generalization, etc. This article attempts to improve domain adaptation and generalization by focusing on the distribution-shift robustness of meta-learning from the task generation perspective. In fact, few-shot fault diagnosis under varying working conditions allows to address the distribution shift problem in a natural way. An unsupervised across-tasks meta-learning strategy with distributional similarity preference is proposed, where the core is the distribution-distance-weighting mechanism. Differently from the naive random meta-train task generation strategy used in existing meta-learning methods, the source instances that present a more similar distribution with respect to the target instances gain larger weightings in the task generation. This strategy leads to a meta-task training set that is enough diverse, and at the same time can be easily learned due to the distribution similarity features of the source tasks. The proposed method introduces the concept of maximum mean discrepancy that is applied to derive the distribution distance of the measurements. Moreover, a model-agnostic meta-learning is applied to realize few-shot fault diagnosis under varying working conditions. The proposed solutions are verified and compared by considering two public datasets used for bearing fault diagnosis. The results show that the proposed strategy outperforms different related few-shot fault diagnosis methods under varying working conditions. Moreover, it is thus proved that, meta-learning with distribution similarity feature represents an effective approach for domain adaptation and generalization.

An eXplainable Artificial Intelligence Methodology on Big Data Architecture

Although artificial intelligence has become part of everyone’s real life, a trust crisis against such systems is occurring, thus increasing the need to explain black-box predictions, especially in the military, medical, and financial domains. Modern eXplainable Artificial Intelligence (XAI) techniques focus on benchmark datasets, but the cognitive applicability of such solutions under big data settings is still unclear due to memory or computation constraints. In this paper, we extend a model-agnostic XAI methodology, named Cluster-Aided Space Transformation for Local Explanation (CASTLE), to be able to deal with high-volume datasets. CASTLE aims to explain the black-box behavior of predictive models by combining both local (i.e., based on the input sample) and global (i.e., based on the whole scope for action of the model) information. In particular, the local explanation provides a rule-based explanation for the prediction of a target instance as well as the directions to update the likelihood of the predicted class. Our extension leverages modern big data technologies (e.g., Apache Spark) to handle the high volume, variety, and velocity of huge datasets. We have evaluated the framework on five datasets, in terms of temporal efficiency, explanation quality, and model significance. Our results indicate that the proposed approach retains the high-quality explanations associated with CASTLE while efficiently handling large datasets. Importantly, it exhibits a sub-linear, rather than exponential, dependence on dataset size, making it a scalable solution for massive datasets or in any big data scenario.

Design of Logistics Sorting Algorithm Based on Deep Learning and Sampling Evaluation

The advancement of automation and Internet of Things technology has bolstered the automation process in the logistics sector. To address the challenge of localizing and generating grasping positions for intelligent robots in logistics sorting, this study developed an algorithm for item localization. The algorithm relies on enhanced YOLOv3 target detection and instance segmentation technologies to design a position generation algorithm for the robotic arm, which was further refined using sampling evaluation. The experimental results showed that the research-improved target detection model performed better on different datasets in terms of F1 value, accuracy and Area under the Curve (AUC) metrics, with the highest values of 95.77%, 94.05%, and 91.30%, respectively, which was effective in localizing document-like parcels. Meanwhile, the instance segmentation algorithm with fused features took significantly lower values than other target detection algorithms in terms of average absolute value error and root mean square error. The accuracy rate and all-class average precision value were higher than other target detection models, and the fluctuation of the value taken was smaller, which was suitable for logistics parcel localization. The position generation model, based on a sampling evaluation, yielded significantly different values compared to other algorithms. The relative position error and absolute trajectory error indexes were all below 0.4. The combined indexes of grasping accuracy and error indicate the superior performance of the research-designed algorithms. They can effectively enhance the sorting effects of real logistics scenarios. This research contributes to the improvement of the automated sorting system through the use of visual robotic arm technology. Additionally, it encourages the development of logistics automation and the establishment of intelligent logistics factories.

An Innovative Deep Learning Approach for Image Semantic and Instance Segmentation

In this study, we propose a segmentation model based on convolutional neural networks (CNNs) to address image segmentation challenges in computer vision. Prior to designing the model, the activation function and other modules of the convolutional neural network were optimized to meet specific requirements. The segmentation task was transformed into binary classification problem to simplify network calculations and improve efficiency. Additionally, the model utilized a mask map obtained from the semantic segmentation model to aid in instance segmentation. Class activation technology was introduced to extract feature mapping maps. The corresponding thermal maps were obtained to achieve target instance segmentation. To further validate the effectiveness of the segmentation model, simulation experiments were conducted on semantic segmentation and instance segmentation respectively. The results show that the accuracy of the basic semantic segmentation model reached 87.58%, while the average accuracy of the entire class of the optimized instance segmentation model reached 97.9%. Therefore, the research and design of image segmentation models demonstrate high accuracy and good robustness.

Optimal heterogeneous domain adaptation for text classification in transfer learning

The prime challenge of unsupervised symmetric heterogeneous cross-domain adaptation is to train the source domain and apply the trained knowledge to the target domain. Most of the existing algorithms for unsupervised transfer learning create the subspace of the source domain features and target domain features for training purposes. It is an extensive computational process as most of the techniques require labeled source data. Many techniques also suffer from the loss of originality of features in both domains. This paper aims to consider the feature vectors of both the source and target domain for training the data based on the similarity of exemplar (feature) vectors of different instances, known as Instance Similarity Feature (ISF). The use of vectorization method for the similarity of features is proposed in this paper. The exemplar vectors are chosen randomly for the target datasets. Hence, to acquire relevant factual data in the knowledge base for training in our research, we worked to increase the domain separation error between source and target instances. To avoid the instability caused due to poor exemplar vector selection, the K-means clustering approach is followed after feature similarity, known as K-means Instance Similarity Feature (KISF). Many existing transfer learning techniques are based on the original feature set, which can cause degeneracy, hence affecting Accuracy. In order to vanquish the limitations of existing approaches, we have introduced novel optimal models with KISF and Ant Lion Optimizer (KISFA), KISF with Particle Swarm Optimization (KISFP) and KISF with Biogeography Based Optimization (KISFB). High-dimensionality can impact efficacy of the model, hence, feature selection with nature-based optimizer namely: Ant Lion Optimizer, Particle Swarm Optimization and Biogeography-Based Optimization are applied. We measure the performance of the proposed models by using Support Vector Machine, Logistic Regression, Random Forest, Naive Baye’s, K-Nearest Neighbor and Decision Tree as classifiers, and Accuracy and F1-score as fitness functions. Extensive experiments are performed on four datasets with 50 iterations. The proposed model is compared with eleven other techniques and our technique outperforms all other techniques in average Accuracy. The validation is performed on the dataset using 10-fold cross-validation. The statistical test was performed using ANOVA, proving that our technique is significantly better than other techniques.

Instance segmentation algorithm for sorting dismantling components of end-of-life vehicles

In the recycling and dismantling procedures of End-of-Life Vehicles, the diverse thicknesses of disassembled parts necessitate specialized recycling processes, underscoring the imperative for efficient classification and recycling methods. Presently, the classification of disassembled components from scrapped vehicles relies heavily on manual visual inspection and caliper measurements, resulting in inefficiency and low accuracy. To overcome the challenges associated with thickness classification for disassembled components of End-of-Life Vehicles, propose a paradigm shift by introducing an intelligent grading strategy. This approach aims to significantly improve efficiency and accuracy in the classification process, offering a more effective solution for the recycling of dismantled parts. In response to the inefficiencies, low accuracy, and safety concerns associated with manual sorting, propose a deep learning model, incorporating an attention mechanism, for accomplishing instance segmentation tasks of End-of-Life Vehicles components. The model introduces a global attention mechanism into the backbone network for feature extraction, which is further integrated into the instance segmentation branch, resulting in a significant enhancement of instance segmentation performance. The model is trained and optimized using a dataset collected from an End-of-Life Vehicles dismantling facility in China. Experimental results show that when the intersection over union is 0.5, the mean Average Precision of the model in target detection and instance segmentation reaches 93.2% and 92.1% respectively., and 77.2% and 72.4% in the intersection over union is 0.5–0.95. In comparison to the unmodified baseline model, the model exhibits improvements of 1.7%, 2%, 5.8%, and 3.1% across evaluation metrics. When contrasted with traditional manual sorting methods, the proposed model exhibits pronounced advantages in terms of accuracy and fairness. In conclusion, the model effectively addresses the challenge of intelligent sorting of dismantled components from End-of-Life Vehicles during the dismantling process.

X-RefSeg3D: Enhancing Referring 3D Instance Segmentation via Structured Cross-Modal Graph Neural Networks

Referring 3D instance segmentation is a challenging task aimed at accurately segmenting a target instance within a 3D scene based on a given referring expression. However, previous methods have overlooked the distinct roles played by different words in referring expressions. Additionally, they have failed to incorporate the positional relationship within referring expressions with the spatial correlations in 3D scenes. To alleviate these issues, we present a novel model called X-RefSeg3D, which constructs a cross-modal graph for the input 3D scene and unites textual and spatial relationships for reasoning via graph neural networks. Our approach begins by capturing object-specific text features, which are then fused with the instance features to construct a comprehensive cross-modal scene graph. Subsequently, we integrate the obtained cross-modal features into graph neural networks, leveraging the K-nearest algorithm to derive explicit instructions from expressions and factual relationships in scenes. This enables the effective capture of higher-order relationships among instances, thereby enhancing feature fusion and facilitating reasoning. Finally, the refined feature undergoes a matching module to compute the ultimate matching score. Experimental results on ScanRefer demonstrate the effectiveness of our method, surpassing previous approaches by a substantial margin of +3.67% in terms of mIOU.

Attention-Based Monocular Depth Estimation Considering Global and Local Information in Remote Sensing Images

Monocular depth estimation using a single remote sensing image has emerged as a focal point in both remote sensing and computer vision research, proving crucial in tasks such as 3D reconstruction and target instance segmentation. Monocular depth estimation does not require multiple views as references, leading to significant improvements in both time and efficiency. Due to the complexity, occlusion, and uneven depth distribution of remote sensing images, there are currently few monocular depth estimation methods for remote sensing images. This paper proposes an approach to remote sensing monocular depth estimation that integrates an attention mechanism while considering global and local feature information. Leveraging a single remote sensing image as input, the method outputs end-to-end depth estimation for the corresponding area. In the encoder, the proposed method employs a dense neural network (DenseNet) feature extraction module with efficient channel attention (ECA), enhancing the capture of local information and details in remote sensing images. In the decoder stage, this paper proposes a dense atrous spatial pyramid pooling (DenseASPP) module with channel and spatial attention modules, effectively mitigating information loss and strengthening the relationship between the target’s position and the background in the image. Additionally, weighted global guidance plane modules are introduced to fuse comprehensive features from different scales and receptive fields, finally predicting monocular depth for remote sensing images. Extensive experiments on the publicly available WHU-OMVS dataset demonstrate that our method yields better depth results in both qualitative and quantitative metrics.

Uncertainty-weighted prototype active learning in domain adaptive semantic segmentation

Generalizing segmentation networks that adapt to unseen target domains is practical to the real-world utility. Recent self-training-based approaches compute the pseudo labels of highly imbalanced target image that are typically biased to the majority classes and basically noisy. To avoid the model falling into an error-prone and suboptimal state, we propose an uncertainty-weighted prototype active learning method to deal with domain adaptation regarding the semantic segmentation task. For assisting active learning to select reliable samples, multiple prototypical anchors strategy is employed to build the target image distribution model and assign accurate pixel-level pseudo labels, which consists of two different stages. In the initial training stage, this model defines each class as a set of non-learnable prototypes, and performs online clustering to identify target instances for annotating in the prototype-based feature space. In the fine-tuning stage, entropy criterion is adopted to generate representative class prototypes set of target domain, then compute the uncertainty as the clustering weight of prototype-level features via a predictive entropy-based strategy. By mixing these latent representations of the target samples and developing feature alignment loss function, our method can achieve more accurate segmentation performance. Extensive experiments on two benchmarks, GTAV → Cityscapes and SYNTHIA → Cityscapes, demonstrate that the proposed model outperforms other state-of-the-art approaches, and ablation study verifies the effectiveness of each key component. We hope our work will open a new view for future research in the field of domain adaptation. The code will be released soon at https://github.com/zihaodong/UPAL.

USER: Unified Semantic Enhancement With Momentum Contrast for Image-Text Retrieval.

As a fundamental and challenging task in bridging language and vision domains, Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality, and its key challenge is to measure the semantic similarity across different modalities. Although significant progress has been achieved, existing approaches typically suffer from two major limitations: (1) It hurts the accuracy of the representation by directly exploiting the bottom-up attention based region-level features where each region is equally treated. (2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism. To address these limitations, we propose a Unified Semantic Enhancement Momentum Contrastive Learning (USER) method for ITR. Specifically, we delicately design two simple but effective Global representation based Semantic Enhancement (GSE) modules. One learns the global representation via the self-attention algorithm, noted as Self-Guided Enhancement (SGE) module. The other module benefits from the pre-trained CLIP module, which provides a novel scheme to exploit and transfer the knowledge from an off-the-shelf model, noted as CLIP-Guided Enhancement (CGE) module. Moreover, we incorporate the training mechanism of MoCo into ITR, in which two dynamic queues are employed to enrich and enlarge the scale of negative sample pairs. Meanwhile, a Unified Training Objective (UTO) is developed to learn from mini-batch based and dynamic queue based samples. Extensive experiments on the benchmark MSCOCO and Flickr30K datasets demonstrate the superiority of both retrieval accuracy and inference efficiency. For instance, compared with the existing best method NAAF, the metric R@1 of our USER on the MSCOCO 5K Testing set is improved by 5% and 2.4% on caption retrieval and image retrieval without any external knowledge or pre-trained model while enjoying over 60 times faster inference speed. Our source code will be released at https://github.com/zhangy0822/USER.