Instance Level Research Articles

Video Moment Retrieval With Noisy Labels.

Video moment retrieval (VMR) aims to localize the target moment in an untrimmed video according to the given nature language query. The existing algorithms typically rely on clean annotations to train their models. However, making annotations by human labors may introduce much noise. Thus, the video moment retrieval models will not be well trained in practice. In this article, we present a simple yet effective video moment retrieval framework via bottom-up schema, which is in end-to-end manners and robust to noisy label training. Specifically, we extract the multimodal features by syntactic graph convolutional networks and multihead attention layers, which are fused by the cross gates and the bilinear approach. Then, the feature pyramid networks are constructed to encode plentiful scene relationships and capture high semantics. Furthermore, to mitigate the effects of noisy annotations, we devise the multilevel losses characterized by two levels: a frame-level loss that improves noise tolerance and an instance-level loss that reduces adverse effects of negative instances. For the frame level, we adopt the Gaussian smoothing to regard noisy labels as soft labels through the partial fitting. For the instance level, we exploit a pair of structurally identical models to let them teach each other during iterations. This leads to our proposed robust video moment retrieval model, which experimentally and significantly outperforms the state-of-the-art approaches on standard public datasets ActivityCaption and textually annotated cooking scene (TACoS). We also evaluate the proposed approach on the different manual annotation noises to further demonstrate the effectiveness of our model.

Ensemble machine learning prediction of hyperuricemia based on a prospective health checkup population.

Objectives: An accurate prediction model for hyperuricemia (HUA) in adults remain unavailable. This study aimed to develop a stacking ensemble prediction model for HUA to identify high-risk groups and explore risk factors. Methods: A prospective health checkup cohort of 40899 subjects was examined and randomly divided into the training and validation sets with the ratio of 7:3. LASSO regression was employed to screen out important features and then the ROSE sampling was used to handle the imbalanced classes. An ensemble model using stacking strategy was constructed based on three individual models, including support vector machine, decision tree C5.0, and eXtreme gradient boosting. Model validations were conducted using the area under the receiver operating characteristic curve (AUC) and the calibration curve, as well as metrics including accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F1 score. A model agnostic instance level variable attributions technique (iBreakdown) was used to illustrate the black-box nature of our ensemble model, and to identify contributing risk factors. Results: Fifteen important features were screened out of 23 clinical variables. Our stacking ensemble model with an AUC of 0.854, outperformed the other three models, support vector machine, decision tree C5.0, and eXtreme gradient boosting with AUCs of 0.848, 0.851 and 0.849 respectively. Calibration accuracy as well as other metrics including accuracy, specificity, negative predictive value, and F1 score were also proved our ensemble model's superiority. The contributing risk factors were estimated using six randomly selected subjects, which showed that being female and relatively younger, together with having higher baseline uric acid, body mass index, γ-glutamyl transpeptidase, total protein, triglycerides, creatinine, and fasting blood glucose can increase the risk of HUA. To further validate our model's applicability in the health checkup population, we used another cohort of 8559 subjects that also showed our ensemble prediction model had favorable performances with an AUC of 0.846. Conclusion: In this study, the stacking ensemble prediction model for HUA was developed, and it outperformed three individual models that compose it (support vector machine, decision tree C5.0, and eXtreme gradient boosting). The contributing risk factors were identified with insightful ideas.

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.

Few-shot Tumor Bud Segmentation Using Generative Model in Colorectal Carcinoma.

Current deep learning methods in histopathology are limited by the small amount of available data and time consumption in labeling the data. Colorectal cancer (CRC) tumor budding quantification performed using H&E-stained slides is crucial for cancer staging and prognosis but is subject to labor-intensive annotation and human bias. Thus, acquiring a large-scale, fully annotated dataset for training a tumor budding (TB) segmentation/detection system is difficult. Here, we present a DatasetGAN-based approach that can generate essentially an unlimited number of images with TB masks from a moderate number of unlabeled images and a few annotated images. The images generated by our model closely resemble the real colon tissue on H&E-stained slides. We test the performance of this model by training a downstream segmentation model, UNet++, on the generated images and masks. Our results show that the trained UNet++ model can achieve reasonable TB segmentation performance, especially at the instance level. This study demonstrates the potential of developing an annotation-efficient segmentation model for automatic TB detection and quantification.

Multi-view contrastive clustering via integrating graph aggregation and confidence enhancement

Multi-view clustering endeavors to effectively uncover consistent clustering patterns across multiple data sources or feature spaces. This field grapples with two key challenges: (1) the effective integration and utilization of consistency and complementarity information from diverse view spaces, and (2) the capturing of structural correlations between data samples in the multi-view context. To address these challenges, this paper proposes the Multi-view contrAstive clustering with Graph Aggregation and confidence enhancement (MAGA) algorithm. Specifically, we employ a deep autoencoder network to learn embedded features for each independent view. To harness consistency and complementarity information, we introduce the Simple Cross-view Spectral Graph Aggregation module. This module utilizes graph convolutional layers to generate view-specific graph embeddings and subsequently aggregates these embeddings from different views into a unified feature space using a cross-view self-attention mechanism. To capture both inter-view and intra-view structural correlations among different samples, we propose a dual representation contrastive learning mechanism, which operates concurrently at both the instance and feature levels. Additionally, we introduce the maximizing cluster assignment confidence mechanism to obtain more compact clustering assignments. As a result, MAGA outperforms 20 competitive methods across nine benchmark datasets, showcasing its superior performance. Code: https://github.com/BJT-bjt/MAGA.

Loose to compact feature alignment for domain adaptive object detection

Recently, great achievements have been made for deep learning based object detection methods. But their performance drops significantly when domain shifts occur. To address this problem, in this work we propose a loose to compact feature alignment method under an unsupervised domain adaptation framework. The entire feature alignment is performed in a divide and conquer manner, so as to distribute the alignment difficulties at two steps. At the first step, we loosen the goal at both image and instance levels. At the image level, a new Mask Guided Foreground Alignment (MGFA) module is proposed to make the alignment focus more on easier foreground regions, leaving the more diverse and more difficult background regions to the second step; at the instance level, we propose a Class-Wise Instance Alignment (CWIA) module with separated domain classifiers for different categories so as to ease the alignment. At the second step, the alignment is performed per pixel and per instance, achieving a more compact and better aligned feature space. We conduct experiments on three different adaptation scenarios, where we achieve comparable results, demonstrating the effectiveness of our proposed method.

Multi-Instance Learning with One Side Label Noise

Multi-instance Learning (MIL) is a popular learning paradigm arising from many real applications. It assigns a label to a set of instances, which is called a bag, and the bag’s label is determined by the instances within it. A bag is positive if and only if it has at least one positive instance. Since labeling bags is more complicated than labeling each instance, we will often face the mislabeling problem in MIL. Furthermore, it is more common that a negative bag has been mislabeled to a positive one, since one mislabeled instance will lead to the change of the whole bag label. This is an important problem that originated from real applications, e.g., web mining and image classification, but little research has concentrated on it as far as we know. In this article, we focus on this MIL problem with one side label noise that the negative bags are mislabeled as positive ones. To address this challenging problem, we propose, to the best our our knowledge, a novel multi-instance learning method with one side label noise. We design a new double weighting approach under traditional framework to characterize the “faithfulness” of each instance and each bag in learning the classifier. Briefly, on the instance level, we employ a sparse weighting method to select the key instances, and the MIL problem with one size label noise is converted to a mislabeled supervised learning scenario. On the bag level, the weights of bags, together with the selected key instances, will be utilized to identify the real positive bags. In addition, we have solved our proposed model by an alternative iteration method with proved convergence behavior. Empirical studies on various datasets have validated the effectiveness of our method.

Boosting Multiple Instance Learning Models for Whole Slide Image Classification: A Model-Agnostic Framework Based on Counterfactual Inference

Multiple instance learning is an effective paradigm for whole slide image (WSI) classification, where labels are only provided at the bag level. However, instance-level prediction is also crucial as it offers insights into fine-grained regions of interest. Existing multiple instance learning methods either solely focus on training a bag classifier or have the insufficient capability of exploring instance prediction. In this work, we propose a novel model-agnostic framework to boost existing multiple instance learning models, to improve the WSI classification performance in both bag and instance levels. Specifically, we propose a counterfactual inference-based sub-bag assessment method and a hierarchical instance searching strategy to help to search reliable instances and obtain their accurate pseudo labels. Furthermore, an instance classifier is well-trained to produce accurate predictions. The instance embedding it generates is treated as a prompt to refine the instance feature for bag prediction. This framework is model-agnostic, capable of adapting to existing multiple instance learning models, including those without specific mechanisms like attention. Extensive experiments on three datasets demonstrate the competitive performance of our method. Code will be available at https://github.com/centurion-crawler/CIMIL.

HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation

Point-supervised Temporal Action Localization (PSTAL) is an emerging research direction for label-efficient learning. However, current methods mainly focus on optimizing the network either at the snippet-level or the instance-level, neglecting the inherent reliability of point annotations at both levels. In this paper, we propose a Hierarchical Reliability Propagation (HR-Pro) framework, which consists of two reliability-aware stages: Snippet-level Discrimination Learning and Instance-level Completeness Learning, both stages explore the efficient propagation of high-confidence cues in point annotations. For snippet-level learning, we introduce an online-updated memory to store reliable snippet prototypes for each class. We then employ a Reliability-aware Attention Block to capture both intra-video and inter-video dependencies of snippets, resulting in more discriminative and robust snippet representation. For instance-level learning, we propose a point-based proposal generation approach as a means of connecting snippets and instances, which produces high-confidence proposals for further optimization at the instance level. Through multi-level reliability-aware learning, we obtain more reliable confidence scores and more accurate temporal boundaries of predicted proposals. Our HR-Pro achieves state-of-the-art performance on multiple challenging benchmarks, including an impressive average mAP of 60.3% on THUMOS14. Notably, our HR-Pro largely surpasses all previous point-supervised methods, and even outperforms several competitive fully-supervised methods. Code will be available at https://github.com/pipixin321/HR-Pro.

Boosting Few-Shot Learning via Attentive Feature Regularization

Few-shot learning (FSL) based on manifold regularization aims to improve the recognition capacity of novel objects with limited training samples by mixing two samples from different categories with a blending factor. However, this mixing operation weakens the feature representation due to the linear interpolation and the overlooking of the importance of specific channels. To solve these issues, this paper proposes attentive feature regularization (AFR) which aims to improve the feature representativeness and discriminability. In our approach, we first calculate the relations between different categories of semantic labels to pick out the related features used for regularization. Then, we design two attention-based calculations at both the instance and channel levels. These calculations enable the regularization procedure to focus on two crucial aspects: the feature complementarity through adaptive interpolation in related categories and the emphasis on specific feature channels. Finally, we combine these regularization strategies to significantly improve the classifier performance. Empirical studies on several popular FSL benchmarks demonstrate the effectiveness of AFR, which improves the recognition accuracy of novel categories without the need to retrain any feature extractor, especially in the 1-shot setting. Furthermore, the proposed AFR can seamlessly integrate into other FSL methods to improve classification performance.

Dual Self-Paced Cross-Modal Hashing

Cross-modal hashing~(CMH) is an efficient technique to retrieve relevant data across different modalities, such as images, texts, and videos, which has attracted more and more attention due to its low storage cost and fast query speed. Although existing CMH methods achieve remarkable processes, almost all of them treat all samples of varying difficulty levels without discrimination, thus leaving them vulnerable to noise or outliers. Based on this observation, we reveal and study dual difficulty levels implied in cross-modal hashing learning, \ie instance-level and feature-level difficulty. To address this problem, we propose a novel Dual Self-Paced Cross-Modal Hashing (DSCMH) that mimics human cognitive learning to learn hashing from ``easy'' to ``hard'' in both instance and feature levels, thereby embracing robustness against noise/outliers. Specifically, our DSCMH assigns weights to each instance and feature to measure their difficulty or reliability, and then uses these weights to automatically filter out the noisy and irrelevant data points in the original space. By gradually increasing the weights during training, our method can focus on more instances and features from ``easy'' to ``hard'' in training, thus mitigating the adverse effects of noise or outliers. Extensive experiments are conducted on three widely-used benchmark datasets to demonstrate the effectiveness and robustness of the proposed DSCMH over 12 state-of-the-art CMH methods.

Про інстанційний розгляд звернень громадян за захистом конституційних прав і свобод в Україні

The article is devoted to the analysis of the constituent elements of citizens’ appeals The types of appeals and ways of their resolution by public authorities and their officials within the limits of the powers granted to them by law are investigated. The grounds and procedure for processing appeals of individuals and legal entities to law enforcement agencies of other countries on the prevention, detection and investigation of crimes are considered. The article concludes that a wide range of appeals to public authorities for protection of violated rights and freedoms indicates that they are guaranteed in administrative (out-of-court) and judicial proceedings. The jurisdictional activity of public authorities when considering citizens’ complaints about violated rights and freedoms has an instance (stage) level, which is a condition for observance of the rule of law and other principles enshrined in the Constitution.

Group benefits instance for data purification

Manually annotating datasets for training deep models is very labor-intensive and time-consuming. To overcome such inferiority, directly leveraging web images to conduct training data becomes a natural choice. Nevertheless, the presence of label noise in web data usually degrades the model performance. Existing methods for combating label noise are typically designed and tested on synthetic noisy datasets. However, they tend to fail to achieve satisfying results on real-world noisy datasets. To this end, we propose a method named GRIP to alleviate the noisy label problem for both synthetic and real-world datasets. Specifically, GRIP utilizes a group regularization strategy that estimates class soft labels to improve noise-robustness. Soft label supervision reduces overfitting on noisy labels and learns inter-class similarities to benefit classification. Furthermore, an instance purification operation globally identifies noisy labels by measuring the difference between each training sample and its class soft label. Through operations at both group and instance levels, our approach integrates the advantages of noise-robust and noise-cleaning methods and remarkably alleviates the performance degradation caused by noisy labels. Comprehensive experimental results on synthetic and real-world datasets demonstrate the superiority of GRIP over the existing state-of-the-art methods. The data and source code of this work have been made available at: https://github.com/NUST-Machine-Intelligence-Laboratory/GRIP.

Synthesizing class labels for highly imbalanced credit card fraud detection data

Acquiring labeled datasets often incurs substantial costs primarily due to the requirement of expert human intervention to produce accurate and reliable class labels. In the modern data landscape, an overwhelming proportion of newly generated data is unlabeled. This paradigm is especially evident in domains such as fraud detection and datasets for credit card fraud detection. These types of data have their own difficulties associated with being highly class imbalanced, which poses its own challenges to machine learning and classification. Our research addresses these challenges by extensively evaluating a novel methodology for synthesizing class labels for highly imbalanced credit card fraud data. The methodology uses an autoencoder as its underlying learner to effectively learn from dataset features to produce an error metric for use in creating new binary class labels. The methodology aims to automatically produce new labels with minimal expert input. These class labels are then used to train supervised classifiers for fraud detection. Our empirical results show that the synthesized labels are of high enough quality to produce classifiers that significantly outperform a baseline learner comparison when using area under the precision-recall curve (AUPRC). We also present results of varying levels of positive-labeled instances and their effect on classifier performance. Results show that AUPRC performance improves as more instances are labeled positive and belong to the minority class. Our methodology thereby effectively addresses the concerns of high class imbalance in machine learning by creating new and effective class labels.

SAR-CDSS: A Semi-Supervised Cross-Domain Object Detection from Optical to SAR Domain

The unique imaging modality of synthetic aperture radar (SAR) has posed significant challenges for object detection, making it more complex to acquire and interpret than optical images. Recently, numerous studies have proposed cross-domain adaptive methods based on convolutional neural networks (CNNs) to promote SAR object detection using optical data. However, existing cross-domain methods focus on image features, lack improvement on input data, and ignore the valuable supervision provided by few labeled SAR images. Therefore, we propose a semi-supervised cross-domain object detection framework that uses optical data and few SAR data to achieve knowledge transfer for SAR object detection. Our method focuses on the data processing aspects to gradually reduce the domain shift at the image, instance, and feature levels. First, we propose a data augmentation method of image mixing and instance swapping to generate a mixed domain that is more similar to the SAR domain. This method fully utilizes few SAR annotation information to reduce domain shift at image and instance levels. Second, at the feature level, we propose an adaptive optimization strategy to filter out mixed domain samples that significantly deviate from the SAR feature distribution to train feature extractor. In addition, we employ Vision Transformer (ViT) as feature extractor to handle the global feature extraction of mixed images. We propose a detection head based on normalized Wasserstein distance (NWD) to enhance objects with smaller effective regions in SAR images. The effectiveness of our proposed method is evaluated on public SAR ship and oil tank datasets.

"Little Man" in the context of the narrative structure of the work: "Overcoat" by N. V. Gogol and "Irrepressible Tambourine" by A.M. Remizov

The subject of the study is the evolution of the "little man" as a literary phenomenon based on the novels "Overcoat" by Gogol and "Irrepressible Tambourine" by Remizov. It is noted that the content of the term under study is not limited to the semantics of the image, which contains a meaningful characteristic of a certain type of literary hero, but depends on the narrative structure of the work. This position of the "little man" in Gogol and Remizov is at the intersection of interrelated factors: details as a carrier of the grotesque in fictional reality and at the level of narrative instance, as well as modifications of a specific genre form, expressed in the original construction of the composition — Gogol overcoming the genre of physiological essay and Remizov household anecdote, respectively. The study shows that detail plays a key role in the poetics of the works of these authors and determines not only the dynamics of the image in a particular work, but also forms an appropriate aspect of influence. Remizov inherits the indirect characterization of a literary character through the details of the external world, adopted by Gogol, but in another way secures with their help the status of the main character of the "Irrepressible Tambourine" as a small person at the narrative level due to a more open organization of the horizon of readers' expectations. The task required the synthesis of a historical and genetic approach with a genrological (genological) one, which made it possible to consider the category of "little man" as an element of poetics, reinterpreted by Remizov in "Irrepressible Tambourine" under the influence of Gogol's work, and its significance in connection with the transformation of the genre model with an individual author's approach to narration. The study showed that Remizov overcomes the epistemological uncertainty of the narrator in the "Overcoat" and, thanks to the details referring to the mythopoeic context (Ancient Russia, Eros), builds the image of a small man (Stratilatov), whose insignificance and mediocrity are less related to the plot of the "Irrepressible Tambourine" than with archetypal references that are carriers of narrative semantics stories and setting the meaningful direction of the author's thought. Thus, the unknowability of the foundations of being by the character is emphasized at the narrative level of the work, enhanced by the grotesque characterization. The corresponding conclusion, according to the author of the article, is relevant in analyzing the system of images of literary heroes in Remizov and his anticipation of the work of European expressionist prose writers with their emphasis on the existential, rather than social source of human suffering.

Comprehensive Evaluation of Multispectral Image Registration Strategies in Heterogenous Agriculture Environment.

This article is focused on the comprehensive evaluation of alleyways to scale-invariant feature transform (SIFT) and random sample consensus (RANSAC) based multispectral (MS) image registration. In this paper, the idea is to extensively evaluate three such SIFT- and RANSAC-based registration approaches over a heterogenous mix containing Triticum aestivum crop and Raphanus raphanistrum weed. The first method is based on the application of a homography matrix, derived during the registration of MS images on spatial coordinates of individual annotations to achieve spatial realignment. The second method is based on the registration of binary masks derived from the ground truth of individual spectral channels. The third method is based on the registration of only the masked pixels of interest across the respective spectral channels. It was found that the MS image registration technique based on the registration of binary masks derived from the manually segmented images exhibited the highest accuracy, followed by the technique involving registration of masked pixels, and lastly, registration based on the spatial realignment of annotations. Among automatically segmented images, the technique based on the registration of automatically predicted mask instances exhibited higher accuracy than the technique based on the registration of masked pixels. In the ground truth images, the annotations performed through the near-infrared channel were found to have a higher accuracy, followed by green, blue, and red spectral channels. Among the automatically segmented images, the accuracy of the blue channel was observed to exhibit a higher accuracy, followed by the green, near-infrared, and red channels. At the individual instance level, the registration based on binary masks depicted the highest accuracy in the green channel, followed by the method based on the registration of masked pixels in the red channel, and lastly, the method based on the spatial realignment of annotations in the green channel. The instance detection of wild radish with YOLOv8l-seg was observed at a mAP@0.5 of 92.11% and a segmentation accuracy of 98% towards segmenting its binary mask instances.

Prediction of lymph node metastasis in operable cervical cancer using clinical parameters and deep learning with MRI data: a multicentre study.

To develop and validate a magnetic resonance imaging-based (MRI) deep multiple instance learning (D-MIL) model and combine it with clinical parameters for preoperative prediction of lymph node metastasis (LNM) in operable cervical cancer. A total of 392 patients with cervical cancer were retrospectively enrolled. Clinical parameters were analysed by logistical regression to construct a clinical model (M1). A ResNet50 structure is applied to extract features at the instance level without using manual annotations about the tumour region and then construct a D-MIL model (M2). A hybrid model (M3) was constructed by M1 and M2 scores. The diagnostic performance of each model was evaluated by the area under the receiver operating characteristic curve (AUC) and compared using the Delong method. Disease-free survival (DFS) was evaluated by the Kaplan‒Meier method. SCC-Ag, maximum lymph node short diameter (LNmax), and tumour volume were found to be independent predictors of M1 model. For the diagnosis of LNM, the AUC of the training/internal/external cohort of M1 was 0.736/0.690/0.732, the AUC of the training/internal/external cohort of M2 was 0.757/0.714/0.765, and the AUC of the training/internal/external cohort of M3 was 0.838/0.764/0.835. M3 showed better performance than M1 and M2. Through the survival analysis, patients with higher hybrid model scores had a shorter time to reach DFS. The proposed hybrid model could be used as a personalised non-invasive tool, which is helpful for predicting LNM in operable cervical cancer. The score of the hybrid model could also reflect the DFS of operable cervical cancer. Lymph node metastasis is an important factor affecting the prognosis of cervical cancer. Preoperative prediction of lymph node status is helpful to make treatment decisions, improve prognosis, and prolong survival time. • The MRI-based deep-learning model can predict the LNM in operable cervical cancer. • The hybrid model has the highest diagnostic efficiency for the LNM prediction. • The score of the hybrid model can reflect the DFS of operable cervical cancer.

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

Annotation scarcity and cross-modality/stain data distribution shifts are two major obstacles hindering the application of deep learning models for nuclei analysis, which holds a broad spectrum of potential applications in digital pathology. Recently, unsupervised domain adaptation (UDA) methods have been proposed to mitigate the distributional gap between different imaging modalities for unsupervised nuclei segmentation in histopathology images. However, existing UDA methods are built upon the assumption that data distributions within each domain should be uniform. Based on the over-simplified supposition, they propose to align the histopathology target domain with the source domain integrally, neglecting severe intra-domain discrepancy over subpartitions incurred by mixed cancer types and sampling organs. In this paper, for the first time, we propose to explicitly consider the heterogeneity within the histopathology domain and introduce open compound domain adaptation (OCDA) to resolve the crux. In specific, a two-stage disentanglement framework is proposed to acquire domain-invariant feature representations at both image and instance levels. The holistic design addresses the limitations of existing OCDA approaches which struggle to capture instance-wise variations. Two regularization strategies are specifically devised herein to leverage the rich subpartition-specific characteristics in histopathology images and facilitate subdomain decomposition. Moreover, we propose a dual-branch nucleus shape and structure preserving module to prevent nucleus over-generation and deformation in the synthesized images. Experimental results on both cross-modality and cross-stain scenarios over a broad range of diverse datasets demonstrate the superiority of our method compared with state-of-the-art UDA and OCDA methods.Graphical abstract

Subdomain-Alignment Data Augmentation for Pipeline Fault Diagnosis: An Adversarial Self-Attention Network

Data augmentation (DA) has the potential to address the issue of imbalanced and insufficient datasets (I&ID) in pipeline fault diagnosis. However, the majority of existing DA methods for time series are inspired by computer vision techniques, ignoring the temporal dynamic properties and fine-grained fault features, which leads to limited performance of the augmentation. To tackle this problem, we introduce a novel DA approach called the subdomain-alignment adversarial self-attention network (SA-ASN), which takes into account both temporal association and semantic correlation. Our approach features a novel temporal association learning (TAL) mechanism, which transfers temporal information from the discriminator to the generator via a customized knowledge-sharing structure, improving the reliability of synthetic long-range associations. Additionally, we introduce a prototype-assisted subdomain alignment (PASA) strategy that forms a hierarchical structure in the synthetic dataset by incorporating local semantic correlation into the model training. With the support of TAL and PASA, our SA-ASN algorithm enhances the authenticity of temporal structure at the instance level and improves the discriminability of fault features at the category level. Our experimental results show that the SA-ASN algorithm provides a more diverse and accurate augmentation of pipeline data. The effectiveness of our SA-ASN algorithm encourages the use of data-driven diagnostic models in complex real-world oilfield pipeline networks.