Loss Of Contrast Research Articles

Dynamic image super-resolution via progressive contrastive self-distillation

Convolutional neural networks (CNNs) are highly successful for image super-resolution (SR). However, they often require sophisticated architectures with high memory cost and computational overhead, significantly restricting their practical deployments on resource-limited devices. In this paper, we propose a novel dynamic contrastive self-distillation (Dynamic-CSD) framework to simultaneously compress and accelerate various off-the-shelf SR models, and explore using the trained model for dynamic inference. In particular, to build a compact student network, a channel-splitting super-resolution network (CSSR-Net) can first be constructed from a target teacher network. Then, we propose a novel contrastive loss to improve the quality of SR images via explicit knowledge transfer. Furthermore, progressive CSD (Pro-CSD) is developed to extend the two-branch CSSR-Net into multi-branch, leading to a switchable model at runtime. Finally, a difficulty-aware branch selection strategy for dynamic inference is given. Extensive experiments demonstrate that the proposed Dynamic-CSD scheme effectively compresses and accelerates several standard SR models such as EDSR, RCAN and CARN.

Diffusion-based normality pre-training for weakly supervised video anomaly detection

Weakly supervised video anomaly detection is the task of detecting anomalous frames in videos where no frame-level labels are provided at training phase. Previous methods usually employed a multiple instance learning (MIL)-based ranking loss to ensure inter-class separation. However, these methods are unable to completely utilise the information from the huge amounts of normal frames. Moreover, the performance of these methods is misguided by the erroneous initial prediction of the MIL-based classifier. Taking these shortcomings into consideration, we propose a diffusion-based normality learning pretrain step, which first involves training a Global–Local Feature Encoder (GLFE) model with only normal videos to understand the feature distribution of normal frames. The resulting pre-trained Global–Local feature encoder is further optimised using Multi-Sequence Contrastive loss using both normal and anomalous videos. Our proposed GLFE model captures long- and short-range temporal features using a Transformer block and pyramid of dilated convolutions in a two-branch setup. The model adaptively learns the relation between the two branch features by introducing the Co-Attention module, which provides a learnable fusion of features. Additionally we introduced a triplet contrastive loss to provide better separation between abnormal and normal frames in anomalous videos. The developed methodology is evaluated through extensive experiments on two public benchmark datasets (UCF-Crime and ShanghaiTech). The results obtained are comparable to or better than the existing state-of-the-art weakly supervised methods.

Adaptive unified contrastive learning with graph-based feature aggregator for imbalanced medical image classification

Medical image datasets are often imbalanced due to biases in data collection and limitations in acquiring data for rare conditions. Addressing class imbalance is crucial for developing reliable deep-learning algorithms capable of effectively handling all classes. Recent class imbalanced methods have investigated the effectiveness of self-supervised learning (SSL) and demonstrated that such learned features offer increased resilience to class imbalance issues and obtain much improved performances over other types of class imbalanced methods. However, existing SSL methods either lack end-to-end capabilities or require substantial memory resources, potentially resulting in sub-optimal features and classifiers and limiting their practical usage. Moreover, the conventional pooling operations (e.g., max-pooling, or average-pooling) tend to generate less discriminative features when datasets pose high inter-class similarities. To alleviate the above issues, in this study, we present a novel end-to-end self-supervised learning framework tailored for imbalanced medical image datasets. Our framework constitutes an adaptive contrastive loss that can dynamically adjust the model’s learning focus between feature learning and classifier learning and a feature aggregation mechanism based on Graph Neural Networks to further enhance feature discriminability. We evaluate the effectiveness of our framework on four medical datasets, and the experimental results highlight its superior performance in imbalanced image classification tasks.

Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition.

This paper addresses a joint training approach applied to a pipeline comprising speech enhancement (SE) and automatic speech recognition (ASR) models, where an acoustic tokenizer is included in the pipeline to leverage the linguistic information from the ASR model to the SE model. The acoustic tokenizer takes the outputs of the ASR encoder and provides a pseudo-label through K-means clustering. To transfer the linguistic information, represented by pseudo-labels, from the acoustic tokenizer to the SE model, a cluster-based pairwise contrastive (CBPC) loss function is proposed, which is a self-supervised contrastive loss function, and combined with an information noise contrastive estimation (infoNCE) loss function. This combined loss function prevents the SE model from overfitting to outlier samples and represents the pronunciation variability in samples with the same pseudo-label. The effectiveness of the proposed CBPC loss function is evaluated on a noisy LibriSpeech dataset by measuring both the speech quality scores and the word error rate (WER). The experimental results reveal that the proposed joint training approach using the described CBPC loss function achieves a lower WER than the conventional joint training approaches. In addition, it is demonstrated that the speech quality scores of the SE model trained using the proposed training approach are higher than those of the standalone-SE model and SE models trained using conventional joint training approaches. An ablation study is also conducted to investigate the effects of different combinations of loss functions on the speech quality scores and WER. Here, it is revealed that the proposed CBPC loss function combined with infoNCE contributes to a reduced WER and an increase in most of the speech quality scores.

Supervised contrastive learning for graph representation enhancement

Graph Neural Networks (GNNs) have exhibited significant success in various applications, but they face challenges when labeled nodes are limited. A novel self-supervised learning paradigm has emerged, enabling GNN training without labeled nodes and even surpassing GNNs with limited labeled data. However, self-supervised methods lack class-discriminative node representations due to the absence of labeled information during training. In this paper, we exploit a supervised graph contrastive learning approach (SGCL) framework to tackle the issue of limited labeled nodes, ensuring coherent grouping of nodes within the same class. We propose augmentation techniques based on a novel centrality function to highlight important topological structures. Additionally, we introduce a supervised contrastive learning method that removes the necessity for negative samples and simplifies complex elements effortlessly. Our approach combines supervised contrastive loss and node similarity regularization while achieving consistent grouping of unlabeled nodes with labeled ones. Furthermore, we utilize the pseudo-labeling technique to propagate label information to distant nodes and address the underfitting problem, especially with low-degree nodes. Experimental results on real-world graphs demonstrate that SGCL outperforms both semi-supervised and self-supervised methods in node classification.

A novel cross-domain adaption network based on Se-Sk-DenseNet for remaining useful life prediction of rolling bearings under different working conditions

Effectively predicting the remaining useful life (RUL) of rolling bearings can ensure reliability and safety, minimize machine downtime, and reduce the operation and maintenance costs of enterprises. To solve the problems of data distribution discrepancy caused by different working conditions and the collected signals containing a lot of useless information and noise, a novel cross-domain adaption network (CDAN) is proposed in this study. Firstly, a novel feature extractor, squeeze-and-excitation (Se)-selective kernel (Sk)-DenseNet, is developed to extract useful critical features from the input data and remove the ineffective features by embedding Se and Sk attention blocks; besides, a new objective loss function consist of the RUL loss, the multi-kernel maximum mean discrepancy loss, the contrastive loss, and the Kullback–Leibler divergence loss, is proposed to solve the problem of data distribution shift; finally, the effectiveness and superiority of CDAN are proved on the PHM2012 bearings dataset. The results demonstrate that CDAN can extract deep critical features and achieve the high cross-domain RUL prediction accuracy under different working conditions.

DeepFLAIR: A neural network approach to mitigate signal and contrast loss in temporal lobes at 7 Tesla FLAIR images

Background and purposeHigher magnetic field strength introduces stronger magnetic field inhomogeneities in the brain, especially within temporal lobes, leading to image artifacts. Particularly, T2-weighted fluid-attenuated inversion recovery (FLAIR) images can be affected by these artifacts. Here, we aimed to improve the FLAIR image quality in temporal lobe regions through image processing of multiple contrast images via machine learning using a neural network. MethodsThirteen drug-resistant MR-negative epilepsy patients (age 29.2 ± 9.4y, 5 females) were scanned on a 7 T MRI scanner. Magnetization-prepared (MP2RAGE) and saturation-prepared with 2 rapid gradient echoes, multi-echo gradient echo with four echo times, and the FLAIR sequence were acquired. A voxel-wise neural network was trained on extratemporal-lobe voxels from the acquired structural scans to generate a new FLAIR-like image (i.e., deepFLAIR) with reduced temporal lobe inhomogeneities. The deepFLAIR was evaluated in temporal lobes through signal-to-noise (SNR), contrast-to-noise (CNR) ratio, the sharpness of the gray-white matter boundary and joint-histogram analysis. Saliency mapping demonstrated the importance of each input image per voxel. ResultsSNR and CNR in both gray and white matter were significantly increased (p < 0.05) in the deepFLAIR's temporal ROIs, compared to the FLAIR. The gray-white matter boundary sharpness was either preserved or improved in 10/13 right-sided temporal regions and was found significantly increased in the ROIs. Multiple image contrasts were influential for the deepFLAIR reconstruction with the MP2RAGE second inversion image being the most important. ConclusionsThe deepFLAIR network showed promise to restore the FLAIR signal and reduce contrast attenuation in temporal lobe areas. This may yield a valuable tool, especially when artifact-free FLAIR images are not available.

Inv-ReVersion: Enhanced Relation Inversion Based on Text-to-Image Diffusion Models

Diffusion models are widely recognized in image generation for their ability to produce high-quality images from text prompts. As the demand for customized models grows, various methods have emerged to capture appearance features. However, the exploration of relations between entities, another crucial aspect of images, has been limited. This study focuses on enabling models to capture and generate high-level semantic images with specific relation concepts, which is a challenging task. To this end, we introduce the Inv-ReVersion framework, which uses inverse relations text expansion to separate the feature fusion of multiple entities in images. Additionally, we employ a weighted contrastive loss to emphasize part of speech, helping the model learn more abstract relation concepts. We also propose a high-frequency suppressor to reduce the time spent on learning low-frequency details, enhancing the model’s ability to generate image relations. Compared to existing baselines, our approach can more accurately generate relation concepts between entities without additional computational costs, especially in capturing abstract relation concepts.

Management Analysis Method of Multivariate Time Series Anomaly Detection in Financial Risk Assessment

The significance of financial risk lies in its impact on economic stability and individual/institutional financial security. Effective risk management is crucial for market confidence and crisis prevention. Current methods for multivariate time series anomaly detection have limitations in adaptability and generalization. To address this, we propose an innovative approach integrating contrastive learning and Generative Adversarial Networks (GANs). We use geometric distribution masking for data augmentation to enhance dataset diversity. Within the GAN framework, we train a Transformer-based autoencoder to capture normal point distributions. We include contrastive loss in the discriminator to ensure robust generalization. Rigorous experiments on four real-world datasets show that our method effectively mitigates overfitting and outperforms state-of-the-art approaches. This enhances anomaly identification in risk management, paving the way for deep learning in finance, and offering insights for future research and practical use.

Temporal, spectral and amplitude characteristics of the Greek fricative /s/ in hearing-impaired and normal-hearing speech

ABSTRACT Fricatives, and especially sibilants, are very frequently misarticulated by speakers with hearing loss. Misarticulations can result in phonemic contrast weakening or loss, compromising intelligibility. The present study focuses on the examination of acoustic characteristics of the Greek alveolar fricative /s/, an articulatorily demanding sound, produced by young adult speakers with profound hearing impairment and with normal hearing. An array of variables was examined using mixed-effects and random forest models aiming to assess the effectiveness of various measures in differentiating hearing-impaired and normal-hearing /s/ production. Significant differences were found in spectral and amplitude measures, but not in temporal measures. In hearing-impaired speech, spectral slope and RMS amplitude had significantly lower values, indicating a more distributed spectrum, suggestive of decreased flow velocity through the fricative constriction. Also, a trend for concentration of energy at lower frequencies was observed suggesting more posterior fricative articulation than normal. Moreover, measures capturing the variation of frequency and amplitude over time revealed different patterns of sibilance development across time than normal, denoting the production of a less well-formed or less sibilant /s/ by speakers with hearing impairment. The investigation of contextual effects on /s/ in hearing-impaired speech showed increased spectral variance, negative skewness and lower kurtosis in the labial (rounded) context /u/ in relation to the nonlabial contexts /i/ and /a/, indicating a more diffuse, less compact spectrum with concentration at high frequencies. Findings are discussed in relation to previous literature on fricative production by speakers with hearing impairment and normal hearing in Greek and other languages.

CoReFace: Sample-guided Contrastive Regularization for Deep Face Recognition

The discriminability of the feature representation is crucial for face recognition. However, previous methods rely solely on the learnable weights of the classification layer, which represent the identities. This reliance could be problematic as the evaluation process depends on the similarity between pairs of face images and requires minimal identity information learned during training. As a result, there is an inconsistency between the training and evaluation processes, which can confuse the feature encoder and hinder the effectiveness of identity-based methods. To address this problem, we propose a novel approach namely Contrastive Regularization for Face Recognition (CoReFace), which applies sample-level regularization in feature representation learning. Specifically, we employ sample-guided contrastive learning to directly regularize the training based on the sample-sample relationship and thus align it with the evaluation process. To avoid image quality degradation, we augment the embeddings instead of the images in order to integrate contrastive learning into face recognition. Additionally, we introduce a new contrastive loss function for the regularization of representation distribution. This function incorporates an adaptive margin and a supervised contrastive mask to ensure stable loss values and prevent interference with the identity supervision signals. Finally, we explore new pair-coupling protocols in order to overcome the problem of semantically repetitive signals in contrastive learning. Extensive experiments demonstrate the efficacy and efficiency of our CoReFace approach, which achieves competitive results compared to state-of-the-art methods. Code could be found https://github.com/IsidoreSong/CoreFace here.

Clustering Method for Signals in the Wideband RF Spectrum Using Semi-Supervised Deep Contrastive Learning

This paper presents the application of self-supervised deep contrastive learning in clustering signals detected in the wideband RF spectrum, presented in the form of spectrograms. Radio clustering is a method of searching for similar signals within the analyzed part of the radio spectrum. Typically, it is based on one or several specific parameters processed from the signal in a given channel. The authors propose a slightly different, innovative approach; thanks to the self-supervised learning of neural networks, there is no need to define specific parameters, and the feature vector, enabling comparison of Euclidean distances between signals, is generated by a deep neural network trained using a contrastive loss function on a dataset containing different radio modulations. The authors describe self-supervised solutions based on contrastive learning and the methods of signal segmentation and augmentation. The training process utilizes a custom database and the Resnet-50 network with a contrastive cost function. Radio clustering is used for autonomous spectrum analysis across wide frequency ranges and enables, among other things, the detection of tactical radio stations operating with widely dispersed frequency-hopping or a significant reduction in computational power required for real-time analysis of a large number of radio signals.

Universal 12-lead ECG representation for signal denoising and cardiovascular disease detection by fusing generative and contrastive learning

With the wide use of electrocardiogram (ECG) technology, more and more ECGs have been collected and stored. However, ECG labeling is costly and laborious, the utilization of unlabeled ECG is still a critical challenge. Self-supervised learning (SSL) is a way to deal with this problem, which can learn representations from unlabeled ECG and use them for downstream tasks. In this study, a novel SSL model that fuses generative learning (denoising autoencoder, DAE) and contrastive learning (CL) is proposed to learn robust representations from unlabeled ECG for downstream denoising and classification. The 12-lead ECGs are separated into one-dimensional single-lead ECGs and ECG-specific noises are added to the original ECGs as the data augmentation method. To improve the model's denoising and feature extraction abilities, the reconstruction loss and contrastive loss are combined during the pretraining phase. In the downstream classification task, a multi-branch network is used to enhance the correlation between different ECG leads. As a result, it improves the denoising performance over the standard DAE by an average of 5.20%. In the classification tasks for the Chapman, PTB-XL, and CPSC2018 databases, compared to existing work, the accuracies in linear probe are increased by at least 2.49%, 1.09%, and 2.61%, respectively. In addition, an SoC (system-on-a-chip) based heterogeneous deployment scheme is designed. It is 13.69–14.26 times faster in inference than pure software deployment and enables real-time detection of ECG signals. The proposed method provides a good way to utilize unlabeled ECG, and the designed deployment scheme has great potential for medical applications.

One-shot font generation via local style self-supervision using Region-Aware Contrastive Loss

Compositional scripts like Hangeul (Korean characters) and Chinese characters involve numerous characters, making manual font design labor-intensive and cost-ineffective work. Although many few-shot font generation methods have been introduced, they have at least one of the limitations, i.e., lacking local styles of font, additional component labeling, and high complexity in network structure and training. To solve these limitations, given our observation that font style can be perceived at a patch-level rather than a component-level, we propose Region-Aware Contrastive loss (RAC-loss) so that the generator can capture the local style by self-supervision. The proposed loss maximizes the style information between patches of the generated image and the style reference image. And we introduce an attention mechanism to the patch-level contrastive loss to handle multiple patch correspondences. This attention learns style similarity between two glyph images, which serves as a patch-correspondence map. RAC-loss gives more fine-grained feedback to the generator than component-level loss, allowing it to incorporate local styles, even in a straightforward structure like a visual geometry group network (VGGNet). This results in a fast inference latency (3.02 ms), and the proposed method achieved 43.18 mean Fréchet Inception Distance (mFID) on the test dataset, a notable decrease of 5.42 compared to the previous method.

Contrastive Unfolding Deraining Network.

Due to the fact that the degradation of image quality caused by rain usually affects outdoor vision tasks, image deraining becomes more and more important. Focusing on the single image deraining (SID) task, in this article, we propose a novel Contrastive Unfolding DEraining Network (CUDEN), which combines the traditional iterative algorithm and deep network, exhibiting excellent performance and nice interpretability. CUDEN transforms the challenge of locating rain streaks into discovering rain features and defines the relationship between the image and feature domains in terms of mapping pairs. To obtain the mapping pairs efficiently, we propose a dynamic multidomain translation (DMT) module for decomposing the original mapping into sub-mappings. To enhance the feature extraction capability of networks, we also propose a new serial multireceptive field fusion (SMF) block, which extracts complex and variable rain features with convolution kernels of different receptive fields. Moreover, we are the first to introduce contrastive learning to the SID task and combine it with perceptual loss to propose a new contrastive perceptual loss (CPL), which is quite generalized and greatly helpful in identifying the appropriate gradient descent direction during training. Extensive experiments on synthetic and real-world datasets demonstrate that our proposed CUDEN outperforms the state-of-the-art (SOTA) deraining networks.

SC-SSL: Self-Correcting Collaborative and Contrastive Co-Training Model for Semi-Supervised Medical Image Segmentation.

Image segmentation achieves significant improvements with deep neural networks at the premise of a large scale of labeled training data, which is laborious to assure in medical image tasks. Recently, semi-supervised learning (SSL) has shown great potential in medical image segmentation. However, the influence of the learning target quality for unlabeled data is usually neglected in these SSL methods. Therefore, this study proposes a novel self-correcting co-training scheme to learn a better target that is more similar to ground-truth labels from collaborative network outputs. Our work has three-fold highlights. First, we advance the learning target generation as a learning task, improving the learning confidence for unannotated data with a self-correcting module. Second, we impose a structure constraint to encourage the shape similarity further between the improved learning target and the collaborative network outputs. Finally, we propose an innovative pixel-wise contrastive learning loss to boost the representation capacity under the guidance of an improved learning target, thus exploring unlabeled data more efficiently with the awareness of semantic context. We have extensively evaluated our method with the state-of-the-art semi-supervised approaches on four public-available datasets, including the ACDC dataset, M&Ms dataset, Pancreas-CT dataset, and Task_07 CT dataset. The experimental results with different labeled-data ratios show our proposed method's superiority over other existing methods, demonstrating its effectiveness in semi-supervised medical image segmentation.

NExT-OOD: Overcoming Dual Multiple-Choice VQA Biases.

In recent years, multiple-choice Visual Question Answering (VQA) has become topical and achieved remarkable progress. However, most pioneer multiple-choice VQA models are heavily driven by statistical correlations in datasets, which cannot perform well on multimodal understanding and suffer from poor generalization. In this paper, we identify two kinds of spurious correlations, i.e., a Vision-Answer bias (VA bias) and a Question-Answer bias (QA bias). To systematically and scientifically study these biases, we construct a new video question answering (videoQA) benchmark NExT-OOD in OOD setting and propose a graph-based cross-sample method for bias reduction. Specifically, the NExT-OOD is designed to quantify models' generalizability and measure their reasoning ability comprehensively. It contains three sub-datasets including NExT-OOD-VA, NExT-OOD-QA, and NExT-OOD-VQA, which are designed for the VA bias, QA bias, and VA&QA bias, respectively. We evaluate several existing multiple-choice VQA models on our NExT-OOD, and illustrate that their performance degrades significantly compared with the results obtained on the original multiple-choice VQA dataset. Besides, to mitigate the VA bias and QA bias, we explicitly consider the cross-sample information and design a contrastive graph matching loss in our approach, which provides adequate debiasing guidance from the perspective of whole dataset, and encourages the model to focus on multimodal contents instead of spurious statistical regularities. Extensive experimental results illustrate that our method significantly outperforms other bias reduction strategies, demonstrating the effectiveness and generalizability of the proposed approach.

Multi-rate modulation encoding via unsupervised learning for audio event detection

Technologies in healthcare, smart homes, security, ecology, and entertainment all deploy audio event detection (AED) in order to detect sound events in an audio recording. Effective AED techniques rely heavily on supervised or semi-supervised models to capture the wide range of dynamics spanned by sound events in order to achieve temporally precise boundaries and accurate event classification. These methods require extensive collections of labeled or weakly labeled in-domain data, which is costly and labor-intensive. Importantly, these approaches do not fully leverage the inherent variability and range of dynamics across sound events, aspects that can be effectively identified through unsupervised methods. The present work proposes an approach based on multi-rate autoencoders that are pretrained in an unsupervised way to leverage unlabeled audio data and ultimately learn the rich temporal dynamics inherent in natural sound events. This approach utilizes parallel autoencoders that achieve decompositions of the modulation spectrum along different bands. In addition, we introduce a rate-selective temporal contrastive loss to align the training objective with event detection metrics. Optimizing the configuration of multi-rate encoders and the temporal contrastive loss leads to notable improvements in domestic sound event detection in the context of the DCASE challenge.

Multimodal Vigilance Estimation With Modality-Pairwise Contrastive Loss.

Nowadays, how to estimate vigilance with higher accuracy has become a hot field of research direction. Although the increasing available modalities opens the door for amazing new possibilities to achieve good performance, the uncertain cross-modal interaction still poses a real challenge to the multimodal fusion. In this paper, a cross-modality alignment method has been proposed based on the contrastive learning for extracting shared but not the same information among modalities. The contrastive learning is adopted to minimize the intermodal differences by maximizing the similarity of semantic representation of modalities. Applying our proposed modeling framework, we evaluated our approach on SEED-VIG dataset consisting of EEG and EOG signals. Experiments showed that our study achieved state-of-the-art multimodal vigilance estimation performance both in intra-subject and inter-subject situations, the average of RMSE/CORR were improved to 0.092/0.893 and 0.144/0.887, respectively. In addition, analysis on the frequency bands showed that theta and alpha activities contain valuable information for vigilance estimation, and the correlation between them and PERCLOS can be significantly improved by contrastive learning. We argue that the proposed method in the inter-subject case could offer the possibility of reducing the high-cost of data annotation, and further analysis may provide an idea for the application of multimodal vigilance regression.

Enhancing action recognition from low-quality skeleton data via part-level knowledge distillation

Skeleton-based action recognition is vital for comprehending human-centric videos and has applications in diverse domains. One of the challenges of skeleton-based action recognition is dealing with low-quality data, such as skeletons that have missing or inaccurate joints. This paper addresses the issue of enhancing action recognition using low-quality skeletons through a general knowledge distillation framework. The proposed framework employs a teacher-student model setup, where a teacher model trained on high-quality skeletons guides the learning of a student model that handles low-quality skeletons. To bridge the gap between heterogeneous high-quality and low-quality skeletons, we present a novel part-based skeleton matching strategy, which exploits shared body parts to facilitate local action pattern learning. An action-specific part matrix is developed to emphasize critical parts for different actions, enabling the student model to distill discriminative part-level knowledge. A novel part-level multi-sample contrastive loss achieves knowledge transfer from multiple high-quality skeletons to low-quality ones, which enables the proposed knowledge distillation framework to include training low-quality skeletons that lack corresponding high-quality matches. Comprehensive experiments conducted on the NTU-RGB+D, Penn Action, and SYSU 3D HOI datasets demonstrate the effectiveness of the proposed knowledge distillation framework.