Irrelevant Regions Research Articles

Identity Retention and Emotion Converted StarGAN for low-resource emotional speaker recognition

In speaker recognition (SR) task, the inconsistency of emotion between enrolled speech and test speech will lead to emotion domain mismatch, which decreases the recognition accuracy and is more serious under low-resource training data condition. To help SR model learn the various emotion distribution when only neutral speech is enrolled, Identity Retention and Emotion Converted StarGAN (IREC-StarGAN) is proposed to generate speech feature in other emotions. Emotion and speaker relevant multi-task loss is designed to retain origin speaker information during adversarial training progress for the generation of emotion feature. To avoid essential speaker or emotion information being contaminated by irrelevant regions, generator uses SC-bottleneck with multi kernel size channels to learn the deep feature of data in various scales. Discriminator adopts densenet with two branches, one of which classifies whether input feature belongs to target emotion domain and the another one focuses on the speaker information. The origin neutral feature will be enrolled with generated feature together to be recognized by Resnet18 for speaker recognition task. The proposed framework obtains the EER of 18.33% on Mandarin Affective Speech Corpus and 8.14% on IEMOCAP, which outperform the existing embedding and GAN methods and reveals the robustness of IREC-StarGAN under low-resource emotion condition.

Assessing saliency models of observers' visual attention on acquired facial differences.

Saliency models that predict observers' visual attention to facial differences could enable psychosocial interventions to help patients and their families anticipate staring behaviors. The purpose of this study was to assess the ability of existing saliency models to predict observers' visual attention to acquired facial differences arising from head and neck cancer and its treatment. Saliency maps predicted by graph-based visual saliency (GBVS), an artificial neural network (ANN), and a face-specific model were compared to observer fixation maps generated from eye-tracking of lay observers presented with clinical facial photographs of patients with a visible or functional impairment manifesting in the head and neck region. We used a linear mixed-effects model to investigate observer and stimulus factors associated with the saliency models' accuracy. The GBVS model predicted many irrelevant regions (e.g., shirt collars) as being salient. The ANN model underestimated observers' attention to facial differences relative to the central region of the face. Compared with GBVS and ANN, the face-specific saliency model was more accurate on this task; however, the face-specific model underestimated the saliency of deviations from the typical structure of human faces. The linear mixed-effects model revealed that the location of the facial difference (midface versus periphery) was significantly associated with saliency model performance. Model performance was also significantly impacted by interobserver variability. Existing saliency models are not adequate for predicting observers' visual attention to facial differences. Extensions of face-specific saliency models are needed to accurately predict the saliency of acquired facial differences arising from head and neck cancer and its treatment.

Transformer tracking with multi-scale dual-attention

Transformer-based trackers greatly improve tracking success rate and precision rate. Attention mechanism in Transformer can fully explore the context information across successive frames. Nevertheless, it ignores the equally important local information and structured spatial information. And irrelevant regions may also affect the template features and search region features. In this work, a multi-scale feature fusion network is designed with box attention and instance attention in Encoder–Decoder architecture based on Transformer. After extracting features, the local information and structured spatial information is learnt by multi-scale box attention, and the global context information is explored by instance attention. Box attention samples grid features from the region of interest. Therefore, it effectively focuses on the region of interest (ROI) and avoids the influence of irrelevant regions in feature extraction. At the same time, instance attention can also pay attention to the context information across successive frames, and avoid falling into local optimum. The long-range feature dependencies are learned in this stage. Extensive experiments are conducted on six challenging tracking datasets to demonstrate the superiority of the proposed tracker MDTT, including UAV123, GOT-10k, LaSOT, VOT2018, TrackingNet, and NfS. In particular, the proposed tracker achieves AUC score of 64.7 % on LaSOT, 78.1 % on TrackingNet and precision score of 89.2 % on UAV123, which outperforms the baseline and most recent advanced trackers.

Mutually communicated model based on multi-parametric MRI for automated segmentation and classification of prostate cancer.

Multiparametric magnetic resonance imaging (mp-MRI) is introduced and established as a noninvasive alternative for prostate cancer (PCa) detection and characterization. To develop and evaluate a mutually communicated deep learning segmentation and classification network (MC-DSCN) based on mp-MRI for prostate segmentation and PCa diagnosis. The proposed MC-DSCN can transfer mutual information between segmentation and classification components and facilitate each other in a bootstrapping way. For classification task, the MC-DSCN can transfer the masks produced by the coarse segmentation component to the classification component to exclude irrelevant regions and facilitate classification. For segmentation task, this model can transfer the high-quality localization information learned by the classification component to the fine segmentation component to mitigate the impact of inaccurate localization on segmentation results. Consecutive MRI exams of patients were retrospectively collected from two medical centers (referred to as center A and B). Two experienced radiologists segmented the prostate regions, and the ground truth of the classification refers to the prostate biopsy results. MC-DSCN was designed, trained, and validated using different combinations of distinct MRI sequences as input (e.g., T2-weighted and apparent diffusion coefficient) and the effect of different architectures on the network's performance was tested and discussed. Data from center A were used for training, validation, and internal testing, while another center's data were used for external testing. The statistical analysis is performed to evaluate the performance of the MC-DSCN. The DeLong test and paired t-test were used to assess the performance of classification and segmentation, respectively. In total, 134 patients were included. The proposed MC-DSCN outperforms the networks that were designed solely for segmentation or classification. Regarding the segmentation task, the classification localization information helped to improve the IOU in center A: from 84.5% to 87.8% (p<0.01) and in center B: from 83.8% to 87.1% (p<0.01), while the area under curve (AUC) of PCa classification was improved in center A: from 0.946 to 0.991 (p<0.02) and in center B: from 0.926 to 0.955 (p<0.01) as a result of the additional information provided by the prostate segmentation. The proposed architecture could effectively transfer mutual information between segmentation and classification components and facilitate each other in a bootstrapping way, thus outperforming the networks designed to perform only one task.

EARDS: EfficientNet and attention-based residual depth-wise separable convolution for joint OD and OC segmentation.

Glaucoma is the leading cause of irreversible vision loss. Accurate Optic Disc (OD) and Optic Cup (OC) segmentation is beneficial for glaucoma diagnosis. In recent years, deep learning has achieved remarkable performance in OD and OC segmentation. However, OC segmentation is more challenging than OD segmentation due to its large shape variability and cryptic boundaries that leads to performance degradation when applying the deep learning models to segment OC. Moreover, the OD and OC are segmented independently, or pre-requirement is necessary to extract the OD centered region with pre-processing procedures. In this paper, we suggest a one-stage network named EfficientNet and Attention-based Residual Depth-wise Separable Convolution (EARDS) for joint OD and OC segmentation. In EARDS, EfficientNet-b0 is regarded as an encoder to capture more effective boundary representations. To suppress irrelevant regions and highlight features of fine OD and OC regions, Attention Gate (AG) is incorporated into the skip connection. Also, Residual Depth-wise Separable Convolution (RDSC) block is developed to improve the segmentation performance and computational efficiency. Further, a novel decoder network is proposed by combining AG, RDSC block and Batch Normalization (BN) layer, which is utilized to eliminate the vanishing gradient problem and accelerate the convergence speed. Finally, the focal loss and dice loss as a weighted combination is designed to guide the network for accurate OD and OC segmentation. Extensive experimental results on the Drishti-GS and REFUGE datasets indicate that the proposed EARDS outperforms the state-of-the-art approaches. The code is available at https://github.com/M4cheal/EARDS.

Pavement Crack Detection in Infrared Images Using a DCNN and CCL Algorithm

In-pavement crack detection, image segmentation techniques can accurately identify and extract cracks in pavement images with complex backgrounds, however, due to the interference of noise and non-crack damage, the resulting irrelevant non-crack regions reduce the extraction accuracy. To address the above problems, this research presents a road crack segmentation model consisting of a skip connection and decoder of U-Net, residual blocks in the encoding part, and the connected components labeling algorithm (CCL) in the extremity. The proposed method, depth-wise separable convolution replaces the standard convolution to reduce the number of parameters and operation cost. Post-processing using the connected components labeling algorithm solves the interference of irrelevant non-cracked regions. 4710 infrared images are used for the training and testing of the model. The testing results show that the new method for the pavement crack detection in different shadowed images is satisfactory, the detection accuracy can be up to 92.23%, and the algorithm comparison proves that the proposed algorithm is much better than that by the widely used traditional algorithms. In terms of similarity, the Dice Similarity Coefficient (DSC) is improved by 5.97%, 5.22%, and 1.49% compared to FCN, U-Net, and DeeplabV3+ networks, respectively. Therefore, the method is effective in detecting cracks in different pavement IR images, and can significantly improve the accuracy of segmentation results, reduce the error rate, improve the segmentation accuracy and robustness, and can extract cracks in pavement IR images well.

Semantic Completion and Filtration for Image–Text Retrieval

Image–text retrieval is a vital task in computer vision and has received growing attention, since it connects cross-modality data. It comes with the critical challenges of learning unified representations and eliminating the large gap between visual and textual domains. Over the past few decades, although many works have made significant progress in image–text retrieval, they are still confronted with the challenge of incomplete text descriptions of images, i.e., how to fully learn the correlations between relevant region–word pairs with semantic diversity. In this article, we propose a novel semantic completion and filtration (SCAF) method to alleviate the above issue. Specifically, the text semantic completion module is presented to generate a complete semantic description of an image using multi-view text descriptions, guiding the model to explore the correlations of relevant region–word pairs fully. Meanwhile, the adaptive structural semantic matching module is presented to filter irrelevant region–word pairs by considering the relevance score of each region–word pair, which facilitates the model to focus on learning the relevance of matching pairs. Extensive experiments show that our SCAF outperforms the existing methods on Flickr30K and MSCOCO datasets, which demonstrates the superiority of our proposed method.

Where you edit is what you get: Text-guided image editing with region-based attention

Leveraging the abundant knowledge learned from pre-trained multi-modal models like CLIP has recently proved to be effective for text-guided image editing. Though convincing results have been made when combining the image generator StyleGAN with CLIP, most methods need to train separate models for different prompts, and irrelevant regions are often changed after editing due to the lack of spatial disentanglement. We propose a novel framework that can edit different images according to different prompts in one model. Besides, an innovative region-based spatial attention mechanism is adopted to explicitly guarantee the locality of editing. Experiments mainly in the face domain verify the feasibility of our framework and show that when multi-text editing and local editing are accomplishable, our method can complete practical applications like sequential editing and regional style transfer.

SSAD: Single-Shot Multi-Scale Attentive Detector for Autonomous Driving

A novel Single-shot Multi-scale detection network with feature fusion and multi-scale attention mechanism is proposed for autonomous driving. The proposed model is referred to as a Single-shot Multi-scale Attentive Detector (SSAD) and it would build feature relations of the feature map in the spatial space. The proposed network highlights pedestrian and vehicle regions on the extracted feature map and also suppresses irrelevant regions, from the global relation information and thereby providing reliable guidance for autonomous driving, while detecting smaller and occluded targets. SSAD design is simple, accurate and computationally efficient. Evaluation results of the proposed network on multiple datasets have achieved considerable and promising results. Experimental results show that the SSAD network when tested on Pascal Voc-2007, INRIA, Caltech, and City Persons datasets, outperforms many state-of-the-art (SOTA) detectors.

Fast Attention CNN for Fine-Grained Crack Segmentation.

Deep learning-based computer vision algorithms, especially image segmentation, have been successfully applied to pixel-level crack detection. The prediction accuracy relies heavily on detecting the performance of fine-grained cracks and removing crack-like noise. We propose a fast encoder-decoder network with scaling attention. We focus on a low-level feature map by minimizing encoder-decoder pairs and adopting an Atrous Spatial Pyramid Pooling (ASPP) layer to improve the detection accuracy of tiny cracks. Another challenge is the reduction in crack-like noise. This introduces a novel scaling attention, AG+, to suppress irrelevant regions. However, removing crack-like noise, such as grooving, is difficult by using only improved segmentation networks. In this study, a crack dataset is generated. It contains 11,226 sets of images and masks, which are effective for detecting detailed tiny cracks and removing non-semantic objects. Our model is evaluated on the generated dataset and compared with state-of-the-art segmentation networks. We use the mean Dice coefficient (mDice) and mean Intersection over union (mIoU) to compare the performance and FLOPs for computational complexity. The experimental results show that our model improves the detection accuracy of fine-grained cracks and reduces the computational cost dramatically. The mDice score of the proposed model is close to the best score, with only a 1.2% difference but two times fewer FLOPs.

LDANet: Automatic lung parenchyma segmentation from CT images

Automatic segmentation of the lung parenchyma from computed tomography (CT) images is helpful for the subsequent diagnosis and treatment of patients. In this paper, based on a deep learning algorithm, a lung dense attention network (LDANet) is proposed with two mechanisms: residual spatial attention (RSA) and gated channel attention (GCA). RSA is utilized to weight the spatial information of the lung parenchyma and suppress feature activation in irrelevant regions, while the weights of each channel are adaptively calibrated using GCA to implicitly predict potential key features. Then, a dual attention guidance module (DAGM) is designed to maximize the integration of the advantages of both mechanisms. In addition, LDANet introduces a lightweight dense block (LDB) that reuses feature information and a positioned transpose block (PTB) that realizes accurate positioning and gradually restores the image resolution until the predicted segmentation map is generated. Experiments are conducted on two public datasets, LIDC-IDRI and COVID-19 CT Segmentation, on which LDANet achieves Dice similarity coefficient values of 0.98430 and 0.98319, respectively, outperforming a state-of-the-art lung segmentation model. Additionally, the effectiveness of the main components of LDANet is demonstrated through ablation experiments.

Multi-Level Residual Feature Fusion Network for Thoracic Disease Classification in Chest X-Ray Images

Motivation: Automated identification of thoracic diseases from chest X-ray images (CXR) is a significant area in computer-aided diagnosis. However, most existing methods have limited ability to extract multi-scale features and accurately capture the spatial location of lesions when dealing with thoracic diseases that exhibit concurrency and large variations in lesion size. Method: Based on the above problems, we propose a multi-level residual feature fusion network (MLRFNet) for classifying thoracic diseases. Our approach can quickly capture receptive field information across different lesion sizes and enhance disease-specific features within the spatial domain on feature maps. The MLRFNet comprises two main components: a feature extractor that learns multi-scale semantic information from chest X-ray images and a multi-level residual feature classifier (MRFC) that refines disease-specific pathological features at spatial locations to reduce interference from irrelevant regions. Additionally, the ECA attention modules connect both components to enable flexible channel-wise focus on critical pathological information. Results: We evaluated the performance of MLRFNet through a series of experiments on two datasets: ChestX-Ray14 and CheXpert. Our results show that MLRFNet achieves an average AUC of 0.853 on the ChestX-Ray14 dataset and 0.904 on the CheXpert dataset. Conclusion: The results of experiments demonstrate that our proposed method works better than the current state-of-the-art baselines. Future work will focus on investigating the interdependencies among labels for thoracic diseases and techniques for model compression.

Scene-Text Oriented Referring Expression Comprehension

Referring expression comprehension (REC) aims to identify and locate a specific object in visual scenes referred to by a natural language expression. Existing studies of REC only focus on basic visual attributes and neglect scene text. Since scene text has the functions of object identification and disambiguation, it is naturally and frequently used to refer to objects. However, existing methods do not explicitly recognize text in images and fail to align scene text mentioned in expressions with the text shown in images, resulting in object localization errors. This study takes the first step toward addressing these limitations. First, we introduce a new task called scene-text oriented referring expression comprehension, which aims to align visual cues and textual semantics of scene text with referring expressions and visual contents. Second, we propose a scene text awareness network that can bridge the gap between texts from two modalities by grounding visual representations of expression-correlated scene texts. Specifically, we propose a correlated text extraction module to solve the problem of lacking semantic understanding, and a correlated region activation module to address the fixed alignment problem and absent alignment problem. These modules ensure that the proposed method focuses on local regions that are most relevant to scene text, thus mitigating the misalignment of scene text with irrelevant regions. Third, to conduct quantitative evaluations, we establish a new benchmark dataset called RefText. Experimental results demonstrate that the proposed method can effectively comprehend scene-text oriented referring expressions and achieves excellent performance.

Adaptive Latent Graph Representation Learning for Image-Text Matching.

Image-text matching is a challenging task due to the modality gap. Many recent methods focus on modeling entity relationships to learn a common embedding space of image and text. However, these methods suffer from distractions of entity relationships such as irrelevant visual regions in an image and noisy textual words in a text. In this paper, we propose an adaptive latent graph representation learning method to reduce the distractions of entity relationships for image-text matching. Specifically, we use an improved graph variational autoencoder to separate the distracting factors and latent factor of relationships and jointly learn latent textual graph representations, latent visual graph representations, and a visual-textual graph embedding space. We also introduce an adaptive cross-attention mechanism to perform feature attending on the latent graph representations across images and texts, thus further narrowing the modality gap to boost the matching performance. Extensive experiments on two public datasets, Flickr30K and COCO, show the effectiveness of our method.

Image segmentation based on improved UNET++

In order to help researchers to perform cell segmentation, computer vision techniques are used to perform cell segmentation on neural cells. The addition of the Attention mechanism based on U-net++ suppresses the activity of irrelevant regions and improves the segmentation efficiency and segmentation accuracy. According to the experimental data, the segmentation accuracy of the method reaches 0.78 and the loss rate is 0.3, which can achieve a better segmentation effect compared with the traditional algorithm.

Detection of Detached Ice-fragments at Martian Polar Scarps Using a Convolutional Neural Network

Repeated high-resolution imaging has revealed current mass wasting in the form of ice block falls at steep scarps of Mars. However, both the accuracy and efficiency of ice-fragments’ detection are limited when using conventional computer vision methods. Existing deep learning methods suffer from the problem of shadow interference and indistinguishability between classes. To address these issues, we proposed a deep learning-driven change detection model that focuses on regions of interest. A convolutional neural network simultaneously analyzed bitemporal images, i.e., pre- and postdetach images. An augmented attention module was integrated in order to suppress irrelevant regions such as shadows while highlighting the detached ice-fragments. A combination of dice loss and focal loss was introduced to deal with the issue of imbalanced classes and hard, misclassified samples. Our method showed a true positive rate of 84.2% and a false discovery rate of 16.9%. Regarding the shape of the detections, the pixel-based evaluation showed a balanced accuracy of 85% and an F1 score of 73.2% for the detached ice-fragments. This last score reflected the difficulty in delineating the exact boundaries of some events both by a human and the machine. Compared with five state-of-the-art change detection methods, our method can achieve a higher F1 score and surpass other methods in excluding the interference of the changed shadows. Assessing the detections of the detached ice-fragments with the help of previously detected corresponding shadow changes demonstrated the capability and robustness of our proposed model. Furthermore, the good performance and quick processing speed of our developed model allow us to efficiently study large-scale areas, which is an important step in estimating the ongoing mass wasting and studying the evolution of the martian polar scarps.

Attention‐enhanced Alexnet for improved radar micro‐Doppler signature classification

AbstractThis work introduces an attention mechanism that can be integrated into any standard convolution neural network to improve model sensitivity and prediction accuracy with minimal computational overhead. The attention mechanism is introduced in a lightweight network – Alexnet and its classification performance for human micro‐Doppler signatures is evaluated. The Alexnet model trained with an attention module can implicitly highlight the salient regions in the radar signatures while suppressing the irrelevant background regions and consistently improving network predictions. Network visualizations are provided through class activation mapping, providing better insights into how the predictions are made. The visualizations demonstrate how the attention mechanism focusses on the region of interest in the radar signatures.

SAR-CGAN: Improved generative adversarial network for EIT reconstruction of lung diseases

The image reconstruction of electrical impedance tomography (EIT) is a nonlinear ill-posed inverse problem, and the reconstructed images tend to have artifacts due to noise in the measurement system. The shallow neural networks are difficult to fully express the nonlinear relationship of the EIT imaging problems. Deep neural networks have demonstrated great potential to remove artifacts from initial conductivity images caused by system noise. In the paper, a novel deep learning method, namely, conditional generative adversarial network with soft-attention gates (AGs) and residual learning (SAR-CGAN), is proposed to solve the above problems of EIT imaging. SAR-CGAN consists of generator and discriminator. Specifically, the AGs are employed by the generator to improve sensitivity and accuracy for lung EIT imaging by suppressing feature activation in irrelevant EIT region. And residual learning framework is introduced in the discriminator to address the problem of model degradation and easily enjoy accuracy gains from greatly increased depth. Extensive confirmatory and comparative experiments with three kinds of lung disease are conducted and compared with CNN, DnCNN, Attention U-Net and CGAN methods. The imaging results indicate that SAR-CGAN method can accurately recover the position and boundaries of lesions in lung images and effective reconstruction can be accomplished for the new conductivity distribution and noisy samples.

A Nested U-Shaped Residual Codec Network for Strip Steel Defect Detection

Strip steel is an important raw material for the related industries, such as aerospace, shipbuilding, and pipelines, and any quality defects in the strip steel would lead to huge economic losses. However, it is still a challenge task to effectively detect the defects from the background of the strip steel due to its complex variations, including variable flaws, chaotic background, and noise invasion. This paper proposes a novel strip steel defect detection method based on a U-shaped residual network, including an encoder and a decoder. The encoder is a fully convolutional neural network in which attention mechanisms are embedded to adequately extract multi-scale defect features and ro ignore irrelevant background regions. The decoder is a U-shaped residual network to capture more contextual data from different scales, without significantly increasing the computational cost due to the pooling operations used in the U-shaped network. Furthermore, a residual refinement module is designed immediately after the decoder to further optimize the coarse defect map. Experimental results show that the proposed method can effectively segment surface defect objects from irrelevant background noise and is superior to other advanced methods with clear boundaries.

Unnecessary radiation exposure during diagnostic radiography in infants in a neonatal intensive care unit: a retrospective cohort study

Unnecessary radiation exposure (URE) during radiographic examination is an issue among infants in neonatal intensive care units (NICUs). The causes of URE have not been fully explored. This study investigated the incidence and identified the causes of URE in infants during diagnostic radiography in a NICU. This was a retrospective cohort study. We retrieved and analysed requests and radiographs taken at a tertiary NICU between September and November 2018. URE was defined as the rate of discordance between requests and images taken (DisBRI) and unnecessary radiation exposure in irrelevant regions (UREIR) during radiography. We compared the rates of URE between very low-birth-weight (VLBW, birth weight < 1500 g) infants and non-VLBW infants. A total of 306 radiographs from 88 infants were taken. The means ± standard deviations (SDs) of gestational age and birth weight were 35.7 ± 3.6 weeks and 2471 ± 816 g, respectively. Each infant underwent an average of 3.5 radiographs. The DisBRI rate was 1.3% and was mostly related to poor adherence to requests. The UREIR rates in thoraco-abdominal babygrams were 89.6% for the head, 14.8% for the elbows and 18.4% for the knee and were mainly related to improper positioning of and collimation in infants while performing radiography. The UREIR rates for the head, knee and ankle were higher in VLBW infants than in non-VLBW infants (94.6% vs. 85.6%, 27.0% vs. 11.5% and 5.4% vs. 0.7%, respectively, p < 0.05).Conclusions: URE during diagnostic radiography is common in sick infants and is mainly related to improper positioning and collimation during examinations. Adherence to protocols when performing radiographic examination or using ultrasonography may be a solution to reduce URE in infants in NICUs.What is Known:• The risk of unnecessary radiation exposure (URE) during radiography has been a common and important issue in sick infants in neonatal intensive care units (NICUs).• The new point-of-care ultrasound (POCUS) technique decreases the need for chest films and prevents radiation exposure in neonates.What is New:• In the NICU, URE is still a common issue in critically ill infants during radiographic examinations. The causes of URE during diagnostic radiography are mainly due to improper positioning and collimation during examinations.• The incidence of URE in irrelevant regions is higher in very low-birth-weight (VLBW) infants than in non-VLBW infants.