Source Code Research Articles

Novel hybrid deep learning based cyber security threat detection model with optimization algorithm

In order to continuously provide services to the company, the Internet of Things (IoT) connects the hardware, software, storing data, and applications that could be utilized as a new port of entry for cyber-attacks. The privacy of IoT is presently very vulnerable to virus threats and software piracy. Threats like this have the potential to capture critical data, harming businesses' finances and reputations. We have suggested a hybrid Deep Learning (DL) strategy in this study to identify malware-infected programs and files that have been illegally distributed over the IoT environment. To detect illegal content utilizing Source code (SC) duplication, the Adaptive TensorFlow deep neural network with Improved Particle Swarm Optimization (IPSO) is suggested. This novel hybrid strategy improves cyber security by fusing cutting-edge DL with optimization methods, providing more effective and accurate detection. With a strong solution for real-time threat identification, the model handles the complexity of contemporary cyberthreats. To highlight the significance of the proxy regarding the SC duplication, the noisy data is filtered using the tokenization and weighting feature approaches. After that, duplication in SC is found using a DL method. To look into software piracy, the dataset was gathered via Google Code Jam (GCJ). Additionally, using the visual representation of color images, the Enhanced Long Short-Term Memory (E-LSTM) was employed to identify suspicious actions in the IoT environment. The Maling dataset is used to gather the malware samples required for testing. The experimental findings show that, in terms of categorization, the suggested method for evaluating cybersecurity threats in IoT surpasses conventional approaches.

VSPACE: exploring virtual spatial representation of articular chondrocytes at the single-cell level.

vSPACE is a web-based application presenting a spatial representation of scRNAseq data obtained from human articular cartilage by emulating the concept of spatial transcriptomics technology, but virtually. This virtual 2D plot presentation of human articular cartage cells generates several zonal distribution patterns, for one or multiple genes at a time, revealing patterns that scientists can appreciate as imputed spatial distribution patterns along the zonal axis. vSPACE is implemented in Python Dash as a web-based toolbox designed for data visualization of zonal gene expression patterns in articular cartilage chondrocytes. This tool is freely accessible at: https://vspace.cse.uconn.edu/The source code and extra materials for this service can be downloaded from: https://github.com/zhacheny/vSPACE.

Efficient license plate recognition in unconstrained scenarios

Automatic license plate recognition (ALPR) is a critical technology for intelligent transportation systems. Most existing ALPR methods are focused on specific application scenarios. Although there are a few methods that focus on unconstrained scenarios, they are very time-consuming. In this work, we propose an efficient ALPR (EALPR) framework, where we can handle distorted license plates (LP) caused by perspective problems with high efficiency. We design a light LPD structure based on efficient object detection methods and use anchor-free strategies for LPD to alleviate the problem of expensive costs. Benefitting from these optimizations and a united framework structure, the proposed EALPR has real-time efficiency. We evaluate our method on five datasets and the results show that our method achieves state-of-the-art accuracy: 98.15% on OpenALPR(EU), 95.61% on OpenALPR(BR), 99.51% on AOLP(RP), 88.81% on SSIG, 79.41% on CD-HARD. Additionally, our method achieves an impressive speed of 74.9 FPS (Frames Per Second), outperforming existing approaches and demonstrating its efficiency. Our source code can be accessed at https://github.com/wechao18/Efficient-alpr-unconstrained.

A wavelet CNN with appropriate feed-allocation and PSO optimized activations for diabetic retinopathy grading

This work modifies the architecture of conventional CNN with the integration of Multi-resolution Analysis (MRA) in a CNN framework for Diabetic Retinopathy (DR) diagnosis and grading. Here, the HF sub-bands are subjected to optimized activations and are directly fed to the fully connected layers, as it encompasses edge features. Unlike FD-Relu, the proposed function preserves significant negative coefficients, compared to the S-Relu, the proposed third-order S-Relu is optimized such that it sustains the activations in the range suitable for the wavelet coefficients. The coefficients of higher-order terms of the proposed 3rd-order S-Relu are optimized with PSO, fitting the maximum energy of the wavelet sub-bands to ensure High Frequency (HF) edge preservation. The authors re-architecture 3 different CNNs published in the Retinal Image analysis field, with spatial and wavelet inputs with optimized activations. The highest accuracy of 96% is attained with the AlexNet re-architecture, with 35,126 fundus images secured from the Kaggle dataset. As we can infer the proposed re-architecture wavelet CNN outperformed the multiscale shallow CNNs, multiscale attention net, and stacked CNNs with a 6.6, 0.3, 0.7 per cent increase in accuracy. The entire implementation of the wavelet CNN is made available under source code.

Exploring sample relationship for few-shot classification

Few-shot classification (FSC) is a challenging problem, which aims to identify novel classes with limited samples. Most existing methods employ vanilla transfer learning or episodic meta-training to learn a feature extractor, and then measure the similarity between the query image and the few support examples of novel classes. However, these approaches merely learn feature representations from individual images, overlooking the exploration of the interrelationships among images. This neglect can hinder the attainment of more discriminative feature representations, thus limiting the potential improvement of few-shot classification performance. To address this issue, we propose a Sample Relationship Exploration (SRE) module comprising the Sample-level Attention (SA), Explicit Guidance (EG) and Channel-wise Adaptive Fusion (CAF) components, to learn discriminative category-related features. Specifically, we first employ the SA component to explore the similarity relationships among samples and obtain aggregated features of similar samples. Furthermore, to enhance the robustness of these features, we introduce the EG component to explicitly guide the learning of sample relationships by providing an ideal affinity map among samples. Finally, the CAF component is adopted to perform weighted fusion of the original features and the aggregated features, yielding category-related embeddings. The proposed method is a plug-and-play module which can be embedded into both transfer learning and meta-learning based few-shot classification frameworks. Extensive experiments on benchmark datasets show that the proposed module can effectively improve the performance over baseline models, and also perform competitively against the state-of-the-art algorithms. The source code is available at https://github.com/Chenguoz/SRE.

MSDAFL: molecular substructure-based dual attention feature learning framework for predicting drug-drug interactions.

Drug-drug interactions (DDIs) can cause unexpected adverse drug reactions, affecting treatment efficacy and patient safety. The need for computational methods to predict DDIs has been growing due to the necessity of identifying potential risks associated with drug combinations in advance. Although several deep learning methods have been recently proposed to predict DDIs, many overlook feature learning based on interactions between the substructures of drug pairs. In this work, we introduce a molecular Substructure-based Dual Attention Feature Learning framework (MSDAFL), designed to fully utilize the information between substructures of drug pairs to enhance the performance of DDI prediction. We employ a self-attention module to obtain a set number of self-attention vectors, which are associated with various substructural patterns of the drug molecule itself, while also extracting interaction vectors representing inter-substructure interactions between drugs through an interactive attention module. Subsequently, an interaction module based on cosine similarity is used to further capture the interactive characteristics between the self-attention vectors of drug pairs. We also perform normalization after the interaction feature extraction to mitigate overfitting. After applying three-fold cross-validation, the MSDAFL model achieved average precision scores of 0.9707, 0.9991, and 0.9987, and area under the receiver operating characteristic curve scores of 0.9874, 0.9934, and 0.9974 on three datasets, respectively. In addition, the experiment results of five-fold cross-validation and cross-datum study also indicate that MSDAFL performs well in predicting DDIs. Data and source codes are available at https://github.com/27167199/MSDAFL.

Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection

Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, a critical yet often overlooked issue is that GNNs primarily rely on aggregating information from adjacent nodes, limiting structural information transfer to single-layer updates. In code graphs, nodes and relationships typically require cross-layer information propagation to fully capture complex program logic and potential vulnerability patterns. Furthermore, while some studies utilize codeLMs to supplement GNNs with code semantic information, existing integration methods have not fully explored the potential of their collaborative effects.To address these challenges, we introduce Vul-LMGNNs that integrates pre-trained CodeLMs with GNNs, leveraging knowledge distillation to facilitate cross-layer propagation of both code semantic knowledge and structural information. Specifically, Vul-LMGNNs utilizes Code Property Graphs (CPGs) to incorporate code syntax, control flow, and data dependencies, while employing gated GNNs to extract structural information in the CPG. To achieve cross-layer information transmission, we implement an online knowledge distillation (KD) program that enables a single student GNN to acquire structural information extracted from a simultaneously trained counterpart through an alternating training procedure. Additionally, we leverage pre-trained CodeLMs to extract semantic features from code sequences. Finally, we propose an ”implicit-explicit” joint training framework to better leverage the strengths of both CodeLMs and GNNs. In the implicit phase, we utilize CodeLMs to initialize the node embeddings of each student GNN. Through online knowledge distillation, we facilitate the propagation of both code semantics and structural information across layers. In the explicit phase, we perform linear interpolation between the CodeLM and the distilled GNN to learn a late fusion model. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to 17 state-of-the-art approaches. Our source code can be accessed via GitHub: https://github.com/Vul-LMGNN/vul-LMGGNN.

Learning to explain is a good biomedical few-shot learner.

Significant progress has been achieved in biomedical text mining using deep learning methods, which rely heavily on large amounts of high-quality data annotated by human experts. However, the reality is that obtaining high-quality annotated data is extremely challenging due to data scarcity (e.g. rare or new diseases), data privacy and security concerns, and the high cost of data annotation. Additionally, nearly all researches focus on predicting labels without providing corresponding explanations. Therefore, in this paper, we investigate a more realistic scenario, biomedical few-shot learning, and explore the impact of interpretability on biomedical few-shot learning. We present LetEx-Learning to explain-a novel multi-task generative approach that leverages reasoning explanations from large language models (LLMs) to enhance the inductive reasoning ability of few-shot learning. Our approach includes (1) collecting high-quality explanations by devising a suite of complete workflow based on LLMs through CoT prompting and self-training strategies, (2) converting various biomedical NLP tasks into a text-to-text generation task in a unified manner, where collected explanations serve as additional supervision between text-label pairs by multi-task training. Experiments are conducted on three few-shot settings across six biomedical benchmark datasets. The results show that learning to explain improves the performances of diverse biomedical NLP tasks in low-resource scenario, outperforming strong baseline models significantly by up to 6.41%. Notably, the proposed method makes the 220M LetEx perform superior reasoning explanation ability against LLMs. Our source code and data are available at https://github.com/cpmss521/LetEx.

A Lightweight Recurrent Learning Network for Sustainable Compressed Sensing

Recently, deep learning-based compressed sensing (CS) has achieved great success in reducing the sampling and computational cost of sensing systems and improving the reconstruction quality. These approaches, however, largely overlook the issue of the computational cost; they rely on complex structures and task-specific operator designs, resulting in extensive storage and high energy consumption in CS imaging systems. In this article, we propose a lightweight but effective deep neural network based on recurrent learning to achieve a sustainable CS system; it requires a smaller number of parameters but obtains high-quality reconstructions. Specifically, our proposed network consists of an initial reconstruction sub-network and a residual reconstruction sub-network. While the initial reconstruction sub-network has a hierarchical structure to progressively recover the image, reducing the number of parameters, the residual reconstruction sub-network facilitates recurrent residual feature extraction via recurrent learning to perform both feature fusion and deep reconstructions across different scales. In addition, we also demonstrate that, after the initial reconstruction, feature maps with reduced sizes are sufficient to recover the residual information, and thus we achieved a significant reduction in the amount of memory required. Extensive experiments illustrate that our proposed model can achieve a better reconstruction quality than existing state-of-the-art CS algorithms, and it also has a smaller number of network parameters than these algorithms. Our source codes are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/C66YU/CSRN</uri> .

NAS-BNN: Neural Architecture Search for Binary Neural Networks

Binary Neural Networks (BNNs) have gained extensive attention for their superior inferencing efficiency and compression ratio compared to traditional full-precision networks. However, due to the unique characteristics of BNNs, designing a powerful binary architecture is challenging and often requires significant manpower. A promising solution is to utilize Neural Architecture Search (NAS) to assist in designing BNNs, but current NAS methods for BNNs are relatively straightforward and leave a performance gap between the searched models and manually designed ones. To address this gap, we propose a novel neural architecture search scheme for binary neural networks, named NAS-BNN. We first carefully design a search space based on the unique characteristics of BNNs. Then, we present three training strategies, which significantly enhance the training of supernet and boost the performance of all subnets. Our discovered binary model family outperforms previous BNNs for a wide range of operations (OPs) from 20M to 200M. For instance, we achieve 68.20% top-1 accuracy on ImageNet with only 57M OPs. In addition, we validate the transferability of these searched BNNs on the object detection task, and our binary detectors with the searched BNNs achieve a novel state-of-the-art result, e.g., 31.6% mAP with 370M OPs, on MS COCO dataset. The source code and models will be released at https://github.com/VDIGPKU/NAS-BNN.

Rethinking Pan-Sharpening in Closed-Loop Regularization.

It is generally known that pan-sharpening is fundamentally a PAN-guided multispectral (MS) image super-resolution problem that involves learning the nonlinear mapping from low-resolution (LR) to high-resolution (HR) MS images. Since an infinite number of HR-MS images can be downsampled to produce the same corresponding LR-MS image, learning the mapping from LR-MS to HR-MS image is typically ill-posed and the space of the possible pan-sharpening functions can be extremely large, making it difficult to estimate the optimal mapping solution. To address the above issue, we propose a closed-loop scheme that learns the two opposite mapping including the pan-sharpening and its corresponding degradation process simultaneously to regularize the solution space in a single pipeline. More specifically, an invertible neural network (INN) is introduced to perform a bidirectional closed-loop: the forward operation for LR-MS pan-sharpening and the backward operation for learning the corresponding HR-MS image degradation process. In addition, given the vital importance of high-frequency textures for the Pan-sharpened MS images, we further strengthen the INN by designing a specified multiscale high-frequency texture extraction module. Extensive experimental results demonstrate that the proposed algorithm performs favorably against state-of-the-art methods qualitatively and quantitatively with fewer parameters. Ablation studies also verify the effectiveness of the closed-loop mechanism in pan-sharpening. The source code is made publicly available at https://github.com/manman1995/pan-sharpening-Team-zhouman/.

MAS-DGAT-Net: A dynamic graph attention network with multibranch feature extraction and staged fusion for EEG emotion recognition

In recent years, with the rise of deep learning technologies, EEG-based emotion recognition has garnered significant attention. However, most existing methods tend to focus on the spatiotemporal information of EEG signals while overlooking the potential topological information of brain regions. To address this issue, this paper proposes a dynamic graph attention network with multi-branch feature extraction and staged fusion (MAS-DGAT-Net), which integrates graph convolutional neural networks (GCN) for EEG emotion recognition. Specifically, the differential entropy (DE) features of EEG signals are first reconstructed into a correlation matrix using the Spearman correlation coefficient. Then, the brain-region connectivity-feature extraction (BCFE) module is employed to capture the brain connectivity features associated with emotional activation states. Meanwhile, this paper introduces a dual-branch cross-fusion feature extraction (CFFE) module, which consists of an attention-based cross-fusion feature extraction branch (A-CFFEB) and a cross-fusion feature extraction branch (CFFEB). A-CFFEB efficiently extracts key channel-frequency information from EEG features by using an attention mechanism and then fuses it with the output features from the BCFE. The fused features are subsequently input into the proposed dynamic graph attention module with a broad learning system (DGAT-BLS) to mine the brain connectivity feature information further. Finally, the deep features output by DGAT-BLS and CFFEB are combined for emotion classification. The proposed algorithm has been experimentally validated on SEED, SEED-IV, and DEAP datasets in subject-dependent and subject-independent settings, with the results confirming the model's effectiveness. The source code is publicly available at: https://github.com/cvmdsp/MAS-DGAT-Net

Automated model discovery for textile structures: The unique mechanical signature of warp knitted fabrics

Textile fabrics have unique mechanical properties, which make them ideal candidates for many engineering and medical applications: They are initially flexible, nonlinearly stiffening, and ultra-anisotropic. Various studies have characterized the response of textile structures to mechanical loading; yet, our understanding of their exceptional properties and functions remains incomplete. Here we integrate biaxial testing and constitutive neural networks to automatically discover the best model and parameters to characterize warp knitted polypropylene fabrics. We use experiments from different mounting orientations, and discover interpretable anisotropic models that perform well during both training and testing. Our study shows that constitutive models for warp knitted fabrics are highly sensitive to an accurate representation of the textile microstructure, and that models with three microstructural directions outperform classical orthotropic models with only two in-plane directions. Strikingly, out of 214=16,384 possible combinations of terms, we consistently discover models with two exponential linear fourth invariant terms that inherently capture the initial flexibility of the virgin mesh and the pronounced nonlinear stiffening as the loops of the mesh tighten. We anticipate that the tools we have developed and prototyped here will generalize naturally to other textile fabrics–woven or knitted, weft knit or warp knit, polymeric or metallic–and, ultimately, will enable the robust discovery of anisotropic constitutive models for a wide variety of textile structures. Beyond discovering constitutive models, we envision to exploit automated model discovery for the generative material design of wearable devices, stretchable electronics, and smart fabrics, as programmable textile metamaterials with tunable properties and functions. Our source code, data, and examples are available at https://github.com/LivingMatterLab/CANN. Statement of significanceTextile structures are rapidly gaining popularity in many biomedical applications including tissue engineering, wound healing, and surgical repair. A precise understanding of their unique mechanical properties is critical to tailor them to their specific functions. Here we integrate mechanical testing and machine learning to automatically discover the best models for knitted polypropylene fabrics. We show that warp knitted fabrics possess a complex symmetry with three distinct microstructural directions. Along these, the behavior is dominated by an exponential linear term that characterize the initial flexibility of the virgin mesh and the nonlinear stiffening as the loops of the fabric tighten. We expect that our technology will generalize naturally to other fabrics and enable the robust discovery of complex anisotropic models for a wide variety of textile structures.

Nearshore optical video object detector based on temporal branch and spatial feature enhancement

The computing power of nearshore and ship-borne devices is limited, posing significant challenges for accurately detecting objects in real-time on such devices. We propose a nearshore video object detector (NVID) to tackle these challenges. Considering the abundance of dynamic entities in the nearshore environment, we have developed you can look more (YCLM) to perceive the temporal characteristics of these objects. Furthermore, to improve the ability to detect objects of different sizes of networks, we designed parallel deformable attention (PDA) based on the spatial features of objects. More importantly, we developed fast re-parameterization convolution (FREConv) and faster conv (FConv). Building on these innovations, we proposed a fast re-parameterization network (FRENet) specifically tailored to produce low-parameter, multi-scale feature outputs. With end-to-end training, our pipeline outperforms other state-of-the-art (SOTA) methods on the nearshore objects (NearshoreObjects) dataset (90.4 average precision (AP) 50 (＋4.7), 9.3 parameters (Params) (−1.0M), 24.8 frames per second (FPS) (Jetson Nano) (＋0.6)). In addition, NVID also achieved excellent results in the on board (OnBoard) dataset (90.3 AP50 (＋2.8), 9.3 params (−1.0M), 26.5 FPS (Jetson Nano) (＋0.8)). The source code can be accessed at https://github.com/Yuanlin-Zhao/NVID.

DACG: Dual Attention and Context Guidance model for radiology report generation

Medical images are an essential basis for radiologists to write radiology reports and greatly help subsequent clinical treatment. The task of generating automatic radiology reports aims to alleviate the burden of clinical doctors writing reports and has received increasing attention this year, becoming an important research hotspot. However, there are severe issues of visual and textual data bias and long text generation in the medical field. Firstly, Abnormal areas in radiological images only account for a small portion, and most radiological reports only involve descriptions of normal findings. Secondly, there are still significant challenges in generating longer and more accurate descriptive texts for radiology report generation tasks. In this paper, we propose a new Dual Attention and Context Guidance (DACG) model to alleviate visual and textual data bias and promote the generation of long texts. We use a Dual Attention Module, including a Position Attention Block and a Channel Attention Block, to extract finer position and channel features from medical images, enhancing the image feature extraction ability of the encoder. We use the Context Guidance Module to integrate contextual information into the decoder and supervise the generation of long texts. The experimental results show that our proposed model achieves state-of-the-art performance on the most commonly used IU X-ray and MIMIC-CXR datasets. Further analysis also proves that our model can improve reporting through more accurate anomaly detection and more detailed descriptions. The source code is available at https://github.com/LangWY/DACG.

Simultaneous Optimization of Discrete and Continuous Parameters Defining a Robot Morphology and Controller.

The morphology and controller design of robots is often a labor-intensive task performed by experienced and intuitive engineers. Automatic robot design using machine learning is attracting increasing attention in the hope that it will reduce the design workload and result in better-performing robots. Most robots are created by joining several rigid parts and then mounting actuators and their controllers. Many studies limit the possible types of rigid parts to a finite set to reduce the computational burden. However, this not only limits the search space, but also prohibits the use of powerful optimization techniques. To find a robot closer to the global optimal design, a method that explores a richer set of robots is desirable. In this article, we propose a novel method to efficiently search for various robot designs. The method combines three different optimization methods with different characteristics. We apply proximal policy optimization (PPO) or soft actor-critic (SAC) as the controller, the REINFORCE algorithm to determine the lengths and other numerical parameters of the rigid parts, and a newly proposed method to determine the number and layout of the rigid parts and joints. Experiments with physical simulations confirm that when this method is used to handle two types of tasks-walking and manipulation-it performs better than simple combinations of existing methods. The source code and videos of our experiments are available online (https://github.com/r-koike/eagent).

An integrative analysis to identify pancancer epigenetic biomarkers

Integrating and analyzing the pancancer data collected from different experiments is crucial for gaining insights into the common mechanisms in the molecular level underlying the development and progression of cancers. Epigenetic study of the pancancer data can provide promising results in biomarker discovery. The genes that are epigenetically dysregulated in different cancers are powerful biomarkers for drug-related studies. This paper identifies the genes having altered expression due to aberrant methylation patterns using differential analysis of TCGA pancancer data of 12 different cancers. We identified a comprehensive set of 115 epigenetic biomarker genes out of which 106 genes having pancancer properties. The correlation analysis, gene set enrichment, protein-protein interaction analysis, pancancer characteristics analysis, and diagnostic modeling were performed on these biomarkers to illustrate the power of this signature and found to be important in different molecular operations related to cancer. An accuracy of 97.56% was obtained on TCGA pancancer gene expression dataset for predicting the binary class tumor or normal. The source code and dataset of this work are available at https://github.com/panchamisuneeth/EpiPanCan.git.

A three-stage model for camouflaged object detection

Camouflaged objects are typically assimilated into their backgrounds and exhibit fuzzy boundaries. The complex environmental conditions and the high intrinsic similarity between camouflaged targets and their surroundings pose significant challenges in accurately locating and segmenting these objects in their entirety. While existing methods have demonstrated remarkable performance in various real-world scenarios, they still face limitations when confronted with difficult cases, such as small targets, thin structures, and indistinct boundaries. Drawing inspiration from human visual perception when observing images containing camouflaged objects, we propose a three-stage model that enables coarse-to-fine segmentation in a single iteration. Specifically, our model employs three decoders to sequentially process subsampled features, cropped features, and high-resolution original features. This proposed approach not only reduces computational overhead but also mitigates interference caused by background noise. Furthermore, considering the significance of multi-scale information, we have designed a multi-scale feature enhancement module that enlarges the receptive field while preserving detailed structural cues. Additionally, a boundary enhancement module has been developed to enhance performance by leveraging boundary information. Subsequently, a mask-guided fusion module is proposed to generate fine-grained results by integrating coarse prediction maps with high-resolution feature maps. Our network shows superior performance without introducing unnecessary complexities. Upon acceptance of the paper, the source code will be made publicly available at https://github.com/clelouch/TSNet.

R-Net: Recursive decoder with edge refinement network for salient object detection

Fully convolutional neural (FCN) networks based on encoder-decoder structures are adopted for salient object detection (SOD) and achieve state-of-the-art performance. However, most of the previous works aggregate multilevel features layer by layer and ignore long-range dependencies of spatial context, leading to imbalanced feature processes and context separation. In addition, the blurry edges obtained from previous SOD methods inevitably influence the detection accuracy. To handle these problems, this study proposes an effective recursion-based SOD network R-Net adopting a partial cascade structure, which contains a recursive decoder module (RDM), a long-range dependency module (LRDM), and an edge refinement module (ERM). The RDM is composed of three subdecoders, which can scale different level features to the same size through recursive pooling and recursive upsampling for full-scale feature fusion. The LRDM bridges the encoder and decoder by weighting multilevel features to establish the long-range feature dependency of the spatial context. And the ERM attached to the decoder introduces the shallow reference feature to refine the blurry edges for obtaining delicate detection results. The SOD experimental results on DUTS-TE, HKU-IS, PASCAL-S, ECSSD, and DUT-OMRON datasets demonstrate that the proposed R-Net is more robust under different complex scenes compared with some state-of-the-art methods. The source codes are available at https://github.com/ZEROICEWANG/R-Net.

HSMix: Hard and soft mixing data augmentation for medical image segmentation

Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. Additionally, traditional data augmentation methods for local image editing augmentation methods generally utilize square regions, which cause a loss of contour information.We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images.Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in https://github.com/DanielaPlusPlus/HSMix.