Symmetric Encoder Research Articles

Utilizing machine learning techniques for enhanced water quality monitoring

ABSTRACT Water quality is an important issue for environmental health. It directly impacts human well-being, ecosystem sustainability and socioeconomic development. This paper provides an overview for water quality assesment by integrating traditional methods with computational technology. Dimensionality reduction is considered an essential preprocessing step in any data analysis task which can be performed by using either feature selection or feature extraction methods. In this study, we propose an autoencoder-based feature selection method that can be used with both labeled and unlabeled data. It can be implemented with an arbitrary number of hidden layers in the symmetric encoder part of the autoencoder and provides results that compare favorably with the results provided by computationally more expensive methods and also provides a quantitatively ordered rank of features for the features in the dataset. Also, our proposed method for water quality assessment has demonstrated remarkable success in efficiently managing and interpreting complex datasets, offering a promising pathway toward effective environmental stewardship and sustainable water resource management. Through its implementation, we aim to contribute to the preservation and protection of water quality for the benefit of present and future generations.

MAS-Net: Multi-Attention Hybrid Network for Superpixel Segmentation

Superpixels, as essential mid-level image representations, have been widely used in computer vision due to their computational efficiency and redundant compression. Compared with traditional superpixel methods, superpixel algorithms based on deep learning frameworks demonstrate significant advantages in segmentation accuracy. However, existing deep learning-based superpixel algorithms suffer from a loss of details due to convolution and upsampling operations in their encoder–decoder structure, which weakens their semantic detection capabilities. To overcome these limitations, we propose a novel superpixel segmentation network based on a multi-attention hybrid network (MAS-Net). MAS-Net is still based on an efficient symmetric encoder–decoder architecture. First, utilizing residual structure based on a parameter-free attention module at the feature encoding stage enhanced the capture of fine-grained features. Second, adoption of a global semantic fusion self-attention module was used at the feature selection stage to reconstruct the feature map. Finally, fusing the channel with the spatial attention mechanism at the feature-decoding stage was undertaken to obtain superpixel segmentation results with enhanced boundary adherence. Experimental results on real-world image datasets demonstrated that the proposed method achieved competitive results in terms of visual quality and metrics, such as ASA and BR-BP, compared with the state-of-the-art approaches.

3D-ISRNet:Generating 3D point clouds through image similarity retrieval in a complex background from a single image

Reconstructing the 3D shape of objects using limited information from a single image is a complex and necessary task. The existing methods for reconstructing point clouds from a single image haven't effectively addressed the issue of background interference in the input image, thus greatly limited their practical application. To address the underlying issue, this paper proposes a 3D-ISRNet aiming to enhance the accuracy of point cloud reconstruction through image similarity retrieval. The 3D-ISRNet retrieves point clouds similar to the object and employs spatial and channel attention mechanisms for image encoding. This aims to enhance the features of the reconstruction area and improve reconstruction accuracy. After the fusion of retrieved point clouds with image features, the combined data undergoes a symmetric encoder to generate the reconstructed point clouds, thereby alleviating the constraints imposed by the initial point clouds on the reconstruction results. 3D-ISRNet demonstrates favorable results on public and real-world datasets.

Representation Learning Based on Vision Transformer

In recent years, with the rapid development of information technology, the volume of image data has grown exponentially. However, these datasets typically contain a large amount of redundant information. To extract effective features and reduce redundancy from images, a representation learning method based on the Vision Transformer (ViT) has been proposed, and to our best knowledge, Transformer was first applied to zero-shot learning (ZSL). The method adopts a symmetric encoder–decoder structure, where the encoder incorporates Multi-Head Self-Attention (MSA) mechanism of ViT to reduce the dimensionality of image features, eliminate redundant information, and decrease computational burden. Consequently, it effectively extracts features, and the decoder is utilized for reconstructing image data. We evaluated the representation learning capability of the proposed method in various tasks, including data visualization, image reconstruction, face recognition, and ZSL. By comparing with state-of-the-art representation learning methods, the outstanding results obtained validate the effectiveness of this method in the field of representation learning.

Efficient image restoration with style-guided context cluster and interaction

Recently, convolutional neural networks (CNNs) and vision transformers (ViTs) have emerged as powerful tools for image restoration (IR). Nonetheless, they encountered some limitations due to their characteristics, such as CNNs sacrificing global reception and ViTs requiring large memory and graphics resources. To address these limitations and explore an alternative approach for improved IR performance, we propose two clustering-based frameworks for general IR tasks, which are style-guided context cluster U-Net (SCoC-UNet) and style-guided clustered point interaction U-Net (SCPI-UNet). The SCoC-UNet adopts a U-shaped architecture, comprising position embedding, Encoder, Decoder, and reconstruction block. Specifically, the input low-quality image is viewed as a set of unorganized points, each of which is first given location information by the continuous relative position embedding method. These points are then fed into a symmetric Encoder and Decoder which utilize style-guided context cluster (SCoC) blocks to extract potential context features and high-frequency information. Although SCoC-UNet has obtained decent performance for image restoration, its SCoC block can only capture connectivity at points within the same cluster, which may ignore long-range dependencies in different clusters. To address this issue, we further propose a SCPI-UNet based on SCoC-UNet, which leverages a style-guided clustered point interaction (SCPI) block in place of the SCoC block. The SCPI block utilizes a cross-attention mechanism to establish the connections of feature points between different clusters. Extensive experimental results demonstrate that the proposed SCoC-UNet and SCPI-UNet can handle several typical IR tasks (i.e., JPEG compression artifact reduction, image denoising, and super-resolution) and achieve superior quantitative and qualitative performance over some state-of-the-art methods.

Research on Pectoral Muscle Segmentation Algorithm of CT Image Based on Deep Learning.

Chronic obstructive pulmonary disease (COPD) is a common respiratory disease, which seriously endangers human health and is also one of the important causes of death. The death rate of COPD in China is the highest in the world, and the problem of under-diagnosis of the disease is very serious. The gold standard for the diagnosis of COPD is lung function examination, and clinical studies have shown that CT and other imaging methods can be included in the auxiliary diagnosis of COPD. CT images can be used to assess pectoral muscle area, which is associated with COPD severity. Patients with lower pectoral muscle area often have more severe expiratory airflow obstruction and other problems. Therefore, the key of the research is to accurately segment the pectoral muscle in CT images. The medical image segmentation method based on deep learning can dig out more abundant internal information of data, so it has gradually become the preferred method in the aspect of medical image segmentation. In this paper, a pectoral muscle segmentation algorithm based on U-Net and its variant U-Net++ is proposed, which is of great significance for evaluating the severity of disease in patients with COPD. The network is composed of symmetrical encoders and decoders, which can effectively learn from very little labelled data by using appropriate data enhancement methods, and is therefore very suitable for medical image segmentation. The experimental results on the data set provided by Jiangsu Province Hospital show that the average Dice coefficient of the proposed algorithm is more than 94% and the average accuracy rate is 91%. The algorithm can accurately segment the pectoral muscle in CT images and has good segmentation performance.

Semantic Segmentation of High-Resolution Remote Sensing Images with Improved U-Net Based on Transfer Learning

Semantic segmentation of high-resolution remote sensing images has emerged as one of the foci of research in the remote sensing field, which can accurately identify objects on the ground and determine their localization. In contrast, the traditional deep learning-based semantic segmentation, on the other hand, requires a large amount of annotated data, which is unsuitable for high-resolution remote sensing tasks with limited resources. It is therefore important to build a semantic segmentation method for high-resolution remote sensing images. In this paper, it is proposed an improved U-Net model based on transfer learning to solve the semantic segmentation problem of high-resolution remote sensing images. The model is based on the symmetric encoder–decoder structure of U-Net. For the encoder, transfer learning is applied and VGG16 is used as the backbone of the feature extraction network, and in the decoder, after upsampling using bilinear interpolation, it is performed multiscale fusion with the feature maps of the corresponding layers of the encoder in turn and is finally obtained the predicted value of each pixel to achieve precise localization. To verify the efficacy of the proposed network, experiments are performed on the ISPRS Vaihingen dataset. The experiments show that the applied method has achieved high-quality semantic segmentation results on the high-resolution remote sensing dataset, and the MIoU is 1.70%, 2.20%, and 2.33% higher on the training, validation, and test sets, respectively, and the IoU is 4.26%, 6.89%, and 5.44% higher for the automotive category compared to the traditional U-Net.

A fusion‐attention swin transformer for cardiac MRI image segmentation

AbstractFor semantic segmentation of cardiac magnetic resonance image (MRI) with low recognition and high background noise, a fusion‐attention Swin Transformer is proposed based on cognitive science and deep learning methods. It has a U‐shaped symmetric encoding–decoding structure with an attention‐based skip connection. The encoder realizes self‐attention for deep feature representation and the decoder up‐samples global features to the corresponding input resolution for pixel‐level segmentation. By introducing a skip connection between the encoder and decoder based on fusion attention, the remote interaction of global information is realized, and the attention to local features and specific channels is enhanced. A public ACDC cardiac MRI image dataset is used for experiments. The segmentation of the left ventricle, right ventricle, and myocardial layer is realized. The method performs well on a small sample dataset, for example, the pixel accuracy obtained by the proposed model is 93.68%, the Dice coefficient is 92.28%, and HD coefficient is 11.18. Compared with the state‐of‐the‐art models, the segmentation precision has been significantly improved, especially for the low recognition and heavily occluded targets.

OCT layer segmentation using U-NET semantic segmentation and RESNET34 encoder-decoder

Images using OCT (Optical Coherence Tomography) for producing cross-sections of images of retina light-sensitive tissue linings behind the human eye's black portions. Segmentation provides a better understanding of the retinal anatomy, which is essential in planning and evaluating treatment options. By measuring retinal thickness and analyzing the arrangement of retinal tissue. A semantic segmentation model based on U-Net deep learning model is proposed. It uses ResNet34 architecture as an encoder and decoder for efficient feature extraction from the input images. A complete convolution encoder-decoder network called U-Net has skip links between the blocks that deal with symmetric encoding and decoding. The proposed framework can segment the layers in the OCT scan accurately.As the light returns to the scanner from the retina, it provides fine-grained cross-sectional photographs of the retina.The effectiveness of the suggested strategy was evaluated at two benchmarks: DeepLabV3Plus and UnetP++.Accuracy and mean Intersection over Union (mIoU) were two of the typical pixel-wise measures that were taken into consideration while evaluating the models' effectiveness. The future enhancement of this work (1) Add other performance evaluation metrics like transmission time, precision, and recall, (2) Deep learning approach use optimization algorithms to improve accuracy value.

MST-UNet: a modified Swin Transformer for water bodies’ mapping using Sentinel-2 images

Deep learning is widely used in remote sensing field of feature recognition. Symmetric encoder–decoder network, such as UNet, is one of the most commonly used image segmentation networks, but the accuracy is often low due to its simple structure. We combine two neural network models of convolutional neural network (CNN) and Swin Transformer called modified Swin Transformer using UNet structure (MST-UNet) to achieve accurate segmentation of water bodies from remote sensing data, with Xiamen City as study area. MST-UNet is based on symmetric encoder–decoder network. We use CNN and Swin Transformer blocks to extract features from input images and capture the interdependence among different pixels, respectively. More attention is paid to global information of images. By four times upsampling to obtain predictions, the results show that the accuracy of MST-UNet is better than UNet and its improved models. The Intersection of Union (IoU), mean IoU, and Dice score on test set reach 87.80%, 92.93%, 93.08%, respectively, which verifies the feasibility of the MST-UNet. This experiment has a reference value for related studies.

Image dehazing algorithm based on ViTGAN and its application

In recent years, many methods based on deep learning have emerged in the field of image dehazing. The ViTGAN generation adversarial network based on Transformer is used to design image dehazing algorithm.In ViTGAN, the generator is improved to achieve the desired effect.The first half of the collaterals can be used as a data encoder to extract the features of the original image. In this part, a lower sampling layer is placed behind each density block.Generate network can regard as the second half of a corresponding to the first encoder decoder, in this part, each layer of the sampling density behind block placed a, for the first half is passed on the image characteristics of sampling, can be reduced by the sampling image characteristics to enlarge to the image of the original size, to ensure that the network output is correct.The feature map of the decoder and the symmetric encoder can be fused to ensure that the features of the decoder can be properly represented.

A Semi-Supervised Semantic Segmentation Method for Blast-Hole Detection

The goal of blast-hole detection is to help place charge explosives into blast-holes. This process is full of challenges, because it requires the ability to extract sample features in complex environments, and to detect a wide variety of blast-holes. Detection techniques based on deep learning with RGB-D semantic segmentation have emerged in recent years of research and achieved good results. However, implementing semantic segmentation based on deep learning usually requires a large amount of labeled data, which creates a large burden on the production of the dataset. To address the dilemma that there is very little training data available for explosive charging equipment to detect blast-holes, this paper extends the core idea of semi-supervised learning to RGB-D semantic segmentation, and devises an ERF-AC-PSPNet model based on a symmetric encoder–decoder structure. The model adds a residual connection layer and a dilated convolution layer for down-sampling, followed by an attention complementary module to acquire the feature maps, and uses a pyramid scene parsing network to achieve hole segmentation during decoding. A new semi-supervised learning method, based on pseudo-labeling and self-training, is proposed, to train the model for intelligent detection of blast-holes. The designed pseudo-labeling is based on the HOG algorithm and depth data, and proved to have good results in experiments. To verify the validity of the method, we carried out experiments on the images of blast-holes collected at a mine site. Compared to the previous segmentation methods, our method is less dependent on the labeled data and achieved IoU of 0.810, 0.867, 0.923, and 0.945, at labeling ratios of 1/8, 1/4, 1/2, and 1.

Semantic SLAM Based on Deep Learning in Endocavity Environment

Traditional endoscopic treatment methods restrict the surgeon’s field of view. New approaches to laparoscopic visualization have emerged due to the advent of robot-assisted surgical techniques. Lumen simultaneous localization and mapping (SLAM) technology can use the image sequence taken by the endoscope to estimate the pose of the endoscope and reconstruct the lumen scene in minimally invasive surgery. This technology gives the surgeon better visual perception and is the basis for the development of surgical navigation systems as well as medical augmented reality. However, the movement of surgical instruments in the internal cavity can interfere with the SLAM algorithm, and the feature points extracted from the surgical instruments may cause errors. Therefore, we propose a modified endocavity SLAM method combined with deep learning semantic segmentation that introduces a convolution neural network based on U-Net architecture with a symmetric encoder–decoder structure in the visual odometry with the goals of solving the binary segmentation problem between surgical instruments and the lumen background and distinguishing dynamic feature points. Its segmentation performance is improved by using pretrained encoders on the network model to obtain more accurate pixel-level instrument segmentation. In this setting, the semantic segmentation is used to reject the feature points on the surgical instruments and reduce the impact caused by dynamic surgical instruments. This can provide more stable and accurate mapping results compared to ordinary SLAM systems.

DGAN-KPN: Deep Generative Adversarial Network and Kernel Prediction Network for Denoising MC Renderings

In this paper, we present a denoising network composed of a kernel prediction network and a deep generative adversarial network to construct an end-to-end overall network structure. The network structure consists of three parts: the Kernel Prediction Network (KPN), the Deep Generation Adversarial Network (DGAN), and the image reconstruction model. The kernel prediction network model takes the auxiliary feature information image as the input, passes through the source information encoder, the feature information encoder, and the kernel predictor, and finally generates a prediction kernel for each pixel. The generated adversarial network model is divided into two parts: the generator model and the multiscale discriminator model. The generator model takes the noisy Monte Carlo-rendered image as the input, passes through the symmetric encoder–decoder structure and the residual block structure, and finally outputs the rendered image with preliminary denoising. Then, the prediction kernel and the preliminarily denoised rendered image is sent to the image reconstruction model for reconstruction, and the prediction kernel is applied to the preliminarily denoised rendered image to obtain a preliminarily reconstructed result image. To further improve the quality of the result and to be more robust, the initially reconstructed rendered image undergoes four iterations of filtering for further denoising. Finally, after four iterations of the image reconstruction model, the final denoised image is presented as the output. This denoised image is applied to the loss function. We compared the results from our approach with state-of-the-art results by using the structural similarity index (SSIM) values and peak signal-to-noise ratio (PSNR) values, and we reported a better performance.

Segmentation of lung airways based on deep learning methods

Precise segmentation of the lung airways is essential for a quantitative assessment of airway diseases. However, because of the complexity of the airway structure and the different thicknesses of the trachea at different positions, it is extremely difficult to segment the fine bronchial structure using chest computed tomography (CT). Traditional lung airway segmentation methods are generally based on the grayscale, geometric shape of the image, or the use of prior knowledge of anatomy. In recent years, deep learning techniques such as fully convolutional neural networks (FCNs) have achieved a great success in the field of image segmentation. Specifically, the symmetric encoder–decoder network represented by U-Net has achieved high accuracy in many medical image segmentation tasks. In the airway segmentation challenge task of the 4th International Symposium on Image Computing and Digital Medicine (ISICDM 2020), 9 of the 12 teams participating in the final round used the U-Net network or one of its other forms, obtaining good results for lung airway segmentation. Methods used to improve the segmentation accuracy include attention mechanisms and multiscale feature information fusion. This article provides a detailed description of the methods used by these 12 teams and analyses their results.

Ground-Based Cloud Detection Using Multiscale Attention Convolutional Neural Network

Cloud detection plays a significant role in ground-based remote sensing observation, and it is quite challenging due to the variations in illumination and cloud form, and the vague boundaries between cloud and sky. In this letter, we propose a novel deep model named multiscale attention convolutional neural network (MACNN) for ground-based cloud detection, which possesses a symmetric encoder–decoder structure. For accurate cloud detection, we design the multiscale module in MACNN to obtain different receptive fields by using different hole rates for the filters, and meanwhile, we propose the attention module in MACNN to learn the attention coefficients in order to reflect different importance of pixels. Furthermore, we release the Tianjin Normal University (TJNU) cloud detection database (TCDD) to provide a comparative study for different methods, and to the best of our knowledge, it is the largest cloud detection database. We conduct a series of experiments on the TCDD, and the experimental results demonstrate that the proposed MACNN outperforms state-of-the-art methods in five quantitative evaluation criteria.

Applying GMEI-GAN to Generate Meaningful Encrypted Images in Reversible Data Hiding Techniques

With the rapid development of information technology, the transmission of information has become convenient. In order to prevent the leakage of information, information security should be valued. Therefore, the data hiding technique has become a popular solution. The reversible data hiding technique (RDH) in particular uses symmetric encoding and decoding algorithms to embed the data into the cover carrier. Not only can the secret data be transmitted without being detected and retrieved completely, but the cover carrier also can be recovered without distortion. Moreover, the encryption technique can protect the carrier and the hidden data. However, the encrypted carrier is a form of ciphertext, which has a strong probability to attract the attention of potential attackers. Thus, this paper uses the generative adversarial networks (GAN) to generate meaningful encrypted images for RDH. A four-stage network architecture is designed for the experiment, including the hiding network, the encryption/decryption network, the extractor, and the recovery network. In the hiding network, the secret data are embedded into the cover image through residual learning. In the encryption/decryption network, the cover image is encrypted into a meaningful image, called the marked image, through GMEI-GAN, and then the marked image is restored to the decrypted image via the same architecture. In the extractor, 100% of the secret data are extracted through the residual learning framework, same as the hiding network. Lastly, in the recovery network, the cover image is reconstructed with the decrypted image and the retrieved secret data through the convolutional neural network. The experimental results show that using the PSNR/SSIM as the criteria, the stego image reaches 45.09 dB/0.9936 and the marked image achieves 38.57 dB/0.9654. The proposed method not only increases the embedding capacity but also maintains high image quality in the stego images and marked images.

Deep Convolutional Symmetric Encoder—Decoder Neural Networks to Predict Students’ Visual Attention

Prediction of visual attention is a new and challenging subject, and to the best of our knowledge, there are not many pieces of research devoted to the anticipation of students’ cognition when solving tests. The aim of this paper is to propose, implement, and evaluate a machine learning method that is capable of predicting saliency maps of students who participate in a learning task in the form of quizzes based on quiz questionnaire images. Our proposal utilizes several deep encoder–decoder symmetric schemas which are trained on a large set of saliency maps generated with eye tracking technology. Eye tracking data were acquired from students, who solved various tasks in the sciences and natural sciences (computer science, mathematics, physics, and biology). The proposed deep convolutional encoder–decoder network is capable of producing accurate predictions of students’ visual attention when solving quizzes. Our evaluation showed that predictions are moderately positively correlated with actual data with a coefficient of 0.547 ± 0.109. It achieved better results in terms of correlation with real saliency maps than state-of-the-art methods. Visual analyses of the saliency maps obtained also correspond with our experience and expectations in this field. Both source codes and data from our research can be downloaded in order to reproduce our results.

Multimodal medical image segmentation using multi-scale context-aware network

Multimodal medical image segmentation with different imaging devices is a key but challenging task in medical image visual analysis and reasoning. Recently, U-Net based networks achieved considerable success in semantic segmentation of medical image. However, U-Net utilizes a skip-connection to connect two symmetric encoder and decoder layers. Although the single granularity information of the encoder layer is preserved through skip connection, the rich multi-scale spatial information is ignored, which greatly affects its performance in the segmentation task. In this paper, a multi-scale context-aware network (CA-Net) for multimodal medical image segmentation is proposed, which captures rich context information with dense skip connection and assigns distinct weights to different channels. CA-Net consists of four key components, namely encoder module, multi-scale context fusion (MCF) module, decoder module, and dense skip connection module. The proposed MCF module extracts multi-scale spatial information through a spatial context fusion (SCF) block, and learn to balance channel-wise features through a Squeeze-and-Excitation (SE) block. Extensive experiments demonstrate that our model achieves state-of-the-art performance on three benchmark datasets of different modalities, including skin lesion segmentation in dermoscopy, lung segmentation in CT images, and blood vessel segmentation in retina images.

SEDRFuse: A Symmetric Encoder–Decoder With Residual Block Network for Infrared and Visible Image Fusion

Image fusion is an important task for computer vision as a diverse range of applications are benefiting from the fusion operation. The existing image fusion methods are largely implemented at the pixel level, which may introduce artifacts and/or inconsistencies, while the computational complexity is relatively high. In this article, we propose a symmetric encoder–decoder with residual block (SEDRFuse) network to fuse infrared and visible images for night vision applications. At the training stage, the SEDRFuse network is trained to create a fixed feature extractor. At the fusing stage, the trained extractor is utilized to extract the intermediate and compensation features, which are generated by the residual block and the first two convolutional layers from the input source images, respectively. Two attention maps, which are derived from the intermediate features, are then multiplied by the intermediate features for fusion. The salient compensation features obtained through elementwise selection are passed to the corresponding deconvolutional layers for processing. Finally, the fused intermediate features and the selected compensation features are decoded to reconstruct the fused image. Experimental results demonstrate that the proposed fusion solution, i.e., SEDRFuse, outperforms the state-of-the-art fusion methods in terms of both subjective and objective evaluations.