Encoder-decoder Paradigm Research Articles

XDTEncoder: A Deep Explainable Arrhythmia Classification Framework for Smart Healthcare

In the context of smart healthcare, the integration of multimedia and digital twin technologies has driven significant advances in telemedicine. Electrocardiogram (ECG) signals, as a part of multimedia healthcare data, provide crucial digital information about the electrical activity of the heart, which is essential for diagnosing arrhythmias and ensuring a healthy life. Arrhythmia classification is a fundamental step in analyzing ECG signals and a critical problem for diagnosing heart diseases. One key challenge in arrhythmia classification is the lack of high accuracy for classifying arrhythmia heartbeats, and another key challenge is the lack of interpretability of decision-making models. This study aims to develop a novel approach to improve the performance of arrhythmia classification while providing explainable diagnostic decision paths. We propose XDTEncoder, an explainable arrhythmia classification framework that leverages multi-level features to classify arrhythmia heartbeats while offering an explainable diagnostic decision path. XDTEncoder is novel in three aspects. (1) It constructs a human-machine collaborative knowledge representation based on the Encoder-Decoder paradigm, which allows our model to classify arrhythmias while producing decision paths for cardiologists. (2) XDTEncoder compares two encoding methods (the binary tree encoding method and the Huffman encoding method) to embed the diagnostic decision tree into the arrhythmia classification framework. (3) XDTEncoder fuses multi-level features to improve the performance of arrhythmias classification. Evaluation on 5 types of arrhythmias in the MIT-BIH database demonstrates that our new approach outperforms state-of-the-art classifiers while providing interpretability.

UIDF-Net: Unsupervised Image Dehazing and Fusion Utilizing GAN and Encoder-Decoder.

Haze weather deteriorates image quality, causing images to become blurry with reduced contrast. This makes object edges and features unclear, leading to lower detection accuracy and reliability. To enhance haze removal effectiveness, we propose an image dehazing and fusion network based on the encoder-decoder paradigm (UIDF-Net). This network leverages the Image Fusion Module (MDL-IFM) to fuse the features of dehazed images, producing clearer results. Additionally, to better extract haze information, we introduce a haze encoder (Mist-Encode) that effectively processes different frequency features of images, improving the model's performance in image dehazing tasks. Experimental results demonstrate that the proposed model achieves superior dehazing performance compared to existing algorithms on outdoor datasets.

Memory-Based Cross-Modal Semantic Alignment Network for Radiology Report Generation.

Generating radiology reports automatically reduces the workload of radiologists and helps the diagnoses of specific diseases. Many existing methods take this task as modality transfer process. However, since the key information related to disease accounts for a small proportion in both image and report, it is hard for the model to learn the latent relation between the radiology image and its report, thus failing to generate fluent and accurate radiology reports. To tackle this problem, we propose a memory-based cross-modal semantic alignment model (MCSAM) following an encoder-decoder paradigm. MCSAM includes a well initialized long-term clinical memory bank to learn disease-related representations as well as prior knowledge for different modalities to retrieve and use the retrieved memory to perform feature consolidation. To ensure the semantic consistency of the retrieved cross modal prior knowledge, a cross-modal semantic alignment module (SAM) is proposed. SAM is also able to generate semantic visual feature embeddings which can be added to the decoder and benefits report generation. More importantly, to memorize the state and additional information while generating reports with the decoder, we use learnable memory tokens which can be seen as prompts. Extensive experiments demonstrate the promising performance of our proposed method which generates state-of-the-art performance on the MIMIC-CXR dataset.

Graph-Collaborated Auto-Encoder Hashing for Multiview Binary Clustering.

Unsupervised hashing methods have attracted widespread attention with the explosive growth of large-scale data, which can greatly reduce storage and computation by learning compact binary codes. Existing unsupervised hashing methods attempt to exploit the valuable information from samples, which fails to take the local geometric structure of unlabeled samples into consideration. Moreover, hashing based on auto-encoders aims to minimize the reconstruction loss between the input data and binary codes, which ignores the potential consistency and complementarity of multiple sources data. To address the above issues, we propose a hashing algorithm based on auto-encoders for multiview binary clustering, which dynamically learns affinity graphs with low-rank constraints and adopts collaboratively learning between auto-encoders and affinity graphs to learn a unified binary code, called graph-collaborated auto-encoder (GCAE) hashing for multiview binary clustering. Specifically, we propose a multiview affinity graphs' learning model with low-rank constraint, which can mine the underlying geometric information from multiview data. Then, we design an encoder-decoder paradigm to collaborate the multiple affinity graphs, which can learn a unified binary code effectively. Notably, we impose the decorrelation and code balance constraints on binary codes to reduce the quantization errors. Finally, we use an alternating iterative optimization scheme to obtain the multiview clustering results. Extensive experimental results on five public datasets are provided to reveal the effectiveness of the algorithm and its superior performance over other state-of-the-art alternatives.

Enhancing COVID-19 CT Image Segmentation: A Comparative Study of Attention and Recurrence in UNet Models.

Imaging plays a key role in the clinical management of Coronavirus disease 2019 (COVID-19) as the imaging findings reflect the pathological process in the lungs. The visual analysis of High-Resolution Computed Tomography of the chest allows for the differentiation of parenchymal abnormalities of COVID-19, which are crucial to be detected and quantified in order to obtain an accurate disease stratification and prognosis. However, visual assessment and quantification represent a time-consuming task for radiologists. In this regard, tools for semi-automatic segmentation, such as those based on Convolutional Neural Networks, can facilitate the detection of pathological lesions by delineating their contour. In this work, we compared four state-of-the-art Convolutional Neural Networks based on the encoder-decoder paradigm for the binary segmentation of COVID-19 infections after training and testing them on 90 HRCT volumetric scans of patients diagnosed with COVID-19 collected from the database of the Pisa University Hospital. More precisely, we started from a basic model, the well-known UNet, then we added an attention mechanism to obtain an Attention-UNet, and finally we employed a recurrence paradigm to create a Recurrent-Residual UNet (R2-UNet). In the latter case, we also added attention gates to the decoding path of an R2-UNet, thus designing an R2-Attention UNet so as to make the feature representation and accumulation more effective. We compared them to gain understanding of both the cognitive mechanism that can lead a neural model to the best performance for this task and the good compromise between the amount of data, time, and computational resources required. We set up a five-fold cross-validation and assessed the strengths and limitations of these models by evaluating the performances in terms of Dice score, Precision, and Recall defined both on 2D images and on the entire 3D volume. From the results of the analysis, it can be concluded that Attention-UNet outperforms the other models by achieving the best performance of 81.93%, in terms of 2D Dice score, on the test set. Additionally, we conducted statistical analysis to assess the performance differences among the models. Our findings suggest that integrating the recurrence mechanism within the UNet architecture leads to a decline in the model's effectiveness for our particular application.

GAGPT-2: A Geometric Attention-based GPT-2 Framework for Image Captioning in Hindi

Image captioning frameworks usually employ an encoder-decoder paradigm, with the encoder receiving abstract image feature vectors as input and the decoder for language modeling. Nowadays, most prominent architectures employ features from region proposals derived from object detection modules. In this work, we propose a novel architecture for image captioning. We employ the object detection module integrated with transformer architecture as an encoder and GPT-2 (Generative Pre-trained Transformer) as a decoder. The encoder utilizes the information of the spatial relationships among detected objects. We introduce a unique methodology for image caption generation in Hindi, which is widely spoken in South Asia and India and is the world’s third most spoken language as well as India’s official language. In terms of BLEU scores, the proposed approach’s performance is comparable to those of other baselines, and the results illustrate that the proposed approach outperforms the other baselines. The efficacy of the proposed approach in generating correct captions is further determined by human assessment in terms of adequacy and fluency.

Just Noticeable Visual Redundancy Forecasting: A Deep Multimodal-Driven Approach

Just noticeable difference (JND) refers to the maximum visual change that human eyes cannot perceive, and it has a wide range of applications in multimedia systems. However, most existing JND approaches only focus on a single modality, and rarely consider the complementary effects of multimodal information. In this article, we investigate the JND modeling from an end-to-end homologous multimodal perspective, namely hmJND-Net. Specifically, we explore three important visually sensitive modalities, including saliency, depth, and segmentation. To better utilize homologous multimodal information, we establish an effective fusion method via summation enhancement and subtractive offset, and align homologous multimodal features based on a self-attention driven encoder-decoder paradigm. Extensive experimental results on eight different benchmark datasets validate the superiority of our hmJND-Net over eight representative methods.

Artificial intelligence of things (AIoT) data acquisition based on graph neural networks: A systematical review

SummaryThe power of artificial intelligence of things (AIoT) stems from adapting machine learning (ML) and artificial intelligence (AI) models into abundant intelligent IoT fields, based on a large data stream with different formats, sizes, and timestamps generated by massive numbers of heterogeneous sensors. On the one hand, data acquisition is the fundamental basis for any AIoT systems, but data sensed by massive IoT devices may be noisy and even contain adversarial samples. On the other hand, ensuring the efficiency and robustness in data acquisition is vitally important for data‐driven ML and AI. Recently, besides perceiving ability, the literature has witnessed great development of empowering things with learning and reasoning ability through deep learning models, including recurrent neural networks (RNNs) and/or convolutional neural network (CNNs). However, the existing works have one significant weakness: fail to explicitly leverage the geospatial implications and latent connections among sensors for high‐quality data acquisition and quality control. Graphs are intrinsically suitable for representing the dependencies and inter‐relationships between AIoT data sensing devices. Due to the ability of capturing the complex interactive relationships between nodes and producing high‐level representations of the graph input, graph neural networks (GNNs) have exploded onto various ML and AI fields, to learn from graph‐structured data. Our review covers the latest progresses in GNN for the fundamental atomic task of data acquisition in AIoT. Instead of surveying the abundant GNN schemes in vertically various IoT sensing applications, this paper systematically reviews the horizontal infrastructure that all AIoT fields should have, that is, AIoT data acquisition, based on GNN and other related emerging AI factors. Our contributions include the following aspects: Provide the latest progresses in GNN for the horizontal task of data acquisition in AIoT, propose the unified GNN pipeline based on encoder–decoder paradigm, and systematically categorize and summarize the emerging technologies helpful to address the issues in AIoT data acquisition, especially the noisy and adversarial data, and point out some future directions about GNN‐based AIoT data acquisition.

CapERA: Captioning Events in Aerial Videos

In this paper, we introduce the CapERA dataset, which upgrades the Event Recognition in Aerial Videos (ERA) dataset to aerial video captioning. The newly proposed dataset aims to advance visual–language-understanding tasks for UAV videos by providing each video with diverse textual descriptions. To build the dataset, 2864 aerial videos are manually annotated with a caption that includes information such as the main event, object, place, action, numbers, and time. More captions are automatically generated from the manual annotation to take into account as much as possible the variation in describing the same video. Furthermore, we propose a captioning model for the CapERA dataset to provide benchmark results for UAV video captioning. The proposed model is based on the encoder–decoder paradigm with two configurations to encode the video. The first configuration encodes the video frames independently by an image encoder. Then, a temporal attention module is added on the top to consider the temporal dynamics between features derived from the video frames. In the second configuration, we directly encode the input video using a video encoder that employs factorized space–time attention to capture the dependencies within and between the frames. For generating captions, a language decoder is utilized to autoregressively produce the captions from the visual tokens. The experimental results under different evaluation criteria show the challenges of generating captions from aerial videos. We expect that the introduction of CapERA will open interesting new research avenues for integrating natural language processing (NLP) with UAV video understandings.

The Application of Artificial Intelligence and Big Data Technology in Basketball Sports Training

INTRODUCTION: Basketball involves a wide variety of complex human motions. Thus, recognizing them with Precision is essential for both training and competition. The subjective perceptions and experiences of the trainers are heavily relied upon while training players. Big data and Artificial Intelligence (AI) technology may be utilized to track athlete training. Sensing their motions may also help instructors make choices that dramatically improve athletic ability. OBJECTIVES: This research paper developed an Action Recognition technique for teaching basketball players using Big Data, and CapsNet called ARBIGNet METHODS: The technique uses a network that is trained using large amounts of data from basketball games called a Whale Optimized Artificial Neural Network (WO-ANN) which is collected using capsules. In order to determine the spatiotemporal information aspects of basketball sports training from videos, this study first employs the Convolution Random Forest (ConvRF) unit. The second accomplishment of this study is creating the Attention Random Forest (AttRF) unit, which combines the RF with the attention mechanism. The study used big data analytics for fast data transmissions. The unit scans each site randomly, focusing more on the region where the activity occurs. The network architecture is then created by enhancing the standard encoder-decoder paradigm. Then, using the Enhanced Darknet network model, the spatiotemporal data in the video is encoded. The AttRF structure is replaced by the standard RF at the decoding step. The ARBIGNet architecture is created by combining these components. RESULTS: The efficiency of the suggested strategy implemented on action recognition in basketball sports training has been tested via experiments, which have yielded 95.5% mAP and 98.8% accuracy.

Retrieval Augmented Convolutional Encoder-decoder Networks for Video Captioning

Video captioning has been an emerging research topic in computer vision, which aims to generate a natural sentence to correctly reflect the visual content of a video. The well-established way of doing so is to rely on encoder-decoder paradigm by learning to encode the input video and decode the variable-length output sentence in a sequence-to-sequence manner. Nevertheless, these approaches often fail to produce complex and descriptive sentences as natural as those from human being, since the models are incapable of memorizing all visual contents and syntactic structures in the human-annotated video-sentence pairs. In this article, we uniquely introduce a Retrieval Augmentation Mechanism (RAM) that enables the explicit reference to existing video-sentence pairs within any encoder-decoder captioning model. Specifically, for each query video, a video-sentence retrieval model is first utilized to fetch semantically relevant sentences from the training sentence pool, coupled with the corresponding training videos. RAM then writes the relevant video-sentence pairs into memory and reads the memorized visual contents/syntactic structures in video-sentence pairs from memory to facilitate the word prediction at each timestep. Furthermore, we present Retrieval Augmented Convolutional Encoder-Decoder Network (R-ConvED), which novelly integrates RAM into convolutional encoder-decoder structure to boost video captioning. Extensive experiments on MSVD, MSR-VTT, Activity Net Captions, and VATEX datasets validate the superiority of our proposals and demonstrate quantitatively compelling results.

Abstractive text summarization of low-resourced languages using deep learning.

Humans must be able to cope with the huge amounts of information produced by the information technology revolution. As a result, automatic text summarization is being employed in a range of industries to assist individuals in identifying the most important information. For text summarization, two approaches are mainly considered: text summarization by the extractive and abstractive methods. The extractive summarisation approach selects chunks of sentences like source documents, while the abstractive approach can generate a summary based on mined keywords. For low-resourced languages, e.g., Urdu, extractive summarization uses various models and algorithms. However, the study of abstractive summarization in Urdu is still a challenging task. Because there are so many literary works in Urdu, producing abstractive summaries demands extensive research. This article proposed a deep learning model for the Urdu language by using the Urdu 1 Million news dataset and compared its performance with the two widely used methods based on machine learning, such as support vector machine (SVM) and logistic regression (LR). The results show that the suggested deep learning model performs better than the other two approaches. The summaries produced by extractive summaries are processed using the encoder-decoder paradigm to create an abstractive summary. With the help of Urdu language specialists, the system-generated summaries were validated, showing the proposed model's improvement and accuracy.

Automated urban planning aware spatial hierarchies and human instructions

Traditional urban planning demands urban experts to spend much time producing an optimal urban plan under many architectural constraints. The remarkable imaginative ability of deep generative learning provides hope for renovating this domain. Existing works are constrained by: (1) neglecting human requirements; (2) omitting spatial hierarchies, and (3) lacking urban plan samples. We propose a novel, deep human-instructed urban planner to fill these gaps and implement two practical frameworks. In the preliminary version, we formulate the task into an encoder–decoder paradigm. The encoder is to learn the information distribution of surrounding contexts, human instructions, and land-use configuration. The decoder is to reconstruct the land-use configuration and the associated urban functional zones. Although it has achieved good results, the generation performance is still unstable due to the complex optimization directions of the decoder. Thus, we propose a cascading deep generative adversarial network (GAN) in this paper, inspired by the workflow of urban experts. The first GAN is to build urban functional zones based on human instructions and surrounding contexts. The second GAN will produce the land-use configuration by considering the built urban functional zones. Finally, we conducted extensive experiments and case studies to validate the effectiveness and superiority of our work.

Partial-to-Partial Point Generation Network for Point Cloud Completion

Point cloud completion aims at predicting dense complete 3D shapes from sparse incomplete point clouds captured from 3D sensors or scanners. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Existing point cloud completion methods follow the encoder-decoder paradigm, in which the complete point clouds are recovered in a coarse-to-fine strategy. However, only using the global feature is difficult and will lead to blurring of the global structure and distortion of local details. To address this problem, we propose a novel Partial-to-Partial Point Generation Network ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet), a learning-based approach for point cloud completion. In <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet, we use a feature disentangle encoder to obtain the global feature, and missing code and novel view partial point cloud are generated conditioned on the view-related missing code. To better aggregate partial point clouds, an attentive sampling module is proposed to sample multiple partial point clouds into the final complete result. Extensive experiments on several public benchmarks demonstrate that our <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet outperforms state-of-the-art point cloud completion methods.

High-Resolution Representation Learning and Recurrent Neural Network for Singing Voice Separation

Music source separation has traditionally followed the encoder-decoder paradigm (e.g., hourglass, U-Net, DeconvNet, SegNet) to isolate individual music components from mixtures. Such networks, however, result in a loss of location-sensitivity, as low-resolution representation drops the useful harmonic patterns over the temporal dimension. We overcame this problem by performing singing voice separation using a high-resolution representation learning (HRNet) system coupled with a long short-term memory (LSTM) module to retain high-resolution feature map and capture the temporal behavior of the acoustic signal. We called this joint combination of HRNet and LSTM as HR-LSTM. The predicted spectrograms produced by this system are close to ground truth and successfully separate music sources, achieving results superior to those realized by past methods. The proposed network was tested using four datasets (DSD100, MIR-1K, Korean Pansori, and Nepal Idol singing voice). Our experiments confirmed that the proposed HR-LSTM outperforms state-of-the-art networks at singing voice separation when the DSD100 dataset is used, performs comparably to alternative methods when the MIR-1K dataset is used, and separates the voice and accompaniment components well when the Pansori and NISVS datasets are used. In addition to proposing and validating our network, we also developed and shared our Nepal Idol dataset.

Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information

Stack Overflow is very helpful for software developers who are seeking answers to programming problems. Previous studies have shown that a growing number of questions are of low quality and thus obtain less attention from potential answerers. Gao et al. proposed an LSTM-based model (i.e., BiLSTM-CC) to automatically generate question titles from the code snippets to improve the question quality. However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens. This paper proposes CCBERT, a deep learning based novel model to enhance the performance of question title generation by making full use of the bi-modal information of the entire question body. CCBERT follows the encoder–decoder paradigm and uses CodeBERT to encode the question body into hidden representations, a stacked Transformer decoder to generate predicted tokens, and an additional copy attention layer to refine the output distribution. Both the encoder and decoder perform the multi-head self-attention operation to better capture the long-range dependencies. This paper builds a dataset containing around 200,000 high-quality questions filtered from the data officially published by Stack Overflow to verify the effectiveness of the CCBERT model. CCBERT outperforms all the baseline models on the dataset. Experiments on both code-only and low-resource datasets show the superiority of CCBERT with less performance degradation. The human evaluation also shows the excellent performance of CCBERT concerning both readability and correlation criteria. CCBERT is capable of automatically capturing the bi-modal semantic information from the entire question body and parsing the long-range dependencies to achieve better performance. Therefore, CCBERT is an effective approach for generating Stack Overflow question titles.

Scalable Discrete Matrix Factorization and Semantic Autoencoder for Cross-Media Retrieval.

Hashing methods have sparked great attention on multimedia tasks due to their effectiveness and efficiency. However, most existing methods generate binary codes by relaxing the binary constraints, which may cause large quantization error. In addition, most supervised cross-modal approaches preserve the similarity relationship by constructing an n×n large-size similarity matrix, which requires huge computation, making these methods unscalable. To address the above challenges, this article presents a novel algorithm, called scalable discrete matrix factorization and semantic autoencoder method (SDMSA). SDMSA is a two-stage method. In the first stage, the matrix factorization scheme is utilized to learn the latent semantic information, the label matrix is incorporated into the loss function instead of the similarity matrix. Thereafter, the binary codes can be generated by the latent representations. During optimization, we can avoid manipulating a large n×n similarity matrix, and the hash codes can be generated directly. In the second stage, a novel hash function learning scheme based on the autoencoder is proposed. The encoder-decoder paradigm aims to learn projections, the feature vectors are projected to code vectors by encoder, and the code vectors are projected back to the original feature vectors by the decoder. The encoder-decoder scheme ensures the embedding can well preserve both the semantic and feature information. Specifically, two algorithms SDMSA-lin and SDMSA-ker are developed under the SDMSA framework. Owing to the merit of SDMSA, we can get more semantically meaningful binary hash codes. Extensive experiments on several databases show that SDMSA-lin and SDMSA-ker achieve promising performance.

SAE-PD-Seq: sequence autoencoder-based pre-training of decoder for sequence learning tasks

Sequence learning approaches require careful tuning of parameters for their success. Pre-trained sequence models exhibit a superior performance compared to the sequence models that are randomly initialized. This work presents a sequence autoencoder based pre-trained decoder approach for sequence learning. This approach is applicable for the tasks in which the output is a sequence. In the proposed method, a SAE (Sequence Auto Encoder) is trained with an objective of reconstructing the input sequence. The weights of the pre-trained SAE are then used to initialize the decoder in the sequence model developed based on the encoder–decoder paradigm. The proposed pre-trained decoder-based approach achieves superior performance as compared to the pre-trained encoder-based approach and the pre-trained encoder- and decoder-based approach. The behavior of the suggested approach is examined using unsupervised pre-training. The proposed method is evaluated for neural machine translation and image caption generation tasks. Outcomes of the experimental studies on benchmark datasets indicate the effectiveness of the proposed approach.

Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder

Zero-shot learning (ZSL) is an effective method to perform the recognition task without any training samples of specific classes. Most existing ZSL models put emphasis on learning an embedding between visual space and semantic space directly. However, few ZSL models research whether the human-designed semantic features are discriminative enough to recognize different classes. Moreover, one-way mapping suffers from the project domain shift problem. In this article, we propose to learn a Discriminative Dual Semantic Auto-encoder (DDSA) based on the encoder-decoder paradigm to solve this problem. DDSA attempts to construct two bidirectional embeddings to connect the visual space and the semantic space with the help of the learned aligned space which includes discriminative information of the visual features and semantic features. Based on the DDSA, we additionally propose a Deep DDSA to capture deep aligned features that are more conducive to zero-shot classification. The key to the proposed framework is that it implicitly exact the principal information from visual space and semantic space to construct aligned features, which is not only semantic-preserving but also discriminative. Extensive experiments on five benchmarks (SUN, CUB, AWA1, AWA2 and aPY) demonstrate the effectiveness of the proposed framework with state-of-the-art performance obtained on both conventional ZSL and generalized ZSL settings.

Class label autoencoder with structure refinement for zero-shot learning

Existing zero-shot learning (ZSL) methods usually learn a projection function between a feature space and a semantic embedding space(text or attribute space) for recognizing unseen classes. However, the projection function is difficult to generalize the relationship description between the feature space and multi-semantic embedding spaces(all kinds of class names are embedded as different vectors in terms of the various semantic interpretation, for example attributes, word vectors, global vectors and hierarchical embedding), which have the diversity characteristic for detailing the different semantic information of the same class. To deal with this issue, we present a novel method to ZSL based on learning class label autoencoder with structure refinement(CLASR). CLASR can not only build a scalable framework for adapting to multi-semantic embedding spaces, but also utilize the encoder-decoder paradigm for constraining the bidirectional projection between the feature space and the class label space. Moreover, CLASR can focus on the fusion model between the relationship of feature classes (visual feature classes structure) and the relevance of the semantic classes (semantic classes structure) for improving zero-shot classification by structure refinement. The CLASR solution can provide both unseen class labels and the relation of the different classes structure(visual feature and semantic classes structure) that can encode the intrinsic structure of classes. Extensive experiments demonstrate the CLASR outperforms the state-of-art methods on four benchmark datasets, which are AwA, CUB, Dogs and ImNet-2.