Multi-scale Local Features Research Articles

MSM-ViT: A multi-scale MobileViT for pulmonary nodule classification using CT images.

Accurate classification of benign and malignant pulmonary nodules using chest computed tomography (CT) images is important for early diagnosis and treatment of lung cancer. In terms of natural image classification, the ViT-based model has greater advantages in extracting global features than the traditional CNN model. However, due to the small image dataset and low image resolution, it is difficult to directly apply the ViT-based model to pulmonary nodule classification. To propose and test a new ViT-based MSM-ViT model aiming to achieve good performance in classifying pulmonary nodules. In this study, CNN structure was used in the task of classifying pulmonary nodules to compensate for the poor generalization of ViT structure and the difficulty in extracting multi-scale features. First, sub-pixel fusion was designed to improve the ability of the model to extract tiny features. Second, multi-scale local features were extracted by combining dilated convolution with ordinary convolution. Finally, MobileViT module was used to extract global features and predict them at the spatial level. CT images involving 442 benign nodules and 406 malignant nodules were extracted from LIDC-IDRI data set to verify model performance, which yielded the best accuracy of 94.04% and AUC value of 0.9636 after 10 cross-validations. The proposed new model can effectively extract multi-scale local and global features. The new model performance is also comparable to the most advanced models that use 3D volume data training, but its occupation of video memory (training resources) is less than 1/10 of the conventional 3D models.

Multi-Branch Deep Learning Framework for Land Scene Classification in Satellite Imagery

Land scene classification in satellite imagery has a wide range of applications in remote surveillance, environment monitoring, remote scene analysis, Earth observations and urban planning. Due to immense advantages of the land scene classification task, several methods have been proposed during recent years to automatically classify land scenes in remote sensing images. Most of the work focuses on designing and developing deep networks to identify land scenes from high-resolution satellite images. However, these methods face challenges in identifying different land scenes. Complex texture, cluttered background, extremely small size of objects and large variations in object scale are the common challenges that restrict the models to achieve high performance. To tackle these challenges, we propose a multi-branch deep learning framework that efficiently combines global contextual features with multi-scale features to identify complex land scenes. Generally, the framework consists of two branches. The first branch extracts global contextual information from different regions of the input image, and the second branch exploits a fully convolutional network (FCN) to extract multi-scale local features. The performance of the proposed framework is evaluated on three benchmark datasets, UC-Merced, SIRI-WHU, and EuroSAT. From the experiments, we demonstrate that the framework achieves superior performance compared to other similar models.

Pyramid multi-loss vision transformer for thyroid cancer classification using cytological smear

Multi-instance learning, a commonly used technique in artificial intelligence for analyzing slides, can be applied to diagnose thyroid cancer based on cytological smears. Since smears do not have multidimensional histological features similar to histopathology, mining potential contextual information and diversity of features is crucial for better classification performance. In this paper, we propose a pyramid multi-loss vision transformer model called PyMLViT, a novel algorithm with two core modules to address these issues. Specifically, we design a pyramid token extraction module to acquire potential contextual information on smears. The pyramid token structure extracts multi-scale local features, and the vision transformer structure further obtains global information through the self-attention mechanism. Furthermore, we construct multi-loss fusion module based on the conventional multi-instance learning framework. With carefully designed bag and patch weight allocation strategies, we incorporate slide-level annotations as pseudo-labels for patches to participate in training, thus enhancing the diversity of supervised information. Extensive experimental results on the real-world dataset show that PyMLViT has a high performance and a competitive number of parameters compared to popular methods for diagnosing thyroid cancer in cytological smears.

Hyperspectral Image Classification Based on Dense Pyramidal Convolution and Multi-Feature Fusion

In recent years, hyperspectral image classification techniques have attracted a lot of attention from many scholars because they can be used to model the development of different cities and provide a reference for urban planning and construction. However, due to the difficulty in obtaining hyperspectral images, only a limited number of pixels can be used as training samples. Therefore, how to adequately extract and utilize the spatial and spectral information of hyperspectral images with limited training samples has become a difficult problem. To address this issue, we propose a hyperspectral image classification method based on dense pyramidal convolution and multi-feature fusion (DPCMF). In this approach, two branches are designed to extract spatial and spectral features, respectively. In the spatial branch, dense pyramid convolutions and non-local blocks are used to extract multi-scale local and global spatial features in image samples, which are then fused to obtain spatial features. In the spectral branch, dense pyramidal convolution layers are used to extract spectral features in image samples. Finally, the spatial and spectral features are fused and fed into fully connected layers to obtain classification results. The experimental results show that the overall accuracy (OA) of the method proposed in this paper is 96.74%, 98.10%, 98.92% and 96.67% on the four hyperspectral datasets, respectively. Significant improvements are achieved compared to the five methods of SVM, SSRN, FDSSC, DBMA and DBDA for hyperspectral classification. Therefore, the proposed method can better extract and exploit the spatial and spectral information in image samples when the number of training samples is limited. Provide more realistic and intuitive terrain and environmental conditions for urban planning, design, construction and management.

Haze removal using deep convolutional neural network for Korea Multi-Purpose Satellite-3A (KOMPSAT-3A) multispectral remote sensing imagery

This paper presents a convolutional neural network to automatically remove the haze distribution using a single multispectral remote sensing image in the raw file format. To train the proposed dehazing network, we synthesized multispectral hazy images using the haze thickness map (HTM) and relative scattering model representing the wavelength-dependent scattering property of the haze distribution. Since the raw multispectral hazy images have a low dynamic range, we cannot accurately estimate the haze distribution directly from them. To differently impose a proper amount of attention to hazy and haze-free regions, we used the HTM from the contrast-enhanced version of the input hazy image. The proposed dehazing network consists of four sub-networks: (i) shallow feature extraction network (SFEN), (ii) cascaded residual dense block network (CRDBN), (iii) multiscale feature extraction network (MFEN), and (iv) refinement network (RN). The densely connected convolutional layers and local residual learning allow the residual dense block (RDB) to extract the abundant local features, and the cascaded architecture further improves the propagation of the local information and gradients. The MFEN is used to extract multiscale local features representing the hierarchical information for the haze distribution and haze-free region. Experimental results demonstrated that the proposed method can achieve improved dehazing performance on Korea Multi-Purpose Satellite-3A (KOMPSAT-3A) multispectral remote sensing imagery without undesired artifacts. In the sense of quantitative assessment, the proposed method produced improved peak signal-to-noise ratio (PSNR) by 10%, structural similarity index measure (SSIM) by 1%, and spectral angle mapper (SAM) by 19% compared with the existing best method.

Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition

In recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although the mainstream methods for Chinese medical named entity recognition can effectively capture global context information, they ignore the potential local information in sentences, and hence cannot extract the local context features through an efficient framework. To overcome these challenges, this paper constructs a diabetes corpus and proposes the RMBC (RoBERTa Multi-scale CNN BiGRU Self-attention CRF) model. This model is a named entity recognition model that unites multi-scale local feature awareness and the self-attention mechanism. This paper first utilizes RoBERTa-wwm to encode the characters; then, it designs a local context-wise module, which captures the context information containing locally important features by fusing multi-window attention with residual convolution at the multi-scale and adds a self-attention mechanism to address the restriction of the bidirectional gated recurrent unit (BiGRU) capturing long-distance dependencies and to obtain global semantic information. Finally, conditional random fields (CRF) are relied on to learn of the dependency between adjacent tags and to obtain the optimal tag sequence. The experimental results on our constructed private dataset, termed DNER, along with two benchmark datasets, demonstrate the effectiveness of the model in this paper.

Multiscale Local Feature Fusion: Marine Microalgae Classification for Few-Shot Learning

In the marine ecological environment, marine microalgae is an important photosynthetic autotrophic organism, which can carry out photosynthesis and absorb carbon dioxide. With the increasingly serious eutrophication of the water body, under certain environmental factors, the rapid propagation of some algae in the water body gradually forms a harmful bloom, which damages the water environment. Therefore, how to identify the beneficial algae and harmful algae quickly and accurately has gradually become the key to solve the problem. There are more than 30,000 species of marine microalgae in the world, and the sample data are few and the characteristics are not obvious. Many of the algae are similar in shape and difficult to distinguish. The few-shot learning task is very challenging. By training very few labeled samples, the deep learning model has excellent recognition ability. Meanwhile, the few-shot classification method based on metric learning has attracted considerable attention. In this paper, in order to make full use of image features and improve the generalization ability of the model, a multi-scale local feature fusion algorithm was proposed to classify marine microalgae with few shots. First, the input image is gridded and multiscale processed, and then it is sent to the CTM category traversal module for feature extraction to obtain local features. A local feature fusion module based on the SE-NET self-attention mechanism is designed to obtain local enhanced features containing global information to improve the generalization ability of the model. Classification is realized by calculating the distance between the sample feature vector of the query set and the prototype of the support set. Under the settings of 5-way 1-shot and 5-way 5-shot, the classification accuracy of the proposed method is improved by 6.08% and 5.5%, respectively. It provides a new idea for microalgae identification and a new opportunity for the sustainable development of new energy.

Anchor-free infrared pedestrian detection based on cross-scale feature fusion and hierarchical attention mechanism

An anchor-free infrared pedestrian detection algorithm is proposed to address the problems of multiscale, partial occlusion and environmental interference in infrared images, which make it difficult for traditional algorithms to perform accurate detection. First, a cross-scale feature fusion module is devised to improve the detection performance of multiscale and partially occluded objects. This module fuses the global features of different residual layers and multiscale local region features, thereby improving the multiscale feature fusion capability of the model while expanding the feature receptive field of the backbone network. Second, the object texture features of infrared images are weak owing to the imaging mechanism, which affects the accuracy of detection model. Thus, we construct a hierarchical attention mapping module to enhance the significance of pedestrian features in complex environments while suppressing the background information. Moreover, to improve the detection performance further, a novel dual-branch head detector is designed to decouple the input features from the channel dimension, which can be used for classification and location. Finally, by introducing the anchor-free concept into the network prediction mechanism, the prediction box can learn ground truth regression parameters by itself, thereby simplifying the network structure and improving the model generalization ability. The experimental results demonstrate that the proposed algorithm can accurately detect multiscale infrared pedestrian objects in complex environments and exhibits better objective evaluation indices compared to other pedestrian detection algorithms. The average precision and recall reached 98.78% and 98.67%, respectively.

MYOLO: A Lightweight Fresh Shiitake Mushroom Detection Model Based on YOLOv3

Fruit and vegetable inspection aids robotic harvesting in modern agricultural production. For rapid and accurate detection of fresh shiitake mushrooms, picking robots must overcome the complex conditions of the growing environment, diverse morphology, dense shading, and changing field of view. The current work focuses on improving inspection accuracy at the expense of timeliness. This paper proposes a lightweight shiitake mushroom detection model called Mushroom You Only Look Once (MYOLO) based on You Only Look Once (YOLO) v3. To reduce the complexity of the network structure and computation and improve real-time detection, a lightweight GhostNet16 was built instead of DarkNet53 as the backbone network. Spatial pyramid pooling was introduced at the end of the backbone network to achieve multiscale local feature fusion and improve the detection accuracy. Furthermore, a neck network called shuffle adaptive spatial feature pyramid network (ASA-FPN) was designed to improve fresh shiitake mushroom detection, including that of densely shaded mushrooms, as well as the localization accuracy. Finally, the Complete Intersection over Union (CIoU) loss function was used to optimize the model and improve its convergence efficiency. MYOLO achieved a mean average precision (mAP) of 97.03%, 29.8M parameters, and a detection speed of 19.78 ms, showing excellent timeliness and detectability with a 2.04% higher mAP and 2.08 times fewer parameters than the original model. Thus, it provides an important theoretical basis for automatic picking of fresh shiitake mushrooms.

Recursive Multi-Relational Graph Convolutional Network for Automatic Photo Selection

Automatic Photo Selection (APS) is a fundamental and important task for further photo cropping and photo enhancement. As the images in a photo series normally have subtle differences, it remains challenging to surface the best photos among highly similar photos. In this work, we propose a Recursive Multi-Relational Graph Convolutional Network (RMGCN) for APS. Specifically, we explore and devise inner-relation and inter-relation graphs to learn informative representations in hierarchical manner. 1) Patch-aware Intra Graph Module (PIGM) captures visual and spatial relations between different patches to characterize the representations in an image. 2) Context-aware Inter Graph Module (CIGM) explicitly exploits mutual comparative relation between different images in a photo series. These two graphs are recursively refined each other by reasoning the graph representations. Then, our model aggregates the output of CIGM with multi-scale local features via the proposed Cross-domain Fusing Gate (CFG) to boost the discriminative ability. Besides, we formulate four companion objectives as soft constraints to improve convergence rate during training. Extensive experiments are conducted on photo-triage dataset, and superior results are reported on different metrics when comparing to the state-of-the-art methods. We also perform rigorous ablations and analysis to validate our approach.

Multi-scale local feature enhanced transformer network for pavement crack detection

目的 道路裂缝是路面病害的早期征兆。定期监测路面状况、及时准确地发现路面裂缝对于交通养护机构降低成本、保证路面结构的可靠性和耐久性以及提高驾驶安全性、舒适性有重要意义。目前基于卷积神经网络的深度学习模型在长距离依赖建模方面存在不足，模型精度难以满足真实路面环境下的裂缝检测任务。一些模型通过引入空间/通道注意力机制进行长距离依赖建模，但是会导致计算量和计算复杂程度增加，无法实现实时检测。鉴于此，本文提出一种基于Transformer编码—解码结构的深度神经网络道路裂缝检测模型CTNet （crack transformernetwork）。方法 该模型主要由Transformer注意力模块、多尺度局部特征增强模块、上采样模块和跨越连接4部分构成。采用Transformer注意力机制能更有效提取全局和长距离依赖关系，克服传统卷积神经网络表征输入信息的短距离相关缺陷。同时，为适应裂缝尺寸变化多样性，将Transformer与多尺度局部特征增强模块相结合，从而有效整合不同尺度局部信息，克服Transformer局部特征建模不足。结果 通过与DeepCrack模型在不同裂缝检测数据集中的比较表明，本文提出的多尺度局部特征增强Transformer网络能快速、准确地分割路面裂缝，且效率更优。定量研究结果表明，CTNet在更有挑战性的CrackLS315数据集中的精度、召回率和F1值达到91.38%、80.38%和85.53%，明显优于对比方法。在CrackWH100数据集中，精度、召回率和F1值进一步提升，分别达到92.70%、90.52%和91.60%。此外，CTNet的训练速率提升至DeepCrack模型的6.78倍。结论 CTNet可以实现强噪声背景下的道路裂缝检测，检测效果优于目前最优方法，且参数量小，易于训练和部署。;Objective The pavement-relevant inspection is focused on the optimization for pavement cracks early-alarming detection and the preservation of pavement structure. However，conventional image processing-based techniques are laborintensive and time-consuming，such as edge detection，threshold segmentation，template matching，and morphology operations. It is challenged to the geometric and spectral complexities of pavement crack and its contexts（e. g. ，illumination variation，oil or water stains，and shadows caused by trees and vehicles）. The convolution neural network（CNN）based deep learning image processing techniques have been developing intensively. However，the CNN-based methods are less effective in long-range dependency modeling，which may cause insufficient detection results in complicated road surface scenarios. Some works are related to attention mechanisms like spatial or channel attention modules，and self-attention modules. However，these attention mechanism-based operations are still challenged for their sophistication and computational cost. Method To detect pavement cracks efficiently and effectively，we develop a novel Transformer-based encoderdecoder neural network，called CTNet，which consists of Transformer blocks，multi-scale local feature enhanced blocks， upsampling blocks，and skip connections. The CTNet can achieve more long-range dependency and global receptive field in terms of multi-head self-attention-based Transformer mechanism. Although Transformer is featured by high running efficiency and low computational overhead demand，it is infeasible to model local contextual information because Token generation can break the connections of neighboring regions. Thus，to capture more multi-scale local information，we design a multi-scale local feature-enhanced block in terms of a multiple dilation ratios-relevant dilation convolution block. Especially，the designed multi-scale local feature enhancement block is melted into each Transformer block for local information complement. Both of local and global low-level contextual features can be captured for feature enhancement. Afterwards，a novel decoder path is implemented to extract high-level features. The decoder consists of the Transformer blocks similar to the up-sampling blocks and the spatial details can be restored for end-to-end segmentation. Result To demonstrate the efficiency and effectiveness of our proposed CTNet，a series of comparative analyses and ablation studies are carried out on three datasets. First，the CTNet can optimize running efficiency，as well as comparable computation overhead and complexity compared to the current UNet，SegNet，DeepCrack，and SwinUnet. Second，CTNet is 6. 78 times faster than the second-best DeepCrack model in terms of training speed. On CrackLS315 dataset，quantitative analyses are also showed that the optimal CTNet is obtained a precision of 91. 38%，a recall of 80. 38%，and a F1 measure of 85. 53% of each；on CrackWH100 dataset，CTNet can obtain a precision of 92. 70%，a recall of 90. 52%，and a F1 measure 91. 60% of each as well. However，it is still challenged to lack of local information when pure-Transformer-based Swin-UNet performed not well compared to fully convolution networks. Furthermore，the CTNet is insufficient to converge when the local blocksenhanced are removed. In summary，the Transformer-based CTNet is beneficial to multi-scenario pavement cracks in terms of the global receptive field. The CTNet can get pavement crack detection results consistently. Conclusion The proposed CTNet has its potentials to deal with noisy pavement images for pavement crack detection.

Expression site agnostic histopathology image segmentation framework by self supervised domain adaption

Motivation:With the sites of antigen expression different, the segmentation of immunohistochemical (IHC) histopathology images is challenging, due to the visual variances. With H&E images highlighting the tissue structure and cell distribution more broadly, transferring more salient features from H&E images can achieve considerable performance on expression site agnostic IHC images segmentation. Methods:To the best of our knowledge, this is the first work that focuses on domain adaptive segmentation for different expression sites. We propose an expression site agnostic domain adaptive histopathology image semantic segmentation framework (ESASeg). In ESASeg, multi-level feature alignment encodes expression site invariance by learning generic representations of global and multi-scale local features. Moreover, self-supervision enhances domain adaptation to perceive high-level semantics by predicting pseudo-labels. Results:We construct a dataset with three IHCs (Her2 with membrane stained, Ki67 with nucleus stained, GPC3 with cytoplasm stained) with different expression sites from two diseases (breast and liver cancer). Intensive experiments on tumor region segmentation illustrate that ESASeg performs best across all metrics, and the implementation of each module proves to achieve impressive improvements. Conclusion:The performance of ESASeg on the tumor region segmentation demonstrates the efficiency of the proposed framework, which provides a novel solution on expression site agnostic IHC related tasks. Moreover, the proposed domain adaption and self-supervision module can improve feature domain adaption and extraction without labels. In addition, ESASeg lays the foundation to perform joint analysis and information interaction for IHCs with different expression sites.

A multi-dimensional residual shrinking network combined with a long short-term memory network for state of charge estimation of Li-ion batteries

The multi-dimensional interaction and noise interference of the original data increase the difficulty of state-of-charge (SoC) estimation. Meanwhile the existing SoC estimators prefer to estimate the SoC of consecutive time steps, which may lead to cumulative errors and data leakage. To solve these problems, a fusion network combining a multi-dimensional residual shrinkage network (MRSN) with a long short-term memory network (LSTM) is proposed for SoC estimation. Meanwhile, a sequence-to-point processing method is utilized to avoid data leakage, which is based on historical input data for estimating SoC at a single time step. Specifically, MRSN uses small sub-networks to remove redundant noise and extracts multi-scale local features on different channels to obtain stronger correlation features. LSTM uses historical inputs to obtain multi-scale temporal correlation. After that the features extracted by both networks are fused in the channel dimension to enhance the accuracy of SoC estimation. Experiments on the public dataset show that the mean absolute error (MAE) of the results is kept within 0.5 % under four different temperature conditions. Furthermore, the MAE is 0.18 % at 25 degrees, which verifies that the proposed network could significantly improve the accuracy of the estimation.

3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification

Point cloud classification is a fundamental task in 3D applications. However, it is challenging to achieve effective feature learning due to the irregularity and unordered nature of point clouds. Lately, 3D Transformers have been adopted to improve point cloud processing. Nevertheless, massive Transformer layers tend to incur huge computational and memory costs. This paper presented a novel hierarchical framework that incorporated convolutions with Transformers for point cloud classification, named 3D Convolution-Transformer Network (3DCTN). It combined the strong local feature learning ability of convolutions with the remarkable global context modeling capability of Transformers. Our method had two main modules operating on the downsampling point sets. Each module consisted of a multi-scale local feature aggregating (LFA) block and a global feature learning (GFL) block, which were implemented by using the Graph Convolution and Transformer respectively. We also conducted a detailed investigation on a series of self-attention variants to explore better performance for our network. Various experiments on ModelNet40 and ScanObjectNN datasets demonstrated that our method achieves state-of-the-art classification performance with a lightweight design. The code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/d62lu/3DCTN</uri> .

Transformers and CNNs fusion network for salient object detection

Recently, Salient Object Detection (SOD) has achieved promising results thanks to the rapid evolution of CNN architectures. However, those CNN-based methods show limited capacity in modeling long-range interaction among pixels. In this paper, we propose to combine the merits from CNN and Transformer and design a unified model for RGB and RGB-D SOD. First, we represent the image with a sequence-to-sequence perspective and use a Transformer-based branch to express long-distance relationships of image tokens to obtain global semantic information and predict a coarse saliency map. Second, we employ a CNN-based branch to extract multi-scale local detail features to predict contour prediction for auxiliary supervision at each level. Finally, we propose the Bi-enhancement Fusion Module to fuse multi-scale cues from two branches to predict a more accurate saliency map. In addition, for RGB-D SOD, to obtain effective cross-modality features, we propose a Cross-modality Multi-Scale Transformer Module and a Depth-induced Enhancement Module to fuse RGB and depth cues in the Transformers branch and the CNNs branch, respectively. Experiments on both RGB and RGB-D SOD datasets demonstrate that our proposed model achieves satisfactory performance compared with state-of-the-art methods.

Learning Motion Constraint-Based Spatio-Temporal Networks for Infrared Dim Target Detections

Efficient infrared dim object detection has been challenged by low signal-to-noise ratios (SNRs). Traditional methods rely on the gradient difference and fixed-parameter model. These methods fail to adapt to sophisticated and variable situations in the real world. To tackle the issue, a deep learning method based on the spatio-temporal network is proposed in this paper. The model is established by the Convolutional Long Short-Term Memory cell (Conv-LSTM) and the 3D Convolution cell (3D-Conv). It is trained to learn the motion constraint of moving targets (spatio-temporal constraint module, called STM) and to fuse the multiscale local feature between the target and background (deep spatial features module, called DFM). In addition, a variable interval search module (state-aware module, called STAM) is added to the inference. The submodule decides to conduct a global search for images only if the target is lost due to fast motion, uncertain obstruction, and frame loss. Comprehensive experiments indicate that the proposed method achieves better performance over all baseline methods. On the mid-wave infrared datasets collected by the authors, the proposed method achieves a 95.87% detection rate. The SNR of the dataset is around 1–3 dB, and the background of the sequence includes sky, asphalt road, and buildings.

A generative adversarial network with multi-scale convolution and dilated convolution res-network for OCT retinal image despeckling

Optical coherence tomography (OCT) has been widely adopted for imaging in various areas, yet it is largely affected by speckle noise generated from the coherent multiple-scattered photons. To alleviate the influences of speckle noise, a generative adversarial network with multi-scale convolution and dilated convolution res-network (MDR-GAN) is proposed in this study. Specifically, a cascade multi-scale module (CMSM) consisting of three convolution and dilated convolution res-network (CD-Rn) blocks is proposed to raise network learning capacity, while a new residual learning method is devised to link the input and output feature maps for feature reconstructions. Among them, CMSM has the characteristics of capturing multi-scale local features of images. Residual learning effectively avoids the degradation problem of the network. Extensive experiments with four retinal OCT datasets are conducted and results are compared with those of the state-of-the-art deep learning networks to verify the effectiveness of the proposed MDR-GAN. Results demonstrate that the denoising effect of MDR-GAN is better than those of the other denoising methods. The peak single-to-noise ratio (PSNR) of MDR-GAN is improved by 2 dB as compared that of Pix2pix, while its equivalent number of looks (ENL) is improved by at least 233.9% as compared with the-state-of-the-art existing methods. Our MDR-GAN code can be download at https://github.com/Austin-Lms/MDR-GAN.

Joint Event Relation Identification Based on Multiscale Convolutional Neural Network and Sharing Strategy

At present, most of the event relation identification work mainly focuses on the sequential temporal and explicit causal relation between events. These methods usually ignore the role of synchronous temporal and implicit causal relation in sentences, which makes the semantic understanding of the model deviate from the text. In this paper, we propose a joint event relation identification model. The model uses bidirectional GRU and multiscale convolutional neural network to obtain the context semantic features and multiscale local semantic features of text, respectively. Then, these two kinds of features are fused to fully obtain the semantics of the text itself. In addition, we build encoders and decoders of event temporal and causal relation, respectively, to obtain the event temporal and causal semantic features from text. In this process, considering the correlation between event timing and causality, we use three different parameter sharing strategies to realize the interaction between event temporal and causal semantic features. The experimental results on the legal field dataset we constructed show that our model has made significant improvements compared with the baseline model. Through experimental analysis, our method can effectively improve the identification performance of synchronous temporal and implicit causality relation.

Single-photon 3D imaging with a multi-stage network.

Active single-photon 3D imaging technology has been applied to 3D imaging of complex scenes in many frontier fields such as biomedicine, remote sensing mapping, etc. However, single-photon 3D imaging with strong background noise is still a major challenge. Several classical algorithms and machine learning methods have been proposed to solve the problem. In this paper, we propose a novel multi-stage synergistic recovery network to reconstruct an accurate depth map. In the model, we first extract multi-scale feature information using encoder and decoder architectures, then combine them with an original resolution network that retains complete spatial location information. Through this way, we can compensate the deficiencies of the original resolution network for multi-scale local feature extraction. Moreover, a self-supervised attention module (SAM) is constructed to weight local features between different stages, optimizing the feature exchange between different stages of the multi-stage architecture network. Our method currently performs the best of all the tested methods.

Multi-Scale LBP Texture Feature Learning Network for Remote Sensing Interpretation of Land Desertification

Land desertification is a major challenge to global sustainable development. Therefore, the timely and accurate monitoring of the land desertification status can provide scientific decision support for desertification control. The existing automatic interpretation methods are affected by factors such as “same spectrum different matter”, “different spectrum same object”, staggered distribution of desertification areas, and wide ranges of ground objects. We propose an automatic interpretation method for the remote sensing of land desertification that incorporates multi-scale local binary pattern (MSLBP) and spectral features based on the above issues. First, a multi-scale convolutional LBP feature extraction network is designed to obtain the spatial texture features of remote sensing images and fuse them with spectral features to enhance the feature representation capability of the model. Then, considering the continuity of the distribution of the same kind of ground objects in local space, we designed an adaptive median filtering method to process the probability map of the extreme learning machine (ELM) classifier output to improve the classification accuracy. Four typical datasets were developed using GF-1 multispectral imagery with the Horqin Left Wing Rear Banner as the study area. Experimental results on four datasets show that the proposed method solves the problem of ill classification and omission in classifying the remote sensing images of desertification, effectively suppresses the effects of “homospectrum” and “heterospectrum”, and significantly improves the accuracy of the remote sensing interpretation of land desertification.