Local CNN Research Articles

Comprehensive Tennis Serve Training System Based on Local Attention-Based CNN Model

Comprehensive Tennis Serve Training System Based on Local Attention-Based CNN Model

A multi-granularity hierarchical network for long- and short-term forecasting on multivariate time series data

Multivariate time series forecasting is a significant research problem in many fields such as economics, finance and transportation, where simultaneous long- and short-term forecasting is required. However, current techniques are typically limited to a single short-term or a long-term forecast. To address the limitation, a novel multi-granularity hierarchical network, GNet-LS, is proposed for long- and short-term forecasting on multivariate time series data, which takes into account the separate role of internal correlation and external relationship. First, the original time series sequence is divided into multiple granular sequences based on downsampling, to reduce error accumulation caused by long-term prediction. In order to discover the external relationships between variables, the CNN module slides over the sequence of variables. The global CNN and local CNN are built to implement periodic and nonperiodic extraction, respectively. Next, a self-attention module is used to model dependencies between the output of local CNN and global CNN. The LSTM networks and attention mechanisms are used to mine internal correlation of the target variable on time series. Then, multiple granular external relationships and internal correlation are obtained in parallel. Finally, external relationships and internal correlation are fused together by splicing and overlay to obtain both long-term and short-term forecasts. The experimental results demonstrate that the proposed GNet-LS outperforms a bunch of compared methods in terms of RSE, CORR, MAE and RMSE.

GLNET: global–local CNN's-based informed model for detection of breast cancer categories from histopathological slides

GLNET: global–local CNN's-based informed model for detection of breast cancer categories from histopathological slides

Satellite Image Classification Using Extended Local Binary Patterns, SVM AND CNN

In many disciplines and applications, image processing has shown to be an effective research tool. Numerous uses, such as disaster response, law enforcement, and environmental monitoring, depend on satellite imaging. Applications demand manual object and device identification from photos. ELBP-SVM technique is used to categorize the satellite images into a set of distinct classes. Although this work is able to classify distinct classes in addition to the class of satellite images, identifying the characteristics of these other classes, such as forest, desert, oceans, etc., is also straightforward because these other classes have some unique characteristics that can be easy to distinguish and therefore easy to classify. This work uses the suggested ELBP approach to first find local binary patterns. The SVM classifies the test picture class after acquiring the extended features. The ELBP-SVM approach is utilized in this study, and the percentage of correctly identified satellite images is 97%. The results discovered and experimentally acquired on MATLAB 2020a are superior to other research currently accessible for the classification of satellite photos.

Online Prediction of Cutting Temperature Using Self-Adaptive Local Learning and Dynamic CNN

To maintain a high-quality machining process, it is necessary to develop an online cutting temperature prediction model accurately. However, using the conventional cutting temperature modeling methods, it is difficult to capture the continuous and multivariate time-variant characteristic of cutting temperature. Recently, local learning and switching dynamics have been employed to identify the time-variant characteristic. However, it requires quantitative and precise analysis with certain switching patterns, which are hard to realize in the machining process. In this article, a self-adaptive local learning method integrated with dynamic convolutional neural networks (DCNN) is proposed. The moving window strategy and t-test analysis of performance maps are first to give the local domains. Then, a DCNN is designed to capture spatial cross-correlations and temporal autocorrelations in each local domain. Finally, the local models are ensemble using Bayesian ensemble learning. The effectiveness of the proposed method is demonstrated through different process information and tool conditions.

DSACNN: Dynamically local self-attention CNN for 3D point cloud analysis

The point cloud is a common 3D representation widely applied in CAX engineering due to its simple data representation and rich semantic information. However, discrete and unordered 3D data structures make it difficult for point clouds to understand semantic information and make them unsuitable for applying standard operators. In this paper, to enhance machine perception of 3D semantic information, we propose a novel approach that can not only directly process point cloud data by a novel convolution-like operator but also dynamically pay attention to local semantic information. First, we design a novel dynamic local self-attention mechanism that can dynamically and flexibly focus on top-level information of the receptive field to learn and understand subtle features. Second, we propose a dynamic self-attention learning block, which adopts the proposed dynamic local self-attention learning convolution operation to directly deal with disordered and irregular point clouds to learn global and local point features while dynamically learning the important local semantic information. Third, the proposed operation can be compatibly applied as an independent component in popular architectures to improve the perception of local semantic information. Numerous experiments demonstrate the advantage of our method for point cloud tasks on datasets from both CAD data and scan data of complex real-world scenes.

Image Retrieval of Tourism Landscape in Rural Revitalization Based on Wireless Communication Network

With the rapid development of information technology, people's acquisition of tourism and other information is increasingly dependent on images and other information. Aiming at the low efficiency of traditional image retrieval methods in processing massive image data, an image retrieval method based on wireless communication network is proposed. Based on the salient area and wireless communication network combined with the hash method to extract local CNN features, simulate INRIA Holidays Data set and Oxford Buildings Data set, and calculate the accuracy and recall rate of search results based on the returned results of tourist attractions pictures. This article designs an experiment to verify the accuracy and recall rate of search results. By comparing the feature hash function to generate the hash code and the Hamming distance between each hash code in the image library, the image is queried, and the final search result is obtained: The more searches, the lower the accuracy and recall rate. This also proves to a certain extent that the CNN feature extraction technology can be used for travel image search, improving the search accuracy by 20%.The wireless communication network is still of great significance to the future social development. It is necessary to conduct in-depth research, not only the image retrieval of tourist attractions proposed in this article but also the potential value of wireless communication networks from multiple angles and more comprehensively.

Local Binary CNN for Diabetic Retinopathy Classification on Fundus Images

Local Binary CNN for Diabetic Retinopathy Classification on Fundus Images

Named Entity Recognition in Electric Power Metering Domain Based on Attention Mechanism

Named Entity Recognition (NER) is one key step for constructing power domain knowledge graph which is increasingly urgent in building smart grid. This paper proposes a new NER model called Att-CNN-BiGRU-CRF which consists of the following five layers. The prefix Att means the model is based on attention mechanism. A joint feature embedding layer combines the character embedding and word embedding based on BERT to obtain more semantic information. A convolutional attention layer combines the local attention mechanism and CNN to capture the relationship of local context. A BiGRU layer extracts higher-level features of power metering text. A global multi-head attention layer optimizes the processing of sentence level information. A CRF layer obtains the output tag sequences. This paper also constructs a corresponding power metering corpus data set with a new entity classification method. The novelties of our work are the five layer model structure and the attention mechanism. Experimental results show that the proposed model has high recall rate 88.16% and precision rate 89.33% which is better than the state-of-the-art models.

Thorax disease classification with attention guided convolutional neural network

This paper considers the task of thorax disease diagnosis on chest X-ray (CXR) images. Most existing methods generally learn a network with global images as input. However, thorax diseases usually happen in (small) localized areas which are disease specific. Thus training CNNs using global images may be affected by the (excessive) irrelevant noisy areas. Besides, due to the poor alignment of some CXR images, the existence of irregular borders hinders the network performance. For addressing the above problems, we propose to integrate the global and local cues into a three-branch attention guided convolution neural network (AG-CNN) to identify thorax diseases. An attention guided mask inference based cropping strategy is proposed to avoid noise and improve alignment in the global branch. AG-CNN also integrates the global cues to compensate the lost discriminative cues by the local branch. Specifically, we first learn a global CNN branch using global images. Then, guided by the attention heatmap generated from the global branch, we infer a mask to crop a discriminative region from the global image. The local region is used for training a local CNN branch. Lastly, we concatenate the last pooling layers of both the global and local branches for fine-tuning the fusion branch. Experiments on the ChestX-ray14 dataset demonstrate that after integrating the local cues with the global information, the average AUC scores are improved by AG-CNN.

Evaluation of Local Descriptors and Deep CNN Features for Face Anti Spoofing

Recently facial recognition Technology are being habitual for various access control requirements and spoof detection in such a system has drawn growing attention. In this paper, we represent by comparison analysis of different local descriptors and off the shelf deep networks for feature extractionLocal Binary Pattern (LBP), SIFT, Histogram of Oriented Gradients (HOG), Shallow CNN, VGG16 and Inception-ResnetV2 for face spoofing detection. Furthermore, we evaluated three Classifiers-Decision Tree, Artificial Neural Network (ANN) and Support Vector Machine (SVM) over the feature extracted through local descriptors and deep networks. The evaluation has been conducted using publicly available YALE face database containing real and fake facial images. Real dataset consists of 5121 entries and fake dataset has 7508 images. The analysis results demonstrate that the best prediction accuracy of real and spoof is obtained with Inception_ResnetV2 features when classified with ANN and about 96.23% accuracy is achieved

Integrating Local CNN and Global CNN for Script Identification in Natural Scene Images

Script identification in natural scene images is a key pre-step for text recognition and is also an indispensable condition for automatic text understanding systems that are designed for multi-language environments. In this paper, we present a novel framework integrating Local CNN and Global CNN both of which are based on ResNet-20 for script identification. We first obtain a lot of patches and segmented images based on the aspect ratios of the images. Subsequently, these patches and segmented images are used as inputs to Local CNN and Global CNN for training, respectively. Finally, to get the final results, the Adaboost algorithm is used to combine the results of Local CNN and Global CNN for decision-level fusion. Benefiting from such a strategy, Local CNN fully exploits the local features of the image, effectively revealing subtle differences among the scripts that are difficult to distinguish such as English, Greek, and Russian. Moreover, Global CNN mines the global features of the image to improve the accuracy of script identification. The experimental results demonstrate that our approach has a good performance on four public datasets.

RGB-D Scene Recognition via Spatial-Related Multi-Modal Feature Learning

RGB-D image-based scene recognition has achieved significant performance improvement with the development of deep learning methods. While convolutional neural networks can learn high-semantic level features for object recognition, these methods still have limitations for RGB-D scene classification. One limitation is that how to learn better multi-modal features for the RGB-D scene recognition is still an open problem. Another limitation is that the scene images are usually not object-centric and with great spatial variability. Thus, vanilla full-image CNN features maybe not optimal for scene recognition. Considering these problems, in this paper, we propose a compact and effective framework for RGB-D scene recognition. Specifically, we make the following contributions: 1) A novel RGB-D scene recognition framework is proposed to explicitly learn the global modal-specific and local modal-consistent features simultaneously. Different from existing approaches, local CNN features are considered for the learning of modal-consistent representations; 2) key Feature Selection (KFS) module is designed, which can adaptively select important local features from the high-semantic level CNN feature maps. It is more efficient and effective than object detection and dense patch-sampling based methods, and; 3) a triplet correlation loss and a spatial-attention similarity loss are proposed for the training of KFS module. Under the supervision of the proposed loss functions, the network can learn import local features of two modalities with no need for extra annotations. Finally, by concatenating the global and local features together, the proposed framework can achieve new state-of-the-art scene recognition performance on the SUN RGB-D dataset and NYU Depth version 2 (NYUD v2) dataset.

Multimodal activity recognition with local block CNN and attention-based spatial weighted CNN

Deep learning based human activity recognition approach combines spatial and temporal information to complete the recognition task. The temporal information is extracted by optical flow, which is always compensated by the warping method in order to achieve better performance. However, these methods usually take the global feature as the starting point, only consider global information of video frames, and ignore local information that reflects the changes of human behavior, causing the algorithm to be sensitive to the external environment such as occlusion, illumination change. In view of the above problems, this paper fuses the local spatial features of video frames, global spatial features and temporal features to recognize different actions, and further extracts the visual attention weight to make constraint on the global spatial features. Experiments show that the algorithm proposed in this paper has better accuracy compared with the existing methods.

Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond

While convolutional neural networks (CNNs) have been excellent for object recognition, the greater spatial variability in scene images typically means that the standard full-image CNN features are suboptimal for scene classification. In this article, we investigate a framework allowing greater spatial flexibility, in which the Fisher vector (FV)-encoded distribution of local CNN features, obtained from a multitude of region proposals per image, is considered instead. The CNN features are computed from an augmented pixel-wise representation consisting of multiple modalities of RGB, HHA, and surface normals, as extracted from RGB-D data. More significantly, we make two postulates: (1) component sparsity—that only a small variety of region proposals and their corresponding FV GMM components contribute to scene discriminability, and (2) modal nonsparsity—that features from all modalities are encouraged to coexist. In our proposed feature fusion framework, these are implemented through regularization terms that apply group lasso to GMM components and exclusive group lasso across modalities. By learning and combining regressors for both proposal-based FV features and global CNN features, we are able to achieve state-of-the-art scene classification performance on the SUNRGBD Dataset and NYU Depth Dataset V2. Moreover, we further apply our feature fusion framework on an action recognition task to demonstrate that our framework can be generalized for other multimodal well-structured features. In particular, for action recognition, we enforce interpart sparsity to choose more discriminative body parts, and intermodal nonsparsity to make informative features from both appearance and motion modalities coexist. Experimental results on the JHMDB and MPII Cooking Datasets show that our feature fusion is also very effective for action recognition, achieving very competitive performance compared with the state of the art.

Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction

Taxi demand prediction is an important building block to enabling intelligent transportation systems in a smart city. An accurate prediction model can help the city pre-allocate resources to meet travel demand and to reduce empty taxis on streets which waste energy and worsen the traffic congestion. With the increasing popularity of taxi requesting services such as Uber and Didi Chuxing (in China), we are able to collect large-scale taxi demand data continuously. How to utilize such big data to improve the demand prediction is an interesting and critical real-world problem. Traditional demand prediction methods mostly rely on time series forecasting techniques, which fail to model the complex non-linear spatial and temporal relations. Recent advances in deep learning have shown superior performance on traditionally challenging tasks such as image classification by learning the complex features and correlations from large-scale data. This breakthrough has inspired researchers to explore deep learning techniques on traffic prediction problems. However, existing methods on traffic prediction have only considered spatial relation (e.g., using CNN) or temporal relation (e.g., using LSTM) independently. We propose a Deep Multi-View Spatial-Temporal Network (DMVST-Net) framework to model both spatial and temporal relations. Specifically, our proposed model consists of three views: temporal view (modeling correlations between future demand values with near time points via LSTM), spatial view (modeling local spatial correlation via local CNN), and semantic view (modeling correlations among regions sharing similar temporal patterns). Experiments on large-scale real taxi demand data demonstrate effectiveness of our approach over state-of-the-art methods.

Superpixel-Based Multiple Local CNN for Panchromatic and Multispectral Image Classification

Recently, very high resolution (VHR) panchromatic and multispectral (MS) remote-sensing images can be acquired easily. However, it is still a challenging task to fuse and classify these VHR images. Generally, there are two ways for the fusion and classification of panchromatic and MS images. One way is to use a panchromatic image to sharpen an MS image, and then classify a pan-sharpened MS image. Another way is to extract features from panchromatic and MS images, respectively, and then combine these features for classification. In this paper, we propose a superpixel-based multiple local convolution neural network (SML-CNN) model for panchromatic and MS images classification. In order to reduce the amount of input data for the CNN, we extend simple linear iterative clustering algorithm for segmenting MS images and generating superpixels. Superpixels are taken as the basic analysis unit instead of pixels. To make full advantage of the spatial-spectral and environment information of superpixels, a superpixel-based multiple local regions joint representation method is proposed. Then, an SML-CNN model is established to extract an efficient joint feature representation. A softmax layer is used to classify these features learned by multiple local CNN into different categories. Finally, in order to eliminate the adverse effects on the classification results within and between superpixels, we propose a multi-information modification strategy that combines the detailed information and semantic information to improve the classification performance. Experiments on the classification of Vancouver and Xi’an panchromatic and MS image data sets have demonstrated the effectiveness of the proposed approach.

Robust face detection using local CNN and SVM based on kernel combination

One key challenge of face detection is the large appearance variations due to some real-world factors, such as viewpoint, extreme illuminations and expression changes, which lead to the large intra-class variations and making the detection algorithm is not robust enough. In this paper, we propose a locality-sensitive support vector machine using kernel combination (LS-KC-SVM) algorithm to solve the above two problems. First, we employ the locality-sensitive SVM (LSSVM) to construct a local model on each local region, which can handle the classification task easier due to smaller within-class variation. Second, motivated by the idea that local features are more robust compared with global features, we use multiple local CNNs to jointly learn local facial features because of the powerful strength of CNN learning characteristic. In order to use this property of local features effectively, we apply the global and local kernels to the features and introduce the combination kernel to the LSSVM. Extensive experiments demonstrate the robustness and efficiency of our algorithm by comparing it with several popular face detection algorithms on the widely used CMU+MIT dataset and FDDB dataset.

SPATIAL DEPTH EXTRACTION USING RANDOM STEREOGRAMS IN ANALOGIC CNN FRAMEWORK

Demonstrated and motivated on human stereo vision analogic CNN algorithms are proposed to extract 3D spatial information from computer-generated random-dot stereograms as well as real scene random-dot like ones produced with simple optical devices, projector and camera. Several aspects of making real scene stereograms are considered to minimize perspective distortion and enable local CNN processing.