Local Convolutional Neural Network Research Articles

Identification of the damage location for the structural sealant based on deep learning

Structural sealants are essential to maintain the safety of panel units of hidden frame glass curtain wall. Damage localization of the concealed structural sealant is still difficult because of the unknown baseline model. In this paper, a convolutional neural network-based identification method is proposed to localize the damage of structural sealants without the baseline model. The method develops a novel input of the convolutional neural network (CNN), multi-symmetry-point images (MSPI), which is encoded by the response of four symmetrical points to impulse excitations. The CNN would identify the damage location by discerning differences among four images. Then, a dataset with 28803 samples, which considers the effect of the multiple damages and noise, was used to train the CNN. Three types of transformed images and four CNN models were compared to optimize the input signals and the configuration of the CNN. A series of numerical examples indicated that the Gram angle difference field is the optimal image transformation method, and improved-DenseNet121 is the optimal CNN for damage localization in structural sealants. Then, several laboratory experiments validate the effectiveness of the optimized image input and CNN with an accuracy rate of 92 % in the identification of damage location.

AI- POWERED BASED BLIND AND VISUALLY IMPAIRED SYSTEM FOR SMART GLASS

Visual impairment causes serious problems in daily life and mobility for millions of people worldwide. Against this background, smart assistive technology is gaining momentum to improve the independence and quality of life of the blind and visually impaired. They have difficulty in daily life because they cannot detect the problems around them, and one of their biggest problems is identifying people. Besides automation, there are many unexplored uses of search engines. This project includes an app that uses search to help blind people see things in front of them safely, and a facial recognition system with visual feedback that can help visually impaired people see familiar and unfamiliar people. The speaker will provide them with sound support. In this work, we use deep learning-based Regional Convolutional Neural Network (Faster R-CNN) to detect and identify people and objects in the environment. Faster local convolutional neural network technology processes and distributes images captured by cameras. The operator takes the image as input. Therefore, this model helps visually impaired people in a simpler way than a white cane. Keywords—Visuallmpairment,Objectdetection,Faster Region Convolutional Neural Network.

Application of deep learning with bivariate models for genomic prediction of sow lifetime productivity-related traits.

Pig breeders cannot obtain phenotypic information at the time of selection for sow lifetime productivity (SLP). They would benefit from obtaining genetic information of candidate sows. Genomic data interpreted using deep learning (DL) techniques could contribute to the genetic improvement of SLP to maximize farm profitability because DL models capture nonlinear genetic effects such as dominance and epistasis more efficiently than conventional genomic prediction methods based on linear models. This study aimed to investigate the usefulness of DL for the genomic prediction of two SLP-related traits; lifetime number of litters (LNL) and lifetime pig production (LPP). Two bivariate DL models, convolutional neural network (CNN) and local convolutional neural network (LCNN), were compared with conventional bivariate linear models (i.e., genomic best linear unbiased prediction, Bayesian ridge regression, Bayes A, and Bayes B). Phenotype and pedigree data were collected from 40,011 sows that had husbandry records. Among these, 3,652 pigs were genotyped using the PorcineSNP60K BeadChip. The best predictive correlation for LNL was obtained with CNN (0.28), followed by LCNN (0.26) and conventional linear models (approximately 0.21). For LPP, the best predictive correlation was also obtained with CNN (0.29), followed by LCNN (0.27) and conventional linear models (approximately 0.25). A similar trend was observed with the mean squared error of prediction for the SLP traits. This study provides an example of a CNN that can outperform against the linear model-based genomic prediction approaches when the nonlinear interaction components are important because LNL and LPP exhibited strong epistatic interaction components. Additionally, our results suggest that applying bivariate DL models could also contribute to the prediction accuracy by utilizing the genetic correlation between LNL and LPP.

Parameter instance learning with enhanced vision transformers for aerial person re‐identification

SummaryIn an agnostic space environment, aerial person re‐identification (Re‐ID) is a task that the query person may not occur in the gallery set, it is considered a subordinate task within the domain of open‐world person Re‐ID, and is a more challenging and practical application research. The aerial person images, captured by unmanned aerial vehicles, present more significant challenges such as weak appearance features, fewer individual person samples and occlusion due to variations in camera height and viewing angles compared to ground‐level images. Most state‐of‐the‐arts person Re‐ID methods developed for open‐world datasets rely heavily on local convolutional neural networks but exhibit suboptimal performance when directly applied to aerial person Re‐ID tasks. In this article, a parameter instance learning based on vision transformers (ViT) model is introduced for the design of aerial person Re‐ID. Initially, we employ a self‐supervised paradigm grounded in parameter instance discrimination, aiming to capture feature alignment and instance similarity. Subsequently, using labeled training data, we optimize the network model through the calculation of two types of loss functions. Finally, we employ a feature enhancement strategy utilizing zero‐padding and displacement techniques. This strategy effectively and directly enhances the robustness of the ViT model against issues such as occlusion and misalignment. We conducted experiments on a Re‐ID dataset to validate the effectiveness of the method. Our approach achieves a mean average precision of 57.31% and a Rank‐1 accuracy of 65.29% on the aerial person Re‐ID dataset PRAI‐1581.

Federated Learning-Enabled Jamming Detection and Waveform Classification for Distributed Tactical Wireless Networks

In this paper, we propose a federated learning (FL)-based jammer detection and waveform classification algorithm for distributed tactical wireless networks (TWNs). More specifically, we consider a distributed TWN with multiple clusters and various types of waveforms used in the presence of a mobile jammer. We analyze the frequency domain of the waveforms received on local servers to extract the unique cyclic frequency from each waveform’s spectral correlation function (SCF). The method is used to detect the peak values in the frequency-cyclic frequency plane. The primary signal’s SCF exhibits peaks at the unique cyclic frequency and the zero cyclic frequency. These features are then used to train local convolutional neural networks (CNNs) to detect the jamming attacks and classify the waveforms. Moreover, a practical distributed TWN is considered in which each cluster head has a partial observation of the TWN with insufficient data samples, and the proposed algorithm exploits the distributed learning feature of FL, i.e., global learning aggregation to detect the existence of jammers and distinguish the types of waveforms received throughout the TWN. We implement a rigorous TWN simulation using MATLAB toolboxes and our proposed algorithm in TensorFlow Federated. The numerical results show that our proposed algorithm outperforms the standalone local SCF-CNN algorithm. We further demonstrate that the SCF feature yields more accuracy than the In-phase and Quadrature features.

Reliable Feature Matching for Spherical Images via Local Geometric Rectification and Learned Descriptor

Spherical images have the advantage of recording full scenes using only one camera exposure and have been becoming an important data source for 3D reconstruction. However, geometric distortions inevitably exist due to the spherical camera imaging model. Thus, this study proposes a reliable feature matching algorithm for spherical images via the combination of local geometric rectification and CNN (convolutional neural network) learned descriptor. First, image patches around keypoints are reprojected to their corresponding tangent planes based on a spherical camera imaging model, which uses scale and orientation data from the keypoints to achieve both rotation and scale invariance. Second, feature descriptors are then calculated from the rectified image patches by using a pre-trained separate detector and descriptor learning network, which improves the discriminability by exploiting the high representation learning ability of the CNN. Finally, after classical feature matching with the ratio test and cross check, refined matches are obtained based on an essential matrix-based epipolar geometry constraint for outlier removal. By using three real spherical images and an incremental structure from motion (SfM) engine, the proposed algorithm is verified and compared in terms of feature matching and image orientation. The experiment results demonstrate that the geometric distortions can be efficiently reduced from rectified image patches, and the increased ratio of the match numbers ranges from 26.8% to 73.9%. For SfM-based spherical image orientation, the proposed algorithm provides reliable feature matches to achieve complete reconstruction with comparative accuracy.

Approximate object location deep visual representations for image retrieval

Image descriptor based on Convolution Neural Network (CNN) has been presented to represent image in recent lots of works, outperforming the traditional feature for image retrieval. In this paper, a new method is proposed to capture real object regions in images for better search. In contrast to the existing method that process image as a whole, our method focuses on searching for approximate object region in an image. The proposed method mainly has the following two aspects: (i) we employ the sliding windows with the multiple scales over the image, to extract corresponding local CNN features. After processing feature, an optimized image representation with several categories of weights has been prepared for image retrieval and re-rank. (ii) On the basis of (i), we construct a new framework called approximate object location to search out the most similar domain, which can be applied in renewed retrieval, in an image for the query. This proposed framework provides a vector consisting of many locals in different scales for an image. According to experiments, our proposed method outperforms most current approaches based on CNN, and excels many previous algorithms based on costly bag-of-words.

Adaptive Local Context and Syntactic Feature Modeling for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a fine-grained sentiment analysis task that consists of two types of subtasks: aspect term extraction and aspect sentiment classification. In the aspect term extraction task, current methods suffer from the lack of fine-grained information in aspect term extraction and difficulty in identifying aspect term boundaries. In the aspect sentiment classification task, the current aspect sentiment classifier cannot adapt itself to the text and determine the local context. To address these two challenges, this work proposes an adaptive semantic relative distance approach based on dependent syntactic analysis, which uses adaptive semantic relative distance to determine the appropriate local context for each text and increase the accuracy of sentiment analysis. Meanwhile, the study also predicts the current word labels by combining local information features extracted by local convolutional neural networks and global information features to precisely locate the word labels. In two subtasks, our proposed model improves accuracy and F1 scores on the SemEval-2014 Task 4 Restaurant and Laptop datasets compared to the state-to-the-art approaches, especially in the aspect sentiment classification subtask.

Convolution Network Enlightened Transformer for Regional Crop Disease Classification

The overarching goal of smart farming is to propose pioneering solutions for future sustainability of humankind. It is important to recognize the image captured for monitoring the growth of plants and preventing diseases and pests. Currently, the task of automatic recognition of crop diseases is to research crop diseases based on deep learning, but the existing classifiers have problems regarding, for example, accurate identification of similar disease categories. Tomato is selected as the crop of this article, and the corresponding tomato disease is the main research point. The vision transformer (VIT) method has achieved good results on image tasks. Aiming at image recognition, tomato plant images serve as this article’s data source, and their structure is improved based on global ViT and local CNN (convolutional neural network) networks, which are built to diagnose disease images. Therefore, the features of plant images can be precisely and efficiently extracted, which is more convenient than traditional artificial recognition. The proposed architecture’s efficiency was evaluated by three image sets from three tomato-growing areas and acquired by drone and camera. The results show that this article method garners an average counting accuracy of 96.30%. It provides scientific support and a reference for the decision-making process of precision agriculture.

FedStack: Personalized activity monitoring using stacked federated learning

Recent advances in remote patient monitoring (RPM) systems can recognize various human activities to measure vital signs, including subtle motions from superficial vessels. There is a growing interest in applying artificial intelligence (AI) to this area of healthcare by addressing known limitations and challenges such as predicting and classifying vital signs and physical movements, which are considered crucial tasks. Federated learning is a relatively new AI technique designed to enhance data privacy by decentralizing traditional machine learning modeling. However, traditional federated learning requires identical architectural models to be trained across the local clients and global servers. This limits global model architecture due to the lack of local models’ heterogeneity. To overcome this, a novel federated learning architecture, FedStack, which supports ensembling heterogeneous architectural client models was proposed in this study. This work offers a protected privacy system for hospitalized in-patients in a decentralized approach and identifies optimum sensor placement. The proposed architecture was applied to a mobile health sensor benchmark dataset from 10 different subjects to classify 12 routine activities. Three AI models, artificial neural network (ANN), convolutional neural network (CNN), and bidirectional long short-term memory (Bi-LSTM) were trained on individual subject data. The federated learning architecture was applied to these models to build local and global models capable of state-of-the-art performances. The local CNN model outperformed ANN and Bi-LSTM models on each subject data. Our proposed work has demonstrated better performance for heterogeneous stacking of the local models compared to homogeneous stacking. Further analysis of the global heterogeneous CNN model determined that the optimum placement of the sensors on human limbs resulted in better activity recognition. This work sets the stage to build an enhanced RPM system that incorporates client privacy to assist with clinical observations for patients in an acute mental health facility and ultimately help to prevent unexpected death.

Graph-Embedded Convolutional Neural Network for Image-Based EEG Emotion Recognition

Emotion recognition from electroencephalograph (EEG) signals has long been essential for affective computing. In this article, we evaluate EEG emotion recognition by converting EEG signals from multiple channels into images such that richer spatial information can be considered and the question of EEG-based emotion recognition can be converted into image recognition. To this end, we propose a novel method to generate continuous images from discrete EEG signals by introducing offset variables following a Gaussian distribution for each EEG channel to alleviate the biased electrode coordinates during image generation. In addition, a novel graph-embedded convolutional neural network (GECNN) method is proposed to combine the local convolutional neural network (CNN) features with global functional features to provide complementary emotion information. In GECNN, the attention mechanism is applied to extract more discriminative local features. Simultaneously, dynamical graph filtering explores the intrinsic relationships between different EEG regions. The local and global functional features are finally fused for emotion recognition. Extensive experiments in subject-dependent and subject-independent protocols are conducted to evaluate the performance of the proposed GECNN model on four datasets, i.e., SEED, SDEA, DREAMER, and MPED.

Spatio-temporal convolutional residual network for regional commercial vitality prediction.

The vitality of commercial entities reflects the business condition of their surrounding area, the prediction of which helps identify the trend of regional development and make investment decisions. The indicators of business conditions, like revenues and profits, can be employed to make a prediction beyond any doubt. Unfortunately, such figures constitute business secrets and are usually publicly unavailable. Thanks to the rapid growing of location based social networks such as Yelp and Foursquare, massive amount of online data has become available for predicting the vitality of commercial entities. In this paper, a Spatio-Temporal Convolutional Residual Neural Network (STCRNN) is proposed for regional commercial vitality prediction, based on public online data, such as reviews and check-ins from mobile apps. Firstly, a commercial vitality map is built to indicate the popularity of business entities. Afterwards, a local convolutional neural network is employed to capture the spatial relationship of surrounding commercial districts on the vitality map. Then, a 3-dimension convolution is applied to deal with both recent and periodic variations, i.e., the sequential and seasonal changes of commercial vitality. Finally, long short-term memory is introduced to synthesize these two variations. In particular, a residual network is used to eliminate gradient vanishing and exploding, caused by the increase of depth of neural networks. Experiments on public Yelp datasets from 2013 to 2018 demonstrate that STCRNN outperforms the current methods in terms of mean square error.

Facial Makeup Detection Using Multi-Scale Local Binary Patterns and Convolutional Neural Network Fusion

This study presents a novel facial makeup detection scheme using fusion of multi-scale Local Binary Patterns (ms-LBP) texture methods and convolutional neural network (CNN) deep learning extractor. Facial makeups affect the accuracy of face recognition systems due to appearance alteration of individuals. In order to fuse the facial features, the proposed scheme first considers concatenation of extracted global histogram of a whole image using three different LBP operators. The LBP scales are used to extract and then concatenate the local histogram of overlapping and nonoverlapping blocks of images. This way utilizes the advantages of microtexton information of local primitives besides the extracted global textures. Finally, the extracted features using CNN feature extractor are combined with global and local multi-scale textures. In general, CNN deep learning extractor learns high-level discriminative characteristics of images and therefore the proposed system improves the facial makeup detection rate by involving both microtexton and discriminative information of facial images. In addition, the proposed method attempts to select the optimized subset of facial features to increase the detection performance of system by applying Particle Swarm Optimization (PSO) technique after feature level fusion. The paper uses Support Vector Machine (SVM) classifier for classifying the facial vectors into makeup or no-makeup classes. The proposed scheme is then evaluated using YMU, VMU and MIW facial makeup databases with consideration of light, medium and heavy makeups on several datasets. Experimental results analysis clarifies the effectiveness of proposed facial makeup detection framework of this study.

Fault diagnosis of a planetary gearbox based on a local bi-spectrum and a convolutional neural network

The transmission paths of vibration signals in the planetary gearboxes are complex. The signals have the characteristics of strong background noise, instability and non-Gaussian. Bi-spectrums can suppress Gaussian colored noise and are suitable for vibration signal processing of planetary gearboxes. In the traditional fault diagnosis methods based on bi-spectrums, the amplitudes of fault characteristic frequency, or the other further quantitative calculations values, are generally used as the basis of fault diagnosis processes. It has been found that bi-spectrum images can directly characterize the faults of the planetary gearboxes. Convolutional neural networks (CNNs) have been used in mechanical fault diagnoses in recent years. One-dimensional original signals are converted into two-dimensional images as CNN input, which is an effective method for mechanical fault diagnoses. At the present time, there has not been any relevant research conducted using bi-spectral images as CNN input. In this study, a fault diagnosis method based on local bi-spectrum and CNN was proposed. A bi-spectral analysis of the vibration signals of the planetary gearbox was first carried out in order to reveal the fault information while retaining the non-Gaussian information. Then, according to the bi-spectrum symmetry, local images containing the main information were taken as the input of the CNN, which reduced the redundancy of the fault information. Then, in order to improve the diagnostic accuracy of the CNN, the key parameters of CNN architecture were optimized. Finally, a CNN diagnosis model was built to realize the classification diagnoses of different fault positions and different fault degrees of planetary gearboxes. This study’s comparison of the diagnosis results of the full bi-spectrum + CNN, original vibration signal + CNN, local bi-spectrum + (support vector machines), and local bi-spectrum + (stacked auto-encoder) showed that the proposed method in this study had achieved both accuracy and rapidity in the fault diagnoses of planetary gearboxes.

A Novel Hybrid Approach Based on Deep CNN Features to Detect Knee Osteoarthritis.

In the recent era, various diseases have severely affected the lifestyle of individuals, especially adults. Among these, bone diseases, including Knee Osteoarthritis (KOA), have a great impact on quality of life. KOA is a knee joint problem mainly produced due to decreased Articular Cartilage between femur and tibia bones, producing severe joint pain, effusion, joint movement constraints and gait anomalies. To address these issues, this study presents a novel KOA detection at early stages using deep learning-based feature extraction and classification. Firstly, the input X-ray images are preprocessed, and then the Region of Interest (ROI) is extracted through segmentation. Secondly, features are extracted from preprocessed X-ray images containing knee joint space width using hybrid feature descriptors such as Convolutional Neural Network (CNN) through Local Binary Patterns (LBP) and CNN using Histogram of oriented gradient (HOG). Low-level features are computed by HOG, while texture features are computed employing the LBP descriptor. Lastly, multi-class classifiers, that is, Support Vector Machine (SVM), Random Forest (RF), and K-Nearest Neighbour (KNN), are used for the classification of KOA according to the Kellgren–Lawrence (KL) system. The Kellgren–Lawrence system consists of Grade I, Grade II, Grade III, and Grade IV. Experimental evaluation is performed on various combinations of the proposed framework. The experimental results show that the HOG features descriptor provides approximately 97% accuracy for the early detection and classification of KOA for all four grades of KL.

White Matter, Gray Matter and Cerebrospinal Fluid Segmentation from Brain Magnetic Resonance Imaging Using Adaptive U-Net and Local Convolutional Neural Network

Abstract According to the World Alzheimer Report 2015, 46 million people are living with dementia in the world. The diagnosis of diseases helps doctors treating patients better. One of the signs of diseases is related to white matter, grey matter and cerebrospinal fluid. Therefore, the automatic segmentation of three tissues in brain imaging especially from magnetic resonance imaging (MRI) plays an important role in medical analysis. In this research, we proposed an effective approach to segment automatically these tissues in three-dimensional (3D) brain MRI. First, a deep learning model is used to segment the sure and unsure regions. In the unsure region, another deep learning model is used to classify each pixel. In the experiments, an adaptive U-net model is used to segment the sure and unsure regions, and the Local Convolutional Neural Network (CNN) model with multiple inputs is used to classify each pixel only in the unsure region. Our method was evaluated with a real image database, Internet Brain Segmentation Repository database, with 18 persons (IBSR 18) (https://www.nitrc.org/projects/ibsr) and compared with state of art methods being the results very promising.

End‐to‐end global to local convolutional neural network learning for hand pose recovery in depth data

Despite recent advances in 3-D pose estimation of human hands, thanks to the advent of convolutional neural networks (CNNs) and depth cameras, this task is still far from being solved in uncontrolled setups. This is mainly due to the highly non-linear dynamics of fingers and self-occlusions, which make hand model training a challenging task. In this study, a novel hierarchical tree-like structured CNN is exploited, in which branches are trained to become specialised in predefined subsets of hand joints called local poses. Further, local pose features, extracted from hierarchical CNN branches, are fused to learn higher order dependencies among joints in the final pose by end-to-end training. Lastly, the loss function used is also defined to incorporate appearance and physical constraints about doable hand motions and deformations. Finally, a non-rigid data augmentation approach is introduced to increase the amount of training depth data. Experimental results suggest that feeding a tree-shaped CNN, specialised in local poses, into a fusion network for modelling joints' correlations and dependencies, helps to increase the precision of final estimations, showing competitive results on NYU, MSRA, Hands17 and SyntheticHand datasets.

A convolutional neural network for common coordinate registration of high-resolution histology images.

MotivationRegistration of histology images from multiple sources is a pressing problem in large-scale studies of spatial -omics data. Researchers often perform ‘common coordinate registration’, akin to segmentation, in which samples are partitioned based on tissue type to allow for quantitative comparison of similar regions across samples. Accuracy in such registration requires both high image resolution and global awareness, which mark a difficult balancing act for contemporary deep learning architectures.ResultsWe present a novel convolutional neural network (CNN) architecture that combines (i) a local classification CNN that extracts features from image patches sampled sparsely across the tissue surface and (ii) a global segmentation CNN that operates on these extracted features. This hybrid network can be trained in an end-to-end manner, and we demonstrate its relative merits over competing approaches on a reference histology dataset as well as two published spatial transcriptomics datasets. We believe that this paradigm will greatly enhance our ability to process spatial -omics data, and has general purpose applications for the processing of high-resolution histology images on commercially available GPUs.Availability and implementationAll code is publicly available at https://github.com/flatironinstitute/st_gridnet.Supplementary information Supplementary data are available at Bioinformatics online.

Genetic-algorithm-based Local Binary Convolutional Neural Network for Gender Recognition

Genetic-algorithm-based Local Binary Convolutional Neural Network for Gender Recognition

Hybrid Convolutional Neural Network for Localization of Epileptic Focus Based on iEEG.

Epileptic focus localization by analysing intracranial electroencephalogram (iEEG) plays a critical role in successful surgical therapy of resection of the epileptogenic lesion. However, manual analysis and classification of the iEEG signal by clinicians are arduous and time-consuming and excessively depend on the experience. Due to individual differences of patients, the iEEG signal from different patients usually shows very diverse features even if the features belong to the same class. Accordingly, automatic detection of epileptic focus is required to improve the accuracy and to shorten the time for treatment. In this paper, we propose a novel feature fusion-based iEEG classification method, a deep learning model termed Time-Frequency Hybrid Network (TF-HybridNet), in which short-time Fourier transform (STFT) and 1d convolution layers are performed on the input iEEG in parallel to extract features of the time-frequency domain and feature maps. And then, the time-frequency features and feature maps are fused and fed to a 2d convolutional neural network (CNN). We used the Bern-Barcelona iEEG dataset for evaluating the performance of TF-HybridNet, and the experimental results show that our approach is able to differentiate the focal from nonfocal iEEG signal with an average classification accuracy of 94.3% and demonstrates an improved accuracy rate compared to the model using only STFT or one-dimensional convolutional layers as feature extraction.