Convolutional Network Architecture Research Articles

Forecast Global Ionospheric TEC: Apply Modified U‐Net on VISTA TEC Data Set

AbstractThis work developed a modified U‐Net model (a convolutional network architecture) to predict global Total Electron Content (TEC) maps. The input includes the current global TEC map, the current F10.7, the time history of the Interplanetary Magnetic Field Bz and SYM‐H in the previous 4 days, the Hour of Day, and the Day of Year. The output is the global TEC map several hours or several days ahead. The modified U‐Net was trained and validated on a brand new TEC database, the VISTA (Video Imputation with SoftImpute, Temporal smoothing and Auxiliary data) TEC database. The VISTA TEC maps can reveal important large‐scale TEC structures and preserve mesoscale structures simultaneously. Taking advantage of the new neural network and the new database, our model achieves an root of the mean squared error from 1.2 TECU to 2.4 TECU as the prediction horizon increases from 1 hr to 7 days. In addition, the model could reveal multiscale structures in the predicted TEC maps.

FedHM: Practical federated learning for heterogeneous model deployments

In this paper, we propose a novel federated learning framework named FedHM that aims to address the challenge of training models on heterogeneous devices with varying architectures. Our approach enables the collaborative training of diverse local models by sharing a fully convolutional network (FCN) architecture that effectively extracts the local-to-global representations. By leveraging the weights with respect to this abstraction as common information across different DNN architectures, FedHM achieves efficient federated learning with minimal computational and communication overhead. We compare FedHM with three federated learning frameworks using two datasets for image classification tasks. Our results show that FedHM achieves high accuracy with considerably lower computational and communication costs compared to the other frameworks.

Multi-task prediction-based graph contrastive learning for inferring the relationship among lncRNAs, miRNAs and diseases.

Identifying the relationships among long non-coding RNAs (lncRNAs), microRNAs (miRNAs) and diseases is highly valuable for diagnosing, preventing, treating and prognosing diseases. The development of effective computational prediction methods can reduce experimental costs. While numerous methods have been proposed, they often to treat the prediction of lncRNA-disease associations (LDAs), miRNA-disease associations (MDAs) and lncRNA-miRNA interactions (LMIs) as separate task. Models capable of predicting all three relationships simultaneously remain relatively scarce. Our aim is to perform multi-task predictions, which not only construct a unified framework, but also facilitate mutual complementarity of information among lncRNAs, miRNAs and diseases. In this work, we propose a novel unsupervised embedding method called graph contrastive learning for multi-task prediction (GCLMTP). Our approach aims to predict LDAs, MDAs and LMIs by simultaneously extracting embedding representations of lncRNAs, miRNAs and diseases. To achieve this, we first construct a triple-layer lncRNA-miRNA-disease heterogeneous graph (LMDHG) that integrates the complex relationships between these entities based on their similarities and correlations. Next, we employ an unsupervised embedding model based on graph contrastive learning to extract potential topological feature of lncRNAs, miRNAs and diseases from the LMDHG. The graph contrastive learning leverages graph convolutional network architectures to maximize the mutual information between patch representations and corresponding high-level summaries of the LMDHG. Subsequently, for the three prediction tasks, multiple classifiers are explored to predict LDA, MDA and LMI scores. Comprehensive experiments are conducted on two datasets (from older and newer versions of the database, respectively). The results show that GCLMTP outperforms other state-of-the-art methods for the disease-related lncRNA and miRNA prediction tasks. Additionally, case studies on two datasets further demonstrate the ability of GCLMTP to accurately discover new associations. To ensure reproducibility of this work, we have made the datasets and source code publicly available at https://github.com/sheng-n/GCLMTP.

Performing Melanoma Diagnosis by an Effective Multi-view Convolutional Network Architecture

Performing Melanoma Diagnosis by an Effective Multi-view Convolutional Network Architecture

Self-attention (SA) temporal convolutional network (SATCN)-long short-term memory neural network (SATCN-LSTM): an advanced python code for predicting groundwater level.

Groundwater level prediction is important for effective water management. Accurately predicting groundwater levels allows decision-makers to make informed decisions about water allocation, groundwater abstraction rates, and groundwater recharge strategies. This study presents a novel model, the self-attention (SA) temporal convolutional network (SATCN)-long short-term memory neural network (SATCN-LSTM), for groundwater level prediction. The SATCN-LSTM model combines the advantages of the SATCN and LSTM models to overcome the limitations of the LSTM model. By utilizing skip connections and self-attention mechanisms, the SATCN model addresses the vanishing gradient problem, identifies relevant data, and captures both short- and long-term dependencies in time series data. By demonstrating the improved performance of the SATCN-LSTM model in terms of mean absolute error and root mean square error (RMSE), and by comparing these results with those reported in previous papers, we have highlighted the advancements and contributions of the proposed model. By improving prediction accuracy, the SATCN-LSTM model enables decision-makers to make informed choices regarding water allocation, groundwater abstraction rates, and drought preparedness. The SATCN-LSTM model contributes to the sustainable and efficient use of groundwater resources by providing reliable information for decision-making processes. The SATCN-LSTM model combines the temporal convolutional network (TCN) architecture with LSTM. TCN is known for its ability to capture short-term dependencies in time series data, while LSTM is effective at capturing long-term dependencies. By integrating both architectures, the SATCN-LSTM model can capture the complex temporal relationships at different scales, leading to improved prediction accuracy. Meteorological data were used to predict GWL. The SATCN-LSTM model outperformed the other models. The SATCN-LSTM model had the lowest mean absolute error (MAE) of 0.09, followed by the self-attention (SA) temporal convolutional network (SATCN) model with an MAE of 0.12. The SALSTM model had an MAE of 0.16, while the TCN-LSTM, temporal convolutional network (TCN), and LSTM models had MAEs of 0.17, 0.22, and 0.23, respectively. The SATCN-LSTM model had the lowest root mean square error of 0.14, followed by SATCN with an RMSE of 0.15. The study results indicated that the SATCN-LSTM model was a robust tool for predicting groundwater level.

Stroke-GFCN: ischemic stroke lesion prediction with a fully convolutional graph network.

The interpretation of image data plays a critical role during acute brain stroke diagnosis, and promptly defining the requirement of a surgical intervention will drastically impact the patient's outcome. However, determining stroke lesions purely from images can be a daunting task. Many studies proposed automatic segmentation methods for brain stroke lesions from medical images in different modalities, though heretofore results do not satisfy the requirements to be clinically reliable. We investigate the segmentation of brain stroke lesions using a geometric deep learning model that takes advantage of the intrinsic interconnected diffusion features in a set of multi-modal inputs consisting of computer tomography (CT) perfusion parameters. We propose a geometric deep learning model for the segmentation of ischemic stroke brain lesions that employs spline convolutions and unpooling/pooling operators on graphs to excerpt graph-structured features in a fully convolutional network architecture. In addition, we seek to understand the underlying principles governing the different components of our model. Accordingly, we structure the experiments in two parts: an evaluation of different architecture hyperparameters and a comparison with state-of-the-art methods. The ablation study shows that deeper layers obtain a higher Dice coefficient score (DCS) of up to 0.3654. Comparing different pooling and unpooling methods shows that the best performing unpooling method is the proportional approach, yet it often smooths the segmentation border. Unpooling achieves segmentation results more adapted to the lesion boundary corroborated with systematic lower values of Hausdorff distance. The model performs at the level of state-of-the-art models without optimized training methods, such as augmentation or patches, with a DCS of . We proposed and evaluated an end-to-end trainable fully convolutional graph network architecture using spline convolutional layers for the ischemic stroke lesion prediction. We propose a model that employs graph-based operations to predict acute stroke brain lesions from CT perfusion parameters. Our results prove the feasibility of using geometric deep learning to solve segmentation problems, and our model shows a better performance than other models evaluated. The proposed model achieves improved metric values for the DCS metric, ranging from 8.61% to 69.05%, compared with other models trained under the same conditions. Next, we compare different pooling and unpooling operations in relation to their segmentation results, and we show that the model can produce segmentation outputs that adapt to irregular segmentation boundaries when using simple heuristic unpooling operations.

ABTCN: an efficient hybrid deep learning approach for atmospheric temperature prediction.

Temperature prediction is an important and significant step for monitoring global warming and the environment to save and protect human lives. The climatology parameters such as temperature, pressure, and wind speed are time-series data and are well predicted with data driven models. However, data-driven models have certain constraints, due to which these models are unable to predict the missing values and erroneous data caused by factors like sensor failure and natural disasters. In order to solve this issue, an efficient hybrid model, i.e., attention-based bidirectional long short term memory temporal convolution network (ABTCN) architecture is proposed. ABTCN uses k-nearest neighbor (KNN) imputation method for handling the missing data. A bidirectional long short term memory (Bi-LSTM) network with self-attention mechanism and temporal convolutional network (TCN) model that aids in the extraction of features from complex data and prediction of long data sequence. The performance of the proposed model is evaluated in comparison to various state-of-the-art deep learning models using error metrics such as MAE, MSE, RMSE, and R2 score. It is observed that our proposed model is superior over other models with high accuracy.

Self-Supervised Learning for Multilevel Skeleton-Based Forgery Detection via Temporal-Causal Consistency of Actions

Skeleton-based human action recognition and analysis have become increasingly attainable in many areas, such as security surveillance and anomaly detection. Given the prevalence of skeleton-based applications, tampering attacks on human skeletal features have emerged very recently. In particular, checking the temporal inconsistency and/or incoherence (TII) in the skeletal sequence of human action is a principle of forgery detection. To this end, we propose an approach to self-supervised learning of the temporal causality behind human action, which can effectively check TII in skeletal sequences. Especially, we design a multilevel skeleton-based forgery detection framework to recognize the forgery on frame level, clip level, and action level in terms of learning the corresponding temporal-causal skeleton representations for each level. Specifically, a hierarchical graph convolution network architecture is designed to learn low-level skeleton representations based on physical skeleton connections and high-level action representations based on temporal-causal dependencies for specific actions. Extensive experiments consistently show state-of-the-art results on multilevel forgery detection tasks and superior performance of our framework compared to current competing methods.

'Earlier than Early' Detection of Breast Cancer in Israeli BRCA Mutation Carriers Applying AI-Based Analysis to Consecutive MRI Scans.

Female BRCA1/BRCA2 (=BRCA) pathogenic variants (PVs) carriers are at a substantially higher risk for developing breast cancer (BC) compared with the average risk population. Detection of BC at an early stage significantly improves prognosis. To facilitate early BC detection, a surveillance scheme is offered to BRCA PV carriers from age 25-30 years that includes annual MRI based breast imaging. Indeed, adherence to the recommended scheme has been shown to be associated with earlier disease stages at BC diagnosis, more in-situ pathology, smaller tumors, and less axillary involvement. While MRI is the most sensitive modality for BC detection in BRCA PV carriers, there are a significant number of overlooked or misinterpreted radiological lesions (mostly enhancing foci), leading to a delayed BC diagnosis at a more advanced stage. In this study we developed an artificial intelligence (AI)-network, aimed at a more accurate classification of enhancing foci, in MRIs of BRCA PV carriers, thus reducing false-negative interpretations. Retrospectively identified foci in prior MRIs that were either diagnosed as BC or benign/normal in a subsequent MRI were manually segmented and served as input for a convolutional network architecture. The model was successful in classification of 65% of the cancerous foci, most of them triple-negative BC. If validated, applying this scheme routinely may facilitate 'earlier than early' BC diagnosis in BRCA PV carriers.

Learning Social Spatio-Temporal Relation Graph in the Wild and a Video Benchmark.

Social relations are ubiquitous and form the basis of social structure in our daily life. However, existing studies mainly focus on recognizing social relations from still images and movie clips, which are different from real-world scenarios. For example, movie-based datasets define the task as the video classification, only recognizing one relation in the scene. In this article, we aim to study the problem of social relation recognition in an open environment. To close the gap, we provide the first video dataset collected from real-life scenarios, named social relation in the wild (SRIW), where the number of people can be huge and vary, and each pair of relations needs to be recognized. To overcome new challenges, we propose a spatio-temporal relation graph convolutional network (STRGCN) architecture, utilizing correlative visual features to recognize social relations intuitively. Our method decouples the task into two classification tasks: person-level and pair-level relation recognition. Specifically, we propose a person behavior and character module to encode moving and static features in two explicit ways. Then we take them as node features to build a relation graph with meaningful edges in a scene. Based on the relation graph, we introduce the graph convolutional network (GCN) and local GCN to encode social relation features which are used for both recognitions. Experimental results demonstrate the effectiveness of the proposed framework, achieving 83.1% and 40.8% mAP in person-level and pair-level classification. Moreover, the study also contributes to the practicality in this field.

A Novel Encoder-Decoder Structure-based Transformer for Fine-Resolution Remote Sensing Images

Full convolution neural network (FCN) based on an encoder-decoder structure has become a standard network in the semantic segmentation domain. Encoder-decoder architecture is an effective means to get finer-grained performance. Encoders constantly extract multilevel features, and then use decoders to gradually introduce low-level features into high-level features. Context information is critical for accurate segmentation, which is the main direction of semantic segmentation at present. So many efforts have been made to make better use of this kind of information, including codec structure, void convolution (expanded convolution), and attention mechanism. However, most of these schemes are based on Resnet or other variants of convolution network FCN, which makes it unable to get rid of the defective local receptive field of convolution itself. In this work, we introduce the pyramid visual converter (PVT) to replace the traditional full convolution network architecture, and design a novel encoder-decoder architecture to more effectively utilize the context information.

Interpreting Deep Graph Convolutional Networks with Spectrum Perspective

Graph convolutional network (GCN) architecture is the basis of many neural networks and has been widely used in processing graph-structured data. When dealing with large and sparse data, deeper GCN models are often required. However, the models suffer from performance degradation as the number of layers increases. The mainstream attribution of the current research is over-smoothing, and there are also gradient vanishing, training difficulties, etc., so a consensus cannot be reached. In this paper, we theoretically analyze the degradation problem by adopting spectral graph theory to globally consider the propagation and transformation components of the GCN architecture, and conclude that the over-smoothing problem caused by the propagation matrices is not the key factor for performance degradation. Afterwards, in addition to using conventional experimental methods, we proposed an experimental analysis strategy under the guidance of random matrix theory to analyze the singular value distribution of the model weight matrix. We concluded that the key factor leading to the degradation of model performance is the transformation component. In the context of a lack of consensus on the problem of model performance degradation, the paper proposes a systematic analysis strategy, as well as theoretical and empirical evidence.

Privacy protection in intelligent vehicle networking: A novel federated learning algorithm based on information fusion

Federated learning is an effective technique to solve the problem of information fusion and information sharing in intelligent vehicle networking. However, most of the existing federated learning algorithms generally have the risk of privacy leakage. To address this security risk, this paper proposes a novel personalized federated learning with privacy preservation (PDP-PFL) algorithm based on information fusion. In the first stage of its execution, the new algorithm achieves personalized privacy protection by grading users’ privacy based on their privacy preferences and adding noise that satisfies their privacy preferences. In the second stage of its execution, PDP-PFL performs collaborative training of deep models among different in-vehicle terminals for personalized learning, using a lightweight dynamic convolutional network architecture without sharing the local data of each terminal. Instead of sharing all the parameters of the model as in standard federated learning, PDP-PFL keeps the last layer local, thus adding another layer of data confidentiality and making it difficult for the adversary to infer the image of the target vehicle terminal. It trains a personalized model for each vehicle terminal by “local fine-tuning”. Based on experiments, it is shown that the accuracy of the proposed new algorithm for PDP-PFL calculation can be comparable to or better than that of the FedAvg algorithm and the FedBN algorithm, while further enhancing the protection of data privacy.

Improved Transformer Based Hashing for Cased-based Pneumonia Retrieval

The deep hash method has been proved to be the most effective nearest neighbor search technique in large-scale image retrieval. However, up to now, deep hashing in medical image retrieval has been dominated by convolution network architecture, such as Resnet. Inspired by Transformer, we integrate Transformer into the deep learning framework. Our framework is mainly composed of two main modules: (1) In order to solve the problem of big difference among different subclasses of medical image categories, the Transformer model is improved in this paper. The external attention is used to replace the self-attention in the original Transformer model, and the feature acquisition ability of the model is improved by capturing the correlation between samples. (2) In addition, in order to learn compact binary coding, we adopt Bayesian framework for end-to-end learning. We conduct experiments on ChestX-ray Pneumonia dataset, and compare the proposed algorithm with the commonly used depth hashing method. Experimental results show that the MAP values under different hash bit codes are better than the existing deep hash methods, which proves the effectiveness of the proposed method.

Application of generative adversarial neural networks for the formation of databases in scanning tunneling microscopy

We discuss the development of a technique for automatic generation of databases of images obtained with a scanning tunneling microscope. An analysis of state-of-the-art methods and means of automatic processing of images obtained from probe and electron microscopes is carried out. We proposed using generative-adversarial networks for generating images taken with a scanning tunneling microscope to form training databases of images. A process of training and comparison of deep convolutional generative adversarial network (DCGAN) architectures using the OpenCV and Keras libraries together with TensorFlow is described, with the best of them identified by computing the metrics IS, FID, and KID. The scaling of images obtained from DCGAN is performed using a method of fine tuning of a super-resolution generative adversarial neural network (SRGAN) and bilinear interpolation based on the Python programming language. An analysis of calculated quantitative metrics values shows that the best results of image generation are obtained using DCGAN96 and SRGAN. It is found that FID and KID metric values for SRGAN method are better than values for bilinear interpolation in all cases except for DCGAN32. All calculations are performed on a GTX GeForce 1070 video card. A method for automatic generation of a scanning tunneling microscope image database based on the stepwise application of DCGAN and SRGAN is developed. Results of generation and comparison of the original image, the one obtained with DCGAN96 and the enlarged image with SRGAN are presented.

Combined space–time reduced-order model with three-dimensional deep convolution for extrapolating fluid dynamics

There is a critical need for efficient and reliable active flow control strategies to reduce drag and noise in aerospace and marine engineering applications. While traditional full-order models based on the Navier–Stokes equations are not feasible, advanced model reduction techniques can be inefficient for active control tasks, especially with strong non-linearity and convection-dominated phenomena. Using convolutional recurrent autoencoder network architectures, deep-learning-based reduced-order models have been recently shown to be effective while performing several orders of magnitude faster than full-order simulations. However, these models encounter significant challenges outside the training data, limiting their effectiveness for active control and optimization tasks. In this study, we aim to improve the extrapolation capability by modifying the network architecture and integrating coupled space–time physics as an implicit bias. Reduced-order models via deep learning generally employ decoupling in spatial and temporal dimensions, which can introduce modeling and approximation errors. To alleviate these errors, we propose a novel technique for learning coupled spatial–temporal correlation using a three-dimensional convolution network. We assess the proposed technique against a standard encoder–propagator–decoder model and demonstrate a superior extrapolation performance. To demonstrate the effectiveness of the three-dimensional convolution network, we consider a benchmark problem of the flow past a circular cylinder at laminar flow conditions and use the spatiotemporal snapshots from the full-order simulations. Our proposed three-dimensional convolution architecture accurately captures the velocity and pressure fields for varying Reynolds numbers. Compared to the standard encoder–propagator–decoder network, the spatiotemporal-based three-dimensional convolution network improves the prediction range of Reynolds numbers outside of the training data.

STJA-GCN: A Multi-Branch Spatial–Temporal Joint Attention Graph Convolutional Network for Abnormal Gait Recognition

Early recognition of abnormal gait enables physicians to determine a prompt rehabilitation plan for patients for the most effective treatment and care. The Kinect depth sensor can easily collect skeleton data describing the position of joints in the human body. However, the default human skeleton model of Kinect includes an excessive number of many joints, which limits the accuracy of the gait recognition methods and increases the computational resources required. In this study, we propose an optimized human skeleton model for the Kinect system and streamline the joints using a center-of-mass calculation. We integrate several techniques to propose an end-to-end, spatial–temporal, joint attention graph convolutional network (STJA-GCN) architecture. We conducted experiments with a fivefold cross-validation on two common datasets of information on abnormal gaits to evaluate the performance of the proposed method. The results show that the STJA-GCN achieved 93.17 and 92.08% accuracy on the two datasets, and compared to the original spatial–temporal graph convolutional network (ST-GCN), the recognition accuracy increases by 9.22 and 20.65%, respectively. Overall, the results demonstrate that the STJA-GCN can accurately recognize abnormal gaits and, thus, can support low-cost rehabilitation assessments at community hospitals or in patients’ homes.

Automatic brain tumor detection using CNN transfer learning approach.

Automatic brain tumor detection is a challenging task as tumors vary in their position, mass, nature, and similarities found between brain lesions and normal tissues. The tumor detection is vital and urgent as it is related to the lifespan of the affected person. Medical experts commonly utilize advanced imaging practices such as magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound images to decide the presence of abnormal tissues. It is a very time-consuming task to extract the tumor information from the enormous quantity of information produced by MRI volumetric data examination using a manual approach. In manual tumor detection, precise identification of tumor along with its details is a complex task. Henceforth, reliable and automatic detection systems are vital. In this paper, convolutional neural network based automated brain tumor recognition approach is proposed to analyze the MRI images and classify them into tumorous and non-tumorous classes. Various convolutional neutral network architectures like Alexnet, VGG-16, GooGLeNet, and RNN are explored and compared together. The paper focuses on the tuning of the hyperparameters for the two architectures namely Alexnet and VGG-16. Exploratory results on BRATS 2013, BRATS 2015, and OPEN I dataset with 621 images confirmed that the accuracy of 98.67% is achieved using CNN Alexnet for automatic detection of brain tumors while testing on 125 images.

Ensemble the recent architectures of deep convolutional networks for skin diseases diagnosis

AbstractIf you decided to utilize deep learning in any image processing application, you would be faced with the issue, “Which architecture should I use?” due to the proliferation of existing CNN models and their advancements. Unfortunately, your answer will only be partially correct because each alternative has its advantage. The underlying idea of this research is to combine recent CNN models instead of selecting just one for optimal accuracy. Our study applied this idea to color lesion images to diagnose skin diseases. By ensembling, the recent CNNs, over 99% classification accuracy and over 97% sensitivity were achieved for the ISIC‐2017 dataset, which contains 2000 lesion images. Our mean sensitivity and AUC values for classifying 10000 color lesion images into seven different skin diseases (ISIC‐2018) were 0.825% and 0.922%, respectively. In categorizing over 25000 images from the ISIC 2019 dataset, our suggested technique achieved a mean sensitivity of over 90%.

Not getting in too deep: A practical deep learning approach to routine crystallisation image classification.

Using a relatively small training set of ~16 thousand images from macromolecular crystallisation experiments, we compare classification results obtained with four of the most widely-used convolutional deep-learning network architectures that can be implemented without the need for extensive computational resources. We show that the classifiers have different strengths that can be combined to provide an ensemble classifier achieving a classification accuracy comparable to that obtained by a large consortium initiative. We use eight classes to effectively rank the experimental outcomes, thereby providing detailed information that can be used with routine crystallography experiments to automatically identify crystal formation for drug discovery and pave the way for further exploration of the relationship between crystal formation and crystallisation conditions.