Neural Network-based Architecture Research Articles

Presentation-level Privacy Protection Techniques for Automated Face Recognition—A Survey

The use of Biometric Facial Recognition (FR) systems have become increasingly widespread, especially since the advent of deep neural network-based architectures. Although FR systems provide substantial benefits in terms of security and safety, the use of these systems also raises significant privacy concerns. This article discusses recent advances in facial identity hiding techniques, focusing on privacy protection approaches that hide or protect facial biometric data before camera devices capture the data. Moreover, we also discuss the state-of-the-art methods used to evaluate such privacy protection techniques. The primary motivation of this survey is to assess the relative performance of facial privacy protection methods and identify open challenges and future work that needs to be considered in this research area.

A Machine Learning Method for Automated Description and Workflow Analysis of First Trimester Ultrasound Scans.

Obstetric ultrasound assessment of fetal anatomy in the first trimester of pregnancy is one of the less explored fields in obstetric sonography because of the paucity of guidelines on anatomical screening and availability of data. This paper, for the first time, examines imaging proficiency and practices of first trimester ultrasound scanning through analysis of full-length ultrasound video scans. Findings from this study provide insights to inform the development of more effective user-machine interfaces, of targeted assistive technologies, as well as improvements in workflow protocols for first trimester scanning. Specifically, this paper presents an automated framework to model operator clinical workflow from full-length routine first-trimester fetal ultrasound scan videos. The 2D+t convolutional neural network-based architecture proposed for video annotation incorporates transfer learning and spatio-temporal (2D+t) modelling to automatically partition an ultrasound video into semantically meaningful temporal segments based on the fetal anatomy detected in the video. The model results in a cross-validation A1 accuracy of 96.10%, F1 = 0.95, precision = 0.94 and recall = 0.95. Automated semantic partitioning of unlabelled video scans (n=250) achieves a high correlation with expert annotations (ρ = 0.95, p = 0.06). Clinical workflow patterns, operator skill and its variability can be derived from the resulting representation using the detected anatomy labels, order, and distribution. It is shown that nuchal translucency (NT) is the toughest standard plane to acquire and most operators struggle to localize high-quality frames. Furthermore, it is found that newly qualified operators spend 25.56% more time on key biometry tasks than experienced operators.

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

This work investigates classification of emotions from full-body movements by using a novel Convolutional Neural Network-based architecture. The model is composed of two shallow networks processing in parallel where the 8-bit RGB images obtained from time intervals of 3D-positional data are the inputs. One network performs a coarse-grained modelling in the time domain while the other one applies a fine-grained modelling. We show that combining different temporal scales into one architecture improves the classification results of a dataset composed of short excerpts of the performances of professional dancers who interpreted four affective states: anger, happiness, sadness, and insecurity. Additionally, we investigate the effect of data chunk duration, overlapping, the size of the input images and the contribution of several data augmentation strategies for our proposed method. Better recognition results were obtained when the duration of a data chunk was longer, and this was further improved by applying balanced data augmentation. Moreover, we test our method on other existing motion capture datasets and compare the results with prior art. In all of the experiments, our results surpassed the state-of-the-art approaches, showing that this method generalizes across diverse settings and contexts.

Sketch2Process: End-to-End BPMN Sketch Recognition Based on Neural Networks

Process models play an important role in various software engineering contexts. Among others, they are used to capture business-related requirements and provide the basis for the development of process-oriented applications in low-code/no-code settings. To support modelers in creating, checking, and maintaining process models, dedicated tools are available. While these tools are generally considered as indispensable to capture process models for their later use, the initial version of a process model is often sketched on a whiteboard or a piece of paper. This has been found to have great advantages, especially with respect to communication and collaboration. It, however, also creates the need to subsequently transform the model sketch into a digital counterpart that can be further processed by modeling and analysis tools. Therefore, to automate this task, various so-called sketch recognition approaches have been defined in the past. Yet, these existing approaches are too limited for use in practice, since they, for instance, require sketches to be created on a digital device or do not address the recognition of edges or textual labels. Against this background, we use this paper to introduce Sketch2Process, the first end-to-end sketch recognition approach for process models captured using BPMN. Sketch2Process uses a neural network-based architecture to recognize the shapes, edges, and textual labels of highly expressive process models, covering 25 types of BPMN elements. To train and evaluate our approach, we created a dataset consisting of 704 hand-drawn and manually annotated BPMN models. Our experiments demonstrate that our approach is highly accurate and consistently outperforms the state of the art.

DFGNN: An interpretable and generalized graph neural network for deepfakes detection

Deepfakes are generated using sophisticated deep-learning models to create fake images or videos. As the techniques for creating deepfakes improve, issues like defamation, impersonation, fraud, and misinformation on social media are becoming more prevalent. Existing deep learning-based deepfakes detection models are not interpretable and don’t generalize well when tested across diverse deepfakes generating techniques and datasets. Therefore, the creation of reliable and effective deepfakes detection algorithms is required which are not only generalizable but also interpretable. This paper introduces a novel graph neural network-based architecture to identify hyper-realist deepfake content. Currently, very limited efforts have been done to address the problem of deepfakes detection using graph neural networks. The proposed model is based on the pyramid structure that takes advantage of multi-scale images property by extracting features with progressively smaller spatial sizes as layer depth increases. The method first sliced the image into patches, which are referred to as nodes, and then constructed a graph by connecting the nearest neighbors. To transform and exchange information between all nodes, the proposed model has two basic modules: GraphNet, which uses graph convolution layers to aggregate and update graph information, and FFN, which has linear layers for the transformation of node features. The effectiveness of the method is assessed using the diverse Deepfake Detection Challenge dataset (DFDC), FaceForensics++ (FF++), World Leaders dataset (WLRD), and the Celeb-DF. To demonstrate the generalizability of the proposed method for accurate deepfakes detection, open/close set, cross-set, and cross-corpora evaluations were also performed. The AUC values of 0.98 on FF++, 0.95 on Celeb-DF, 0.92 on DFDC, and 1.00 on most of the sets of WLRD datasets demonstrate the efficacy of the method for identifying manipulated facial images produced using various deepfakes techniques.

Fault Diagnosis of Bearings and Gears Based on LiteNet With Feature Aggregation

Significant progress has been made in the current fault diagnosis algorithms. However, they do not consider computational resources and require expensive equipment to complete the training of the models. To immediately complete model training and obtain higher accuracy rates using cheaper equipment, reduce the equipment cost in the industry, and build a bridge for industrial fault diagnosis with neural networks, this paper proposes a convolutional neural network-based architecture that uses a small number of computational resources with high accuracy. Simultaneously, a loss function is proposed that can further improve the accuracy of the network model without consuming too many computational resources. According to experimental comparisons, the proposed technique has clear advantages in terms of accuracy, computational resources, training time, and stability.

VieCap4H Challenge 2021: Vietnamese Image Captioning for Healthcare Domain using Swin Transformer and Attention-based LSTM

This study presents our approach on the automatic Vietnamese image captioning for healthcare domain in text processing tasks of Vietnamese Language and Speech Processing (VLSP) Challenge 2021, as shown in Figure~\ref{fig:example}. In recent years, image captioning often employs a convolutional neural network-based architecture as an encoder and a long short-term memory (LSTM) as a decoder to generate sentences. These models perform remarkably well in different datasets. Our proposed model also has an encoder and a decoder, but we instead use a Swin Transformer in the encoder, and a LSTM combined with an attention module in the decoder. The study presents our training experiments and techniques used during the competition. Our model achieves a BLEU4 score of 0.293 on the vietCap4H dataset, and the score is ranked the 3$^{rd}$ place on the private leaderboard. Our code can be found at \url{https://git.io/JDdJm}.

How many neurons are needed to make a short-term prediction of the Bitcoin exchange rate?

The goal of our work was to select a neural network architecture that would give the best prediction of the Bitcoin exchange rate using historical data. Our work fits into the very important topic of predicting the value of the cryptocurrency exchange rate, and makes use of recent data which, as a result of the high Bitcoin exchange rate dynamics of the last year, differs significantly from those of previous years. We propose and test a number of neural network-based architectures and conduct a discussion of the results. Unlike previous state-of-the-art works, we conducted a comprehensive comparison of three different neural network-based models: MLP (multilayer perceptron), LSTM (long short-term memory) and CNN (convolutional neural network). We tested them for a wide range of parameters. The results we present are, to the best of our knowledge, the most up to date when it comes to the application of artificial intelligence methods for the prediction of cryptocurrency exchange rates. The best-performing architectures were used for a website that gives real-time predictions of the Bitcoin exchange rate. The website is available at http://stpbtc-ii.up.krakow.pl/. Source codes of our research are available to download in order to make our experiment reproducible.

Pathological Analysis of Brain Tumor in MRI Images Using CAR-UNET Model Architecture

The most dangerous type of organ failure in humans is a brain tumor. A brain tumor’s incorrect segmentation and classification are critical for treatment and early diagnosis. Several Deep neural network-based architectures have recently been developed to improve brain tumor classification performance. However, brain tumor classification performance must be improved, which is a difficult area of research. The goal of this study is to analyze different types of brain tumors and how to classify them to increase the survival rate of people with brain tumors. The CAR-U-Net (Concatenation and Residual) image classification method is proposed in this paper to help with brain tumor segmentation and classification research. The baseline U-Net architecture employs concatenation and residual connections. The changes in the network help in discovering varied features by expanding the specific receptive field. We consider two factors for a better diagnosis system: finding missing feature maps and eliminating unfeatured feature maps. The residual connection can solve the over learn or Null feature map problem, while the concatenation connection can solve the missing feature maps problems. This model has been tested on the BraTS2017 Challenge datasets. The network’s concatenation and residual connections, which are used for better deep supervision and tumor differentiation, are accurate. The accuracy, sensitivity, specificity, and dice score were used to compare the performance quantitatively. The proposed system achieved 94.12% Accuracy and 97.16% sensitivity which is higher than the existing systems such as U-net, Residual U-net and, Bayesian SVNN. Also, the proposed system got the dice score coefficient of 90.32. which is higher compared to conventional U-net and Residual U-net. The proposed CAR-UNET model outperformed current best-practice techniques. The high-performance capacity of this model can help bioinformatics, medicine, and early diagnosis researchers.

A comparison of classical and machine learning-based phenotype prediction methods on simulated data and three plant species.

Genomic selection is an integral tool for breeders to accurately select plants directly from genotype data leading to faster and more resource-efficient breeding programs. Several prediction methods have been established in the last few years. These range from classical linear mixed models to complex non-linear machine learning approaches, such as Support Vector Regression, and modern deep learning-based architectures. Many of these methods have been extensively evaluated on different crop species with varying outcomes. In this work, our aim is to systematically compare 12 different phenotype prediction models, including basic genomic selection methods to more advanced deep learning-based techniques. More importantly, we assess the performance of these models on simulated phenotype data as well as on real-world data from Arabidopsis thaliana and two breeding datasets from soy and corn. The synthetic phenotypic data allow us to analyze all prediction models and especially the selected markers under controlled and predefined settings. We show that Bayes B and linear regression models with sparsity constraints perform best under different simulation settings with respect to explained variance. Further, we can confirm results from other studies that there is no superiority of more complex neural network-based architectures for phenotype prediction compared to well-established methods. However, on real-world data, for which several prediction models yield comparable results with slight advantages for Elastic Net, this picture is less clear, suggesting that there is a lot of room for future research.

DNN-PP: A novel Deep Neural Network approach and its applicability in drug-related property prediction

In this study we propose the Deep Neural Network Properties Predictor (DNN-PP) which is a deep neural network-based architecture that learns molecular representation to enhance the process of molecular properties prediction. This work’s main novelty is the use of two separate blocks of operations, where each block learns a representation. Then the two latent feature vectors are combined and fed into a few dense layers ended by a regression or classification layer. Furthermore, our DNN-PP employs a stacked attention mechanism and proposes a collection of atomic and bond features to provide additional insights into a molecular structure. Therefore, while various approaches for molecular properties prediction have already been developed, our architecture can capture detailed structure–property relationship information. The performance of the proposed DNN-PP was tested on a standard benchmark for molecular machine learning. The results show that our method outperforms state-of-the-art models. Take for instance hydration free energies prediction, where DNN-PP obtains the smallest predictive error, i.e., 0.73 on the test set, while the best competitive approach reports an error of 1.07 on unseen data (lower is better).

CRCCN-Net: Automated framework for classification of colorectal tissue using histopathological images

Colorectal cancer has a high mortality rate that continuously affects human life globally. Early detection of it extends human life and helps in preventing disease. Histopathological inspection is a frequently used approach to diagnose and detect colorectal cancer. Visual inspection of histopathological diagnosis requires more inspection time and the decision depends on the subjective perception of clinicians. This work proposed lightweight, less complex convolutional neural network-based architecture for automated classification of multi-class colorectal tissue histopathological images using two publicly available datasets, colorectal histology, and NCT-CRC-HE-100K, respectively. Histopathological images are provided as input to pre-trained models Xception, InceptionResNetV2, DenseNet121, VGG16, and the proposed network colorectal cancer classification convolutional neural network. This is the first study that compares the computational time of different deep learning architectures for the classification of colorectal tissue. The developed network requires less computational time for training compared to other pre-trained models. Accuracy, sensitivity, precision, false-positive rate, false-negative rate, specificity, F-1 score, and area under the curve have been used to evaluate the performance of the proposed architecture. The proposed network attained an accuracy of 93.50%, and 96.26% on the colorectal histology dataset, and NCT-CRC-HE-100K dataset, respectively. On the merged dataset, an accuracy of 99.21% is achieved by the newly developed network. The comparative analysis shows that the proposed framework outperformed existing state-of-the-art approaches. Clinicians may install the presented CRCCN-Net to confirm the diagnosis in the hospitals.

Hurricane Damage Detection From Satellite Imagery Using Convolutional Neural Networks

Hurricanes are one of the most disastrous natural phenomena occurring on Earth that cause loss of human lives and immense damage to property as well. For assessment of this damage, windshield survey is commonly used, which is an error-prone and time-consuming method. For solving this problem, computer vision comes into the picture. In this paper, a convolutional neural network-based architecture has been proposed to classify the post-hurricane satellite imagery into damaged and undamaged building classes accurately. The model consists of five convolutional and five pooling layers followed by a flattening layer and two dense layers. For this, a dataset of Hurricane Harvey has been considered having 23000 satellite images each of size 128 X 128 pixels. With the proposed model, the author has achieved an accuracy of 92.91%, F1-score of 93%, sensitivity of 93.34%, specificity of 92.47%, and precision of 92.65% at a learning rate of 0.0001 and 30 epochs. Also, low false positive rate of 7.53% and false negative rate of 6.66% were obtained.

Complete Blood Cell Detection and Counting Based on Deep Neural Networks

Complete blood cell (CBC) counting has played a vital role in general medical examination. Common approaches, such as traditional manual counting and automated analyzers, were heavily influenced by the operation of medical professionals. In recent years, computer-aided object detection using deep learning algorithms has been successfully applied in many different visual tasks. In this paper, we propose a deep neural network-based architecture to accurately detect and count blood cells on blood smear images. A public BCCD (Blood Cell Count and Detection) dataset is used for the performance evaluation of our architecture. It is not uncommon that blood smear images are in low resolution, and blood cells on them are blurry and overlapping. The original images were preprocessed, including image augmentation, enlargement, sharpening, and blurring. With different settings in the proposed architecture, five models are constructed herein. We compare their performance on red blood cells (RBC), white blood cells (WBC), and platelet detection and deeply investigate the factors related to their performance. The experiment results show that our models can recognize blood cells accurately when blood cells are not heavily overlapping.

Deep Neural Network Models for Colon Cancer Screening.

Simple SummaryDeep learning models have been shown to achieve high performance in diagnosing colon cancer compared to conventional image processing and hand-crafted machine learning methods. Hence, several studies have focused on developing hybrid learning, end-to-end, and transfer learning techniques to reduce manual interaction and for labelling the regions of interest. However, these weak learning techniques do not always provide a clear diagnosis. Therefore, it is necessary to develop a clear explainable learning method that can highlight factors and form the basis of clinical decisions. However, there has been little research carried out employing such transparent approaches. This study discussed the aforementioned models for colon cancer diagnosis.Early detection of colorectal cancer can significantly facilitate clinicians’ decision-making and reduce their workload. This can be achieved using automatic systems with endoscopic and histological images. Recently, the success of deep learning has motivated the development of image- and video-based polyp identification and segmentation. Currently, most diagnostic colonoscopy rooms utilize artificial intelligence methods that are considered to perform well in predicting invasive cancer. Convolutional neural network-based architectures, together with image patches and preprocesses are often widely used. Furthermore, learning transfer and end-to-end learning techniques have been adopted for detection and localization tasks, which improve accuracy and reduce user dependence with limited datasets. However, explainable deep networks that provide transparency, interpretability, reliability, and fairness in clinical diagnostics are preferred. In this review, we summarize the latest advances in such models, with or without transparency, for the prediction of colorectal cancer and also address the knowledge gap in the upcoming technology.

Enhancing imputation techniques performance utilizing uncertainty aware predictors and adversarial learning

One crucial problem for applying machine learning algorithms to real-world datasets is missing data. The objective of data imputation is to fill the missing values in a dataset to resemble the completed dataset as accurately as possible. Many methods are proposed in the literature that mostly differs on the objective function and types of the variables considered. The performance of traditional machine learning methods is low when there is a nonlinear and complex relationship between features. Recently, deep learning methods are introduced to estimate data distribution and generate values for missing entries. However, these methods are originally developed for large datasets and custom data types such as image, video, and text. Thus, adopting these methods for small and structured datasets that are prevalent in real-world applications is not straightforward and often yields unsatisfactory results. Also, both types of methods do not consider uncertainty in the imputed data. We address these issues by developing a simple neural network-based architecture that works well with small and tabular datasets and utilizing a novel adversarial strategy to estimate the uncertainty of imputed data. The estimated uncertainty scores of features are then passed to the imputer module, and it fills the missing values by paying more attention to more reliable feature values. It results in an uncertainty-aware imputer with a promising performance. Extensive experiments conducted on some real-world datasets confirm that the proposed methods considerably outperform state-of-the-art imputers. Meanwhile, their execution time is not costly compared to peer state-of-the-art methods.

Monocular three-dimensional object detection using data augmentation and self-supervised learning in autonomous driving

Monocular three-dimensional (3D) object detection (OD) is an essential and challenging task in the domain of autonomous driving. Modern convolution neural network-based architectures for OD heavily rely on data augmentation (DA) and self-supervised learning (SSL). However, they have been relatively less explored for monocular 3D OD, especially in the field of autonomous driving. DAs for two-dimensional OD techniques do not directly extend to the 3D objects. Literature shows that this requires adaptation of the 3D geometry of the input scene and synthesis of new viewpoints. This requires accurate depth information of the scene which may not be available always. We propose augmentations for monocular 3D OD without creating view synthesis. The proposed method uses DA with SSL approach via multiobject labeling as the pretext task. We evaluate the proposed DA-SSL approach on RTM3D detection model (baseline), with and without the application of DA. The results demonstrate improvements between 2% and 3% in mAP 3D and 0.9% to 1.5% BEV scores using SSL over the baseline scores. We propose an inverse class frequency weighted (ICFW) mAP score that highlights improvements in detection for low-frequency classes in a class imbalanced datasets with long tails. We observe improvements in both ICFW mAP 3D and Bird’s Eye View (BEV) scores to take into account the class imbalance in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) validation dataset. We achieve 4% to 5% increase in ICFW metrics with the pretext task.

Deep learning-based brain tumor segmentation on limited sequences of magnetic resonance imaging.

2054 Background: Deep learning algorithms trained to segment brain tumors from magnetic resonance imaging (MRI) perform well in ideal conditions supplied by curated datasets. An example is the MICCAI BraTS 2018 dataset, which provides a set of four MRI sequences (T1, T1 post-gadolinium contrast-enhanced, T2, FLAIR) on 285 samples, along with ground truth segmentations annotated by experts. In real-world settings, however, it is not uncommon for patients to undergo MRI with a limited number of image sequence acquisitions. We examined the effect of restricting the imaging sequence set when training a deep learning model for brain tumor segmentation. Our goal was to identify a limited subset that still achieved acceptably high performance. Methods: In our experiments, we used the convolutional neural network-based U-Net architecture. Instead of the standard BraTS task, we focused on sub-segmenting specific areas of brain tumors: the active, enhancing tumor (AT) and the tumor core (TC; AT plus the cystic/necrotic core). We trained the architecture on 285 samples of the BraTS 2018 dataset using one or both of two sequences: T1 contrast-enhanced (T1CE) and FLAIR. Using each training model, we predicted segmentation masks given a volumetric MR image of a brain tumor on 66 samples in the held-out validation dataset. We submitted the predicted segmentations to the Center for Biomedical Image Computing and Analytics (CBICA) imaging portal for evaluation, which reports prediction accuracy by returning dice scores on each prediction. Results: As shown in the Table, the U-Net model using T1CE alone yielded high performances, with median dice scores of 0.84461 and 0.88267, when segmenting AT and TC, respectively. On the other hand, using FLAIR alone generated lower performances on the same tasks of segmenting the AT (0.25996) and the TC regions (0.51153). Combining T1CE and FLAIR provided no additional performance improvement compared with T1CE used as a standalone sequence. Conclusions: We achieved high performances in two brain tumor sub-segmentation tasks using a limited number of MRI sequences in training a deep-learning algorithm. T1CE alone produced high performances on both TC and AT segmentations. Given the ubiquity of the T1CE sequence, the ability to achieve high performance tumor segmentations in the face of limited image sequence availability is critical when applying our algorithm in real-world clinical or research settings. [Table: see text]

WBATimeNet: A deep neural network approach for VM Live Migration in the cloud

Live Migration (LM) of Virtual Machines (VMs) is an important activity for most cloud platforms, including Azure. LM impacts the availability of VMs, due to which workloads running on them may get affected adversely. Azure cloud infrastructure consists of many nodes and their associated VMs. Many VMs (up to a few million) are candidates to undergo LM at any given time. During the process of LM, it is essential to ensure very low or no adverse impact on the running workload of the customer. Thus, of the millions of VMs, predicting which ones to live migrate based on their low utilization of resources is a critical task. To solve this, we propose a novel deep learning network WBATimeNet, which uses Multivariate Time Series data of Memory, CPU, and Disk to predict which VM should be live migrated. WBATimeNet is a deep neural network-based architecture that uses White-box Adversarial Training to address the high variability and uncertainty of time series data. The experimental results illustrate that WBATimeNet outperforms baseline models by a large margin and helps maintain the increased availability of VMs in Azure during the LM process.

Visualization of Customized Convolutional Neural Network for Natural Language Recognition.

For analytical approach-based word recognition techniques, the task of segmenting the word into individual characters is a big challenge, specifically for cursive handwriting. For this, a holistic approach can be a better option, wherein the entire word is passed to an appropriate recognizer. Gurumukhi script is a complex script for which a holistic approach can be proposed for offline handwritten word recognition. In this paper, the authors propose a Convolutional Neural Network-based architecture for recognition of the Gurumukhi month names. The architecture is designed with five convolutional layers and three pooling layers. The authors also prepared a dataset of 24,000 images, each with a size of 50 × 50. The dataset was collected from 500 distinct writers of different age groups and professions. The proposed method achieved training and validation accuracies of about 97.03% and 99.50%, respectively for the proposed dataset.