Neural Network-based Architecture Research Articles

Ransomware Classification with Deep Neural Network and Bi-LSTM

Malicious attacks, malware, and ransomware families present essential risks to cybersecurity and may result in significant harm to computer systems, data clusters, networks, and mobile apps across a range of industries. Recently, there has been interest in ransomware classification using DNN and Bi-LSTM. DNN, a subset of machine learning techniques, has been found to improve ransomware detection and classification precision and efficacy. Ransomware has been affecting commercial, public, and governmental organizations' networks and computer systems for more than a decade, enabling new dynamic detection techniques to help DNNs detect ransomware. However, deep neural network-based architectures and DL classifiers (such as DNN, and Bi-LSTM classifiers) will be employed to detect ransomware. These networks may learn to correctly identify and categorize new ransomware incidents by integrating various datasets, including known and unknown ransomware samples. The classification of ransomware detection has been thoroughly investigated, and a model incorporating classic DL techniques with DNN and Bi-LSTM-based architecture will be proposed. A model execution experiment will be carried out to facilitate comparative testing of various approaches. This study focuses on the detection and classification of ransomware using DNN and Bi-LSTM. This study provides the groundwork for future investigations into the issues with ransomware detection. To protect against several ransomware attack types, deep neural networks have become an effective tool for ransomware detection. These networks combine machine learning and deep learning techniques.

Enhanced Linear and Vision Transformer-Based Architectures for Time Series Forecasting

Time series forecasting has been a challenging area in the field of Artificial Intelligence. Various approaches such as linear neural networks, recurrent linear neural networks, Convolutional Neural Networks, and recently transformers have been attempted for the time series forecasting domain. Although transformer-based architectures have been outstanding in the Natural Language Processing domain, especially in autoregressive language modeling, the initial attempts to use transformers in the time series arena have met mixed success. A recent important work indicating simple linear networks outperform transformer-based designs. We investigate this paradox in detail comparing the linear neural network- and transformer-based designs, providing insights into why a certain approach may be better for a particular type of problem. We also improve upon the recently proposed simple linear neural network-based architecture by using dual pipelines with batch normalization and reversible instance normalization. Our enhanced architecture outperforms all existing architectures for time series forecasting on a majority of the popular benchmarks.

Analyzing the Combined Effects of Sarcasm and Emotion for Gender Prediction

“Women are bitchy but men are sarcastic”, such comments reveal the relationship between gender and sarcasm. Automatic gender identification can play a crucial role in services that depend on data about a user’s background. Although for some social media users the gender of a user is typically unavailable due to privacy and anonymity. Based on the notion that male and female users may express their thoughts and sentiments differently in their posts, social media accounts can be examined using their posts (text) in order to automatically identify the gender of an anonymous user. In the current work, efforts are made in analyzing the effects of emotion and sarcasm intended by the users in their tweets for predicting gender. Sarcasm + emotion aided gender prediction systems are developed using different machine learning and neural network-based architectures. In our developed model, tweet features are extracted by using pre-trained GloVe embeddings. The sarcasm intensity is concatenated with the corresponding tweet representation and at last classification layer is used to predict the gender labels. For the experimentation purpose, the PAN-2018 dataset has been used. We have also shown the effect of utilizing emotion, and sarcasm information over gender prediction using different models

Exploring deep echo state networks for image classification: a multi-reservoir approach

Echo state networks (ESNs) belong to the class of recurrent neural networks and have demonstrated robust performance in time series prediction tasks. In this study, we investigate the capability of different ESN architectures to capture spatial relationships in images without transforming them into temporal sequences. We begin with three pre-existing ESN-based architectures and enhance their design by incorporating multiple output layers, customising them for a classification task. Our investigation involves an examination of the behaviour of these modified networks, coupled with a comprehensive performance comparison against the baseline vanilla ESN architecture. Our experiments on the MNIST data set reveal that a network with multiple independent reservoirs working in parallel outperforms other ESN-based architectures for this task, achieving a classification accuracy of 98.43%. This improvement on the classical ESN architecture is accompanied by reduced training times. While the accuracy of ESN-based architectures lags behind that of convolutional neural network-based architectures, the significantly lower training times of ESNs with multiple reservoirs operating in parallel make them a compelling choice for learning spatial relationships in scenarios prioritising energy efficiency and rapid training. This multi-reservoir ESN architecture overcomes standard ESN limitations regarding memory requirements and training times for large networks, providing more accurate predictions than other ESN-based models. These findings contribute to a deeper understanding of the potential of ESNs as a tool for image classification.

A dual contrastive learning-based graph convolutional network with syntax label enhancement for aspect-based sentiment classification

Introduction: Aspect-based sentiment classification is a fine-grained sentiment classification task. State-of-the-art approaches in this field leverage graph neural networks to integrate sentence syntax dependency. However, current methods fail to exploit the data augmentation in encoding and ignore the syntactic relation in sentiment delivery.Methods: In this work, we propose a novel graph neural network-based architecture with dual contrastive learning and syntax label enhancement. Specifically, a contrastive learning-based contextual encoder is designed, integrating sentiment information for semantics learning. Moreover, a weighted label-enhanced syntactic graph neural network is established to use both the syntactic relation and syntax dependency, which optimizes the syntactic weight between words. A syntactic triplet between words is generated. A syntax label-based contrastive learning scheme is developed to map the triplets into a unified feature space for syntactic information learning.Results: Experiments on five publicly available datasets show that our model substantially outperforms the baseline methods.Discussion: As such, the proposed method shows its effectiveness in aspect-based sentiment classification tasks.

PmcaNet: Pyramid multiscale channel attention network for electron microscopy image segmentation

Recent advances in high-throughput electron microscopy (EM) have revolutionized the examination of microstructures by enabling fast EM image generation. However, accurately segmenting EM images remains challenging due to inherent characteristics, including low contrast and subtle grayscale variations. Moreover, as manually annotated EM images are limited, it is usually impractical to utilize deep learning techniques for EM image segmentation. To address these challenges, the pyramid multiscale channel attention network (PmcaNet) is specifically designed. PmcaNet employs a convolutional neural network-based architecture and a multiscale feature pyramid to effectively capture global context information, enhancing its ability to comprehend the intricate structures within EM images. To enable the rapid extraction of channel-wise dependencies, a novel attention module is introduced to enhance the representation of intricate nonlinear features within the images. The performance of PmcaNet is evaluated on two general EM image segmentation datasets as well as a homemade dataset of superalloy materials, regarding pixel-wise accuracy and mean intersection over union (mIoU) as evaluation metrics. Extensive experiments demonstrate that PmcaNet outperforms other models on the ISBI 2012 dataset, achieving 87.85% pixel-wise accuracy and 73.11% mean intersection over union (mIoU), while also advancing results on the Kathuri and SEM-material datasets.

The AI Race: Why Current Neural Network-based Architectures are a Poor Basis for Artificial General Intelligence

Artificial General Intelligence is the idea that someday an hypothetical agent will arise from artificial intelligence (AI) progresses, and will surpass by far the brightest and most gifted human minds. This idea has been around since the early development of AI. Since then, scenarios on how such AI may behave towards humans have been the subject of many fictional and research works. This paper analyzes the current state of artificial intelligence progresses, and how the current AI race with the ever faster release of impressive new AI methods (that can deceive humans, outperform them at tasks we thought impossible to tackle by AI a mere decade ago, and that disrupt the job market) have raised concerns that Artificial General Intelligence (AGI) might be coming faster that we thought. In particular, we focus on 3 specific families of modern AIs to develop the idea that deep neural networks, which are the current backbone of nearly all artificial intelligence methods, are poor candidates for any AGI to arise due to their many limitations, and therefore that any threat coming from the recent AI race does not lie in AGI but in the limitations, uses, and lack of regulations of our current models and algorithms. This article appears in the AI & Society track.

Structure-aware graph neural network based deep transfer learning framework for enhanced predictive analytics on diverse materials datasets

Modern data mining methods have demonstrated effectiveness in comprehending and predicting materials properties. An essential component in the process of materials discovery is to know which material(s) will possess desirable properties. For many materials properties, performing experiments and density functional theory computations are costly and time-consuming. Hence, it is challenging to build accurate predictive models for such properties using conventional data mining methods due to the small amount of available data. Here we present a framework for materials property prediction tasks using structure information that leverages graph neural network-based architecture along with deep-transfer-learning techniques to drastically improve the model’s predictive ability on diverse materials (3D/2D, inorganic/organic, computational/experimental) data. We evaluated the proposed framework in cross-property and cross-materials class scenarios using 115 datasets to find that transfer learning models outperform the models trained from scratch in 104 cases, i.e., ≈90%, with additional benefits in performance for extrapolation problems. We believe the proposed framework can be widely useful in accelerating materials discovery in materials science.

Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature Review.

Speech disorders profoundly impact the overall quality of life by impeding social operations and hindering effective communication. This study addresses the gap in systematic reviews concerning machine learning-based assistive technology for individuals with speech disorders. The overarching purpose is to offer a comprehensive overview of the field through a Systematic Literature Review (SLR) and provide valuable insights into the landscape of ML-based solutions and related studies. The research employs a systematic approach, utilizing a Systematic Literature Review (SLR) methodology. The study extensively examines the existing literature on machine learning-based assistive technology for speech disorders. Specific attention is given to ML techniques, characteristics of exploited datasets in the training phase, speaker languages, feature extraction techniques, and the features employed by ML algorithms. This study contributes to the existing literature by systematically exploring the machine learning landscape in assistive technology for speech disorders. The originality lies in the focused investigation of ML-speech recognition for impaired speech disorder users over ten years (2014-2023). The emphasis on systematic research questions related to ML techniques, dataset characteristics, languages, feature extraction techniques, and feature sets adds a unique and comprehensive perspective to the current discourse. The systematic literature review identifies significant trends and critical studies published between 2014 and 2023. In the analysis of the 65 papers from prestigious journals, support vector machines and neural networks (CNN, DNN) were the most utilized ML technique (20%, 16.92%), with the most studied disease being Dysarthria (35/65, 54% studies). Furthermore, an upsurge in using neural network-based architectures, mainly CNN and DNN, was observed after 2018. Almost half of the included studies were published between 2021 and 2022).

ConTraNet: A hybrid network for improving the classification of EEG and EMG signals with limited training data

ObjectiveBio-Signals such as electroencephalography (EEG) and electromyography (EMG) are widely used for the rehabilitation of physically disabled people and for the characterization of cognitive impairments. Successful decoding of these bio-signals is however non-trivial because of the time-varying and non-stationary characteristics. Furthermore, existence of short- and long-range dependencies in these time-series signal makes the decoding even more challenging. State-of-the-art studies proposed Convolutional Neural Networks (CNNs) based architectures for the classification of these bio-signals, which are proven useful to learn spatial representations. However, CNNs because of the fixed size convolutional kernels and shared weights pay only uniform attention and are also suboptimal in learning short-long term dependencies, simultaneously, which could be pivotal in decoding EEG and EMG signals. Therefore, it is important to address these limitations of CNNs. To learn short- and long-range dependencies simultaneously and to pay more attention to more relevant part of the input signal, Transformer neural network-based architectures can play a significant role. Nonetheless, it requires a large corpus of training data. However, EEG and EMG decoding studies produce limited amount of the data. Therefore, using standalone transformers neural networks produce ordinary results. In this study, we ask a question whether we can fix the limitations of CNN and transformer neural networks and provide a robust and generalized model that can simultaneously learn spatial patterns, long-short term dependencies, pay variable amount of attention to time-varying non-stationary input signal with limited training data. ApproachIn this work, we introduce a novel single hybrid model called ConTraNet, which is based on CNN and Transformer architectures that contains the strengths of both CNN and Transformer neural networks. ConTraNet uses a CNN block to introduce inductive bias in the model and learn local dependencies, whereas the Transformer block uses the self-attention mechanism to learn the short- and long-range or global dependencies in the signal and learn to pay different attention to different parts of the signals. Main resultsWe evaluated and compared the ConTraNet with state-of-the-art methods on four publicly available datasets (BCI Competition IV dataset 2b, Physionet MI-EEG dataset, Mendeley sEMG dataset, Mendeley sEMG V1 dataset) which belong to EEG-HMI and EMG-HMI paradigms. ConTraNet outperformed its counterparts in all the different category tasks (2-class, 3-class, 4-class, 7-class, and 10-class decoding tasks). SignificanceWith limited training data ConTraNet significantly improves classification performance on four publicly available datasets for 2, 3, 4, 7, and 10-classes compared to its counterparts.

Convolutional neural network-based classification and monitoring models for lung cancer detection: 3D perspective approach

Recent developments in technology and research have offered a wide variety of new techniques for image and data analysis within the medical field. Medical research helps doctors and researchers acquire not only knowledge about health and new diseases, but also techniques of prevention and treatment. In particular, radiomic analysis is mainly used to extract quantitative data from medical images and to build a model strong enough to diagnose focal diseases. However, finding a model capable to fit all patient situations is not an easy task. In this paper frame prediction models and classification models are reported in order to predict the evolution of a given data series and determine whether an anomaly exists or not. This article also shows how to build and make use of a convolutional neural network-based architecture aiming to accomplish prediction task for medical images, not only using common computer tomography scans, but also 3D volumes.

A convolutional neural network-based architecture for health monitoring of joint damages in a steel plane frame structure under temperature variability

A convolutional neural network-based architecture for health monitoring of joint damages in a steel plane frame structure under temperature variability

An Efficient and Effective Framework for Intestinal Parasite Egg Detection Using YOLOv5.

Intestinal parasitic infections pose a grave threat to human health, particularly in tropical and subtropical regions. The traditional manual microscopy system of intestinal parasite detection remains the gold standard procedure for diagnosing parasite cysts or eggs. This approach is costly, time-consuming (30 min per sample), highly tedious, and requires a specialist. However, computer vision, based on deep learning, has made great strides in recent years. Despite the significant advances in deep convolutional neural network-based architectures, little research has been conducted to explore these techniques' potential in parasitology, specifically for intestinal parasites. This research presents a novel proposal for state-of-the-art transfer learning architecture for the detection and classification of intestinal parasite eggs from images. The ultimate goal is to ensure prompt treatment for patients while also alleviating the burden on experts. Our approach comprised two main stages: image pre-processing and augmentation in the first stage, and YOLOv5 algorithms for detection and classification in the second stage, followed by performance comparison based on different parameters. Remarkably, our algorithms achieved a mean average precision of approximately 97% and a detection time of only 8.5 ms per sample for a dataset of 5393 intestinal parasite images. This innovative approach holds tremendous potential to form a solid theoretical basis for real-time detection and classification in routine clinical examinations, addressing the increasing demand and accelerating the diagnostic process. Our research contributes to the development of cutting-edge technologies for the efficient and accurate detection of intestinal parasite eggs, advancing the field of medical imaging and diagnosis.

Glitch in the matrix: A large scale benchmark for content driven audio–visual forgery detection and localization

Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of audio or audio–visual manipulations that can completely change the meaning of the video content. To addresses this gap, we propose and benchmark a new dataset, Localized Audio Visual DeepFake (LAV-DF), consisting of strategic content-driven audio, visual and audio–visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations. We further improve (i.e. BA-TFD+) the baseline method by replacing the backbone with a Multiscale Vision Transformer and guide the training process with contrastive, frame classification, boundary matching and multimodal boundary matching loss functions. The quantitative analysis demonstrates the superiority of BA-TFD+ on temporal forgery localization and deepfake detection tasks using several benchmark datasets including our newly proposed dataset. The dataset, models and code are available at https://github.com/ControlNet/LAV-DF.

Multisource neural network feature map fusion: An efficient strategy to detect plant diseases

The use of RGB cameras or multispectral imaging systems can provide a wide range of applications for crop monitoring, plant phenotyping and disease detection. Although several approaches have been proposed, they increasingly use convolutional neural network-based architectures, which have, however, become increasingly cumbersome for improving classification results and difficult to train with few labeled data. Other increasingly popular approaches consist of using an ensemble of convolutional neural networks, in which each model solves a different problem. Since the inference is time- and resource-consuming due to the execution of multiple models, recent works have focused on transferring knowledge from an ensemble of models to a compact model to obtain better performance. In this paper, we propose an original approach that improves both accuracy and speed by reusing feature maps extracted by heterogeneous models from different data. Linked to each model, a transformation block allows keeping the correct number of feature maps and changing their dimension if necessary. To generate the feature maps, we only need the first layers of the ensemble models, thus taking advantage of ensemble learning methods, while adding only a few layers of a second model dedicated to aggregation of features. This approach allows an ensemble of models to be combined with different architectures that can process different data, such as several representations of the same input image or multispectral images, while being fast enough at the inference stage. This approach is adapted to hierarchical classification tasks by re-exploiting the same feature maps with different transformation blocks, offering accuracy gains in tasks not handled by the ensemble model. The results are provided for the PlantVillage dataset, with RGB images converted to three different color spaces, and for a custom Grapevine Yellow dataset, with multispectral images acquired with two different multispectral cameras.

Augmenting Deep Residual Surrogates with Fourier Neural Operators for Rapid Two-Phase Flow and Transport Simulations

Summary Accurate numerical modeling of multiphase flow and transport mechanisms is essential to study varied, complex physical phenomena including flow in subsurface oil and gas reservoirs and subsurface aquifers subject to CO2 sequestration. State-of-the-art complete physics-based solvers suffer from many computational challenges. High-fidelity data-driven surrogate models that solve the governing partial differential equations (PDEs) have the potential to optimize the time to solution and increase confidence in critical business and engineering decisions through better quantification of solution statistics. We leverage the recently proposed Fourier neural operators (FNOs) with quasilinear time complexity to capture the spectral information from feature maps to solve the coupled porous flow and transport PDEs. Embedding Fourier layers within the residual blocks results in a highly effective structure that, while achieving competitive accuracy, also enables efficient training of deeper networks with a dramatically reduced number of trainable parameters. The resulting novel deep-learning (DL) architecture is coined as FResNet++. FResNet++ uses squeeze and excitation blocks, atrous spatial pyramid pooling (ASPP), and attention blocks to increase its sensitivity to the relevant features and capture multiscale information, and it is specifically tuned to operate optimally to learn from and predict numerically simulated flow (pressure and saturation) fields. We demonstrate the ability of FResNet++ to generalize over multiple high-dimensional input parameter spaces that describe subsurface permeability and porosity heterogeneity. The resulting DL architecture accurately captures the complex interplay between viscous forces and highly heterogeneous permeability and porosity fields. We investigate two-phase flow in porous media, which is the archetypal problem for reservoir simulation giving rise to a system of nonlinearly coupled PDEs with highly heterogeneous coefficients. We show in blind tests that FResNet++ predicts saturation fields more accurately compared to ResU-Net and original FNO with fully connected linear layers. We additionally investigate the effects of using alternative loss functions and an alternative way of utilizing FResNet++ to increase its effectiveness. For the first time in the literature, we show that the spatiotemporal evolution of pressure and saturation fields can be jointly predicted with good accuracy using a single FResNet++ network over long time horizons in response to previously unseen permeability and porosity fields. After a moderate training investment on graphics processing units (GPUs), FResNet++ yields a speedup of at least four orders of magnitude compared to a conventional numerical PDE solver and operates with notably fewer trainable parameters compared to the original FNO. Our numerical experiments validate that FNOs can be utilized in various convolutional neural network-based architectures and can effectively substitute for repetitive physics-based forward simulations for scenario testing.

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices.

This paper considers the task of appearance-based localization: visual place recognition from omnidirectional images obtained from catadioptric cameras. The focus is on designing an efficient neural network architecture that accurately and reliably recognizes indoor scenes on distorted images from a catadioptric camera, even in self-similar environments with few discernible features. As the target application is the global localization of a low-cost service mobile robot, the proposed solutions are optimized toward being small-footprint models that provide real-time inference on edge devices, such as Nvidia Jetson. We compare several design choices for the neural network-based architecture of the localization system and then demonstrate that the best results are achieved with embeddings (global descriptors) yielded by exploiting transfer learning and fine tuning on a limited number of catadioptric images. We test our solutions on two small-scale datasets collected using different catadioptric cameras in the same office building. Next, we compare the performance of our system to state-of-the-art visual place recognition systems on the publicly available COLD Freiburg and Saarbrücken datasets that contain images collected under different lighting conditions. Our system compares favourably to the competitors both in terms of the accuracy of place recognition and the inference time, providing a cost- and energy-efficient means of appearance-based localization for an indoor service robot.

Presentation-level Privacy Protection Techniques for Automated Face Recognition—A Survey

The use of Biometric Facial Recognition (FR) systems have become increasingly widespread, especially since the advent of deep neural network-based architectures. Although FR systems provide substantial benefits in terms of security and safety, the use of these systems also raises significant privacy concerns. This article discusses recent advances in facial identity hiding techniques, focusing on privacy protection approaches that hide or protect facial biometric data before camera devices capture the data. Moreover, we also discuss the state-of-the-art methods used to evaluate such privacy protection techniques. The primary motivation of this survey is to assess the relative performance of facial privacy protection methods and identify open challenges and future work that needs to be considered in this research area.

A Machine Learning Method for Automated Description and Workflow Analysis of First Trimester Ultrasound Scans.

Obstetric ultrasound assessment of fetal anatomy in the first trimester of pregnancy is one of the less explored fields in obstetric sonography because of the paucity of guidelines on anatomical screening and availability of data. This paper, for the first time, examines imaging proficiency and practices of first trimester ultrasound scanning through analysis of full-length ultrasound video scans. Findings from this study provide insights to inform the development of more effective user-machine interfaces, of targeted assistive technologies, as well as improvements in workflow protocols for first trimester scanning. Specifically, this paper presents an automated framework to model operator clinical workflow from full-length routine first-trimester fetal ultrasound scan videos. The 2D+t convolutional neural network-based architecture proposed for video annotation incorporates transfer learning and spatio-temporal (2D+t) modelling to automatically partition an ultrasound video into semantically meaningful temporal segments based on the fetal anatomy detected in the video. The model results in a cross-validation A1 accuracy of 96.10%, F1 = 0.95, precision = 0.94 and recall = 0.95. Automated semantic partitioning of unlabelled video scans (n=250) achieves a high correlation with expert annotations (ρ = 0.95, p = 0.06). Clinical workflow patterns, operator skill and its variability can be derived from the resulting representation using the detected anatomy labels, order, and distribution. It is shown that nuchal translucency (NT) is the toughest standard plane to acquire and most operators struggle to localize high-quality frames. Furthermore, it is found that newly qualified operators spend 25.56% more time on key biometry tasks than experienced operators.

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

This work investigates classification of emotions from full-body movements by using a novel Convolutional Neural Network-based architecture. The model is composed of two shallow networks processing in parallel where the 8-bit RGB images obtained from time intervals of 3D-positional data are the inputs. One network performs a coarse-grained modelling in the time domain while the other one applies a fine-grained modelling. We show that combining different temporal scales into one architecture improves the classification results of a dataset composed of short excerpts of the performances of professional dancers who interpreted four affective states: anger, happiness, sadness, and insecurity. Additionally, we investigate the effect of data chunk duration, overlapping, the size of the input images and the contribution of several data augmentation strategies for our proposed method. Better recognition results were obtained when the duration of a data chunk was longer, and this was further improved by applying balanced data augmentation. Moreover, we test our method on other existing motion capture datasets and compare the results with prior art. In all of the experiments, our results surpassed the state-of-the-art approaches, showing that this method generalizes across diverse settings and contexts.