Hybrid Neural Architecture Research Articles

Toward an intelligent computing system for the early diagnosis of Alzheimer's disease based on the modular hybrid growing neural gas.

The proportion of older people will soon include nearly a quarter of the world population. This leads to an increased prevalence of non-communicable diseases such as Alzheimer's disease (AD), a progressive neurodegenerative disorder and the most common dementia. mild cognitive impairment (MCI) can be considered its prodromal stage. The early diagnosis of AD is a huge issue. We face it by solving these classification tasks: MCI-AD and cognitively normal (CN)-MCI-AD. An intelligent computing system has been developed and implemented to face both challenges. A non-neural preprocessing module was followed by a processing one based on a hybrid and ontogenetic neural architecture, the modular hybrid growing neural gas (MyGNG). The MyGNG is hierarchically organized, with a growing neural gas (GNG) for clustering followed by a perceptron for labeling. For each task, 495 and 819 patients from the Alzheimer's disease neuroimaging initiative (ADNI) database were used, respectively, each with 211 characteristics. Encouraging results have been obtained in the MCI-AD classification task, reaching values of area under the curve (AUC) of 0.96 and sensitivity of 0.91, whereas 0.86 and 0.9 in CN-MCI-AD. Furthermore, a comparative study with popular machine learning (ML) models was also performed for each of these tasks. The MyGNG proved to be a better computational solution than the other ML methods analyzed. Also, it had a similar performance to other deep learning schemes with neuroimaging. Our findings suggest that our proposal may be an interesting computing solution for the early diagnosis of AD.

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network

NeuroWave-Net: Enhancing epileptic seizure detection from EEG brain signals via advanced convolutional and long short-term memory networks

<abstract> <p>This study presented a new approach to seizure classification utilizing electroencephalogram (EEG) data. We introduced the NeuroWave-Net, an innovative hybrid model that seamlessly integrates convolutional neural networks (CNN) and long short-term memory (LSTM) architectures. Unlike conventional methods, our model capitalized on CNN's proficiency in feature extraction and LSTM's prowess in classifying seizure. The key strength of the NeuroWave-Net lies in its ability to combine these distinct architectures, synergizing their capabilities for enhanced accuracy in identifying seizure conditions within EEG data. Our proposed model exhibited outstanding performance, achieving a classification accuracy of 99.48%. This study contributed to the advancement of seizure classification models, providing a robust and streamlined approach for accurate categorization within EEG datasets. NeuroWave-Net stands as a testament to the potential of hybrid neural network architectures in neurological diagnostics.</p> </abstract>

USAGE OF ARTIFICIAL NEURAL NETWORKS IN THE DIAGNOSIS OF KNEE JOINT DISORDERS

Following article address the issue of automatic knee disorder diagnose with usage of neural networks. We proposed several hybrid neural net architectures which aim to successfully classify abnormality using MRI (magnetic resonance imaging) images acquired from publicly available dataset. To construct such combinations of models we used pretrained Alexnet, Resnet18 and Resnet34 downloaded from Torchvision. Experiments showed that for certain abnormalities our models can achieve up to 90% accuracy.

Quantitative comparison of the computational complexity of optical, digital and hybrid neural network architectures for image classification tasks.

By implementing neuromorphic paradigms in processing visual information, machine learning became crucial in an ever-increasing number of applications of our everyday lives, ever more performing but also computationally demanding. While a pre-processing of the information passively in the optical domain, before optical-electronic conversion, can reduce the computational requirements for a machine learning task, a comprehensive analysis of computational requirements for hybrid optical-digital neural networks is thus far missing. In this work we critically compare and analyze the performance of different optical, digital and hybrid neural network architectures with respect to their classification accuracy and computational requirements for analog classification tasks of different complexity. We show that certain hybrid architectures exhibit a reduction of computational requirements of a factor >10 while maintaining their performance. This may inspire a new generation of co-designed optical-digital neural network architectures, aimed for applications that require low power consumption like remote sensing devices.

Leveraging Machine Learning for Road Accident Analysis

Road accidents result in high human and economic costs globally. This paper examines how advanced machine learning techniques can support enhanced analysis of road accident data to uncover patterns and insights to guide traffic safety interventions. Novel machine learning methods proposed include hybrid neural network architectures optimized using nature-inspired algorithms and interpretable rule-based tree ensemble techniques. Our investigation commences with the training and evaluation of each model on a diverse dataset comprising various road-related features. The performance metrics, including accuracy predictive capabilities. The results reveal nuanced strengths and weaknesses in each approach.

Attention-Based 1D CNN-BiLSTM Hybrid Model Enhanced with FastText Word Embedding for Korean Voice Phishing Detection

In the increasingly complex domain of Korean voice phishing attacks, advanced detection techniques are paramount. Traditional methods have achieved some degree of success. However, they often fail to detect sophisticated voice phishing attacks, highlighting an urgent need for enhanced approaches to improve detection performance. Addressing this, we have designed and implemented a novel artificial neural network (ANN) architecture that successfully combines data-centric and model-centric AI methodologies for detecting Korean voice phishing attacks. This paper presents our unique hybrid architecture, consisting of a 1-dimensional Convolutional Neural Network (1D CNN), a Bidirectional Long Short-Term Memory (BiLSTM), and Hierarchical Attention Networks (HANs). Our evaluations using the real-world KorCCVi v2 dataset demonstrate that the proposed architecture effectively leverages the strengths of CNN and BiLSTM to extract and learn contextually rich features from word embedding vectors. Additionally, implementing word and sentence attention mechanisms from HANs enhances the model’s focus on crucial features, considerably improving detection performance. Achieving an accuracy score of 99.32% and an F1 score of 99.31%, our model surpasses all baseline models we trained, outperforms several existing solutions, and maintains comparable performance to others. The findings of this study underscore the potential of hybrid neural network architectures in improving voice phishing detection in the Korean language and pave the way for future research. This could involve refining and expanding upon this model to tackle increasingly sophisticated voice phishing strategies effectively or utilizing larger datasets.

ENCODE - Ensemble neural combination for optimal dimensionality encoding in time-series forecasting

Nowadays, predictive models based on a time series are widely used in many fields, from geology to healthcare and from traffic management to industrial production. One of the most important tasks in machine and deep learning is designing predictive algorithms that provide increasingly reliable and accurate forecasts from a time series. This work proposes a novel ensemble approach to producing predictions in a multivariate framework. Its main idea is to reduce data dimensionality through an encoding technique, with the aim to extract useful information via single predictive procedures and then to gather all the processed data through a combiner to give the final forecast. In our framework, the combiner is composed of a hybrid neural architecture: a Convolutional Neural Network to extract local patterns from the predictions and a Recurrent Neural Network to infer information about the temporal patterns of the time series. Furthermore, a fully connected network is adopted to merge these two components and to provide the prediction. Extensive experiments on different datasets, both public and private, related to different applications have been carried out. Comparisons of the errors with conventional methods and state-of-the-art strategies confirm both accuracy and robustness of the proposed ensemble. Finally, we also show a comparison in terms of computational time, both in the hyperparameter optimization and forecasting tasks.

EEG-Based Emotion Classification in Financial Trading Using Deep Learning: Effects of Risk Control Measures

Day traders in the financial markets are under constant pressure to make rapid decisions and limit capital losses in response to fluctuating market prices. As such, their emotional state can greatly influence their decision-making, leading to suboptimal outcomes in volatile market conditions. Despite the use of risk control measures such as stop loss and limit orders, it is unclear if these strategies have a substantial impact on the emotional state of traders. In this paper, we aim to determine if the use of limit orders and stop loss has a significant impact on the emotional state of traders compared to when these risk control measures are not applied. The paper provides a technical framework for valence-arousal classification in financial trading using EEG data and deep learning algorithms. We conducted two experiments: the first experiment employed predetermined stop loss and limit orders to lock in profit and risk objectives, while the second experiment did not employ limit orders or stop losses. We also proposed a novel hybrid neural architecture that integrates a Conditional Random Field with a CNN-BiLSTM model and employs Bayesian Optimization to systematically determine the optimal hyperparameters. The best model in the framework obtained classification accuracies of 85.65% and 85.05% in the two experiments, outperforming previous studies. Results indicate that the emotions associated with Low Valence and High Arousal, such as fear and worry, were more prevalent in the second experiment. The emotions associated with High Valence and High Arousal, such as hope, were more prevalent in the first experiment employing limit orders and stop loss. In contrast, High Valence and Low Arousal (calmness) emotions were most prominent in the control group which did not engage in trading activities. Our results demonstrate the efficacy of our proposed framework for emotion classification in financial trading and aid in the risk-related decision-making abilities of day traders. Further, we present the limitations of the current work and directions for future research.

A hybrid neural architecture search for hyperspectral image classification

Convolution neural network (CNN)is widely used in hyperspectral image (HSI) classification. However, the network architecture of CNNs is usually designed manually, which requires careful fine-tuning. Recently, many technologies for neural architecture search (NAS) have been proposed to automatically design networks, further improving the accuracy of HSI classification to a new level. This paper proposes a circular kernel convolution-β-decay regulation NAS-confident learning rate (CK-βNAS-CLR) framework to automatically design the neural network structure for HSI classification. First, this paper constructs a hybrid search space with 12 kinds of operation, which considers the difference between enhanced circular kernel convolution and square kernel convolution in feature acquisition, so as to improve the sensitivity of the network to hyperspectral information features. Then, the β-decay regulation scheme is introduced to enhance the robustness of differential architecture search (DARTS) and reduce the discretization differences in architecture search. Finally, we combined the confidence learning rate strategy to alleviate the problem of performance collapse. The experimental results on public HSI datasets (Indian Pines, Pavia University) show that the proposed NAS method achieves impressive classification performance and effectively improves classification accuracy.

USPoint: Self-Supervised Interest Point Detection and Description for Ultrasound-Probe Motion Estimation During Fine-Adjustment Standard Fetal Plane Finding.

Ultrasound (US)-probe motion estimation is a fundamental problem in automated standard plane locating during obstetric US diagnosis. Most recent existing recent works employ deep neural network (DNN) to regress the probe motion. However, these deep regressionbased methods leverage the DNN to overfit on the specific training data, which is naturally lack of generalization ability for the clinical application. In this paper, we are back to generalized US feature learning rather than deep parameter regression. We propose a self-supervised learned local detector and descriptor, named USPoint, for US-probe motion estimation during the fine-adjustment phase of fetal plane acquisition. Specifically, a hybrid neural architecture is designed to simultaneously extract a local feature, and further estimate the probe motion. By embedding a differentiable USPoint-based motion estimation inside the proposed network architecture, the USPoint learns the keypoint detector, scores and descriptors from motion error alone, which doesn't require expensive human-annotation of local features. The two tasks, local feature learning and motion estimation, are jointly learned in a unified framework to enable collaborative learning with the aim of mutual benefit. To the best of our knowledge, it is the first learned local detector and descriptor tailored for the US image. Experimental evaluation on real clinical data demonstrates the resultant performance improvement on feature matching and motion estimation for potential clinical value. A video demo can be found online: https://youtu.be/JGzHuTQVlBs.

Comparative models for multi-step ahead wind speed forecasting applied for expected wind turbine power output prediction

This paper investigates six of the most widely used wind speed forecasting models for a combination of statistical and physical methods for the purpose of Wind Turbine Power Generation (WTPG) prediction in Cameroon. Statistical method based on both single static and dynamic neural networks architectures and two hybrid neural networks architectures in comparison to ARIMA model are employed for multi-step ahead wind speed forecasting in two Datasets in Bapouh, Cameroon. The physical method is used to estimate 1 day ahead expected WTPG for each Dataset using the previous predicted wind speed from better forecasting models. The obtained results of multi-step ahead forecasting showed that the ARIMA and nonlinear autoregression with exogenous input neural network (NARXNN) models perform well the wind speed forecasting than other forecasting models in both Datasets. The better performances of ARIMA are achieved with one-step ahead and two-step ahead forecasting, while NARXNN is better with one-step ahead forecasting. But NARXNN models have more computational time than other models such as ARIMA models. Furthermore, the effectiveness of employed hybrid method for WTPG prediction is proven.

Boosting Intelligent Data Analysis in Smart Sensors by Integrating Knowledge and Machine Learning.

The presented paper proposes a hybrid neural architecture that enables intelligent data analysis efficacy to be boosted in smart sensor devices, which are typically resource-constrained and application-specific. The postulated concept integrates prior knowledge with learning from examples, thus allowing sensor devices to be used for the successful execution of machine learning even when the volume of training data is highly limited, using compact underlying hardware. The proposed architecture comprises two interacting functional modules arranged in a homogeneous, multiple-layer architecture. The first module, referred to as the knowledge sub-network, implements knowledge in the Conjunctive Normal Form through a three-layer structure composed of novel types of learnable units, called L-neurons. In contrast, the second module is a fully-connected conventional three-layer, feed-forward neural network, and it is referred to as a conventional neural sub-network. We show that the proposed hybrid structure successfully combines knowledge and learning, providing high recognition performance even for very limited training datasets, while also benefiting from an abundance of data, as it occurs for purely neural structures. In addition, since the proposed L-neurons can learn (through classical backpropagation), we show that the architecture is also capable of repairing its knowledge.

Multimodal sentiment and emotion recognition in hyperbolic space

Prior approaches for multimodal sentiment and emotion recognition (SER) exploit input data representations and neural networks based on the classical Euclidean geometry. Recently, however, the hyperbolic metric proved to be a powerful tool for data mapping, being able to capture the hierarchical structure of the relations among elements in the data. In this paper we propose the use of hyperbolic learning for SER, and show that the inclusion in the neural network of hyperbolic structures mapping the input into the hyperbolic space can improve the quality of the predictions. The benefits brought by the hyperbolic features are evaluated by developing extensions of existing methods following two approaches. From one side, we modified state-of-the-art models by including hyperbolic output layers. From the other, we generated hybrid neural network architectures by combining hyperbolic and Euclidean layers according to different schemes. The proposed hyperbolic models were tested on several classification tasks applied to benchmark multimodal SER datasets. Experiments gave strong evidence that in both simple and complex networks the introduction of a hyperbolic structure results in an improvement of the model accuracy. Specifically, the combined use of hyperbolic and Euclidean layers showed superior performance in almost all the classification tasks.

A Real-Time Clinical Decision Support System, for Mild Cognitive Impairment Detection, Based on a Hybrid Neural Architecture.

Clinical procedure for mild cognitive impairment (MCI) is mainly based on clinical records and short cognitive tests. However, low suspicion and difficulties in understanding test cut-offs make diagnostic accuracy being low, particularly in primary care. Artificial neural networks (ANNs) are suitable to design computed aided diagnostic systems because of their features of generating relationships between variables and their learning capability. The main aim pursued in that work is to explore the ability of a hybrid ANN-based system in order to provide a tool to assist in the clinical decision-making that facilitates a reliable MCI estimate. The model is designed to work with variables usually available in primary care, including Minimental Status Examination (MMSE), Functional Assessment Questionnaire (FAQ), Geriatric Depression Scale (GDS), age, and years of education. It will be useful in any clinical setting. Other important goal of our study is to compare the diagnostic rendering of ANN-based system and clinical physicians. A sample of 128 MCI subjects and 203 controls was selected from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The ANN-based system found the optimal variable combination, being AUC, sensitivity, specificity, and clinical utility index (CUI) calculated. The ANN results were compared with those from medical experts which include two family physicians, a neurologist, and a geriatrician. The optimal ANN model reached an AUC of 95.2%, with a sensitivity of 90.0% and a specificity of 84.78% and was based on MMSE, FAQ, and age inputs. As a whole, physician performance achieved a sensitivity of 46.66% and a specificity of 91.3%. CUIs were also better for the ANN model. The proposed ANN system reaches excellent diagnostic accuracy although it is based only on common clinical tests. These results suggest that the system is especially suitable for primary care implementation, aiding physicians work with cognitive impairment suspicions.

A transformer-based approach to irony and sarcasm detection

Figurative language (FL) seems ubiquitous in all social media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of natural language processing, mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper, we employ advanced deep learning methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work (Potamias et al., in: International conference on engineering applications of neural networks, Springer, Berlin, pp 164–175, 2019), we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which is further enhanced with the employment and devise of a recurrent convolutional neural network. With this setup, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state-of-the-art methodologies and systems. Results demonstrate that the proposed methodology achieves state-of-the-art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.

Hybrid neural networks for big data classification

Two new hybrid neural architectures combining morphological neurons and perceptrons are introduced in this paper. The first architecture, called Morphological - Linear Neural Network (MLNN) consists of a hidden layer of morphological neurons and an output layer of classical perceptrons has the capability of extracting features. The second architecture, called Linear-Morphological Neural Network (LMNN) is composed of one or several perceptron layers as a feature extractor, it is then followed by an output layer of morphological neurons for non-linear classification. Both architectures are trained by stochastic gradient descent. One of the main contributions of this paper is to show that the morphological layer offers a greater capacity to extract features than the perceptron layer. This claim is supported both theoretically and experimentally. We prove that the morphological layer possesses a greater capacity per computation unit to segment the 2D input space than the perceptron layer. In other words, adding more hyper-boxes produces more response regions than adding hyperplanes. From an empirical point of view, we test the two new models on 25 standard datasets at low dimensionality and one big data dataset. The result is that MLNN requires a lesser number of learning parameters than the other tested architectures while achieving better accuracies.

Generalised Hough Transform for Speech Pattern Classification

While typical hybrid neural network architectures for automatic speech recognition (ASR) use a context window of frame-based features, this may not be the best approach to capture the wider temporal context, which contains phonetic and linguistic information that is equally important. In this paper, we introduce a system that integrates both the spectral and geometrical shape information from the acoustic spectrum, inspired by research in the field of machine vision. In particular, we focus on the Generalized Hough Transform (GHT), which is a sophisticated technique that can model the geometrical distribution of speech information over the wider temporal context. To integrate the GHT as part of a hybrid-ASR system, we propose to use a neural network, with features derived from the probabilistic Hough voting step of the GHT, to implement an improved version of the GHT where the output of the network represents the conventional target class posteriors. A major advantage of our approach is that each step of the GHT is highly interpretable, particularly compared to deep neural network (DNN) systems which are commonly treated as powerful black-box classifiers that give little insight into how the output is achieved. Experiments are carried out on two speech pattern classification tasks. The first is the TIMIT phoneme classification, which demonstrates the performance of the approach on a standard ASR task. The second is a spoken word recognition challenge, which highlights the flexibility of the approach to capture phonetic information within a longer temporal context.

Using Neural Pattern Classifiers to Quantify the Modularity of Conflict-Control Mechanisms in the Human Brain

Resolving conflicting sensory and motor representations is a core function of cognitive control, but it remains uncertain to what degree control over different sources of conflict is implemented by shared (domain general) or distinct (domain specific) neural resources. Behavioral data suggest conflict-control to be domain specific, but results from neuroimaging studies have been ambivalent. Here, we employed multivoxel pattern analyses that can decode a brain region's informational content, allowing us to distinguish incidental activation overlap from actual shared information processing. We trained independent sets of "searchlight" classifiers on functional magnetic resonance imaging data to decode control processes associated with stimulus-conflict (Stroop task) and ideomotor-conflict (Simon task). Quantifying the proportion of domain-specific searchlights (capable of decoding only one type of conflict) and domain-general searchlights (capable of decoding both conflict types) in each subject, we found both domain-specific and domain-general searchlights, though the former were more common. When mapping anatomical loci of these searchlights across subjects, neural substrates of stimulus- and ideomotor-specific conflict-control were found to be anatomically consistent across subjects, whereas the substrates of domain-general conflict-control were not. Overall, these findings suggest a hybrid neural architecture of conflict-control that entails both modular (domain specific) and global (domain general) components.

English

This paper proposes a solution methodology for a missile defence problem using theatremissile defence (TMD) concept. In the missile defence scenario, the concept of TMD is generallyused for the optimal allocation of interceptors to counter the attack missiles. The problem iscomputationally complex due to the presence of enormous state space. The Learning vectorquantiser–Radial basis function (LVQ-RBF) multi-agent hybrid neural architecture is used as thelearning structure, and Q-learning as the learning method. The LVQ-RBF multi-agent hybridneural architecture overcomes the complex state space issue using the partitioning and weightedlearning approach. The proposed LVQ-RBF multi- agent hybrid architecture improvises thelearning performance by the local and global error criterion. The state space is explored withinitial coarse partitioning by LVQ neural network. The fine partitioning of the state space isperformed using the multi-agent RBF neural network. The discrete reward scheme is used forLVQ-RBF multi-agent hybrid neural architecture. It has a hierarchical architecture which enablesquicker convergence without the loss of accuracy. The simulation of the TMD is performed with500 assets and six priority of assets.