Explicit Feature Engineering Research Articles

Advances in Sentiment Analysis in Deep Learning Models and Techniques

The article investigates the advantages, disadvantages, and areas of research that need more exploration regarding deep learning architectures used in sentiment analysis. These architectures let models learn complex language features from data without explicit feature engineering, changing sentiment analysis. The models' capacity to capture long-range dependencies has improved their context and nuanced expression interpretation, especially in long or metaphorical texts. Deep learning sentiment analysis algorithms have improved, yet they still face obstacles. The complexity of these models raises ethical questions about bias and transparency. They also require huge, annotated datasets and computational resources, which limits their use in resource-constrained contexts. Adopting deep learning models requires balancing performance and practicality. Explore critical deep learning sentiment analysis research gaps. Cross-domain and cross-lingual sentiment analysis requires context- and language-specific models. Textual and non-textual multimodal sentiment analysis offers untapped potential for complex sentiment interpretation. Responsible AI deployment requires model interpretability, robustness against adversarial assaults, and domain consistency. Finally, deep learning and sentiment analysis have changed our knowledge of human emotion. Accuracy and contextual comprehension have improved, but model transparency, data prerequisites, and practical applicability remain issues. Overcoming these restrictions and exploring research gaps will enable responsible sentiment analysis AI innovation.

MOD3NN: A Framework for Automatic Signal Modulation Detection Using 3D CNN

In this work, we present an application of a three-dimensional convolutional neural network for the task of automatic modulation recognition from raw I/Q signal data. Raw I/Q signal data exhibits a special “helical” structure that can be exploited with three-dimensional convolutions (3D convolutions) to learn spatio-temporal features from the signal for the problem of modulation recognition. By tweaking the convolutional filters to learn the helical symmetry of the data, we can design a shallow network for automatic modulation recognition (AMR). We present the results of our experiments with raw I/Q signal data collected in an uncalibrated radio frequency (RF) environment using several different modulation schemes. We show that with our methods and implementation, we can achieve around 99 % accuracy for automatic modulation recognition, for a variety of practical modulation techniques without the need for explicit feature engineering.

An End-to-End Machine Learning based Unified Architecture for Non-Intrusive Load Monitoring

Non-Intrusive Load Monitoring (NILM) or load disaggregation aims to analyze power consumption by decomposing the energy measured at the aggregate level into constituent appliances level. The conventional load disaggregation framework consists of signal processing and machine learning-based pipelined architectures, respectively for explicit feature extraction and decision making. Manual feature selection in such load disaggregation frameworks leads to biased decisions that eventually reduce system performance. This paper presents an efficient End-to-End (E2E) approach-based unified architecture using Gated Recurrent Units (GRU) for NILM. The proposed approach eliminates explicit feature engineering and has a unified classification and prediction model for appliance power. This eventually reduces the computational cost and enhances response time. The performance of the proposed system is compared with conventional algorithms' with the use of recall, precision, accuracy, F1 score, the relative error in total energy and Mean Absolute Error (MAE). These evaluation metrics are calculated on the power consumption of top priority appliances of Reference Energy Disaggregation Dataset (REDD). The proposed architecture with an overall accuracy of 91.2 and MAE of 25.23 outperforms conventional methods for all electrical appliances. It has been showcased through a series of experiments that feature extraction and event-based approaches for NILM can readily be replaced with E2E deep learning techniques allowing simpler and cost-efficient implementation pathways.

Decoding and interpreting cortical signals with a compact convolutional neural network

Objective. Brain–computer interfaces (BCIs) decode information from neural activity and send it to external devices. The use of Deep Learning approaches for decoding allows for automatic feature engineering within the specific decoding task. Physiologically plausible interpretation of the network parameters ensures the robustness of the learned decision rules and opens the exciting opportunity for automatic knowledge discovery. Approach. We describe a compact convolutional network-based architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also propose a novel theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time. The obtained spatial and frequency patterns characterizing the neuronal populations pivotal to the specific decoding task can then be interpreted by fitting appropriate spatial and dynamical models. Main results. We first tested our solution using realistic Monte-Carlo simulations. Then, when applied to the ECoG data from Berlin BCI competition IV dataset, our architecture performed comparably to the competition winners without requiring explicit feature engineering. Using the proposed approach to the network weights interpretation we could unravel the spatial and the spectral patterns of the neuronal processes underlying the successful decoding of finger kinematics from an ECoG dataset. Finally we have also applied the entire pipeline to the analysis of a 32-channel EEG motor-imagery dataset and observed physiologically plausible patterns specific to the task. Significance. We described a compact and interpretable CNN architecture derived from the basic principles and encompassing the knowledge in the field of neural electrophysiology. For the first time in the context of such multibranch architectures with factorized spatial and temporal processing we presented theoretically justified weights interpretation rules. We verified our recipes using simulations and real data and demonstrated that the proposed solution offers a good decoder and a tool for investigating motor control neural mechanisms.

Deep Learning for EEG Seizure Detection in Preterm Infants.

EEG is the gold standard for seizure detection in the newborn infant, but EEG interpretation in the preterm group is particularly challenging; trained experts are scarce and the task of interpreting EEG in real-time is arduous. Preterm infants are reported to have a higher incidence of seizures compared to term infants. Preterm EEG morphology differs from that of term infants, which implies that seizure detection algorithms trained on term EEG may not be appropriate. The task of developing preterm specific algorithms becomes extra-challenging given the limited amount of annotated preterm EEG data available. This paper explores novel deep learning (DL) architectures for the task of neonatal seizure detection in preterm infants. The study tests and compares several approaches to address the problem: training on data from full-term infants; training on data from preterm infants; training on age-specific preterm data and transfer learning. The system performance is assessed on a large database of continuous EEG recordings of 575[Formula: see text]h in duration. It is shown that the accuracy of a validated term-trained EEG seizure detection algorithm, based on a support vector machine classifier, when tested on preterm infants falls well short of the performance achieved for full-term infants. An AUC of 88.3% was obtained when tested on preterm EEG as compared to 96.6% obtained when tested on term EEG. When re-trained on preterm EEG, the performance marginally increases to 89.7%. An alternative DL approach shows a more stable trend when tested on the preterm cohort, starting with an AUC of 93.3% for the term-trained algorithm and reaching 95.0% by transfer learning from the term model using available preterm data. The proposed DL approach avoids time-consuming explicit feature engineering and leverages the existence of the term seizure detection model, resulting in accurate predictions with a minimum amount of annotated preterm data.

Deep learning for biological age estimation.

Modern machine learning techniques (such as deep learning) offer immense opportunities in the field of human biological aging research. Aging is a complex process, experienced by all living organisms. While traditional machine learning and data mining approaches are still popular in aging research, they typically need feature engineering or feature extraction for robust performance. Explicit feature engineering represents a major challenge, as it requires significant domain knowledge. The latest advances in deep learning provide a paradigm shift in eliciting meaningful knowledge from complex data without performing explicit feature engineering. In this article, we review the recent literature on applying deep learning in biological age estimation. We consider the current data modalities that have been used to study aging and the deep learning architectures that have been applied. We identify four broad classes of measures to quantify the performance of algorithms for biological age estimation and based on these evaluate the current approaches. The paper concludes with a brief discussion on possible future directions in biological aging research using deep learning. This study has significant potentials for improving our understanding of the health status of individuals, for instance, based on their physical activities, blood samples and body shapes. Thus, the results of the study could have implications in different health care settings, from palliative care to public health.

Deep Embedded Non-Redundant Clustering

Complex data types like images can be clustered in multiple valid ways. Non-redundant clustering aims at extracting those meaningful groupings by discouraging redundancy between clusterings. Unfortunately, clustering images in pixel space directly has been shown to work unsatisfactory. This has increased interest in combining the high representational power of deep learning with clustering, termed deep clustering. Algorithms of this type combine the non-linear embedding of an autoencoder with a clustering objective and optimize both simultaneously. None of these algorithms try to find multiple non-redundant clusterings. In this paper, we propose the novel Embedded Non-Redundant Clustering algorithm (ENRC). It is the first algorithm that combines neural-network-based representation learning with non-redundant clustering. ENRC can find multiple highly non-redundant clusterings of different dimensionalities within a data set. This is achieved by (softly) assigning each dimension of the embedded space to the different clusterings. For instance, in image data sets it can group the objects by color, material and shape, without the need for explicit feature engineering. We show the viability of ENRC in extensive experiments and empirically demonstrate the advantage of combining non-linear representation learning with non-redundant clustering.

Drug-drug interaction extraction from biomedical texts using long short-term memory network.

The simultaneous administration of multiple drugs increases the probability of interaction among them, as one drug may affect the activities of others. This interaction among drugs may have a positive or negative impact on the therapeutic outcomes. Thus, identification of unknown drug-drug interactions (DDIs) is of significant concern for improving the safety and efficacy of drug consumption. Although multiple DDI resources exist, it is becoming infeasible to maintain these up-to-date manually with the number of biomedical texts growing at a fast pace. Most existing methods model DDI extraction as a classification problem and rely mainly on handcrafted features, and certain features further depend on domain-specific tools. Recently, neural network models using latent features have been demonstrated to yield similar or superior performance compared to existing models. In this study, we present three long short-term memory (LSTM) network models, namely B-LSTM, AB-LSTM, and Joint AB-LSTM. All three models use word and position embedding as latent features; thus, they do not rely on explicit feature engineering. Furthermore, the use of a bidirectional LSTM (Bi-LSTM) network allows for extraction of implicit features from an entire sentence. The two models AB-LSTM and Joint AB-LSTM also apply attentive pooling in the Bi-LSTM layer output in order to assign weights to features. Our experimental results on the SemEval-2013 DDI extraction dataset indicate that the Joint AB-LSTM model produces reasonable performance (F-score: 69.39%) even with the simple architecture.

A Global Model for Concept-to-Text Generation

Concept-to-text generation refers to the task of automatically producing textual output from non-linguistic input. We present a joint model that captures content selection ("what to say") and surface realization ("how to say") in an unsupervised domain-independent fashion. Rather than breaking up the generation process into a sequence of local decisions, we define a probabilistic context-free grammar that globally describes the inherent structure of the input (a corpus of database records and text describing some of them). We recast generation as the task of finding the best derivation tree for a set of database records and describe an algorithm for decoding in this framework that allows to intersect the grammar with additional information capturing fluency and syntactic well-formedness constraints. Experimental evaluation on several domains achieves results competitive with state-of-the-art systems that use domain specific constraints, explicit feature engineering or labeled data.