2D Signals Research Articles

Automatic Emotion Recognition From Multi-Band EEG Data Based on a Deep Learning Scheme With Effective Channel Attention

Automatic emotion recognition using electroencephalogram (EEG) has obtained a wide range of attention in the domain of human-computer interaction (HCI) owing to the notable differences in brain activities in the event of different types of emotions. In this paper, a novel emotion recognition approach is proposed based on a deep learning scheme utilizing the temporal, spatial, and frequency effects of the EEG signal. As neural firing provides a pathway to elicit emotions, temporal, spatial, and frequency sub-band analysis of EEG signals uncovers salient information to categorize different classes of emotions. In this regard, temporal data from each channel are divided into major spectral bands and 2D signal matrices are constructed by combining the temporal information of different frequency band signals. After concatenating all signal matrices obtained from the available channels, a 3D feature space is obtained, which can better characterize emotion variations and, thus, better classification performance is obtained. The feature space is applied to a 2D deep neural network where the band information is passed to the depth dimension of the neural network. In order to highlight the important channels, a channel attention mechanism is proposed with the neural network to distribute the weights among the channels according to the contribution. Hence, the modified feature space effectively captures distinctive information about specific channels in the context of emotion recognition. In this study, detailed and extensive experimentations are carried out on a publicly available DEAP dataset and a very satisfactory performance is obtained for the valence and the arousal domain in 2-class scenario for the subject-dependent case. The average accuracies obtained for valence and arousal domain in binary class problem are 97.06% and 96.93%, respectively.

Detecting Lombard Speech Using Deep Learning Approach.

Robust Lombard speech-in-noise detecting is challenging. This study proposes a strategy to detect Lombard speech using a machine learning approach for applications such as public address systems that work in near real time. The paper starts with the background concerning the Lombard effect. Then, assumptions of the work performed for Lombard speech detection are outlined. The framework proposed combines convolutional neural networks (CNNs) and various two-dimensional (2D) speech signal representations. To reduce the computational cost and not resign from the 2D representation-based approach, a strategy for threshold-based averaging of the Lombard effect detection results is introduced. The pseudocode of the averaging process is also included. A series of experiments are performed to determine the most effective network structure and the 2D speech signal representation. Investigations are carried out on German and Polish recordings containing Lombard speech. All 2D signal speech representations are tested with and without augmentation. Augmentation means using the alpha channel to store additional data: gender of the speaker, F0 frequency, and first two MFCCs. The experimental results show that Lombard and neutral speech recordings can clearly be discerned, which is done with high detection accuracy. It is also demonstrated that the proposed speech detection process is capable of working in near real-time. These are the key contributions of this work.

MLMRS-Net: Electroencephalography (EEG) motion artifacts removal using a multi-layer multi-resolution spatially pooled 1D signal reconstruction network

Electroencephalogram (EEG) signals suffer substantially from motion artifacts when recorded in ambulatory settings utilizing wearable sensors. Because the diagnosis of many neurological diseases is heavily reliant on clean EEG data, it is critical to eliminate motion artifacts from motion-corrupted EEG signals using reliable and robust algorithms. Although a few deep learning-based models have been proposed for the removal of ocular, muscle, and cardiac artifacts from EEG data to the best of our knowledge, there is no attempt has been made in removing motion artifacts from motion-corrupted EEG signals: In this paper, a novel 1D convolutional neural network (CNN) called multi-layer multi-resolution spatially pooled (MLMRS) network for signal reconstruction is proposed for EEG motion artifact removal. The performance of the proposed model was compared with ten other 1D CNN models: FPN, LinkNet, UNet, UNet+, UNetPP, UNet3+, AttentionUNet, MultiResUNet, DenseInceptionUNet, and AttentionUNet++ in removing motion artifacts from motion-contaminated single-channel EEG signal. All the eleven deep CNN models are trained and tested using a single-channel benchmark EEG dataset containing 23 sets of motion-corrupted and reference ground truth EEG signals from PhysioNet. Leave-one-out cross-validation method was used in this work. The performance of the deep learning models is measured using three well-known performance matrices viz. mean absolute error (MAE)-based construction error, the difference in the signal-to-noise ratio (ΔSNR), and percentage reduction in motion artifacts (η). The proposed MLMRS-Net model has shown the best denoising performance, producing an average ΔSNR, η, and MAE values of 26.64 dB, 90.52%, and 0.056, respectively, for all 23 sets of EEG recordings. The results reported using the proposed model outperformed all the existing state-of-the-art techniques in terms of average η improvement.

Open-source, highly efficient, post-acquisition synchronization for 4D dual-contrast imaging of the mouse embryonic heart over development with optical coherence tomography.

Dynamic imaging of the beating embryonic heart in 3D is critical for understanding cardiac development and defects. Optical coherence tomography (OCT) plays an important role in embryonic heart imaging with its unique imaging scale and label-free contrasts. In particular, 4D (3D + time) OCT imaging enabled biomechanical analysis of the developing heart in various animal models. While ultrafast OCT systems allow for direct volumetric imaging of the beating heart, the imaging speed remains limited, leading to an image quality inferior to that produced by post-acquisition synchronization. As OCT systems become increasingly available to a wide range of biomedical researchers, a more accessible 4D reconstruction method is required to enable the broader application of OCT in the dynamic, volumetric assessment of embryonic heartbeat. Here, we report an open-source, highly efficient, post-acquisition synchronization method for 4D cardiodynamic and hemodynamic imaging of the mouse embryonic heart. Relying on the difference between images to characterize heart wall movements, the method provides good sensitivity to the cardiac activity when aligning heartbeat phases, even at early stages when the heart wall occupies only a small number of pixels. The method works with a densely sampled single 3D data acquisition, which, unlike the B-M scans required by other methods, is readily available in most commercial OCT systems. Compared with an existing approach for the mouse embryonic heart, this method shows superior reconstruction quality. We present the robustness of the method through results from different embryos with distinct heart rates, ranging from 1.24 Hz to 2.13 Hz. Since the alignment process operates on a 1D signal, the method has a high efficiency, featuring sub-second alignment time while utilizing ∼100% of the original image files. This allows us to achieve repeated, dual-contrast imaging of mouse embryonic heart development. This new, open-source method could facilitate research using OCT to study early cardiogenesis.

Use of Neural Networks and Decision Trees in Compression of 2D and 3D Digital Signals

Use of Neural Networks and Decision Trees in Compression of 2D and 3D Digital Signals

Synaptic Activity and Hardware Footprint of Spiking Neural Networks in Digital Neuromorphic Systems

Spiking neural networks are expected to bring high resources, power, and energy efficiency to machine learning hardware implementations. In this regard, they could facilitate the integration of Artificial Intelligence in highly constrained embedded systems, such as image classification in drones or satellites. If their logic resource efficiency is widely accepted in the literature, their energy efficiency still remains debated. In this article, a novel high-level metric is used to characterize the expected energy efficiency gain when using Spiking Neural Networks (SNN) instead of Formal Neural Networks (FNN) for hardware implementation: Synaptic Activity Ratio (SAR). This metric is applied to a selection of classification tasks including images and 1D signals. Moreover, a high-level estimator for logic resources, power usage, execution time, and energy is introduced for neural network hardware implementations on FPGA, based on four existing accelerator architectures covering both sequential and parallel implementation paradigms for both spiking and formal coding domains. This estimator is used to evaluate the reliability of the Synaptic Activity Ratio metric to characterize spiking neural network energy efficiency gain on the proposed dataset benchmark. This study led to the conclusion that spiking domain offers significant power and energy savings in sequential implementations. This study also shows that synaptic activity is a critical factor that must be taken into account when addressing low-energy systems.

A CNN model based on innovative expansion operation improving the fault diagnosis accuracy of drilling pump fluid end

Accurate fault diagnosis is critical to the safe and reliable operation of the drilling pump. The challenge in fault diagnosis of the fluid end of the drilling pump is to extract the fault features more accurately. This paper proposes an innovative expansion operation to convert the one-dimensional (1D) signal into three-dimensional (3D) images with 3 channels, which not only increases the visibility of feature information but also reduces the influence of noise. To increase the identification types and improve the identification accuracy, this paper provides a Convolutional Neural Network (CNN) drilling pump fluid end fault diagnosis method based on the expansion operation. Firstly, to solve the problem that the feature information is not obvious in the 3D images, the expansion operation converts the 1D vibration signals into 3D images to generate images with more obvious features. Then, the AlexNet model is investigated by changing the number of neurons in the first fully connected layer. Compared to directly generated 3D images, grayscale images generated by Do’s method, and grayscale images generated by Do’s method combined with expansion operation, the diagnosis accuracy of the AlexNet model based on the 3D images generated by expansion operation is the highest. Finally, the experiment of the proposed fault diagnosis method of the fluid end of the drilling pump is carried out. Results show that the method proposed in this paper identifies 9 types of faults of the drilling pump’s fluid end with a mean accuracy of 98.00%, which is significantly higher than that of other diagnostic methods. This research provides a new approach to more accurately diagnose the drilling pump fluid end faults.

An efficient approach for guided wave structural monitoring of switch rails via deep convolutional neural network-based transfer learning

Data-driven approaches in structural health monitoring have received increasing attention, especially advances in deep learning-based methods, which have further driven the development of data-driven damage detection. Due to the limited availability of guided wave samples and the imbalance between data classes, this study proposes a deep convolutional neural network-based transfer learning (DCTL) approach for the structure monitoring of switch rails using guided wave monitoring signals. A pretrained model based on Inception-ResNet-V2 was adopted and fine-tuned. Different methods for converting 1D signals into 2D images were investigated to find the optimal approach that meets practical monitoring requirements. Affine transformations were used for data augmentation to improve generalization ability and to avoid the overfitting of the training model. Two types of guided wave monitoring experiments on the foot and web of switch rails were conducted to evaluate the proposed method against different conventional methods in the field of switch rails. In addition, the DCTL method was investigated, with and without pretrained weights, along with different frozen layers. The classification results show that the proposed method can identify damage in challenging situations and outperforms conventional methods.

A feature-level multi-sensor fusion approach for in-situ quality monitoring of selective laser melting

Selective laser melting (SLM) is a commonly used technique in additive manufacturing to produce metal components with complex geometries and high precision. However, the poor process reproducibility and unstable product reliability has hindered its wide adoption in practice. Hence, there is a pressing demand for in-situ quality monitoring and real-time process control. In this paper, a feature-level multi-sensor fusion approach is proposed to combine acoustic emission signals with photodiode signals to realize in-situ quality monitoring for intelligence-driven production of SLM. An off-axial in-situ monitoring system featuring a microphone and a photodiode is developed to capture the process signatures during the building process. According to the 2D porosity and 3D density measurements, the collected acoustic and optical signals are grouped into three categories to indicate the quality of the produced parts. In consideration of the laser scanning information, an approach to transform the 1D signal to 2D image is developed. The converted images are then used to train a convolutional neural network so as to extract and fuse the features derived from the two individual sensors. In comparison with several baseline models, the proposed multi-sensor fusion approach achieves the best performance in quality monitoring.

Performance evaluation of CNN and R-CNN based line by line analysis algorithms for fibre placement defect classification

The Automated Fibre Placement process is commonly used in aerospace for the manufacturing of structural components, but requires a subsequent inspection to meet the corresponding safety requirements. In order to improve this mostly manual inspection step, machine learning methods for the interpretation of 2D surface images are being increasingly utilised in research. Depending on the manufacturing process, a very long time can elapse between the appearance of a manufacturing defect and its recognition. Hence, in this paper Convolutional and Recurrent Neural Network techniques are presented that allow a line-by-line analysis of the incoming height profile scans of a Laser Line Scan Sensor as a 1D signal, which enables a direct reaction to a defect, even if only one or a few individual height profiles of the defect have been recorded. The combination of Convolutional and Recurrent Neural Network structures is particularly beneficial for this application. The investigations in this paper are especially interesting for developers of automated inspection systems in composite engineering.

Numerical Analysis of Energy Recovery of Hybrid Loader Actuators Based on Parameters Optimization

The conventional loader actuator hydraulic system suffers from the potential energy waste problem of the boom arm. This study proposes a hydraulic control method and control strategy for the energy recovery and regeneration of a hybrid loader arm. When the boom arm drops, the piston side of the boom cylinder charges the accumulator, and the system achieves energy recovery. When the boom arm rises, the accumulator releases hydraulic energy to drive the energy regeneration hydraulic motor to provide energy for the system, and the system achieves energy regeneration. The system’s principle analysis and the mathematical model are completed based on Boyle’s, Newton’s second law, and the flow continuity principle. The simulation model is established using AMESim 2D mechanical library, HCD library, and signal library. Under the typical working condition, 50-type wheel loader numerical simulation research is conducted, and the system cylinder motion characteristics, accumulator charging and discharging performance, system energy recovery, and regeneration performance are obtained. On this basis, energy recovery and regeneration efficiency are selected as optimization objectives. The optimal designs of accumulator and energy regeneration hydraulic motor parameters are carried out to obtain the influence law of accumulator and hydraulic motor parameters on system energy recovery and regeneration, and the energy-saving effect of the system is analyzed. The results show that the optimized parameters effectively improve the system energy recovery and regeneration efficiency and reduce engine fuel consumption. The system provides a reference for designing an energy recovery system and researching the energy-saving technology of loaders.

Regulated 2D Grayscale Image for Finding Power Quality Abnormalities in Actual Data

It is possible to preserve power quality by classifying and identifying abnormalities. Prior studies focused on enhancing the PQD classification performance in one-dimensional (1D) CNNs. Recently, various image conversion methods have been established to facilitate CNN for PQD classification. PQD is a 1D signal that needs to be converted to a 2D image through data pre-processing since 2D images may include more PQD information than 1D signals. However, the PQD data used for the power quality classifier is synthetic PQD produced using mathematical models with parameter modifications in accordance with IEEE Std. 1159, which places limitations on prior research. This study uses data from the Amrita Honeywell Hackathon 2021 to examine how the response-based 2D deep CNN power quality classifier responds to actual field power quality disruptions. The results of the study show that a 2D deep CNN with regulated 2D grayscale pictures based on a process-regulated 2D image matrix can classify real data power quality disturbances with accuracy, precision, recall, and F1-score of 98.80%, 98.99%, and 98.60%, respectively. Additionally, 2D images can potentially contain more PQD data than 1D signals, enhancing identification performance on actual data.

Lightweight Multireceptive Field CNN for 12-Lead ECG Signal Classification.

The electrical activity produced during the heartbeat is measured and recorded by an ECG. Cardiologists can interpret the ECG machine's signals and determine the heart's health condition and related causes of ECG signal abnormalities. However, cardiologist shortage is a challenge in both developing and developed countries. Moreover, the experience of a cardiologist matters in the accurate interpretation of the ECG signal, as the interpretation of ECG is quite tricky even for experienced doctors. Therefore, developing computer-aided ECG interpretation is required for its wide-reaching effect. 12-lead ECG generates a 1D signal with 12 channels among the well-known time-series data. Classical machine learning can develop automatic detection, but deep learning is more effective in the classification task. 1D-CNN is being widely used for CVDS detection from ECG datasets. However, adopting a deep learning model designed for computer vision can be problematic because of its massive parameters and the need for many samples to train. In many detection tasks ranging from semantic segmentation of medical images to time-series data classification, multireceptive field CNN has improved performance. Notably, the nature of the ECG dataset made performance improvement possible by using a multireceptive field CNN (MRF-CNN). Using MRF-CNN, it is possible to design a model that considers semantic context information within ECG signals with different sizes. As a result, this study has designed a multireceptive field CNN architecture for ECG classification. The proposed multireceptive field CNN architecture can improve the performance of ECG signal classification. We have achieved a 0.72 F1 score and 0.93 AUC for 5 superclasses, a 0.46 F1 score and 0.92 AUC for 20 subclasses, and a 0.31 F1 score and 0.92 AUC for all the diagnostic classes of the PTB-XL dataset.

Fitness Movement Types and Completeness Detection Using a Transfer-Learning-Based Deep Neural Network.

Fitness is important in people’s lives. Good fitness habits can improve cardiopulmonary capacity, increase concentration, prevent obesity, and effectively reduce the risk of death. Home fitness does not require large equipment but uses dumbbells, yoga mats, and horizontal bars to complete fitness exercises and can effectively avoid contact with people, so it is deeply loved by people. People who work out at home use social media to obtain fitness knowledge, but learning ability is limited. Incomplete fitness is likely to lead to injury, and a cheap, timely, and accurate fitness detection system can reduce the risk of fitness injuries and can effectively improve people’s fitness awareness. In the past, many studies have engaged in the detection of fitness movements, among which the detection of fitness movements based on wearable devices, body nodes, and image deep learning has achieved better performance. However, a wearable device cannot detect a variety of fitness movements, may hinder the exercise of the fitness user, and has a high cost. Both body-node-based and image-deep-learning-based methods have lower costs, but each has some drawbacks. Therefore, this paper used a method based on deep transfer learning to establish a fitness database. After that, a deep neural network was trained to detect the type and completeness of fitness movements. We used Yolov4 and Mediapipe to instantly detect fitness movements and stored the 1D fitness signal of movement to build a database. Finally, MLP was used to classify the 1D signal waveform of fitness. In the performance of the classification of fitness movement types, the mAP was 99.71%, accuracy was 98.56%, precision was 97.9%, recall was 98.56%, and the F1-score was 98.23%, which is quite a high performance. In the performance of fitness movement completeness classification, accuracy was 92.84%, precision was 92.85, recall was 92.84%, and the F1-score was 92.83%. The average FPS in detection was 17.5. Experimental results show that our method achieves higher accuracy compared to other methods.

Temporal super-resolution of echocardiography using a novel high-precision non-polynomial interpolation

Higher frame rates are very useful for improving medical diagnosis in fast-moving parts of the heart, especially in the valves. To this end, we propose a non-polynomial interpolation method for increasing the frame rate in echocardiography. Besides describing the proposed method, we present two additional contributions: (1) we obtain a closed-form solution, which is continuous and infinitely differentiable; (2) we provide an error analysis of the method. The resulting error bound indicates that the interpolation method is reliable. Finally, to show the efficiency of our proposal in temporal super-resolution, i.e., the increase in frame rate, we apply it to three types of datasets, including a 1D signal, a simulated dataset, and B-mode echocardiography images. Our experimental results show that the Mean Squared Error of the proposed method is reduced from 0.6 to 0.3, while having the same computational complexity compared to cubic B-spline. The quantitative results also indicate that, even with lower selection rates, we can reach a high performance reconstruction while the image quality is not degraded significantly.

SAR Signal Formation and Image Reconstruction of a Moving Sea Target

Maritime application of synthetic aperture radar (SAR) technology for sea-target surveillance and imaging is considered in this study. A SAR scenario, including the kinematics of a SAR satellite and a ship moving on the sea, along with the geometry of the target, are analytically described. A linear frequency modulation (LFM) waveform is applied for the target’s illumination. Based on the target’s geometry, SAR and target kinematics and the LFM waveform, a SAR signal model is synthesized. It is proven that the process of signal formation is a transformation of the three-dimensional (3D) image into a two-dimensional (2D) signal, whereas the target’s 2D imaging is an inverse transformation of the 2D signal into the target’s 2D image. SAR signal components, linear Fourier terms and higher-order phase terms are analytically derived and discussed in detail. Moreover, it is proven that SAR image reconstruction is a motion-compensation procedure, i.e., it removes all phases induced by first- and higher-order motion. Based on the SAR signal analysis, an illustrative iterative image-reconstruction algorithm is derived. The quality of the imaging is evaluated by an entropy cost function. Simulation experiments are carried out to verify the correctness of the theoretical statements in respect of SAR signal formation and image reconstruction.

An Alternative Computation of the Entropy of 1D Signals Based on Geometric Properties

The objective of this work is to present a novel methodology based on the computation of a couple of geometric characteristics of the position of the data points in 1D signal to propose an alternative estimation of signal entropy. The conditions to be fulfilled by the signal are minimal; only those necessary to meet the sampling theorem requirement are enough. This work shows some examples in which the proposed methodology can distinguish among signals that cannot be differentiated by other in-use alternatives. Additionally an original example where the usual ordinal pattern algorithm to compute entropy is not applicable, is presented and analyzed. The proposal developed through this work carries some advantages over other alternatives and constitutes a true advancement in the pathway to compute the distribution function of the sequential points of 1D signals later used to compute the entropy of the signal.

Comparison of spectrogram, persistence spectrum and percentile spectrum based image representation performances in drone detection and classification using novel HMFFNet: Hybrid Model with Feature Fusion Network

Drone detection and classification are gaining great importance in military and civilian applications because of their availability and widespread usages. This study presents a new framework, named as Hybrid Model with Feature Fusion Network (HMFFNet), for drone detection and classification using Radio Frequency (RF) signals. It is composed of three stages. In the first stage, 1D RF signals are segmented to specific sizes and each segment is transformed into 2D signals in terms of spectrogram, persistence spectrum and percentile spectrum images to be employed in the well-known deep learning architectures through transfer learning approach. In the second stage, VGG19, one of these architectures, is utilized for feature extraction using the transformed images. Then the features are fused in various combinations to improve their discriminative property. Finally, these features are processed by a Support Vector Machine (SVM) classifier in the last stage of the proposed framework. To evaluate the performance of the HMFFNet, numerous experiments were conducted on the publicly available DroneRF dataset. The experimental results showed that the performance of the novel percentile spectrum based features are better compared to the widely-used and good-performing spectrogram based features in drone detection and classification. Also, fusion of features from three drone representing images further improve the performance of the proposed frameworks. For the quantitative evaluation, experiments yield to the accuracy rates of 100%, 99.55%, and 97.75% for 2-Class drone existence, 4-Class drone type detection and 10-Class drone mode detection, respectively, showing higher results than that of other state-of-the-art studies.

Learning-based high-quality image recovery from 1D signals obtained by single-pixel imaging

As an innovative and computational imaging technique, it is critical for single-pixel imaging (SPI) to achieve a high reconstruction quality. However, the reconstruction image quality in conventional SPI is heavily dependent on the sampling rate. In this work, we present a learning-based SPI approach for high-quality image reconstruction under a low sampling rate. A bucket detector included in our optical setup is used to collect the one-dimensional (1D) signals reflected from a two-dimensional (2D) object and an end-to-end generative adversarial network (EGAN) which is pre-trained with simulated data is utilized to implement the reconstruction of the object. The results show that the proposed approach is able to produce high quality approximations of 2D images from optically collected 1D bucket signals at a very low sampling ratio. It can also be shown a better performance can be achieved by compared with previous studies on the same dataset, such as conventional SPI, compressive-sensing ghost imaging (CSGI) and U-net-based SPI approaches.

MIMO Radar Imaging Based on Two-dimensional Compressive Sensing Recovery Algorithm

Conventional sparse signal recovery-based MIMO radar imaging method rearranges the received two-dimensional (2D) signals into a vector, However, transforming the two dimensional equation to one dimensional equation then solving linear equation based sparse signal recovery problem requires huge memory and computational budget. A two-dimensional compressed sensing MIMO radar imaging algorithm based on improved smooth L0 norm is proposed in this paper. Firstly, a two-dimensional MIMO radar imaging model is established to transform the imaging problem into a sparse optimization problem. Then, the two-dimensional negative exponential function sequence is used as the smoothed function sequence to approximate the L0 norm. The two-dimensional imaging of MIMO radar is realized by cyclic iteration and gradient projection algorithm. Simulation results show that the proposed algorithm has advantages over the traditional compressed sensing algorithms.