Two-dimensional Convolutional Network Research Articles

Intelligent speech elderly rehabilitation learning assistance system based on deep learning and sensor networks

Recently, deep learning has been proved to significantly improve the quality of speech recognition. Convolutional neural networks are often used in speech recognition tasks because of their special network structure and powerful learning function. In order to solve the problem that the traditional convolutional neural network can not reflect the one-dimensional basic attributes of speech signal, this paper proposes to set the number of frames for one dimension of convolution kernel, and use one-dimensional model and two-dimensional convolutional network model for speech recognition. By moving the convolution kernel of time axis and frequency band, it can adapt to the time change of speech signal and maintain the correlation between frequency bands to a great extent. At the same time, this paper also discusses the speech signal preprocessing, feature parameter extraction and regularization algorithm. Due to the lack of hospital resources, the lag of information technology, the poor ability of accompanying and other reasons, the current accompanying service for elderly rehabilitation can not meet the needs of elderly patients. The rapid development and wide use of information technology provide opportunities for the optimization of health services for the elderly. In view of the difficulty of word memory and interpretation in current rehabilitation courses, a VR intelligent teaching system based on intelligent voice technology is developed. The application results show that the system can effectively improve the ability of language expression and word writing. At present, the system of intelligent speech function has not been completed, and lacks speech synthesis function. The next research will focus on the use of speech synthesis technology, in order to realize the man-machine dialogue between people and the system, and show a more real training situation.

An Adaptive Multi-Feature Fusion Network for Predicting Overall Survival of Patients with Head and Neck Cancer

Accurate prognostic prediction could allow personalized treatment to achieve optimal clinical outcome. We aimed to develop a highly predictive overall survival model, considering the complementary relationships between clinical information, traditional radiomics and deep image information, to further improve the overall prediction accuracy by constructing a richer feature set and adaptive weighting. A total of 427 patients with Oropharyngeal Cancer (OPC) patients from the TCIA database were included. 341 cases were used for training, 86 cases were used as an independent cohort. Patient characteristics, including TMN, age, gender, HPV status, smoking or drinking status, etc. were considered as potential predictors. Traditional radiomics features of gross tumor volume (GTV) was extracted from planning CT using open-source software. In addition, a two-dimensional convolutional network (2D_CNN) was designed to extract deep image features. An adaptive multi-feature fusion network was developed to predict overall survival of patients based on three types of features. The fusion network integrates an attention mechanism to the channel dimension to obtain proper weighting of each channel in the feature graph through the fully connected network by focusing on effective feature channels and automatic learning according to the loss, thus improving the utilization rate of effective features. The model performance was evaluated using the area-under-ROC-curve (AUC), accuracy, precision, recall, f1-score. The AUCs of predictive models based on clinical features, traditional radiomics features and deep image features were 0.7, 0.61 and 0.72, respectively. Combining patient characteristics, radiomic features and deep imaging features, the AUCs of the prediction models was significantly improved to 0.85 and 0.86 (with attention mechanisms) for the independent test cohort (Table 1). The proposed adaptive multi-channel network assigned effective weights to the potential predictors, selectively enhanced useful features while suppressed irrelevant features, enabling more accurate feature map weights. We demonstrated the improved predictive value, with a multi-channel fusion network integrated with an attention mechanism, for overall survival of OPC patients.

Using Histogram Extracted from Satelite Imagery and Convolutional Network to Predict GRDP in Java Region

Inequality is one of the problems faced by all countries in the world, including Indonesia. The data used to measure development inequality between regions mostly use GRDP data. However, the GRDP data issued by BPS has a deficiency, it was released after the current year, and this figure is provisional. Therefore, a new data source is needed that can be used to estimate the value of economic activity so that it can be used to measure the level of inequality in development in a region. Nighttime Light (NTL) satellite imagery data can be an alternative to see socioeconomic activity in an area and have been shown to have a strong correlation with socioeconomic activity. In this study, we used VIIRS NTL satellite imagery data and Dynamic World land cover data to estimate GRDP. Rather than using statistical features for each area of interest, we use features in the form of histograms extracted from NTL images and land cover images for each area of interest. Using a histogram, we do not lose spatial information from satellite imagery. Then we proposed a deep learning method in the form of a one-dimensional convolutional neural network using the Huber loss function. This model obtained good precision with an R-square value of 0.8549, beating the baseline method with two-dimensional convolutional networks. The use of the Huber loss function can improve the performance of the model, which has a smaller total loss and has a smoother gradient.

A light-weight neural network for marine acoustic signal recognition suitable for fiber-optic hydrophones

Fiber-optic hydrophones based on optical fiber sensing and communication technology have developed rapidly in recent years. With the advantages of high sensitivity, small size, and large dynamic range, they are easy to detect underwater acoustic signals, showing great potential in the field of marine ecological protection. Researchers can monitor the marine ecological environment by recognizing underwater optical fiber sensing signals collected by fiber-optic hydrophones in real-time. Deep learning models can achieve high recognition accuracy, but they usually have large amounts of parameters that makes them difficult to be deployed in mobile terminals of marine survey ships. To solve the key problem, this paper discussed the design of the Deep Light-Weight Attention Residual Network (DLA-ResNet). It introduces the depth-wise separable convolution (DSC) and the convolutional block attention module (CBAM) into ResNet18 to make its performance better. ResNet18 is a two-dimensional convolutional network (CNN) model applying residual blocks. Experiments verify that the accuracy of DLA-ResNet improved from ResNet18 is increased by about 2.04% and its size is reduced by 85.67%. The proposed scheme in this paper realizes an average recognition rate of 96.50% for eight types of underwater optical fiber sensing signals in the application of marine ecological environment monitoring. Moreover, its model size is only 6.12 MB and it only takes 27.90 ms, which is available to meet the needs of reliability and effectiveness.

Two-dimensional convolution-based power system reliability assessment

With the rise in carbon emissions, it has become fashionable to construct a new power system using new energy as the primary raw material for power generation. This, however, might result in a significant rise in the number of system states and frequent changes in the system state. These issues raise the bar for the effectiveness of system dependability evaluation. Traditional evaluation procedures necessitate computing Optimal Power Flow (OPF) for many systems states one by one, which is unquestionably time-consuming. For that purpose, this research proposes a reliability evaluation approach based on two-dimensional convolutional networks. Branch circuit fault messages are analyzed with generator fault messages. Multiple channel state data from power system node admittance matrices, power generation, and energy demand are fed into a two-dimensional convolutional neural network. This is used to determine the regression connection between load curtailments and system status data and to forecast load curtailments. Finally, it is applied to the RTS-79 system, and experimental results demonstrate that the technique has certain advantages in terms of computing speed and accuracy.

Two-Dimensional Convolutional Neural Network for Depression Episodes Detection in Real Time Using Motor Activity Time Series of Depresjon Dataset.

Depression is a common illness worldwide, affecting an estimated 3.8% of the population, including 5% of all adults, in particular, 5.7% of adults over 60 years of age. Unfortunately, at present, the ways to evaluate different mental disorders, like the Montgomery–Åsberg depression rating scale (MADRS) and observations, need a great effort, on part of specialists due to the lack of availability of patients to obtain the necessary information to know their conditions and to detect illness such as depression in an objective way. Based on data analysis and artificial intelligence techniques, like Convolutional Neural Network (CNN), it is possible to classify a person, from the mental status examination, into two classes. Moreover, it is beneficial to observe how the data of these two classes are similar in different time intervals. In this study, a motor activity database was used, from which the readings of 55 subjects of study (32 healthy and 23 with some degree of depression) were recorded with a small wrist-worn accelerometer to detect the peak amplitude of movement acceleration and generate a transient voltage signal proportional to the rate of acceleration. Motor activity data were selected per patient in time-lapses of one day for seven days (one week) in one-minute intervals. The data were pre-processed to be given to a two-dimensional convolutional network (2D-CNN), where each record of motor activity per minute was represented as a pixel of an image. The proposed model is capable of detecting depression in real-time (if this is implemented in a mobile device such as a smartwatch) with low computational cost and accuracy of 76.72% In summary, the model shows promising abilities to detect possible cases of depression, providing a helpful resource to identify the condition and be able to take the appropriate follow-up for the patient.

BMCSA:Multi-feature spatial convolution semantic matching model based on BERT

This paper proposes a multi-feature spatial convolutional semantic matching model (BMCSA) based on BERT by enriching different feature spatial information of semantic features. BMCSA employs the BERT model to extract the semantic features of the text, then uses the two-dimensional convolutional network to extract different feature spatial information, and finally combines the Attention mechanism to capture the global feature spatial information. We use two different semantic matching data sets and a text inference data set to verify the effectiveness of the proposed model. Experimental results prove that BMCSA is better than the baseline model.

Spatial–temporal interaction module for action recognition

Video action recognition methods based on deep learning can be divided into two types: two-dimensional convolutional networks (2D-ConvNets) relied and three-dimensional convolutional networks (3D-ConvNets) relied. 2D-ConvNets are more efficient to learn spatial features, but cannot capture temporal relationships directly. 3D-ConvNets can jointly learn spatial–temporal features, but their learning is time-consuming because of a large number of networks’ parameters. We therefore propose an effective spatial–temporal interaction (STI) module. The 2D spatial convolution and the one-dimensional temporal convolution are combined through attention mechanism in STI to learn the spatial–temporal information effectively and efficiently. The computation cost of the proposed method is far less than 3D convolution. The proposed STI module can be combined with 2D-ConvNets to obtain the effect of 3D-ConvNets with far fewer parameters, and it can also be inserted into 3D-ConvNets to improve their ability to learn spatial–temporal features, so as to improve the recognition accuracy. Experimental results show that the proposed method outperforms the existing counterparts on benchmark datasets.

Student achievement prediction using deep neural network from multi-source campus data

Finding students at high risk of poor academic performance as early as possible plays an important role in improving education quality. To do so, most existing studies have used the traditional machine learning algorithms to predict students’ achievement based on their behavior data, from which behavior features are extracted manually thanks to expert experience and knowledge. However, owing to an increase in the varieties and overall volume of behavioral data, it has become more and more challenging to identify high-quality handcrafted features. In this paper, we propose an end-to-end deep learning model that automatically extracts features from students’ multi-source heterogeneous behavior data to predict academic performance. The key innovation of this model is that it uses long short-term memory networks to capture inherent time-series features for each type of behavior, and it takes two-dimensional convolutional networks to extract correlation features among different behaviors. We conducted experiments with four types of daily behavior data from students of the university in Beijing. The experimental results demonstrate that the proposed deep model method outperforms several machine learning algorithms.

Human Activity Classification Based on Point Clouds Measured by Millimeter Wave MIMO Radar With Deep Recurrent Neural Networks

We investigate the feasibility of classifying human activities measured by a MIMO radar in the form of a point cloud. If a human subject is measured by a radar system that has a very high angular azimuth and elevation resolution, scatterers from the body can be localized. When precisely represented, individual points form a point cloud whose shape resembles that of the human subject. As the subject engages in various activities, the shapes of the point clouds change accordingly. We propose to classify human activities through recognition of point cloud variations. To construct a dataset, we used an FMCW MIMO radar to measure 19 human subjects performing 7 activities. The radar had 12 TXs and 16 RXs, producing a 33×31 virtual array with approximately 3.5 degrees of angular resolution in azimuth and elevation. To classify human activities, we used a deep recurrent neural network (DRNN) with a two-dimensional convolutional network. The convolutional filters captured point clouds' features at time instance for sequential input into the DRNN, which recognized time-varying signatures, producing a classification accuracy exceeding 97%.

Physics Validation of Novel Convolutional 2D Architectures for Speeding Up High Energy Physics Simulations

The precise simulation of particle transport through detectors remains a key element for the successful interpretation of high energy physics results. However, Monte Carlo based simulation is extremely demanding in terms of computing resources. This challenge motivates investigations of faster, alternative approaches for replacing the standard Monte Carlo technique.We apply Generative Adversarial Networks (GANs), a deep learning technique, to replace the calorimeter detector simulations and speeding up the simulation time by orders of magnitude. We follow a previous approach which used three-dimensional convolutional neural networks and develop new two-dimensional convolutional networks to solve the same 3D image generation problem faster. Additionally, we increased the number of parameters and the neural networks representational power, obtaining a higher accuracy. We compare our best convolutional 2D neural network architecture and evaluate it versus the previous 3D architecture and Geant4 data. Our results demonstrate a high physics accuracy and further consolidate the use of GANs for fast detector simulations.

Computer-Aided Recognition and Analysis of Abnormal Behavior in Video

In intelligent computer-aided video abnormal behavior recognition, pedestrian behavior analysis technology can detect and handle abnormal behaviors in time, which has great practical value in ensuring social safety. We analyze a deep learning video behavior recognition network that has advantages in current research. The network first sparsely sampled the input video to obtain the video frame of each video segment, and then used a two-dimensional convolutional network to extract the characteristics of each video frame, then used a three-dimensional network to fuse them. The method realizes the recognition of long-term and short-term actions in the video at the same time. In order to overcome the shortcoming of the large amount of calculation in the 3D convolution part of the network, this paper proposes an improvement to this module in the network, and proposes a mobile 3D convolution network structure. Aiming at the problem of low utilization of long-term motion features in video sequences, this paper constructs a deep residual module by introducing long and short-term memory networks, residual connection design, etc., to fully and effectively utilize the long-term dynamic features in video sequences. Aiming at the problem of large differences in similar actions and small differences between classes in abnormal behavior videos, this paper proposes a 2CSoftmax function based on double center loss to optimize the network model, which is beneficial to maximize the distance between classes and minimize the distance between classes, so as to realize the classification and recognition of similar actions and improve the recognition accuracy.

Classification of hyperspectral images of the interior of fruits and vegetables using a 2D convolutional neuronal network

This work evaluates the performance of a Deep Learning technique for classification of challenging hyperspectral images of the interior of fruits and vegetables when they are combined. Some of these samples have low contrast, similar colour features, and their skins or characteristic shapes are lost when cut to expose their interiors. We implemented a two-dimensional convolutional neural network for this classification task and compared their results against the technique of support vector machines. We randomly selected a group of 13 hyperspectral images from a public database containing information of the interior of 42 fruits and vegetables. Using parts of these 13 selected images, we constructed three artificial hyperspectral images merging these parts differently. We applied the two proposed techniques over the three of them. The comparison of the classification results shows that the two-dimensional convolutional neural network over-performs the support vector machine in all three composite images. The two-dimensional convolutional neural network exceeded 98% classification accuracy in all of them. These results show that the two-dimensional convolutional network benefits from the spatial and spectral data in the images obtaining proper levels of classification even in samples mixed in complex contexts, as it can occur in the food or pharmaceutical industries.

Speech enhancement and speaker verification with convolutional neural networks

Convolutional neural networks have been used with great success for image processing tasks. The convolution filters can act as powerful feature detectors in images to find edges and shapes. Time domain speech signals are one dimensional, but frequency domain information can be viewed in the same way as an image, allowing one to use two-dimensional convolutional networks for tasks such as speech enhancement. In this paper, recently reported deep neural network preprocessing methods used for dereverberation are examined to explore their ability to remove noise and or reverberation. A convolutional network is then introduced and compared to the previous methods to explore the relative strengths and weaknesses of each approach. Comparisons are made through experimentation in the context of speech quality and speaker identification tasks.