1D Vector Research Articles

Handwritten Digit Recognition Using Logistic Regression

With applications in optical character recognition (OCR), digitizing historical documents, and automating data entry procedures, handwritten digit recognition is crucial to pattern recognition and machine learning. This work develops a handwritten digit recognition system using logistic regression. The main objective is to use the MNIST dataset to develop a precise and effective model for classifying handwritten numbers from 0 to 9. Data collection, preprocessing, model installation, training, assessment, and analysis are all included in the methodology. Normalizing pixel values and flattening 2D picture matrices into 1D vectors are steps in the preprocessing process. Using gradient descent optimization, the logistic regression model set up for multi- class classification minimizes the cost function. A testing set will be used to assess performance, and metrics including F1 score, recall, accuracy, and precision will be used. A very precise logistic regression model for handwritten digit identification is the anticipated result. The model's advantages and disadvantages will be emphasized through thorough analysis and visualization, which will include learning weights, biases, and incorrectly categorized images. This work offers useful solutions for automation, postal services, and digital archiving, furthering the field of pattern recognition and highlighting the significance of logistic regression in machine learning.

Noise attenuation in distributed acoustic sensing data using a guided unsupervised deep learning network

Distributed acoustic sensing (DAS) is a promising technology introducing a new paradigm in the acquisition of high-resolution seismic data. However, DAS data often show weak signals compared with the background noise, especially in challenging installation environments. In this study, we develop a new approach to denoise DAS data that leverages an unsupervised deep learning (DL) model, eliminating the need for labeled training data. The input DAS data undergo band-pass filtering to eliminate high-frequency content. Subsequently, a continuous wavelet transform (CWT) is performed, and the finest scale is used to guide the DL model in reconstructing the DAS signal. First, we extract 2D patches from the band-pass filtered data and the CWT scale of the data. Then, these patches are converted using an unrolling mechanism into 1D vectors to form the input of the DL model. A self-attention layer is included in each layer to extract the spatial relation between the band-pass filtered data and the CWT scale. Through an iterative process, the DL model tunes its parameters to suppress DAS noise, with the band-pass filtered data serving as the target for the network. The denoising performance of our framework is validated using field examples from the San Andreas Fault Observatory at Depth and Frontier Observatory for Research in Geothermal Energy data sets, where the data are recorded by a fiber-optic cable. Comparative analyses against three benchmark methods reveal the robust denoising performance of our framework.

Long-range attention classification for substation point cloud

Point cloud classification for substation is challenging due to the occluding layout as well as the complexity of methods. Most previous methods trade inference speed for precision by building complex extractors. To alleviate the paradox of performance and complexity trade-off, this paper proposes the lightweight long-range attention classification for substation point cloud, including shuffle channel attention(SCA) and dilated channel attention(DCA). First, SCA captures local cross-channel interaction via 1D convolution and global interaction via channel shuffling, shown promising performance. Second, to further reduce the amount of computation involved in shuffling, we propose a more elegant method DCA by resizing the 1D vector after pooling into 2D feature map. Note that proposed DCA are implemented by just 2D convolution, determining the coverage of cross-channel interaction, with a significantly higher inference speed. Furthermore, we develop a heuristic algorithm to adaptively determine parameters like kernel size, shuffle group and size of 2D feature map. Besides ModelNet40 and ScanObjectNN(PB_T50_RS), this paper also selects ten of main objects in substation for training and testing. Finally, experimental results suggest that proposed methods bring notable performance gain, nearly 1%. DCA reaches 93.801% and just increases 0.0003M parameters but 15.5% faster than SCA. We also make an attempt to simplify proposed methods via reducing the coding dimensions and coding blocks. And simplified DCA achieves 93.4% performance with almost quarter parameters and 112% faster, guaranteeing both efficiency and effectiveness. A simplified version is used to achieve the highest accuracy of 85.115% and 83.304% on ScanObjectNN, with the highest improvements of 1.735% and 2.391%. This paper also conducts robustness test where different proportions of points are missing. Results show that when 93.75% missing, the accuracy only decreases by 4.093%, and can still reach 89%, which is significantly ahead of other methods.

Point Cloud Quality Assessment Using a One-Dimensional Model Based on the Convolutional Neural Network.

Recent advancements in 3D modeling have revolutionized various fields, including virtual reality, computer-aided diagnosis, and architectural design, emphasizing the importance of accurate quality assessment for 3D point clouds. As these models undergo operations such as simplification and compression, introducing distortions can significantly impact their visual quality. There is a growing need for reliable and efficient objective quality evaluation methods to address this challenge. In this context, this paper introduces a novel methodology to assess the quality of 3D point clouds using a deep learning-based no-reference (NR) method. First, it extracts geometric and perceptual attributes from distorted point clouds and represent them as a set of 1D vectors. Then, transfer learning is applied to obtain high-level features using a 1D convolutional neural network (1D CNN) adapted from 2D CNN models through weight conversion from ImageNet. Finally, quality scores are predicted through regression utilizing fully connected layers. The effectiveness of the proposed approach is evaluated across diverse datasets, including the Colored Point Cloud Quality Assessment Database (SJTU_PCQA), the Waterloo Point Cloud Assessment Database (WPC), and the Colored Point Cloud Quality Assessment Database featured at ICIP2020. The outcomes reveal superior performance compared to several competing methodologies, as evidenced by enhanced correlation with average opinion scores.

BrutNet: A novel approach for violence detection and classification using DCNN with GRU

AbstractAutomatic Violence Detection and Classification (AVDC) with deep learning has garnered significant attention in computer vision research. This paper presents a novel approach for combining a custom Deep Convolutional Neural Network (DCNN) with a Gated Recurrent Unit (GRU) in developing a new AVDC model called BrutNet. Specifically, a time‐distributed DCNN (TD‐DCNN) is developed to generate a compact 2D representation with 512 spatial features per frame from a set of equally‐spaced frames of dimension 16090 in short video segments. Further to leverage the temporal information, a GRU layer is utilised, generating a condensed 1D vector that enables binary classification of violent or non‐violent content through multiple dense layers. Overfitting is addressed by incorporating dropout layers with a rate of 0.5, while the hidden and output layers employ rectified linear unit (ReLU) and sigmoid activations, respectively. The model is trained on the NVIDIA Tesla K80 GPU through Google Colab, demonstrating superior performance compared to existing models across various video datasets, including hockey fights, movie fights, AVD, and RWF‐2000. Notably, the model stands out by requiring only 3.416 million parameters and achieving impressive test accuracies of 97.62%, 100%, 97.22%, and 86.43% on the respective datasets. Thus, BrutNet exhibits the potential to emerge as a highly efficient and robust AVDC model in support of greater public safety, content moderation and censorship, computer‐aided investigations, and law enforcement.

Probabilistic analysis of tunnel convergence in spatially variable soil based on Gaussian process regression

Traditional probabilistic analysis of tunnel convergence in spatially variable soil typically involves using the random finite difference method, which is computationally intensive. In this study, we propose an efficient machine learning method based on the Gaussian Process Regression (GPR) model for predicting the horizontal and vertical convergences of a high-speed railway tunnel. In the proposed approach, the soil Young's modulus random field is simplified from a 2D matrix to a 1D vector, which serves as the input of the GPR model. Considering the prior knowledge about the probability distribution of the tunnel convergence, the horizontal and vertical convergences are transformed into two random variables that can be predicted effectively by the GPR model. Notably, the developed GPR model requires only 1.97% of the computational cost to achieve similar prediction accuracy as the 2D-convolutional neural network model. A training dataset comprising 1400 samples can deliver satisfactory prediction performance of the surrogate model under a certain scale of fluctuation (SOF). Moreover, selecting 700 samples from each random field optimally composes a mixed dataset for training the GPR model. The findings of this study suggest that the trained GPR model provides excellent performance for the probabilistic analysis of the tunnel convergences considering the spatially variable soil Young's modulus.

A Multi-Input Convolutional Neural Networks Model for Earthquake Precursor Detection Based on Ionospheric Total Electron Content

Earthquakes occur all around the world, causing varying degrees of damage and destruction. Earthquakes are by their very nature a sudden phenomenon and predicting them with a precise time range is difficult. Some phenomena may be indicators of physical conditions favorable for large earthquakes (e.g., the ionospheric Total Electron Content (TEC)). The TEC is an important parameter used to detect pre-earthquake changes by measuring ionospheric disturbances and space weather indices, such as the global geomagnetic index (Kp), the storm duration distribution (Dst), the sunspot number (R), the geomagnetic storm index (Ap-index), the solar wind speed (Vsw), and the solar activity index (F10.7), have also been used to detect pre-earthquake ionospheric changes. In this study, the feasibility of the 6th-day earthquake prediction by the deep neural network technique using the previous five consecutive days is investigated. For this purpose, a two-staged approach is developed. In the first stage, various preprocessing steps, namely TEC signal improvement and time-frequency representation-based TEC image construction, are performed. In the second stage, a multi-input convolutional neural network (CNN) model is designed and trained in an end-to-end fashion. This multi-input CNN model has a total of six inputs, and five of the inputs are designed as 2D and the sixth is a 1D vector. The 2D inputs to the multi-input CNN model are TEC images and the vector input is concatenated space weather indices. The network branches with the 2D inputs contain convolution, batch normalization, and Rectified Linear Unit (ReLU) activation layers, and the branch with the 1D input contains a ReLU activation layer. The ReLU activation outputs of all the branches are flattened and then concatenated. And the classification is performed via fully connected, softmax, and classification layers, respectively. In the experimental work, earthquakes with a magnitude of Mw5.0 and above that occurred in Turkey between 2012 and 2019 are used as the dataset. The TEC data were recorded by the Turkey National Permanent GNSS Network-Active (TNPGN-Active) Global Navigation Satellite System (GNSS) stations. The TEC data five days before the earthquake were marked as “precursor days” and the TEC data five days after the earthquake were marked as “normal days”. In total, 75% of the dataset is used to train the proposed method and 25% of the dataset is used for testing. The classification accuracy, sensitivity, specificity, and F1-score values are obtained for performance evaluations. The results are promising, and an 89.31% classification accuracy is obtained.

CSIT: Channel Spatial Integrated Transformer for human pose estimation

AbstractHuman keypoints detection is different from general detection tasks and requires networks that can learn visual information and anatomical constraints. Since CNN is excellent in extracting texture features of images and transformer can learn the correlation among keypoints well, many CTPNets (CNN+transformer type human pose estimation networks) have emerged. However, these networks are unconcerned with the processing of the features extracted from the CNN and naturally expand only from the channel dimension, ignoring the spatial features in the visual information that are essential for complex detection tasks like pose estimation. So the channel spatial integrated transformer for human pose estimation, termed CSIT, is proposed. The visual information are summarized as texture and spatial information, and a parallel network is used to expand the feature maps in the channel and spatial dimensions to learn texture features and spatial features respectively. In addition, anatomically constrained information is learned by keypoint embeddings. At the end of the network, the 1D vector representation method with more advanced performance and more compatible with transformer's characteristics is used to predict keypoints. Experiments show that CSIT outperforms the mainstream CTPNets on the COCO test‐dev dataset, and also show satisfactory results on the MPII dataset.

Large‐scale 3D fast Fourier transform computation on a GPU

AbstractWe propose a novel graphics processing unit (GPU) algorithm that can handle a large‐scale 3D fast Fourier transform (i.e., 3D‐FFT) problem whose data size is larger than the GPU's memory. A 1D FFT‐based 3D‐FFT computational approach is used to solve the limited device memory issue. Moreover, to reduce the communication overhead between the CPU and GPU, we propose a 3D data‐transposition method that converts the target 1D vector into a contiguous memory layout and improves data transfer efficiency. The transposed data are communicated between the host and device memories efficiently through the pinned buffer and multiple streams. We apply our method to various large‐scale benchmarks and compare its performance with the state‐of‐the‐art multicore CPU FFT library (i.e., fastest Fourier transform in the West [FFTW]) and a prior GPU‐based 3D‐FFT algorithm. Our method achieves a higher performance (up to 2.89 times) than FFTW; it yields more performance gaps as the data size increases. The performance of the prior GPU algorithm decreases considerably in massive‐scale problems, whereas our method's performance is stable.

A novel emotion recognition method based on 1D-DenseNet

Emotion recognition based on EEG (electroencephalogram) is one of the keys to improve communication between doctors and patients, which has attracted much more attention in recent years. While the traditional algorithms are generally based on using the original EEG sequence signal as input, they neglect the bad influence of noise that is difficult to remove and the great importance of shallow features for the recognition process. As a result, there is a difficulty in recognizing and analyzing emotions, as well as a stability error in traditional algorithms. To solve this problem, in this paper, a new method of EEG emotion recognition based on 1D-DenseNet is proposed. Firstly, we extract the band energy and sample entropy of EEG signal to form a 1D vector instead of the original sequence signal to reduce noise interference. Secondly, we construct a 1D-Densenet model, which takes the above-mentioned 1D vector as the input, and then connects the shallow manual features of the input layer and the output of each convolution layer as the input of the next convolution layer. This model increases the influence proportion of shallow features and has good performance. To verify the effectiveness of this method, the MAHNOB-HCI and DEAP datasets are used for analysis and the average accuracy of emotion recognition reaches 90.02% and 93.51% respectively. To compare with the current research results, the new method proposed in this paper has better classification effect. Simple preprocessing and high recognition accuracy make it easy to be applied to real medical research.

PointMapNet: Point Cloud Feature Map Network for 3D Human Action Recognition

3D human action recognition is crucial in broad industrial application scenarios such as robotics, video surveillance, autonomous driving, or intellectual education, etc. In this paper, we present a new point cloud sequence network called PointMapNet for 3D human action recognition. In PointMapNet, two point cloud feature maps symmetrical to depth feature maps are proposed to summarize appearance and motion representations from point cloud sequences. Specifically, we first convert the point cloud frames to virtual action frames using static point cloud techniques. The virtual action frame is a 1D vector used to characterize the structural details in the point cloud frame. Then, inspired by feature map-based human action recognition on depth sequences, two point cloud feature maps are symmetrically constructed to recognize human action from the point cloud sequence, i.e., Point Cloud Appearance Map (PCAM) and Point Cloud Motion Map (PCMM). To construct PCAM, an MLP-like network architecture is designed and used to capture the spatio-temporal appearance feature of the human action in a virtual action sequence. To construct PCMM, the MLP-like network architecture is used to capture the motion feature of the human action in a virtual action difference sequence. Finally, the two point cloud feature map descriptors are concatenated and fed to a fully connected classifier for human action recognition. In order to evaluate the performance of the proposed approach, extensive experiments are conducted. The proposed method achieves impressive results on three benchmark datasets, namely NTU RGB+D 60 (89.4% cross-subject and 96.7% cross-view), UTD-MHAD (91.61%), and MSR Action3D (91.91%). The experimental results outperform existing state-of-the-art point cloud sequence classification networks, demonstrating the effectiveness of our method.

CNN parameter design based on fault signal analysis and its application in bearing fault diagnosis

As a representative deep learning network, Convolutional Neural Network (CNN) has been extensively used in bearing fault diagnosis and many good results have been reported. In Prognostics and Health Management (PHM) field, the CNN’s input size is usually designed as a 1D vector or 2D square matrix, and the convolution kernel size is also defined as a square shape like 3 × 3 and 5 × 5, which are directly adopted from the image recognition. Though satisfying results can be obtained, CNN with such parameter specifications is not optimal and efficient. To this end, this paper elaborated the physical characteristics of bearing acceleration signals to guide the CNN design. First, the fault period under different fault types and shaft rotation frequency were used to determine the size of CNN’s input. Next, an exponential function was involved in fitting the envelope of decaying acceleration signal during each fault period, and signal length within different decaying ratios was used to define the CNN’s kernel size. Finally, the designed CNN was validated with the Case Western Reserve University bearing dataset and Paderborn University bearing dataset. Results confirm that the physics-guided CNN (PGCNN) with rectangular input shape and rectangular convolution kernel works better than the baseline CNN with higher accuracy and smaller uncertainty. The feasibility of designing CNN parameters with physics-guided rules derived from bearing fault signal analysis has also been verified.

Stacking ensemble learning models for daily runoff prediction using 1D and 2D CNNs

In recent years, applications of convolutional neural networks (CNNs) to runoff prediction have received some attention due to their excellent feature extraction capabilities. However, existing studies are still limited since merely either 1D or 2D CNNs are developed to predict runoff. In this study, a stacking ensemble learning model for daily runoff prediction based on different types of 1D and 2D CNNs is proposed and applied to the Quinebaug River Basin, Connecticut, USA. The structure of the CNN models is developed with reference to the classic LeNet5 network. Especially, the predictors are reconstructed into 1D vectors and 2D matrices with 10-, 20- and 30-day time steps. Totally 18 member models are constructed through selecting 3 representative 1D and 2D CNN models with 3 time steps. The simple average method (SAM) is used to integrate different CNN member models. The results show that the performance of the same-type SAM based on either 1D or 2D CNNs improves because it can counteract the effects of both positive and negative predicted values by the member models to some extent. Furthermore, the mixed-type SAM models based on both 1D and 2D CNN member models can further improve the prediction accuracy. The optimal model SAM15 consists of two 1D CNNs and four 2D CNNs. Compared with the optimal CNN member models, SAM15 reduces the validation RMSE by about 13% and improves the validation R and NSE by about 3% and 7%, respectively. This study highlights that the proposed stacking ensemble learning model can improve the daily runoff prediction accuracy through integration of the nonlinear fitting ability of 1D and 2D CNN member models.

Unsupervised deep learning for 3D interpolation of highly incomplete data

We propose to denoise and reconstruct the 3D seismic data simultaneously using an unsupervised deep learning (DL) framework, which does not require any prior information about the seismic data and is free of labels. We use an iterative process to reconstruct the 3D highly incomplete seismic data. For each iteration, we use the DL framework to denoise the 3D seismic data and initially reconstruct the missing traces. Then, the projection onto convex sets (POCS) algorithm is used for further enhancement of the seismic data reconstruction. The output of the POCS is considered as the input for the DL network in the next iteration. We use a patching technique to extract 3D seismic patches. Because the proposed DL network consists of several fully connected layers, each extracted patch needs to be converted to a 1D vector. In addition, we use an attention mechanism to enhance the learning capability of the proposed DL network. We evaluate the performance of the proposed framework using several synthetic and field examples and find that the proposed method outperforms all benchmark methods.

Deep 1D Landmark Representation Learning for Space Target Pose Estimation

Monocular vision-based pose estimation for known uncooperative space targets plays an increasingly important role in on-orbit operations. The existing state-of-the-art methods of space target pose estimation build the 2D-3D correspondences to recover the space target pose, where space target landmark regression is a key component of the methods. The 2D heatmap representation is the dominant descriptor in landmark regression. However, its quantization error grows dramatically under low-resolution input conditions, and extra post-processing is usually needed to compute the accurate 2D pixel coordinates of landmarks from heatmaps. To overcome the aforementioned problems, we propose a novel 1D landmark representation that encodes the horizontal and vertical pixel coordinates of a landmark as two independent 1D vectors. Furthermore, we also propose a space target landmark regression network to regress the locations of landmarks in the image using 1D landmark representations. Comprehensive experiments conducted on the SPEED dataset show that the proposed 1D landmark representation helps the proposed space target landmark regression network outperform existing state-of-the-art methods at various input resolutions, especially at low resolutions. Based on the 2D landmarks predicted by the proposed space target landmark regression network, the error of space target pose estimation is also smaller than existing state-of-the-art methods under all input resolution conditions.

Linking co-expression modules with phenotypes.

The method for quantifying the association between co-expression module and clinical trait of interest requires application of dimensionality reduction to summaries modules as one dimensional (1D) vector. However, these methods are often linked with information loss. The amount of information lost depends upon the percentage of variance captured by the reduced 1D vector. Therefore, it is of interest to describe a method using analysis of rank (AOR) to assess the association between module and clinical trait of interest. This method works with clinical traits represented as binary class labels and can be adopted for clinical traits measured in continuous scale by dividing samples in two groups around median value. Application of the AOR method on test data for muscle gene expression profiles identifies modules significantly associated with diabetes status.

A Deep Convolutional Neural Network for Prediction of Peptide Collision Cross Sections in Ion Mobility Spectrometry.

Most frequently, the identification of peptides in mass spectrometry-based proteomics is carried out using high-resolution tandem mass spectrometry. In order to increase the accuracy of analysis, additional information on the peptides such as chromatographic retention time and collision cross section in ion mobility spectrometry can be used. An accurate prediction of the collision cross section values allows erroneous candidates to be rejected using a comparison of the observed values and the predictions based on the amino acids sequence. Recently, a massive high-quality data set of peptide collision cross sections was released. This opens up an opportunity to apply the most sophisticated deep learning techniques for this task. Previously, it was shown that a recurrent neural network allows for predicting these values accurately. In this work, we present a deep convolutional neural network that enables us to predict these values more accurately compared with previous studies. We use a neural network with complex architecture that contains both convolutional and fully connected layers and comprehensive methods of converting a peptide to multi-channel 1D spatial data and vector. The source code and pre-trained model are available online.

Riemannian proximal stochastic gradient descent for sparse 2DPCA

Sparse two-dimensional principal component analysis (2DPCA) is an effective dimensionality reduction and data processing technique, which has been successfully applied in face recognition. Sparse 2DPCA is based on 2D image matrices instead of 1D vectors so it can well exploit the structural information of image pixels. The sparse 2DPCA model is regarded as a non-convex and non-smooth composite optimization problem. The manifold proximal gradient algorithm (ManPG) that is efficient and robust can globally converge to an optimal point in solving this kind of problem on the Stiefel manifold, such as sparse PCA. ManPG uses the proximal gradient method for the non-smooth term of this problem, which yields a convex subproblem on the tangent space. By solving this subproblem, a descent direction on the tangent space is obtained. Then a new update point on the manifold is obtained via retraction. However, in solving large-scale problems, the computational complexity of ManPG is quite high since ManPG requires calculating the full gradient in each iteration. In order to reduce the computational complexity of each iteration, a more computationally efficient Riemannian proximal stochastic gradient descent algorithm (RPSGD) is proposed, which utilizes the stochastic gradient instead of the full gradient in ManPG. We prove that the iteration complexity of the proposed RPSGD is O(1/ϵ4) while it maintains the sublinear convergence for the large-scale sparse 2DPCA. The experimental results on face recognition and handwritten digits classification data sets show RPSGD has the advantages of small-time overhead, high test accuracy, good reconstructed image quality, and relatively sparse solution in solving the large-scale sparse 2DPCA.

Strip-Map SAR Image Formulation Based on the Modified Alternating Split Bregman Method

Conventional compressive sensing (CS)-based imaging methods allow images to be reconstructed from a small amount of data, while they suffer from high computational burden even for a moderate scene. To address this problem, this paper presents a novel two-dimensional (2D) CS imaging algorithm for strip-map synthetic aperture radars (SARs) with zero squint angle. By introducing a 2D separable formulation to model the physical procedure of the SAR imaging, we separate the large measurement matrix into two small ones, and then the induced algorithm can deal with 2D signal directly instead of converting it into 1D vector. As a result, the computational load can be reduced significantly. Furthermore, thanks to its superior performance in maintaining contour information, the gradient space of the SAR image is exploited and the total variation (TV) constraint is incorporated to improve resolution performance. Due to the non-differentiable property of the TV regularizer, it is difficult to directly solve the induced TV regularization problem. To overcome this problem, an improved split Bregman method is presented by formulating the TV minimization problem into a sequence of unconstrained optimization problem and Bregman updates. It yields an accurate and simple solution. Finally, the synthesis and real experiment results demonstrate that the proposed algorithm remains competitive in terms of high resolution and high computational efficiency.

Using ATR-FTIR spectra and convolutional neural networks for characterizing mixed plastic waste

We present a convolutional neural network (CNN) framework for classifying different types of plastic materials that are commonly found in mixed plastic waste (MPW) streams. The CNN framework uses experimental ATR-FTIR (attenuated total reflection-Fourier transform infrared spectroscopy) spectra to classify ten different plastic types. An important aspect of this type of spectral data is that it can be collected in real-time; as such, this approach provides an avenue for enabling the high-throughput characterization of MPW. The proposed CNN architecture (which we call PlasticNet) uses a Gramian angular representation of the spectra. We show that this 2-dimensional (2D) matrix representation highlights correlations between different frequencies (wavenumber) and leads to significant improvements in classification accuracy, compared to the direct use of spectra (a 1D vector representation). We also demonstrate that PlasticNet can reach an overall classification accuracy of over 87% and can classify certain plastics with 100% accuracy. Our framework also uses saliency maps to analyze spectral features that are most informative.