Convolution Research Articles

A novel remaining useful life prediction based on transfer hybrid deep neural network under variable working conditions

Deep learning-based methods for remaining useful life prediction (RUL) usually require the precondition that the training and test data obey the same distribution. In engineering applications, mechanical equipment is frequently under different working conditions, which can lead to significant differences in the distribution of collected data and difficulties in obtaining labels. This paper proposed a novel RUL prediction method based on transfer hybrid deep neural network to solve the above problems. Firstly, a degradation feature extraction strategy and a clustering hybrid feature screening strategy are proposed to enrich the information content of degradation features and obtain manual features with significant degradation trends. Then, a multi-stage shrinkage attention temporal convolution network is used to adaptively extract strongly expressive and information-rich deep features from the raw data. Next, a bidirectional convolutional gated recurrent unit based on bidirectional learning and convolutional operations is designed to achieve the fusion of manual and deep features and improve the quality of degradation features. Finally, the unsupervised domain adaptation strategy is used to reduce the differences in the distribution of degradation features between training and test data and to achieve feature alignment. This paper validates the effectiveness of the proposed method on six transfer tasks. The experimental results show that the RUL prediction effectiveness of the proposed method is better than other methods.

MRNet: rolling bearing fault diagnosis in noisy environment based on multi-scale residual convolutional network

Abstract Vibration signal collection of rolling bearings in the complex working environment often suffers from significant noise interference, rendering traditional fault diagnosis methods ineffective. To address this challenge, we propose a multi-scale residual convolutional network (MRNet) for diagnosing rolling bearing faults in noisy environments. The MRNet model features multiple convolution branches, each of which utilizes kernels with different sizes to capture fault information at different scales, so this multi-scale framework excels at extracting both local and global information from raw fault vibration signals, enhancing fault recognition accuracy. Additionally, we introduce residual blocks to maintain global information during the convolution operations, preventing useful feature information loss. To further improve global feature extraction capability of the network model, a lightweight Transformer module is developed and incorporated, compensating for some global information that the network’s front-end might fail to capture. The effectiveness of MRNet is validated by using two publicly available rolling bearing fault datasets and our own experiment dataset. The verification results indicate that MRNet outperforms other comparative models, particularly for complex fault diagnosis in noisy environments.

Exploring high-order correlation for hyperspectral image denoising with hypergraph convolutional network

High-order correlation is an important property of hyperspectral images (HSIs) and has been widely investigated in model-based HSI denoising. However, the existing deep learning-based HSI denoising approaches have not fully utilized the high-order correlation. Hypergraph convolutional networks have shown great potential in capturing the high-order correlation. Therefore, in this paper, we propose a novel HSI denoising method by employing hypergraph convolution to characterize the high-order correlation at the patch level. Specifically, our framework is a symmetrically skip-connected 3D encoder–decoder architecture, which enhances the extraction and utilization of local features. Furthermore, to integrate competently the hypergraph convolutional modules into the 3D framework, we devise a dimensional transformation module that facilitates the fusion of 3D convolution and hypergraph convolution. Notably, in the hypergraph convolution operation, we use a data-driven technique to acquire the incidence matrix of a hypergraph, efficiently constructing the HSI into a high-order structure. Our proposed method excels in HSI denoising performance compared to state-of-the-art approaches, evidenced by extensive experiments on synthetic and real-world noisy HSIs.

A combined neural ODE-Bayesian optimization approach to resolve dynamics and estimate parameters for a modified SIR model with immune memory

We propose a novel hybrid approach that integrates Neural Ordinary Differential Equations (NODEs) with Bayesian optimization to address the dynamics and parameter estimation of a modified time-delay-type Susceptible-Infected-Removed (SIR) model incorporating immune memory. This approach leverages a neural network to produce continuous multi-wave infection profiles by learning from both data and the model. The time-delay component of the SIR model, expressed through a convolutional integral, results in an integro-differential equation. To resolve these dynamics, we extend the NODE framework, employing a Runge-Kutta solver, to handle the challenging convolution integral, enabling us to fit the data and learn the parameters and dynamics of the model. Additionally, through Bayesian optimization, we enhance prediction accuracy while focusing on long-term dynamics. Our model, applied to COVID-19 data from Mexico, South Africa, and South Korea, effectively learns critical time-dependent parameters and provides accurate short- and long-term predictions. This combined methodology allows for early prediction of infection peaks, offering significant lead time for public health responses.

Temperature compensation method for impedance signals of bolt loosening under temperature varying conditions using TransUnet

Abstract Piezoelectric transducer based electro-mechanical impedance technology is an effective and reliable detection method for bolt structure loosening under external load. However, temperature change usually causes some non-loosening factors to change the impedance spectrum of bolt structure. In order to reduce the misjudgment of loosening damage caused by temperature change, it is necessary to construct a temperature compensation model for impedance spectrum. In this paper, the convolutional structure in U-net and Transformer are effectively combined to form a TransUnet deep neural network structure suitable for input and output of one-dimensional data. Using impedance data between 10 °C and 50 °C and temperature as input to the network. After convolution operation, the convolutional block attention module is embedded in the U-net to optimize the encoder transmission characteristics and enhance the performance of the skip connection in the traditional U-shaped structure. The temperature compensation rate (TCR) is defined to measure the effect of temperature compensation. Then the lightweight convolutional neural network structure is used to recognize the bolt loosening damage of the compensated impedance signal. The generalization ability of the TransUnet was tested using impedance data that is not in the training dataset. The results show that the TransUnet proposed in the paper can realize the temperature compensation of multi-peak impedance signals. The TCR and recognition accuracy of bolt loose damage reaches reach 0.003 and 95.7%, respectively, which is 11% higher than that without temperature compensation. At 60 °C, the TCR and the identification accuracy of loosening damage can still reach 0.0055 and 90.2%, respectively, which show that the TransUnet has strong generalization ability.

Vibration displacement measurement of bridge structural models using image super-resolution reconstruction and visual object detection network

Abstract Visual vibration measurement has emerged in the field of structural health monitoring in recent years, but it still has some shortcomings in terms of resolution, recognition rate and real-time performance. Considering the three aspects of recovering high-frequency image details, improving the compactness of the target bounding box, and reducing the computational time, we use the constructed image super-resolution reconstruction model and target detection model to measure the vibration displacement of the bridge structural model. First, we integrate the Transformer module into the Unet network with a simple structure. The Swin and Global Transformer Unet (SGTU) module constructed in this form can reduce the computational cost while reconstructing the large-resolution feature map target, and it can sharply edge information of the vibration target. We use the framework of the YOLOv5 algorithm as the backbone, and use the GhostBottleneck (GB) module to reduce the time for convolution operations to generate similar features. In addition, the proposed DWCBottleneck (DWCB) fusion module is also able to achieve high-level semantic fusion and network depth expansion with minimal computational cost. Finally, the center point offset of the bounding box predicted by the model can be used to obtain the displacement offset of the object in the image sequence. The position information of the target in the first frame image is used as the reference frame for calculating the offset, and the vibration displacement of the flexible structure in the image coordinate system is obtained by calculating the deviation of the displacement between the remaining frames and the first frame. We perform qualitative and quantitative comparisons in three aspects: video super-resolution reconstruction, visual detection robustness, and sensor vibration measurement displacement using a homemade vibration image dataset. The time-frequency domain displacement curves regressed by the visual vibration measurement algorithm are compared with the curves acquired after accelerometer acquisition, indicating the necessity of super-resolution reconstruction in visual vibration measurement.

Efficient Convolutional Neural Networks Utilizing Fine-Grained Fast Fourier Transforms

Convolutional Neural Networks (CNNs) are among the most prevalent deep learning techniques employed across various domains. The computational complexity of CNNs is largely attributed to the convolution operations. These operations are computationally demanding and significantly impact overall model performance. Traditional CNN implementations convert convolutions into matrix operations via the im2col (image to column) technique, facilitating parallelization through advanced BLAS libraries. This study identifies and investigates a significant yet intricate pattern of data redundancy within the matrix-based representation of convolutions, a pattern that, while complex, presents opportunities for optimization. Through meticulous analysis of the redundancy inherent in the im2col approach, this paper introduces a mathematically succinct matrix representation for convolution, leading to the development of an optimized FFT-based convolution with finer FFT granularity. Benchmarking demonstrates that our approach achieves an average speedup of 14 times and a maximum speedup of 17 times compared to the regular FFT convolution. Similarly, it outperforms the Im2col+GEMM approach from NVIDIA’s cuDNN library, achieving an average speedup of three times and a maximum speedup of five times. Our FineGrained FFT convolution approach, when integrated into Caffe, a widely used deep learning framework, leads to significant performance gains. Evaluations using synthetic CNNs designed for real-world applications show an average speedup of 1.67 times. Furthermore, a modified VGG network variant achieves a speedup of 1.25 times.

InfoFlowNet: A multi-head attention-based self-supervised learning model with surrogate approach for uncovering brain effective connectivity

Deciphering brain network topology can enhance the depth of neuroscientific knowledge and facilitate the development of neural engineering methods. Effective connectivity, which gauges the directional influences among brain regions, is pivotal for these studies. This research introduces InfoFlowNet, a novel self-supervised learning model designed to infer causal relationships in electroencephalogram (EEG) data, aiming at capturing the complex information flow among brain regions. Specifically, InfoFlowNet employs convolution operations and multi-head self-attention mechanisms with a masking feature, which ignores self-connections to focus on inter-regional dynamics. Additionally, a new causal magnitude estimation is introduced by assessing the impact of shuffled surrogate data on signal prediction to quantify causal influence. Experiments using synthetic data and real EEG datasets demonstrate that InfoFlowNet effectively uncovers time-varying causal relationships. Compared with the Granger causality model (GCM) and the temporal causal discovery framework (TCDF), InfoFlowNet shows superior sensitivity in detecting significant causal edges. For instance, in a psychomotor vigilance task, InfoFlowNet identified 8 out of 16 significant causal edges, significantly outperforming GCM and TCDF. Furthermore, InfoFlowNet successfully passes the whiteness test on the model's residuals, and advanced analyses illustrate the advantages of utilizing multi-head attention and masking mechanisms to capture effective connectivity. In sum, InfoFlowNet represents a significant advancement in the analysis of effective connectivity, offering a deeper understanding of brain network dynamics. The model's ability to discern complex causal interactions supports its potential application in broader neuroscientific research and applications.

Dress Code Monitoring Method in Industrial Scene Based on Improved YOLOv8n and DeepSORT.

Deep learning-based object detection has become a powerful tool in dress code monitoring. However, even state-of-the-art detection models inevitably suffer from false alarms or missed detections, especially when handling small targets such as hats and masks. To overcome these limitations, this paper proposes a novel method for dress code monitoring using an improved YOLOv8n model, the DeepSORT tracking, and a new dress code judgment criterion. We improve the YOLOv8n model through three means: (1) a new neck structure named FPN-PAN-FPN (FPF) is introduced to enhance the model's feature fusion capability, (2) Receptive-Field Attention convolutional operation (RFAConv) is utilized to better capture the difference in information brought by different positions, and a (3) Focused Linear Attention (FLatten) mechanism is added to expand the model's receptive field. This improved YOLOv8n model increases mAP while reducing model size. Next, DeepSORT is integrated to obtain instance information across multi-frames. Finally, we adopt a new judgment criterion to conduct real-scene dress code monitoring. The experimental results show that our method effectively identifies instances of dress violations, reduces false alarms, and improves accuracy.

RSE-YOLOv8: An Algorithm for Underwater Biological Target Detection.

Underwater target detection is of great significance in underwater ecological assessment and resource development. To better protect the environment and optimize the development of underwater resources, we propose a new underwater target detection model with several innovations based on the YOLOv8 framework. Firstly, the SAConv convolutional operation is introduced to redesign C2f, the core module of YOLOv8, to enhance the network's feature extraction capability for targets of different scales. Secondly, we propose the RFESEConv convolution module instead of the conventional convolution operation in neural networks to cope with the degradation of image channel information in underwater images caused by light refraction and reflection. Finally, we propose an ESPPF module to further enhance the model's multi-scale feature extraction efficiency. Simultaneously, the overall parameters of the model are reduced. Compared to the baseline model, the proposed one demonstrates superior advantages when deployed on underwater devices with limited computational resources. The experimental results show that we have achieved significant detection accuracy on the underwater dataset, with an mAP@50 of 78% and an mAP@50:95 of 43.4%. Both indicators are 2.1% higher compared to the baseline models. Additionally, the proposed model demonstrates superior performance on other datasets, showcasing its strong generalization capability and robustness. This research provides new ideas and methods for underwater target detection and holds important application value.

A parallel neural networks for emotion recognition based on EEG signals

Our study proposes a novel Parallel Temporal–Spatial-Frequency Neural Network (PTSFNN) for emotion recognition. The network processes EEG signals in the time, frequency, and spatial domains simultaneously to extract discriminative features. Despite its relatively simple architecture, the proposed model achieves superior performance. Specifically, PTSFNN first applies wavelet transform to the raw EEG signals and then reconstructs the coefficients based on frequency hierarchy, thereby achieving frequency decomposition. Subsequently, the core part of the network performs three independent parallel convolution operations on the decomposed signals, including a novel graph convolutional network. Finally, an attention mechanism-based post-processing operation is designed to effectively enhance feature representation. The features obtained from the three modules are concatenated for classification, with the cross-entropy loss function being adopted. To evaluate the model’s performance, extensive experiments are conducted on the SEED and SEED-IV public datasets. The experimental results demonstrate that PTSFNN achieves excellent performance in emotion recognition tasks, with classification accuracies of 87.63% and 74.96%, respectively. Comparative experiments with previous state-of-the-art methods confirm the superiority of our proposed model, which can efficiently extract emotion information from EEG signals.

FM‐YOLOv8:Lightweight gesture recognition algorithm

AbstractIn practical production applications, the efficiency and success rate of gesture recognition directly affect the user experience and work efficiency. However, the existing gesture recognition models have the problem of a large number of model parameters and high computational complexity, which makes them unable to meet the needs of end‐to‐end industrial deployment. To solve these problems, this article proposes a gesture recognition model based on YOLOv8. First, FasterNet is adopted as the backbone of YOLOv8, which significantlydecrease the number of parameters and made the model more lightweight. By reducing the number of parameters, the computational complexity of the model can be reduced while maintaining the performance of the model, and the operation efficiency of the model can be improved. Second, recombination convolution ScConv is introduced to replace common convolution operations to further improve the model's efficiency. Recombination convolution can reduce the computation and make up for the loss of precision to some extent. Finally, the MDPIoU loss function is used to optimize target location and prediction, to improve the accuracy of the model. The MDPIoU loss function can better deal with the problem of target boundary frame positioning and prediction so that the model can locate and predict gestures more accurately in gesture recognition tasks. Experiments on a data set containing 10 types of gestures show that the number of parameters and floating point calculations of the improved network model are reduced by 45% and 42.7%, respectively, while the accuracy is unchanged. The improved model can be deployed on edge terminals, providing efficient and accurate gesture recognition.

Multi-sensor temporal-spatial graph network fusion empirical mode decomposition convolution for machine fault diagnosis

Multi-sensor time-series data at different locations contains not only temporal correlation information but also spatial correlation information which is treasure for machine fault diagnosis. Existing graph construction methods mainly apply different data analysis methods to connect nodes and edges. Few works, however, consider the location of the sensor itself and temporal correlation information of multi-sensor time-series data. To mine the relationship between spatial information and temporal information, the multi-sensor temporal-spatial graph is constructed in this paper. Hereinto, the different data points of multi-sensor are severed as different nodes which represents the spatial feature information. The temporal information is contained between different nodes of the same sensor. Moreover, an empirical mode decomposition graph convolution network (EGCN) is proposed to extract the feature. Specifically, the traditional graph convolution operator is changed to empirical mode decomposition which can decompose the input features into multiple intrinsic modal features to achieve adaptive feature extraction and improve the representation capability of the network. Finally, the different fault types can be classified by fully connected layers. Experiments from different test rigs demonstrate that the proposed method achieves a diagnostic accuracy exceeding 99 % under limited fault samples.

Framework for UAV-based river flow velocity determination employing optical recognition

The determination of river velocity is important for hydromorphological analyses and river monitoring systems. Indirect measurements of river velocity using videos recorded by unmanned aerial vehicles (UAV) allow fast and cost-effective processing of information about the river stretch. This paper presents a method for computing flow velocity of the river surface using deep supervised model RAFT to determine the optical flow in combination with image pre-processing by convolutional operations. Moreover, the windiness coefficients and variance score were proposed to evaluate reliability of the collected data and the obtained results of optical flow detection. Various image pre-processing techniques were applied, namely the selection of the analysed area and the number of convolutional operations to select the one with the lowest variance score. This score represents the consistency of the river flow velocity during the video and can be used to filter out unreliable results. The numerical experiments were performed using the videos and directly measured velocity values of 4 shallow rivers in Lithuania collected during the field surveys. The optical velocity estimation method showed good correspondence to the directly measured values for the velocity range from 0 m/s to 0.8 m/s in the points with low variance score up to 0.192 that represents the first quartile of the variance. The optical flow method tends to underestimate the velocity up to 0.5 m/s for the quartiles with the higher variance scores. It was shown that in most cases the lowest variance score value was obtained using pre-processing techniques without convolutional operations. However, the need to analyse various pre-processing techniques arises from the different origin of the objects moving on the river surface.

Reconstructing hyperspectral images of textiles from a single RGB image utilizing the multihead self-attention mechanism

Hyperspectral images possess abundant information and play a pivotal role in enhancing the accuracy of color difference detection in textiles. However, traditional hyperspectral imaging methods necessitate costly equipment and intricate operational procedures. A novel deep learning model based on a multihead attention mechanism was proposed in this article to facilitate the extensive application of hyperspectral imaging technology in textile quality inspection. This model enabled the reconstruction of the hyperspectral information of plain weave textiles from a single RGB image. In this model, encoder-decoder architecture and pyramid pooling convolutional operations were employed to integrate multiscale features of plain weave cotton-linen textiles. This could capture details and contextual information in textile images more precisely, enhancing the accuracy of hyperspectral image reconstruction. Simultaneously, an attention mechanism was introduced to increase the model’s receptive field and improve its focus on key regions in the input image and feature maps. This resulted in a reduced weighting of redundant information during network learning, leading to an improved feature extraction capability of the network. Through these methods, successful reconstructions of plain weave textiles hyperspectral information from a single RGB image was achieved. Quantitative and qualitative tests were conducted on two datasets, namely, the NTIRE 2020 dataset and a self-made textile dataset, to evaluate the performance of the proposed method. The approach proposed in this article exhibited promising results on both datasets. Specifically, the reconstructed textile hyperspectral images achieved a root mean square error of 0.0344, a peak signal-to-noise ratio of 29.945, a spectral angle mapper of 3.753, and a structural similarity index measure of 0.955 on the textile dataset. In the reconstructed hyperspectral colorimetric experiment, the maximum value of average color difference was 2.641. These results demonstrate that the method can meet the requirements for textile color measurement applications.

NKDFF-CNN: A convolutional neural network with narrow kernel and dual-view feature fusion for multitype gesture recognition based on sEMG

Deep learning algorithms have been widely applied to gesture recognition based on multi-channel surface electromyography (sEMG). However, the limitations in feature extraction capabilities of existing algorithms have restricted the performance of multitype gesture recognition. To address this challenge, we propose a novel sEMG-based gesture recognition algorithm, namely, Narrow Kernel and Dual-view Feature Fusion Convolutional Neural Network (NKDFF-CNN). Firstly, to overcome the issue of traditional square kernel convolution operation, which loses channel independence features, we employ the narrow kernel convolution in the model to learn time-related features in each independent channel of sEMG, resulting in obtaining representative correlation information between specific muscles and gestures. Then, the dual-view structure is used to capture both shallow and deep features, which are fused at the decision level. Thus, the multi-dimensional feature information is extracted. The NKDFF-CNN is further extended to ACCNKDFF-CNN by introducing acceleration signals for multimodal feature integration. Experimental validation on the NinaPro DB2 dataset demonstrates the superior classification performance of NKDFF-CNN, achieving 88.03 % accuracy for 49 hand gestures, outperforming other state-of-the-art MSFF-net. In addition, the ACCNKDFF-CNN model with multimodal feature information significantly improved the accuracy to 95.25 %. We also validated the proposed NKDFF-CNN on NinaPro DB3 with the disabled subjects and the NinaPro DB4 with healthy subjects. The results showcased that the NKDFF-CNN achieved advanced accuracies of 70.58 % and 85.91 % for the multitype hand gestures classification, respectively, showing the high generalization ability of the proposed model. As a consequence, the proposed NKDFF-CNN method achieved superior recognition performance in both accuracy and generality compared to other advanced models. Thus, it provides a reliable algorithm for research in fields such as rehabilitative medicine.

AI-Enabled Sensor Fusion of Time-of-Flight Imaging and mmWave for Concealed Metal Detection.

In the field of detection and ranging, multiple complementary sensing modalities may be used to enrich information obtained from a dynamic scene. One application of this sensor fusion is in public security and surveillance, where efficacy and privacy protection measures must be continually evaluated. We present a novel deployment of sensor fusion for the discrete detection of concealed metal objects on persons whilst preserving their privacy. This is achieved by coupling off-the-shelf mmWave radar and depth camera technology with a novel neural network architecture that processes radar signals using convolutional Long Short-Term Memory (LSTM) blocks and depth signals using convolutional operations. The combined latent features are then magnified using deep feature magnification to reveal cross-modality dependencies in the data. We further propose a decoder, based on the feature extraction and embedding block, to learn an efficient upsampling of the latent space to locate the concealed object in the spatial domain through radar feature guidance. We demonstrate the ability to detect the presence and infer the 3D location of concealed metal objects. We achieve accuracies of up to 95% using a technique that is robust to multiple persons. This work provides a demonstration of the potential for cost-effective and portable sensor fusion with strong opportunities for further development.

Convolution equation and operators on the euclidean motion group

Let $G = \mathbb{R}^2\rtimes SO(2)$ be the Euclidean motion group, let g be the Lie algebra of G and let U(g) be the universal enveloping algebra of g. Then U(g) is an infinite dimensional, linear associative and non-commutative algebra consisting of invariant differential operators on G. The Dirac measure on G is represented by $\delta_G$, while the convolution product of functions or measures on G is represented by $\ast$. Among other notable results, it is demonstrated that for each u in U(g), there is a distribution E on G such that the convolution equation $u\ast E = \delta_G$ is solved by method of convolution. Further more, it is established that the (convolution) operator $A^\prime : C^\infty_c(G)\rightarrow C^\infty(G),$, which is defined as $A^\prime f = f\ast T^n\delta(t)$ extends to a bounded linear operator on $L^2(G)$, for $f\in C^\infty_c(G)$, the space of infinitely differentiable functions on G with compact support. Furthermore, we demonstrate that the left convolution operator LT denoted as $L_Tf = T\ast f$ commutes with left translation, for $T\in D^\prime(G)$.

High‐order multilayer attention fusion network for 3D object detection

AbstractThree‐dimensional object detection based on the fusion of 2D image data and 3D point clouds has become a research hotspot in the field of 3D scene understanding. However, different sensor data have discrepancies in spatial position, scale, and alignment, which severely impact detection performance. Inappropriate fusion methods can lead to the loss and interference of valuable information. Therefore, we propose the High‐Order Multi‐Level Attention Fusion Network (HMAF‐Net), which takes camera images and voxelized point clouds as inputs for 3D object detection. To enhance the expressive power between different modality features, we introduce a high‐order feature fusion module that performs multi‐level convolution operations on the element‐wise summed features. By incorporating filtering and non‐linear activation, we extract deep semantic information from the fused multi‐modal features. To maximize the effectiveness of the fused salient feature information, we introduce an attention mechanism that dynamically evaluates the importance of pooled features at each level, enabling adaptive weighted fusion of significant and secondary features. To validate the effectiveness of HMAF‐Net, we conduct experiments on the KITTI dataset. In the “Car,” “Pedestrian,” and “Cyclist” categories, HMAF‐Net achieves mAP performances of 81.78%, 60.09%, and 63.91%, respectively, demonstrating more stable performance compared to other multi‐modal methods. Furthermore, we further evaluate the framework's effectiveness and generalization capability through the KITTI benchmark test, and compare its performance with other published detection methods on the 3D detection benchmark and BEV detection benchmark for the “Car” category, showing excellent results. The code and model will be made available on https://github.com/baowenzhang/High‐order‐Multilayer‐Attention‐Fusion‐Network‐for‐3D‐Object‐Detection.

Species-specific model based on sequence and structural information for ubiquitination sites prediction

Ubiquitination is a common post-translational modification of proteins in eukaryotic cells, and it is also a significant method of regulating protein biological function. Computational methods for predicting ubiquitination sites can serve as a cost-effective and time-saving alternative to experimental methods. Existing computational methods often build classifiers based on protein sequence information, physical and chemical properties of amino acids, evolutionary information, and structural parameters. However, structural information about most proteins cannot be found in existing databases directly. The features of proteins differ among species, and some species have small amounts of ubiquitinated proteins. Therefore, it is necessary to develop species-specific models that can be applied to datasets with small sample sizes. To solve these problems, we propose a species-specific model (SSUbi) based on a capsule network, which integrates proteins’ sequence and structural information. In this model, the feature extraction module is composed of two sub-modules that extract multi-dimensional features from sequence and structural information respectively. In the submodule, the convolution operation is used to extract encoding dimension features, and the channel attention mechanism is used to extract feature map dimension features. After integrating the multi-dimensional features from both types of information, the species-specific capsule network further converts the features into capsule vectors and classifies species-specific ubiquitination sites. The experimental results show that SSUbi can effectively improve the prediction performance of species with small sample sizes and outperform other models.