Three-dimensional Convolution Research Articles

Alzheimer’s disease diagnosis by 3D-SEConvNeXt

Alzheimer’s disease (AD) constitutes a fatal neurodegenerative disorder and represents the most prevalent form of dementia among the elderly population. Traditional manual AD classification methods, such as clinical diagnosis, are known to be time-consuming and labor-intensive, with relatively low accuracy. Therefore, our work aims to develop a new deep learning framework to tackle this challenge. Our proposed model integrates ConvNeXt with three-dimensional (3D) convolution and incorporates a 3D Squeeze-and-Excitation (3D-SE) attention mechanism to enhance early classification of AD. The experimental data is sourced from the publicly accessible Alzheimer’s disease Neuroimaging Initiative (ADNI) database, with raw Magnetic Resonance Imaging (MRI) data preprocessed using SPM12 software. Subsequently, the preprocessed data is input into the 3D-SEConvNeXt network to perform four classification tasks: distinguishing between AD and Normal Control (NC), Mild Cognitive Impairment (MCI) and NC, AD and MCI, as well as AD, MCI, and NC. The experimental results indicate that the 3D-SEConvNeXt model consistently outperforms alternative models in terms of accuracy, achieving commendable outcomes in early AD diagnostic tasks.

Three-dimensional dynamic gesture recognition method based on convolutional neural network

With the rapid advancement of virtual reality, dynamic gesture recognition technology has become an indispensable and critical technique for users to achieve human–computer interaction in virtual environments. The recognition of dynamic gestures is a challenging task due to the high degree of freedom and the influence of individual differences and the change of gesture space. To solve the problem of low recognition accuracy of existing networks, an improved dynamic gesture recognition algorithm based on ResNeXt architecture is proposed. The algorithm employs three-dimensional convolution techniques to effectively capture the spatiotemporal features intrinsic to dynamic gestures. Additionally, to enhance the model’s focus and improve its accuracy in identifying dynamic gestures, a lightweight convolutional attention mechanism is introduced. This mechanism not only augments the model’s precision but also facilitates faster convergence during the training phase. In order to further optimize the performance of the model, a deep attention submodule is added to the convolutional attention mechanism module to strengthen the network’s capability in temporal feature extraction. Empirical evaluations on EgoGesture and NvGesture datasets show that the accuracy of the proposed model in dynamic gesture recognition reaches 95.03% and 86.21%, respectively. When operating in RGB mode, the accuracy reached 93.49% and 80.22%, respectively. These results underscore the effectiveness of the proposed algorithm in recognizing dynamic gestures with high accuracy, showcasing its potential for applications in advanced human–computer interaction systems.

Multimodal registration network with multi-scale feature-crossing.

A critical piece of information for prostate intervention and cancer treatment is provided by the complementary medical imaging modalities of ultrasound (US) and magnetic resonance imaging (MRI). Therefore, MRI-US image fusion is often required during prostate examination to provide contrast-enhanced TRUS, in which image registration is a key step in multimodal image fusion. We propose a novel multi-scale feature-crossing network for the prostate MRI-US image registration task. We designed a feature-crossing module to enhance information flow in the hidden layer by integrating intermediate features between adjacent scales. Additionally, an attention block utilizing three-dimensional convolution interacts information between channels, improving the correlation between different modal features. We used 100 cases randomly selected from The Cancer Imaging Archive (TCIA) for our experiments. A fivefold cross-validation method was applied, dividing the dataset into five subsets. Four subsets were used for training, and one for testing, repeating this process five times to ensure each subset served as the test set once. We test and evaluate our technique using fivefold cross-validation. The cross-validation trials result in a median target registration error of 2.20 mm on landmark centroids and a median Dice of 0.87 on prostate glands, both of which were better than the baseline model. In addition, the standard deviation of the dice similarity coefficient is 0.06, which suggests that the model is stable. We propose a novel multi-scale feature-crossing network for the prostate MRI-US image registration task. A random selection of 100 cases from The Cancer Imaging Archive (TCIA) was used to test and evaluate our approach using fivefold cross-validation. The experimental results showed that our method improves the registration accuracy. After registration, MRI and TURS images were more similar in structure and morphology, and the location and morphology of cancer were more accurately reflected in the images.

Water pollution classification and detection by hyperspectral imaging.

This study utilizes spectral analysis to quantify water pollutants by analyzing the images of biological oxygen demand (BOD). In this study, a total of 2545 images depicting water quality pollution were generated due to the absence of a standardized water pollution detection method. A novel snap-shot hyperspectral imaging (HSI) conversion algorithm has been developed to conduct spectral analysis on traditional RGB images. In order to demonstrate the effectiveness of the developed HSI algorithm, two distinct three-dimensional convolution neural networks (3D-CNN) are employed to train two separate datasets. One dataset is based on the HSI conversion algorithm (HSI-3DCNN), while the other dataset is the traditional RGB dataset (RGB-3DCNN). The images depicting water quality pollution were categorized into three distinct groups: Good, Normal, and Severe, based on the extent of pollution severity. A comparison was conducted between the HSI and RGB models, focusing on precision, recall, F1-score, and accuracy. The water pollution model's accuracy improved from 76% to 80% when the RGB-3DCNN was substituted with the HSI-3DCNN. The results suggest that the HSI has the capacity to enhance the effectiveness of water pollution detection compared to the RGB model.

Residual Spatiotemporal Convolutional Neural Network Based on Multisource Fusion Data for Approaching Precipitation Forecasting

Approaching precipitation forecast refers to the prediction of precipitation within a short time scale, which is usually regarded as a spatiotemporal sequence prediction problem based on radar echo maps. However, due to its reliance on single-image prediction, it lacks good capture of sudden severe convective events and physical constraints, which may lead to prediction ambiguities and issues such as false alarms and missed alarms. Therefore, this study dynamically combines meteorological elements from surface observations with upper-air reanalysis data to establish complex nonlinear relationships among meteorological variables based on multisource data. We design a Residual Spatiotemporal Convolutional Network (ResSTConvNet) specifically for this purpose. In this model, data fusion is achieved through the channel attention mechanism, which assigns weights to different channels. Feature extraction is conducted through simultaneous three-dimensional and two-dimensional convolution operations using a pure convolutional structure, allowing the learning of spatiotemporal feature information. Finally, feature fitting is accomplished through residual connections, enhancing the model’s predictive capability. Furthermore, we evaluate the performance of our model in 0–3 h forecasting. The results show that compared with baseline methods, this network exhibits significantly better performance in predicting heavy rainfall. Moreover, as the forecast lead time increases, the spatial features of the forecast results from our network are richer than those of other baseline models, leading to more accurate predictions of precipitation intensity and coverage area.

Adaptive temporal compression for reduction of computational complexity in human behavior recognition

The research on video analytics especially in the area of human behavior recognition has become increasingly popular recently. It is widely applied in virtual reality, video surveillance, and video retrieval. With the advancement of deep learning algorithms and computer hardware, the conventional two-dimensional convolution technique for training video models has been replaced by three-dimensional convolution, which enables the extraction of spatio-temporal features. Specifically, the use of 3D convolution in human behavior recognition has been the subject of growing interest. However, the increased dimensionality has led to challenges such as the dramatic increase in the number of parameters, increased time complexity, and a strong dependence on GPUs for effective spatio-temporal feature extraction. The training speed can be considerably slow without the support of powerful GPU hardware. To address these issues, this study proposes an Adaptive Time Compression (ATC) module. Functioning as an independent component, ATC can be seamlessly integrated into existing architectures and achieves data compression by eliminating redundant frames within video data. The ATC module effectively reduces GPU computing load and time complexity with negligible loss of accuracy, thereby facilitating real-time human behavior recognition.

Data-driven distributionally robust day-ahead dispatch for active distribution networks based on improved conditional generative adversarial network

With the increasing penetration of renewable energy generation (REG), strong uncertainty poses a significant challenge to the rationality of day-ahead dispatch plans in distribution systems. We propose a data-driven, distributionally robust day-ahead dispatch method for active distribution networks (ADNs) based on an improved conditional generative adversarial network (CGAN). First, an improved CWGAN-GP model is constructed based on three-dimensional convolution (Conv3D) and the Wasserstein distance to obtain the day-ahead REG scenarios, considering the correlation features of photovoltaic (PV) and wind turbines (WTs), which reduces the conservatism of scenario generation. Accordingly, the Gaussian mixture model (GMM) clustering method based on moment information was used to construct an ambiguity set. Moreover, a two-stage distributionally robust day-ahead dispatch model for an ADN is established, which realizes the fusion of the data-driven method and distributionally robust optimization (DRO) mechanism model. Finally, the validity of the data-driven DRO (DDRO) method is verified through a case analysis.

Bi-State Prediction Network Model for Mixed Traffic Congestion Prediction

Forecasting traffic congestion has become an important issue than ever before. Information provided by traffic congestion prediction methods becomes helpful to traffic management agencies to take suitable actions in due time. On the other hand, with prior information, passengers can plan their trip routes accordingly. In this paper, we tackle the problem of mixed traffic congestion prediction and develop a new Bi-State Prediction Network (BiSPNet). Two heads form BiSPNet. Each head information passed through the layers of three-dimensional convolution (Conv3D), two-dimensional convolutional long short-term memory (ConvLSTM2D), and two-dimensional convolution (Conv2D). Then the dense layer merges both head outputs. These layers capacitate the model to capture the characteristics of multiple inputs. The proposed model is validated by the ground truth data collected by the video cameras installed on the roadways in Delhi, India. Experimental results confirm that the prediction accuracy of the BiSPNet model surpasses that of several state-of-the-art benchmark models. In particular, the BiSPNet reduces mean absolute error by 53.23%, 40.95%, and 0.38%, respectively, in congestion score prediction for 5, 10, and 15-min time horizons compared to the mixed deep learning network (MDLNet). The source data and code can be accessed on GitHub at the following link: https://github.com/amanojup/ .

A deep learning-based method for the prediction of temporal lobe injury in patients with nasopharyngeal carcinoma

PurposeTo establish a deep learning-based model to predict radiotherapy-induced temporal lobe injury (TLI). Materials and methodsSpatial features of dose distribution within the temporal lobe were extracted using both the three-dimensional convolution (C3D) network and the dosiomics method. The Minimal Redundancy-Maximal-Relevance (mRMR) method was employed to rank the extracted features and select the most relevant ones. Four machine learning (ML) classifiers, including logistic regression (LR), k-nearest neighbors (kNN), support vector machines (SVM) and random forest (RF), were used to establish prediction models. Nested sampling and hyperparameter tuning methods were applied to train and validate the prediction models. For comparison, a prediction model base on the conventional D0.5cc of the temporal lobe obtained from dose volume (DV) histogram was established. The area under the receiver operating characteristic (ROC) curve (AUC) was utilized to compare the predictive performance of the different models. ResultsA total of 127 nasopharyngeal carcinoma (NPC) patients were included in the study. In the model based on C3D deep learning features, the highest AUC value of 0.843 was achieved with 5 features. For the dosiomics features model, the highest AUC value of 0.715 was attained with 1 feature. Both of these models demonstrated superior performance compared to the prediction model based on DV parameters, which yielded an AUC of 0.695. ConclusionThe prediction model utilizing C3D deep learning features outperformed models based on dosiomics features or traditional parameters in predicting the onset of TLI. This approach holds promise for predicting radiation-induced toxicities and guide individualized radiotherapy.

Efficient Data-Driven Gap Filling of Satellite Image Time Series Using Deep Neural Networks with Partial Convolutions

Abstract The abundance of gaps in satellite image time series often complicates the application of deep learning models such as convolutional neural networks for spatiotemporal modeling. Based on previous work in computer vision on image inpainting, this paper shows how three-dimensional spatiotemporal partial convolutions can be used as layers in neural networks to fill gaps in satellite image time series. To evaluate the approach, we apply a U-Net-like model on incomplete image time series of quasi-global carbon monoxide observations from the Sentinel-5 Precursor (Sentinel-5P) satellite. Prediction errors were comparable to two considered statistical approaches while computation times for predictions were up to three orders of magnitude faster, making the approach applicable to process large amounts of satellite data. Partial convolutions can be added as layers to other types of neural networks, making it relatively easy to integrate with existing deep learning models. However, the approach does not provide prediction uncertainties and further research is needed to understand and improve model transferability. The implementation of spatiotemporal partial convolutions and the U-Net-like model is available as open-source software. Significance Statement Gaps in satellite-based measurements of atmospheric variables can make the application of complex analysis methods such as deep learning approaches difficult. The purpose of this study is to present and evaluate a purely data-driven method to fill incomplete satellite image time series. The application on atmospheric carbon monoxide data suggests that the method can achieve prediction errors comparable to other approaches with much lower computation times. Results highlight that the method is promising for larger datasets but also that care must be taken to avoid extrapolation. Future studies may integrate the approach into more complex deep learning models for understanding spatiotemporal dynamics from incomplete data.

Investigating a three-dimensional convolution recognition model for acoustic emission signal analysis during uniaxial compression failure of coal

Acoustic emission predicts coal sample failure, vital for early warnings and monitoring. The study emphasizes using acoustic emission to predict coal sample failure patterns and establish a discriminative model for monitoring. It employs a lightweight 3D convolutional model combining DenseNet with Group Convolution (GC) and the "squeeze-and-excitation" (SE) module. Coal samples from Xinglongzhuang Coal Mine underwent uniaxial compression, and a subset of experiments with prominent features was selected. The resulting acoustic emission parameters were then transformed into spatiotemporal image sequences for input to the model. Successfully identifying hazardous coal damage stages, all four network structures (DenseNet, DenseNet + GC, DenseNet + SE, DenseNet + GC + SE) achieved over 98.29% predictive accuracy in verification samples. The model exhibited a recall rate of over 98.02% for high-risk samples, effectively capturing spatiotemporal information. DenseNet + GC + SE showed a probability distribution focusing on different risk levels. By integrating group convolution and SE modules, this model significantly reduced both model and time complexity while preserving precision, enhancing efficiency. It effectively discerns diverse acoustic emission features through group convolution and evaluates their significance using the SE module. Compared to the traditional C3D neural network, this model greatly reduces complexity and time requirements while improving efficiency in distinguishing between various damage stages.

The identification of minor impact collisions in a long video for detecting property damages caused by fleeing vehicles using three-dimensional convolutional neural network

Abstract A parked vehicle damaged by a hit-and-run can only be repaired at the expense of the owner, unless the fleeing vehicle is identified and the driver apprehended. Identifying the fleeing vehicle involves using a video investigation method that searches for perpetrators through CCTV footage of the crime scene. When the length of the recorded video is long, the investigation may require an extended amount of time from the investigator, resulting in an added burden on their daily work. Some commercial companies are using object recognition and tracking technology to detect hit-and-run incidents; however, detecting small movements of a vehicle during a minor collision still remains a challenge. Therefore, there is a need for a system that can detect small movement in a vehicle in a lengthy video. Automatic recognition and tracking require a sufficient amount of training dataset. However, such a dataset for hit-and-run incidents is not publicly available. One of the reasons behind this scarcity is that it may violate personal information protection acts. On the other hand, instead of using real accident videos, we could use actors to simulate such accident scenes. Although this may be feasible, creating such a dataset would require substantial costs. In this paper, we describe a new dataset for hit-and-run incidents. We collected 833 hit-and-run videos by recreating a parking lot using miniaturized cars. This dataset has been made publicly available through Kaggle. We used three-dimensional convolution neural network, which is frequently used in the field of action recognition, to detect small movements of vehicles during hit-and-run incidents. In addition, the proportion of the area that surrounds the target vehicle to the min-max box of the vehicle itself and the length of the input frame are varied to compare the accuracy. As a result, we were able to achieve better accuracy by using the lowest proportion and the shortest input frame.

Tropical cyclone trajectory based on satellite remote sensing prediction and time attention mechanism ConvLSTM model

The accurate and timely prediction of tropical cyclones is of paramount importance in mitigating the impact of these catastrophic meteorological events. Presently, methods for predicting tropical cyclones based on satellite remote sensing images encounter notable challenges, including the inadequate extraction of three-dimensional spatial features and limitations in long-term forecasting. As a response to these challenges, this study introduces the Temporal Attention Mechanism ConvLSTM (TAM-CL) model, designed to conduct thorough spatiotemporal feature extraction on three-dimensional atmospheric reanalysis data of tropical cyclones. By leveraging ConvLSTM with three-dimensional convolution kernels, our model enhances the extraction of three-dimensional spatiotemporal features. Furthermore, an attention mechanism is integrated to bolster long-term prediction accuracy by emphasizing crucial temporal nodes. In the evaluation of tropical cyclone track and intensity forecasts across 24, 48, and 72 h, TAM-CL demonstrates a notable reduction in prediction errors, thereby underscoring its efficacy in forecasting both cyclone tracks and intensities. This contributes to an effective exploration of the application of deep networks in conjunction with atmospheric reanalysis data.

Hyperspectral Image Super-Resolution Based on Feature Diversity Extraction

Deep learning is an important research topic in the field of image super-resolution. Problematically, the performance of existing hyperspectral image super-resolution networks is limited by feature learning for hyperspectral images. Nevertheless, the current algorithms exhibit some limitations in extracting diverse features. In this paper, we address limitations to existing hyperspectral image super-resolution networks, focusing on feature learning challenges. We introduce the Channel-Attention-Based Spatial–Spectral Feature Extraction network (CSSFENet) to enhance hyperspectral image feature diversity and optimize network loss functions. Our contributions include: (a) a convolutional neural network super-resolution algorithm incorporating diverse feature extraction to enhance the network’s diversity feature learning by elevating the matrix rank, (b) a three-dimensional (3D) feature extraction convolution module, the Channel-Attention-Based Spatial–Spectral Feature Extraction Module (CSSFEM), to boost the network’s performance in both the spatial and spectral domains, (c) a feature diversity loss function designed based on the image matrix’s singular value to maximize element independence, and (d) a spatial–spectral gradient loss function introduced based on space and spectrum gradient values to enhance the reconstructed image’s spatial–spectral smoothness. In contrast to existing hyperspectral super-resolution algorithms, we used four evaluation indexes, PSNR, mPSNR, SSIM, and SAM, and our method showed superiority during testing with three common hyperspectral datasets.

High Efficient 3D Convolution Feature Compression

In this paper, a high efficient 3D convolution feature compression method is proposed. This method is mainly used to compress the three-dimensional convolution deep features extracted by video analysis. Gather the compressed features to the cloud server instead of all video features. This method can solve the problem that the data aggregation of video big data analysis requires a large amount of network bandwidth. Quantization is a common method for feature compression, but the existing quantization-based methods often carry out model training and quantization in stages, which makes the robustness of quantization results poor. To solve this problem, the method proposed in this paper is to apply the feature quantization operation directly to the network and train it with the analysis task, and use the parameter iterative optimization method to solve the non-differentiable problem of quantization operation. Different from the deep features extracted from images or single objects, the 3D convolution features extracted from video clips have high time-domain redundancy. In this paper, by serializing the three-dimensional convolution features, the time-domain prediction coding method is used to remove the time-domain redundancy of the three-dimensional convolution features, to improve the feature compression ratio. The experimental results show that this method can only use 1 bit to represent the elements in the three-dimensional convolution deep feature. When the analysis accuracy loss is no more than 1%, the feature compression ratio can reach 4500 times compared with the original feature data, and the data transmission can be reduced by 96%.

A novel parameter dense three-dimensional convolution residual network method and its application in classroom teaching.

Improving the rationality and accuracy of classroom quality analysis is crucial in modern education. Traditional methods, such as questionnaires and manual recordings, are resource-intensive and subjective, leading to inconsistent results. As a solution, computer vision (CV) technologies have emerged as powerful tools for real-time classroom monitoring. This study proposes a novel Dense 3D Convolutional Residual Network (D3DCNN_ResNet) to recognize students' expressions and behaviors in English classrooms. The proposed method combines Single Shot Multibox Detector (SSD) for target detection with an improved D3DCNN_ResNet model. The network applies 3D convolution in both spatial and temporal domains, with shortcut connections from residual blocks to increase network depth. Dense connections are introduced to enhance the flow of high- and low-level features. The model was tested on two datasets: the CK+ dataset for expression recognition and the KTH dataset for behavior recognition. The experiments show that the proposed method is highly efficient in optimizing model training and improving recognition accuracy. On the CK+ dataset, the model achieved an expression recognition accuracy of 97.94%, while on the KTH dataset, the behavior recognition accuracy reached 98.86%. The combination of residual blocks and dense connections reduced feature redundancy and improved gradient flow, leading to better model performance. The results demonstrate that the D3DCNN_ResNet is well-suited for classroom quality analysis and has the potential to enhance teaching strategies by providing real-time feedback on student engagement.

Anomaly Detection Based on a 3D Convolutional Neural Network Combining Convolutional Block Attention Module Using Merged Frames.

With the recent rise in violent crime, the real-time situation analysis capabilities of the prevalent closed-circuit television have been employed for the deterrence and resolution of criminal activities. Anomaly detection can identify abnormal instances such as violence within the patterns of a specified dataset; however, it faces challenges in that the dataset for abnormal situations is smaller than that for normal situations. Herein, using datasets such as UBI-Fights, RWF-2000, and UCSD Ped1 and Ped2, anomaly detection was approached as a binary classification problem. Frames extracted from each video with annotation were reconstructed into a limited number of images of 3×3, 4×3, 4×4, 5×3 sizes using the method proposed in this paper, forming an input data structure similar to a light field and patch of vision transformer. The model was constructed by applying a convolutional block attention module that included channel and spatial attention modules to a residual neural network with depths of 10, 18, 34, and 50 in the form of a three-dimensional convolution. The proposed model performed better than existing models in detecting abnormal behavior such as violent acts in videos. For instance, with the undersampled UBI-Fights dataset, our network achieved an accuracy of 0.9933, a loss value of 0.0010, an area under the curve of 0.9973, and an equal error rate of 0.0027. These results may contribute significantly to solve real-world issues such as the detection of violent behavior in artificial intelligence systems using computer vision and real-time video monitoring.

Synergistic exploitation of localized spectral-spatial and temporal information with DNNs for multisensor-multitemporal image-based crop classification

The challenge of performing efficient and reliable crop classification with multisensor multitemporal (MSMT) images in mixed land cover scenarios i.e. presence of small land parcels (area < 20,000-meter square) of crops and other land covers such as built-up or grasslands, is significant. Specially in countries (ex. India) where diverse crops are practiced in small land parcels. This challenge can be addressed if deep neural network (DNN) based models can exploit all three i.e. spatial, spectral, and temporal information of a crop, present in the MSMT images, efficiently and effectively. Therefore, this study presents a novel DNN based model that exploits all three information in a synergistic fashion to achieve improved crop classification. At first, the model increases the significance of local spectral information via a strategy that creates versions of spectral band set wherein neighbourhood of spectral bands is permuted. Then, the model utilizes three-dimensional convolutions, in a time-distributed fashion, to extract local spectral-spatial features. Finally, the model utilizes bidirectional long short-term memory or LSTM-RNNs to extract the temporal information embedded in the time-distributed feature-space created after the convolutions. The developed model is trained and evaluated on Sentinel-1 and Sentinel-2 MSMT data to achieve a 6-class classification including two major crops grown in the region. One of the proposed models namely Perm-3D-CRNN-v1 showed a 97.77 % overall accuracy on test samples and reflected satisfactory on quantitative analysis. The localized spectral-spatial convolutions created prominent class-specific features whereas the bidirectional information flow in the recurrent layer improved the exploitation of crop-phenology type features making the model perform efficiently.

Three-dimensional hybrid fusion networks for current-based bearing fault diagnosis

Abstract Intelligent fault diagnosis (IFD) techniques commonly use vibration-based measurements to perform health monitoring of critical rotating components in industrial systems. However, these vibration-based approaches may be limited in cost-sensitive applications, because the installation of vibration sensors is inconvenient and vibration sensors are expensive. Considering the difficulties of IFD using only current-related information from the motor current signal (MCS), this paper proposes a three-dimensional hybrid-fusion neural network (3D-HFN) that can automatically perform both data- and feature-level fusion of multi-phase current signals for MCS-based IFD of the rolling bearing. The 3D-HFN consists of the multivariate variational mode decomposition (MVMD) and an improved three-dimensional convolution neural network (3D-CNN). Firstly, MVMD is proposed to process multi-phase current signals, which adaptively acquire several intrinsic mode functions with mode-alignment properties. Subsequently, signal-to-image conversion and 3D stacking methods are used to construct 3D-like data in the current-phase dimension, which can fully preserve the interaction relationship between different phases using data-level fusion. Finally, an improved 3D-CNN with multiscale feature fusion and the smooth maximum unit is proposed to learn the 3D-like data and identify different health conditions for the rolling bearing. An open-source dataset with composite bearing faults is used to validate the merits of the proposed method. Experimental results show that the proposed approach has achieved more reliable diagnosis performance than other hand-crafted or 2D/3D-CNN-based algorithms in MCS-based IFD of the rolling bearing.

Multi-dimensional stereo face reconstruction for psychological assistant diagnosis in medical meta-universe

Virtual reality/augmented reality (VR/AR) technology is very important for meta-cosmic medicine. It is the technical link between the material world and the virtual world. The realization of face-to-face psychological diagnosis with three-dimensional holography is a difficult problem in meta-cosmic medicine. Because psychological diagnosis needs to observe rapidly changing microexpressions, it is difficult to capture them. At present, most of the research is based on two dimensions, which cannot meet the real-time reconstruction requirements of meta-universe medicine. To solve these problems, this paper introduces a 3D facial reconstruction algorithm composed of multiresolution feature fusion and attention mechanisms. It can be used for psychological assistant diagnosis of meta-cosmic medicine. First, face alignment and 3D coordinate construction of the feature point cloud are carried out through a 3D convolution coding network. After that, multiresolution cost convolution is used to extract and match the color, facial bones, and depth features of texture images. Finally, the transposed three-dimensional convolution is used to decode and output. To achieve the real-time reconstruction of psychotherapy, we designed a three-dimensional channel attention mechanism module for acceleration. This makes this method suitable for the construction of virtual doctors and virtual patients in the virtual platform of the medical universe of psychological medicine and can also complete the construction of the virtual teaching platform of the medical universe.