Residual Learning Framework Research Articles

Hybrid connected attentional lightweight network for gangue intelligent segmentation in top-coal caving face

The estimation of gangue content is the main basis for intelligent top coal caving mining by computer vision, and the automatic segmentation of gangue is crucial to computer vision analysis. However, it is still a great challenge due to the degradation of images and the limitation of computing resources. In this paper, a hybrid connected attentional lightweight network (HALNet) with high speed, few parameters and high accuracy is proposed for gangue intelligent segmentation on the conveyor in the top-coal caving face. Firstly, we propose a deep separable dilation convolution block (DSDC) combining deep separable convolution and dilation convolution, which can provide a larger receptive field to learn more information and reduce the size and computational cost of the model. Secondly, a bridging residual learning framework is designed as the basic unit of encoder and decoder to minimize the loss of semantic information in the process of feature extraction. An attention fusion block (AFB) with skip pathway is introduced to capture more representative and distinctive features through the fusion of high-level and low-level features. Finally, the proposed network is trained through the expanded dataset, and the gangue image segmentation results are obtained by pixel-by-pixel classification method. The experimental results show that the proposed HALNet reduces about 57 percentage parameters compared with U-Net, and achieves state-of-the art performance on dataset.

SAR-CGAN: Improved generative adversarial network for EIT reconstruction of lung diseases

The image reconstruction of electrical impedance tomography (EIT) is a nonlinear ill-posed inverse problem, and the reconstructed images tend to have artifacts due to noise in the measurement system. The shallow neural networks are difficult to fully express the nonlinear relationship of the EIT imaging problems. Deep neural networks have demonstrated great potential to remove artifacts from initial conductivity images caused by system noise. In the paper, a novel deep learning method, namely, conditional generative adversarial network with soft-attention gates (AGs) and residual learning (SAR-CGAN), is proposed to solve the above problems of EIT imaging. SAR-CGAN consists of generator and discriminator. Specifically, the AGs are employed by the generator to improve sensitivity and accuracy for lung EIT imaging by suppressing feature activation in irrelevant EIT region. And residual learning framework is introduced in the discriminator to address the problem of model degradation and easily enjoy accuracy gains from greatly increased depth. Extensive confirmatory and comparative experiments with three kinds of lung disease are conducted and compared with CNN, DnCNN, Attention U-Net and CGAN methods. The imaging results indicate that SAR-CGAN method can accurately recover the position and boundaries of lesions in lung images and effective reconstruction can be accomplished for the new conductivity distribution and noisy samples.

A multiple frequency bands parallel spatial–temporal 3D deep residual learning framework for EEG-based emotion recognition

Electroencephalography (EEG) based emotion recognition has become a hot research issue in the field of cognitive interaction and brain-computer interface (BCI). How to build a deep learning model which can fully learn frequency-spatial–temporal representation from complex emotional EEG data and has good neurological interpretability is still challenging. In this paper, a novel multiple frequency bands parallel spatial–temporal 3D deep residual learning framework (MFBPST-3D-DRLF) is proposed for EEG-based emotion recognition. Firstly, a new optimal frequency bands selection method based on group sparse regression is designed for characteristic analysis on frequency domain. Secondly, spatial–temporal 3D feature representations of multiple frequency bands are generated in the data preparation stage for fully expressing the discriminative local patterns among brain responses of different emotional states. Finally, a novel parallel 3D deep residual networks architecture is elaborately constructed to simultaneously extract high level abstract features and achieve accurate classification. Emotional EEG recognition performance of the proposed method has been evaluated on two benchmark datasets, namely SEED and SEED-IV. The proposed MFBPST-3D-DRLF achieves 96.67% and 88.21% on both datasets, outperforming several state-of-the-art algorithms. In addition, investigations on the intermediate results and model parameters reveal that neural signatures associated with different emotional states are traceable and gamma band is most suitable for EEG based emotion recognition.

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy.

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre-defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high- and low-polarization dynamic (out-of-plane) ferroelastic domain walls in a PbTiO3 (PTO) thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics.

Water Leakage and Crack Identification in Tunnels Based on Transfer-Learning and Convolutional Neural Networks

In order to solve the problems of long artificial time consumption, the inability to standardize the degree of damage, and the difficulty of maintaining data in traditional tunnel disease detection methods, this paper proposes the use of Residual Network (ResNet) models for tunnel water leakage and crack detection. ResNet proposes a residual learning framework to ease the training of networks that are deeper than those previously used. Furthermore, ResNet explicitly reformulates the layers as learning the residual functions of the reference layer inputs, rather than learning the unreferenced functions. The ResNet model is built on the Tensorflow Deep Learning (DL) framework and transfer-learning is used to optimize the model. The ResNet-V1 can be obtained by pre-training in ImageNet. The fully connected layers of the ResNet-V1 were modified to four classifications of tunnel disease. Then, the SoftMax function is used to recognize the tunnel diseases. Four network structures have been chosen, i.e., ResNet34 and ResNet50, with and without Transfer-learning, respectively. Those models were selected for testing and training on the sample dataset, and these four network structures were compared and analyzed using five types of evaluation indicators, which are the confusion matrix, accuracy, precision, recall ratio and F1. In identifying tunnel cracks and water leakage, the accuracy of ResNet50 and ResNet34 using the transfer-learning were 96.30% and 91.29%, and the accuracy of ResNet50 was 5.01% higher than that of ResNet34; for the network structure without the transfer-learning, the accuracy of ResNet50 was 90.36% and ResNet34’s accuracy was 87.87%. These data show that the accuracy of ResNet50 is higher than that of ResNet34 with or without the transfer-learning, and the deep structure framework is superior in the identification of tunnel diseases; secondly, comparing the network structures with and without the transfer-learning, it can be found that using the Transfer-Learning can improve the ResNet network’s accuracy for tunnel disease identification. The experiments and reliability analysis demonstrate the intelligent tunnel disease identification method proposed in this paper, and its good robustness and generalization performance. This method can be used for the rapid identification of cracks and water leakage in a tunnel survey, construction and maintenance, which has practical engineering implications for tunnel disease detection.

Residual Spatial Attention Kernel Generation Network for Hyperspectral Image Classification With Small Sample Size

With the rapid development of deep learning, the convolutional neural networks (CNNs) have been widely used in hyperspectral image classification (HSIC) and achieved excellent performance. However, CNNs reuse the same kernel weights over different locations, resulting in the insufficient capability of capturing diversity spatial interactions. Moreover, CNNs usually require a large amount of training samples to optimize the learnable parameters. When training samples are limited, the classification performance of CNN tends to drop off a cliff. To tackle the aforementioned issues, a novel residual spatial attention kernel generation network (RSAKGN) is proposed for HSIC. First, a spatial attention kernel generation module (SAKGM) is built to extract discriminative semantic features, which can dynamically calculate the attention weights to generate specific spatial attention kernels over different locations. Then, we combine the SAKGM with residual learning framework by embedding the SAKGM into a bottleneck residual block to obtain the residual spatial attention block (RSAB). The RSAKGN is constructed by stacking several RSABs. Experimental results on three public HSI datasets demonstrate that the proposed RSAKGN method outperforms several state-of-the-arts with small sample size.

Applying GMEI-GAN to Generate Meaningful Encrypted Images in Reversible Data Hiding Techniques

With the rapid development of information technology, the transmission of information has become convenient. In order to prevent the leakage of information, information security should be valued. Therefore, the data hiding technique has become a popular solution. The reversible data hiding technique (RDH) in particular uses symmetric encoding and decoding algorithms to embed the data into the cover carrier. Not only can the secret data be transmitted without being detected and retrieved completely, but the cover carrier also can be recovered without distortion. Moreover, the encryption technique can protect the carrier and the hidden data. However, the encrypted carrier is a form of ciphertext, which has a strong probability to attract the attention of potential attackers. Thus, this paper uses the generative adversarial networks (GAN) to generate meaningful encrypted images for RDH. A four-stage network architecture is designed for the experiment, including the hiding network, the encryption/decryption network, the extractor, and the recovery network. In the hiding network, the secret data are embedded into the cover image through residual learning. In the encryption/decryption network, the cover image is encrypted into a meaningful image, called the marked image, through GMEI-GAN, and then the marked image is restored to the decrypted image via the same architecture. In the extractor, 100% of the secret data are extracted through the residual learning framework, same as the hiding network. Lastly, in the recovery network, the cover image is reconstructed with the decrypted image and the retrieved secret data through the convolutional neural network. The experimental results show that using the PSNR/SSIM as the criteria, the stego image reaches 45.09 dB/0.9936 and the marked image achieves 38.57 dB/0.9654. The proposed method not only increases the embedding capacity but also maintains high image quality in the stego images and marked images.

Convolutional neural network with coarse-to-fine resolution fusion and residual learning structures for cross-modality image synthesis

Multi-modality MRI has been increasingly used for disease diagnosis and prognosis evaluation. However, the acquisitions of multi-modal images always face some practical difficulties such as the increased cost and patient discomfort. In this paper, we proposed an improved 3D UNet model for MRI modality synthesis and use the synthetic modalities to improve the performance of the following image analysis tasks such as tumor segmentation and tumor grading. We developed our model with three improvements on the basic 3D UNet model. First, we constructed a coarse-to-fine resolution fusion network to improve the ability of structural texture preservation. Second, we designed a convolutional fusion-based residual learning framework to learn mapping function, which leads to more detail preservation as well as less artifacts in the background. Third, for many-to-one synthesis, to combine the input multi-modalities more effectively, we introduced a new network module to fuse the UNet output feature maps of each modality. One private and two public datasets including a total of 493 cases were used for synthesis validation. Our proposed method surpasses the best existing method about 2.0 dB and 0.8 dB in PSNRs on the two datasets, respectively. Furthermore, two separate radiomics applications were used to demonstrate the value of our modality synthesis method. In the image segmentation application, integrating the modalities synthesized by our method has improved the segmentation Dice by 1.3%. In gliomas grading diagnosis application, integrating the synthesized modalities improved the diagnostic accuracy by 8.3%.

Deep Convolutional Neural Network for Large-Scale Date Palm Tree Mapping from UAV-Based Images

Large-scale mapping of date palm trees is vital for their consistent monitoring and sustainable management, considering their substantial commercial, environmental, and cultural value. This study presents an automatic approach for the large-scale mapping of date palm trees from very-high-spatial-resolution (VHSR) unmanned aerial vehicle (UAV) datasets, based on a deep learning approach. A U-Shape convolutional neural network (U-Net), based on a deep residual learning framework, was developed for the semantic segmentation of date palm trees. A comprehensive set of labeled data was established to enable the training and evaluation of the proposed segmentation model and increase its generalization capability. The performance of the proposed approach was compared with those of various state-of-the-art fully convolutional networks (FCNs) with different encoder architectures, including U-Net (based on VGG-16 backbone), pyramid scene parsing network, and two variants of DeepLab V3+. Experimental results showed that the proposed model outperformed other FCNs in the validation and testing datasets. The generalizability evaluation of the proposed approach on a comprehensive and complex testing dataset exhibited higher classification accuracy and showed that date palm trees could be automatically mapped from VHSR UAV images with an F-score, mean intersection over union, precision, and recall of 91%, 85%, 0.91, and 0.92, respectively. The proposed approach provides an efficient deep learning architecture for the automatic mapping of date palm trees from VHSR UAV-based images.

Learning Gated Non-Local Residual for Single-Image Rain Streak Removal

This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.

FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Depth Completion

Depth completion aims to recover a dense depth map from a sparse depth map with the corresponding color image as input. Recent approaches mainly formulate the depth completion as a one-stage end-to-end learning task, which outputs dense depth maps directly. However, the feature extraction and supervision in one-stage frameworks are insufficient, limiting the performance of these approaches. To address this problem, we propose a novel end-to-end residual learning framework, which formulates the depth completion as a two-stage learning task, i.e., a sparse-to-coarse stage and a coarse-to-fine stage. First, a coarse dense depth map is obtained by a simple CNN framework. Then, a refined depth map is further obtained using a residual learning strategy in the coarse-to-fine stage with coarse depth map and color image as input. Specially, in the coarse-to-fine stage, a channel shuffle extraction operation is utilized to extract more representative features from color image and coarse depth map, and an energy based fusion operation is exploited to effectively fuse these features obtained by channel shuffle operation, thus leading to more accurate and refined depth maps. We achieve SoTA performance in RMSE on KITTI benchmark. Extensive experiments on other datasets future demonstrate the superiority of our approach over current state-of-the-art depth completion approaches.

Effect of Scan Time on Neuro 18F-Fluorodeoxyglucose Positron Emission Tomography Image Generated Using Deep Learning

The purpose of this study was to generate the PET images with high signal-to-noise ratio (SNR) acquired for typical scan durations (H-PET) from short scan time PET images with low SNR (L-PET) using deep learning and to evaluate the effect of scan time on the quality of predicted PET image. A convolutional neural network (CNN) with a concatenated connection and residual learning framework was implemented. PET data from 27 patients were acquired for 900 s, starting 60 minutes after the intravenous administration of FDG using a commercial PET/CT scanner. To investigate the effect of scan time on the quality of the predicted H-PETs, 10 s, 30 s, 60 s, and 120 s PET data were generated by sorting the 900 s LMF data into the LMF data acquired for each scan time. Twenty-three of the 27 patient images were used for training of the proposed CNN and the remaining four patient images were used for test of the CNN. The predicted H-PETs generated by the CNN were compared to ground-truth H-PETs, L-PETs, and filtered L-PETs processed with four commonly used denoising algorithms. The peak signal-to-noise ratios (PSNRs), normalized root mean square errors (NRMSEs), and average regionof- interest (ROI) differences as a function of scan time were calculated. The quality of the predicted H-PETs generated by the CNN was superior to that of the L-PETs and filtered L-PETs. Lower NRMSEs and higher PSNRs were also obtained from predicted H-PETs compared to the L-PETs and filtered L-PETs. ROI differences in the predicted H-PETs were smaller than those of the L-PETs. The quality of the predicted H-PETs gradually improved with increasing scan times. The lowest NRMSEs, highest PSNRs, and smallest ROI differences were obtained using the predicted H-PETs for 120 s. Various performance test results for the proposed CNN indicate that it is possible to generate H-PETs from neuro FDG L-PETs using the proposed CNN method, which might allow reductions in both scan time and injection dose.

Residual multi-task learning for facial landmark localization and expression recognition

Facial landmark localization and expression recognition are two important and highly relevant topics in facial analysis. However, few works focus on using the complementary information between the two tasks to improve the performance. In this paper, we propose a residual multi-task learning framework to predict the two tasks simultaneously. Different from previous multi-task learning methods which directly train a deep multi-task network with additional branches and losses, we propose a novel residual learning module to further strengthen the linkages between the two tasks. Benefit from the proposed residual learning module, one task can learn complementary information from the other task, leading to the performance promotion. Another problem for the multi-task learning is the lack of training data with multi-task labels. For example, there is no landmark localization annotation for the two widely-used FER dataset (AffectNet and RAF), vice versa. To solve this problem, we propose an association learning method to further enhance the connection between the two tasks. Based on this connection, the dataset with single-task labels can be used in the multi-task learning. Extensive experiments are conducted on four popular datasets (i.e. 300-W, AFLW for landmark localization and AffectNet, RAF for expression recognition), demonstrating the effectiveness of the proposed algorithm.

Expectile regression via deep residual networks

Expectile is a generalization of the expected value in probability and statistics. In finance and risk management, the expectile is considered to be an important risk measure due to its connection with gain–loss ratio and its coherent and elicitable properties. Linear multiple expectile regression was proposed in 1987 for estimating the conditional expectiles of a response given a set of covariates. Recently, more flexible nonparametric expectile regression models were proposed based on gradient boosting and kernel learning. In this paper, we propose a new nonparametric expectile regression model by adopting the deep residual network learning framework and name it Expectile NN. Extensive numerical studies on simulated and real datasets demonstrate that Expectile NN has very competitive performance compared with existing methods. We explicitly specify the architecture of Expectile NN so that it is easy to be reproduced and used by others. Expectile NN is the first deep learning model for nonparametric expectile regression.

PET/CT for Brain Amyloid: A Feasibility Study for Scan Time Reduction by Deep Learning.

This study was to develop a convolutional neural network (CNN) model with a residual learning framework to predict the full-time 18F-florbetaben (18F-FBB) PET/CT images from corresponding short-time scans. In this retrospective study, we enrolled 22 cognitively normal subjects, 20 patients with mild cognitive impairment, and 42 patients with Alzheimer disease. Twenty minutes of list-mode PET/CT data were acquired and reconstructed as the ground-truth images. The short-time scans were made in either 1, 2, 3, 4, or 5 minutes. The CNN with a residual learning framework was implemented to predict the ground-truth images of 18F-FBB PET/CT using short-time scans with either a single-slice or a 3-slice input layer. Model performance was evaluated by quantitative and qualitative analyses. Additionally, we quantified the amyloid load in the ground-truth and predicted images using the SUV ratio. On quantitative analyses, with increasing scan time, the normalized root-mean-squared error and the SUV ratio differences between predicted and ground-truth images gradually decreased, and the peak signal-to-noise ratio increased. On qualitative analysis, the predicted images from the 3-slice CNN model showed better image quality than those from the single-slice model. The 3-slice CNN model with a short-time scan of at least 2 minutes achieved comparable, quantitative prediction of full-time 18F-FBB PET/CT images, with adequate to excellent image quality. The 3-slice CNN model with a residual learning framework is promising for the prediction of full-time 18F-FBB PET/CT images from short-time scans.

Deep Residual Network in Network.

Deep network in network (DNIN) model is an efficient instance and an important extension of the convolutional neural network (CNN) consisting of alternating convolutional layers and pooling layers. In this model, a multilayer perceptron (MLP), a nonlinear function, is exploited to replace the linear filter for convolution. Increasing the depth of DNIN can also help improve classification accuracy while its formation becomes more difficult, learning time gets slower, and accuracy becomes saturated and then degrades. This paper presents a new deep residual network in network (DrNIN) model that represents a deeper model of DNIN. This model represents an interesting architecture for on-chip implementations on FPGAs. In fact, it can be applied to a variety of image recognition applications. This model has a homogeneous and multilength architecture with the hyperparameter “L” (“L” defines the model length). In this paper, we will apply the residual learning framework to DNIN and we will explicitly reformulate convolutional layers as residual learning functions to solve the vanishing gradient problem and facilitate and speed up the learning process. We will provide a comprehensive study showing that DrNIN models can gain accuracy from a significantly increased depth. On the CIFAR-10 dataset, we evaluate the proposed models with a depth of up to L = 5 DrMLPconv layers, 1.66x deeper than DNIN. The experimental results demonstrate the efficiency of the proposed method and its role in providing the model with a greater capacity to represent features and thus leading to better recognition performance.

DRHNet: A Deep Residual Network Based on Heterogeneous Kernel for Steganalysis

Convolutional neural networks as steganalysis have problems such as poor versatility, long training time, and limited image size. For these problems, we present a heterogeneous kernel residual learning framework called DRHNet—Dual Residual Heterogeneous Network—to save time on the networks during the training phase. Instead of using the image as an input of the network, we extract and merge the images into a feature matrix using the rich model and use the generated feature matrix as the real input of the network. The architecture we proposed has good versatility and can reduce the computation and the number of parameters while still getting higher accuracy. On BOSSbase 1.01, we evaluate the performance of DRHNet in the setting of the spatial domain and frequency domain. The preliminary experimental results show that DRHNet shows excellent steganalysis performance against the state-of-the-art steganographic algorithms.

CSART: Channel and spatial attention-guided residual learning for real-time object tracking

Siamese networks have achieved great success in object tracking due to the balance of precision and speed. However, Siamese trackers usually utilize the local feature of the last layer, which may degrade tracking performance in some difficult scenarios. In this paper, we propose a novel Channel and Spatial Attention-guided Residual learning framework for Tracking, referred to as CSART, which can improve feature representation of Siamese networks by exploiting self-attention mechanism to capture powerful contextual information. Specifically, to be efficient and seamless integration, different kinds of self-attention are appended on the template and search branches of Siamese networks respectively, that model global semantic inter-dependencies in channel and spatial dimensions. To avoid representation degradation, we consider to adaptively aggregate basic feature and its attention-weighted features with residual learning. Furthermore, a joint loss consisting of classic logistic loss as well as focal softmax loss is designed to emphasize difficult samples and guide the learning process of the whole model. Benefiting from the above scheme, CSART alleviates the over-fitting problem to some extent and enhances the discriminability. Extensive experiments on six popular tracking datasets indicate that the proposed tracker achieves better performance with a speed of 65 fps than other state-of-the-art trackers.

Multi-Scale Gradients Self-Attention Residual Learning for Face Photo-Sketch Transformation

Face sketch synthesis, as a key technique for solving face sketch recognition, has made considerable progress in recent years. Due to the difference of modality between face photo and face sketch, traditional exemplar-based methods often lead to missed texture details and deformation while synthesizing sketches. And limited to the local receptive field, Convolutional Neural Networks-based methods cannot deal with the interdependence between features well, which makes the constraint of facial features insufficient; as such, it cannot retain some details in the synthetic image. Moreover, the deeper the network layer is, the more obvious the problems of gradient disappearance and explosion will be, which will lead to instability in the training process. Therefore, in this paper, we propose a multi-scale gradients self-attention residual learning framework for face photo-sketch transformation that embeds a self-attention mechanism in the residual block, making full use of the relationship between features to selectively enhance the characteristics of specific information through self-attention distribution. Simultaneously, residual learning can keep the characteristics of the original features from being destroyed. In addition, the problem of instability in GAN training is alleviated by allowing discriminator to become a function of multi-scale outputs of the generator in the training process. Based on cycle framework, the matching between the target domain image and the source domain image can be constrained while the mapping relationship between the two domains is established so that the tasks of face photo-to-sketch synthesis (FP2S) and face sketch-to-photo synthesis (FS2P) can be achieved simultaneously. Both Image Quality Assessment (IQA) and experiments related to face recognition show that our method can achieve state-of-the-art performance on the public benchmarks, whether using FP2S or FS2P.

Multi-Branch Deep Residual Learning for Clustering and Beamforming in User-Centric Network

In existing works of deep learning-based resource allocation, the scalability degrades heavily with the increase of network complexity, which is due to their limited learning ability of shallow neural networks and insufficient knowledge of network. Nowadays, to address the growth of cell density, cooperative beamforming in user-centric network (UCN) is emerged, where the additional degrees of freedom of multi-antenna and cell coordination aggravate the challenges. This letter proposes a deep residual learning framework, UcnBeamNet, to enhance the ability of approximating the iterative algorithm for sum rate maximization, where multi-branch subnets are connected in parallel to extract extra information. Specifically, a weighted minimum mean square error (WMMSE)-based algorithm is derived to determine the optimal clusters and beamforming matrices; then UcnBeamNet is trained to learn the input-output mapping and provide direct insight of UCN from association matrices in addition to plural inputs. Extensive experiments demonstrate UcnBeamNet still reaches 90.38% sum-rate relative to conventional algorithm even with a large network size, and achieves more than 50, $000\times $ speed up in computational efficiency.