Self-supervised Deep Learning Research Articles

Motion correction for native myocardial T1 mapping using self-supervised deep learning registration with contrast separation.

In myocardial T1 mapping, undesirable motion poses significant challenges because uncorrected motion can affect T1 estimation accuracy and cause incorrect diagnosis. In this study, we propose and evaluate a motion correction method for myocardial T1 mapping using self-supervised deep learning based registration with contrast separation (SDRAP). A sparse coding based method was first proposed to separate the contrast component from T1 -weighted (T1w) images. Then, a self-supervised deep neural network with cross-correlation (SDRAP-CC) or mutual information as the registration similarity measurement was developed to register contrast separated images, after which signal fitting was performed on the motion corrected T1w images to generate motion corrected T1 maps. The registration network was trained and tested in 80 healthy volunteers with images acquired using the modified Look-Locker inversion recovery (MOLLI) sequence. The proposed SDRAP was compared with the free form deformation (FFD) registration method regarding (1) Dice similarity coefficient (DSC) and mean boundary error (MBE) of myocardium contours, (2) T1 value and standard deviation (SD) of T1 fitting, (3) subjective evaluation score for overall image quality and motion artifact level, and (4) computation time. Results showed that SDRAP-CC achieved the highest DSC of 85.0 ± 3.9% and the lowest MBE of 0.92 ± 0.25 mm among the methods compared. Additionally, SDRAP-CC performed the best by resulting in lower SD value (28.1 ± 17.6 ms) and higher subjective image quality scores (3.30 ± 0.79 for overall quality and 3.53 ± 0.68 for motion artifact) evaluated by a cardiologist. The proposed SDRAP took only 0.52 s to register one slice of MOLLI images, achieving about sevenfold acceleration over FFD (3.7 s/slice).

Self-Supervised Deep Learning Framework for Anomaly Detection in Traffic Data

Self-Supervised Deep Learning Framework for Anomaly Detection in Traffic Data

Self-Supervised Deep Learning for Improved Image-Based Wave-Front Sensing

Phase retrieval from supervised learning neural networks is restricted due to the problem of obtaining labels. To address this situation, in the present paper, we propose a phase retrieval model of self-supervised physical deep learning combined with a complete physical model to represent the image-formation process. The model includes two parts: one is MobileNet V1, which is used to map the input samples to the Zernike coefficients, the other one is an optical imaging system and it is used to obtain the point spread function for training the model. In addition, the loss function is calculated based on the similarity between the input and the output to realize self-supervised learning. The root-mean-square (RMS) of the wave-front error (WFE) between the input and reconstruction is 0.1274 waves in the situation of D/r0 = 20 in the simulation. By comparison, The RMS of WFE is 0.1069 waves when using the label to train the model. This method retrieves numerous wave-front errors in real time in the presence of simulated detector noise without relying on label values. Moreover, this method is more suitable for practical applications and is more robust than supervised learning. We believe that this technology has great applications in free-space optical communication.

Spatial-temporal 3D dependency matching with self-supervised deep learning for monocular visual sensing

Monocular visual sensing is the task of using a camera to estimate the scene depth, optical flow and camera pose. In this paper, we propose a spatial–temporal 3D dependency matching approach that enforces the robustness of continuous frames matching for monocular visual sensing. 3D structure and warped depth based geometry backpropagation are used to encourage jointly learning the view depth, optical flow and camera pose employing a novel self-supervised neural network from monocular sequences. We designed two different iterative convolutional prediction sub-networks, where the optical flow obtained by depth and camera pose is iteratively used for depth prediction. A virtual frame method is proposed to optimize the optical flow of moving objects. The salient feature of the proposed learning framework is completely unsupervised, requiring only consecutive monocular images for training and testing. Evaluation on publicly benchmark datasets shows that our unsupervised learning model significantly outperforms previous methods and achieves better performance compared with previously unsupervised manners and achieves comparable results with supervised ones.

Self-Supervised Deep Learning to Reconstruct Seismic Data With Consecutively Missing Traces

Seismic data processing requires careful interpolation or reconstruction to restore the regularly or irregularly missing traces. In practice, seismic data with consecutively missing traces are quite common, which will lead to a great challenge for conventional interpolation or reconstruction methods. To effectively reconstruct the successively blank traces in seismic data, we proposed a self-supervised deep learning approach, with which the convolutional neural network is trained in a supervised manner with pseudolabels obtained from unlabeled observed data. The pseudolabels are automatically generated by randomly masking the observed data to simulate the consecutively missing scenario. We train a nested U-Net structure (UNet++) with a hybrid loss function so that the local and global structural information can be captured to ensure the quality of reconstruction. A two-step reconstruction workflow is designed to recover the missing recordings with respect to both the receivers and sources. Synthetic and field data examples demonstrate that the proposed self-supervised learning can effectively reconstruct the corrupted seismic data.

Missing Shots and Near-Offset Reconstruction of Marine Seismic Data With Towered Streamers via Self-Supervised Deep Learning

Missing Shots and Near-Offset Reconstruction of Marine Seismic Data With Towered Streamers via Self-Supervised Deep Learning

A Multidirectional Deep Neural Network for Self-Supervised Reconstruction of Seismic Data

Seismic studies exhibit gaps in the recorded data due to surface obstacles. To fill in the gaps with self-supervised deep learning, the network learns to predict different events from the recorded parts of data and then applies it to reconstruct the missing parts of the same dataset. We propose two improvements to the task: a rearrangement of the data, and a new deep-learning approach. We rearrange the traces of a 2D acquisition line as 3D data cubes, sorting the traces by the source and receiver coordinates. This 3D representation offers more information about the structure of the seismic events and allows a coherent reconstruction of them. However, learning the structure of events in 3D cubes is more complicated than in 2D images while the size of the training dataset is limited. Thus, we propose a specific architecture and training strategy to take advantage of 3D data samples, while benefiting from the simplicity of 2D reconstructions. Our proposed multidirectional convolutional neural network has two parallel branches trained to perform 2D reconstructions along the vertical and horizontal directions and a small 3D part that combines their results. We use our method to reconstruct data gaps resulting from several missing shots in a benchmark synthetic and a real land dataset. Compared to a conventional 3D U-net, our network learns to reconstruct the events more accurately. Compared to 2D U-nets, our method avoids the discontinuities that arise from the 2D reconstruction of each trace of the missing shot gathers.

Self-Supervised Deep Learning for 3D Gravity Inversion

The gravity method is one of the non-destructive geophysical methods, which aims to estimate the 3D subsurface density distribution of geological bodies from the observed 2D surface gravity anomalies. Recently, deep learning has achieved great success in solving ill-posed problems including gravity inversion. The limitation of the current deep learning methods for gravity inversion is the difference between synthetic and field data. Thus, we introduce a self-supervised estimation method for 3D gravity inversion (SSGI). SSGI learns the field data directly by closed-loop of the inversion model and forward model. The proposed inversion model contains an encoder, an expander, a decoder, and a 3D refiner. Since the forward model is built according to the law of universal gravitation, SSGI can optimize the inversion model by minimizing the mean absolute error of the original and reconstructed gravity anomalies. Besides, SSGI constrains the inversion model by a guide-line in the auxiliary loop. Since the guide-line corresponds to the sampling or average of the density matrix, minimizing the mean absolute error between the original guide-line and the generated guide-line can reduce the uncertainty of inversion. The experimental results demonstrate that the proposed SSGI achieves state-of-the-art performance in 3D gravity inversion.

Dropout-Based Robust Self-Supervised Deep Learning for Seismic Data Denoising

Incoherent noise suppression is an indispensable step in seismic data processing. Recently, deep learning (DL) methods have gained commendable success in seismic data denoising, one of which is the supervised DL denoising method using clean data as the training label, whereas the cost of obtaining clean data is high. We investigate a robust self-supervised DL denoising method without using clean data. Bernoulli-sampled training pairs of the raw noisy data produced by the dropout layer are served to train the NN, and a Monte Carlo (MC) self-integrated technique results in further improving the denoising quality of the trained NN during the testing. Compared with the f-x deconvolution (FXDECON), deep image prior (DIP), and sparse autoencoder (SAE) methods via synthetic and real data examples, the proposed method outperforms these methods for enhancing the signal-to-noise ratio (SNR) and reducing the signal loss.

Self-Supervised Deep Learning for Nonlinear Seismic Full Waveform Inversion

Seismic full waveform inversion (FWI) is able to build high-resolution velocity model based on the full information carried by seismic wave. However, FWI requires an accurate enough initial model to ensure convergence. In this paper, we propose a new nonlinear FWI method to mitigate the initial model dependence problem. Specifically, we firstly propose a nonlinear operator within the hybrid model- and data-driven framework based on the frequency controllable envelope operator (FCEO) and a deep learning architecture U-Net. FCEO is used to obtain the envelope of a band-limited data and U-Net realizes the mapping from this envelope to that corresponding to a lower frequency band. The U-Net is trained in a self-supervised manner that avoids the reliance on labeled data and benefits the generalization ability. Based on the nonlinear operator, a nonlinear FWI method is proposed by defining a new misfit function. In addition, the calculation of gradient is derived using the adjoint-state method. Using numerical examples, we investigate the performance of the proposed nonlinear operator and the new nonlinear FWI method. The results clearly demonstrate that the proposed nonlinear operator is effective in obtaining low-frequency envelope data, and the new nonlinear FWI method has advantages over common method in mitigating cycle-skipping and building an initial model for conventional FWI.

A CNN-Based Self-Supervised Synthetic Aperture Radar Image Denoising Approach

Synthetic aperture radar (SAR) plays an essential role in earth observation and projection due to its capability to penetrate clouds, which makes it possible to monitor terrestrial surfaces under all weather conditions. Multiplicative noise often occurs in the SAR signal, hampering the retrieval of information from SAR imagery. Convolutional neural networks (CNNs) have been used in many computer vision tasks and are helpful in image denoising. However, current CNN-based denoising approaches inevitably lead to a ``washed out'' effect that loses spatial details. Another limitation is that most typical CNN-based denoising models require a noise-free image for training. To address these issues, we propose a novel end-to-end self-supervised SAR denoising model: Enhanced Noise2Noise (EN2N), which can be trained without a noise-free image. To enhance the quality of the result images, the perceptual features from a pre-learned CNN are introduced to restore the spatial details by a hybrid loss function. Experiments show that our proposed method outperforms the typical denoising methods in terms of noise reduction and feature preservation based on image quality metrics. Also, the new hybrid loss could enhance the spatial details significantly. The good performance maintains the robustness throughout time, which reduces the uncertainty in time-series SAR caused by random noise. Benefiting from optimization of graphics processing unit (GPU) and multi-threading, the proposed method has higher computation efficiency than traditional methods. This study demonstrates the great potential of using our self-supervised deep learning approaches for SAR image denoising in the future.

A Practical Approach for Hyperspectral Unmixing Using Deep Learning

The deep learning methods have started showing promising results for spectral unmixing. We observe that many of them need direct supervision in the form of unmixed components, which is scarcely available in practice. At the same time, some require extensive data for supervised training in order to handle high implicit noise. In this letter, we propose a two-stage fully connected self-supervised deep learning network for alleviating these practical issues in performing blind hyperspectral unmixing. Given the data, the first stage (inverse model) jointly estimates the endmembers and abundances, whereas the second stage (forward model) learns the physics of hyperspectral image acquisition. The central idea is to reconstruct the hyperspectral input vector using estimated endmembers and abundances at the inverse model, which best presents the input vector’s underlying physics to the forward model. The network is jointly optimized against a two-stage loss function (in measurement domain) during the training, and we decouple the second stage at the time of inference. AdamW is used to optimize the loss function, while ReLU with a dropout of 0.3 is employed to avoid possible overfitting. The proposed network requires only the sensed hyperspectral data and learns the unmixing function from data itself, even at a lower signal-to-noise ratio (SNR). Experiments are conducted on synthetic data at different SNRs and two real benchmark hyperspectral data. The efficacy of the proposed model is evaluated and compared with the state of the art both qualitatively and quantitatively.

Rotor Localization and Phase Mapping of Cardiac Excitation Waves Using Deep Neural Networks.

The analysis of electrical impulse phenomena in cardiac muscle tissue is important for the diagnosis of heart rhythm disorders and other cardiac pathophysiology. Cardiac mapping techniques acquire local temporal measurements and combine them to visualize the spread of electrophysiological wave phenomena across the heart surface. However, low spatial resolution, sparse measurement locations, noise and other artifacts make it challenging to accurately visualize spatio-temporal activity. For instance, electro-anatomical catheter mapping is severely limited by the sparsity of the measurements, and optical mapping is prone to noise and motion artifacts. In the past, several approaches have been proposed to create more reliable maps from noisy or sparse mapping data. Here, we demonstrate that deep learning can be used to compute phase maps and detect phase singularities in optical mapping videos of ventricular fibrillation, as well as in very noisy, low-resolution and extremely sparse simulated data of reentrant wave chaos mimicking catheter mapping data. The self-supervised deep learning approach is fundamentally different from classical phase mapping techniques. Rather than encoding a phase signal from time-series data, a deep neural network instead learns to directly associate phase maps and the positions of phase singularities with short spatio-temporal sequences of electrical data. We tested several neural network architectures, based on a convolutional neural network (CNN) with an encoding and decoding structure, to predict phase maps or rotor core positions either directly or indirectly via the prediction of phase maps and a subsequent classical calculation of phase singularities. Predictions can be performed across different data, with models being trained on one species and then successfully applied to another, or being trained solely on simulated data and then applied to experimental data. Neural networks provide a promising alternative to conventional phase mapping and rotor core localization methods. Future uses may include the analysis of optical mapping studies in basic cardiovascular research, as well as the mapping of atrial fibrillation in the clinical setting.

Tone mapping high dynamic range images based on region-adaptive self-supervised deep learning

This paper presents a region-adaptive self-supervised deep learning (RASSDL) technique for high dynamic range (HDR) image tone mapping. The RASSDL tone mapping operator (TMO) is a convolutional neural network (CNN) trained on local image regions that can seamlessly tone map images of arbitrary sizes. The training of RASSDL TMO is through the design of a self-supervising target that automatically adapts to the local image regions based on their information contents. The self-supervising target is designed to ensure the tone-mapped output achieves a balance between preserving the relative contrast of the original scene and the visibilities of the fine details to achieve faithful reproduction of the HDR scene. Distinguishing from many existing TMOs that require manual tuning of parameters, RASSDL is parameter-free and completely automatic. Experimental results demonstrate that RASSDL TMO can achieve state-of-the-art performance in terms of preserving overall contrasts, revealing fine details, and being free from visual artifacts.

The potential of self-supervised networks for random noise suppression in seismic data

Noise suppression is an essential step in many seismic processing workflows. A portion of this noise, particularly in land datasets, presents itself as random noise. In recent years, neural networks have been successfully used to denoise seismic data in a supervised fashion. However, supervised learning always comes with the often unachievable requirement of having noisy-clean data pairs for training. Using blind-spot networks, we redefine the denoising task as a self-supervised procedure where the network uses the surrounding noisy samples to estimate the noise-free value of a central sample. Based on the assumption that noise is statistically independent between samples, the network struggles to predict the noise component of the sample due to its randomicity, whilst the signal component is accurately predicted due to its spatio-temporal coherency. Illustrated on synthetic examples, the blind-spot network is shown to be an efficient denoiser of seismic data contaminated by random noise with minimal damage to the signal; therefore, providing improvements in both the image domain and down-the-line tasks, such as post-stack inversion. To conclude our study, the suggested approach is applied to field data and the results are compared with two commonly used random denoising techniques: FX-deconvolution and sparsity-promoting inversion by Curvelet transform. By demonstrating that blind-spot networks are an efficient suppressor of random noise, we believe this is just the beginning of utilising self-supervised learning in seismic applications. • Application of self-supervised deep learning for random noise suppression. • Blind-spot networks shown to effectively reduce white and coloured noise. • SNR improvements seen in the image, frequency and acoustic impedance domains. • Network successfully trained and applied to post-stack field data. • Blind-spot approach comparable to commonly used noise suppression procedures.

Visual SLAM-Based Robotic Mapping Method for Planetary Construction.

With the recent discovery of water-ice and lava tubes on the Moon and Mars along with the development of in-situ resource utilization (ISRU) technology, the recent planetary exploration has focused on rover (or lander)-based surface missions toward the base construction for long-term human exploration and habitation. However, a 3D terrain map, mostly based on orbiters’ terrain images, has insufficient resolutions for construction purposes. In this regard, this paper introduces the visual simultaneous localization and mapping (SLAM)-based robotic mapping method employing a stereo camera system on a rover. In the method, S-PTAM is utilized as a base framework, with which the disparity map from the self-supervised deep learning is combined to enhance the mapping capabilities under homogeneous and unstructured environments of planetary terrains. The overall performance of the proposed method was evaluated in the emulated planetary terrain and validated with potential results.

Deep Learning in Therapeutic Antibody Development.

Deep learning applied to antibody development is in its adolescence. Low data volumes and biological platform differences make it challenging to develop supervised models that can predict antibody behavior in actual commercial development steps. But successes in modeling general protein behaviors and early antibody models give indications of what is possible for antibodies in general, particularly since antibodies share a common fold. Meanwhile, new methods of data collection and the development of unsupervised and self-supervised deep learning methods like generative models and masked language models give the promise of rich and deep data sets and deep learning architectures for better supervised model development. Together, these move the industry toward improved developability , lower costs, and broader access of biotherapeutics .

20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction.

High spatial and temporal resolution across the whole brain is essential to accurately resolve neural activities in fMRI. Therefore, accelerated imaging techniques target improved coverage with high spatio-temporal resolution. Simultaneous multi-slice (SMS) imaging combined with in-plane acceleration are used in large studies that involve ultrahigh field fMRI, such as the Human Connectome Project. However, for even higher acceleration rates, these methods cannot be reliably utilized due to aliasing and noise artifacts. Deep learning (DL) reconstruction techniques have recently gained substantial interest for improving highly-accelerated MRI. Supervised learning of DL reconstructions generally requires fully-sampled training datasets, which is not available for high-resolution fMRI studies. To tackle this challenge, self-supervised learning has been proposed for training of DL reconstruction with only undersampled datasets, showing similar performance to supervised learning. In this study, we utilize a self-supervised physics-guided DL reconstruction on a 5-fold SMS and 4-fold in-plane accelerated 7T fMRI data. Our results show that our self-supervised DL reconstruction produce high-quality images at this 20-fold acceleration, substantially improving on existing methods, while showing similar functional precision and temporal effects in the subsequent analysis compared to a standard 10-fold accelerated acquisition.

Picket: guarding against corrupted data in tabular data during learning and inference

Data corruption is an impediment to modern machine learning deployments. Corrupted data can severely bias the learned model and can also lead to invalid inferences. We present, Picket, a simple framework to safeguard against data corruptions during both training and deployment of machine learning models over tabular data. For the training stage, Picket identifies and removes corrupted data points from the training data to avoid obtaining a biased model. For the deployment stage, Picket flags, in an online manner, corrupted query points to a trained machine learning model that due to noise will result in incorrect predictions. To detect corrupted data, Picket uses a self-supervised deep learning model for mixed-type tabular data, which we call PicketNet. To minimize the burden of deployment, learning a PicketNet model does not require any human-labeled data. Picket is designed as a plugin that can increase the robustness of any machine learning pipeline. We evaluate Picket on a diverse array of real-world data considering different corruption models that include systematic and adversarial noise during both training and testing. We show that Picket consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier detection models.

Self-supervised delineation of geologic structures using orthogonal latent space projection

We have developed two machine-learning frameworks that could assist in automated lithostratigraphic interpretation of seismic volumes without any manual hand labeling from an experienced seismic interpreter. The first framework is an unsupervised hierarchical clustering model used to divide seismic images from a volume into a certain number of clusters determined by the algorithm. The clustering framework uses a combination of density and hierarchical techniques to determine the size and homogeneity of the clusters. The second framework consists of a self-supervised deep learning framework to label regions of geologic interest in seismic images. It projects the latent space of an encoder-decoder architecture onto two orthogonal subspaces, from which it learns to delineate regions of interest in the seismic images. To demonstrate an application of both frameworks, a seismic volume is clustered into various contiguous clusters, from which four clusters are selected based on distinct seismic patterns: horizons, faults, salt domes, and chaotic structures. Images from the selected clusters are used to train the encoder-decoder network. The output of the encoder-decoder network is a probability map of the possibility that an amplitude reflection event belongs to an interesting geologic structure. The structures are delineated using the probability map. The delineated images are further used to posttrain a segmentation model to extend our results to full vertical sections. The results on vertical sections indicate that we can factorize a seismic volume into its corresponding structural components. Finally, we find that our deep learning framework can be modeled as an attribute extractor and we compare our attribute result with various existing attributes in the literature and determine the competitive performance with them.