Large-scale Training Datasets Research Articles

Generalizing deep learning electronic structure calculation to the plane-wave basis.

Deep neural networks capable of representing the density functional theory (DFT) Hamiltonian as a function of material structure hold great promise for revolutionizing future electronic structure calculations. However, a notable limitation of previous neural networks is their compatibility solely with the atomic-orbital (AO) basis, excluding the widely used plane-wave (PW) basis. Here we overcome this critical limitation by proposing an accurate and efficient real-space reconstruction method for directly computing AO Hamiltonian matrices from PW DFT results. The reconstruction method is orders of magnitude faster than traditional projection-based methods to convert PW results to the AO basis, and the reconstructed Hamiltonian matrices can faithfully reproduce the PW electronic structure, thus bridging the longstanding gap between the AO basis deep learning electronic structure approach and PW DFT. Advantages of the PW methods, such as high accuracy, high flexibility and wide applicability, thus can be all integrated into deep learning electronic structure methods without sacrificing these methods' inherent benefits. This allows for the construction of large-scale and high-fidelity training datasets with the help of PW DFT results towards the development of precise and broadly applicable deep learning electronic structure models.

Domain Decomposition for Data-Driven Reduced Modeling of Large-Scale Systems

This paper focuses on the construction of accurate and predictive data-driven reduced models of large-scale numerical simulations with complex dynamics and sparse training datasets. In these settings, standard, single-domain approaches may be too inaccurate or may overfit and hence generalize poorly. Moreover, processing large-scale datasets typically requires significant memory and computing resources, which can render single-domain approaches computationally prohibitive. To address these challenges, we introduce a domain-decomposition formulation into the construction of a data-driven reduced model. In doing so, the basis functions used in the reduced model approximation become localized in space, which can increase the accuracy of the domain-decomposed approximation of the complex dynamics. The decomposition furthermore reduces the memory and computing requirements to process the underlying large-scale training dataset. We demonstrate the effectiveness and scalability of our approach in a large-scale three-dimensional unsteady rotating-detonation rocket engine simulation scenario with more than 75 million degrees of freedom and a sparse training dataset. Our results show that compared to the single-domain approach, the domain-decomposed version reduces both the training and prediction errors for pressure by up to 13% and up to 5% for other key quantities, such as temperature, and fuel, and oxidizer mass fractions. Lastly, our approach decreases the memory requirements for processing by almost a factor of four, which in turn reduces the computing requirements as well.

Synthesizing Feature-Aligned and Category-aware Electronic Medical Records for Intracranial Aneurysm Rupture Prediction.

Rupture prediction is crucial for precise treatment and follow-up management of patients with intracranial aneurysms (IAs). Considerable machine learning (ML) methods have been proposed to improve rupture prediction by leveraging electronic medical records (EMRs), however, data scarcity and category imbalance strongly influence performance. Thus, we propose a novel data synthesis method i.e., Transformer-based conditional GAN (TransCGAN), to synthesize highly authentic and category-aware EMRs to address above challenges. Specifically, we first align feature-wise context relationship and distribution between synthetic and original data to enhance synthetic data quality. To achieve this, we first integrate the Transformer structure into GAN to match the contextual relationship by processing the long-range dependencies among clinical factors and introduce a statistical loss to maintain distributional consistency by constraining the mean and variance of the synthesis features. Additionally, a conditional module is designed to assign the category of the synthesis data, thereby addressing the challenge of category imbalance. Subsequently, the synthetic data are merged with the original data to form a large-scale and category-balanced training dataset for IAs rupture prediction. Experimental results show that using TransCGAN's synthetic data enhances classifier performance, achieving AUC of 0.89 and outperforming state-of-the-art resampling methods by 5 %-33 % in F1 score.

MF-Match: A Semi-Supervised Model for Human Action Recognition.

Human action recognition (HAR) technology based on radar signals has garnered significant attention from both industry and academia due to its exceptional privacy-preserving capabilities, noncontact sensing characteristics, and insensitivity to lighting conditions. However, the scarcity of accurately labeled human radar data poses a significant challenge in meeting the demand for large-scale training datasets required by deep model-based HAR technology, thus substantially impeding technological advancements in this field. To address this issue, a semi-supervised learning algorithm, MF-Match, is proposed in this paper. This algorithm computes pseudo-labels for larger-scale unsupervised radar data, enabling the model to extract embedded human behavioral information and enhance the accuracy of HAR algorithms. Furthermore, the method incorporates contrastive learning principles to improve the quality of model-generated pseudo-labels and mitigate the impact of mislabeled pseudo-labels on recognition performance. Experimental results demonstrate that this method achieves action recognition accuracies of 86.69% and 91.48% on two widely used radar spectrum datasets, respectively, utilizing only 10% labeled data, thereby validating the effectiveness of the proposed approach.

A maximum entropy-driven support vector classification model for seismic collapse fragility curves estimation of reinforced concrete frame structures

A novel method called maximum entropy-driven support vector classification (MED-SVC) for estimating the seismic collapse fragility curves of reinforced concrete (RC) frames is proposed in this paper. In comparison to kernel-based learning (KBL) methods, a subset selection process based on the maximum Renyi entropy criterion is embedded into the proposed MED-SVC, which can actively select an informative small-scale subset from a large-scale training dataset. The selected subset can accurately reflect the data distribution of the large-scale training dataset, which can be further used to explicitly establish a high-dimensional feature variable. Due to this, the computational complexity of training the proposed MED-SVC model only depends on the dimension of the feature variable, which can effectively solve the problem related to the low efficacy of training the KBL model in the context of a large-scale training dataset. Meanwhile, the proposed MED-SVC model can also reduce the negative effect of randomness in the subset selection process on predictive performance. The robustness, accuracy, and efficiency of the proposed MED-SVC model are thoroughly validated by comparison with random sampling driven SVC (RSD-SVC), KBL methods, random forest, neural network, and naive bayes based on 33,000 damage data of RC frames under earthquakes. Then, the proposed MED-SVC model is applied to estimate the seismic collapse fragility curve of an RC frame structure outside the aforementioned dataset. The results show that the proposed MED-SVC model can generate seismic collapse fragility curves that agree well with those produced by the finite element method (FEM) and performs more robustly than RSD-SVC.

A high-quality dataset featuring classified and annotated cervical spine X-ray atlas

Recent research in computational imaging largely focuses on developing machine learning (ML) techniques for image recognition in the medical field, which requires large-scale and high-quality training datasets consisting of raw images and annotated images. However, suitable experimental datasets for cervical spine X-ray are scarce. We fill the gap by providing an open-access Cervical Spine X-ray Atlas (CSXA), which includes 4963 raw PNG images and 4963 annotated images with JSON format (JavaScript Object Notation). Every image in the CSXA is enriched with gender, age, pixel equivalent, asymptomatic and symptomatic classifications, cervical curvature categorization and 118 quantitative parameters. Subsequently, an efficient algorithm has developed to transform 23 keypoints in images into 77 quantitative parameters for cervical spine disease diagnosis and treatment. The algorithm’s development is intended to assist future researchers in repurposing annotated images for the advancement of machine learning techniques across various image recognition tasks. The CSXA and algorithm are open-access with the intention of aiding the research communities in experiment replication and advancing the field of medical imaging in cervical spine.

Usability of novel beat-by-beat ECG annotation software portal for generating large-scale machine learning training datasets

Abstract Introduction Despite the availability of numerous large publicly available ECG datasets, there is a distinct absence of specialized software to enable the beat-by-beat annotation and labeling, limiting the ability to generate high-quality machine learning training datasets. This study analyzed the usability of such a novel software interface. Through a survey of several cardiologists and clinic staff, several metrics were gauged, including user satisfaction, system and app simplicity, user interface, and effectiveness. The survey results gleaned insight into the users’ experiences, providing an understanding of the software’s user-friendliness and potential in developing a large-scale machine learning annotated ECG training dataset to advance cardiac health research. Methods 3 cardiology physicians, and 9 cardiac telemetry unit staff were surveyed following use of the SafeBeat Annotation Software Platform to annotate a large publicly available dataset ECGs (n=10,646) beat-by-beat. The validated Post-Study System Usability Questionnaire (PSSUQ) and mobile Health App Usability Questionnaire (MAUQ) were used to assess the overall usability of the software interface. Results are reported as mean ± standard deviation. Results Users completed 10,646 ECG beat-by-beat annotations in 62 days, and averaged 164.0 ECGs per day. Physicians rated the software interface highly, with an overall mean PSSUQ of 6.4 ± 0.8 and MAUQ of 6.4 ± 0.7. Staff also rated the software interface highly, with an overall mean PSSUQ of 6.7 ± 0.6 and MAUQ of 6.8 ± 0.5. Overall 92% of the users reported that they strongly agreed they would use the application again and 83% strongly agreed that it was easy to learn how to use the system. Conclusion The usability analysis of this novel beat-by-beat ECG annotation software revealed that it was a highly effective tool for both cardiologists and clinic staff. Both the system and app were reported to be simple to learn, comfortable to use, and easily navigable. With most users responding that they could become productive quickly using this system, there is clear potential for it to be used to develop a large-scale machine learning dataset.Figure 1

A novel full-resolution convolutional neural network for urban-fringe-rural identification: A case study of urban agglomeration region

Monitoring urbanization processes is important because they are often accompanied by intensive landscape pattern transitions and pluralistic socioeconomic changes. To effectively monitor urban expansion and support regional planning, it is essential to develop a fast, accurate and universal urban–rural classification model, especially identifying the dynamic spatial patterns of urban, urban–rural fringe and rural areas. Although deep learning can effectively detect land cover changes, its applications in urban–rural identification have received little attention due to a lack of high-quality training datasets. In this study, we develop a novel transferable full-resolution convolutional neural network (FR-Net) to identify urban-fringe-rural areas. A large-scale training dataset was constructed using field surveys and aerial photography, and a data cube was stacked by multiple typical socio-natural indicators. We took the Beijing-Tianjin-Hebei (BTH) urban agglomeration region in China as a case study and identified spatiotemporal evolutions of urban-fringe-rural areas from 2000 to 2020. The results indicated that over the past two decades, the urban–rural fringe expanded outward with urban areas, and both areas gradually increased, with an inverted U-shaped growth rate. Accurate identification of these fringes can benefit regional urban–rural planning and social governance. Based on the identification results, complex socio-ecological impacts of urbanization could be further explored. Testing demonstrated that the developed FR-Net model has high accuracy and robustness. Our developed open-source FR-Net model exhibits transferability and can be applied to multi-scale urbanized areas.

Generating bimodal privacy-preserving data for face recognition

The performance of state-of-the-art face recognition systems depends crucially on the availability of large-scale training datasets. However, increasing privacy concerns nowadays accompany the collection and distribution of biometric data, which has already resulted in the retraction of valuable face recognition datasets. The use of synthetic data represents a potential solution, however, the generation of privacy-preserving facial images useful for training recognition models is still an open problem. Generative methods also remain bound to the visible spectrum, despite the benefits that multispectral data can provide. To address these issues, we present a novel identity-conditioned generative framework capable of producing large-scale recognition datasets of visible and near-infrared privacy-preserving face images. The framework relies on a novel identity-conditioned dual-branch style-based generative adversarial network to enable the synthesis of aligned high-quality samples of identities determined by features of a pretrained recognition model. In addition, the framework incorporates a novel filter to prevent samples of privacy-breaching identities from reaching the generated datasets and improve both identity separability and intra-identity diversity. Extensive experiments on six publicly available datasets reveal that our framework achieves competitive synthesis capabilities while preserving the privacy of real-world subjects. The synthesized datasets also facilitate training more powerful recognition models than datasets generated by competing methods or even small-scale real-world datasets. Employing both visible and near-infrared data for training also results in higher recognition accuracy on real-world visible spectrum benchmarks. Therefore, training with multispectral data could potentially improve existing recognition systems that utilize only the visible spectrum, without the need for additional sensors.

Adapting low-dose CT denoisers for texture preservation using zero-shot local noise-level matching.

On enhancing the image quality of low-dose computed tomography (LDCT), various denoising methods have achieved meaningful improvements. However, they commonly produce over-smoothed results; the denoised images tend to be more blurred than the normal-dose targets (NDCTs). Furthermore, many recent denoising methods employ deep learning(DL)-based models, which require a vast amount of CT images (or image pairs). Our goal is to address the problem of over-smoothed results and design an algorithm that works regardless of the need for a large amount of training dataset to achieve plausible denoising results. Over-smoothed images negatively affect the diagnosis and treatment since radiologists had developed clinical experiences with NDCT. Besides, a large-scale training dataset is often not available in clinical situations. To overcome these limitations, we propose locally-adaptive noise-level matching (LANCH), emphasizing the output should retain the same noise-level and characteristics to that of the NDCT without additionaltraining. We represent the NDCT image as the pixel-wisely weighted sum of an over-smoothed output from off-the-shelf denoiser (OSD) and the difference between the LDCT image and the OSD output. Herein, LANCH determines a 2D ratio map (i.e., pixel-wise weight matrix) by locally matching the noise-level of output and NDCT, where the LDCT-to-NDCT device flux (mAs) ratio reveals the NDCT noise-level. Thereby, LANCH can preserve important details in LDCT, and enhance the sharpness of the noise-free regions. Note that LANCH can enhance any LDCT denoisers without additional training data (i.e., zero-shot). The proposed method is applicable to any OSD denoisers, reporting significant texture plausibility development over the baseline denoisers in quantitative and qualitative manners. It is surprising that the denoising accuracy achieved by our method with zero-shot denoiser was comparable or superior to that of the best training-based denoisers; our result showed 1% and 33% gains in terms of SSIM and DISTS, respectively. Reader study with experienced radiologists shows significant image quality improvements, a gain of +1.18 on a five-point mean opinion scorescale. In this paper, we propose a technique to enhance any low-dose CT denoiser by leveraging the fundamental physical relationship between the x-ray flux and noise variance. Our method is capable of operating in a zero-shot condition, which means that only a single low-dose CT image is required for the enhancement process. We demonstrate that our approach is comparable or even superior to supervised DL-based denoisers that are trained using numerous CT images. Extensive experiments illustrate that our method consistently improves the performance of all tested LDCT denoisers.

AL-MRIS: An Active Learning-Based Multipath Residual Involution Siamese Network for Few-Shot Hyperspectral Image Classification

In hyperspectral image (HSI) classification scenarios, deep learning-based methods have achieved excellent classification performance, but often rely on large-scale training datasets to ensure accuracy. However, in practical applications, the acquisition of hyperspectral labeled samples is time consuming, labor intensive and costly, which leads to a scarcity of obtained labeled samples. Suffering from insufficient training samples, few-shot sample conditions limit model training and ultimately affect HSI classification performance. To solve the above issues, an active learning (AL)-based multipath residual involution Siamese network for few-shot HSI classification (AL-MRIS) is proposed. First, an AL-based Siamese network framework is constructed. The Siamese network, which has relatively low demand for sample data, is adopted for classification, and the AL strategy is integrated to select more representative samples to improve the model’s discriminative ability and reduce the costs of labeling samples in practice. Then, the multipath residual involution (MRIN) module is designed for the Siamese subnetwork to obtain the comprehensive features of the HSI. The involution operation was used to capture the fine-grained features and effectively aggregate the contextual semantic information of the HSI through dynamic weights. The MRIN module comprehensively considers the local features, dynamic features and global features through multipath residual connections, which improves the representation ability of HSIs. Moreover, a cosine distance-based contrastive loss is proposed for the Siamese network. By utilizing the directional similarity of high-dimensional HSI data, the discriminability of the Siamese classification network is improved. A large number of experimental results show that the proposed AL-MRIS method can achieve excellent classification performance with few-shot training samples, and compared with several state-of-the-art classification methods, the AL-MRIS method obtains the highest classification accuracy.

FedOSS: Federated Open Set Recognition via Inter-Client Discrepancy and Collaboration.

Open set recognition (OSR) aims to accurately classify known diseases and recognize unseen diseases as the unknown class in medical scenarios. However, in existing OSR approaches, gathering data from distributed sites to construct large-scale centralized training datasets usually leads to high privacy and security risk, which could be alleviated elegantly via the popular cross-site training paradigm, federated learning (FL). To this end, we represent the first effort to formulate federated open set recognition (FedOSR), and meanwhile propose a novel Federated Open Set Synthesis (FedOSS) framework to address the core challenge of FedOSR: the unavailability of unknown samples for all anticipated clients during the training phase. The proposed FedOSS framework mainly leverages two modules, i.e., Discrete Unknown Sample Synthesis (DUSS) and Federated Open Space Sampling (FOSS), to generate virtual unknown samples for learning decision boundaries between known and unknown classes. Specifically, DUSS exploits inter-client knowledge inconsistency to recognize known samples near decision boundaries and then pushes them beyond decision boundaries to synthesize discrete virtual unknown samples. FOSS unites these generated unknown samples from different clients to estimate the class-conditional distributions of open data space near decision boundaries and further samples open data, thereby improving the diversity of virtual unknown samples. Additionally, we conduct comprehensive ablation experiments to verify the effectiveness of DUSS and FOSS. FedOSS shows superior performance on public medical datasets in comparison with state-of-the-art approaches. The source code is available at https://github.com/CityU-AIM-Group/FedOSS.

Learning to Answer Visual Questions from Web Videos.

Recent methods for visual question answering rely on large-scale annotated datasets. Manual annotation of questions and answers for videos, however, is tedious, expensive and prevents scalability. In this work, we propose to avoid manual annotation and generate a large-scale training dataset for video question answering making use of automatic cross-modal supervision. We leverage a question generation transformer trained on text data and use it to generate question-answer pairs from transcribed video narrations. Given narrated videos, we then automatically generate the HowToVQA69M dataset with 69M video-question-answer triplets. To handle the open vocabulary of diverse answers in this dataset, we propose a training procedure based on a contrastive loss between a video-question multi-modal transformer and an answer transformer. We introduce the zero-shot VideoQA task and the VideoQA feature probe evaluation setting and show excellent results. Furthermore, our method achieves competitive results on MSRVTT-QA, ActivityNet-QA, MSVD-QA and How2QA datasets. We also show that our approach generalizes to another source of web video and text data. We generate the WebVidVQA3M dataset from videos with alt-text annotations, and show its benefits for training VideoQA models. Finally, for a detailed evaluation we introduce iVQA, a new VideoQA dataset with reduced language bias and high-quality manual annotations.

Weighted residual network for SAR automatic target recognition with data augmentation.

Decades of research have been dedicated to overcoming the obstacles inherent in synthetic aperture radar (SAR) automatic target recognition (ATR). The rise of deep learning technologies has brought a wave of new possibilities, demonstrating significant progress in the field. However, challenges like the susceptibility of SAR images to noise, the requirement for large-scale training datasets, and the often protracted duration of model training still persist. This paper introduces a novel data augmentation strategy to address these issues. Our method involves the intentional addition and subsequent removal of speckle noise to artificially enlarge the scope of training data through noise perturbation. Furthermore, we propose a modified network architecture named weighted ResNet, which incorporates residual strain controls for enhanced performance. This network is designed to be computationally efficient and to minimize the amount of training data required. Through rigorous experimental analysis, our research confirms that the proposed data augmentation method, when used in conjunction with the weighted ResNet model, significantly reduces the time needed for training. It also improves the SAR ATR capabilities. Compared to existing models and methods tested, the combination of our data augmentation scheme and the weighted ResNet framework achieves higher computational efficiency and better recognition accuracy in SAR ATR applications. This suggests that our approach could be a valuable advancement in the field of SAR image analysis.

Deep learning techniques for biometric security: A systematic review of presentation attack detection systems

Biometric technology, including finger vein, fingerprint, iris, and face recognition, is widely used to enhance security in various devices. In the past decade, significant progress has been made in improving biometric systems, thanks to advancements in deep convolutional neural networks (DCNN) and computer vision (CV), along with large-scale training datasets. However, these systems have become targets of various attacks, with presentation attacks (PAs) being prevalent and easily executed. PAs involve displaying videos, images, or full-face masks to trick biometric systems and gain unauthorized access. Many authors are currently focusing on detecting these presentation attacks (PAD) and have developed several methods, particularly those based on deep learning (DL), which have shown superior performance compared to other techniques. This survey article focuses on manuscripts related to deep learning presentation attack detection, spoof attack detection using deep learning, and anti-spoofing deep learning methods for biometric finger vein, fingerprint, iris, and face recognition. The studies were primarily sourced from four digital research libraries: ACM, Science Direct, Springer, and IEEE Xplore. The article presents a comprehensive review of DL-based PAD systems, examining recent literature on DL-based PAD methods in finger vein, fingerprint, iris, and face detection systems. Through extensive research of the literature, recent algorithms and their solutions for relevant PAD approaches are thoroughly analyzed. Additionally, the article provides a performance analysis and highlights the most promising research findings. The discussion section addresses current issues, opportunities for advancement, and potential solutions associated with deep learning-based PAD methods. This study is valuable to various community users seeking to understand the significance of this technology and its recent applicability in the development of biometric technology for deep learning.

Image restoration based on vector quantization for robotic automatic welding

For the purpose of robot automatic welding based on a laser stripe welding seam tracking system, a weld noise image restoration algorithm based on vector quantization and synthetic fractal noise is proposed to solve the problems caused by strong noise interference and limited noise dataset in welding seam tracking. The weld noise image restoration is a process of noise loss. To achieve this process, a weld noise image restoration learning model based on feature encoding, vector quantization, and feature decoding is constructed, which includes feature encoding to extract image features, vector quantization to produce noise loss, and feature decoding to complete image restoration. To enhance the generalization ability of the weld noise image restoration model and overcome the difficulty of limited data collection in on-site welding, a synthetic welding noise model is proposed based on fractal theory, multidimensional Gaussian distribution, and random region generation algorithm. A large-scale training dataset is generated by randomly initializing parameters, and the potential mapping relationship between the weld noise image and the noise-free image is constructed. Compared with the limited dataset obtained in on-site welding, the synthetic dataset makes the image restoration model more generalizable, and the similarity between the restored welding seam image and the original image reaches 0.85. In the welding seam tracking experiment, for multiple sets of different data, the average tracking accuracy of the welding seam tracking algorithm based on image restoration is 0.2 mm, which verifies that the proposed image restoration algorithm can improve the robustness of automatic welding.

Volumetric MRI with sparse sampling for MR-guided 3D motion tracking via sparse prior-augmented implicit neural representation learning.

Volumetric reconstruction of magnetic resonance imaging (MRI) from sparse samples is desirable for 3D motion tracking and promises to improve magnetic resonance (MR)-guided radiation treatment precision. Data-driven sparse MRI reconstruction, however, requires large-scale training datasets for prior learning, which is time-consuming and challenging to acquire in clinical settings. To investigate volumetric reconstruction of MRI from sparse samples of two orthogonal slices aided by sparse priors of two static 3D MRI through implicit neural representation (NeRP) learning, in support of 3D motion tracking during MR-guided radiotherapy. A multi-layer perceptron network was trained to parameterize the NeRP model of a patient-specific MRI dataset, where the network takes 4D data coordinates of voxel locations and motion states as inputs and outputs corresponding voxel intensities. By first training the network to learn the NeRP of two static 3D MRI with different breathing motion states, prior information of patient breathing motion was embedded into network weights through optimization. The prior information was then augmented from two motion states to 31 motion states by querying the optimized network at interpolated and extrapolated motion state coordinates. Starting from the prior-augmented NeRP model as an initialization point, we further trained the network to fit sparse samples of two orthogonal MRI slices and the final volumetric reconstruction was obtained by querying the trained network at 3D spatial locations. We evaluated the proposed method using 5-min volumetric MRI time series with 340ms temporal resolution for seven abdominal patients with hepatocellular carcinoma, acquired using golden-angle radial MRI sequence and reconstructed through retrospective sorting. Two volumetric MRI with inhale and exhale states respectively were selected from the first 30s of the time series for prior embedding and augmentation. The remaining 4.5-min time series was used for volumetric reconstruction evaluation, where we retrospectively subsampled each MRI to two orthogonal slices and compared model-reconstructed images to ground truth images in terms of image quality and the capability of supporting 3D target motion tracking. Across the seven patients evaluated, the peak signal-to-noise-ratio between model-reconstructed and ground truth MR images was 38.02±2.60dB and the structure similarity index measure was 0.98±0.01. Throughout the 4.5-min time period, gross tumor volume (GTV) motion estimated by deforming a reference state MRI to model-reconstructed and ground truth MRI showed good consistency. The 95-percentile Hausdorff distance between GTV contours was 2.41±0.77mm, which is less than the voxel dimension. The mean GTV centroid position difference between ground truth and model estimation was less than 1mm in all three orthogonal directions. A prior-augmented NeRP model has been developed to reconstruct volumetric MRI from sparse samples of orthogonal cine slices. Only one exhale and one inhale 3D MRI were needed to train the model to learn prior information of patient breathing motion for sparse image reconstruction. The proposed model has the potential of supporting 3D motion tracking during MR-guided radiotherapy for improved treatment precision and promises a major simplification of the workflow by eliminating the need for large-scale training datasets.

Few-shot image generation with reverse contrastive learning

Generative models, such as Generative Adversarial Networks (GANs), have recently shown remarkable capabilities in various generation tasks. However, the success of these models heavily depends on the availability of a large-scale training dataset. When the size of the training dataset is limited, the quality and diversity of the generated results suffer from severe degradation. In this paper, we propose a novel approach, Reverse Contrastive Learning (RCL), to address the problem of high-quality and diverse image generation under few-shot settings. The success of RCL benefits from a two-sided, powerful regularization. Our proposed regularization is designed based on the correlation between generated samples, which can effectively utilize the latent feature information between different levels of samples. It does not require any auxiliary information or augmentation techniques. A series of qualitative and quantitative results show that our proposed method is superior to the existing State-Of-The-Art (SOTA) methods under the few-shot setting and is still competitive under the low-shot setting, showcasing the effectiveness of RCL. Code will be released upon acceptance at https://github.com/gouayao/RCL.

Adversarially Learning Occlusions by Backpropagation for Face Recognition.

With the accomplishment of deep neural networks, face recognition methods have achieved great success in research and are now being applied at a human level. However, existing face recognition models fail to achieve state-of-the-art performance in recognizing occluded face images, which are common scenarios captured in the real world. One of the potential reasons for this is the lack of large-scale training datasets, requiring labour-intensive and costly labelling of the occlusions. To resolve these issues, we propose an Adversarially Learning Occlusions by Backpropagation (ALOB) model, a simple yet powerful double-network framework used to mitigate manual labelling by contrastively learning the corrupted features against personal identity labels, thereby maximizing the loss. To investigate the performance of the proposed method, we compared our model to the existing state-of-the-art methods, which function under the supervision of occlusion learning, in various experiments. Extensive experimentation on LFW, AR, MFR2, and other synthetic masked or occluded datasets confirmed the effectiveness of the proposed model in occluded face recognition by sustaining better results in terms of masked face recognition and general face recognition. For the AR datasets, the ALOB model outperformed other advanced methods by obtaining a 100% recognition rate for images with sunglasses (protocols 1 and 2). We also achieved the highest accuracies of 94.87%, 92.05%, 78.93%, and 71.57% TAR@FAR = 1 × 10-3 in LFW-OCC-2.0 and LFW-OCC-3.0, respectively. Furthermore, the proposed method generalizes well in terms of FR and MFR, yielding superior results in three datasets, LFW, LFW-Masked, and MFR2, and producing accuracies of 98.77%, 97.62%, and 93.76%, respectively.

Mapping annual center-pivot irrigated cropland in Brazil during the 1985–2021 period with cloud platforms and deep learning

As an important agricultural facility that supports efficient crop production and precise water management, center-pivot irrigated cropland (CPIC) plays a critical role in increasing crop yield and ensuring food security globally. Since the 1980s, CPIC has expanded significantly and has become one of the most valuable irrigated croplands in Brazil. However, national-scale and long-term CPIC products are currently unavailable, making spatiotemporal analysis for related ecological and economic factors challenging. In this study, we proposed a novel framework combining cloud platforms and deep learning algorithms to produce annual CPIC maps (ACMs) with 30 m spatial resolution in Brazil from 1985 to 2021. Leveraging all available Landsat-5, 7, and 8 images in Brazil, we generated three-band annual composite images to obtain stable image features for CPIC and alleviate the heterogeneity of CPIC. Then we constructed a large-scale training dataset covering different climatic zones of Brazil and spanning multiple years. Based on this dataset a CNN model was trained and applied to the whole of Brazil and generalized to other years. We deployed the CNN model to the cloud platform and combined it with Google Earth Engine to produce Brazil's annual CPIC Maps (ACMs). Moreover, we analyzed the spatiotemporal dynamics of CPIC and explored the impact of CPIC expansion on forests. The accuracy assessment demonstrated that our ACMs reached an average mIoU of 0.947 ± 0.021 for all study years, and the estimated CPIC area was highly consistent with official statistics. Our results indicated that Brazilian CPICs reached 1.8 × 106 ha in 2021, which was 45 times more than that in 1985. CPIC expansion occurred mainly in the states of Minas Gerais, Goiás, and Bahia over the past few decades, with an average annual increase in area of over 8,000 ha. We also found that CPIC expansion had encroached on 42,000 ha of shrubs and 25,000 ha of forests since 1985, and the impact was continuing and intensifying. To the best of our knowledge, the produced ACMs are the longest and most spatially detailed CPIC products in Brazil that reflect the CPIC spatiotemporal changes in the last four decades and can provide essential information services for Brazil’s water resource management and agricultural decision-making.