Adversarial Network Research Articles

Label Augmentation to Improve Generalization of Deep Learning Semantic Segmentation of Laparoscopic Images

Training Deep Neural Networks to solve semantic segmentation is a challenging problem with small-size labeled datasets, leading to overfitting. This is especially problematic in medical images, and in particular, in laparoscopic surgery images. In this context, ground-truth segmentation labels are available only for a small set of images with few patients. Besides, inter-patient variability is very high in practice. Models trained for a specific setup and a set of patients usually performs poorly when deployed in a new environment. This work proposes a new training strategy that improves the generalization accuracy of current state-of-the-art semantic segmentation methods applied to laparoscopic images. Our approach is based on training a discriminator network, which learns to detect segmentation errors, producing a dense segmentation error map. Unlike in adversarial networks, we train the discriminator offline by synthetically altering ground-truth segmentation labels with simple morphological and geometric operations. We then use the discriminator to train a segmentation neural network, by minimizing the discriminator predicted error jointly with a standard segmentation loss. This strategy results in segmentation models that are significantly more accurate when tested in unseen images than those only relying on data augmentation. This technique is very suitable to boost the performance of any state-of-the-art segmentation network and can be combined with other data augmentation strategies. This paper evaluates and validates our proposal by training and testing common state-of-the-art segmentation models in publicly available semantic segmentation datasets, specialized in laparoscopic and endoscopic surgery. The results show that our methods are effective, obtaining a significant improvement in terms of segmentation accuracy, especially in challenging small-size datasets.

Towards Accurate and Robust Domain Adaptation Under Multiple Noisy Environments.

In many non-stationary environments, machine learning algorithms usually confront the distribution shift scenarios. Previous domain adaptation methods have achieved great success. However, they would lose algorithm robustness in multiple noisy environments where the examples of source domain become corrupted by label noise, feature noise, or open-set noise. In this paper, we report our attempt toward achieving noise-robust domain adaptation. We first give a theoretical analysis and find that different noises have disparate impacts on the expected target risk. To eliminate the effect of source noises, we propose offline curriculum learning minimizing a newly-defined empirical source risk. We suggest a proxy distribution-based margin discrepancy to gradually decrease the noisy distribution distance to reduce the impact of source noises. We propose an energy estimator for assessing the outlier degree of open-set-noise examples to defeat the harmful influence. We also suggest robust parameter learning to mitigate the negative effect further and learn domain-invariant feature representations. Finally, we seamlessly transform these components into an adversarial network that performs efficient joint optimization for them. A series of empirical studies on the benchmark datasets and the COVID-19 screening task show that our algorithm remarkably outperforms the state-of-the-art, with over 10% accuracy improvements in some transfer tasks.

Image quality estimation based on visual perception using adversarial networks in autonomous vehicles

Image quality estimation based on visual perception using adversarial networks in autonomous vehicles

An Improved Spatiotemporal Fusion Algorithm for Monitoring Daily Snow Cover Changes With High Spatial Resolution

Considering the tradeoff between spatial resolution and temporal resolution, spatiotemporal fusion has become a promising technique to monitor snow cover dynamics with both high spatial and temporal resolutions. The representative spatiotemporal fusion methods, e.g. Spatial Temporal Data Fusion Approach (STDFA), usually exist obvious phenomenon of spectral distortion when the surface reflectance changes nonlinearly, which affects the quality of the spatiotemporal fusion image. To address this issue, an effective STDFA-Matching-Pix2pix-Generative Adversarial Network (SMPG) algorithm combining the unmixing-based method, deep learning method, pre-matching and post-matching module is proposed to reduce the spectral distortion of STDFA fusion image. The high-temporal-low-spatial (HTLS) resolution MOD09GA data and high-spatial-low-temporal resolution (HSLT) Landsat 8 data are selected in this study. SMPG algorithm is firstly employed to obtain daily high-spatial-high-temporal (HSHT) images, and then daily snow cover results with a spatial resolution of 30 m are obtained by calculating the normalized difference snow index (NDSI). SMPG algorithm is further compared with STDFA, Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), Flexible Spatiotemporal DAta Fusion (FSDAF), Swin SpatioTemporal Fusion Model (SwinSTFM), and Generative Adversarial Network-based SpatioTemporal Fusion Model (GAN-STFM). The experimental results indicate that the proposed algorithm yields better overall performance in daily spatiotemporal fusion image and snow cover result with a spatial resolution of 30 m. The mean correlation coefficient (CC) of SMPG can achieve 0.962, which is 0.06-0.36 higher than that of other spatiotemporal fusion methods. The error between the percentage of snow cover area obtained through SMPG and validation data is within 0.84%.

Self-Attention-Based Adversarial Autoencoder Network for Surface Defect Detection

Self-Attention-Based Adversarial Autoencoder Network for Surface Defect Detection

Stock Ranking Prediction Based on an Adversarial Game Neural Network

Stock Ranking Prediction Based on an Adversarial Game Neural Network

Колоризация эскизов на основе генеративно-состязательных нейронных сетей

Generative adversarial neural networks (GAN) successfully perform automatic colorization of character sketches. Such models can be further improved in such aspects as increasing the throughput, improving the colorization quality, and reducing the model size. Steps aimed at improving the colorization quality are taken. Known solutions are reviewed, and an initial GAN model with eight blocks in the generator encoder and decoder is developed and trained based on the review results. The second GAN model is obtained on the basis of the first one by including residual blocks in the generator encoder blocks with simultaneously using the attention blocks and residual blocks in the generator decoder. The third GAN model has been developed based on the second one: one block is added to the generator encoder and decoder. All models, including the initial and modified ones, have been trained on the same data set. The models have been trained either with or without using additional information about the image color (the color palette of the reference image or color hint labels). The trained models are evaluated with respect to the quality of the images they generate (colored sketches), determined by the Frechet Inception Distance metric. All modified GAN models generate images with quality superior to that of the initial model.

Adversarial Transformation Network with Adaptive Perturbations for Generating Adversarial Examples

Adversarial Transformation Network with Adaptive Perturbations for Generating Adversarial Examples

Simultaneously Improve Transferability and Discriminability for Adversarial Domain Adaptation

Although adversarial domain adaptation enhances feature transferability, the feature discriminability will be degraded in the process of adversarial learning. Moreover, most domain adaptation methods only focus on distribution matching in the feature space; however, shifts in the joint distributions of input features and output labels linger in the network, and thus, the transferability is not fully exploited. In this paper, we propose a matrix rank embedding (MRE) method to enhance feature discriminability and transferability simultaneously. MRE restores a low-rank structure for data in the same class and enforces a maximum separation structure for data in different classes. In this manner, the variations within the subspace are reduced, and the separation between the subspaces is increased, resulting in improved discriminability. In addition to statistically aligning the class-conditional distribution in the feature space, MRE forces the data of the same class in different domains to exhibit an approximate low-rank structure, thereby aligning the class-conditional distribution in the label space, resulting in improved transferability. MRE is computationally efficient and can be used as a plug-and-play term for other adversarial domain adaptation networks. Comprehensive experiments demonstrate that MRE can advance state-of-the-art domain adaptation methods.

A Prediction Method for the RUL of Equipment for Missing Data

We present a prediction framework to estimate the remaining useful life (RUL) of equipment based on the generative adversarial imputation net (GAIN) and multiscale deep convolutional neural network and long short-term memory (MSDCNN-LSTM). The method we proposed addresses the problem of missing data caused by sensor failures in engineering applications. First, a binary matrix is used to adjust the proportion of “0” to simulate the number of missing data in the engineering environment. Then, the GAIN model is used to impute the missing data and approximate the true sample distribution. Finally, the MSDCNN-LSTM model is used for RUL prediction. Experiments are carried out on the commercial modular aero-propulsion system simulation (C-MAPSS) dataset to validate the proposed method. The prediction results show that the proposed method outperforms other methods when packet loss occurs, showing significant improvements in the root mean square error (RMSE) and the score function value.

Few-Shot Abnormal Network Traffic Detection Based on Multi-scale Deep-CapsNet and Adversarial Reconstruction

Detecting various attacks and abnormal traffic in the network is extremely important to network security. Existing detection models used massive amounts of data to complete abnormal traffic detection. However, few-shot attack samples can only be intercepted in certain special scenarios. In addition, the discrimination of traffic attributes will also be affected by the change of feature attitude. But the traditional neural network model cannot detect this kind of attitude change. Therefore, the accuracy and efficiency of few-shot sample abnormal traffic detection are very low. In this paper, we proposed a few-shot abnormal network traffic detection method. It was composed of the multi-scale Deep-CapsNet and adversarial reconstruction. First, we designed an improved EM vector clustering of the Deep-CapsNet. The attitude transformation matrix was used to complete the prediction from low-level to high-level features. Second, a multi-scale convolutional capsule was designed to optimize the Deep-CapsNet. Third, an adversarial reconstruction classification network (ARCN) was proposed. The supervised source data classification and the unsupervised target data reconstruction were achieved. Moreover, we proposed an adversarial training strategy, which alleviated the noise interference during reconstruction. Fourth, the few-shot sample classification were obtained by combining multi-scale Deep-CapsNet and adversarial reconstruction. The ICSX2012 and CICIDS2017 datasets were used to verify the performance. The experimental results show that our method has better training performance. Moreover, it has the highest accuracy in two-classification and multi-classification. Especially it has good anti-noise performance and short running time, which can be used for real-time few-shot abnormal network traffic detection.

Generation of Realistic Navigation Paths for Web Site Testing Using RNN and GAN

For applications that have not yet been launched, a reliable way for creating online navigation logs may be crucial, enabling developers to test their products as though they were being used by real users. This might lead to faster and lower-cost program testing and enhancement, especially in terms of usability and interaction. In this work we propose a method for using deep learning approaches such as recurrent neural networks (RNN) and generative adversarial neural networks (GANN) to produce high-quality weblogs. Eventually, we can utilize the created data for automated testing and improvement of Web sites prior to their release with the aid of model-driven development tools such as IFML Editor.

Machine Translation in Low-Resource Languages by an Adversarial Neural Network

Existing Sequence-to-Sequence (Seq2Seq) Neural Machine Translation (NMT) shows strong capability with High-Resource Languages (HRLs). However, this approach poses serious challenges when processing Low-Resource Languages (LRLs), because the model expression is limited by the training scale of parallel sentence pairs. This study utilizes adversary and transfer learning techniques to mitigate the lack of sentence pairs in LRL corpora. We propose a new Low resource, Adversarial, Cross-lingual (LAC) model for NMT. In terms of the adversary technique, LAC model consists of a generator and discriminator. The generator is a Seq2Seq model that produces the translations from source to target languages, while the discriminator measures the gap between machine and human translations. In addition, we introduce transfer learning on LAC model to help capture the features in rare resources because some languages share the same subject-verb-object grammatical structure. Rather than using the entire pretrained LAC model, we separately utilize the pretrained generator and discriminator. The pretrained discriminator exhibited better performance in all experiments. Experimental results demonstrate that the LAC model achieves higher Bilingual Evaluation Understudy (BLEU) scores and has good potential to augment LRL translations.

Explicit feature disentanglement for visual place recognition across appearance changes

In the long-term deployment of mobile robots, changing appearance brings challenges for localization. When a robot travels to the same place or restarts from an existing map, global localization is needed, where place recognition provides coarse position information. For visual sensors, changing appearances such as the transition from day to night and seasonal variation can reduce the performance of a visual place recognition system. To address this problem, we propose to learn domain-unrelated features across extreme changing appearance, where a domain denotes a specific appearance condition, such as a season or a kind of weather. We use an adversarial network with two discriminators to disentangle domain-related features and domain-unrelated features from images, and the domain-unrelated features are used as descriptors in place recognition. Provided images from different domains, our network is trained in a self-supervised manner which does not require correspondences between these domains. Besides, our feature extractors are shared among all domains, making it possible to contain more appearance without increasing model complexity. Qualitative and quantitative results on two toy cases are presented to show that our network can disentangle domain-related and domain-unrelated features from given data. Experiments on three public datasets and one proposed dataset for visual place recognition are conducted to illustrate the performance of our method compared with several typical algorithms. Besides, an ablation study is designed to validate the effectiveness of the introduced discriminators in our network. Additionally, we use a four-domain dataset to verify that the network can extend to multiple domains with one model while achieving similar performance.

A Comprehensive Review on Image Synthesis with Adversarial Networks: Theory, Literature, and Applications

A Comprehensive Review on Image Synthesis with Adversarial Networks: Theory, Literature, and Applications

A proof of concept for machine learning-based virtual knapping using neural networks

Prehistoric stone tools are an important source of evidence for the study of human behavioural and cognitive evolution. Archaeologists use insights from the experimental replication of lithics to understand phenomena such as the behaviours and cognitive capacities required to manufacture them. However, such experiments can require large amounts of time and raw materials, and achieving sufficient control of key variables can be difficult. A computer program able to accurately simulate stone tool production would make lithic experimentation faster, more accessible, reproducible, less biased, and may lead to reliable insights into the factors that structure the archaeological record. We present here a proof of concept for a machine learning-based virtual knapping framework capable of quickly and accurately predicting flake removals from 3D cores using a conditional adversarial neural network (CGAN). We programmatically generated a testing dataset of standardised 3D cores with flakes knapped from them. After training, the CGAN accurately predicted the length, volume, width, and shape of these flake removals using the intact core surface information alone. This demonstrates the feasibility of machine learning for investigating lithic production virtually. With a larger training sample and validation against archaeological data, virtual knapping could enable fast, cheap, and highly-reproducible virtual lithic experimentation.

Machine Learning-Based Condition Monitoring for PV Systems: State of the Art and Future Prospects

To ensure the continuity of electric power generation for photovoltaic systems, condition monitoring frameworks are subject to major enhancements. The continuous uniform delivery of electric power depends entirely on a well-designed condition maintenance program. A just-in-time task to deal with several naturally occurring faults can be correctly undertaken via the cooperation of effective detection, diagnosis, and prognostic analyses. Therefore, the present review first outlines different failure modes to which all photovoltaic systems are subjected, in addition to the essential integrated detection methods and technologies. Then, data-driven paradigms, and their contribution to solving this prediction problem, are also explored. Accordingly, this review primarily investigates the different learning architectures used (i.e., ordinary, hybrid, and ensemble) in relation to their learning frameworks (i.e., traditional and deep learning). It also discusses the extension of machine learning to knowledge-driven approaches, including generative models such as adversarial networks and transfer learning. Finally, this review provides insights into different works to highlight various operating conditions and different numbers and types of failures, and provides links to some publicly available datasets in the field. The clear organization of the abundant information on this subject may result in rigorous guidelines for the trends adopted in the future.

Robust Multimodal Representation Learning With Evolutionary Adversarial Attention Networks

Multimodal representation learning is beneficial for many multimedia-oriented applications, such as social image recognition and visual question answering. The different modalities of the same instance (e.g., a social image and its corresponding description) are usually correlational and complementary. Most existing approaches for multimodal representation learning are not effective to model the deep correlation between different modalities. Moreover, it is difficult for these approaches to deal with the noise within social images. In this article, we propose a deep learning-based approach named evolutionary adversarial attention networks (EAANs), which combines the attention mechanism with adversarial networks through evolutionary training, for robust multimodal representation learning. Specifically, a two-branch visual-textual attention model is proposed to correlate visual and textual content for joint representation. Then adversarial networks are employed to impose regularization upon the representation by matching its posterior distribution to the given priors. Finally, the attention model and adversarial networks are integrated into an evolutionary training framework for robust multimodal representation learning. Extensive experiments have been conducted on four real-world datasets, including PASCAL, MIR, CLEF, and NUS-WIDE. Substantial performance improvements on the tasks of image classification and tag recommendation demonstrate the superiority of the proposed approach.

Automatic Meniscus Segmentation Using Adversarial Learning-Based Segmentation Network with Object-Aware Map in Knee MR Images.

Meniscus segmentation from knee MR images is an essential step when analyzing the length, width, height, cross-sectional area, surface area for meniscus allograft transplantation using a 3D reconstruction model based on the patient’s normal meniscus. In this paper, we propose a two-stage DCNN that combines a 2D U-Net-based meniscus localization network with a conditional generative adversarial network-based segmentation network using an object-aware map. First, the 2D U-Net segments knee MR images into six classes including bone and cartilage with whole MR images at a resolution of 512 × 512 to localize the medial and lateral meniscus. Second, adversarial learning with a generator based on the 2D U-Net and a discriminator based on the 2D DCNN using an object-aware map segments the meniscus into localized regions-of-interest with a resolution of 64 × 64. The average Dice similarity coefficient of the meniscus was 85.18% at the medial meniscus and 84.33% at the lateral meniscus; these values were 10.79%p and 1.14%p, and 7.78%p and 1.12%p higher than the segmentation method without adversarial learning and without the use of an object-aware map with the Dice similarity coefficient at the medial meniscus and lateral meniscus, respectively. The proposed automatic meniscus localization through multi-class can prevent the class imbalance problem by focusing on local regions. The proposed adversarial learning using an object-aware map can prevent under-segmentation by repeatedly judging and improving the segmentation results, and over-segmentation by considering information only from the meniscus regions. Our method can be used to identify and analyze the shape of the meniscus for allograft transplantation using a 3D reconstruction model of the patient’s unruptured meniscus.

Political and Technical Information Exchange in Adversarial Policy Networks (WITHDRAWN)

Information exchange is one of the essential elements of learning processes in policy subsystems or networks. The role of scientific and technical information in structuring policy subsystems or networks is a foundational tenet of the Advocacy Coalition Framework. Previous research focuses on trust and political influence as drivers of information exchange between actors in networks or subsystems. Few studies examine how low levels of trust and political power disparity, common in adversarial policy networks, are associated with information exchange. Using the quadratic assignment procedure multiple regression, we analyze political and technical information flows in an adversarial local hydraulic fracturing policy network in New York. We found that low levels of trust between policy actors increase political information exchange, and a larger power disparity constraints political information exchange, despite existing informal collaborative relationships. Also, low levels of trust strengthen technical information exchange between collaborative actors in the network. These findings extend research on policy networks and ACF scholarship by pinpointing novel contingencies of political and technical information exchange in an adversarial network.