Great Generalization Ability Research Articles

Siamese Adversarial Network for image classification of heavy mineral grains

The identification of heavy mineral grains based on microscopic images can significantly reduce the time and economic cost of the identification. There are several deep learning models to realize end-to-end identification of mineral image recently. However, due to the variety and complexity of mineral images, the existing models are difficult to accurately recognize heavy mineral grains in microscopic images. Here we propose the Siamese Adversarial Network (SAN) for image classification of the heavy mineral grains, which is the first time to focus on addressing the domain difference of heavy mineral images from different basins. In more details, we design a Siamese feature encoder to extract features of both the plane-polarized and cross-polarized images as internal representation of heavy mineral grains. The features are reconstructed to discard domain-related information by adversarial training the heavy mineral classifier and domain discriminator. The identification performance of the models under the three mixed domain experiments is consistently higher than the performance under the same domain settings respectively which shows that the model we proposed achieves a great generalization ability on unseen domains.

IDNA-ABT: advanced deep learning model for detecting DNA methylation with adaptive features and transductive information maximization.

DNA methylation plays an important role in epigenetic modification, the occurrence, and the development of diseases. Therefore, identification of DNA methylation sites is critical for better understanding and revealing their functional mechanisms. To date, several machine learning and deep learning methods have been developed for the prediction of different DNA methylation types. However, they still highly rely on manual features, which can largely limit the high-latent information extraction. Moreover, most of them are designed for one specific DNA methylation type, and therefore cannot predict multiple methylation sites in multiple species simultaneously. In this study, we propose iDNA-ABT, an advanced deep learning model that utilizes adaptive embedding based on Bidirectional Encoder Representations from Transformers (BERT) together with transductive information maximization (TIM). Benchmark results show that our proposed iDNA-ABT can automatically and adaptively learn the distinguishing features of biological sequences from multiple species, and thus perform significantly better than the state-of-the-art methods in predicting three different DNA methylation types. In addition, TIM loss is proven to be effective in dichotomous tasks via the comparison experiment. Furthermore, we verify that our features have strong adaptability and robustness to different species through comparison of adaptive embedding and six handcrafted feature encodings. Importantly, our model shows great generalization ability in different species, demonstrating that our model can adaptively capture the cross-species differences and improve the predictive performance. For the convenient use of our method, we further established an online webserver as the implementation of the proposed iDNA-ABT. Our proposed iDNA-ABT and data are freely accessible via http://server.wei-group.net/iDNA_ABT and our source codes are available for downloading in the GitHub repository (https://github.com/YUYING07/iDNA_ABT). Supplementary data are available at Bioinformatics online.

Active learning-based KNN-Monte Carlo simulation on the probabilistic fracture assessment of cracked structures

The probability of fracture (POF) assessment of complex cracked structures is a difficult task in the reliability assessment of engineering structures. Due to the complexity of structure, balancing efficiency and accuracy is the top concern in POF calculation. In this study, a novel probabilistic solution method called AKNN-MCS (Active learning-based K-Nearest Neighbors-Monte Carlo Simulation) is proposed. Combining the active learning strategy and the KNN algorithm, this method could get accurate POF results using a few samples. In detail, POF calculation is treated as a classification problem. A learning function is proposed to select sample points near the limit state surface. Then the selected sample points are added into training data set T. A convergence criterion is defined to decide when to stop the enrichment of T. Thanks to the above active learning strategy, the trained KNN model could have a great generalization ability with only a few training samples required. The proposed method is validated by POF assessment of a finite thickness plate containing a surface semi-elliptical crack and POF assessment of the CT specimen. Results show that AKNN-MCS is three or four orders of magnitude more efficient than MCS for almost identical POF results.

Boosting the generalization ability of Vis-NIR-spectroscopy-based regression models through dimension reduction and transfer learning

Calibration regression models based on visible and near-infrared (Vis/NIR) spectroscopy are now widely used in the rapid non-destructive prediction of agricultural products' quality parameters. However, the distributions of products' quality parameters and spectral responses are different in various batches of products, so a calibration regression model built on one dataset might be ineffective when tested in another. How to improve the generalization ability of models is a crucial problem and is formulated as ‘Calibration Transfer’. Calibration transfer was firstly proposed to eliminate the difference of spectral responses between spectrometers, but now it has been applied in the transfer between different domains of spectral samples or different components. In this paper, we proposed two robust models with great generalization ability in the calibration transfer task, respectively using dimension reduction and transfer learning, namely SPRS (Standard normal variate, Partial least squares dimension reduction, Ridge regression, Slope/bias) and SNV-based Aug-TrAdaBoost.R2. We tested the two models in spectral datasets of tea leaves to predict the moisture content of samples, it was found that SPRS and SNV-based Aug-TrAdaBoost.R2 can reach great performance over both source and target domains across different batches, different varieties, and different classes of tea leaf samples. SPRS and SNV-based Aug-TrAdaBoost.R2 achieved R2 values of 0.9314 and 0.9895 in cross-tea-class prediction whereas traditional calibration method PLSR + S/B only achieved 0.4874. SPRS had low computation complexity and was more robust while SNV-based Aug-TrAdaBoost.R2 had higher accuracy in target domain prediction but was computation-consuming. The two proposed models showed the potentials of online automatic quality parameters prediction and high-accuracy prediction across domains of various samples.

Are CEOs Different?

ABSTRACTUsing 2,603 executive assessments, we study how CEO candidates differ from candidates for other top management positions, particularly CFOs. More than half of the variation in the 30 assessed characteristics is explained by four factors that we interpret as general ability, execution (vs. interpersonal), charisma (vs. analytical), and strategic (vs. managerial). CEO candidates have more extreme factor scores that differ significantly from those of CFO candidates. Conditional on being considered, candidates with greater general ability and interpersonal skills are more likely to be hired. These and our previous results on CEO success suggest that boards overweight interpersonal skills in hiring CEOs.

Few-Shot Learning by a Cascaded Framework With Shape-Constrained Pseudo Label Assessment for Whole Heart Segmentation.

Automatic and accurate 3D cardiac image segmentation plays a crucial role in cardiac disease diagnosis and treatment. Even though CNN based techniques have achieved great success in medical image segmentation, the expensive annotation, large memory consumption, and insufficient generalization ability still pose challenges to their application in clinical practice, especially in the case of 3D segmentation from high-resolution and large-dimension volumetric imaging. In this paper, we propose a few-shot learning framework by combining ideas of semi-supervised learning and self-training for whole heart segmentation and achieve promising accuracy with a Dice score of 0.890 and a Hausdorff distance of 18.539 mm with only four labeled data for training. When more labeled data provided, the model can generalize better across institutions. The key to success lies in the selection and evolution of high-quality pseudo labels in cascaded learning. A shape-constrained network is built to assess the quality of pseudo labels, and the self-training stages with alternative global-local perspectives are employed to improve the pseudo labels. We evaluate our method on the CTA dataset of the MM-WHS 2017 Challenge and a larger multi-center dataset. In the experiments, our method outperforms the state-of-the-art methods significantly and has great generalization ability on the unseen data. We also demonstrate, by a study of two 4D (3D+T) CTA data, the potential of our method to be applied in clinical practice.

FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search.

Deep neural networks achieve remarkable performance in many computer vision tasks. Most state-of-the-art (SOTA) semantic segmentation and object detection approaches reuse neural network architectures designed for image classification as the backbone, commonly pre-trained on ImageNet. However, performance gains can be achieved by designing network architectures specifically for detection and segmentation, as shown by recent neural architecture search (NAS) research for detection and segmentation. One major challenge though is that ImageNet pre-training of the search space representation (a.k.a. super network) or the searched networks incurs huge computational cost. In this paper, we propose a Fast Network Adaptation (FNA++) method, which can adapt both the architecture and parameters of a seed network (e.g., an ImageNet pre-trained network) to become a network with different depths, widths, or kernel sizes via a parameter remapping technique, making it possible to use NAS for segmentation and detection tasks a lot more efficiently. In our experiments, we apply FNA++ on MobileNetV2 to obtain new networks for semantic segmentation, object detection, and human pose estimation that clearly outperform existing networks designed both manually and by NAS. We also implement FNA++ on ResNets and NAS networks, which demonstrates a great generalization ability. The total computation cost of FNA++ is significantly less than SOTA segmentation and detection NAS approaches: 1737× less than DPC, 6.8× less than Auto-DeepLab, and 8.0× less than DetNAS. A series of ablation studies are performed to demonstrate the effectiveness, and detailed analysis is provided for more insights into the working mechanism. Codes are available at https://github.com/JaminFong/FNA.

Person Retrieval in Surveillance Videos Via Deep Attribute Mining and Reasoning

Person retrieval largely relies on the appearance features of pedestrians. This task is rather more difficult in surveillance videos due to the limitations of extracting robust appearance features brought by the cross-view and cross-camera data with lower image resolution, motion blur, occlusion and other kinds of image degradation. To build up a more reliable person retrieval system, recent works introduced appearance attribute models to describe and distinguish different persons with high-level semantic concepts. Despite the progress of previous works, the value of utilizing appearance attributes is still under-explored. On one hand, existing methods lack for concise and precise attribute representations that are specific for each attribute category and, in the meantime, are able to filter noisy information in irrelevant spatial locations and useless patterns. On the other hand, correlation and reasoning between different attributes are neglected, which could generate more useful information and add more robustness to the retrieval system. In this paper, we propose an Attribute Mining and Reasoning (AMR) framework which is capable to handle the issues in question. The AMR makes better use of appearance attributes with two main components. First, the AMR disentangles the representations of different attributes by localizing their spatial positions and identifying their effective patterns in a weakly supervised manner. To achieve more reliable localization, we propose the Attribute Localization Ensemble (ALE) module that is consisted of multiple localization heads and a voting mechanism. Second, we introduce the Attribute Reasoning (AR) module to correlate different attributes together with the global appearance features and discover their latent relations to generate more comprehensive descriptions of pedestrians. Extensive experiments on DukeMTMC-ReID and Market-1501 datasets demonstrate the effectiveness of the proposed AMR framework as well as its superiority over the existing state-of-the-art methods. The AMR model also shows great generalization ability on the unseen CUHK03 dataset when it is only trained on Market-1501 dataset.

An Optimized Random Forest Model and Its Generalization Ability in Landslide Susceptibility Mapping: Application in Two Areas of Three Gorges Reservoir, China

Numerous researches have been published on the application of landslide susceptibility assessment models; however,they were only applied in the same areas as the models were originated,the effect of applying the models to other areas than the origin of the models has not been explored. This study is purposed to develop an optimized random forest (RF) model with best ratios of positive-to-negative cells and 10-fold cross-validation for landslide susceptibility mapping (LSM),and then explore its generalization ability not only in the area where the model is originated but also in area other than the origin of the model. Two typical counties (Fengjie County and Wushan County) in the Three Gorges Reservoir area,China,which have the same terrain and geological conditions,were selected as an example. To begin with,landslide inventory was prepared based on field investigations,satellite images,and historical records,and 1 522 landslides were then identified in Fengjie County. 22 landslide-conditioning factors under the influence of topography,geology,environmental conditions,and human activities were prepared. Then,combined with 10-fold cross-validation,three typical ratios of positive-to-negative cells,i.e.,1:1,1:5,and 1:10,were adopted for comparative analyses. An optimized RF model (Fengjie-based model) with the best ratios of positive-to-negative cells and 10-fold cross-validation was constructed. Finally,the Fengjie-based model was applied to Fengjie County and Wushan County,and the confusion matrix and area under the receiver operating characteristic (ROC) curve value (AUC) were used to estimate the accuracy. The Fengjie-based model delivered high stability and predictive capability in Fengjie County,indicating a great generalization ability of the model to the area where the model is originated. The LSM in Wushan County generated by the Fengjie-based model had a reasonable reference value,indicating the Fengjie-based model had a great generalization ability in area other than the origin of the model. The Fengjie-based model in this study could be applied in other similar areas/countries with the same terrain and geological conditions,and a LSM may be generated without collecting landslide information for modeling,so as to reduce workload and improve efficiency in practice.

Projection wavelet weighted twin support vector regression for OFDM system channel estimation

In this paper, an efficient projection wavelet weighted twin support vector regression (PWWTSVR) based orthogonal frequency division multiplexing system (OFDM) system channel estimation algorithm is proposed. Most Channel estimation algorithms for OFDM systems are based on the linear assumption of channel model. In the proposed algorithm, the OFDM system channel is consumed to be nonlinear and fading in both time and frequency domains. The PWWTSVR utilizes pilot signals to estimate response of nonlinear wireless channel, which is the main work area of SVR. Projection axis in optimal objective function of PWWRSVR is sought to minimize the variance of the projected points due to the utilization of a priori information of training data. Different from traditional support vector regression algorithm, training samples in different positions in the proposed PWWTSVR model are given different penalty weights determined by the wavelet transform. The weights are applied to both the quadratic empirical risk term and the first-degree empirical risk term to reduce the influence of outliers. The final regressor can avoid the overfitting problem to a certain extent and yield great generalization ability for channel estimation. The results of numerical experiments show that the propose algorithm has better performance compared to the conventional pilot-aided channel estimation methods.

Prediction of miRNA targets by learning from interaction sequences.

MicroRNAs (miRNAs) are involved in a diverse variety of biological processes through regulating the expression of target genes in the post-transcriptional level. So, it is of great importance to discover the targets of miRNAs in biological research. But, due to the short length of miRNAs and limited sequence complementarity to their gene targets in animals, it is challenging to develop algorithms to predict the targets of miRNA accurately. Here we developed a new miRNA target prediction algorithm using a multilayer convolutional neural network. Our model learned automatically the interaction patterns of the experiment-validated miRNA:target-site chimeras from the raw sequence, avoiding hand-craft selection of features by domain experts. The performance on test dataset is inspiring, indicating great generalization ability of our model. Moreover, considering the stability of miRNA:target-site duplexes, our method also showed good performance to predict the target transcripts of miRNAs.

Auto-GAN: Self-Supervised Collaborative Learning for Medical Image Synthesis

In various clinical scenarios, medical image is crucial in disease diagnosis and treatment. Different modalities of medical images provide complementary information and jointly helps doctors to make accurate clinical decision. However, due to clinical and practical restrictions, certain imaging modalities may be unavailable nor complete. To impute missing data with adequate clinical accuracy, here we propose a framework called self-supervised collaborative learning to synthesize missing modality for medical images. The proposed method comprehensively utilize all available information correlated to the target modality from multi-source-modality images to generate any missing modality in a single model. Different from the existing methods, we introduce an auto-encoder network as a novel, self-supervised constraint, which provides target-modality-specific information to guide generator training. In addition, we design a modality mask vector as the target modality label. With experiments on multiple medical image databases, we demonstrate a great generalization ability as well as specialty of our method compared with other state-of-the-arts.

A pointer network based deep learning algorithm for unconstrained binary quadratic programming problem

Combinatorial optimization problems have been widely used in various fields. And many types of combinatorial optimization problems can be generalized into the model of unconstrained binary quadratic programming (UBQP). Therefore, designing an effective and efficient algorithm for UBQP problems will also contribute to solving other combinatorial optimization problems. Pointer network is an end-to-end sequential decision structure and combines with deep learning technology. With the utilization of the structural characteristics of combinatorial optimization problems and the ability to extract the rule behind the data by deep learning, pointer network has been successfully applied to solve several classical combinatorial optimization problems. In this paper, a pointer network based algorithm is designed to solve UBQP problems. The network model is trained by supervised learning (SL) and deep reinforcement learning (DRL) respectively. Trained pointer network models are evaluated by self-generated benchmark dataset and ORLIB dataset respectively. Experimental results show that pointer network model trained by SL has strong learning ability to specific distributed dataset. Pointer network model trained by DRL can learn more general distribution data characteristics. In other words, it can quickly solve problems with great generalization ability. As a result, the framework proposed in this paper for UBQP has great potential to solve large scale combinatorial optimization problems.

Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation.

With the rapid development of flexible vision sensors and visual sensor networks, computer vision tasks, such as object detection and tracking, are entering a new phase. Accordingly, the more challenging comprehensive task, including instance segmentation, can develop rapidly. Most state-of-the-art network frameworks, for instance, segmentation, are based on Mask R-CNN (mask region-convolutional neural network). However, the experimental results confirm that Mask R-CNN does not always successfully predict instance details. The scale-invariant fully convolutional network structure of Mask R-CNN ignores the difference in spatial information between receptive fields of different sizes. A large-scale receptive field focuses more on detailed information, whereas a small-scale receptive field focuses more on semantic information. So the network cannot consider the relationship between the pixels at the object edge, and these pixels will be misclassified. To overcome this problem, Mask-Refined R-CNN (MR R-CNN) is proposed, in which the stride of ROIAlign (region of interest align) is adjusted. In addition, the original fully convolutional layer is replaced with a new semantic segmentation layer that realizes feature fusion by constructing a feature pyramid network and summing the forward and backward transmissions of feature maps of the same resolution. The segmentation accuracy is substantially improved by combining the feature layers that focus on the global and detailed information. The experimental results on the COCO (Common Objects in Context) and Cityscapes datasets demonstrate that the segmentation accuracy of MR R-CNN is about 2% higher than that of Mask R-CNN using the same backbone. The average precision of large instances reaches 56.6%, which is higher than those of all state-of-the-art methods. In addition, the proposed method requires low time cost and is easily implemented. The experiments on the Cityscapes dataset also prove that the proposed method has great generalization ability.

CDL: A Cloud Detection Algorithm Over Land for MWHS-2 Based on the Gradient Boosting Decision Tree

This article presents a standalone cloud detection algorithm over the land (CDL) for microwave humidity sounder-2 (MWHS-2), which is characterized by the first operational satellite sensor measuring 118.75 GHz. The CDL is based on the advanced machine learning algorithm gradient boosting decision tree, which achieves the state-of-the-art performance on tabular data with high accuracy, fast training speed, great generalization ability, and weight factor ranking of predictors (or features). Given that the new-generation weather radar of China (CINRAD) provides improved cloud information with extensive temporal-spatial coverage, the observations from CINRAD are used to train the algorithm in this study. There are four groups of radiometric information employed to evaluate the CDL: all frequency ranges from MWHS-2 (all-algorithm), the humidity channels near 183.31 GHz (hum-algorithm), the temperature channels near 118.75 GHz (tem-algorithm), and the window channels at 89 and 150 GHz (win-algorithm). It is revealed that the tem-algorithm (around 118.75 GHz) has a superior performance for CDL along with the optimal values of most evaluation metrics. Although the all-algorithm uses all available frequencies, it shows inferior ability for CDL. Followed are the win-algorithm and hum-algorithm, and the win-algorithm performs better. The analysis also indicates that the latitude, zenith angle, and the azimuth are the top-ranking features for all four algorithms. The presented algorithm CDL can be applied in the quality control processes of assimilating microwave radiances or in the retrieval of atmospheric and surface parameters for cloud filtering.

Adversarial Learning for Joint Optimization of Depth and Ego-Motion.

In recent years, supervised deep learning methods have shown a great promise in dense depth estimation. However, massive high-quality training data are expensive and impractical to acquire. Alternatively, self-supervised learning-based depth estimators can learn the latent transformation from monocular or binocular video sequences by minimizing the photometric warp error between consecutive frames, but they suffer from the scale ambiguity problem or have difficulty in estimating precise pose changes between frames. In this paper, we propose a joint self-supervised deep learning pipeline for depth and ego-motion estimation by employing the advantages of adversarial learning and joint optimization with spatial-temporal geometrical constraints. The stereo reconstruction error provides the spatial geometric constraint to estimate the absolute scale depth. Meanwhile, the depth map with an absolute scale and a pre-trained pose network serves as a good starting point for direct visual odometry (DVO). DVO optimization based on spatial geometric constraints can result in a fine-grained ego-motion estimation with the additional backpropagation signals provided to the depth estimation network. Finally, the spatial and temporal domain-based reconstructed views are concatenated, and the iterative coupling optimization process is implemented in combination with the adversarial learning for accurate depth and precise ego-motion estimation. The experimental results show superior performance compared with state-of-the-art methods for monocular depth and ego-motion estimation on the KITTI dataset and a great generalization ability of the proposed approach.

Hyperspectral Image Classification Using Random Occlusion Data Augmentation

Convolutional neural networks (CNNs) have become a powerful tool for remotely sensed hyperspectral image (HSI) classification due to their great generalization ability and high accuracy. However, owing to the huge amount of parameters that need to be learned and to the complex nature of HSI data itself, these approaches must deal with the important problem of overfitting, which can lead to inadequate generalization and loss of accuracy. In order to mitigate this problem, in this letter, we adopt random occlusion, a recently developed data augmentation (DA) method for training CNNs, in which the pixels of different rectangular spatial regions in the HSI are randomly occluded, generating training images with various levels of occlusion and reducing the risk of overfitting. Our results with two well-known HSIs reveal that the proposed method helps to achieve better classification accuracy with low computational cost.

Energy-Efficient Power Control in Wireless Networks With Spatial Deep Neural Networks

The energy-efficient power control of interfering links in a large wireless network is a challenging task. In this paper, we propose a deep learning based power control scheme, termed PowerNet, that uses the devices' geographical location information (GLI). We show that it is possible to bypass the complex channel estimation process and directly perform power control with GLI when the channel state information (CSI) can be viewed as a function of distance dependent path-loss. The time consuming and complex channel estimation process can thus be avoided. Moreover, with a proper training, PowerNet transforms the on-line complexity to off-line training, and is amenable for real-time services. Different from conventional deep neural network (DNN) that adopts fully connected structure, the proposed PowerNet leverages convolutional layers to better capture the interference pattern across different links in large wireless networks and utilizes deep residual learning to further enhance its robustness. Simulation results demonstrate that PowerNet can achieve a near-optimal performance at a remarkably high speed without explicit channel estimation. PowerNet also exhibits a great generalization ability in terms of problem sizes and channel fading types.

Wavelet transform-based weighted $$\nu$$-twin support vector regression

In this paper, an efficient wavelet transform-based weighted $$\nu$$-twin support vector regression (WTWTSVR) is proposed, inspired by twin support vector regression (TSVR) and $$\nu$$-twin support vector machine-based regression. TSVR and its improved models work faster than support vector regression because they solve a pair of smaller-sized quadratic programming problems. However, they give the same emphasis to the training samples, i.e., all of the training samples share the same weights, and prior information is not used, which leads to the degradation of performance. Motivated by this, samples in different positions in the proposed WTWTSVR model are given different penalty weights determined by the wavelet transform. The weights are applied to both the quadratic empirical risk term and the first-degree empirical risk term to reduce the influence of outliers. The final regressor can avoid the overfitting problem to a certain extent and yield great generalization ability. Numerical experiments on artificial datasets and benchmark datasets demonstrate the feasibility and validity of our proposed algorithm.

Online Fault Diagnosis of External Short Circuit for Lithium-Ion Battery Pack

Battery safety is one of the most crucial issues in the utilization of lithium-ion batteries (LiBs) for all-climate electric vehicles. Short circuit, overcharge, and overheat are three common field failures of LiBs. In this paper, online fault diagnosis for external short circuit (ESC) of LiB packs is investigated. The experiments are carried out to obtain and compare ESC characteristics of 18650-type NMC battery pack and single cell. Based on the analysis of experimental results, a two-step equivalent circuit model is established to describe the ESC process and an online model-based scheme is proposed to diagnose ESC faults of battery packs. The proposed scheme is evaluated by experimental data. The results show that it can effectively diagnose ESC faults in 3.5 s after their occurrences with the terminal voltage error less than 25 mV. The proposed scheme has shown great generalization ability. ESC faults of battery packs under different number of cells connected in series and unavailable current information can also be diagnosed at the terminal voltage error less than 48 and 60 mV, respectively.