Distribution Of Training Samples Research Articles

MSO‐DETR: Metric space optimization for few‐shot object detection

AbstractIn the metric‐based meta‐learning detection model, the distribution of training samples in the metric space has great influence on the detection performance, and this influence is usually ignored by traditional meta‐detectors. In addition, the design of metric space might be interfered with by the background noise of training samples. To tackle these issues, we propose a metric space optimisation method based on hyperbolic geometry attention and class‐agnostic activation maps. First, the geometric properties of hyperbolic spaces to establish a structured metric space are used. A variety of feature samples of different classes are embedded into the hyperbolic space with extremely low distortion. This metric space is more suitable for representing tree‐like structures between categories for image scene analysis. Meanwhile, a novel similarity measure function based on Poincaré distance is proposed to evaluate the distance of various types of objects in the feature space. In addition, the class‐agnostic activation maps (CCAMs) are employed to re‐calibrate the weight of foreground feature information and suppress background information. Finally, the decoder processes the high‐level feature information as the decoding of the query object and detects objects by predicting their locations and corresponding task encodings. Experimental evaluation is conducted on Pascal VOC and MS COCO datasets. The experiment results show that the effectiveness of the authors’ method surpasses the performance baseline of the excellent few‐shot detection models.

Twice Class Bias Correction for Imbalanced Semi-supervised Learning

Differing from traditional semi-supervised learning, class-imbalanced semi-supervised learning presents two distinct challenges: (1) The imbalanced distribution of training samples leads to model bias towards certain classes, and (2) the distribution of unlabeled samples is unknown and potentially distinct from that of labeled samples, which further contributes to class bias in the pseudo-labels during the training. To address these dual challenges, we introduce a novel approach called Twice Class Bias Correction (TCBC). We begin by utilizing an estimate of the class distribution from the participating training samples to correct the model, enabling it to learn the posterior probabilities of samples under a class-balanced prior. This correction serves to alleviate the inherent class bias of the model. Building upon this foundation, we further estimate the class bias of the current model parameters during the training process. We apply a secondary correction to the model's pseudo-labels for unlabeled samples, aiming to make the assignment of pseudo-labels across different classes of unlabeled samples as equitable as possible. Through extensive experimentation on CIFAR10/100-LT, STL10-LT, and the sizable long-tailed dataset SUN397, we provide conclusive evidence that our proposed TCBC method reliably enhances the performance of class-imbalanced semi-supervised learning.

DCDiff: Dual-Granularity Cooperative Diffusion Models for Pathology Image Analysis.

Whole Slide Images (WSIs) are paramount in the medical field, with extensive applications in disease diagnosis and treatment. Recently, many deep-learning methods have been used to classify WSIs. However, these methods are inadequate for accurately analyzing WSIs as they treat regions in WSIs as isolated entities and ignore contextual information. To address this challenge, we propose a novel Dual-Granularity Cooperative Diffusion Model (DCDiff) for the precise classification of WSIs. Specifically, we first design a cooperative forward and reverse diffusion strategy, utilizing fine-granularity and coarse-granularity to regulate each diffusion step and gradually improve context awareness. To exchange information between granularities, we propose a coupled U-Net for dual-granularity denoising, which efficiently integrates dual-granularity consistency information using the designed Fine- and Coarse-granularity Cooperative Aware (FCCA) model. Ultimately, the cooperative diffusion features extracted by DCDiff can achieve cross-sample perception from the reconstructed distribution of training samples. Experiments on three public WSI datasets show that the proposed method can achieve superior performance over state-of-the-art methods. The code is available at https://github.com/hemo0826/DCDiff.

Bounded exponential loss function based AdaBoost ensemble of OCSVMs

As a commonly used ensemble method, AdaBoost has drawn much consideration in the field of machine learning. However, AdaBoost is highly sensitive to outliers. The performance of AdaBoost may be greatly deteriorated when the training samples are polluted by outliers. For binary and multi-class classifications, there have emerged many approaches to improving the robustness of AdaBoost against outliers. Unfortunately, there are too few researches on enhancing the robustness of AdaBoost against outliers in the case of one-class classification. In this study, the exponential loss function of AdaBoost is replaced by a more robust one to improve the anti-outlier ability of the conventional AdaBoost based ensemble of one-class support vector machines (OCSVMs). Furthermore, based on the redesigned loss function, the update formulae for the weights of base classifiers and the probability distribution of training samples are reformulated towards the AdaBoost ensemble of OCSVMs. The empirical error upper bound is derived from the theoretical viewpoint. Experimental outcomes upon the artificial and benchmark data sets show that the presented ensemble approach is more robust against outliers than its related methods.

Desirable molecule discovery via generative latent space exploration

Drug molecule design is a classic research topic. Drug experts traditionally design molecules relying on their experience. Manual drug design is time-consuming and may produce low-efficacy and off-target molecules. With the popularity of deep learning, drug experts are beginning to use generative models to design drug molecules. A well-trained generative model can learn the distribution of training samples and infinitely generate drug-like molecules similar to the training samples. The automatic process improves design efficiency. However, most existing methods focus on proposing and optimizing generative models. How to discover ideal molecules from massive candidates is still an unresolved challenge. We propose a visualization system to discover ideal drug molecules generated by generative models. In this paper, we investigated the requirements and issues of drug design experts when using generative models, i.e., generating molecular structures with specific constraints and finding other molecular structures similar to potential drug molecular structures. We formalized the first problem as an optimization problem and proposed using a genetic algorithm to solve it. For the second problem, we proposed using a neighborhood sampling algorithm based on the continuity of the latent space to find solutions. We integrated the proposed algorithms into a visualization tool, and a case study for discovering potential drug molecules to make KOR agonists and experiments demonstrated the utility of our approach.

AdaBoost model-based rural governance co-integration and sharing mechanism

Abstract In this paper, we first study the implementation process of the AdaBoost model, initialize the weight distribution of training samples so that each sample weight is equal, then generate weak classifiers and calculate the minimum weighted error rate corresponding to each weak classifier, and then linearly combine the basic classifiers into strong classifiers by using weighted summation. Then the basic classifier weighting parameter calculation formula is improved by combining the characteristics of the logistics distribution function. Finally, the algorithm is used to analyze the rural governance willingness of river country residents, as well as to analyze the rural governance needs of rural residents at three levels: living conditions, public health and service facilities, to identify the types of rural governance, and to explore the rural governance patterns of different types of farm households. At the level of living conditions, farmers who wish to improve cooking fuel in 54 households, heating conditions in 61 households, bathing conditions in 40 households and family toilets in 53 households account for 43%, 49%, 32% and 42%, respectively. A total of 80 farming households, accounting for 29.41% of the research farming households, were optimized and upgraded, including 66 households in the key development area and 14 households in the production and living security area, accounting for 82.5% and 17.5% of the optimized and upgraded type, respectively. The research method of this paper has important references and reference significance for the construction of rural governance.

Transfer Discriminant Softmax Regression with Weighted MMD

As an efficient classical machine learning classifier, the Softmax regression uses cross-entropy as the loss function. Therefore, it has high accuracy in classification. However, when there is inconsistency between the distribution of training samples and test samples, the performance of traditional Softmax regression models will degrade. A transfer discriminant Softmax regression model called Transfer Discriminant Softmax Regression with Weighted MMD (TDS-WMMD) is proposed in this paper. With this method, the Weighted Maximum Mean Divergence (WMMD) is introduced into the objective function to reduce the marginal distribution and conditional distribution between domains both locally and globally, realizing the cross domain transfer of knowledge. In addition, to further improve the classification performance of the model, Linear Discriminant Analysis (LDA) is added to the label iteration refinement process to improve the class separability of the designed method by keeping the same kind of samples together and the different kinds of samples repeling each other. Finally, after conducting classification experiments on several commonly used public transfer learning datasets, the results verify that the designed method can enhance the knowledge transfer ability of the Softmax regression model, and deliver higher classification performance compared with other current transfer learning classifiers.

Kernel Robust Hypothesis Testing

The problem of robust hypothesis testing is studied, where under the null and the alternative hypotheses, the data-generating distributions are assumed to be in some uncertainty sets, and the goal is to design a test that performs well under the worst-case distributions over the uncertainty sets. In this paper, uncertainty sets are constructed in a data-driven manner using kernel method, i.e., they are centered around empirical distributions of training samples from the null and alternative hypotheses, respectively; and are constrained via the distance between kernel mean embeddings of distributions in the reproducing kernel Hilbert space, i.e., maximum mean discrepancy (MMD). The Bayesian setting and the Neyman-Pearson setting are investigated. For the Bayesian setting where the goal is to minimize the worst-case error probability, an optimal test is firstly obtained when the alphabet is finite. When the alphabet is infinite, a tractable approximation is proposed to quantify the worst-case average error probability, and a kernel smoothing method is further applied to design test that generalizes to unseen samples. A direct robust kernel test is also proposed and proved to be exponentially consistent. For the Neyman-Pearson setting, where the goal is to minimize the worst-case probability of miss detection subject to a constraint on the worst-case probability of false alarm, an efficient robust kernel test is proposed and is shown to be asymptotically optimal. Numerical results are provided to demonstrate the performance of the proposed robust tests.

Supervised Classification of UML Class Diagrams Based on F-KNB

Often most software development doesn’t start from scratch but applies previously developed artifacts. These reusable artifacts are involved in various phases of the software life cycle, ranging from requirements to maintenance. Software design as the high level of software development process has an important impact on the following stages, so its reuse is gaining more and more attention. Unified modeling language (UML) class diagram as a modeling tool has become a de facto standard of software design, and thus its reuse also becomes a concern accordingly. So far, the related research on the reuse of UML class diagrams has focused on matching and retrieval. As a large number of class diagrams enter the repository for reuse, classification has become an essential task. The classification is divided into unsupervised classification (also known as clustering) and supervised classification. In our previous work, we discussed the clustering of UML class diagrams. In this paper, we focus on only the supervised classification of UML class diagrams and propose a supervised classification method. A novel ensemble classifier F-KNB combining both dependent and independent construction ideas is built. The similarity of class diagrams is described, in which the semantic, structural and hybrid matching is defined, respectively. The extracted feature elements are used in base classifiers F-KNN and F-NBs that are constructed based on improved K-nearest neighbors (KNNs) and Naive Bayes (NB), respectively. A series of experimental results show that the proposed ensemble classifier F-KNB shows a good classification quality and efficiency under the condition of variable size and distribution of training samples.

A Symmetrical Siamese Network Framework With Contrastive Learning for Pose-Robust Face Recognition.

Face recognition has achieved remarkable success owing to the development of deep learning. However, most of existing face recognition models perform poorly against pose variations. We argue that, it is primarily caused by pose-based long-tailed data - imbalanced distribution of training samples between profile faces and near-frontal faces. Additionally, self-occlusion and nonlinear warping of facial textures caused by large pose variations also increase the difficulty in learning discriminative features of profile faces. In this study, we propose a novel framework called Symmetrical Siamese Network (SSN), which can simultaneously overcome the limitation of pose-based long-tailed data and pose-invariant features learning. Specifically, two sub-modules are proposed in the SSN, i.e., Feature-Consistence Learning sub-Net (FCLN) and Identity-Consistence Learning sub-Net (ICLN). For FCLN, the inputs are all face images on training dataset. Inspired by the contrastive learning, we simulate pose variations of faces and constrain the model to focus on the consistent areas between the original face image and its corresponding virtual pose face images. For ICLN, only profile images are used as inputs, and we propose to adopt Identity Consistence Loss to minimize the intra-class feature variation across different poses. The collaborative learning of two sub-modules guarantees that the parameters of network are updated in a relatively equal probability between near-frontal face images and profile images, so that the pose-based long-tailed problem can be effectively addressed. The proposed SSN shows comparable results over the state-of-the-art methods on several public datasets. In this study, LightCNN is selected as the backbone of SSN, and existing popular networks also can be used into our framework for pose-robust face recognition.

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models.

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.

Deep-Learning Based Injection Attacks Detection Method for HTTP

In the context of the new era of high digitization and informatization, the emergence of the internet and artificial intelligence technologies has profoundly changed people’s lifestyles. The traditional cyber attack detection has become increasingly weak in the context of the increasingly complex network environment in the new era, and deep learning technology has begun to play a significant role in the field of network security. There are many kinds of attacks against web applications, which are very harmful, including SQL (Structured Query Language) injection, XSS (Cross-Site Scripting), and command injection. Based on the detection of SQL injection and XSS attacks, this paper combines the detection of command injection attacks, which are also very harmful, and proposes a multi-classification detection method for web injection attacks. We extract features in the URL (Uniform Resource Locator) and request body of HTTP (Hyper Text Transfer Protocol) requests and combine deep learning technology to build a multi-classification model for injection attacks. Firstly, aiming at the problem of imbalanced distribution of training samples and low detection accuracy of command injection attack, a sample generation method is proposed. The experimental results show that the proposed method ensures a higher detection rate of command injection attacks and lower false alarms. Secondly, we propose a more expressive feature fusion model, which effectively combines the features extracted by deep learning with the discrete features extracted manually. The experimental results show that the feature fusion model proposed in this work is more effective compared with a single deep learning model. The accuracy of the model is improved by about 1%.

Center Prediction Loss for Re-identification

The training loss function that enforces certain training sample distribution patterns plays a critical role in building a re-identification (ReID) system. Besides the basic requirement of discrimination, i.e., the features corresponding to different identities should not be mixed, additional intra-class distribution constraints, such as features from the same identities should be close to their centers, have been adopted to construct losses. Despite the advances of various new loss functions, it is still challenging to strike the balance between the need of reducing the intra-class variation and allowing certain distribution freedom. Traditional intra-class losses try to shrink samples of the same class into one point in the feature space and may easily drop their intra-class similarity structure. In this paper, we propose a new loss based on center predictivity, that is, a sample must be positioned in a location of the feature space such that from it we can roughly predict the location of the center of same-class samples. The prediction error is then regarded as a loss called Center Prediction Loss (CPL). Unlike most existing metric learning loss functions, CPL involves learnable parameters, i.e., the center predictor, which brings a remarkable change in the properties of the loss. In particular, it allows higher freedom in intra-class distributions. And the parameters in CPL will be discarded after training. Extensive experiments on various real-world ReID datasets show that the proposed loss can achieve superior performance and can also be complementary to existing losses.

Transient stability assessment with time-adaptive method based on spatial distribution

For power system transient stability assessment (TSA), the differences of characteristics between critical situations are not obvious in the early post-fault stage, so high accuracy can’t be achieved. With the development of fault, the differences are more obvious, and the evaluation accuracy is improved. Towards the trade-off between accuracy and timeliness, a spatial distribution-based time-adaptive (SDTA) method is proposed. Firstly, several long short-term memory (LSTM) classifiers of different cycles construct a time-adaptive framework. Then, the overlapping region where misclassifications are likely to occur is selected according to the spatial distribution of training samples. The TSA results of samples in the non-overlapping region are directly output, and the reliability of samples in the overlapping region is evaluated. Finally, the distance ratio-based trust score (DRTS) is proposed as a reliability index. The reliability of overlapping samples is evaluated by employing the changing trend of DRTS. If the classifications at the current cycle are assessed as reliable, the TSA results are output. Otherwise, the system will wait for the features of the next cycle for prediction. The proposed method is tested in the IEEE-39 bus system and a realistic system. The case studies demonstrate that the scheme meets the requirements of TSA.

Impact assessment of humanitarian demining using object-based peri-urban land cover classification and morphological building detection from VHR Worldview imagery

This study utilised very high-resolution (VHR) imagery to monitor and evaluate the impact of humanitarian demining activities in a peri-urban environment in Afghanistan. Identifying buildings and mapping the spatial distribution of different types of land cover is of great practical significance for demining organisations, such as the HALO Trust, to assess the impact of their mine-clearance activities, by quantifying change and the growth of a population in a specific area over time. This study had two main objectives: (i) to map the post-clearance land cover, and (ii) to detect and quantify the change in the number and area of buildings. Two independent workflows were implemented and evaluated. To map land cover, this study investigated the implementation of various machine learning algorithms in object-based image classification (OBIA) of VHR satellite imagery (Worldview-1,2,3). Image segmentation was carried out using the Large-Scale Mean-Shift (LSMS) algorithm, before classification was performed based on a machine learning Random Forests (RF) approach. Different parameters and spatial distribution of training samples were tested to analyse the model's performance. Further analysis determined that by using only the segments' mean value per spectral band (Red, Green, Blue), data redundancy in the training stage was eliminated. The final classified map had an overall accuracy of 90.67% and a total built-area of 643,660.28 m2 was detected in the 4.11 km2 study area. To detect and quantify buildings present in the study area, an alternative, automatic, unsupervised approach based on the morphological building index (MBI) was implemented using MATLAB. Two VHR (0.5 m) panchromatic images acquired by WorldView-1 in 2008 and 2018 were processed using a series of multi-scale and multi-directional morphological operators, before a series of post-processing thresholds were applied to refine the output. Parameters were systematically optimised for the datasets and their sensitivity analysed. By comparing the output to manually labelled reference maps, producer's and user's accuracy of 78.63/81.70% and 75.30/77.91% were attained for 2008 and 2018 imagery respectively. The built-area was found to increase from 58,705 to 611,920 m2 over the ten-year period, indicating a significant increase in the number of people resident in the area. Whilst OBIA was found to be more accurate than MBI for the 2018 pansharpened imagery, with producer's and user's accuracy of 91.00/90.60% versus 84.65/79.44% respectively, it must be considered that the MBI, an unsupervised method requiring no training, offers a fast solution if the sole objective is building detection.

Single MR image super-resolution via channel splitting and serial fusion network

In magnetic resonance imaging (MRI), spatial resolution is an important and critical imaging parameter that represents how much information is contained in a unit space. Acquiring high-resolution MRI data usually takes a long scanning time and is subject to motion artifacts due to hardware, physical, and physiological limitations. Single image super-resolution (SISR) based on deep learning is an effective and promising alternative technique to improve the native spatial resolution of magnetic resonance (MR) images. However, because of the simple diversity and single distribution of training samples, the effective training of deep models with medical training samples and improvement of the tradeoff between model performance and computing overhead are major challenges. In addition, deeper networks are more difficult to effectively train since the information is gradually weakened as the network deepens. In this paper, a novel channel splitting and serial fusion network (CSSFN) is presented for single MR image super-resolution. The proposed CSSFN splits hierarchical features into a series of subfeatures, which are then integrated together in a serial manner. Hence, the network becomes deeper and can discriminatively and reasonably deal with the subfeatures. Moreover, a dense global feature fusion (DGFF) is adopted to integrate the intermediate features, which further promotes the information flow in the network and helps to stabilize model training. Extensive experiments on several typical MR images show the superiority of our CSSFN models to other advanced SISR methods.

A short-term load forecasting model based on mixup and transfer learning

When the amount of historical load data is insufficient, the use of deep learning for load forecasting is prone to overfitting. This paper proposes a short-term electric load forecasting model based on mixup and transfer learning. Mixup is introduced to achieve data enhancement, which improves the generalization ability of the forecasting model by expanding the distribution of training samples. Transfer the historical load of other users with similar consumption patterns to supplement data samples and avoid overfitting of the forecasting model. The similarity between the target load sequence and the source load sequence is measured by the maximal information coefficient (MIC). Finally, the expanded load data adopts a long short-term memory (LSTM) neural network that can learn the long-term dependence in the time series for load forecasting. Experimental results on the residential data set demonstrate that the model proposed in this paper improves the accuracy of load forecasting in scenarios where historical data is scarce.

Geometric imbalanced deep learning with feature scaling and boundary sample mining

Data imbalance is a significant factor affecting classification performance in computer vision. In particular, data imbalance is harmful to classification learning and representation learning. To address this issue, this paper proposes a geometric deep learning framework combined with Feature Scaling Module (FSM) and Boundary Samples Mining Module (BSMM). Considering the geometric information in sample distributions of training samples, FSM is proposed to scale the features by hypersphere radius of each class, which improves the representation ability of minority classes. Meanwhile, it is noteworthy that the relationships and information between samples are essential for classification. Therefore, BSMM is proposed to mine the boundary samples by Gabriel Graph that takes the relationships into account. Finally, a loss scheduler is designed to adjust the training process of these two modules. With the scheduler, the model first learns representation and then focuses more on minority classes gradually. Extensive experiments on three benchmark datasets demonstrate the advantages of the proposed learning framework over the state-of-the-art models for solving the imbalance problem.

SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement with Multi-Scale Perception.

Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement network, called SGUIE-Net, in which we introduce semantic information as high-level guidance via region-wise enhancement feature learning. Accordingly, we propose semantic region-wise enhancement module to better learn local enhancement features for semantic regions with multi-scale perception. After using them as complementary features and feeding them to the main branch, which extracts the global enhancement features on the original image scale, the fused features bring semantically consistent and visually superior enhancements. Extensive experiments on the publicly available datasets and our proposed dataset demonstrate the impressive performance of SGUIE-Net. The code and proposed dataset are available at https://trentqq.github.io/SGUIE-Net.html.

DRFL-VAT: Deep Representative Feature Learning With Virtual Adversarial Training for Semisupervised Classification of Hyperspectral Image

While deep learning algorithms have achieved good results in hyperspectral image (HSI) classification, several supervised classification algorithms rely on a large number of labeled samples to get adequate performance. Collecting a large number of labeled samples is expensive in many real applications. To address this issue, a novel semisupervised HSI classification framework called deep representative feature learning (DRFL) with virtual adversarial training (DRFL-VAT) is developed in this article. By embedding the local manifold learning (LML) into the fully connected layers of a convolutional neural network (CNN), our newly developed DRFL can learn representative features. The VAT regularization is adopted to exploit the prediction label distribution of training samples and addresses the overfitting problem. Finally, the objective function of DRFL-VAT is solved by a customized algorithm. We test our method on three widely public HSI datasets and our results show that our method is competitive when compared to other state-of-the-art approaches.