Supervision Information Research Articles

Learning From Architectural Redundancy: Enhanced Deep Supervision in Deep Multipath Encoder–Decoder Networks

Deep encoder-decoders are the model of choice for pixel-level estimation due to their redundant deep architectures. Yet they still suffer from the vanishing supervision information issue that affects convergence because of their overly deep architectures. In this work, we propose and theoretically derive an enhanced deep supervision (EDS) method which improves on conventional deep supervision (DS) by incorporating variance minimization into the optimization. A new structure variance loss is introduced to build a bridge between deep encoder-decoders and variance minimization, and provides a new way to minimize the variance by forcing different intermediate decoding outputs (paths) to reach an agreement. We also design a focal weighting strategy to effectively combine multiple losses in a scale-balanced way, so that the supervision information is sufficiently enforced throughout the encoder-decoders. To evaluate the proposed method on the pixel-level estimation task, a novel multipath residual encoder is proposed and extensive experiments are conducted on four challenging density estimation and crowd counting benchmarks. The experimental results demonstrate the superiority of our EDS over other paradigms, and improved estimation performance is reported using our deeply supervised encoder-decoder.

Breaking the resolution barrier: A low-to-high network for large-scale high-resolution land-cover mapping using low-resolution labels

Large-scale high-resolution land-cover mapping is a way to comprehend the Earth’s surface and resolve the ecological and resource challenges facing humanity. High-resolution (≤1 m) remotely sensed images can now be captured more easily, with wider coverage, as sensors and satellites develop. Nevertheless, the synchronous renewal of land-cover maps is still challenging when using the common land-cover mapping methods, due to the requirement for high-resolution land-cover labels. Abundant low-resolution (∼30 m) land-cover products are available for use as alternative label sources, but the resolution gap between these products and the growing volume of high-resolution imagery is a barrier yet to be overcome. In this paper, to break through this obstacle, we propose a low-to-high network (L2HNet) to automatically generate high-resolution land-cover maps from high-resolution images by taking only low-resolution land-cover products as the training labels, thus getting rid of the requirement for finely labeled samples during the large-scale map updating process. Firstly, to obtain the mapping results with rich details, we propose a resolution-preserving (RP) backbone that contains parallel multi-scale convolutional layers for extracting the high-resolution features from the images. Furthermore, to settle the label noise issue caused by the mismatched resolution, a confident area selection (CAS) module and a low-to-high (L2H) loss function, with weak and unsupervised strategies, are designed for obtaining reliable supervision information from the coarse labels. The experimental results obtained for six administrative states located in the Chesapeake Bay watershed of the United States show that L2HNet outperforms several of the state-of-the-art methods and the mainstream land-cover mapping methods in creating 1-m resolution land-cover maps by taking 30-m resolution land-cover products as training labels. As a further application, L2HNet was also adopted to produce the first 1-m resolution land-cover map with level II classification hierarchy for the entire state of Maryland in the United States, which covers an area of about 33,872 km2. The land-cover map of Maryland is publicly available at http://hipag.whu.edu.cn/L2HNet.html.

Dual Mutual Learning for Cross-Modality Person Re-Identification

Cross-modality person re-identification (Re-ID) is more challenging than traditional visible Re-ID due to the huge cross-modality gap from heterogeneous images. To alleviate this problem, existing methods often utilize a dual path learning framework equipped with metric loss to learn discriminative features. Despite effectiveness, the inevitable degeneration of intra-modality discrimination by taking cross-modality discrimination into consideration is unsolvable. Such degeneration substantially hinders the model’s capability of further improving feature representations. To mitigate this degeneration, we propose a Dual Mutual Learning (DML) method for cross-modality Re-ID which conducts mutual learning between the cross-modality and each of two single modalities. We design a triple-branch deep model containing the RGB and IR branches and the cross-modality branch. The cross-modality branch is designed to learn modality-invariant feature subspace for appearance similarity measurement. Both the RGB branch and IR branch provide attention supervision information to the cross-modality branch for attention feature alignment so as to enhance the intra-modality discrimination. Experimental results on two standard benchmarks demonstrate DML is superior to state-of-the-art methods.

Real-time panoptic segmentation with relationship between adjacent pixels and boundary prediction

Panoptic segmentation has recently received increasing attention since it generates coherent scene segmentation by unifying semantic and instance segmentation. The most popular methods for panoptic segmentation are currently based on an instance segmentation framework with a semantic segmentation branch in parallel. However, these methods are too bloated for real-world applications. In this paper, we propose a simple yet effective fully convolutional network for fast panoptic segmentation. Instead of directly generating the mask for each instance, we leverage a simple graph convolutional layer to construct a pixel relationship head to predict the relationship between two adjacent pixels and determine whether they belong to the same instance. Besides, we leverage boundary information to enhance supervision information and help our method distinguish adjacent objects. Combining predicted category labels for each pixel from the semantic segmentation branch, we can generate a unified panoptic segmentation mask in a parameter-free step. We demonstrate our method’s effectiveness on MS COCO dataset and Cityscapes dataset, which obtain competitive results.

Dlsa: Semi-supervised partial label learning via dependence-maximized label set assignment

Semi-supervised partial label learning is an emerging weakly supervised learning paradigm dealing with partially labeled data and unlabeled data simultaneously. The supervision information is weak and even missing for training instances. Most existing researches simply propagate labels to unlabeled data and focus on inducing a weighted classifier model, ignoring the importance of acquiring reliable label confidences for training instances. In this paper, by label set assignment, the valid label supervision information can be preserved and propagated to unlabeled instances to a greater extent. Then, the training instances are projected into a lower-dimensional feature space. Reliable label confidence can be obtained by maximizing the dependence between the feature description and the associated label information. After that, the predictive model can be induced by preserving the intrinsic manifold structure in both feature space and label space. Accordingly, a novel semi-supervised partial label learning approach via dependence-maximized label set assignment (Dlsa) is proposed, where an accurate and robust model can be induced based on reliable label confidences obtained by label set assignment and dependence-maximized dimensionality reduction. Extensive experiments on five real-world data sets validate the effectiveness and superiority of the proposed method.

Pseudolabel‐guided multiview consensus graph learning for semisupervised classification

Semisupervised multiview learning gains extensive research attention due to its strong capability to utilize the heterogeneous features and the label information of a few labeled samples. However, the supervision information is not well utilized in the process of exploring the consensus structure of the multiview data. In this paper, we propose a novel unified pseudolabel-guided multiview consensus (PMvC) learning framework for the semisupervised classification problem, which learns the consensus structure of multiview data by fully exploiting the supervised information of labeled samples. Specifically, PMvC first assigns multiple pseudolabels to the unlabeled samples by selecting the nearest labeled sample in each view separately, and then labels the part of unlabeled samples by selecting the pseudolabel that agrees across all views. By doing so, the high-confident pseudolabeled samples can be selected to enlarge the labeled sample pool and the supervision information can be exploited further in the learning process. In addition, to capture the consensus structure of the multiview data, PMvC learns a consensus graph from the view-specific self-representation graph guided by enhanced supervision information, which better preserves the manifold structure of samples. Meanwhile, the label information is also propagated from the labeled samples to the unlabeled samples by the learned consensus graph simultaneously. Accordingly, an effective optimization algorithm is derived to find the optimal solution for PMvC. Extensive experiment results on several real-world data sets demonstrate the feasibility and superiority of PMvC. The source code of PMvC is available at https://github.com/justcallmewilliam/PMvC.

Cognitive Impairment Analysis of Myotonic Dystrophy via Weakly Supervised Classification of Neuropsychological Features.

The myotonic dystrophies (DM1 and DM2) are dominantly inherited disorders that cause pathological changes throughout the body. Many individuals with DM experience cognitive, behavioral and other functional central nervous system effects that impact their quality of life. The extent of psychological impairment that will develop in each patient is variable and unpredictable. Hence, it is difficult to get strong supervision information like fully ground truth labels for all cognitive involvement patterns. This study is to assess cognitive involvement among healthy controls and patients with DM. The DM cognitive impairment pattern observation is modeled in a weakly supervised setting and supervision information is used to transform the input feature space to a more discriminative representation suitable for pattern observation. This study incorporated results from 59 adults with DM and 92 control subjects. The developed system categorized the neuropsychological testing data into five cognitive clusters. The quality of the obtained clustering solution was assessed using an internal validity metric. The experimental results show that the proposed algorithm can discover interesting patterns and useful information from neuropsychological data, which will be be crucial in planning clinical trials and monitoring clinical performance. The proposed system resulted in an average classification accuracy of 88%, which is very promising considering the unique challenges present in this population.

The Effectiveness of Si Sagu (Teacher Academic Supervision System) in Academic Supervision Process for Teachers at 3T Area

The purpose of this study was to determine the effectiveness of the SI SAGU Academic Supervision (Information System for Teacher Academic Supervision) model in the academic supervision process in the 3T (Foremost, Outermost, Disadvantaged) area in Bengkayang Regency, West Kalimantan Province. The research method used in this research is a quantitative research method by testing the SI SAGU Academic Supervision model used in the academic supervision process in the 3T area. The research subjects in this study were teachers, principals, and school supervisors in Bengkayang Regency. The instruments used in the study were practical questionnaires and interview guidelines. The instrument used has gone through validation and testing by expert validators, so it is feasible to be used in the research process. The data collection technique in this study was documentation of the results of observations, then giving a perception questionnaire to the SAGU SI users consisting of teachers, school principals, and school supervisors. The results showed that the Teacher Academic Supervision Management Information System (SI SAGU) was effectively used in the academic supervision process in the 3T area, namely Bengkayang Regency. Through SI SAGU, the supervision process can be carried out even though there are some geographical constraints and access to each school. The findings in this study are the effectiveness of the application of SISAGU in the 3T area in the academic supervision process, supported by the active participation and enthusiasm of all relevant stakeholders, namely teachers, schools, principals, school supervisors, and the education office.

Max-Margin Deep Diverse Latent Dirichlet Allocation With Continual Learning.

Deep probabilistic aspect models are widely utilized in document analysis to extract the semantic information and obtain descriptive topics. However, there are two problems that may affect their applications. One is that common words shared among all documents with low representational meaning may reduce the representation ability of learned topics. The other is introducing supervision information to hierarchical topic models to fully utilize the side information of documents that is difficult. To address these problems, in this article, we first propose deep diverse latent Dirichlet allocation (DDLDA), a deep hierarchical topic model that can yield more meaningful semantic topics with less common and meaningless words by introducing shared topics. Moreover, we develop a variational inference network for DDLDA, which helps us to further generalize DDLDA to a supervised deep topic model called max-margin DDLDA (mmDDLDA) by employing max-margin principle as the classification criterion. Compared to DDLDA, mmDDLDA can discover more discriminative topical representations. In addition, a continual hybrid method with stochastic-gradient MCMC and variational inference is put forward for deep latent Dirichlet allocation (DLDA)-based models to make them more practical in real-world applications. The experimental results demonstrate that DDLDA and mmDDLDA are more efficient than existing unsupervised and supervised topic models in discovering highly discriminative topic representations and achieving higher classification accuracy. Meanwhile, DLDA and our proposed models trained by the proposed continual learning approach cannot only show good performance on preventing catastrophic forgetting but also fit the evolving new tasks well.

Self-Supervised Graph Neural Networks via Diverse and Interactive Message Passing

By interpreting Graph Neural Networks (GNNs) as the message passing from the spatial perspective, their success is attributed to Laplacian smoothing. However, it also leads to serious over-smoothing issue by stacking many layers. Recently, many efforts have been paid to overcome this issue in semi-supervised learning. Unfortunately, it is more serious in unsupervised node representation learning task due to the lack of supervision information. Thus, most of the unsupervised or self-supervised GNNs often employ \textit{one-layer GCN} as the encoder. Essentially, the over-smoothing issue is caused by the over-simplification of the existing message passing, which possesses two intrinsic limits: blind message and uniform passing. In this paper, a novel Diverse and Interactive Message Passing (DIMP) is proposed for self-supervised learning by overcoming these limits. Firstly, to prevent the message from blindness and make it interactive between two connected nodes, the message is determined by both the two connected nodes instead of the attributes of one node. Secondly, to prevent the passing from uniformness and make it diverse over different attribute channels, different propagation weights are assigned to different elements in the message. To this end, a natural implementation of the message in DIMP is the element-wise product of the representations of two connected nodes. From the perspective of numerical optimization, the proposed DIMP is equivalent to performing an overlapping community detection via expectation-maximization (EM). Both the objective function of the community detection and the convergence of EM algorithm guarantee that DMIP can prevent from over-smoothing issue. Extensive evaluations on node-level and graph-level tasks demonstrate the superiority of DIMP on improving performance and overcoming over-smoothing issue.

Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization

Multimodal summarization with multimodal output (MSMO) generates a summary with both textual and visual content. Multimodal news report contains heterogeneous contents, which makes MSMO nontrivial. Moreover, it is observed that different modalities of data in the news report correlate hierarchically. Traditional MSMO methods indistinguishably handle different modalities of data by learning a representation for the whole data, which is not directly adaptable to the heterogeneous contents and hierarchical correlation. In this paper, we propose a hierarchical cross-modality semantic correlation learning model (HCSCL) to learn the intra- and inter-modal correlation existing in the multimodal data. HCSCL adopts a graph network to encode the intra-modal correlation. Then, a hierarchical fusion framework is proposed to learn the hierarchical correlation between text and images. Furthermore, we construct a new dataset with relevant image annotation and image object label information to provide the supervision information for the learning procedure. Extensive experiments on the dataset show that HCSCL significantly outperforms the baseline methods in automatic summarization metrics and fine-grained diversity tests.

Image Difference Captioning with Pre-training and Contrastive Learning

The Image Difference Captioning (IDC) task aims to describe the visual differences between two similar images with natural language. The major challenges of this task lie in two aspects: 1) fine-grained visual differences that require learning stronger vision and language association and 2) high-cost of manual annotations that leads to limited supervised data. To address these challenges, we propose a new modeling framework following the pre-training-finetuning paradigm. Specifically, we design three self-supervised tasks and contrastive learning strategies to align visual differences and text descriptions at a fine-grained level. Moreover, we propose a data expansion strategy to utilize extra cross-task supervision information, such as data for fine-grained image classification, to alleviate the limitation of available supervised IDC data. Extensive experiments on two IDC benchmark datasets, CLEVR-Change and Birds-to-Words, demonstrate the effectiveness of the proposed modeling framework. The codes and models will be released at https://github.com/yaolinli/IDC.

Sentiment Lexical Strength Enhanced Self-supervised Attention Learning for sentiment analysis

In Natural Language Processing (NLP), attention mechanism is often used to quantify the importance of the context word in sentiment prediction. However, it tends to focus on high-frequency words, while ignoring low-frequency words that have an active effect in some positions. In this paper, we propose a Sentiment Lexical Strength Enhanced Self-supervised Attention Learning (SLS-ESAL) approach. Specifically, we iteratively mine attention supervision information from all input sentences. Then we use weights quantified by sentiment lexical strength to enhance attention learning in final training, which enables our model to continue to focus on the active context words in different positions and eliminate the effects of the misleading context ones. Experiments on three datasets show that our approach can improve sentiment analysis performance and verify attention weights can be used as an explanation for text classification.

Unified dual-label semi-supervised learning with top-k feature selection

Semi-supervised feature selection alleviates the annotation burden of supervised feature learning by exploiting data under a handful of supervision information. The mainstream technique is to employ a linear regression framework that jointly learns labeled and unlabeled samples. However, existing approaches always encounter the deficiencies in two aspects: 1) the performance of models are severely degenerated once predicted labels are unreliable; 2) the balance of objectives in regards to two types of data are not well considered. In the article, we propose unified dual-label semi-supervised learning for top-k feature selection. The technique defines a soft label matrix to indicate the probability of samples belonging to each class. From the probability, the model could recognize unclassifiable samples that lay around the boundaries. Meanwhile, the label matrix is equipped with an exponent parameter γ. It endows the soft labels dual effects that the labeled and unlabeled data are tactfully discriminated. For the purpose of feature selection, we impose the ℓ2,0-norm constraint on the projection matrix, such that the exact top-k features are picked out. An iteration algorithm is designed to solve the given problem, by which large-scale data are facilely tackled. We conduct experiments that validate the superiority of the proposed method against the state-of-the-art competitors.

Research on the Construction of Financial Supervision Information System Based on Machine Learning

In order to fully implement systematic, continuous and effective supervision of financial institutions and promote the safe, steady, and efficient operation of China’s financial system, this research needs to develop a fully intelligent financial supervision information system, so as to take measures to effectively prevent and resolve financial risks. In this paper, based on ML (machine learning), an LSTM (long short-term memory) model with good comprehensive performance is built. This model is different from the existing scorecard model which relies on statistical learning. It not only further reduces the dependence on financial experts, but also has the ability of rapid iteration. The post-loan risk early warning model based on RF (random forest) algorithm is designed. The model parameters are optimized, which makes the risk early warning model have higher accuracy. The results show that when the data amount is 10,000 to 50,000, the accuracy of the model is relatively high when it is input into the model. When the amount of data is low, the overall throughput of the model is still very high. It shows that the pre-warning model of post-lending risk constructed in this paper has a strong risk prediction ability.

Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark

Tiny object detection (TOD) in aerial images is challenging since a tiny object only contains a few pixels. State-of-the-art object detectors do not provide satisfactory results on tiny objects due to the lack of supervision from discriminative features. Our key observation is that the Intersection over Union (IoU) metric and its extensions are very sensitive to the location deviation of the tiny objects, which drastically deteriorates the quality of label assignment when used in anchor-based detectors. To tackle this problem, we propose a new evaluation metric dubbed Normalized Wasserstein Distance (NWD) and a new RanKing-based Assigning (RKA) strategy for tiny object detection. The proposed NWD-RKA strategy can be easily embedded into all kinds of anchor-based detectors to replace the standard IoU threshold-based one, significantly improving label assignment and providing sufficient supervision information for network training. Tested on four datasets, NWD-RKA can consistently improve tiny object detection performance by a large margin. Besides, observing prominent noisy labels in the Tiny Object Detection in Aerial Images (AI-TOD) dataset, we are motivated to meticulously relabel it and release AI-TOD-v2 and its corresponding benchmark. In AI-TOD-v2, the missing annotation and location error problems are considerably mitigated, facilitating more reliable training and validation processes. Embedding NWD-RKA into DetectoRS, the detection performance achieves 4.3 AP points improvement over state-of-the-art competitors on AI-TOD-v2. Datasets, codes, and more visualizations are available at: https://chasel-tsui.github.io/AI-TOD-v2/.

Gaussian guided IoU: A better metric for balanced learning on object detection

Most anchor-based detectors use intersection over union (IoU) to assign targets to anchors during training. However, IoU did not pay enough attention to the proximity of the anchor's centre to the centre of the truth box, resulting in two issues: (1) the most slender objects were given just one anchor, resulting in insufficient supervision information for slender objects during training; (2) IoU cannot accurately represent the degree of alignment between the feature's receptive field at the anchor's centre and the object. As a result, some features with good alignment degrees are missing, while others with poor alignment degrees are used, reducing the model's localisation accuracy. To address these issues, we first created a Gaussian Guided IoU (GGIoU), which prioritises the proximity of the anchor's centre to the truth box's centre. We then proposed GGIoU-balanced learning methods, including GGIoU-guided assignment strategy and GGIoU-balanced localisation loss. This method can assign multiple anchors to each slender object, favouring features that are well-aligned with the objects during the training process. A large number of experiments show that GGIoU-balanced learning can solve the aforementioned problems and significantly improve the detection model's performance.

The devil in the tail: Cluster consolidation plus cluster adaptive balancing loss for unsupervised person re-identification

Unsupervised person re-identification (Re-ID) is to retrieve pedestrians from different camera views without supervision information. State-of-the-art methods are usually built upon training a convolution neural network with pseudo labels generated by clustering. Unfortunately, the pseudo labels are highly unbalanced and heavily noisy, carrying ineffective or even erroneous supervision information. To address these deficiencies, we present an effective clustering and reorganization approach, called Cluster Consolidation, which aims to separate a small proportion of unreliable data points from each cluster. This approach benefits to improve the quality of the pseudo labels, but also yields more tiny clusters. Thus, we further propose a Cluster Adaptive Balancing (CAB) loss to effectively train the network with the imbalance pseudo labels, where our CAB loss is able to automatically balance the importance of each cluster. We conduct extensive experiments on widely used person Re-ID benchmark datasets and demonstrate the effectiveness of our proposals.

SELF-LLP: Self-supervised learning from label proportions with self-ensemble

In this paper, we tackle the problem called learning from label proportions (LLP), where the training data is arranged into various bags, with only the proportions of different categories in each bag available. Existing efforts mainly focus on training a model with only the limited proportion information in a weakly supervised manner, thus result in apparent performance gap to supervised learning, as well as computational inefficiency. In this work, we propose a multi-task pipeline called SELF-LLP to make full use of the information contained in the data and model themselves. Specifically, to intensively learn representation from the data, we leverage the self-supervised learning as a plug-in auxiliary task to learn better transferable visual representation. The main insight is to benefit from the self-supervised representation learning with deep model, as well as improving classification performance by a large margin. Meanwhile, in order to better leverage the implicit benefits from the model itself, we incorporate the self-ensemble strategy to guide the training process with an auxiliary supervision information, which is constructed by aggregating multiple previous network predictions. Furthermore, a ramp-up mechanism is further employed to stabilize the training process. In the extensive experiments, our method demonstrates compelling advantages in both accuracy and efficiency over several state-of-the-art LLP approaches.

Target-Aware State Estimation for Visual Tracking

Trackers based on the IoU prediction network (IoU-Net) have shown superior performance, which refines a coarse bounding box to an accurate one by maximizing the IoU between the target and the coarse box. However, the traditional IoU-Net is less effective in exploiting the limited but crucial supervision information contained in the initial frame, including the discriminative information between the target and backgrounds and the structure information of the initial target. Missing such information makes the IoU-Net less robust to background distractors and diverse variations of the target appearance. To address this issue, we propose a target-aware state estimation network for visual tracking. A gradient-guided feature adjustment module is built on an online discriminative model to generate target-aware features for constructing the state estimation network; it conveys the online learned discriminative information into the offline trained state estimation network. In addition, we propose a structure-aware integration module and embed it into the state estimation network, enabling the tracker to explicitly model the structure information of the initial target. Extensive experimental results on the VOT2018, OTB2015, UAV123, NFS30, TC128, TrackingNet, LaSOT, and VOT2018-LT datasets demonstrate that the proposed approach performs favorably against state-of-the-art trackers.