Supervisory Information Research Articles

Quality-relevant feature extraction method based on teacher-student uncertainty autoencoder and its application to soft sensors

Supervised representation learning based on the teacher-student framework can extract quality-related features for soft sensors, in which the teacher network extracts representation information for the student network as supervision information. In traditional applications, the teacher network is heavy and is difficult to train, so the teacher network is conventionally pre-trained. However, the pre-training of the teacher network is unnecessary if the training process is not complicated so that it is meaningful to joint optimize the teacher-student network. In our application, the teacher-student framework is used to extract quality-related representation information for soft sensors. The objective is to maximize the mutual information of representation information and supervision information, in which the inconsistency of distributions between observed information and supervisory information is modeled as isotropic Gaussian noise. The objective is decoupled through analysis under some approximate assumptions so that the alternative iteration method can be used to update the parameters of the model. The proposed quality-related feature extraction method is applied to soft sensors combined with a traditional just-in-time learning method. Our experiments show that the prediction performance of our representation extraction method is better than other existing representation extraction algorithms.

An Explainable Laser Welding Defect Recognition Method Based on Multi-Scale Class Activation Mapping

Vision-based online defect recognition can provide insights for laser welding quality control systems. Although the visual signal contains richer quality information than the one-dimensional signal, the quality features contained in the visual signal are more abstract. To improve the explainability of current convolutional neural networks (CNNs) for laser welding defect recognition (LWDR), a class activation mapping method based on multi-scale fusion features (CAM-MSFF) is proposed. In addition, a multi-scale features adaptive fusion method is proposed with three steps of feature squeeze, feature mapping, and feature recalibrating. In order to facilitate the learning and utilization of multi-scale features by the proposed method, supervisory information is applied to multiple scales. The experimental results show that the proposed CAM-MSFF method has higher accuracy and convergence speed than the conventional model. The results of the explainability tests show that the proposed method can provide a more accurate and human-comprehensible explanation of the model’s decision basis.

Semi-Supervised Building Footprint Generation With Feature and Output Consistency Training

Accurate and reliable building footprint maps are vital to urban planning and monitoring, and most existing approaches fall back on convolutional neural networks (CNNs) for building footprint generation. However, one limitation of these methods is that they require strong supervisory information from massive annotated samples for network learning. State-of-the-art semi-supervised semantic segmentation networks with consistency training can help to deal with this issue by leveraging a large amount of unlabeled data, which encourages the consistency of model output on data perturbation. Considering that rich information is also encoded in feature maps, we propose to integrate the consistency of both features and outputs in the end-to-end network training of unlabeled samples, enabling to impose additional constraints. Prior semi-supervised semantic segmentation networks have established the cluster assumption, in which the decision boundary should lie in the vicinity of low sample density. In this work, we observe that for building footprint generation, the low-density regions are more apparent at the intermediate feature representations within the encoder than the encoder's input or output. Therefore, we propose an instruction to assign the perturbation to the intermediate feature representations within the encoder, which considers the spatial resolution of input remote sensing imagery and the mean size of individual buildings in the study area. The proposed method is evaluated on three datasets with different resolutions: Planet dataset (3 m/pixel), Massachusetts dataset (1 m/pixel), and Inria dataset (0.3 m/pixel). Experimental results show that the proposed approach can well extract more complete building structures and alleviate omission errors.

Semi-Supervised Structured Subspace Learning for Multi-View Clustering.

Multi-view clustering aims at simultaneously obtaining a consensus underlying subspace across multiple views and conducting clustering on the learned consensus subspace, which has gained a variety of interest in image processing. In this paper, we propose the Semi-supervised Structured Subspace Learning algorithm for clustering data points from Multiple sources (SSSL-M). We explicitly extend the traditional multi-view clustering with a semi-supervised manner and then build an anti-block-diagonal indicator matrix with small amount of supervisory information to pursue the block-diagonal structure of the shared affinity matrix. SSSL-M regularizes multiple view-specific affinity matrices into a shared affinity matrix based on reconstruction through a unified framework consisting of backward encoding networks and the self-expressive mapping. The shared affinity matrix is comprehensive and can flexibly encode complementary information from multiple view-specific affinity matrices. An enhanced structural consistency of affinity matrices from different views can be achieved and the intrinsic relationships among affinity matrices from multiple views can be effectively reflected in this manner. Technically, we formulate the proposed model as an optimization problem, which can be solved by an alternating optimization scheme. Experimental results over seven different benchmark datasets demonstrate that better clustering results can be obtained by our method compared with the state-of-the-art approaches.

Hierarchical Feature Selection Based on Label Distribution Learning

Hierarchical classification learning, which organizes data categories into a hierarchical structure, is an effective approach for large-scale classification tasks. The high dimensionality of data feature space, represented in hierarchical class structures, is one of the main research challenges. In addition, the class hierarchy often introduces imbalanced class distributions and causes overfitting. In this paper, we propose a feature selection method based on label distribution learning to address the above challenges. The crux is to alleviate the class imbalance problem and learn a discriminative feature subset for hierarchical classification process. Due to correlation between different class categories in the hierarchical tree structure, sibling categories can provide additional supervisory information for each learning sub tasks, which, in turn, alleviates the problem of under-sampling of minority categories. Therefore, we transform hierarchical labels to a hierarchical label distribution to represent this correlation. After that, a discriminative feature subset is selected recursively, by the common features and label-specific feature constraints, to ensure that downstream classification tasks can achieve the best performance. Experiments and comparisons, using seven well-established feature selection algorithms on six real data sets with different degrees of imbalance, demonstrate the superiority of the proposed method.

Audio Matters in Video Super-Resolution by Implicit Semantic Guidance

Video super-resolution (VSR) aims to use multiple consecutive low-resolution frames to recover the corresponding high-resolution frames. However, existing VSR methods only consider videos as image sequences, ignoring another essential timing informationaudio, while in fact, there is a semantic link between audio and vision, and extensive studies have shown that audio can provide supervisory information in visual networks. Meanwhile, the addition of semantic priors has been proven to be effective in super-resolution (SR) tasks, but a pretrained segmentation network is required to obtain semantic segmentation maps. By contrast, audio as the information contained in the video itself can be directly used. Therefore, in this study, we propose a novel and pluggable multiscale audiovisual fusion (MS-AVF) module to enhance VSR performance by exploiting the relevant audio information, which can be regarded as implicit semantic guidance compared with the kind of explicit segmentation priors. Specifically, we first fuse audiovisual features on the semantic feature maps of different granularities of the target frames, and then through a top-down multiscale fusion approach, feedback high-level semantics to the underlying global visual features layer by layer, thereby providing effective audio implicit semantic guidance for VSR. Experimental results show that audio can further improve the VSR effect. Moreover, by visualizing the learned attention mask, the proposed end-to-end model can automatically learn potential audiovisual semantic links, especially improving the accuracy and effectiveness of the SR of sound sources and their surrounding regions.

Semi-Supervised Leukocyte Segmentation Based on Adversarial Learning With Reconstruction Enhancement

The number, relative ratio, and appearance of peripheral blood leukocytes can assist doctors to diagnose diseases such as lymphoma and leukemia. Therefore, segmentation of peripheral blood leukocytes in blood smear images plays a crucial role in the diagnosis of certain diseases. We propose a semi-supervised leukocyte segmentation method under the framework of adversarial learning, in which a discriminator is trained to differentiate the segmentation maps coming either from the ground truth or from the segmentation network. Specifically, we first construct a lightweight leukocyte segmentation network by modifying DeepLabV3+ with simplified MobileNetV2. In addition, a reconstruction decoder is developed as a constraint on feature extraction, to recover the original leukocyte image from its segmentation result. Then, we design the discriminator in a fully convolutional manner, which as well enables semi-supervised learning by inferring reliable supervisory information for unlabeled images. Quantitative and qualitative experimental results on four image datasets demonstrate the superiority of the proposed method over several state-of-the-art methods, achieving the highest average F1 score of 0.9717 and mIoU of 0.9493 with only 20% labeled training samples.

Dual semi-supervised convex nonnegative matrix factorization for data representation

Semi-supervised nonnegative matrix factorization (NMF) has received considerable attention in machine learning and data mining. A new semi-supervised NMF method, called dual semi-supervised convex nonnegative matrix factorization (DCNMF), is proposed in this paper for fully using the limited label information. Specifically, DCNMF simultaneously incorporates the pointwise and pairwise constraints of labeled samples as dual supervisory information into convex NMF, which results in a better low-dimensional data representation. Moreover, DCNMF imposes the nonnegative constraint only on the coefficient matrix but not on the base matrix. Consequently, DCNMF can process mixed-sign data, and hence enlarge the range of applications. We derive an efficient alternating iterative algorithm for DCNMF to solve the optimization, and analyze the proposed DCNMF method in terms of the convergence and computational complexity. We also discuss the relationships between DCNMF and several typical NMF based methods. Experimental results illustrate that DCNMF outperforms the related state-of-the-art NMF methods on nonnegative and mixed-sign datasets for clustering applications.

Decoupled self-supervised label augmentation for fully-supervised image classification

Self-supervised label augmentation has emerged as an effective means to overcome the data scarcity problem for supervised vision tasks. Existing rotation-based self-supervised label augmentation methods either impose or relax the rotation invariance constraint on the primary classifier, which omit necessary supervisory information and may degrade the classification performance depending on the given working data. To overcome this problem, we propose a decoupled self-supervised label augmentation method to enhance the feature representation for fully-supervised image classification. Experimental results on diverse datasets demonstrate that the proposed method consistently outperforms state-of-the-art data augmentation methods. The proposed method is also complementary to conventional data augmentation methods, such as AutoAugment and Fast AutoAugment.

No-reference image quality assessment for contrast-changed images via a semi-supervised robust PCA model

Contrast plays an important role in human perception of image quality. In this paper, we propose a metric for no-reference quality assessment of contrast-changed images by using a novel semi-supervised robust PCA, which can realize feature selection and denoising simultaneously, guided by the available supervisory information. To select features adaptively, the information-oriented features (e.g. entropy and natural scene statistics) and appearance-oriented features (e.g. colorfulness) are adopted. The proposed model is formulated as a constraint optimization problem, which is further casted to a convex problem and solved via augmented Lagrangian multiplier method. Extensive experimental results on CCID2014, CSIQ, SIQAD and TID2013 databases show that the proposed semi-supervised image quality metric based on robust PCA (SIQMR) provides a more accurate prediction than other metrics on the human perception of contrast variations.

Information Management Database System Managerial Supervision of School Supervisors

This article aimed at managing information technology based school supervisory managerial supervision database. The approach in this study uses a system approach with a waterfall model. The steps of developing the model are requirements analysis, design, implementation and unit testing, integration and system test. The results show that School supervisors in the North Musi Rawas District Office still use the manual system, and design of managerial supervisory information systems for principals are designed based on planned flowcharts. The results of expert validation, the supervisory managerial supervisory database includes; curriculum and learning administration, class administration, school administration and management, organization and institutions, facilities and infrastructure, staffing, funding, students, community participation, school cultural environment are appropriate to be used. The appropriate Implementation of the School Supervisory Managerial with Supervision Database SIM.
 Keywords: Management Information System, Management Supervision, System Managerial Supervision

Supervisory information and loss aversion

This study examines the optimality of allowing corrupt interactions (bribery, framing and extortion) between the supervisor and the agent when the agent is loss averse. We show that although inducing bribery leads to effective usage of the supervisory information, preventing all corrupt interactions between the supervisor and the agent by disregarding some of supervisory information can be optimal.

Semisupervised Adaptive Symmetric Non-Negative Matrix Factorization.

As a variant of non-negative matrix factorization (NMF), symmetric NMF (SymNMF) can generate the clustering result without additional post-processing, by decomposing a similarity matrix into the product of a clustering indicator matrix and its transpose. However, the similarity matrix in the traditional SymNMF methods is usually predefined, resulting in limited clustering performance. Considering that the quality of the similarity graph is crucial to the final clustering performance, we propose a new semisupervised model, which is able to simultaneously learn the similarity matrix with supervisory information and generate the clustering results, such that the mutual enhancement effect of the two tasks can produce better clustering performance. Our model fully utilizes the supervisory information in the form of pairwise constraints to propagate it for obtaining an informative similarity matrix. The proposed model is finally formulated as a non-negativity-constrained optimization problem. Also, we propose an iterative method to solve it with the convergence theoretically proven. Extensive experiments validate the superiority of the proposed model when compared with nine state-of-the-art NMF models.

Joint Learning of Multiple Latent Domains and Deep Representations for Domain Adaptation.

In domain adaptation, the automatic discovery of multiple latent source domains has succeeded by capturing the intrinsic structure underlying the source data. Different from previous works that mainly rely on shallow models for domain discovery, we propose a novel unified framework based on deep neural networks to jointly address latent domain prediction from source data and deep representation learning from both source and target data. Within this framework, an iterative algorithm is proposed to alternate between 1) utilizing a new probabilistic hierarchical clustering method to separate the source domain into latent clusters and 2) training deep neural networks by using the domain membership as the supervision to learn deep representations. The key idea behind this joint learning framework is that good representations can help to improve the prediction accuracy of latent domains and, in turn, domain prediction results can provide useful supervisory information for feature learning. During the training of the deep model, a domain prediction loss, a domain confusion loss, and a task-specific classification loss are effectively integrated to enable the learned feature to distinguish between different latent source domains, transfer between source and target domains, and become semantically meaningful among different classes. Trained in an end-to-end fashion, our framework outperforms the state-of-the-art methods for latent domain discovery, as validated by extensive experiments on both object classification and human action-recognition tasks.

Structured subspace learning-induced symmetric nonnegative matrix factorization

Symmetric NMF (SNMF) is able to determine the inherent cluster structure with the constructed graph. However, the mapping between the empirically constructed similarity representation and the desired one may contain complex structural and hierarchical information, which is not easy to capture with single learning approaches. Then, we propose a novel Structured Subspace Learning-induced Symmetric Nonnegative Matrix Factorization (SSLSNMF) in this paper. Based on the similarity space to be learned, SSLSNMF further learns a latent subspace, which considers the global and local structure of the data. Since SSLSNMF is formulated in a semi-supervised way, the supervisory information with constraints of cannot-link and must-link is also utilized. To guarantee that the latent similarity subspace to be discriminative, the global and local structures of the data as well as structural consistencies for limited labels in a semi-supervised manner are learned simultaneously in the proposed framework. An effective alternating iterative algorithm is proposed with proved convergence. Experiments conducted on six benchmark datasets show that better results can be obtained compared with state-of-the-art methods.

Wasserstein Distance-Based Auto-Encoder Tracking

Most of the existing visual object trackers are based on deep convolutional feature maps, but there have fewer works about finding new features for tracking. This paper proposes a novel tracking framework based on a full convolutional auto-encoder appearance model, which is trained by using Wasserstein distance and maximum mean discrepancy . Compared with previous works, the proposed framework has better performance in three aspects, including appearance model, update scheme, and state estimation. To address the issues of the original update scheme including poor discriminant performance under limited supervisory information, sample pollution caused by long term object occlusion, and sample importance unbalance, in this paper, a novel latent space importance weighting algorithm, a novel sample space management algorithm, and a novel IOU-based label smoothing algorithm are proposed respectively. Besides, an improved weighted loss function is adopted to address the sample imbalance issue. Finally, to improve the state estimation accuracy, the combination of Kullback-Leibler divergence and generalized intersection over union is introduced. Extensive experiments are performed on the three widely used benchmarks, and the results demonstrate the state-of-the-art performance of the proposed method. Code and models are available at https://github.com/wahahamyt/CAT.git .

End-to-end video text detection with online tracking

Text in videos usually acts as important semantic cues, which is helpful to video analysis. Video text detection is considered as one of the most difficult tasks in document analysis due to the following two challenges: 1) the difficulties caused by video scenes, i.e., motion blur, illumination changes, and occlusion; 2) the properties of text including variants of fonts, languages, orientations, and shapes. Most existing methods try to improve the video text detection through video text tracking, but treat these two tasks separately. This can significantly increase the amount of calculations and cannot take full advantage of the supervisory information of both tasks. In this work, we introduce explainable descriptor, combines appearance, geometry and PHOC features, to establish a bridge between detection and tracking and build an end-to-end video text detection model with online tracking to address these challenges together. By integrating these two branches into one trainable framework, they can promote each other and the computational cost is significantly reduced. Besides, the introduce explainable descriptor also make our end-to-end model have inherent interpretability. Experiments on existing video text benchmarks including ICDAR 2013 Video, DOST, Minetto and YVT verify the role of explainable descriptors in improving model expression ability and the proposed method significantly outperforms state-of-the-art methods. Our method improves F-score by more than 2% on all datasets and achieves 81.52% on the MOTA of the Minetto dataset.

A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds.

In this article, we propose a novel self-training approach named Crowd-SDNet that enables a typical object detector trained only with point-level annotations (i.e., objects are labeled with points) to estimate both the center points and sizes of crowded objects. Specifically, during training, we utilize the available point annotations to supervise the estimation of the center points of objects directly. Based on a locally-uniform distribution assumption, we initialize pseudo object sizes from the point-level supervisory information, which are then leveraged to guide the regression of object sizes via a crowdedness-aware loss. Meanwhile, we propose a confidence and order-aware refinement scheme to continuously refine the initial pseudo object sizes such that the ability of the detector is increasingly boosted to detect and count objects in crowds simultaneously. Moreover, to address extremely crowded scenes, we propose an effective decoding method to improve the detector's representation ability. Experimental results on the WiderFace benchmark show that our approach significantly outperforms state-of-the-art point-supervised methods under both detection and counting tasks, i.e., our method improves the average precision by more than 10% and reduces the counting error by 31.2%. Besides, our method obtains the best results on the crowd counting and localization datasets (i.e., ShanghaiTech and NWPU-Crowd) and vehicle counting datasets (i.e., CARPK and PUCPR+) compared with state-of-the-art counting-by-detection methods. The code will be publicly available at https://github.com/WangyiNTU/Point-supervised-crowd-detection.

Deep Unsupervised Hashing for Large-Scale Cross-Modal Retrieval Using Knowledge Distillation Model.

Cross-modal hashing encodes heterogeneous multimedia data into compact binary code to achieve fast and flexible retrieval across different modalities. Due to its low storage cost and high retrieval efficiency, it has received widespread attention. Supervised deep hashing significantly improves search performance and usually yields more accurate results, but requires a lot of manual annotation of the data. In contrast, unsupervised deep hashing is difficult to achieve satisfactory performance due to the lack of reliable supervisory information. To solve this problem, inspired by knowledge distillation, we propose a novel unsupervised knowledge distillation cross-modal hashing method based on semantic alignment (SAKDH), which can reconstruct the similarity matrix using the hidden correlation information of the pretrained unsupervised teacher model, and the reconstructed similarity matrix can be used to guide the supervised student model. Specifically, firstly, the teacher model adopted an unsupervised semantic alignment hashing method, which can construct a modal fusion similarity matrix. Secondly, under the supervision of teacher model distillation information, the student model can generate more discriminative hash codes. Experimental results on two extensive benchmark datasets (MIRFLICKR-25K and NUS-WIDE) show that compared to several representative unsupervised cross-modal hashing methods, the mean average precision (MAP) of our proposed method has achieved a significant improvement. It fully reflects its effectiveness in large-scale cross-modal data retrieval.

Facial Expression Recognition With Confidence Guided Refined Horizontal Pyramid Network

Facial expression recognition has become one of the most studied applications in computer vision and human-computer interaction. Part-level features constitute one the most appealing breakthrough to offer fine-grained information and have been intensively studied consequently. The main prerequisite is accurate location of facial parts with additional cues, which is turned to be another difficulty. Rather than directly locating parts, this paper proposes a confidence guided Refined Horizontal Pyramid Network (RHPN) to fully exploit various partial information of a given facial image. It can learn discriminative features with multiple facial granularities. Inconsistencies within each granularity are efficiently limited. Specifically, we first design a Horizontal Pyramid Network (HPN) for classification using the uniform partial feature representations at different horizontal pyramid scales. It successfully enhances the discriminative capabilities of various facial parts. Then, a refinement mechanism is added to HPN due to the fact that the uniform partition inevitably incurs outliers in each part and introduces unreasonable similarity between different parts. It re-assigns these outliers to the parts they are closest to, resulting in the refined parts with enhanced within-stripe consistency. Due to the lack of explicit supervisory information, we design an induced training strategy additionally for efficiently training of RHPN. For avoiding the finagle of prediction based on the separate feature vector, we prefer the confidence guided prediction as the final classification result. Experiments verify the effectiveness of our network for increasing the performance on facial expression recognition. More importantly, it surpasses the state-of-the-art by a large margin on the lab-controlled Oulu-CASIA dataset and the real world RAF-DB dataset.