Semi-supervised Neural Network Research Articles

Reconstruction of elasticity modulus distribution base on semi-supervised neural network

Accurate reconstruction of tissue elasticity modulus distribution has always been an important challenge in ultrasound elastography. Considering that existing deep learning-based supervised reconstruction methods only use simulated displacement data with random noise in training, which cannot fully provide the complexity and diversity brought by in-vivo ultrasound data, this study introduces the use of displacement data obtained by tracking in-vivo ultrasound radio frequency signals (i.e., real displacement data) during training, employing a semi-supervised approach to enhance the prediction accuracy of the model. Experimental results indicate that in phantom experiments, the semi-supervised model augmented with real displacement data provides more accurate predictions, with mean absolute errors and mean relative errors both around 3%, while the corresponding data for the fully supervised model are around 5%. When processing real displacement data, the area of prediction error of semi-supervised model was less than that of fully supervised model. The findings of this study confirm the effectiveness and practicality of the proposed approach, providing new insights for the application of deep learning methods in the reconstruction of elastic distribution from in-vivo ultrasound data.

Semi-Supervised Neural Network for Complex Lithofacies Identification Using Well Logs

Semi-Supervised Neural Network for Complex Lithofacies Identification Using Well Logs

Joint task semi-supervised semantic segmentation for TRUS image

Transrectal ultrasound (TRUS) is an important tool for prostate observation, playing a crucial role in medical processes such as multimodal image registration and surgical planning. However, TRUS images are often plagued by blurry boundary information and artifacts, making manual annotation a time-consuming and labor-intensive task, which results in insufficient pixel-labeled data for training neural networks. Therefore, this paper introduces a semi-supervised neural network based on joint tasks for TRUS image segmentation, which combines confidence information from target localization and semantic segmentation to generate pseudo-labels for collectively training unlabeled samples. Additionally, this method supports the generation of pseudo-labels using only location information for weakly supervised training of neural networks. Compared to other state-of-the-art models, our results show that this approach outperforms in segmentation performance with a smaller dataset, achieving an mean Dice similarity coefficient and mean Jaccard similarity coefficient of 92.8 % and 88.4 %, respectively. Furthermore, the experiments reveal that reducing the coupling of two decoders' parameters between the segmentation and detection tasks improves segmentation accuracy as dataset size decreases.

A3SOM, abstained explainable semi-supervised neural network based on self-organizing map.

In the sea of data generated daily, unlabeled samples greatly outnumber labeled ones. This is due to the fact that, in many application areas, labels are scarce or hard to obtain. In addition, unlabeled samples might belong to new classes that are not available in the label set associated with data. In this context, we propose A3SOM, an abstained explainable semi-supervised neural network that associates a self-organizing map to dense layers in order to classify samples. Abstained classification enables the detection of new classes and class overlaps. The use of a self-organizing map in A3SOM allows integrated visualization and makes the model explainable. Along with describing our approach, this paper shows that the method is competitive with other classifiers and demonstrates the benefits of including abstention rules. A use case is presented on breast cancer subtype classification and discovery to show the relevance of our method in real-world medical problems.

A Semi-Supervised Speech Deception Detection Algorithm Combining Acoustic Statistical Features and Time-Frequency Two-Dimensional Features.

Human lying is influenced by cognitive neural mechanisms in the brain, and conducting research on lie detection in speech can help to reveal the cognitive mechanisms of the human brain. Inappropriate deception detection features can easily lead to dimension disaster and make the generalization ability of the widely used semi-supervised speech deception detection model worse. Because of this, this paper proposes a semi-supervised speech deception detection algorithm combining acoustic statistical features and time-frequency two-dimensional features. Firstly, a hybrid semi-supervised neural network based on a semi-supervised autoencoder network (AE) and a mean-teacher network is established. Secondly, the static artificial statistical features are input into the semi-supervised AE to extract more robust advanced features, and the three-dimensional (3D) mel-spectrum features are input into the mean-teacher network to obtain features rich in time-frequency two-dimensional information. Finally, a consistency regularization method is introduced after feature fusion, effectively reducing the occurrence of over-fitting and improving the generalization ability of the model. This paper carries out experiments on the self-built corpus for deception detection. The experimental results show that the highest recognition accuracy of the algorithm proposed in this paper is 68.62% which is 1.2% higher than the baseline system and effectively improves the detection accuracy.

Consensus graph learning-based self-supervised ensemble clustering

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.

Top-K Pseudo Labeling for Semi-Supervised Image Classification

In this paper, a top-k pseudo labeling method for semi-supervised self-learning is proposed. Pseudo labeling is a key technology in semi-supervised self-learning. Briefly, the quality of the pseudo label generated largely determined the convergence of the neural network and the accuracy obtained. In this paper, the authors use a method called top-k pseudo labeling to generate pseudo label during the training of semi-supervised neural network model. The proposed labeling method helps a lot in learning features from unlabeled data. The proposed method is easy to implement and only relies on the neural network prediction and hyper-parameter k. The experiment results show that the proposed method works well with semi-supervised learning on CIFAR-10 and CIFAR-100 datasets. Also, a variant of top-k labeling for supervised learning named top-k regulation is proposed. The experiment results show that various models can achieve higher accuracy on test set when trained with top-k regulation.

Expression regulation of genes is linked to their CpG density distributions around transcription start sites.

The CpG dinucleotide and its methylation behaviors play vital roles in gene regulation. Previous studies have divided genes into several categories based on the CpG intensity around transcription starting sites and found that housekeeping genes tend to possess high CpG density, whereas tissue-specific genes are generally characterized by low CpG density. In this study, we investigated how the CpG density distribution of a gene affects its transcription and regulation pattern. Based on the CpG density distribution around transcription starting site, by means of a semi-supervised neural network we designed, which took data augmentation into account, we divided the human genes into three categories, and genes within each cluster shared similar CpG density distribution. Not only sequence properties, these different clusters exhibited distinctly different structural features, regulatory mechanisms, correlation patterns between the expression level and CpG/TpG density, and expression and epigenetic mark variations during tumorigenesis. For instance, the activation of cluster 3 genes relies more on 3D genome reorganization, compared with cluster 1 and 2 genes, whereas cluster 2 genes showed the strongest correlation between gene expression and H3K27me3. Genes exhibiting uncoupled correlation between gene regulation and histone modifications are mainly in cluster 3. These results emphasized that the usage of epigenetic marks in gene regulation is partially rooted in the sequence property of genes such as their CpG density distribution and explained to some extent why the relation between epigenetic marks and gene expression is controversial.

Manifold Learning-Based Semisupervised Neural Network for Hyperspectral Image Classification

Feature extraction (FE), an important preprocessing step in hyperspectral image (HSI) classification, has received growing attention in the remote sensing community. In recent years, the FE ability of deep learning (DL) methods has been widely recognized. However, most DL models focus on training networks with strong nonlinear mapping ability. They fail to explore the intrinsic manifold structure in HSI, and their performance depends on large size of the labeled training set. To address the above problems, a novel FE approach, termed manifold learning-based semisupervised neural network (MSSNet), was proposed in this article. By introducing the graph embedding (GE) framework, MSSNet develops a semisupervised graph model to explore the manifold structure in HSI with both labeled and unlabeled data. On the basis of this graph model, MSSNet constructs a combined loss function to take into account the metric of difference values and the exploration of manifold margins; thus, it reduces the difference between the predictive value and the actual value to enhance the separability of the features extracted by the network. Experiments conducted on real-world HSI datasets demonstrate that the performance of the proposed MSSNet outperforms some related state-of-the-art FE approaches.

DeepBHCP: Deep neural network algorithm for solving backward heat conduction problems

This paper extends a deep neural network method, a semi-supervised one, to solve backward heat conduction problems which have been long-standing computational challenges due to being ill-posed. The effectiveness and robustness of the methodology are demonstrated through various problems including different types of boundary conditions, several types of time-dependent thermal diffusivity factors, and a variety of domains for two-dimensional tests. In spite of traditional methods, there is no need to apply any regularization technique. According to simulation results, this revolutionary strategy can efficiently and accurately extract the pattern of the solutions even when the noise corruption up to ten percent is imposed on input data. Moreover, when the final time is increased further, this approach is efficient in recovering the data at the initial time, which accentuates the method's robustness. To demonstrate the superiority of the semi-supervised neural network process, we make a comparison with the radial basis functions finite difference (RBF-FD) method which is a localized RBF method.

Online flu epidemiological deep modeling on disease contact network

The surveillance and preventions of infectious disease epidemics such as influenza and Ebola are important and challenging issues. It is therefore crucial to characterize the disease progress and epidemics process efficiently and accurately. Computational epidemiology can model the progression of the disease and its underlying contact network, but as yet lacks the ability to process of real-time and fine-grained surveillance data. Social media, on the other hand, provides timely and detailed disease surveillance but is insensible to the underlying contact network and disease model. To address these challenges simultaneously, this paper proposes a novel semi-supervised neural network framework that integrates the strengths of computational epidemiology and social media mining techniques for influenza epidemiological modeling. Specifically, this framework learns social media users’ health states and intervention actions in real time, regularized by the underlying disease model and contact network. The learned knowledge from social media can then be fed into the computational epidemic model to improve the efficiency and accuracy of disease diffusion modeling. We propose an online optimization algorithm that iteratively processes the above interactive learning process. The extensive experimental results provided demonstrated that our approach can not only outperform competing methods by a substantial margin in forecasting disease outbreaks, but also characterize the individual-level disease progress and diffusion effectively and efficiently.

Adoption of the activation function fusion approach to identify human activity recognition in a semi-supervised neural network

INTRODUCTION: Neural networks are a popular type of algorithm for human activity monitoring which can build intelligent systems from labelled data in an automated fashion. Obtaining accurately labelled data is costly; it requires time and effort, which can be cumbersome because it interrupts the user activity stream. In conjunction with the ubiquitous presence of embedded technology, neural networks present new research opportunities for human activity monitoring in smart home environments.OBJECTIVES: We propose a human activity classification method that requires a limited amount of labelled data, which consists of a concatenation method for classifying human activities built upon the fusion of neural network activation functions.METHODS: Our methodology builds a neural network model that receives the sensor data through the input layer to then distribute it among the different vertical hidden layers, which implement different activation functions simultaneously. Next a hidden layer combines activation functions by utilising a concatenation method. Finally, the neural network provides classes to the unlabelled sensing data. We conducted an evaluation utilising an open-access dataset. We compared the activity recognition accuracy of our approach utilising 25%, 50%, and 75% of labelled data against a conventional shallow neural network trained with the 100% of labelled data available.RESULTS: Results show an improvement in the accuracy of the activity classification regardless of the portion of labelled data available. It was observed that the highest achieved accuracy when using 25% of activation function fusion data outperformed results compared to when using 100% of labelled data in a conventional shallow network (i.e., increase in accuracy of 2.7%, 3.7%, 4.8%, and 0.9% across the activity recognition of four subjects).CONCLUSION: The approach proposed showed an improvement in the accuracy of classifying human activity when a limited amount of labelled data is available.

A dynamic semisupervised feedforward neural network clustering

AbstractAn efficient single-layer dynamic semisupervised feedforward neural network clustering method with one epoch training, data dimensionality reduction, and controlling noise data abilities is discussed to overcome the problems of high training time, low accuracy, and high memory complexity of clustering. Dynamically after the entrance of each new online input datum, the code book of nonrandom weights and other important information about online data as essentially important information are updated and stored in the memory. Consequently, the exclusive threshold of the data is calculated based on the essentially important information, and the data is clustered. Then, the network of clusters is updated. After learning, the model assigns a class label to the unlabeled data by considering a linear activation function and the exclusive threshold. Finally, the number of clusters and density of each cluster are updated. The accuracy of the proposed model is measured through the number of clusters, the quantity of correctly classified nodes, and F-measure. Briefly, in order to predict the survival time, the F-measure is 100% of the Iris, Musk2, Arcene, and Yeast data sets and 99.96% of the Spambase data set from the University of California at Irvine Machine Learning Repository; and the superior F-measure results in between 98.14% and 100% accuracies for the breast cancer data set from the University of Malaya Medical Center. We show that the proposed method is applicable in different areas, such as the prediction of the hydrate formation temperature with high accuracy.

An efficient semisupervised feedforward neural network clustering

AbstractWe developed an efficient semisupervised feedforward neural network clustering model with one epoch training and data dimensionality reduction ability to solve the problems of low training speed, accuracy, and high memory complexity of clustering. During training, a codebook of nonrandom weights is learned through input data directly. A standard weight vector is extracted from the codebook, and the exclusive threshold of each input instance is calculated based on the standard weight vector. The input instances are clustered based on their exclusive thresholds. The model assigns a class label to each input instance through the training set. The class label of each unlabeled input instance is predicted by considering a linear activation function and the exclusive threshold. Finally, the number of clusters and the density of each cluster are updated. The accuracy of the proposed model was measured through the number of clusters and the quantity of correctly classified nodes, which was 99.85%, 100%, and 99.91% of the Breast Cancer, Iris, and Spam data sets from the University of California at Irvine Machine Learning Repository, respectively, and the superiorFmeasure results between 98.29% and 100% accuracies for the breast cancer data set from the University of Malaya Medical Center to predict the survival time.

Character recognition using Fourier descriptors and a new form of dynamic semisupervised neural network

A new neural network is presented for pattern recognition tasks. This new network, called the Dynamic Supervised Forward-Propagation Network (DSFPN), although based upon the unsupervised Counterpropagation Network (CPN), trains using a supervised algorithm. In addition it allows unsupervised dynamic growth of the learning layer permitting unknown subclasses to be learnt. The classification capabilities of the network are tested using handwritten numerals presented as Fourier descriptors. The results are compared with those of a Back Propagation Network (BPN) and a Counter-propagation Network. This comparison shows the new network can provide as high a classification accuracy as the BPN and train in a time comparable to the CPN. On average the DSFPN trained in 1 1353 of the epochs required to train the BPN whilst producing a considerably higher accuracy than the CPN.

Automated tuning of an electronic circuit board using the artificial neural network approach

Manufacturing of electronic circuits for microwave communication boards often requires tuning of different circuit characteristics by manual adjustment of several trimmer components, including the trimmer's resistance and capacitance. This manual tuning process was automated by applying the artificial neural network modeling approach. In the considered tuning process, which required manual adjustment of a set of trimmers, multiple specification criteria had to be satisfied by several trimmer rotations. The tuning process was described in terms of three independent steps: the circuit output measurement, trimmer selection, and trimmer rotation. The trimmer selection was performed by a semi-supervised neural network, which learned the patterns of circuit characteristics and the deviations between the ideal and practical outputs. Another network was developed for determination of trimmer rotation rate. The results, based on computer simulation of the tuning process, showed that the developed system improved performance of the tuning process, allowing for automation of the microwave circuit board tuning task in a real manufacturing environment.