Unsupervised Data Research Articles

Efficient Binary Static Code Data Flow Analysis Using Unsupervised Learning

The ever-increasing need to ensure that code is reliably, efficiently and safely constructed has fueled the evolution of popular static binary code analysis tools. In identifying potential coding flaws in binaries, tools such as IDA Pro are used to disassemble the binaries into an opcode/ assembly language format in support of manual static code analysis. Because of the highly manual and resource-intensive nature involved with analyzing large binaries, the probability of overlooking potential coding irregularities and inefficiencies is quite high. In this paper, a light-weight, unsupervised data flow methodology is described which uses highly correlated data flow graph (CDFGs) to identify coding irregularities such that analysis time and required computing resources are minimized. Such analysis accuracy and efficiency gains are achieved by using a combination of graph analysis and unsupervised machine learning techniques which allows an analyst to focus on the most statistically significant flow patterns while performing binary static code analysis.

Using Gaussian mixture model clustering to explore morphology and standardized production of ceramic vessels: A case study of pottery from Late Bronze Age Greece

This paper examines vessel morphology and degrees of standardization of ceramic vessels using Gaussian mixture model cluster analysis (GMMC). GMMC is an unsupervised data mining technique that identifies natural groups in a dataset. This project first tests whether GMMC can classify certain shape categories correctly according to human assigned Furumark Shape (FS) pot types using basic vessel dimensions. We then propose that vessels can be considered standardized if they were grouped into the cluster of their own shape category (or pot type). Vessel data are derived from the Late Bronze Age Petsas House ceramic workshop, located in the settlement of Mycenae in southern Greece. The sample comes from a sealed well deposit found within the workshop. GMMC identified three clusters within a group of 488 pots that correspond to three known vessel types with a high degree of sensitivity and specificity so that clustered shapes mostly align with shape categories, and each shape can be defined as standardized. The maximization of information enabled by GMMC and the ability to analyze multiple interrelated variables can thus indicate cognitive approaches to vessel production, such as the perception of vessel shape by potters, and socio-economic factors relating to use by consumers.

Contrastive learning from label distribution: A case study on text classification

State-of-the-art text classification models are dominated by deep neural networks, but they still struggle to the issue of poor generalization ability when using cross entropy loss for training. One of the reasons is the training samples are usually annotated with hard labels, ignoring the inter-label relationship among these labels. This results in the outputted distributions to violate the label-correlation. Although the widely used contrastive learning is able to learn highly-expressive feature representations that generalize well across the training and test sets, contrastive learning in the embedding space is difficult to model the label correlation and may still output unreasonable label distributions. In this paper, we suggest contrastive learning using label distributions and present a novel label-level contrastive learning (LLCL) paradigm, which can constrain the unreasonable label distributions from model outputs. We hypothesize that the label distributions of the instances in the same class are more similar than those from other classes. We introduce two label-level contrastive learning losses, namely supervised contrastive learning and self-supervised contrastive learning. After adding our proposed losses to the cross-entropy loss as regularizer for the training text classification model, our model obtains the average improvement of 0.74% over the strong RoBERTa-Large baseline on ten datasets. Particularly, our contrastive learning in the label space can effectively capture label correlations better than that in the embedding space, achieving an improvement of 6.2% in the top-2 accuracy on the SST-5 dataset. We also demonstrate that our proposed method is more effective, especially in the text classification tasks with a large label space or limited labeled data. Last but not least, our model does not rely on any kind of specialized architectures, data augmentation methods, or additional unsupervised data.

Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition

Deep Learning approaches have brought solutions, with impressive performance, to general classification problems where wealthy of annotated data are provided for training. In contrast, less progress has been made in continual learning of a set of non-stationary classes, mainly when applied to unsupervised problems with streaming data.Here, we propose a novel incremental learning approach which combines a deep features encoder with an Open-Set Dynamic Ensembles of SVM, to tackle the problem of identifying individuals of interest (IoI) from streaming face data. From a simple weak classifier trained on a few video-frames, our method can use unsupervised operational data to enhance recognition. Our approach adapts to new patterns avoiding catastrophic forgetting and partially heals itself from miss-adaptation. Besides, to better comply with real world conditions, the system was designed to operate in an open-set setting. Results show a benefit of up to 15% F1-score increase respect to non-adaptive state-of-the-art methods.

Contrastive time–frequency learning for radar signal sorting

With the development of radar technology, the electromagnetic environment becomes more and more complex. The acquisition of labeled data is costly, and the existing radar signal sorting and recognition methods have poor generalization capability. To address this problem, a radar signal recognition technique based on time–frequency analysis and contrastive learning is proposed in this paper. The classification process is improved according to the characteristics of radar signals without using labels. The proposed method, contrastive time–frequency learning, can use the unsupervised data to sort and identify radar signals. By augmenting the data with various channel conditions, the generalization ability of the classification model is also improved, especially for the situation with low SNR. The experiments prove the good performance of the proposed method, and classification accuracy for radar signals can be up to 98.17%.

Topological structural analysis based on self-adaptive growing neural network for shape feature extraction

Self-organizing neural network has been widely used to extract the topological structure of image shape because it features topological preservation, dynamic adaptation, clustering and dimensionality reduction. However, it is difficult to automatically extract the topology structure with an appropriate number of neurons from the complex and diverse data. In this paper, a novel self-organizing neural network called self-adaptive growing neural network (SAGNN) is proposed, which can generate an appropriate number of neurons autonomously according to the size of input data without setting the total number of neurons in advance. Firstly, Similarity Evaluation Index (SEI) is proposed to evaluate the similarity between the output network and the input space. Then, on the basis of growing neural gas (GNG) network, the SEI as a network growth control condition is introduced into the SAGNN, so that the SAGNN can grow neurons on demand until the expected quantization error is not significantly improved. Experiments involving both artificial and real data sets show that SAGNN can extract the topological structure from the unsupervised data without any prior conditions (including the appropriate number of neurons).

A generalized framework for unsupervised learning and data recovery in computational fluid dynamics using discretized loss functions

The authors present generalized finite-volume-based discretized loss functions integrated into pressure-linked algorithms for physics-based unsupervised training of neural networks (NNs). In contrast to automatic differentiation-based counterparts, discretized loss functions leverage well-developed numerical schemes of computational fluid dynamics (CFD) for tailoring NN training specific to the flow problems. For validation, neural network-based solvers (NN solvers) are trained by posing equations such as the Poisson equation, energy equation, and Spalart–Allmaras model as loss functions. The predictions from the trained NNs agree well with the solutions from CFD solvers while also providing solution time speed-ups of up to seven times. Another application of unsupervised learning is the novel hybrid loss functions presented in this study. Hybrid learning combines the information from sparse or partial observations with a physics-based loss to train the NNs accurately and provides training speed-ups of up to five times compared with a fully unsupervised method. Also, to properly utilize the potential of discretized loss functions, they are formulated in a machine learning (ML) framework (TensorFlow) integrated with a CFD solver (OpenFOAM). The ML-CFD framework created here infuses versatility into the training by giving loss functions access to the different numerical schemes of the OpenFOAM. In addition, this integration allows for offloading the CFD programming to OpenFOAM, circumventing bottlenecks from manually coding new flow conditions in a solely ML-based framework like TensorFlow.

NoC Application Mapping Optimization Using Reinforcement Learning

Application mapping is one of the early stage design processes aimed to improve the performance of Network-on-Chip. Mapping is an NP-hard problem. A massive amount of high-quality supervised data is required to solve the application mapping problem using traditional neural networks. In this article, a reinforcement learning–based neural framework is proposed to learn the heuristics of the application mapping problem. The proposed reinforcement learning–based mapping algorithm (RL-MAP) has actor and critic networks. The actor is a policy network, which provides mapping sequences. The critic network estimates the communication cost of these mapping sequences. The actor network updates the policy distribution in the direction suggested by the critic. The proposed RL-MAP is trained with unsupervised data to predict the permutations of the cores to minimize the overall communication cost. Further, the solutions are improved using the 2-opt local search algorithm. The performance of RL-MAP is compared with a few well-known heuristic algorithms, the Neural Mapping Algorithm (NMA) and message-passing neural network-pointer network-based genetic algorithm (MPN-GA). Results show that the communication cost and runtime of the RL-MAP improved considerably in comparison with the heuristic algorithms. The communication cost of the solutions generated by RL-MAP is nearly equal to MPN-GA and improved by 4.2% over NMA, while consuming less runtime.

Grouping of Village Status in West Java Province Using the Manhattan, Euclidean and Chebyshev Methods on the K-Mean Algorithm

The Ministry of Villages, Development of Disadvantaged Areas and Transmigration (Ministry of Village PDTT) is a ministry within the Indonesian Government in charge of rural and rural development, empowerment of rural communities, accelerated development of disadvantaged areas, and transmigration. Village Potential Data for 2014 (Podes 2014) in West Java Province is data issued by the Central Statistics Agency in collaboration with the Ministry of Village PDTT which is in unsupervised data format, consists of 5319 village data. The Podes 2014 data in West Java Province were made based on the level of village development (village specific) in Indonesia, by making the village as the unit of analysis. Base on the Regulation of the Minister of Villages, Disadvantaged Areas and Transmigration of the Republic of Indonesia number 2 of 2016 concerning the village development index, the Village is classified into 5 village status, namely Very Disadvantaged Village, Disadvantaged Village, Developing Village, Advanced Village and Independent Village based on the ability to manage and increase the potential of social, economic and ecological resources. Village status is in fact inseparable from village development that is under government funding support. However, village development funds have not been distributed effectively and accurately according to the conditions and potential of the village due to the lack of clear information about the status of the village. Therefore, the information regarding the villages priority in term of which villages needs more funding and attention from the government is still lacking. Data mining is a method that can be used to group objects in a data into classes that have the same criteria (clustering). One of the algorithms that can be used for the clustering process is the k-means algorithm. Data grouping using k-means is done by calculating the closest distance from data to a centroid point. In this study, different types of distance calculation in the K-means algorithm are compared. Those types are Manhattan, Euclidean and Chebyshev. Validation tests have been carried out using the execution time and Davies Bouldin index. From this test, the data Village Potential 2014 in West Java province have grouped all the 5 status of the village with the obtained number of villages for each cluster is a cluster village Extremely Backward many as 694 villages, cluster Villages 567 villages, cluster village Evolving as much as 1440 villages, the cluster with Desa Maju1557 villages and the cluster Independent Village for 1061 villages. For distance calculation, Chebyshev has the most efficient accumulation time of 1 second compared to Euclidean 1.6 seconds and Manhattan 2.4 seconds. Meanwhile, the Euclidean method has the value, Davies Index most optimal which is 0.886 compared to the Manhattan method 0.926 and Chebyshev 0.990.

Characterizing driver speeding behavior when using partial-automation in real-world driving

Objective: Speeding is a prevalent and complex risky behavior that can be affected by many factors. Understanding how drivers speed is important for developing countermeasures, especially as new automation features emerge. The current study seeks to identify and describe types of real-world speeding behaviors with and without the use of partial-automation. Methods: This study used a combination of supervised and unsupervised data analysis techniques to assess relevant factors in real-world speeding epochs, extracted from the MIT Advanced Vehicle Technology Naturalistic Driving Study, and classified them into distinct speeding behaviors. Speeding epochs were defined as traveling at least 5 mph over the speed limit for a minimum duration of 3 s. Vehicle speed-exceedance profiles were characterized over time using Dynamic Time Warping and included in multivariate models that evaluated the associations between different features of the speeding epochs, such as speeding duration and magnitude. Finally, the identified features were used to cluster speeding behaviors using the Gower dissimilarity measure. Results: The analysis yielded four types of behaviors in both partially-automated and manual driving: (i) Incidental speeding (low duration, low magnitude), (ii) Moderate speeding (low duration, moderate magnitude), (iii) Elevated speeding (moderate duration, high magnitude), and (iv) Extended speeding (long duration, high magnitude). When comparing the behaviors with and without partial-automation use, both Incidental and Moderate speeding were found to have significantly longer durations with partial-automation than manual driving. Elevated speeding was found to be more prevalent and associated with higher magnitudes during manual than with partially-automated driving. Finally, although Extended speeding was more prevalent during automation use, it was associated with a lower mean and maximum speed magnitude compared to Extended speeding during manual driving. Conclusions: This work highlights the variability in speeding behavior between and within partially-automated and manual driving. The design of systems that mitigate risky speeding behaviors should consider targeting divergent behaviors observed between manual and automated driving as a mechanism to mitigate the prevalence of the different behaviors associated with each state.

Unsupervised Learning and Online Anomaly Detection

The Large Hadron Collider (LHC) demands a huge amount of computing resources to deal with petabytes of data generated from High Energy Physics (HEP) experiments and user logs, which report user activity within the supporting Worldwide LHC Computing Grid (WLCG). An outburst of data and information is expected due to the scheduled LHC upgrade, viz., the workload of the WLCG should increase by 10 times in the near future. Autonomous system maintenance by means of log mining and machine learning algorithms is of utmost importance to keep the computing grid functional. The aim is to detect software faults, bugs, threats, and infrastructural problems. This paper describes a general-purpose solution to anomaly detection in computer grids using unstructured, textual, and unsupervised data. The solution consists in recognizing periods of anomalous activity based on content and information extracted from user log events. This study has particularly compared One-class SVM, Isolation Forest (IF), and Local Outlier Factor (LOF). IF provides the best fault detection accuracy, 69.5%.

Quality Assessment of Medicinal Plants via Chemometric Exploration of Quantitative NMR Data: A Review

Since ancient times, herbal medicines (HM) have played a vital role in worldwide healthcare systems. It is therefore critical that a thorough evaluation of the quality and control of its complicated chemical makeup be conducted, in order to ensure its efficacy and safety. The notion of HM chemical prints, which aim to acquire a full characterization of compound chemical matrices, has become one of the most persuasive techniques for HM quality evaluation during the last few decades. The link between NMR and chemometrics is discussed in this article. The chemometric latent variable technique has been shown to be extremely valuable in inductive studies of biological systems as well as in solving industrial challenges. The results of unsupervised data exploration utilizing main component analysis as well as the multivariate curve resolution, were various. On the other hand, many contemporary NMR applications in metabolomics and quality control are based on supervised regression or classification analyses.

Deer hunting optimization technique for clustering unsupervised data in data mining

Unsupervised data clustering investigation is a standout among the most valuable tools and is an informative task in data mining that looks to characterize similar articles’ gatherings. One of the eminent algorithms for the clustering field is K-means clustering. Scholars recommended enhancing the nature of K-means, and optimization algorithms were hybridized. In this study, a heuristic calculation, deer hunting optimization algorithm (DHOA), was adjusted for K-means data clustering by altering the fundamental parameters of DHOA calculation, which are propelled from the characteristic enlivened calculations. During this work, a new human-based descriptive DHOA has been developed following a human deer hunting strategy. In order to attack the fawn, hunters update their positions based on the movement of the leader and backward movement while also considering the angle of the deer. In this work, the DHOA was hybridized with K-means clustering and the performance of the proposed approach is tested against UCI repository data with different algorithms.

Versatile unsupervised anomaly detection method for RTE-based networks

Reliability and dependability are critical demands of the fourth industrial revolution that Real-time Ethernet (RTE) networks have to meet. The use of anomaly detection and prevention techniques can further enhance existing RTE networks. This work presents a general and efficient anomaly detection strategy based on machine learning techniques. The proposal is of general purpose since only normal (i.e not anomalous) traffic data and statistical features are used during the training phase of the classifier. These features are derived from data sets extracted from unsupervised traffic data using a sliding window algorithm. The efficiency of the proposal depends on the proper selection of the sliding window algorithm’ parameters. In this work, an optimization strategy appropriately selects the algorithm’s step and window size. An Adapted Correlation based on Feature Selection indicates the most relevant features reducing the data dimensionality, improving classifier performance and efficiency. Finally, a One-Class Support Vector Machine algorithm is trained and used for classifying the traffic data sets. The authors applied the proposed anomaly detection strategy in real network data (PROFINET and Ethernet/IP Networks). The results demonstrate the proposal’s efficiency and accuracy.

A sequence-to-sequence remaining useful life prediction method combining unsupervised LSTM encoding-decoding and temporal convolutional network

Remaining useful life (RUL) prediction methods based on deep neural networks (DNNs) have received much attention in recent years. The collected time-series signals are usually processed by the sliding time window method into several segments with the same sequence length as the input. However, the signal processing is not only time-consuming, but also relies too much on personal experience. Moreover, the length of the time window affects the prediction results and the prediction range. Obviously, it is more desirable to remove the data processing and use an entire time series signal as the input for predicting the RUL, i.e. sequence-to-sequence RUL prediction. In order to remove the shortcomings of signal processing, this paper uses a long short-term memory (LSTM) and encoding-decoding framework to construct an unsupervised sequence data processing model. Then, a temporal convolutional network, based on a convolutional neural network, is used to further process the output data of the unsupervised sequence data processing model. The proposed sequence-to-sequence RUL prediction method not only maintains the complete sequence of the data, but has a good capability for data processing. The open access C-MAPSS simulation datasets are used for validation. The validation results show that the proposed method can realize unsupervised sequence signal reconstruction. Moreover, it has better prediction results and prediction efficiency.

A survey on the use of association rules mining techniques in textual social media

The incursion of social media in our lives has been much accentuated in the last decade. This has led to a multiplication of data mining tools aimed at obtaining knowledge from these data sources. One of the greatest challenges in this area is to be able to obtain this knowledge without the need for training processes, which requires structured information and pre-labelled datasets. This is where unsupervised data mining techniques come in. These techniques can obtain value from these unstructured and unlabelled data, providing very interesting solutions to enhance the decision-making process. In this paper, we first address the problem of social media mining, as well as the need for unsupervised techniques, in particular association rules, for its treatment. We follow with a broad overview of the applications of association rules in the domain of social media mining, specifically, their application to the problems of mining textual entities, such as tweets. We also focus on the strengths and weaknesses of using association rules for solving different tasks in textual social media. Finally, the paper provides a perspective overview of the challenges that association rules must face in the next decade within the field of social media mining.

Discretionary Cut-In Driving Behavior Risk Assessment Based on Naturalistic Driving Data

Cut-in driving behavior is one of the basic micro traffic actions for vehicles. A risk assessment helps vehicles execute the behavior well and determine how to react to the same maneuver from other traffic participants when following a leading car or truck. This article presents a discretionary cut-in driving behavior risk assessment method based on field driving data and a united algorithm that is designed to a combine decision tree and a support vector machine to achieve enhanced sensitivity for the riskiest traffic conditions. To build the learning database, a wavelet method is employed to filter naturalistic driving data, incorporating the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means approach. An unsupervised data learning method is used to categorize the impact on vehicles in the target lane, indicated by a target vehicle’s average and maximum deceleration, into three groups. Experiment results based on self-collected and public databases show that tested vehicles are aware of the risk presented by other cars’ and trucks’ cut-in driving as well as their own impact on traffic participants in the target lane.

A Short Text Clustering Approaches in Social Media

In general, handling bag-of-words representation or TF-IDF is an arduous task with short text clustering as it results in sparse vector representation mainly for short texts. This paper provides an in-depth study on learning predictor features through an auto encoder model and sentence embedding technique. Assignment of user from any clustering technique as a process of supervision in order to update the weight of the encoder analysis. Short text related datasets validate of measure effective methods and algorithms for short text unsupervised data. Among this short text are not able to clustering in some social media. This paper discusses the challenge and methods to cluster in social media short text challenging issues. Clustering methods of algorithms to sort the problem to recovery the short text using K-means algorithms of Convolution Neural Network method. This paper provides a comprehensive review of short text prediction using the clustering algorithm, explore the research challenges, and open issues in this area.

Improving spatial variation of ground-level PM2.5 prediction with contrastive learning from satellite imagery

Convolutional Neural Networks (CNNs) are a promising technique to predict highly localized fine particulate matter (i.e., PM2.5 levels) based on high-resolution satellite imagery. Unfortunately, CNNs typically require large amounts of supervised data to perform well, whereas this application generally has lots of unsupervised data (all satellite imagery) and relatively sparse supervised data (measurements from ground sensors). Previous work used transfer learning from another visual task to initialize the CNN weights; however, we hypothesize that standard transfer learning strategies would bias the CNN to focus on irrelevant details of the image for our applications. Instead, we develop a novel framework called Spatiotemporal Contrastive Learning (SCL) to pre-train the CNN. We test both regular contrastive learning and SCL on predicting PM2.5 levels from satellite images in two different cities, Delhi and Beijing, and compare to CNNs with parameters initialized randomly and by transfer learning. Our results show that regular contrastive learning and our SCL frameworks both manage to better capture spatial variation of ground-level PM2.5 concentrations compared to traditional initialization schemes, and that this performance gap increases as the number of ground sensors decreases, implying that the approach will be even more valuable in cities with fewer ground sensors. Our work demonstrates that contrastive learning is a powerful pre-training technique to build better spatial maps of PM2.5, and can be broadly applied in related situations.

Multiclass anomaly detection for unsupervised and semi-supervised data based on a combination of negative selection and clonal selection algorithms

A key challenge in anomaly detection is the imbalance between the amounts of normal and abnormal signal data. Specifically, the amount of abnormal signal data is considerably less than that of normal signal data. To solve this problem, techniques for detecting abnormalities using only normal signal data derived from artificial immune systems (AISs) have been investigated. A representative example is the negative selection algorithm (NSA), which classifies data and detects anomalies using only normal signals through a process that mimics the underlying principle of vertebrate immunity. However, the NSA is optimized to detect only two classes of anomalies. Therefore, in this study, we developed a multiclass anomaly detection algorithm that hybridizes the principles of NSA and the clonal selection algorithm (CSA). We improved this algorithm using unsupervised and semi-supervised learning algorithms to conveniently detect anomalies at actual industrial sites. This paper presents a process for applying an AIS algorithm to anomaly detection using the evolution of data-based anomaly-detection algorithms. In particular, we leveraged the NSA principle of classification through semi-supervised learning to enable multiple classifications of unlabeled data. The obtained detector data formed clones optimized by the CSA and had constant memory, thus improving the classification accuracy and reducing run time. The proposed algorithm was validated using an intelligent maintenance system bearing dataset and a vacuum deposition equipment dataset.