Unsupervised Learning Neural Network Research Articles

Detection of malicious and non-malicious website visitors using unsupervised neural network learning

Distributed denials of service (DDoS) attacks are recognized as one of the most damaging attacks on the Internet security today. Recently, malicious web crawlers have been used to execute automated DDoS attacks on web sites across the WWW. In this study, we examine the use of two unsupervised neural network (NN) learning algorithms for the purpose web-log analysis: the Self-Organizing Map (SOM) and Modified Adaptive Resonance Theory 2 (Modified ART2). In particular, through the use of SOM and modified ART2, our work aims to obtain a better insight into the types and distribution of visitors to a public web-site based on their browsing behavior, as well as to investigate the relative differences and/or similarities between malicious web crawlers and other non-malicious visitor groups. The results of our study show that, even though there is a pretty clear separation between malicious web-crawlers and other visitor groups, 52% of malicious crawlers exhibit very ‘human-like’ browsing behavior and as such pose a particular challenge for future web-site security systems. Also, we show that some of the feature values of malicious crawlers that exhibit very ‘human-like’ browsing behavior are not significantly different than the features values of human visitors. Additionally, we show that Google, MSN and Yahoo crawlers exhibit distinct crawling behavior.

An unsupervised learning algorithm: application to the discrimination of seismic events and quarry blasts in the vicinity of Istanbul

Abstract. The results of the application of an unsupervised learning (neural network) approach comprising a Self Organizing Map (SOM), to distinguish micro-earthquakes from quarry blasts in the vicinity of Istanbul, Turkey, are presented and discussed. The SOM is constructed as a neural classifier and complementary reliability estimator to distinguish seismic events, and was employed for varying map sizes. Input parameters consisting of frequency and time domain data (complexity, spectral ratio, S/P wave amplitude peak ratio and origin time of events) extracted from the vertical components of digital seismograms were estimated as discriminants for 179 (1.8 < Md < 3.0) local events. The results show that complexity and amplitude peak ratio parameters of the observed velocity seismogram may suffice for a reliable discrimination, while origin time and spectral ratio were found to be fuzzy and misleading classifiers for this problem. The SOM discussed here achieved a discrimination reliability that could be employed routinely in observatory practice; however, about 6% of all events were classified as ambiguous cases. This approach was developed independently for this particular classification, but it could be applied to different earthquake regions.

Unsupervised Clustering of Web Sessions to Detect Malicious and Non-malicious Website Users

Denial of Service (DoS) attacks are recognized as one of the most damaging attacks on the Internet security today. Recently, malicious web crawlers have been used to execute automated DoS attacks on web sites across the WWW. In this study, we examine the use of two unsupervised neural network (NN) learning algorithms for the purpose web-log analysis: the Self- Organizing Map (SOM) and Modified Adaptive Resonance Theory 2 (Modified ART2). In particular, through the use of SOM and Modified ART2, our work aims to obtain a better insight into the types and distribution of visitors to a public web-site based on their link-traversal behaviour, as well as to investigate the relative differences and/or similarities between malicious web crawlers and other non-malicious visitor groups. The results of our study show that, even though there is a pretty clear separation between malicious web-crawlers and other visitor groups, around 8% of malicious crawlers exhibit very ‘human-like’ browsing behaviour and as such pose a particular challenge for future web-site security systems.

자기조직도에서 최소생성나무의 활용

비지도 학습 신경망모형의 한 종류인 자기조직도(self-organizing map: SOM)는 고차원 자료를 차원축소하고 저차원지도를 통해 유사한 개체를 군집화하는 방법이며 다양한 분야의 데이터에 적용되고 있다. 한편 최소생성나무(minimal spanning tree: MST)는 개체점들을 닫힌 루프 없이 가장 짧게 선분으로 연결하는 그래프 방법이다. 본 연구에서는 부노드 자기조직도에 최소생성나무를 적용하여 부노드 간 거리를 근사적으로 나타내는 자료 시각화 방법과 자기조직도의 최적 형태와 크기를 결정하기 위한 거리 측도를 제안하였다. 또한 피서의 붓꽃자료와 실제 유전자발현자료 및 모의생성 자료에 적용하여 이 방법의 유용성을 살펴보았다. As one of the unsupervised learning neural network methods, self-organizing maps(SOM) are applied to various fields. It reduces the dimension of multidimensional data by representing observations on the low dimensional manifold. On the other hand, the minimal spanning tree(MST) of a graph that achieves the most economic subset of edges connecting all components by a single open loop. In this study, we apply the MST technique to SOM with subnodes. We propose SOM's with embedded MST and a distance measure for optimum choice of the size and shape of the map. We demonstrate the method with Fisher's Iris data and a real gene expression data. Simulated data sets are also analyzed to check the validity of the proposed method.

Application of self-organizing maps in analysis of wave soldering process

This paper presents an overview of a data analysis method based on self-organizing maps (SOM), a well-known unsupervised neural network learning algorithm, which was applied to a lead-free wave soldering process. The aim of the study was to determine whether the neural network modeling method could be a useful and time-saving way to analyze data from a discrete manufacturing process, such as wave soldering, which is a widely used technique in the electronics industry to solder components on printed circuit boards. The data variables were mostly various process parameters, but also some solder defect numbers were present in the data as a measure of the product quality. The data analysis procedure went as follows. At first, the process data were modeled using the SOM-algorithm. Next, the neuron reference vectors of the formed map were clustered to reveal the desired dominating elements of each territory of the map. At the final stage, the clusters were utilized as sub-models to indicate variable dependencies in these sub-models. The results show that the method presented here can be a good way to analyze this type of process data because interesting interactions between certain process parameters and solder defects were found by means of this data-driven modeling method.

Unsupervised learning neural network with convex constraint: Structure and algorithm

This paper proposes a kind of unsupervised learning neural network model, which has special structure and can realize an evaluation and classification of many groups by the compression of data and the reduction of dimension. The main characteristics of the samples are learned after being trained. In order to realize unsupervised learning of neural network structure with convex constraint, an iterative computation method is proposed that makes use of alternating projection between two convex sets. The final example proves that this method can detect instructions without a mass of supervised data and it converges fast.

Classifying Construction Contractors Using Unsupervised-Learning Neural Networks

Contractor prequalification involves the screening of contractors by a project owner to determine their competence to complete the project on time, within budget, and to expected quality standards. The process of prequalification involves a large number of contractors, each being represented by many attributes. A neural network model was applied to aid in the prequalification process by classifying contractors into groups based on similarity in performance using the financial ratios of liquidity, activity, profitability, and leverage. Contractors are represented in this model by patterns in four-dimensional space. Patterns of similar performance tend to form clusters intercepting regions of low pattern density in between. A neuron with weights is used as a classifier to set a decision boundary between clusters. The method basically iterates the neuron weights to move the decision boundary to a place of low pattern density. Then, the statistical hypothesis testing of the mean difference of two independent samples was used to validate the classification of the parent class to the two child classes considering the four ratios separately. The method was used hierarchically to classify a group of 245 contractors into classes of small numbers. Finally, the inferred procedure of classification proves that the neural network model classified the four-dimension pattern representing contractors efficiently.

Uma abordagem estatística para a previsão de potência reativa em sistemas elétricos

As previsões das potências ativa e reativa se constituem numa ferramenta de extremo valor na monitoração de Sistemas de Energia Elétrica. Com base nisso, este trabalho tem por objetivo, apresentar uma nova técnica de previsão horária de potência reativa a curto prazo, por subestação, baseado na linearidade existente entre as potências ativa e reativa, ou seja, a potência reativa pode ser explicada diretamente pela potência ativa através de uma regressão linear. A fim de melhorar o desempenho da previsão, são incorporadas ao modelo simples, variáveis defasadas das potências acopladas a uma correção para autocorrelação serial (Método Iterativo de Cochrane-Orcutt). E, além disso, como os dados de potência reativa apresentam comportamento heterocedástico, o método de estimação dos coeficientes do modelo por mínimos quadrados não se mostra adequado, por isso, foi utilizada uma solução robusta conhecida como mínimos quadrados reponderados iterativamente (IRLS). A previsão de potência reativa a curto-prazo é dividida em "in sample" e "out of sample", onde esta é aplicada a períodos horários em até um mês à frente. Com intuito de aumentar a precisão dos resultados, os dados são classificados através de uma rede neural não supervisionada conhecida como mapas auto-organizáveis de Kohonen (SOM).

Organizing and Visualizing Software Repositories Using the Growing Hierarchical Self-Organizing Map

A software repository, a place where reusable components are stored and searched for, is a key ingredient for instituting and popularizing software reuse. It is vital that a software repository should be well-organized and provide efficient tools for developers to locate reusable components that meet their requirements. The growing hierarchical self-organizing map (GHSOM), an unsupervised learning neural network, is a powerful data mining technique for the clustering and visualization of large and complex data sets. The resulting maps, serving as retrieval interfaces, can be beneficial to developers in obtaining better insight into the structure of a software repository and increasing their understanding of the relationships among software components. The GHSOM, which is an improvement over the basic self-organizing map (SOM), can adapt its architecture during its learning process and expose the hierarchical structure that exists in the original data. In this paper, we demonstrate the potential of the GHSOM for the organization and visualization of a collection of reusable components stored in a software repository, and compare the results with the ones obtained by using the traditional SOM.

An extended self-organizing map network for market segmentation—a telecommunication example

Kohonen's self-organizing map (SOM) network is an unsupervised learning neural network that maps an n-dimensional input data to a lower dimensional output map while maintaining the original topological relations. The extended SOM network further groups the nodes on the output map into a user specified number of clusters. In this research effort, we applied this extended version of SOM networks to a consumer data set from American Telephone and Telegraph Company (AT&T). Results using the AT&T data indicate that the extended SOM network performs better than the two-step procedure that combines factor analysis and K-means cluster analysis in uncovering market segments.

Design of cellular manufacturing systems using Latent Semantic Indexing and Self Organizing Maps

A new, efficient clustering method for solving the cellular manufacturing problem is presented in this paper. The method uses the part-machine incidence matrix of the manufacturing system to form machine cells, each of which processes a family of parts. By doing so, the system is decomposed into smaller semi-independent subsystems that are managed more effectively improving overall performance. The proposed method uses Self Organizing Maps (SOMs), a class of unsupervised learning neural networks, to perform direct clustering of machines into cells, without first resorting to grouping parts into families as done by previous approaches. In addition, Latent Semantic Indexing (LSI) is employed to significantly reduce the complexity of the problem resulting in more effective training of the network, significantly improved computational efficiency, and, in many cases, improved solution quality. The robustness of the method and its computational efficiency has been investigated with respect to the dimension of the problem and the degree of dimensionality reduction. The effectiveness of grouping has been evaluated by comparing the results obtained with those of the k-means classical clustering algorithm.

주성분 자기조직화 지도 PC-SOM

자기조직화 지도(SOM)은 T. 코호넨의 주도하에 개발된 비지도 학습 신경망 모형이다. 그 동안 패턴인식과 문서검색 분야에 주로 응용되어 왔기 때문에 통계학 분야에서는 덜 알려졌으나, 최근 K-평균 군집화에 대한 대안적 데이터 마이닝 기법으로 활용되기 시작하였다. 본 연구에서는 SOM의 한 버전인 PC-SOM(주성분 자기조직화 지도)을 제안하고 활용 예를 제시하고자 한다. PC-SOM은 1차원적 SOM 알고리즘을 반복 수행하여 2차원, 3차원 등의 SOM을 얻는 방법이기 때문에 기존 SOM과는 달리 사전 Map의 크기를 확정할 필요가 없다. 또한, 기존 SOM에 비하여 향상된 시각화를 가능하게 한다. Self-organizing map (SOM), a unsupervised learning neural network, has been developed by T. Kohonen since 1980's. Main application areas were pattern recognition and text retrieval. Because of that, it has not been spread to statisticians until late. Recently, SOM's are frequently drawn in data mining fields. Kohonen's SOM, however, needs improvements to become a statistician's standard tool. First, there should be a good guideline as for the size of map. Second, an enhanced visualization mode is wanted. In this study, principal components self-organizing map (PC-SOM), a modification of Kohonen's SOM, is proposed to meet such needs. PC-SOM performs one-dimensional SOM during the first stage to decompose input units into node weights and residuals. At the second stage, another one-dimensional SOM is applied to the residuals of the first stage. Finally, by putting together two stages, one obtains two-dimensional SOM. Such procedure can be easily expanded to construct three or more dimensional maps. The number of grid lines along the second axis is determined automatically, once that of the first axis is given by the data analyst. Furthermore, PC-SOM provides easily interpretable map axes. Such merits of PC-SOM are demonstrated with well-known Fisher's iris data and a simulated data set.

A pattern reordering approach based on ambiguity detection for online category learning

Pattern reordering is proposed as an alternative to sequential and batch processing for online category learning. Upon detecting that the categorization of a new input pattern is ambiguous, the input is postponed for a predefined time, after which it is reexamined and categorized for good. This approach is shown to improve the categorization performance over purely sequential processing, while yielding a shorter input response time, or latency, than batch processing. In order to examine the response time of processing schemes, the latency of a typical implementation is derived and compared to lower bounds. Gaussian and softmax models are derived from reject option theory and are considered for detecting ambiguity and triggering pattern postponement. The average latency and Rand Adjusted clustering score of reordered, sequential, and batch processing are compared through computer simulation using two unsupervised competitive learning neural networks and a radar pulse data set.

A new intelligent SOFM-based sampling plan for advanced process control

Sample measurement inspecting for a process parameter is a necessity in semiconductor manufacturing because of the prohibitive amount of time involved in 100% inspection while maintaining sensitivity to all types of defects and abnormality. In current industrial practice, sample measurement locations are chosen approximately evenly across the wafer, in order to have all regions of the wafer equally well represented, but they are not adequate if process-related defective chips are distributed with spatial pattern within the wafer. In this paper, we propose the methodology for generating effective measurement sampling plan for process parameter by applying the Self-Organizing Feature Map (SOFM) network, unsupervised learning neural network, to wafer bin map data within a certain time period. The sampling plan specifies which chips within the wafer need to be inspected, and how many chips within the wafer need to be inspected for a good sensitivity of 100% wafer coverage and defect detection. We finally illustrate the effectiveness of our proposed sampling plan using actual semiconductor fab data.

Analysis of gene expression data using self-organizing maps

DNA microarray technologies together with rapidly increasing genomic sequence information is leading to an explosion in available gene expression data. Currently there is a great need for efficient methods to analyze and visualize these massive data sets. A self-organizing map (SOM) is an unsupervised neural network learning algorithm which has been successfully used for the analysis and organization of large data files. We have here applied the SOM algorithm to analyze published data of yeast gene expression and show that SOM is an excellent tool for the analysis and visualization of gene expression profiles.

Noise injection into inputs in sparsely connected Hopfield and winner-take-all neural networks

In this paper, we show that noise injection into inputs in unsupervised learning neural networks does not improve their performance as it does in supervised learning neural networks. Specifically, we show that training noise degrades the classification ability of a sparsely connected version of the Hopfield neural network, whereas the performance of a sparsely connected winner-take-all neural network does not depend on the injected training noise.

Unsupervised neural network learning procedures for feature extraction and classification

In this article, we review unsupervised neural network learning procedures which can be applied to the task of preprocessing raw data to extract useful features for subsequent classification. The learning algorithms reviewed here are grouped into three sections: information-preserving methods, density estimation methods, and feature extraction methods. Each of these major sections concludes with a discussion of successful applications of the methods to real-world problems.

Developing a nonsymbolic phonetic notation for speech synthesis

The goal of the research presented here is to apply unsupervised neural network learning methods to some of the lower-level problems in speech synthesis currently performed by rule-based systems. T...

PS-33-9 An unsupervised learning neural network system in the pattern recognition of MUAPS

PS-33-9 An unsupervised learning neural network system in the pattern recognition of MUAPS

A hybrid artificial neural network—differential dynamic programming approach for short-term hydro scheduling

Abstract In this paper, a hybrid artificial neural network-differential dynamic programming (ANN-DDP) method for the scheduling of short-term hydro generation is developed. The purpose of short-term hydro scheduling is to find the optimal amounts of generated powers for the hydro units in the system for the next N (N= 24 in this work) hours in the future. In the proposed method, the DDP procedures are performed offline on historical load data. The results are compiled and valuable information is obtained by using ANN algorithms. The DDP algorithm is then performed online according to the obtained information to give the hydro generation schedule for the forecasted load. Two types of ANN algorithm, the supervised learning neural network by Rumelhart et al. and the unsupervised learning neural network by Kohonen, are employed and compared in this paper. The effectiveness of the proposed approach is demonstrated by the short-term hydro scheduling of Taiwan power system which consists of ten hydro plants. It is concluded from the results that the proposed approach can significantly reduce the execution time of the conventional differential dynamic programming algorithm which is required to reach proper hydro generation schedules.