Self-organizing Map Method Research Articles

Segmentation of Natural Gas Customers in Industrial Sector Using Self-Organizing Map (SOM) Method

The usage of the natural gas which is non-renewable energy, needs to be more efficient. Therefore, customer segmentation becomes necessary to set up a marketing strategy to be right on target or to determine an appropriate fee. This research was conducted at PT PGN using one of data mining method, i.e. Self-Organizing Map (SOM). The clustering process is based on the characteristic of its customers as a reference to create the customer segmentation of natural gas customers. The input variables of this research are variable of area, type of customer, the industrial sector, the average usage, standard deviation of the usage, and the total deviation. As a result, 37 cluster and 9 segment from 838 customer data are formed. These 9 segments then employed to illustrate the general characteristic of the natural gas customer of PT PGN.

Self-organizing maps for indications of airborne polychlorinated biphenyl (PCBs) and organochlorine pesticide (OCPs) dependence on spatial and meteorological parameters

This paper investigates the relation of polychlorinated biphenyls (PCBs) and organochlorine pesticides (OCPs) in air samples with meteorological parameters (temperature, atmospheric pressure and relative humidity) using the Kohonen self-organizing map (SOM). Both gas- and particle-adsorbed phase of 20 PCB congeners and 7 OCPs including the three new ones (α-HCH, β-HCH, and γ-HCH) listed in the Stockholm Convention were collected during a one-year period at urban locations in Zagreb (Croatia). Moving beyond existing studies, the SOM analysis showed that the meteorological characteristics of transient seasons such as spring had no influence on the dissimilarities in the behavior of PCBs and OCPs. Towards the identification of pollutant spatial patterns, the SOM did not isolate a clear phenomenon probably due to the absence of local pollution sources contributing to the elevated concentrations of these compounds. Overall, our results have shown that the SOM method, by recognizing significant differences among PCB and OCP seasonality, could be recommended in the analysis of pollutant distribution depending on temperature and atmospheric pressure.

Long-Term Trends, Variability and Extremes of In Situ Sea Surface Temperature Measured Along the Eastern Adriatic Coast and its Relationship to Hemispheric Processes

This paper examines long-term series of in situ sea surface temperature (SST) data measured at nine coastal and one open sea stations along the eastern Adriatic Sea for the period 1959–2015. Monthly and yearly averages were used to document SST trends and variability, while clustering and connections to hemispheric indices were achieved by applying the Principal Component Analysis (PCA) and Self-Organizing Maps (SOM) method. Both PCA and SOM revealed the dominance of temporal changes with respect to the effects of spatial differences in SST anomalies, indicating the prevalence of hemispheric processes over local dynamics, such as bora wind spatial inhomogeneity. SST extremes were connected with blocking atmospheric patterns. A substantial warming between 1979 and 2015, in total exceeding 1 °C, was preceded by a period with a negative SST trend, implying strong multidecadal variability in the Adriatic. The strongest connection was found between yearly SST and the East Atlantic (EA) pattern, while North Atlantic Oscillation (NAO) and East Atlantic/West Russia (EAWR) patterns were found to also affect February SST values. Quantification of the Adriatic SST and their connection to hemispheric indices allow for more precise projections of future SST, considered to be rather important for Adriatic thermohaline circulation, biogeochemistry and fisheries, and sensitive to ongoing climate change.

Techno-Economic analysis of solar photovoltaic power plant for small scale fish processing in Kota Langsa - a case study

In Langsa, fisheries are the sector leaders by fulfilling a capacity of about 6,050 tons per year and on the other hand, fish-aquaculture reaches 1,200 tons per year on average. The fish processing is conducted through catches and aquaculture. The facilities on which this processing takes place are divided into an ice factory unit, a gutting and cutting unit, a drying unit and a curing unit. However, the energy and electricity costs during the production process has become major constraint because of the increase in the fishermen’s production and income. In this study, the potential and cost-effectiveness of photovoltaic solar power plant to meet the energy demands of fish processing units have been analysed. The energy requirements of fish processing units have reached an estimate of 130 kW, while the proposed design of solar photovoltaic electricity generation is of 200 kW in an area of 0,75 hectares. In this analysis, given the closeness between the location of the processing units and the fish supply auctions, the assumption is made that the photovoltaic plants (OTR) were installed on the roof of the building as compared to the solar power plants (OTL) installed on the outside of the location. The results shows that the levelized cost of OTR instalation is IDR 1.115 per kWh, considering 25 years of plant life-span at 10% of discount rate, with a simple payback period of 13.2 years. OTL levelized energy, on the other hand, is at IDR 997.5 per kWh with a simple payback period of 9.6 years. Blood is an essential component of living creatures in the vascular space. For possible disease identification, it can be tested through a blood test, one of which can be seen from the form of red blood cells. The normal and abnormal morphology of the red blood cells of a patient is very helpful to doctors in detecting a disease. With the advancement of digital image processing technology can be used to identify normal and abnormal blood cells of a patient. This research used self-organizing map method to classify the normal and abnormal form of red blood cells in the digital image. The use of self-organizing map neural network method can be implemented to classify the normal and abnormal form of red blood cells in the input image with 93,78% accuracy testing.

The morphological classification of normal and abnormal red blood cell using Self Organizing Map

Blood is an essential component of living creatures in the vascular space. For possible disease identification, it can be tested through a blood test, one of which can be seen from the form of red blood cells. The normal and abnormal morphology of the red blood cells of a patient is very helpful to doctors in detecting a disease. With the advancement of digital image processing technology can be used to identify normal and abnormal blood cells of a patient. This research used self-organizing map method to classify the normal and abnormal form of red blood cells in the digital image. The use of self-organizing map neural network method can be implemented to classify the normal and abnormal form of red blood cells in the input image with 93,78% accuracy testing.

Wind regimes and their relation to synoptic variables using self-organizing maps

Abstract. This study exemplifies the ability of the self-organizing maps (SOM) method to directly define well known wind regimes over Israel during the entire year, except summer period, at 12:00 UTC. This procedure may be applied at other hours and is highly relevant to future automatic climatological analysis and applications. The investigation is performed by analysing surface wind measurements from 53 Israel Meteorological Service stations. The relation between the synoptic variables and the wind regimes is revealed from the averages of ECMWF ERA-INTERIM reanalysis variables for each SOM wind regime. The inspection of wind regimes and their average geopotential anomalies has shown that wind regimes relate to the gradient of the pressure anomalies, rather than to the specific isobars pattern. Two main wind regimes – strong western and the strong eastern or northern – are well known over this region. The frequencies of the regimes according to seasons is verified. Strong eastern regimes are dominant during winter, while strong western regimes are frequent in all seasons.

Logistic regression analysis for the identification of the metastasis-associated signaling pathways of osteosarcoma

Osteosarcoma (OS) is the most common histological type of primary bone cancer. The present study was designed to identify the key genes and signaling pathways involved in the metastasis of OS. Microarray data of GSE39055 were downloaded from the Gene Expression Omnibus database, which included 19 OS biopsy specimens before metastasis (control group) and 18 OS biopsy specimens after metastasis (case group). After the differentially expressed genes (DEGs) were identified using the Linear Models for Microarray Analysis package, hierarchical clustering analysis and unsupervised clustering analysis were performed separately, using orange software and the self-organization map method. Based upon the Database for Annotation, Visualization and Integrated Discovery tool and Cytoscape software, enrichment analysis and protein-protein interaction (PPI) network analysis were conducted, respectively. After function deviation scores were calculated for the significantly enriched terms, hierarchical clustering analysis was performed using Cluster 3.0 software. Furthermore, logistic regression analysis was used to identify the terms that were significantly different. Those terms that were significantly different were validated using other independent datasets. There were 840 DEGs in the case group. There were various interactions in the PPI network [including intercellular adhesion molecule-1 (ICAM1), transforming growth factor β1 (TGFB1), TGFB1-platelet-derived growth factor subunit B (PDGFB) and PDGFB-platelet-derived growth factor receptor-β (PDGFRB)]. Regulation of cell migration, nucleotide excision repair, the Wnt signaling pathway and cell migration were identified as the terms that were significantly different. ICAM1, PDGFB, PDGFRB and TGFB1 were identified to be enriched in cell migration and regulation of cell migration. Nucleotide excision repair and the Wnt signaling pathway were the metastasis-associated pathways of OS. In addition, ICAM1, PDGFB, PDGFRB and TGFB1, which were involved in cell migration and regulation of cell migration may affect the metastasis of OS.

The analysis of methods effectiveness of automated non-destructive testing of products based on Data Mining methods

Purpose of the study: the analysis of the effectiveness of automated nondestructive testing methods within the objectives of data clustering on the use of short-wave electromagnetic radiation in flaw detection. Research methods: Kohonen self-organizing maps (SOM). The relevance of the work is that due to the increased demand for quality and reliability of products are becoming increasingly important physical methods for automated control of metals and products thereof that do not require cutting or fracture specimens of finished productes. The article noted common features of methods of short-wave electromagnetic control of products. The effectiveness of the Data Mining approach to the construction of a hypothesis on the interrelationships of data groups on non-destructive testing of products is substantiated. As an instrument, the method of self-organizing Kohonen maps was chosen. An example of a part of training data and neural network configuration parameters performing the task of visualization and clustering is given. It is concluded about the lead electromagnetic methods of automated control of complex products in production. The resulting distance matrix and the cluster map are shown. An example of applying the results of analysis to the problem of testing spot welded joints is considered. Given the further directions of research is to develop a computer image processing techniques in the framework of automated non-destructive testing systems.

Application of unsupervised pattern recognition approaches for exploration of rare earth elements in Se-Chahun iron ore, central Iran

Abstract. The use of efficient methods for data processing has always been of interest to researchers in the field of earth sciences. Pattern recognition techniques are appropriate methods for high-dimensional data such as geochemical data. Evaluation of the geochemical distribution of rare earth elements (REEs) requires the use of such methods. In particular, the multivariate nature of REE data makes them a good target for numerical analysis. The main subject of this paper is application of unsupervised pattern recognition approaches in evaluating geochemical distribution of REEs in the Kiruna type magnetite–apatite deposit of Se-Chahun. For this purpose, 42 bulk lithology samples were collected from the Se-Chahun iron ore deposit. In this study, 14 rare earth elements were measured with inductively coupled plasma mass spectrometry (ICP-MS). Pattern recognition makes it possible to evaluate the relations between the samples based on all these 14 features, simultaneously. In addition to providing easy solutions, discovery of the hidden information and relations of data samples is the advantage of these methods. Therefore, four clustering methods (unsupervised pattern recognition) – including a modified basic sequential algorithmic scheme (MBSAS), hierarchical (agglomerative) clustering, k-means clustering and self-organizing map (SOM) – were applied and results were evaluated using the silhouette criterion. Samples were clustered in four types. Finally, the results of this study were validated with geological facts and analysis results from, for example, scanning electron microscopy (SEM), X-ray diffraction (XRD), ICP-MS and optical mineralogy. The results of the k-means clustering and SOM methods have the best matches with reality, with experimental studies of samples and with field surveys. Since only the rare earth elements are used in this division, a good agreement of the results with lithology is considerable. It is concluded that the combination of the proposed methods and geological studies leads to finding some hidden information, and this approach has the best results compared to using only one of them.

Neural Network Self-Learning Model for Complex Assessment of Drinking Water Safety for Consumers

We need to take into complex assessment a set of influencing factors of drinking water safety. This raises the task of developing an integrated methodology assessing the safety of drinking water that reaches the consumers. For the integrated assessment of the safety of drinking water, the method of clustering was chosen, namely, the neural network method of Kohonen self-organizing maps. Zones were separated by the method of cluster neural network analysis. The zones are characterized by different content of metal cations in drinking water, levels of carcinogenic and non-carcinogenic risk to the health of the child population, and the probability of the receipt of metal cations with potable water to consumers.

Behavioral Biometrics for User Authentication Using Self-Refreshing SOM

Password protection has been a crucial issue in computer security. Traditional password protection using string does not provide sufficient security for user authentication. On the other hand, implementation of static biometrics in replace of password protection is costly as specific devices are needed. This research proposes the use of Self-Organizing Map (SOM) method for keystroke pattern recognition as an alternative to password protection. A self-refreshing SOM method was also implemented to increase the adaptability of SOM toward inconsistencies in typing pattern. Two main experiments were conducted to examine the ability of SOM for user recognition and the ability of self-refreshing SOM to deal with typing pattern inconsistencies. The first experiment shows the ability of SOM to authenticate users with very accurate classification. Users were successfully recognized and non-users were successfully rejected. Second experiment shows the ability of selfrefreshing SOM to deal with inconsistencies in typing pattern of a same user in which smoother transition of data trend was obtained.

StreamExplorer: A Multi-Stage System for Visually Exploring Events in Social Streams

Analyzing social streams is important for many applications, such as crisis management. However, the considerable diversity, increasing volume, and high dynamics of social streams of large events continue to be significant challenges that must be overcome to ensure effective exploration. We propose a novel framework by which to handle complex social streams on a budget PC. This framework features two components: 1) an online method to detect important time periods (i.e., subevents), and 2) a tailored GPU-assisted Self-Organizing Map (SOM) method, which clusters the tweets of subevents stably and efficiently. Based on the framework, we present StreamExplorer to facilitate the visual analysis, tracking, and comparison of a social stream at three levels. At a macroscopic level, StreamExplorer uses a new glyph-based timeline visualization, which presents a quick multi-faceted overview of the ebb and flow of a social stream. At a mesoscopic level, a map visualization is employed to visually summarize the social stream from either a topical or geographical aspect. At a microscopic level, users can employ interactive lenses to visually examine and explore the social stream from different perspectives. Two case studies and a task-based evaluation are used to demonstrate the effectiveness and usefulness of StreamExplorer.Analyzing social streams is important for many applications, such as crisis management. However, the considerable diversity, increasing volume, and high dynamics of social streams of large events continue to be significant challenges that must be overcome to ensure effective exploration. We propose a novel framework by which to handle complex social streams on a budget PC. This framework features two components: 1) an online method to detect important time periods (i.e., subevents), and 2) a tailored GPU-assisted Self-Organizing Map (SOM) method, which clusters the tweets of subevents stably and efficiently. Based on the framework, we present StreamExplorer to facilitate the visual analysis, tracking, and comparison of a social stream at three levels. At a macroscopic level, StreamExplorer uses a new glyph-based timeline visualization, which presents a quick multi-faceted overview of the ebb and flow of a social stream. At a mesoscopic level, a map visualization is employed to visually summarize the social stream from either a topical or geographical aspect. At a microscopic level, users can employ interactive lenses to visually examine and explore the social stream from different perspectives. Two case studies and a task-based evaluation are used to demonstrate the effectiveness and usefulness of StreamExplorer.

Winter Wind Regimes over Israel Using Self-Organizing Maps

AbstractThe aim of this study is to objectively define and automatically reconstruct surface wind regimes over Israel. Unlike other previous studies that subjectively examined case studies or applied the semiobjective synoptic classification (SOC), this study shows the ability of the method of self-organizing maps (SOM) to directly define well-known wind regimes at the synoptic hours (0000, 0600, 1200, and 1800 UTC) during the winter. This ability sets the groundwork for future automatic climatological analysis and applications. The investigation is performed by analyzing surface wind measurements from 53 Israel Meteorological Service stations. The relation between the synoptic variables and the wind regimes is revealed from the averages of ECMWF interim reanalysis (ERA-Interim) variables at each SOM wind regime. The inspection of wind regimes and their average pressure anomalies has shown that wind regimes relate to the gradient of the pressure anomalies rather than to the specific isobaric pattern. Two main wind regimes—strong western and strong eastern—are well known over this region. During daytime, SOM classification identifies these two regimes while SOC reveals only strong western regimes since SOC considers depth of the pressure gradients only in the case of low pressure centers. In accordance with previous studies, two main groups—winter low and high pressure centers and/or Red Sea troughs—are related to the strong westerly or easterly wind regimes with low diurnal variability and high daily persistence. Regimes under weak pressure gradients have higher diurnal variability, relatively lower steadiness, and weaker speed. Their daily persistence is not necessarily low.

Characterisation of Academic Journal Publications Using Text Mining Techniques

The ever-growing volume of published academic journals and the implicit knowledge that can be derived from them has not fully enhanced knowledge development but rather resulted into information and cognitive overload. However, publication data are textual, unstructured and anomalous. Analysing such high dimensional data manually is time consuming and this has limited the ability to make projections and trends derivable from the patterns hidden in various publications. This study was designed to develop and use intelligent text mining techniques to characterise academic journal publications. Journals Scoring Criteria by nineteen rankers from 2001 to 2013 of 50th edition of Journal Quality List (JQL) were used as criteria for selecting the highly rated journals. The text-miner software developed was used to crawl and download the abstracts of papers and their bibliometric information from the articles selected from these journal articles. The datasets were transformed into structured data and cleaned using filtering and stemming algorithms. Thereafter, the data were grouped into series of word features based on bag of words document representation. The highly rated journals were clustered using Self-Organising Maps (SOM) method with attribute weights in each cluster.

Slippage and immobilization detection for planetary exploration rovers via machine learning and proprioceptive sensing

AbstractThis paper presents a new methodology where machine learning is used for detecting various levels of slip in the context of planetary exploration robotic missions. This methodology aims at employing proprioceptive rover sensor signals. Consequently, no operational complexity is added to the rover's commanding and it is independent of lighting conditions. Two supervised learning methods (Support Vector Machines and Artificial Neural Networks) are compared to two unsupervised learning approaches (K‐means and Self‐Organizing Maps (SOM)). Physical experiments using a single‐wheel testbed equipped with an MSL spare wheel and a real planetary exploration rover validate the implemented methodology. Performance is evaluated in terms of well‐known metrics both considering single data points and subsets of consecutive data points (moving median filter). Computation time and storage requirements are also examined. One of the SOM‐based algorithms, semantic SOM method, demonstrates a proper balance between the benefits of supervised learning algorithms (high success rate, >96%) and the advantages of unsupervised learning methods (low storage requirements, 5 kb, and no need of manually‐labeled training data). This paper also addresses the most convenient placement of IMU sensors on the rover chassis such that slippage detection is maximized.

QSAR studies of the bioactivity of hepatitis C virus (HCV) NS3/4A protease inhibitors by multiple linear regression (MLR) and support vector machine (SVM)

In this study, quantitative structure-activity relationship (QSAR) models using various descriptor sets and training/test set selection methods were explored to predict the bioactivity of hepatitis C virus (HCV) NS3/4A protease inhibitors by using a multiple linear regression (MLR) and a support vector machine (SVM) method. 512 HCV NS3/4A protease inhibitors and their IC50 values which were determined by the same FRET assay were collected from the reported literature to build a dataset. All the inhibitors were represented with selected nine global and 12 2D property-weighted autocorrelation descriptors calculated from the program CORINA Symphony. The dataset was divided into a training set and a test set by a random and a Kohonen’s self-organizing map (SOM) method. The correlation coefficients (r2) of training sets and test sets were 0.75 and 0.72 for the best MLR model, 0.87 and 0.85 for the best SVM model, respectively. In addition, a series of sub-dataset models were also developed. The performances of all the best sub-dataset models were better than those of the whole dataset models. We believe that the combination of the best sub- and whole dataset SVM models can be used as reliable lead designing tools for new NS3/4A protease inhibitors scaffolds in a drug discovery pipeline.

Regional frequency analysis using Growing Neural Gas network

The delineation of hydrologically homogeneous regions is an important issue in regional hydrological frequency analysis. In the present study, an application of the Growing Neural Gas (GNG) network for hydrological data clustering is presented. The GNG is an incremental and unsupervised neural network, which is able to adapt its structure during the training procedure without using a prior knowledge of the size and shape of the network. In the GNG algorithm, the Minimum Description Length (MDL) measure as the cluster validity index is utilized for determining the optimal number of clusters (sub-regions). The capability of the proposed algorithm is illustrated by regionalizing drought severities for 40 synoptic weather stations in Iran. To fulfill this aim, first a clustering method is applied to form the sub-regions and then a heterogeneity measure is used to test the degree of heterogeneity of the delineated sub-regions. According to the MDL measure and considering two different indices namely CS and Davies–Bouldin (DB) in the GNG network, the entire study area is subdivided in two sub-regions located in the eastern and western sides of Iran. In order to evaluate the performance of the GNG algorithm, a number of other commonly used clustering methods, like K-means, fuzzy C-means, self-organizing map and Ward method are utilized in this study. The results of the heterogeneity measure based on the L-moments approach reveal that only the GNG algorithm successfully yields homogeneous sub-regions in comparison to the other methods.

Moisture sources and pathways associated with the spatial variability of seasonal extreme precipitation over Canada

Nine regions with spatially coherent seasonal 3-day total precipitation extremes across Canada were identified using a clustering method that is compliant to the extreme value theory. Using storm back-trajectory analyses, we then identified possible moisture sources and pathways that are conducive to occurrences of seasonal extreme precipitation events in four seasons for the nine regions identified. Moisture pathways for all extreme precipitation events were clustered to nine dominant moisture pathway patterns using the self-organizing map method. Results show that horizontal moisture pathway patterns and their occurrences were not evidently different between seasons. However, warm (summer and fall) and cold (winter and spring) seasons show considerable differences in the spreading of moisture sources in all nine regions, even though many sources do not frequently contribute to extreme precipitation events. In all four seasons, terrestrial evapotranspiration had provided major moisture sources to many extreme precipitation events occurred in inland regions. Central Canada had received more widespread moisture sources over surrounding oceans of North America than western and eastern Canada, because of more diverse moisture pathway patterns for central Canada that transport moisture from all surrounding oceans to central Canada. Extreme precipitation in southwestern Canada mainly resulted from atmospheric rivers over the North Pacific Ocean. For northwestern Canada, moisture pathway patterns were from the northern Pacific, Arctic and northern Atlantic oceans, even though more than 78% of trajectories for northwestern Canada were from the North Pacific. Westerlies from the North Pacific Ocean and northern polar jet streams controlled dominant pathways to central and eastern Canada. More extreme precipitation events over Canada were fed by the Arctic Ocean in warm than in cold seasons.

Probabilistic Forecasting of Surface Ozone with a Novel Statistical Approach

AbstractThe recent change in the Environmental Protection Agency’s surface ozone regulation, lowering the surface ozone daily maximum 8-h average (MDA8) exceedance threshold from 75 to 70 ppbv, poses significant challenges to U.S. air quality (AQ) forecasters responsible for ozone MDA8 forecasts. The forecasters, supplied by only a few AQ model products, end up relying heavily on self-developed tools. To help U.S. AQ forecasters, this study explores a surface ozone MDA8 forecasting tool that is based solely on statistical methods and standard meteorological variables from the numerical weather prediction (NWP) models. The model combines the self-organizing map (SOM), which is a clustering technique, with a stepwise weighted quadratic regression using meteorological variables as predictors for ozone MDA8. The SOM method identifies different weather regimes, to distinguish between various modes of ozone variability, and groups them according to similarity. In this way, when a regression is developed for a specific regime, data from the other regimes are also used, with weights that are based on their similarity to this specific regime. This approach, regression in SOM (REGiS), yields a distinct model for each regime taking into account both the training cases for that regime and other similar training cases. To produce probabilistic MDA8 ozone forecasts, REGiS weighs and combines all of the developed regression models on the basis of the weather patterns predicted by an NWP model. REGiS is evaluated over the San Joaquin Valley in California and the northeastern plains of Colorado. The results suggest that the model performs best when trained and adjusted separately for an individual AQ station and its corresponding meteorological site.

A Hybrid Climate Model for Rainfall Forecasting Based on Combination of Self-Organizing Map and Analog Method

A hybrid climate model (HCM) is a novel proposed model based on the combination of self-organizing map (SOM) and analog method (AM). The main purpose was to improve the accuracy in rainfall forecasting using HCM. In combination process of HCM, SOM algorithm classifies high dimensional input data to low dimensional of several disjointed clusters in which similar input is grouped. AM searches the future day that has similar property with the day in the past. Consequently, the analog day is mapped to each cluster of SOM to investigate rainfall. In this study, the input data, geopotential height at 850 hPa from the Climate Forecast System Reanalysis (CFSR) are training set data and also the complete rainfall data at 30-meteorological stations from Thai meteorological department (TMD) are observed. To improve capability of rainfall forecasting, three different measures were evaluated. The experimental results showed that the performance of HCM is better than the traditional AM. It is illustrated that the HCM can forecast rainfall proficiently.