Gap Statistic Research Articles

Abstract P3-07-10: Triple negative breast cancer tumors subtyping by means of integrated transcriptome and proteome analyses

Abstract Background: Heterogeneity and lack of targeted therapies represent the two main impediments to precision treatment of triple-negative breast cancer (TNBC) and therefore, molecular subtyping and identification of therapeutic pathways are required to optimize medical care. The aim of the present work was to confirm unsupervised analysis of TNBC transcriptomic data by means of proteomics. Methods: Transcriptome and proteome of 83 TNBC macrodissected tumors were screened in parallel. These patients were described in a previous study [Jézéquel, et al., Breast Cancer research (2015) 17, 43). Transcriptome screening was performed using Affymetrix Human Genome U133 Plus 2.0 Arrays (AffymetrixÒ, Santa Clara, CA). Proteome profiling was performed by means of iTRAQ-OFFGEL-LC-MS/MS approach [Campone, et al, Mol Cell Proteomics (2015) 14, 2936-2946]. Results: Unsupervised analysis of transcriptomic data identified three molecular clusters within TNBC: one molecular apocrine (C1: 20%) and two basal-like-enriched (C2: 47% and C3: 33%). C2 presented pro-tumorigenic immune response and C3 exhibited adaptive immune response. iTRAQ-OFFGEL-LC-MS/MS screening identified 366 out of 1,929 unique proteins, which were quantified in at least 70% of TNBC tumors and therefore could be used for analysis. Principal component analysis (PCA) with projection of 83 TNBC onto the first principal plane showed inhomogeneous distribution: one largest group (n = 77) and two outlier groups composed of three tumors, which have been eliminated for the rest of the work. In order to look for the existence of a partition of 77 TNBC cohort based on proteomics data, we performed clustering analysis using fuzzy clustering. Gap statistic was used to estimate the optimal number of clusters. This number was equal to one, whatever the metric. PCA and estimation of the number of clusters results lead us to conclude that iTRAQ-OFFGEL-LC-MS/MS data could not be used alone to subtype our 77 TNBC cohort most probably due to insufficient information content of proteomics matrix. TNBC cluster assignment, based on transcriptomics was applied to these tumors. Sixty-two out 366 ANOVA analyses were significant between the three clusters (p < 0.05). Twenty-two differentially expressed proteins between C1, C2 and C3 belonged to biological categories, which characterized these TNBC clusters. Gene Ontology enrichment analysis based on the set of proteins highly expressed in C2 compared to C1 and C3 (n = 21) displayed enrichment in genes coding for protein involved in extracellular matrix, wound response and RNA splicing. Table 1.Proteins found differentially expressed between TNBC cluster defined by means of transcriptomicsProteinsBiological categoriesClustersK2C7, K2C8, K1C18, K1C19LuminalC1FAS, UGDHAndrogen induced (molecular apocrine) LDHBBasal-likeC2PLMN, POSTN, FLNB, TENA, PLOD3, FSCN1, SERPH, FINCInvasion, extracellular matrix MOESBasal-likeC3STAT1, SYWC, AMPL, SAMH1Interferon pathway IGKC, IGHMImmunoglobulines Conclusion: Although iTRAQ-OFFGEL-LC-MS/MS screening did not contain enough information for cluster identification, 22 proteins, which were differentially expressed between the three clusters corroborate transcriptomic subtyping of TNBC. Citation Format: Campone M, Guette C, Lasla H, Gouraud W, Guérin-Charbonnel C, Jézéquel P. Triple negative breast cancer tumors subtyping by means of integrated transcriptome and proteome analyses [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P3-07-10.

Rethinking Equal Pay in Sweden in Light of the Transparency Approaches in the UK and Iceland

The question can be raised whether there is a need in Sweden to work with issues of equal pay. The gender wage gap (GWG) statistics in Sweden are fairly consistent. According to the OECD gender wag ...

Discriminating copper geochemical anomalies and assessment of their reliability using a combination of sequential Gaussian simulation and gap statistic methods in Hararan area, Kerman, Iran

In geochemical exploration, there are various techniques such as univariate and multivariate statistical methods available for recognition of anomalous areas. Univariate techniques are usually utilized to estimate the threshold value, which is the smallest quantity among the values representing the anomalous areas. In this work, a combination of the Sequential Gaussian Simulation (SGS) and Gap Statistics (GS) methods was utilized as a new technique to estimate the threshold and to visualize the anomalous regions in the Hararan area, which is located in SE Iran, and consists of copper mineralization that seems to be connected to a porphyry Cu-Mo system. Furthermore, the most important advantage of this method is the reliable assessment of the anomalous areas. In other words, the anomalous areas were discriminated in terms of their probability values. The regions with high probability values were reliable and appropriate to locate the drilling points for a detailed exploration. It not only decreases the risk, cost, and time of exploration but also increases the drilling point reliability and precision of reserve estimation after drilling. In this research work, the results of analysis of 607 lithogeochemical samples for the element Cu were used. The SGS method was performed on the transformed data and 50 realizations were obtained. In the next step, the back-transformed realizations were utilized to obtain an E-type map, which was the average of 50 realizations. Moreover, the results of the GS method showed that the Cu threshold value was 228 ppm in the area. Therefore, using the E-type map, areas with values greater than 228 ppm were introduced as the anomalous areas. Finally, the probability map of the exceeding threshold values was acquired, and the anomalous districts located in the southern part of the studied area were considered as more reliable regions for future detailed exploration and drilling.

Drought assessment in Vojvodina (Serbia) using k-means cluster analysis

Droughts are natural hazards that endanger the safety of population, their property and can create serious agricultural and ecological problems over the affected region. An analysis of the Standardized Precipitation Index (SPI) was performed by using the database of sixty years (1956–2016) of the monthly precipitation for the nine meteorological stations in Vojvodina region (Serbia). The SPI has been used for drought determination. The present study investigates the application of k-means clustering methods on the SPI at the 12-month timescale values to detect distinct drought clusters. For the purpose of determining the optimal number of clusters, the Gap Statistics was used. The results show that the total of four clusters (regions) can be identified in Vojvodina: three stations are listed in region 1 (Sombor, Palic, and Novi Sad), two stations in region 2 (Bac and Sremska Mitrovica), region 3 is a single-station region (Bela Crkva), while in region 4, three stations are grouped (Kikinda, Zrenjanin, and Vrsac). The Mann-Kendall test has shown that only in region 1 there is a trend in SPI values ranging from arid towards more humid conditions in the 1956–2016 period. In other regions no trend was observed in the data series. These results could contribute to water resources and agricultural planning and management in Vojvodina region and also confirm the usefulness of clustering methods for drought regionalization.

Optimized K-Means Clustering Model based on Gap Statistic

Big data has become famous to process, store and manage massive volumes of data. Clustering is an essential phase in big data analysis for many real-life application areas uses clustering methodology for result analysis. The data clustered sets have become a challenging issue in the field of big data analytics. Among all clustering algorithm, the K-means algorithm is the most widely used unsupervised clustering approach as seen from past. The K-means algorithm is the best adapted for deciding similarities between objects based on distance measures with small datasets. Existing clustering algorithms require scalable solutions to manage large datasets. However, for a particular domain-specific problem the initial selection of K is still a significant concern. In this paper, an optimized clustering approach presented which is calculated the optimal number of clusters (k) for specific domain problems. The proposed approach is an optimal solution based on the cluster performance measure analysis based on gab statistic. By observation, the experimental results prove that the proposed model can efficiently enhance the speed of the clustering process and accuracy by reducing the computational complexity of the standard k-means algorithm which achieves 76.3%.

Optimal Risk Pooling for Area-Yield Insurance Design: A Machine-Learning Approach

Area-yield insurance has gained much attention in acdademy and practice due to its capability of addressing information asymmetry and data limitation, as well as its low administrative cost. However, by its index insurance nature, area-yield insurance suffers the problem of basis risk. The objective of this paper is to reduce the basis risk of area-yield crop insurance by finding the optimal risk pooling using machine learning algorithms. Two clustering algorithms, including the K-means algorithm and the Gaussian Mixture Model (GMM) optimized with Expectation-Maximization (EM) algorithm, are investigated and compared in searching alternative grouping methods. Five model selection algorithms are utilized to determine the optimized number of risk pools by taking the weather, location, and historical yield information into consideration. The five model selection algorithms are Elbow Point of Within Group Sum of Squares (WSS), validity measurement, Average Silhouette Index, Gap Statistic, and Weighted Stochastic Block Model (WSBM). Finally, the proposed optimal risk pooling algorithms are empirically tested and cross-compared with the U.S. corn production and weather data from 12 States in the Midwest region.

Using the geochemical behavior of silver element in the gap statistics method and multivariate analysis in Cu-Mo porphyry in Hararan area, Kerman province

Using the geochemical behavior of silver element in the gap statistics method and multivariate analysis in Cu-Mo porphyry in Hararan area, Kerman province

IntraClusTSP—An Incremental Intra-Cluster Refinement Heuristic Algorithm for Symmetric Travelling Salesman Problem

The Symmetric Traveling Salesman Problem (sTSP) is an intensively studied NP-hard problem. It has many important real-life applications such as logistics, planning, manufacturing of microchips and DNA sequencing. In this paper we propose a cluster level incremental tour construction method called Intra-cluster Refinement Heuristic (IntraClusTSP). The proposed method can be used both to extend the tour with a new node and to improve the existing tour. The refinement step generates a local optimal tour for a cluster of neighbouring nodes and this local optimal tour is then merged into the global optimal tour. Based on the performed evaluation tests the proposed IntraClusTSP method provides an efficient incremental tour generation and it can improve the tour efficiency for every tested state-of-the-art methods including the most efficient Chained Lin-Kernighan refinement algorithm. As an application example, we apply IntraClusTSP to automatically determine the optimal number of clusters in a cluster analysis problem. The standard methods like Silhouette index, Elbow method or Gap statistic method, to estimate the number of clusters support only partitional (single level) clustering, while in many application areas, the hierarchical (multi-level) clustering provides a better clustering model. Our proposed method can discover hierarchical clustering structure and provides an outstanding performance both in accuracy and execution time.

Filter-based unsupervised feature selection using Hilbert–Schmidt independence criterion

Feature selection is a fundamental preprocess before performing actual learning; especially in unsupervised manner where the data are unlabeled. Essentially, when there are too many features in the problem, dimensionality reduction through discarding weak features is highly desirable. In this paper, we present a framework for unsupervised feature selection based on dependency maximization between the samples similarity matrices before and after deleting a feature. In this regard, a novel estimation of Hilbert–Schmidt independence criterion (HSIC), more appropriate for high-dimensional data with small sample size, is introduced. Its key idea is that by eliminating the redundant features and/or those have high inter-relevancy, the pairwise samples similarity is not affected seriously. Also, to handle the diagonally dominant matrices, a heuristic trick is used in order to reduce the dynamic range of matrix values. In order to speed up the proposed scheme, the gap statistic and k-means clustering methods are also employed. To assess the performance of our method, some experiments on benchmark datasets are conducted. The obtained results confirm the efficiency of our unsupervised feature selection scheme.

Automatic Independent Component Scalp Map Analysis of Electroencephalogram During Motor Preparation.

This work presents a method for automatic independent component (IC) scalp map analysis of electroencephalogram during motor preparation in visuomotor tasks. The strength of this approach is the analysis of the IC scalp maps based on the apriori given mask. This uses an image processing approach, comparable to visual classification used by experts, to automate the selection of relevant ICs in visuomotor tasks. Thirty iterations of the Infomax ICA algorithm were used to test the reliability of the ICs. ICs above 95% quality index were used for IC scalp topography image analysis. Here, we used a linkage-clustering algorithm for IC clustering and gap statistic to estimate the number of clusters. After classifying the components with our approach, the labels were compared to those from well-known MARA ("Multiple Artifact Rejection Algorithm") - an open-source EEGLAB plug-in. It was found that 334 of the 568 labels were in-agreement. MARA labeled 81 out of the 177 source-related components, and 238 out of the 319 non-source-related components, as artifacts. Here, the strength of our approach lies in using an image-processing algorithm to identify the task-specific ICs whereas MARA focuses on the automatic classification of the artifactual ICs by combining stereotyped artifact-specific spatial and temporal features that depend on the electrode montage. After "artefactual" ICs are removed, task-specific ICs still remains to be identified from the remaining "good" ICs where our scalp topography image analysis approach can be applied. Our IC scalp topography image analysis is focused on task-specific IC selection based on an apriori mask, which is not limited to specific EEG features and/or electrode configurations for high-density EEG.

Cumulative lifetime maternal stress and epigenome-wide placental DNA methylation in the PRISM cohort

ABSTRACTEvolving evidence links maternal stress exposure to changes in placental DNA methylation of specific genes regulating placental function that may have implications for the programming of a host of chronic disorders. Few studies have implemented an epigenome-wide approach. Using the Infinium HumanMethylation450 BeadChip (450K), we investigated epigenome-wide placental DNA methylation in relation to maternal experiences of traumatic and non-traumatic stressors over her lifetime assessed using the Life Stressor Checklist-Revised (LSC-R) survey (n = 207). We found differential DNA methylation at epigenome-wide statistical significance (FDR = 0.05) for 112 CpGs. Additionally, we observed three clusters that exhibited differential methylation in response to high maternal lifetime stress. Enrichment analyses, conducted at an FDR = 0.20, revealed lysine degradation to be the most significant pathway associated with maternal lifetimes stress exposure. Targeted enrichment analyses of the three largest clusters of probes, identified using the gap statistic, were enriched for genes associated with endocytosis (i.e., SMAP1, ANKFY1), tight junctions (i.e., EPB41L4B), and metabolic pathways (i.e., INPP5E, EEF1B2). These pathways, also identified in the top 10 KEGG pathways associated with maternal lifetime stress exposure, play important roles in multiple physiological functions necessary for proper fetal development. Further, two genes were identified to exhibit multiple probes associated with maternal lifetime stress (i.e., ANKFY1, TM6SF1). The methylation status of the probes belonging to each cluster and/or genes exhibiting multiple hits, may play a role in the pathogenesis of adverse health outcomes in children born to mothers with increased lifetime stress exposure.

Analisis Cluster dengan Data Outlier Menggunakan Centroid Linkage dan K-Means Clustering untuk Pengelompokkan Indikator HIV/AIDS di Indonesia

Cluster analysis is a method to group data (objects) or observations based on their similarities. Objects that become members of a group have similarities among them. Cluster analyses used in this research are K-means clustering and Centroid Linkage clustering. K-means clustering, which falls under non-hierarchical cluster analysis, is a simple and easy to implement method. On the other hand, Centroid Linkage clustering, which belongs to hierarchical cluster analysis, is useful in handling outliers by preventing them skewing the cluster analysis. To keep it simple, outliers are often removed even though outliers often contain important information. HIV/AIDS is a serious challenge for global public health since HIV/AIDS is an infectious disease attacking body’s immune system that in turn lowering the ability to fight infections which in the end causing death. HIV/AIDS indicators data in Indonesia contain outliers. This research uses gap statistic to define the number of clusters based on HIV/AIDS indicators that groups Indonesia provinces into 7 clusters. By comparing S­w­/S­b ratio, Centroid Linkage clustering is more homogenous than K-means clustering. Using clustering, the government shall be able to create a better policy for fighting HIV/AIDS based on the dominant indicators in each cluster.

PurTreeClust: A Clustering Algorithm for Customer Segmentation from Massive Customer Transaction Data

Clustering of customer transaction data is an important procedure to analyze customer behaviors in retail and e-commerce companies. Note that products from companies are often organized as a product tree, in which the leaf nodes are goods to sell, and the internal nodes (except root node) could be multiple product categories. Based on this tree, we propose the “personalized product tree”, named purchase tree, to represent a customer’s transaction records. So the customers’ transaction data set can be compressed into a set of purchase trees. We propose a partitional clustering algorithm, named PurTreeClust, for fast clustering of purchase trees. A new distance metric is proposed to effectively compute the distance between two purchase trees. To cluster the purchase tree data, we first rank the purchase trees as candidate representative trees with a novel separate density, and then select the top $k$ customers as the representatives of $k$ customer groups. Finally, the clustering results are obtained by assigning each customer to the nearest representative. We also propose a gap statistic based method to evaluate the number of clusters. A series of experiments were conducted on ten real-life transaction data sets, and experimental results show the superior performance of the proposed method.

Clustering by reordering of similarity and Laplacian matrices: Application to galaxy clusters

Abstract Similarity metrics, kernels and similarity-based algorithms have gained much attention due to their increasing applications in information retrieval, data mining, pattern recognition and machine learning. Similarity Graphs are often adopted as the underlying representation of similarity matrices and are at the origin of known clustering algorithms such as spectral clustering. Similarity matrices offer the advantage of working in object–object (two-dimensional) space where visualization of clusters similarities is available instead of object-features (multi-dimensional) space. In this paper, sparse ϵ -similarity graphs are constructed and decomposed into strong components using appropriate methods such as Dulmage–Mendelsohn permutation (DMperm) and/or Reverse Cuthill–McKee (RCM) algorithms. The obtained strong components correspond to groups (clusters) in the input (feature) space. Parameter ϵ i is estimated locally, at each data point i from a corresponding narrow range of the number of nearest neighbors. Although more advanced clustering techniques are available, our method has the advantages of simplicity, better complexity and direct visualization of the clusters similarities in a two-dimensional space. Also, no prior information about the number of clusters is needed. We conducted our experiments on two and three dimensional, low and high-sized synthetic datasets as well as on an astronomical real-dataset. The results are verified graphically and analyzed using gap statistics over a range of neighbors to verify the robustness of the algorithm and the stability of the results. Combining the proposed algorithm with gap statistics provides a promising tool for solving clustering problems. An astronomical application is conducted for confirming the existence of 45 galaxy clusters around the X-ray positions of galaxy clusters in the redshift range [0.1..0.8]. We re-estimate the photometric redshifts of the identified galaxy clusters and obtain acceptable values compared to published spectroscopic redshifts with a 0.029 standard deviation of their differences.

A METRIC THEORY OF MINIMAL GAPS

We study the minimal gap statistic for fractional parts of sequences of the form $\mathcal A^\alpha = \{\alpha a(n)\}$ where $\mathcal A = \{a(n)\}$ is a sequence of distinct of integers. Assuming that the additive energy of the sequence is close to its minimal possible value, we show that for almost all $\alpha$, the minimal gap $\delta_{\min}^\alpha(N)=\min\{\alpha a(m)-\alpha a(n)\bmod 1: 1\leq m\neq n\leq N\}$ is close to that of a random sequence.

A divisive clustering method for functional data with special consideration of outliers

Fil: Justel, Ana. Universidad Autonoma de Madrid; Espana. Universidad Carlos III de Madrid; Espana

Applying the combination of U-statistic and Mahalanobis distance as a multivariate structural method for the delineation of geochemical anomalies

ABSTRACTThere are several statistical methods for separating anomalous values from background leading to determination of anomalous areas. These methods range from simple approaches to complicated ones and include nonstructural and structural methods. Structural methods take the sampling locations and their spatial relation into account for estimating the anomalous areas. The U-statistic method is one of the most important structural methods. It considers the location of samples and carries out the statistical analysis of the data without judging from a geochemical point of view and tries to separate subpopulations and also to determine anomalous areas. In the present study, two nonstructural methods including gap statistic and Mahalanobis distance method (as a multivariate method) are used and U-statistic is considered as a structural method to assess prospective areas of Parkam district. Results show that the samples indicated by U-statistic and gap statistic methods as anomalous are more regular and less dispersed compared to those indicated by other nonstructural methods. Both gap statistic and U-statistic methods have shown almost similar results for estimating threshold and separation, but U-statistic method is even better than gap statistic because the U-statistic method devotes a new value to each sample which can be combined with other methods. For this reason, the combination of U-statistic and Mahalanobis distance method has been employed to separate geochemical anomaly from background. Combination results show that the performance of these two methods is more accurate than using just one of them. Finally, bivariate lithogeochemical map of the study area is provided for Cu and Mo which has been prepared using combination of the U-statistic and Mahalanobis distance method. In this map, the delineated Cu-Mo mineralization is closely associated with the defined zone of potassic alteration, which is also consistent with the field and microscopic observation of the Cu mineralization in this alteration zone. Moreover, it is associated with the phyllic alteration and is spatially conformable with the zone defined for it.

A finer view of the conditional galaxy luminosity function and magnitude-gap statistics

The gap between first and second ranked galaxy magnitudes in groups is often considered a tracer of their merger histories, which in turn may affect galaxy properties, and also serves to test galaxy luminosity functions (LFs). We remeasure the conditional luminosity function (CLF) of the Main Galaxy Sample of the SDSS in an appropriately cleaned subsample of groups from the Yang catalog. We find that, at low group masses, our best-fit CLF have steeper satellite high ends, yet higher ratios of characteristic satellite to central luminosities in comparison with the CLF of Yang et al. (2008). The observed fractions of groups with large and small magnitude gaps as well as the Tremaine & Richstone (1977) statistics, are not compatible with either a single Schechter LF or with a Schechter-like satellite plus lognormal central LF. These gap statistics, which naturally depend on the size of the subsamples, and also on the maximum projected radius, $R_{\rm max}$, for defining the 2nd brightest galaxy, can only be reproduced with two-component CLFs if we allow small gap groups to preferentially have two central galaxies, as expected when groups merge. Finally, we find that the trend of higher gap for higher group velocity dispersion, $\sigma_{\rm v}$, at given richness, discovered by Hearin et al. (2013), is strongly reduced when we consider $\sigma_{\rm v}$ in bins of richness, and virtually disappears when we use group mass instead of $\sigma_{\rm v}$. This limits the applicability of gaps in refining cosmographic studies based on cluster counts.

Abstract 549: Clustering analysis of next-generation sequencing T cell repertoire data in sipuleucel-T treated prostate cancer patients

Abstract Immunotherapy has demonstrated significant clinical benefit in different cancers. T cells are a crucial component of the adaptive immune system and mediate anti-tumoral immunity. Antigen-specific recognition by T cells is via T cell receptor (TCR) which is the product of somatic V(D)J gene recombination with the addition/subtraction of nontemplated bases at recombination junctions. Next generation TCR sequencing effectively profiles the TCR repertoire. Currently TCR analyses quantify diversity across single clones, however, due to the low overlap of clones across samples, such analyses are limited to a single sample. Here we extend our previous analysis pipeline to track and examine TCR repertoire across time by focusing on V and J gene segments, overcoming the limitation of contemporary analytical approaches and thus obtain statistical inference across subjects directly. The published data of TCR profiling of serial peripheral blood mononuclear cells (PBMC) from three healthy subjects and five prostate cancer patients enrolled in an open-label, Phase II clinical trial of neoadjuvant sipuleucel-T (sip-T) were used as the basis of this analyses. After consolidating the clonal abundance for each combination of V and J gene segments, we calculated Pielou’s evenness, which demonstrates that sip-T treated subjects displayed greater clonal expansion compared to healthy subjects (P<0.001). The Circos table viewer was used to circularly visualize the distribution of the combination of V and J gene segments. Sip-T treated subjects were successfully distinguished from healthy subjects with 12 V and J gene combinations explaining the majority of variance of the patients by applying principle components analysis (PCA). Furthermore, we developed a customized clustering workflow to cluster the combination of V and J gene segments based on the change in abundance over time, with gap statistics employed to estimate the optimal number of clusters and k-means algorithm used for partitioning. We found that for all the prostate cancer patients assessed, TCRBV06/TCRBJ01, TCRBV05/TCRBJ02 and TCRBV06/TCRBJ02, which were in the same cluster with the highest frequencies, expanded after first treatment and were maintained at high frequencies at later time points. The results demonstrate that in the setting of immune modulation after sip-T treatment, the V(D)J recombination is not entirely random, and that this non-randomness can be used to distinguish T cell repertoires from sip-T treated subjects versus healthy subjects. The use of additional information of V and J gene segments enables to investigate the profiling of TCR repertoire from a different angle and add another layer of understanding of TCR repertoire. The application of PCA and the customized clustering complete our initial workflow for TCR sequencing data. Citation Format: Li Zhang, Sounak Chakraborty, Jason Cham, David Oh, Nadeem Sheikh, Lawrence Fong. Clustering analysis of next-generation sequencing T cell repertoire data in sipuleucel-T treated prostate cancer patients [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 549. doi:10.1158/1538-7445.AM2017-549

Design of electricity tariff plans using gap statistic for K-means clustering based on consumers monthly electricity consumption data

PurposeElectricity consumption around the world and in India is continuously increasing over the years. Presently, there is a huge diversity in electricity tariffs across states in India. This paper aims to focus on development of new tariff design method using K-means clustering and gap statistic.Design/methodology/approachNumbers of tariff plans are selected using gap-statistic for K-means clustering and regression analysis is used to deduce new tariffs from existing tariffs. The study has been carried on nearly 27,000 residential consumers from Sangli city, Maharashtra State, India.FindingsThese tariff plans are proposed with two objectives: first, possibility to shift consumer’s from existing to lower tariff plan for saving electricity and, second, to increase revenue by increasing tariff charges using Pay-by-Use policy.Research limitations/implicationsThe study can be performed on hourly or daily data using automatic meter reading and to introduce Time of Use or demand based tariff.Practical implicationsThe proposed study focuses on use of data mining techniques for tariff planning based on consumer’s electricity usage pattern. It will be helpful to detect abnormalities in consumption pattern as well as forecasting electricity usage.Social implicationsConsumers will be able to decide own monthly electricity consumption and related tariff leading to electricity savings, as well as high electricity consumption consumers have to pay more tariff charges for extra electricity usage.Originality/valueTo remove the disparity in various tariff plans across states and country, proposed method will help to provide a platform for designing uniform tariff for entire country based on consumer’s electricity consumption data.