Email Clustering Research Articles

Email thread sentiment sequence identification using PLSA clustering algorithm

Email messaging is the most common way of providing effective communication between internauts. Consequently, the total sent and received emails count will be increased. But, the internaut can't remember all such emails. Even though email thread identification approaches give satisfactory benefits to the internauts, but they may fail to alert them for a cause to identify the sentiments behind an email thread. To address, this issue Probabilistic Latent Semantic Analysis clustering algorithm has been used in this paper to identify the email sentiment thread sequence. The sentiment and the thread sequence within the emails have been discovered as clustering sentiment polarity and temporal categories with the help of PLSA clusters. At the initial stage, we used three feature extraction methods, latent semantic analysis (LDA), bag of words (BoW), TF-IDF and SentiWordNet (SWN) lexicon for generating sentiment features of email. Next, Probabilistic Latent Semantic Analysis algorithm is used to form email clusters based on sentiment features. Thus, it helps to identify thread sentiment and sequence of sentiment threads. Email threads give a mechanism by which any user will be able to find out the sequence in the thread on the basis of sentiment analysis of email related to a specific set of communication during a specific time period. Various parameters evaluation measures have been considered in this work to evaluate the proposed model such as accuracy, precision, recall and F-measure, and the proposed algorithm is compared with other standard algorithms. Furthermore, a statistical test has also been performed.

An Unsupervised Approach for Content-Based Clustering of Emails Into Spam and Ham Through Multiangular Feature Formulation

The rapid growth of spam email attacks and the inherent malicious dynamism within those attacks on a range of social, personal and business activities warrants an intelligent and automated anti-spam framework. Attempts like malware propagation, identity theft, sensitive data pilfering, monetary as well as reputational damage are sharply increasing, endangering the privacy of the victim. Current solutions that are rather incomplete when the multidimensional feature range of email, is taken into account. We believe a methodology based on Artificial Intelligence, especially unsupervised machine learning is the way forward. This research attempts to investigating the application of unsupervised learning for the clustering of Spam and Ham emails. The overall goal of the research is to develop an unsupervised framework that solely depends on unsupervised methodologies through a clustering approach that includes multiple algorithms, primarily using the email content (body) and the subject header. The clustering has been done on a novel binary dataset of 22,000 entries of ham and spam emails, composed of ten features (reduced from eleven to ten after the feature reduction). Seven out of these ten features are unique to this study, engineered to represent impactful analytical email characteristics from a multiangular point of view. Out of five different clustering algorithms investigated in this work, OPTICS produced the optimum clustering demonstrating a 0.26% higher average efficacy than its nearest performer DBSCAN. The average balanced accuracy for OPTICS and DBSCAN was found to be ≈75.76%.

A Novel Approach to Control Emails Notification using NLP

The Digital era made Emails the omnipresent medium of information exchange and communication. One has to deal with a huge number of emails per day on a professional as well as personal level. This increases the chance of overlooking important and relevant emails. In this paper, we propose a novel approach to control Emails Notifications using NLP. Its objective is to notify only the top “n” number of prioritized emails by using Natural Language Processing Technique. The entire work has been carried in three phases: “the Spam Filtering and Clustering of emails”, “Similarity detection of new mail and allotment of emails to Clusters for notification” and “Cluster wise Ranking and Prediction of which email to notify”. Also, a simple “Similarity Detection algorithm” is proposed using the important words from the subject of the email.

Efficient Clustering of Emails Into Spam and Ham: The Foundational Study of a Comprehensive Unsupervised Framework

The spread and adoption of spam emails in malicious activities like information and identity theft, malware propagation, monetary and reputational damage etc. are on the rise with increased effectiveness and diversification. Without doubt these criminal acts endanger the privacy of many users and businesses’. Several research initiatives have taken place to address the issue with no complete solution until now; and we believe an intelligent and automated methodology should be the way forward to tackle the challenges. However, till date limited studies have been conducted on the applications of purely unsupervised frameworks and algorithms in tackling the problem. To explore and investigate the possibilities, we intend to propose an anti-spam framework that fully relies on unsupervised methodologies through a multi-algorithm clustering approach. This article presents an in-depth analysis on the methodologies of the first component of the framework, examining only the domain and header related information found in email headers. A novel method of feature reduction using an ensemble of ‘unsupervised’ feature selection algorithms has also been investigated in this study. In addition, a comprehensive novel dataset of 100,000 records of ham and spam emails has been developed and used as the data source. Key findings are summarized as follows: I) out of six different clustering algorithms used – Spectral and K-means demonstrated acceptable performance while OPTICS projected the optimum clustering with an average of 3.5% better efficiency than Spectral and K-means, validated through a range of validations processes II) The other three algorithms- BIRCH, HDBSCAN and K-modes, did not fare well enough. III) The average balanced accuracy for the optimum three algorithms has been found to be ≈94.91%, and IV) The proposed feature reduction framework achieved its goal with high confidence.

Learning from the Field and Its Listserv: Issues That Concern Health Literacy Practitioners.

This study assesses the content of email messages posted to the Health Literacy Discussion List (HLDL) during a two-year period. The study identifies issues of concern to list subscribers, describes the purposes the list serves for health professionals, and contributes to the health literacy literature by providing an email listserv as a research corpus. The authors conducted an inductive qualitative analysis of email posts to the HLDL from October 2013 to October 2015. Using an iterative process, the authors identified descriptive categories for types of posts and topics of posts. The first (SKR) and second (JM) authors reviewed subject lines of all 2,036 posts and brainstormed type and topic categories, independently read and sorted a random sample of 200 posts into those categories, and then discussed discrepancies. Based on the latter experience, the authors combined, added, or excluded certain categories and jointly created a detailed description for each type and topic category. We then sorted another random sample of 200 posts and generated a list of key words relating emails to topic categories. A Cohen's kappa reliability coefficient was calculated to establish intercoder reliability. The third author (RVR) then conducted key word searches for sorting the remaining 1,836 email posts. The existence and frequency of email clusters and the content of emails in these clusters were used to identify and explore in greater detail the "hot topics" of interest to the field. Our analysis suggests the utility of the HLDL as a platform for sharing information and resources, announcements and calls for action, technical assistance and professional discourse.

Learning from the field and its listserv: Issues that concern health literacy practitioners

This study assesses the content of email messages posted to the Health Literacy Discussion List (HLDL) during a two-year period. The study identifies issues of concern to list subscribers, describes the purposes the list serves for health professionals, and contributes to the health literacy literature by providing an email listserv as a research corpus. The authors conducted an inductive qualitative analysis of email posts to the HLDL from October 2013 to October 2015. Using an iterative process, the authors identified descriptive categories for types of posts and of posts. The first (SKR) and second (JM) authors reviewed subject lines of all 2,036 posts and brainstormed type and topic categories, independently read and sorted a random sample of 200 posts into those categories, and then discussed discrepancies. Based on the latter experience, the authors combined, added, or excluded certain categories and jointly created a detailed description for each type and topic category. We then sorted another random sample of 200 posts and generated a list of key words relating emails to topic categories. A Cohen's kappa reliability coefficient was calculated to establish intercoder reliability. The third author (RVR) then conducted key word searches for sorting the remaining 1,836 email posts. The existence and frequency of email clusters and the content of emails in these clusters were used to identify and explore in greater detail the hot topics of interest to the field. Our analysis suggests the utility of the HLDL as a platform for sharing information and resources, announcements and calls for action, technical assistance and professional discourse.

Email thread identification using latent Dirichlet allocation and non-negative matrix factorization based clustering techniques

Emails are the most popular and effective way of communicating over the internet. A number of applications are available today for computers and mobile devices for email messaging. Email messaging is constantly getting more popular and, as a result, numbers of sent and received emails are also increasing. It is very difficult for a user to remember emails and relate newer incoming emails to previous communications made on similar topics. Email threads provide a mechanism using which a user can obtain sequences of emails for a particular set of communication in a time frame and provides a number of benefits to users. In this work two email thread identification algorithms based on a nested textual clustering approach are presented. The work is planned in two stages; in the first stage two popular text clustering approaches, latent Dirichlet allocation and non-negative matrix factorization, are applied over the email messages to form the email clusters. Then in the second stage, clustering is again performed over the created email clusters to identify the email threads using threading features. Performance parameters like accuracy, precision, recall and F-measure are evaluated for the presented thread identification algorithms.

O-means: An Optimized Clustering Method for Analyzing Spam Based Attacks

In recent years, the number of spam emails has been dramatically increasing and spam is recognized as a serious internet threat. Most recent spam emails are being sent by bots which often operate with others in the form of a botnet, and skillful spammers try to conceal their activities from spam analyzers and spam detection technology. In addition, most spam messages contain URLs that lure spam receivers to malicious Web servers for the purpose of carrying out various cyber attacks such as malware infection, phishing attacks, etc. In order to cope with spam based attacks, there have been many efforts made towards the clustering of spam emails based on similarities between them. The spam clusters obtained from the clustering of spam emails can be used to identify the infrastructure of spam sending systems and malicious Web servers, and how they are grouped and correlate with each other, and to minimize the time needed for analyzing Web pages. Therefore, it is very important to improve the accuracy of the spam clustering as much as possible so as to analyze spam based attacks more accurately. In this paper, we present an optimized spam clustering method, called O-means, based on the K-means clustering method, which is one of the most widely used clustering methods. By examining three weeks of spam gathered in our SMTP server, we observed that the accuracy of the O-means clustering method is about 87% which is superior to the previous clustering methods. In addition, we define 12 statistical features to compare similarity between spam emails, and we determined a set of optimized features which makes the O-means clustering method more effective.

Managing email overload with an automatic nonparametric clustering system

Email overload is a recent problem that there is increasingly difficulty that people have to process the large number of emails received daily. Currently, this problem becomes more and more serious and it has already affected the normal usage of email as a knowledge management tool. It has been recognized that categorizing emails into meaningful groups can greatly save cognitive load to process emails, and thus this is an effective way to manage the email overload problem. However, most current approaches still require significant human input for categorizing emails. In this paper, we develop an automatic email clustering system, underpinned by a new nonparametric text clustering algorithm. This system does not require any predefined input parameters and can automatically generate meaningful email clusters. The evaluation shows our new algorithm outperforms existing text clustering algorithms with higher efficiency and quality in terms of computational time and clustering quality measured by different gauges. The experimental results also well match the labeled human clustering results.

Name Discrimination and E-mail Clustering Using Unsupervised Clustering of Similar Contexts

Article Name Discrimination and E-mail Clustering Using Unsupervised Clustering of Similar Contexts was published on August 1, 2008 in the journal Journal of Intelligent Systems (volume 17, issue 1-3).