Enron Email Corpus Research Articles

A comparative study on feature selection and adaptive strategies for email foldering using the ABC-DynF framework

Email foldering is a challenging problem mainly due to its high dimensionality and dynamic nature. This work presents ABC-DynF, an adaptive learning framework with dynamic feature space that we use to compare several incremental and adaptive strategies to cope with these two difficulties. Several studies have been carried out using datasets from the ENRON email corpus and different configuration settings of the framework. The main aim is to study how feature ranking methods, concept drift monitoring, adaptive strategies and the implementation of a dynamic feature space can affect the performance of Bayesian email classification systems.

From $K$-Means to Higher-Way Co-Clustering: Multilinear Decomposition With Sparse Latent Factors

Co-clustering is a generalization of unsupervised clustering that has recently drawn renewed attention, driven by emerging data mining applications in diverse areas. Whereas clustering groups entire columns of a data matrix, co-clustering groups columns over select rows only, i.e., it simultaneously groups rows and columns. The concept generalizes to data “boxes” and higher-way tensors, for simultaneous grouping along multiple modes. Various co-clustering formulations have been proposed, but no workhorse analogous to <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means has emerged. This paper starts from <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means and shows how co-clustering can be formulated as a constrained multilinear decomposition with sparse latent factors. For three- and higher-way data, uniqueness of the multilinear decomposition implies that, unlike matrix co-clustering, it is possible to unravel a large number of possibly overlapping co-clusters. A basic multi-way co-clustering algorithm is proposed that exploits multilinearity using Lasso-type coordinate updates. Various line search schemes are then introduced to speed up convergence, and suitable modifications are proposed to deal with missing values. The imposition of latent sparsity pays a collateral dividend: it turns out that sequentially extracting one co-cluster at a time is almost optimal, hence the approach scales well for large datasets. The resulting algorithms are benchmarked against the state-of-art in pertinent simulations, and applied to measured data, including the ENRON e-mail corpus.

Dyad and Triad Census Analysis of Crisis Communication Network

Dyad and triad census summarize much of the network level structural information of a given directed network. They have been found very useful in analyzing structural properties of social networks. This study aims to explore crisis communication network by following dyad and triad census analysis approach to investigate the association of microlevel communication patterns with organizational crisis. This study further tests hypothesis related to the process of data generation and tendency of the structural pattern of transitivity using dyad and triad census output. The changing communication network at Enron Corporation during the period of its crisis is analyzed in this study. Significant differences in the presence of different isomorphism classes or microlevel patterns of both dyad and triad census are noticed in crisis and non-crisis period network of Enron email corpus. It is also noticed that crisis communication network shows more transitivity compared to the non-crisis communication network.

From once upon a time to happily ever after: Tracking emotions in mail and books

In this paper, we show how sentiment analysis can be used in tandem with effective visualizations to quantify and track emotions in mail and books. We study a number of specific datasets and show, among other things, how collections of texts can be organized for affect-based search and how books portray different entities through co-occurring emotion words. Analysis of the Enron Email Corpus reveals that there are marked differences across genders in how they use emotion words in work-place email. Finally, we show that fairy tales have more extreme emotion densities than novels.

A Framework for the Forensic Investigation of Unstructured Email Relationship Data

The continued reliance on email communications ensures that it remains a major source of evidence during a digital investigation. Emails comprise both structured and unstructured data. Structured data provides qualitative information to the forensics examiner and is typically viewed through existing tools. Unstructured data is more complex as it comprises information associated with social networks, such as relationships within the network, identification of key actors and power relations, and there are currently no standardised tools for its forensic analysis. This paper posits a framework for the forensic investigation of email data. In particular, it focuses on the triage and analysis of unstructured data to identify key actors and relationships within an email network. This paper demonstrates the applicability of the approach by applying relevant stages of the framework to the Enron email corpus. The paper illustrates the advantage of triaging this data to identify (and discount) actors and potential sources of further evidence. It then applies social network analysis techniques to key actors within the data set. This paper posits that visualisation of unstructured data can greatly aid the examiner in their analysis of evidence discovered during an investigation.

Statistical inference on attributed random graphs: Fusion of graph features and content: An experiment on time series of Enron graphs

Fusion of information from graph features and content can provide superior inference for an anomaly detection task, compared to the corresponding content-only or graph feature-only statistics. In this paper, we design and execute an experiment on a time series of attributed graphs extracted from the Enron email corpus which demonstrates the benefit of fusion. The experiment is based on injecting a controlled anomaly into the real data and measuring its detectability.

Organizational Communication Networks and its Structural Changes Correlates to Organizational Disintegration

Email networks in contemporary organizations are fairly representative of the underlying communications networks. We show that changes in communication networks have implications for studying organization disintegration. In this paper, we analyzed the changing communication network structure at Enron Corporation during the period of its disintegration (2000-2001). Our goal was to understand how communication patterns and structure were affected by organizational disintegration. Drawing on (social) network disintegration theory, we tested several propositions using the Enron email corpus: 1) Number of cliques increases 2) Communication network becomes increasingly centralized, and 3) Connectedness among the top management executives increases, as organizations move towards disintegration. The results of the analyses and their implications indicate that during organizational disintegration process there are: higher level of clique activities, tendency toward greater decentralization, and increased connectivity among top management.

Using PLSI-U to detect insider threats by datamining e-mail

Despite a technology bias that focuses on external electronic threats, insiders pose the greatest threat to an organisation. This paper discusses an approach to assist investigators in identifying potential insider threats. We discern employees' interests from e-mail using an extended version of PLSI. These interests are transformed into implicit and explicit social network graphs, which are used to locate potential insiders by identifying individuals who feel alienated from the organisation or have a hidden interest in a sensitive topic. By applying this technique to the Enron e-mail corpus, a small number of employees appear as potential insider threats.

Topic and Role Discovery in Social Networks with Experiments on Enron and Academic Email

Previous work in social network analysis (SNA) has modeled the existence of links from one entity to another, but not the attributes such as language content or topics on those links. We present the Author-Recipient-Topic (ART) model for social network analysis, which learns topic distributions based on the direction-sensitive messages sent between entities. The model builds on Latent Dirichlet Allocation (LDA) and the Author-Topic (AT) model, adding the key attribute that distribution over topics is conditioned distinctly on both the sender and recipient---steering the discovery of topics according to the relationships between people. We give results on both the Enron email corpus and a researcher's email archive, providing evidence not only that clearly relevant topics are discovered, but that the ART model better predicts people's roles and gives lower perplexity on previously unseen messages. We also present the Role-Author-Recipient-Topic (RART) model, an extension to ART that explicitly represents people's roles.

Using Author Topic to detect insider threats from email traffic

One means of preventing insider theft is by stopping potential insiders from becoming actual thieves. This article discusses an approach to assist managers in identifying potential insider threats. By using the Author Topic [Rosen-Zvi Michal, Griffiths Thomas, Steyvers Mark, Smyth Padhraic. The author-topic model for authors and documents. In: Proceedings of the 20th conference on uncertainty in artificial intelligence; 2004. p. 487–94.] clustering algorithm, we discern employees' interests from their daily emails. These interests then provide a means to create an implicit and an explicit social network graph. This approach locates potential insiders by finding individuals who either (1) feel alienated from the organization (a key warning sign of a possible disgruntled worker) or (2) have a hidden interest in a sensitive (e.g. proprietary or classified) topic. In both cases, this is revealed when someone demonstrates an interest in a topic but does not share that interest with anyone in the organization. By applying this technique to the Enron email corpus, we produce coherent, sensible topics and reveal Sherron Watkins, the famous Enron whistleblower, as a potential insider threat from the viewpoint of the individuals behind the Enron scandal.

Communication Networks from the Enron Email Corpus “It's Always About the People. Enron is no Different”

The Enron email corpus is appealing to researchers because it represents a rich temporal record of internal communication within a large, real-world organization facing a severe and survival-threatening crisis. We describe how we enhanced the original corpus database and present findings from our investigation undertaken with a social network analytic perspective. We explore the dynamics of the structure and properties of the organizational communication network, as well as the characteristics and patterns of communicative behavior of the employees from different organizational levels. We found that during the crisis period, communication among employees became more diverse with respect to established contacts and formal roles. Also during the crisis period, previously disconnected employees began to engage in mutual communication, so that interpersonal communication was intensified and spread through the network, bypassing formal chains of communication. The findings of this study provide valuable insight into a real-world organizational crisis, which may be further used for validating or developing theories and dynamic models of organizational crises; thereby leading to a better understanding of the underlying causes of, and response to, organization failure.