Heavy Hitters Research Articles

Mining Frequent Items Over the Distributed Hierarchical Continuous Weighted Data Streams in Internet of Things

Recently, with the increasing supply of band width and the diversification of applications in the Internet of Things (IoT), it has been a challenging problem to identify frequent items (also called heavy hitters) in high-speed and dynamically changing data streams. As well as, these data streams are from multiple sources in a distributed environment. To solve it, we propose the distributed-tracking schemes for continuously mining frequent items in the multi-level, non-regular tree-based communication structure. Our method employs a combination of local tracking and delays updating at each node to produce highly communication-efficient and space-efficient solutions. To reduce the communication cost, it only sends the frequency increments to violate a pre-defined threshold through a hierarchy of intermediate nodes, which is interposed between the monitoring nodes and the root node. With the information gathered, the root node continuously reports the set of frequent items. Two optimization approaches are proposed to minimize the worst-case total communication and minimize the worst-case maximum load on any link under any input streams. We perform extensive simulations with real traffic traces to evaluate the performances of the two optimization approaches.

An Optimal Algorithm for ℓ 1 -Heavy Hitters in Insertion Streams and Related Problems

We give the first optimal bounds for returning the ℓ 1 -heavy hitters in a data stream of insertions, together with their approximate frequencies, closing a long line of work on this problem. For a stream of m items in { 1, 2, … , n } and parameters 0 < ε < φ ⩽ 1, let f i denote the frequency of item i , i.e., the number of times item i occurs in the stream. With arbitrarily large constant probability, our algorithm returns all items i for which f i ⩾ φ m , returns no items j for which f j ⩽ (φ −ε) m , and returns approximations f˜ i with | f˜ i − f i | ⩽ ε m for each item i that it returns. Our algorithm uses O (ε −1 log φ −1 + φ −1 log n + log log m ) bits of space, processes each stream update in O (1) worst-case time, and can report its output in time linear in the output size. We also prove a lower bound, which implies that our algorithm is optimal up to a constant factor in its space complexity. A modification of our algorithm can be used to estimate the maximum frequency up to an additive ε m error in the above amount of space, resolving Question 3 in the IITK 2006 Workshop on Algorithms for Data Streams for the case of ℓ 1 -heavy hitters. We also introduce several variants of the heavy hitters and maximum frequency problems, inspired by rank aggregation and voting schemes, and show how our techniques can be applied in such settings. Unlike the traditional heavy hitters problem, some of these variants look at comparisons between items rather than numerical values to determine the frequency of an item.

Delayed Installation and Expedited Eviction: An Alternative Approach to Reduce Flow Table Occupancy in SDN Switches

Limited flow table size in switches is a major concern for SDN applications. The common approach to overcome this problem is to identify elephant flows and solely focus on them. However, there is no gold standard to assess the effectiveness of such greedy solutions. In this paper, we formally define this problem by choosing a cost function ( hit ratio ) and an objective function to optimize the ( average table occupancy ) and present the optimum solution (i.e., theoretical gold standard) for it. We model the problem as a knapsack problem, analyze how its solution minimizes the table occupancy, and the similarities to and differences from the default idle timeout mechanism used in OpenFlow. We also present a new approach to minimize flow table occupancy based on the insight gained from the knapsack model analysis. Our solution expedites rule evictions by forecasting the TCP flow termination from RST/FIN packets and delays rule installation by incubating non-TCP flows. It reduces average flow table occupancy between 16%–62% in various networks with less than 1.5% reduction in hit ratio. Using three real-world packet traces, we compare the performance of our solution with the theoretically optimum solution, the static idle timeout approach used in current OpenFlow systems, and heavy hitter detection approaches that are commonly used to solve this problem. We provide in-depth analysis of when and where our approach outperforms other solutions, while discussing why it might be better to use rate-based heavy hitter detection in some scenarios.

Fast flow volume estimation

The increasing popularity of jumbo frames means growing variance in the size of packets transmitted in modern networks. Consequently, network monitoring tools must maintain explicit traffic volume statistics rather than settle for packet counting as before. We present constant time algorithms for volume estimations in streams and sliding windows, which are faster than previous work. Our solutions are formally analyzed and are extensively evaluated over multiple real-world packet traces as well as synthetic ones. For streams, we demonstrate a run-time improvement of up to 2.4X compared to the state of the art. On sliding windows, we exhibit a memory reduction of over 100X on all traces and an asymptotic runtime improvement to a constant. Finally, we apply our approach to hierarchical heavy hitters and achieve an empirical 2.4-7X speedup.

SharesSkew: An algorithm to handle skew for joins in MapReduce

In this paper we offer an algorithm which computes the multiway join efficiently in MapReduce even when the data is skewed. Handling skew is one of the major challenges in query processing and computing joins is both important and costly. When data is huge distributed computational platforms must be used. The algorithm Shares for computing multiway joins in MapReduce has been shown to be efficient in various scenarios. It optimizes on the communication cost which is the amount of data that is transferred from the mappers to the reducers. However it does not handle skew. Our algorithm distributes Heavy Hitter (HH) valued records separately by using an adaptation of the Shares algorithm to achieve minimum communication cost. HH values of an attribute is decided by our algorithm and depends on the sizes of the relations (or the part of the relations with HH) and how these sizes interrelate with each other. Unlike other recent algorithms for computing multiway joins in MapReduce, which put a constraint on the number of reducers used, our algorithm puts a constraint on the size (number of tuples) of each reducer. We argue that this choice results in even distribution of the data to the reducers. Furthermore, we investigate a family of multiway joins for which a simpler variant of our algorithm can handle skew. We offer closed forms for computing the parameters of our algorithm for chain and symmetric joins.

Efficient Identification of TOP-K Heavy Hitters over Sliding Windows

Due to the increasing volume of network traffic and growing complexity of network environment, rapid identification of heavy hitters is quite challenging. To deal with the massive data streams in real-time, accurate and scalable solution is required. The traditional method to keep an individual counter for each host in the whole data streams is very resource-consuming. This paper presents a new data structure called FCM and its associated algorithms. FCM combines the count-min sketch with the stream-summary structure simultaneously for efficient TOP-K heavy hitter identification in one pass. The key point of this algorithm is that it introduces a novel filter-and-jump mechanism. Given that the Internet traffic has the property of being heavy-tailed and hosts of low frequencies account for the majority of the IP addresses, FCM periodically filters the mice from input streams to efficiently improve the accuracy of TOP-K heavy hitter identification. On the other hand, considering that abnormal events are always time sensitive, our algorithm works by adjusting its measurement window to the newly arrived elements in the data streams automatically. Our experimental results demonstrate that the performance of FCM is superior to the previous related algorithm. Additionally this solution has a good prospect of application in advanced network environment.

Computing Hierarchical Summary from Two-Dimensional Big Data Streams

There are many application domains, where hierarchical data is inherent, but surprisingly, there are few techniques for mining patterns from such important data. Hierarchical Heavy Hitters (HHH) and multilevel and Cross-Level Association Rules (CLAR) mining are well-known hierarchical pattern mining techniques. The problem in these techniques; however, is that they focus on capturing only global patterns from data but cannot identify local contextual patterns. Another problem in these techniques is that they treat all data items in the transaction equally and do not consider the sequential nature of the relationship among items within a transaction; hence, they cannot capture the correlation semantic within the transactions of the data items. There are many applications such as clickstream mining, healthcare data mining, network monitoring, and recommender systems, which require to identify local contextual patterns and correlation semantics. In this work, we introduce a new concept, which can capture the sequential nature of the relationship between pairs of hierarchical items at multiple concept levels and can capture local contextual patterns within the context of the global patterns. We call this notion Hierarchically Correlated Heavy Hitters (HCHH). Specifically, the proposed approach finds the correlation between items corresponding to hierarchically discounted frequency counts. We have provided formal definitions of the proposed concept and developed algorithmic approaches for computing HCHH in data streams efficiently. The proposed HCHH algorithm have deterministic error guarantees, and space bounds. It requires $O(\frac{\eta}{\epsilon _p\epsilon _s})$ memory, where $\eta$ is a small constant, and $\epsilon _p\in$ [0,1], $\epsilon _s\in$ [0,1] are user defined parameters on upper bounds of estimation error. We have compared the proposed HCHH concept with existing hierarchical pattern mining approaches both theoretically as well as experimentally.

Sketch Acceleration on FPGA and its Applications in Network Anomaly Detection

Sketch, a highly accurate data stream summarization technique, has gained much interest in the research community in recent years. Because of its sub-linear memory complexity, Sketch-based techniques consume significantly less memory than the traditional per-item-state techniques for processing high throughput data streams. One of the major applications of Sketch is in network anomaly detection, which is critical for network management and security in both Internet and data centers. In these applications, throughput is a key performance metric. Due to the low memory complexity, Sketch-based techniques can be supported by the fast onchip storage of the state-of-the-art computing platforms to achieve high throughput. In this work, we first propose a generic architecture on FPGA to accelerate Sketch and adopt it to 2 widely used Sketches: Count-min Sketch and K-ary Sketch. We propose online Sketchbased algorithms for 2 key network anomaly detection tasks: heavy hitter detection and heavy change detection. We adopt the proposed generic architecture for Sketch to accelerate these online algorithms. The post place-and-route results on a state-of-the-art FPGA show that our generic architecture can accelerate both Count-min Sketch and K-ary Sketch to over 150Gbps, demonstrating significant throughput performance improvements compared with other Sketch acceleration techniques. Our architectures for online anomaly detection tasks sustain 100-150 Gbps throughput for various system configurations.

Monsanto jumps on base editing

Monsanto is getting a head start in accessing a new gene-editing tool called DNA base editing that it hopes will help it improve major row crops. The agriculture giant says it will invest $100 million in a plant genome start-up called Pairwise. The firm was founded by gene-editing heavy hitters David Liu and Feng Zhang of Broad Institute of MIT & Harvard and J. Keith Joung of Massachusetts General Hospital and Harvard Medical School. Base editing is a type of gene editing that uses guide RNA and a form of the CRISPR/Cas9 enzyme to change adenine-thymine base pairs to guanine-cytosine pairs. In contrast, older CRISPR/Cas9 tools cut the double-stranded DNA at a particular point, and natural repair mechanisms put it back together, generally stopping the function of a gene. In human health, the instability of cytosine can create genetic mutations that lead to inherited diseases. Gene editing can turn back

Midwest Maize: How Corn Shaped the US Heartland by Cynthia Clampitt

Reviewed by: Midwest Maize: How Corn Shaped the US Heartland by Cynthia Clampitt Kelly J. Sisson Lessens Cynthia Clampitt, Midwest Maize: How Corn Shaped the US Heartland. Urbana: University of Illinois Press, 2015. 304 pp. $19.95. Corn—a remarkably malleable product of thousands of years of human ingenuity—offers tremendous grist for the writer’s mill. Scholars and journalists have long examined the grain’s significance within and across Native American populations; the grain’s utility in permitting colonization and settlement; the growth of the US corn belt as technologies like plows and cultivators, alongside grain elevators and railroads, linked urban and rural spaces and turned corn into a commodity; how politics, technological developments, and consumer demands transformed the nature of corn—and with it—modern American food systems; how the grain’s materiality and economic significance enabled and encouraged local communities to build bona fide palaces with and in honor of the grain; how the federal government’s interests in procuring the grain and producing young farmers engendered particular beliefs about gender, sexuality, and race; and how, in no uncertain terms, its post-Columbian spread transformed peoples’ lives around the globe. Thus Midwest Maize: How Corn Shaped the US Heartland, a new release by Cynthia Clampitt, is in the company of heavy hitters. And while Clampitt proposes that hers is the first to look at “what corn is, what it means, and how it shaped the region that came to be known as the Midwest” (3), her heavy reliance on previous scholarship renders that claim dubious. What, then, does it contribute? Midwest Maize begins with corn’s Mesoamerican beginnings before moving to a brief overview of corn biology and a short chapter on the corn belt’s growth. Next up are histories of urban-rural linkages and farmers’ needs for markets; the changing work of agricultural production; a short chapter on corn as a food directly consumed by humans; and another on corn’s role as a food for livestock. (Here, she touches on current debates about corn vs. grass feeding and conventional vs. organic, though this reviewer wishes that Clampitt would have used more data to back up statements about the ecologic [End Page 93] and economic merits of conventionally grown, corn-fed beef). A short (and distractingly located) chapter on popcorn follows. Next is another short but promising chapter, simply entitled “Transformations.” (Given that it examines many ways in which humans have turned corn into other products, it would have benefitted from a more thorough treatment of High-Fructose Corn Syrup.) A chapter assessing longstanding attempts at scientifically improving corn follows (and here again, a deeper assessment of the current state of genetically modified organism affairs would have been dandy). Chapter eleven (like the popcorn chapter) pauses to acknowledge mid-western corn celebrations. But greater historical context and more carefully nuanced explanations as to how, for example, the Chicago, Omaha and St. Louis world’s fairs established “that corn was the foundation of what had been built in the Heartland” (167) would have improved things immensely. Chapters twelve and thirteen, which Clampitt herself recognizes are poorly divided, seek to show how midwesterners became “connected to and dependent on corn” (173) from the mid-nineteenth century onward, and seem to form the book’s heart. The last two chapters focus first on corn recipes—here, her background as a food writer is much appreciated—and then present a smattering of current debates about modern corn geopolitics. One wishes, however, that instead of closing with the not-extraordinary summation that “it is fairly certain that corn will be at the center of the culture, economy, and foodways of the Midwest” (237), that she might have probed whether such a future even is possible, given the likelihood that climate change will transform the future of corn belt agriculture. While none of the chapters are terrible in and of themselves, the text as a whole has an uncertainty about its scope and purpose. It is unclear whether her interest lies in understanding the myriad ways humans have eaten corn as a vegetable or a starch, in defining the “heartland” as something different from the Midwest, or in grappling with how...

A learning-based measurement framework for traffic matrix inference in software defined networks

In this paper, we propose an intelligent framework for Traffic Matrix (TM) inference in Software Defined Networks (SDN) where the Ternary Content Addressable Memory (TCAM) entries of switches are partitioned into two parts to: 1) effectively aggregate part of incoming flows for aggregate measurements, and 2) de-aggregate and directly measure the most informative flows for per-flow measurements. These measurements are then processed to effectively estimate the size of network flows. Under hard resource constraints of limited TCAM sizes, we show how to design the optimal and efficient-compressed flow aggregation matrices. We propose an optimal Multi-Armed Bandit (MAB) based algorithm to adaptively measure the most rewarding flows. We evaluate the performance of our framework using real traffic traces from different network environments and by considering two main applications: TM estimation and Heavy Hitter (HH) detection. Moreover, we have implemented a prototype of our framework in Mininet to demonstrate its effectiveness.

The Time of My Life

The Time of My Life

SDPA: Toward a Stateful Data Plane in Software-Defined Networking

As the prevailing technique of software-defined networking (SDN), open flow introduces significant programmability, granularity, and flexibility for many network applications to effectively manage and process network flows. However, open flow only provides a simple “match-action” paradigm and lacks the functionality of stateful forwarding for the SDN data plane, which limits its ability to support advanced network applications. Heavily relying on SDN controllers for all state maintenance incurs both scalability and performance issues. In this paper, we propose a novel stateful data plane architecture (SDPA) for the SDN data plane. A co-processing unit, forwarding processor (FP), is designed for SDN switches to manage state information through new instructions and state tables. We design and implement an extended open flow protocol to support the communication between the controller and FP. To demonstrate the practicality and feasibility of our approach, we implement both software and hardware prototypes of SDPA switches, and develop a sample network function chain with stateful firewall, domain name system (DNS) reflection defense, and heavy hitter detection applications in one SDPA-based switch. Experimental results show that the SDPA architecture can effectively improve the forwarding efficiency with manageable processing overhead for those applications that need stateful forwarding in SDN-based networks.

Parallel mining of time-faded heavy hitters

In this paper we present PFDCMSS (Parallel Forward Decay Count–Min Space Saving) which, to the best of our knowledge, is the world first message–passing parallel algorithm for mining time–faded heavy hitters. The algorithm is a parallel version of the recently published FDCMSS (Forward Decay Count–Min Space Saving) sequential algorithm. We formally prove its correctness by showing that the underlying data structure, a sketch augmented with a Space Saving stream summary holding exactly two counters, is mergeable. Whilst mergeability of traditional sketches derives immediately from theory, we show that, instead, merging our augmented sketch is non trivial. Nonetheless, the resulting parallel algorithm is fast and simple to implement. The very large volumes of modern datasets in the context of Big Data present new challenges that current sequential algorithms can not cope with; on the contrary, parallel computing enables near real time processing of very large datasets, which are growing at an unprecedented scale. Our algorithm’s implementation, taking advantage of the MPI (Message Passing Interface) library, is portable, reliable and provides cutting–edge performance. Extensive experimental results confirm that PFDCMSS retains the extreme accuracy and error bound provided by FDCMSS whilst providing excellent parallel scalability. Our contributions are three-fold: (i) we prove the non trivial mergeability of the augmented sketch used in the FDCMSS algorithm; (ii) we derive PFDCMSS, a novel message–passing parallel algorithm; (iii) we experimentally prove that PFDCMSS is extremely accurate and scalable, allowing near real time processing of large datasets. The result supports both casual users and seasoned, professional scientists working on expert and intelligent systems.

A Technique for Efficient Query Estimation over Distributed Data Streams

Distributed data stream mining in a sliding window has emerged recently, due to its applications in many domains including large Telecoms and Internet Service Providers, financial tickers, ATM and credit card operations in banks and transactions in retail chains. Many of these large-scale applications prohibit monitoring data centrally at a single location due to their massive volume of the data; therefore, data acquisition, processing, and mining tasks are often distributed to a number of processing nodes, which monitor their local streams and exchange only the summary of data either periodically or on demand. While this offer many advantages, distributed stream applications possess significant challenges including problems related to an online analysis of the recent data, communication efficiency and various estimation of various complex queries. There are few existing techniques which solve problems related to distributed sliding window data stream; however, those techniques are focused on solving only simple problems and require high space, query, and communication cost, which can be a bottleneck for many of these large scale applications. In this paper, we propose an efficient query estimation technique by constructing a small sketch of the data stream. The constructed sketch uses a deterministic sliding window model and can estimate various complex queries, for both centralized and distributed applications; including point queries (i.e., range queries and heavy hitter queries), quantiles, inner product, and self-join size queries, with deterministic guarantees on the precision. The proposed approach improves upon recent existing work for these problems, in terms of the memory and query cost in a centralized setting and in terms of communication cost and merge complexity in a distributed setting. It requires $O(\frac{1}{\epsilon ^2}\log {(\epsilon N)})$ memory (where $0 is a user defined parameter), can provide estimates in $O(1)$ time, and processes each incoming record in $O(1)$ amortized time. Detailed experimental analysis, both in centralized and distributed settings demonstrates that in practice the proposed approach uses about six times less memory, and has about eight times less query time when compared to ECM sketches. In a distributed application, the proposed technique also significantly improves (around seven times) on the communication cost between distributed sites.

Fast and accurate mining of correlated heavy hitters

The problem of mining correlated heavy hitters (CHH) from a two-dimensional data stream has been introduced recently, and a deterministic algorithm based on the use of the Misra–Gries algorithm has been proposed by Lahiri et al. to solve it. In this paper we present a new counter-based algorithm for tracking CHHs, formally prove its error bounds and correctness and show, through extensive experimental results, that our algorithm outperforms the Misra–Gries based algorithm with regard to accuracy and speed whilst requiring asymptotically much less space.

On the most representative summaries of network user activities

A summary of a user’s Internet activities, such as web visitations, can provide information that closely reflects their interests and preferences. However, automating the summarization process is not trivial as the summary should strike a good balance between generality and specificity, while there is no gold standard for doing so.In our approach to summarizing user information, dubbed SUM, we develop two scoring mechanisms that cooperatively optimize for polarizing criteria. After mapping user activity information onto a category tree, the scoring mechanisms highlight the most representative tree node (or summary); the node provides an aggregated view of the activities most characteristic of the user. We evaluate our approach by using web activity on the network of a large Cellular Service Provider and summarizing it to devise interests of individual users as well as groups. We compare SUM against an algorithm that discovers Hierarchical Heavy Hitter and show that SUM uncovers previously unknown information about users.

Dynamic range majority data structures

Given a set P of n coloured points on the real line, we study the problem of answering range α-majority (or “heavy hitter”) queries on P. More specifically, for a query range Q, we want to return each colour that is assigned to more than an α-fraction of the points contained in Q. We present a new data structure for answering range α-majority queries on a dynamic set of points, where α∈(0,1). Our data structure uses O(n) space, supports queries in O((lg⁡n)/α) time, and updates in O((lg⁡n)/α) amortized time. If the coordinates of the points are integers, then the query time can be improved to O(lg⁡n/(αlg⁡lg⁡n)). For constant values of α, this improved query time matches an existing lower bound, for any data structure with polylogarithmic update time. We also generalize our data structure to handle sets of points in d dimensions, for d≥2, as well as dynamic arrays, in which each entry is a colour.

Virus control

Internet technology has become an important part of our daily lives, meaning that monitoring and identifying abnormal behaviours in network traffic are both crucial for proper operation and cybersecurity countermeasures. Moreover, the proliferation of computer networks has lead to an ever-increasing demand for bandwidth of 100Gbps and more. Thus, the scaling of monitoring and measuring capabilities for collecting accurate statistics has become a challenging issue. The group explained that their work is suitable for anomaly detection for high-speed networking and security applications Professor Yu-Kuen Lai, the lead author on this paper, spoke to us about what he's been working on: “We have developed a filtering scheme to assist superspreader [i.e., a host that infects disproportionally more secondary contacts than other hosts also infected with the same virus] detection in high-speed network traffic. Specifically, we use bitmap sketch data structures to filter out most of the non-superspreader sources from the network traffic with high probability”. It is known that in most of the network devices, the space of fast memory (SRAM) is limited. Therefore, by using this scheme, the memory space required by the measurement module to track the source IP in the network traffic can be reduced while achieving acceptable accuracy. The prototype implementation can process network traffic at a throughput of 27Gbps. Owing to the high speed of internet traffic, tracking all packets and flows to enable complex analysis in real-time is difficult. This is due to limited processing time and memory space. Sketch, an alternative to packet sampling, is a compact data structure capable of shortening substantial numbers of data elements. It relies on probabilistic properties to guarantee the accurate estimation of various attributes of the network traffic. Consequently, it is widely used in high-speed network measurement and monitoring applications. In addition to the superspreader detection application presented in this paper, Sketch algorithms are commonly used for various types of applications, such as distinct flow estimation, heavy hitter identification, traffic change detection, entropy estimation, and so on. Lai also talked to us about the limitations of this method of packet sampling: ‘as discussed in this paper, the sketch-guided filtering works by using a bitmap sketch, which is updated for each incoming packet during an observation period of measurement. When the number of bits set is approaching the number of bits in the bitmap, the filter performance decreases. This scheme is dependent on a predefined threshold parameter and the observation period length. This is a common problem when using a bitmap data structure in this kind of traffic monitoring application. To avoid this problem, we can set the length of an observation period to a small number, or increase the bitmap size, which in turn, increases the memory requirement.’ Sketch-guided filtering can reduce the total memory requirement in the hardware implementation of superspreader detection systems. However, compared to those which use a large amount of memory to count all sources and exact fan-out, it comes at the cost of a slight decrease in detection accuracy and the fan-out estimation. Counting the fan-out of each source in high-speed network traffic is prohibitively expensive in term of processing time and memory space. Most of the previous works of the superspreader detection focused on the detection algorithm and the fan-out estimation accuracy without filtering the sources of low fan-out. The proposed Sketch-guided filtering scheme first eliminates the low fan-out sources, and counts only the fan-out of spreader sources. It can be used in high-speed network systems with a lower memory requirement as compared to previous approaches. Professor Lai hopes that ‘in the short term these results can lead to more ideas and research on how to remove the unnecessary sources efficiently, so that the monitoring process can identify the anomalies accurately. In the long term, we expect that the sketch-guided filtering scheme can be used in real application on network devices for assisting superspreader detection. However, before reaching that stage, more works need to be done to verify the performance of the filter. Furthermore, an intelligent threshold setting, and a method to handle the filter “saturation” problem should be developed.’ The main innovation of the proposed system is to use the sketch-guided filtering mechanism to remove sources with potentially low fanout

From Streamlines to Fast Marching: Rapid Simulation and Performance Assessment of Shale-Gas Reservoirs by Use of Diffusive Time of Flight as a Spatial Coordinate

Summary Current industry practice for characterization and assessment of unconventional reservoirs mostly uses empirical decline-curve analysis or analytic rate- and pressure-transient analysis. High-resolution numerical simulation with local perpendicular bisector (PEBI) grids and global corner-point grids has also been used to examine complex nonplanar fracture geometry, interaction between hydraulic and natural fractures, and implications for the well performance. Although the analytic tools require many simplified assumptions, numerical-simulation techniques are computationally expensive and do not provide the more-geometric understanding derived from the depth-of-investigation (DOI) and drainage-volume calculations. We propose a novel approach for rapid field-scale performance assessment of shale-gas reservoirs. Our proposed approach is dependent on a high-frequency asymptotic solution of the diffusivity equation in heterogeneous reservoirs and serves as a bridge between simplified analytical tools and complex numerical simulation. The high-frequency solution leads to the Eikonal equation (Paris and Hurd 1969), which is solved for a “diffusive time of flight” (DTOF) that governs the propagation of the “pressure front” in the reservoir. The Eikonal equation can be solved by use of the fast-marching method (FMM) to determine the DTOF, which generalizes the concept of DOI to heterogeneous and fractured reservoirs. It provides an efficient means to calculate drainage volume, pressure depletion, and well performance and can be significantly faster than conventional numerical simulation. More importantly, in a manner analogous to streamline simulation, the DTOF can also be used as a spatial coordinate to reduce the 3D diffusivity equation to a 1D equation, leading to a comprehensive simulator for rapid performance prediction of shale-gas reservoirs. The speed and versatility of our proposed method makes it ideally suited for high-resolution reservoir characterization through integration of static and dynamic data. The major advantages of our proposed approach are its simplicity, intuitive appeal, and computational efficiency. We demonstrate the power and utility of our method by use of a field example that involves history matching, uncertainty analysis, and performance assessment of a shale-gas reservoir in east Texas. A sensitivity study is first performed to systematically identify the “heavy hitters” affecting the well performance. This is followed by history matching and an uncertainty analysis to identify the fracture parameters and the stimulated-reservoir volume. A comparison of model predictions with the actual well performance shows that our approach is able to reliably predict the pressure depletion and rate decline.