Architecture and capabilities of a data warehouse for ATM research

Abstract

This paper describes the design, implementation, and use of a data warehouse that supports air traffic management (ATM) research at NASA’s Ames Research Center. The data warehouse, dubbed Sherlock, has been in development since 2009 and is a crucial piece of the ATM research infrastructure used by Ames and its partners. Sherlock comprises several components, including a database, a webbased user interface, and supplementary services for query and visualization. The information stored includes raw data collected from the National Airspace System (NAS), parsed and processed data, derived data, and reports derived from pre-defined queries. The raw data include a variety of flight information from live streams of FAA operational systems, weather observations and forecasts, and NAS advisories and statistics. The modified data comprise parsed and merged data sources and metadata, enabling parameterized searches for data of interest. The derived data represent the results of research analyses deemed to be of significant interest to a wide cross-section of users. Sherlock is implemented on an Oracle 11g database, with supplemental services built on open-source packages and custom software. It contains over 20 TB of data spanning several years, and more data are added daily. It has supported several research studies, such as finding similar days in the NAS and predicting imposition of traffic flow management restrictions. Planned enhancements include integrated search across data sources and the capability for large-scale analytics.