Some Benefits of Storing AE Data in a Modern Database Format

Abstract

During the development of our first AE system in the mid-1980s, we decided to store AE data in a binary file format. Back then, this was the best solution since data could be accessed randomly, not only sequentially, and stored to disk at highest speed and compactness. Several tasks can read the data simultaneously for online analysis. Since then the raw hit and parametric data are stored to a file with extension PRI. Waveform data and features extracted from it are stored to files with extension TRA and TRF, respectively. The fact that this format is still in use today speaks for its advantages. Its capability of simultaneously storing and reading of over hundred thousand hits per second during online analysis is still unsurpassed. A bottleneck is caused by the complexity of managing rather different kinds of data, such as hit driven and time driven features, waveforms, external parameters, labels, and others. This bottleneck makes it difficult to add new features and functions, and to give other parties’ software read-access to all kinds of AE data in real time. Meanwhile the development in PC technology as well as in data management presents some nice alternatives in storing and organizing acquired AE data. The most flexible way in overcoming the limitations of the current file format and to address the upcoming challenges lies in using a database engine that organizes and manages measurement data. Trials have proven that performance of certain database engines satisfies our expectations and requirements with regard to speed, safety, and efficiency. These results as well as the upcoming challenges convinced us to start with the development of a new file format based on a modern SQL (Structured Query Language) database engine. SQLite is now implemented in our latest software release (R2013.0809) with excellent stability and maturity. Online analysis of over 100,000 hits per second is ongoing supported. It was never before so easy to support new features and functions by an AE data file format. The update to the new file format is included in the normal yearly update rate. Interested parties are invited to request an evaluation key that allows trying out the new software.