Managing research data with self-documenting files

Abstract

Processing biomedical research data is frequently complex due to the evolutionary nature of experiments and the requisite modification of analysis software. For the past several years we have been evolving a set of software tools designed to improve our ability to respond to evolving experimental designs. These tools allow the investigator to easily manipulate the research data and specify desired data transformations at run time. Coupling of analysis software to research data files is dependent on data files that are commented in a manner similar to that used in programming languages. The resulting annotated data files are self-documenting, and their use facilitates visual interpretation of displayed data as well as automatic processing of subsets of data. Here we present a formal description of a self-documenting file and describe several software tools that facilitate processing of biomedical research data.