Implementing HDF5 as a filesystem: design and operational considerations

Enabling HDF5 files to function through the operating system as traditional filesystems offers a number of intriguing possibilities. The foremost concept is to allow non-HDF application software to transparently interact with HDF5 files & datasets, thereby enhancing methods for interoperability and high performance across user formats and user communities. A second option is to use the HDF5 filesystem as a project management tool for organizing disparate datasets into a coherent and comprehensive data compilation, thereby making long-term archiving more tenable through a systemic method of provenance management. A third capability is to insert a new type of community filter between application software and the operating system, allowing for the creation of standardized community data structures by user communities. Because of the robust API capabilities of HDF5, any filesystem derived shall have features beyond traditional filesystem designs, such as dedicated virtual filenames that perform unique functions such multiple simultaneous filenames within this filesystem, or automated concatenation of datasets into a single visible filename. Described are prototypes, proposed design strategies, and operational objectives.

Back to Agenda

Last modified: 06/02/2017