|
|
|
A preponderance of data from NASA's Earth Observing System (EOS) are archived in the HDF Version 4 (HDF4) format. The long-term preservation of these data is critical for climate and other scientific studies going many decades into the future.
Its rich structure, platform independence, and Application Programming Interfaces (API) and libraries make HDF4 very effective for working with the large and complex collection of EOS data products. Unfortunately, these features are achieved by employing a complex internal byte layout of HDF4 files, so future readability of HDF4 data depends on the preservation of the software that can interpret that layout. Having a way to access HDF4 data independent of a library could improve its viability as an archive format, and consequently give confidence that HDF4 data will be readily accessible forever, even if the HDF4 API and library are gone.
To address the need to simplify long-term access to EOS data stored in HDF4, a collaborative study between The HDF Group and NASAs Earth Science Data Centers investigated a new approach to accessing data in HDF4 files based on the creation of independent maps that describe the data in HDF4 files, and tools that can use these maps to recover data from those files. With this approach, relatively simple programs could extract the data from an HDF4 file, bypassing the need for the HDF4 library.
This report will describe the HDF4 mapping study, which included an assessment of the range of HDF4 formatted data held by NASA, development of a prototype HDF4 layout mapping language and format, and development of prototype tools to create layout maps and to read HDF4 data using layout maps. The report will also describe future plans to put the layout map approach into practice.
Back to Agenda