|
|
|
Update on HDF, including recent changes to the software, new releases, THG collaborations, and future plans. Session will include an overview of the HDF4.2r2, HDF5 1.6.6, and 1.8.0 releases, as well as updates on completed and on-going THG projects including crash-proofing HDF5, efficient append to HDF5 datasets, and indexing in HDF5.
We will introduce the fully functional HDF5-OPeNDAP data handler in this talk. The features we have added into the data handler includes
At the end, we will show demos of using an OPeNDAP client to read HDF-EOS5 Aura data.
This talk presents the HDF Group’s approach to software engineering. We will share with our users day-to-day maintenance practices and operations at THG along with the future steps to take to assure robustness, sustainability and low cost maintenance of the HDF software.
We will discuss new features of the HDF5 1.8.0 release and how they may affect different aspects of application’s performance from files sizes to raw data I/O speed. We will also discuss best practices on using those features.
Current status of HDF-EOS and access tools will be summarized. Update on HDF-EOS, HDFView plug-in and The HDF-EOS to GeoTIFF (HEG) conversion tool, including recent changes to the software, ongoing maintenance, upcoming releases, future plans, and issues will be discussed.
The Hierarchical Data Format (HDF) has been a data format standard in NASA's Earth Observing System Data and Information System (EOSDIS) since the 1990s. Its rich structure, platform independence, full-featured Application Programming Interface (API), and internal compression make it very useful for archiving science data and utilizing them with a rich set of software tools. However, a key drawback for long-term archiving is the complex internal byte layout of HDF files, requiring one to use the API to access HDF data. This makes the long-term readability of HDF data for a given version dependent on long-term allocation of resources to support that version.
The majority of the data from NASA’s Earth Observing System (EOS) have been archived in HDF Version 4 (HDF4) format. To address the long-term archival issues for these data a collaborative study between The HDF Group and NASAs EOSDIS data centers is underway. One of the first activities undertaken has been an assessment of the range of HDF4 formatted data held by NASA to determine the capabilities inherent in the HDF format that have been used in practice. Based on the results of this assessment, methods for producing a map of the layout of the HDF Version 4 files held by NASA will be prototyped using a markup-language-based HDF tool to map the layout of the HDF Version 4 files. The resulting maps should allow a separate program to read the file without recourse to the HDF API. To verify this, two independent tools based solely on the map files will be developed and tested with a variety of data products archived by NASA.
Funded by NOAA's Scientific Data Stewardship (SDS) program, we will investigate how to store NASA ECS data and metadata into HDF5 Archival Information Packages (AIP). To achieve this, the HDF4 to HDF5 conversion tool will be enhanced so that converted ECS data can be read through the NetCDF4 interface. In addition, metadata tools that convert ECS collection and granule level metadata to NOAA's collection level and NARA's METS standard will be developed.
There is a growing interest in finding ways to integrate HDF5 with traditional database applications. Some applications have used HDF5 for storing data and metadata associated with a given test, experiment, or simulation, then used a traditional relational database management system for querying about that data. Others use HDF5 itself like a database by storing indexes in HDF5 files along with the data, then implementing a query API to query the data. In this talk, we will present some of the ways that HDF5 has been used in support of database-like operations, and discuss possible future directions for this kind of usage for HDF5.
HDF Explorer is a data visualization program that reads Hierarchical Data Format files (HDF, HDF-EOS and HDF5) and also netCDF data files.HDF Explorer offers a simple yet powerful interface for the visualization of HDF data. The data is just a click of the mouse away. Data is first viewed in a tree-like interface, and then optionally loaded and visualized in a variety of ways. HDF Explorer features include fast access to data, grid, scalar and vector views. It also allows exporting your data either as a ASCII text file or a bitmap image.
HDF Explorer supports HDF5 dimension scales.
Last modified: 06/02/2017