How to Read and Visualize HDF-EOS2 data Using IDL

IDLexternal (Interactive Data Language) is a programming language for scientific analysis and visualization. IDL provides APIs for reading and writing several file formats including HDF-EOS2 . This page presents a few examples on how to read and visualize HDF-EOS2 data via IDL.

IDL provides a set of functions for handling HDF-EOS2 files. Since each of them is equivalent to an HDF-EOS2 C API, those who are familiar with the HDF-EOS2 library can easily learn how to handle HDF-EOS2 files in IDL. For example, EOS_GD_OPEN, EOS_GD_ATTACH and EOS_GD_READFIELD are equivalent to GDopen, GDattach and GDreadfield, respectively.

Grid data

HDF-EOS grid data is special in that it does not store longitude and latitude values. Instead, HDF-EOS grid keeps only several parameters that can generate the entire longitude and latitude values. This approach can save much space, but makes it difficult to retrieve those values unless HDF-EOS API is used.

The degree of difficulty of obtaining longitude and latitude values depends on the projection method. The easiest method is the geographic projection. We will cover this case using a real AMSR-E AE_RnGdexternal file from NSIDC. Download the file here.

To access a data field in a grid data, first open the enclosing HDF-EOS2 file using EOS_GD_OPEN. Then, using the descriptor returned by that API, open the grid.

Since IDL does not require a variable declaration, neither file_id nor grid_id needs to be declared in advance. In this example(Figure 1), the accessed grid is named MonthlyRainTotal_GeoGrid, which resides in an HDF-EOS2 file named AMSR_E_L3_RainGrid_B05_200707.hdf.

Figure 1 Accessing a grid
FILE_NAME='AMSR_E_L3_RainGrid_B05_200707.hdf'
GRID_NAME='MonthlyRainTotal_GeoGrid'

file_id = EOS_GD_OPEN(FILE_NAME)
grid_id = EOS_GD_ATTACH(file_id, GRID_NAME)

Then, data fields, dimensions and attributes in this grid can be accessed.

EOS_GD_READFIELD reads data from a data field in a grid. As shown in Figure 2, the name of the data field being read is RrLandRain. Note that grid_id, the first argument, is the descriptor returned by EOS_GD_ATTACH. The third argument rrland is the buffer where the data are stored. Unlike using C APIs, users do not need to allocate memory for this buffer; memory is automatically allocated. Also, users do not need to specify the type of rrland because IDL is a dynamically typed language.

Figure 2 Reading a data field in a grid
DATAFIELD_NAME='RrLandRain'
status = EOS_GD_READFIELD(grid_id, DATAFIELD_NAME, rrland)

After finishing reading all fields, one needs to close the grid using EOS_GD_DETACH(Figure 3). Similarly, one needs to close the grid file using EOS_GD_CLOSE.

Figure 3 Closing the grid and the file
status = EOS_GD_DETACH(grid_id)
status = EOS_GD_CLOSE(file_id)
grid_id is what EOS_GD_ATTACH returned, and file_id is what EOS_GD_OPEN returned. This relation is implied by function names.

A few more steps are required to visualize the data field we just read. To generate a more meaningful plot, the data field needs to be associated with coordinates. As of our best knowledge, IDL does not automatically generate longitude and latitude values. Therefore, users need to provide these values. Note that the following explanation only applies to the geographic projection. For other projection methods, different methods are required.

The geographic projection, which is used by MonthlyRainTotal_GeoGrid, maps meridians to equally spaced, vertical straight lines and maps circles of latitude to evenly spread, horizontal straight lines[1]. This implies that we can precisely interpolate all longitude and latitude values if we know the following:

From the range and the number of points, we can get the space between two adjacent points, which can be called the scale. Then, we can say that the i-th longitude value is (i + offsetX) * scaleX + leftX. offsetX is given by Pixel Registration. Further explanation of Pixel Registration is out of the scope of this document. One can assume that the offset related to Pixel Registration(offsetX or offsetY) can be 0, 0.5 or 1. For more information, refer to the HDF-EOS2 Reference[2].

We need to retrieve the value of leftX, rightX, numX,offsetX, upperY,lowerY,numY and offsetY. To accomplish this, one may use HDF Viewexternal. After opening the HDF-EOS2 file in HDFView, the structure of this file appears in the left frame. In that frame, right click the AMSR_E_L3_RainGrid_B05_200707 text and issue Show Properties; the Properties window is opened(Figure 4). Click the Attributes tab to view StructMetadata.0 that includes its name, value, type and array size. Clicking the cell that represents the value (on the second column) to print the value at the bottom of that window, which gives us all information we need.

XDim (72) and YDim (28) are the values for numX and numY, respectively. Also, UpperLeftPointMtrs and LowerRightMtrs can be used to get the ranges. All numbers of UpperLeftPointMtrs and LowerRightMtrs are in DMS format[3]; so, those values actually mean 0 degree, 70 degrees, 360 degrees, and -70 degrees. As the name of UpperLeft, and LowerRight imply, we can get 0 as leftX, 360 as rightX, 70 as upperY and -70 as lowerY.

Another factor that affects the longitude and latitude values is the Pixel Registration option. Since Figure 4 does not specify this option, the default value HDFE_CENTER is assumed. HDFE_CENTER assigns 0.5 to both offsetX and offsetY.

Now, we have enough information from the file to proceed.

Form the above information, the scaleX is 5 degree and the scaleY is -5 degree. The longitude values are (i + 0.5) * 5 + 0 where i = 0, 1, 2, ..., 71. Similarly, the latitude values are (j + 0.5) * -5 + 70 where j = 0, 1, 2, ..., 27. These series of values can be conveniently generated by the following IDL code:
Figure 5 Calculating longitude and latitude values for the geographic projection
lon = FINDGEN(72) * 5 + 2.5
lat = FINDGEN(28) * (-5) + 67.5
FINDGEN is an IDL function that generates a series of values. After executing the above two lines of code, lon becomes 2.5, 7.5, 12.5, ..., 357.5, and lat becomes 67.5, 62.5, 57.5, ..., -67.5.

Finally, one can use CONTOUR function to draw a plot using the actual data, longitude values and latitude values(Figure 6). As CONTOUR has many options, users may need to refer to the detailed document about this function. Also, MAP_SET can be used to set the style of plots.

Figure 6 Visualizing a data field
MAP_SET, /GRID, /CONTINENTS

contourlevels = FINDGEN(13) * 25
CONTOUR, rrland, lon, lat, /OVERPLOT, LEVELS=contourlevels, /CELL_FILL, MIN_VALUE=0
/GRID turns on the dashed line along the same latitude or the same longitude, and /CONTINENTS draws continental boundaries. /OVERPLOT prevents the grid of latitude and longitude lines and continental boundaries from being erased when the contour is drawn. LEVELS defines the contour levels. Here, we defined LEVELS as 0, 25, 50, ..., 300. /CELL_FILL fills the contour plot. MIN_VALUE is set to 0 to ignore negative values because negative values don't have physical meaning for the rain rate, and the fill value was -1.

In the example, both lon and lat are one-dimensional array even though rrland is two-dimensional. Both the second argument and the third argument can be either one-dimensional or two-dimensional. For this case, each element of rrland is mapped to each element of the Cartesian product of lon and lat. The other case will be explained in the next section.

CONTOUR function will spawn a window that contains the contour plot(Figure 7).

See the complete code here. This code has one procedure called AMSR_E_L3_RainGrid; so, users can use this procedure by compiling the procedure and executing the compiled procedure as follows:

Figure 8 Using AMSR_E_L3_RainGrid procedure
.COMPILE AMSR_E_L3_RainGrid
AMSR_E_L3_RainGrid

Swath data

Unlike grid data, swath data contains geolocation fields such as Longitude and Latitude. This allows users to draw plots without the extra step to calculate longitude and latitude values; these values can be read from geolocation fields. However, swath data may contain dimension maps that allow the size of geolocation fields smaller or larger than that of the related data fields. If this is the case, retrieving longitude and latitude values becomes complicated. We will cover the case without dimension maps.

We will use one swath file from AMSR-E/Aqua L2A Global Swath Spatially-Resampled Brightness Temperatures product. Download the sample file here.

The steps to open, read and close a swath file is similar to the steps of handling a grid file.

Figure 9 Accessing a swath
FILE_NAME='AMSR_E_L2A_BrightnessTemperatures_V09_200206190029_D.hdf'
SWATH_NAME='Low_Res_Swath'

file_id = EOS_SW_OPEN(FILE_NAME)
swath_id = EOS_SW_ATTACH(file_id, SWATH_NAME)

DATAFIELD_NAME='23.8H_Approx._Res.3_TB_(not-resampled)'
status = EOS_SW_READFIELD(swath_id, DATAFIELD_NAME, data)
status = EOS_SW_READFIELD(swath_id, 'Longitude', lon)
status = EOS_SW_READFIELD(swath_id, 'Latitude', lat)

status = EOS_SW_DETACH(swath_id)
status = EOS_SW_CLOSE(file_id)
The name of APIs is slightly different; EOS_SW_OPEN is used instead of EOS_GD_OPEN. As the function name implies, functions that contain SW are for swath, and functions that contain GD are for grid. The above code reads one data field 23.8H_Approx._Res.3_TB_(not-resampled) and two geolocation fields, Longitude and Latitude. Usually, a swath data contains both geolocation fields, and they provide longitude values and latitude values.

As EOS_GD_READFIELD does, EOS_SW_READFIELD stores data elements at the third argument. Now that lon and lat contain longitude values and latitude values, one can pass those variables to CONTOUR function as the following arguments:

Figure 10 Visualizing a swath field
MAP_SET, /GRID, /CONTINENTS
CONTOUR, data, lon, lat, /OVERPLOT, NLEVELS=20, /CELL_FILL
NLEVELS=20 creates 20 equally spaced contour levels. This can be more convenient than using LEVELS. In this example, the second and third arguments of CONTOUR are two-dimensional because both Longitude and Latitude are two-dimensional. Then, each element of data is mapped to each element of lon and lat.

See the complete code here. This code has one procedure called AMSR_E_L2A_BrightnessTemperatures; so, users can use this procedure by compiling the procedure and executing the compiled procedure as follows:

Figure 11 Using AMSR_E_L2A_BrightnessTemperatures procedure
.COMPILE AMSR_E_L2A_BrightnessTemperatures
AMSR_E_L2A_BrightnessTemperatures

Figure 12 shows the result.

For comprehensive list of NASA IDL Examples with codes and plots, click here

References


Last modified: 11/11/2020
About Us | Contact Info | Archive Info | Disclaimer
Sponsored by Subcontract number 4400528183 under Raytheon Contract number NNG15HZ39C, funded by NASA / Maintained by The HDF Group