How to Visualize HDF-EOS data Using IDL via OPeNDAP

IDL external is a solution for data visualization and it can visualize a remote HDF-EOS file through OPeNDAP .

Using NetCDF APIs for IDL 8.4 or above

IDL version 8.4 and later includes support for OPeNDAP using netCDF4 library.

Figure 1. Define an OPeNDAP URL with NASA Earthdata authentication information.

      url = 'https://username:password@acdisc.gesdisc.eosdis.nasa.gov/opendap/Aqua_AIRS_Level3/AIRS3QPM.006/2003/AIRS.2003.01.01.L3.RetQuant_IR031.v6.0.9.0.G13220001334.hdf'

    

Call the NCDF_OPEN function. You can inquire the contents of the data using NCDF_LIST.

Figure 2. Open data from the URL

      file_id = NCDF_OPEN(url)

      NCDF_LIST, url, /VARIABLES

You can retrieve a variable (e.g., Temperature_MW_A) data and geolocation data from OPeNDAP resource using NCDF_VARGET. In addtion, you can retrieve attribute value of a variable using NCDF_ATTGET.

Figure 3. Get data from the URL

    var_id = NCDF_VARID(file_id, 'Temperature_MW_A')

    NCDF_VARGET, file_id, var_id, data

    lat_id = NCDF_VARID(file_id, 'Latitude')

    NCDF_VARGET, file_id,lat_id, lat

    lon_id = NCDF_VARID(file_id, 'Longitude')

    NCDF_VARGET, file_id, lon_id, lon

    NCDF_ATTGET, file_id, var_id, '_FillValue', fillvalue

Once data are read correctly, you can follow the same visualization technique in our comprehensive example and generate plot (Figure 4.)

Figure 4. Contour plot for AIRS dataset

You can see the complete IDL OPeNDAP code from here. You need to change username and password in the complete code.

Using OPeNDAP IDL Client for IDL 8.3 or below

OPeNDAP IDL Client method is not going to work with NASA server that requires Earthdata login.

OPeNDAP used to provide the OPeNDAP IDL Client that is no longer supported. In this example, we assume that you have both IDL software and the OPeNDAP IDL Client installed on your system. The IDL is a commercial software that requires a purchase of license. You can download the OPeNDAP IDL Client from here external for free.

OPeNDAP IDL Client (idl-client 3.6.1 external ) requires OCAPI (ocapi 1.4.3) and cURL library externpal . If you use IDL 8.0 or above, the latest cURL library will not work with ocapi 1.4.3 and idl-client 3.6.1. It will cause OPENDAP_CONNECT() failure. Please use curl version 7.14.0 if you see such failure. In addition, the make install may fail during idl-client 3.6.1 installation. If it fails, you should copy manually the newly-built libraries under the .libs/ directory to your install directory (e.g., /path/to/install/prefix/lib). You can use ls -a UNIX command to list the hidden .libs/ directory under the unpacked idl-client source directory.

The first step is to create a URL variable. Define an OPeNDAP URL variable that serves an HDF-EOS file. You can download the file in this example here.

Figure 5. Define an OPeNDAP URL

      url = 'https://eosdap.hdfgroup.org:8080/opendap/data/NASAFILES/hdf4/AIRS.2003.02.05.L3.RetStd_H001.v6.0.12.0.G14112124328.hdf'
    

Call the OPENDAP_GET function. This function is provided by the OPeNDAP IDL client library. If IDL failed to retrieve the data specified in URL, you'll see an error message failed to retrieve OPeNDAP data. Also, the return value of stat will be 0. You can view the contents of the data by typing help, /str, data.

Figure 6. Open data from the URL

      stat = OPENDAP_GET(url, data, mode='data')

      PRINT, stat

      HELP, /str, data

From the contents, you'll see that OPeNDAP Array is represented as an IDL STRUCT variable.

Figure 7. Get data description from the URL

       
      TOPOGRAPHY
      
      STRUCT
      
      -> <Anonymous> Array[1]

You can retrieve a variable (e.g., topography) data and geolocation data inside the OPeNDAP Grid as follows.

Figure 8. Get data from the URL

      dataset = data.TOPOGRAPHY.TOPOGRAPHY

      lat = data.LATITUDE.LATITUDE

      lon = data.LONGITUDE.LONGITUDE

      levels = 12

As you can infer from the above code, IDL represents an OPeNDAP Grid data as a structure(e.g., data.TOPOGRAPHY). Inside the Grid structure, each member variable name(e.g., data.TOPOGRAPHY.lat) becomes another IDL structure and its array data is referred as the member of its own structure (e..g., data.TOPOGRAPHY.lat.lat). The levels will be used later to generate a contour map.

Now we're ready to draw the dataset defined above. The following code prepares a window with 12 colors and grid map. The CONTOUR command fills the grid map with HDF-EOS data(e.g., data.TOPOGRAPHY.TOPOGRAPHY.TOPOGRAPHY).

Figure 9. Display data from the URL

      DEVICE, DECOMPOSED=0

      LOADCT, 33, NCOLORS=12, BOTTOM=3

      WINDOW, 0, TITLE='AIRS.2008.10.27.L3 Topography', XSIZE=600, YSIZE=400

      MAP_SET, /GRID, /CONTINENT

      CONTOUR, dataset, lon, lat, /OVERPLOT, /FILL, C_COLORS=Indgen(levels)+3, BACKGROUND=1, NLEVELS=levels

Figure 10. Contour plot for Topography dataset

As you can see from the Figure 10, the color filled contour can hide the original map grid and continents because the dataset covers the world. Thus, we want to draw the outlines of the continent again. The following codes will add the grid lines and world maps on top of the color-filled contour map.

Figure 11. Draw grids and maps

      MAP_GRID, /LABEL, COLOR=255

      MAP_CONTINENTS, COLOR=255

The Figure 12 shows the final IDL image.

Figure 12. Contour plot for Topography dataset with Map

You can see the complete code from here.

Last modified: 11/11/2020