Usage of NCL to Access remote HDF-EOS5 MLS Files

1. Introduction

In this document we describe how to use NCL^[1] to visualize a remote HDF-EOS5 MLS files served by NASA GES DISC Hyrax OPeNDAP server external . For the general usages of NCL, please refer to the NCL user's guides^[2].

2. URS Registration and Setting Up Cookies

NASA GES DISC OPeNDAP server access requires user registration and cookies. If you do not have Earthdata Login, please register external first and have username and password ready. Then, authorize data access by following How to Authorize NASA GES DISC DATA ARCHIVE in Earthdata Login instruction.

Once you have Earthdata Login username and password, you need to set up cookies to access data following GES DISC Earthdata Login for Data Access guide. We highlight the key steps from the NASA's guide here with complete sample files.

To set up cookies properly, you will need the following 3 files in your home directory (e.g., /home/eoshelp on Linux or /Users/eoshelp on Mac assuming that eoshelp is your UNIX system's login name). Click each file how it looks like.

2.1 CREATE .netrc file for login / password

The first file .netrc should have Earthdata login and password. For example, if your username is hdfeos and password is 1234abcd, the file should have the following line.

machine urs.earthdata.nasa.gov login hdfeos password 1234abcd

Please edit the above file to match your username and password. Since the file contains the password, make sure that others cannot see it by changing permission.

$chmod go-rwx /home/eoshelp/.netrc

2.2 Create .urs_cookies file using CURL or WGET

The second file, .urs_cookies, should be created automatically by either wget or curl command.

If you like to use wget, issue the following command to create the cookie file. Please note that proper escaping with '\' is necessary.

  
  $wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies http://acdisc.gesdisc.eosdis.nasa.gov:80/opendap/HDF-EOS5/Aura_MLS_Level2/ML2BRO.004/2016/MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5.dods\?BrO_L2gpValue\[0:1:0\]\[0:1:0\] -O test.dods
      

If you like to use curl, issue the following command to create the cookie file. Please note that proper escaping with '\' is necessary.

    
        $curl -n -c ~/.urs_cookies -b ~/.urs_cookies -L -g --url http://acdisc.gesdisc.eosdis.nasa.gov:80/opendap/HDF-EOS5/Aura_MLS_Level2/ML2BRO.004/2016/MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5.dods\?BrO_L2gpValue\[0:1:0\]\[0:1:0\] -o test.dods
      

If the above command succeeds, you will get the following output when you check the content of test.dods file.

      $more test.dods

      "test.dods" may be a binary file.  See it anyway? y

      Dataset {

       Float32 BrO_L2gpValue[BrO_Latitude = 1][BrO_Pressure = 1];

      } MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5;

     Data:

     ^@^@^@^A^@^@^@^A^@^@^@^@

If you don't see the above output, your login/password and system is not working properly with NASA GES DISC OPeNDAP server. You cannot access data until this step works. If you get the same output, you can delete the temporary output test.dods file.

2.3 CREATE .dodsrc FILE for .NETRC and .URS_COOKIES files

The final step is to create .dodsrc file if you don't already have one and add the following lines.

HTTP.COOKIEJAR=/Users/eoshelp/.urs_cookies

HTTP.NETRC=/Users/eoshelp/.netrc

3. How to Open and Read OPeNDAP data with NCL

The following code shows a typical way to use NCL to open and read a variable through OPeNDAP. This code needs to be typed at the prompt that NCL shows.

Figure 1 A typical NCL code to read and plot OPeNDAP data

      load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"

      load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"

      load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"

      begin 

      url = "http://acdisc.gesdisc.eosdis.nasa.gov:80/opendap/HDF-EOS5/Aura_MLS_Level2/ML2BRO.004/2016/"

      fName = "MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5" 

      if ( isfilepresent(url+fName) ) then

      f = addfile ( url + fName, "r")

      end if

      ; Check if the remote file is opened successfully

      ; and what variables are available.

      ; print(f)

In the above code, the url points to the remote MLS file being served by OPeNDAP. Use print command to check if the url file is opened successfully and to see what variables are available inside the file.

The sample MLS file name is MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5. You can download it from here via FTP if you want to try NCL OPeNDAP access with local Hyrax server.

4. Setting up key MLS Variables

The next code snippet shows a typical way to handle MLS variables. MLS file holds the key data under L2gpValue and related geo-location information under Latitude, Longitude, Time, and Pressure variable.

Figure 2 Handle MLS specific variables

      T = f->L2gpValue 

      lat = f->Latitude

      lon = f->Longitude

      lev = f->Pressure

      time = f->Time

      ; Re-scale the values.

      T = T*1e7

      T@units= T@units + "*1e-7"

In the above code, we re-scaled the L2gpValue values since they are too small. You need to adjust the scale factor depending on the MLS product.

5. Visualizing MLS Data

The final code snippet is almost identical to the official NCL MLS example code in NCL HDF Application Page.

Figure 3 MLS Data Visualization

      ; The rest of the code is almost identical to hdf5eos_3.ncl

      printMinMax(T, True)

      print("==============")

      ; Create an "elapsed time" variable.

      telapse
      
       = (time - time(0))/60

      telapse@long_name = "Elapsed Time (minutes)"

      telapse@units
      
       = "minutes since "+time(0)

      ; Plot

      pltType= "pdf"

      pltName= fName+".L2gpValue.ncl"

wks = gsn_open_wks (pltType,pltName); open workstation
gsn_define_colormap(wks,"amwg") ; choose colormap
dumb= NhlNewColor(wks,0.7,0.7,0.7); add gray to colormap

res = True
res@gsnMaximize= True ; make ps/eps/pdf
res@gsnSpreadColors= True
res@gsnSpreadColorEnd= -2 ; do not use gray for contours

      res@gsnPaperOrientation= "portrait" ; force portrait

res@tiMainString = fName

res@cnFillOn = True
res@cnLinesOn= False
res@cnLineLabelsOn = False
res@cnFillMode = "RasterFill" ; faster
res@cnRasterSmoothingOn= True

res@lbLabelAutoStride= True
res@lbOrientation= "Vertical"

      ; Change these values based on min/max values of L2gpValue dataset. 
res@cnLevelSelectionMode = "ManualLevels" ; set manual contour levels
res@cnMinLevelValF = -6. ; set min contour level
res@cnMaxLevelValF = 6. ; set max contour level
res@cnLevelSpacingF=1. ; set contour spacing

res@trYReverse = True ; reverse y-axis
T&lev= lev; assign "pressure" coordinates
T&time = telapse; assign temporal coordinates

res@gsnYAxisIrregular2Linear = True
res@tmYLFormat = "f"; force minimal precision 
res@tiYAxisString= "Pressure (hPa)"

res@gsnLeftString= T@title

plot= gsn_csm_contour (wks,T(lev|:,time|:) , res)

      ; Create trajectory plot

      mpres
      
      = True
      
      ; Plot options desired.

      mpres@gsnFrame
      
       = False ; Don't advance the frame
mpres@gsnMaximize= True
mpres@mpLandFillColor= "gray70"; color of land

      mpres@gsnPaperOrientation= "portrait"; force portrait
plot = gsn_csm_map_ce(wks,mpres) ; Draw map

gsres= True; "Graphic Style" resources
gsres@gsMarkerSizeF= 10.0; Marker size
gsres@gsMarkerThicknessF = 1.0 ; Marker thickness
gsres@gsMarkerColor= "Blue"; Marker color
gsres@gsMarkerIndex= 1 ; Marker style

mpres@tiMainString = fName

gsn_polymarker(wks,plot,lon,lat,gsres) ; plot trajectory
frame(wks)

      end

In the above code, we adjusted the res@cnMinLevelValF and res@cnMaxLevelValF to get a nice graph since the values in that range are the most meaningful values based on the pressure level.

See the complete code here. Run the code as follows:

Figure 4 Running MLS NCL code

ncl mls.ncl

Then, it will create a MLS-Aura_L2GP-BrO_v04-23-c03_2016d302.he5.ncl.pdf file as a result. The PostScript file will have two images like Figure 5 and Figure 6.

Figure 5 Contour plot for MLS L2gpValue dataset

Figure 6 Geo-location plots for MLS L2gpValue dataset

6. References

[1] NCAR Common Language (NCL). http://www.ncl.ucar.edu/
[2] NCL Documentation. NCL. http://www.ncl.ucar.edu/Document/index.shtml

Last modified: 06/02/2017