Reading NCEP/NCAR Reanalysis NetCDF data

From Metograds Wiki
Jump to: navigation, search

Originally written by Takemasa Miyoshi to briefly describe several ways to read NCEP/NCAR Reanalysis data of NetCDF format.

NCEP/NCAR Reanalysis data is freely available on the web at http://www.cdc.noaa.gov/cdc/reanalysis/. The data is in the NetCDF format which can be read by various ways such as C, Fortran, IDL, Matlab, and GrADS. There are tips for reading the data files provided in their website. A sample program in fortran is provided there, but I had a difficulty to run it using NetCDF and UDUNITS libraries. Thus, I tried GrADS, IDL, and Matlab, and finally, I am satisfied with using Matlab to read and manipulate the data.


Methods of Reading Data

Fortran

  • [GOOD] speedy and easy to make various computations using the data once the data are read
  • [BAD] It's rather hard to compile/link and run

To use fortran to read the Reanalysis data, NetCDF and UDUNITS libraries are necessary. They can be downloaded from UNIDATA's website. By usual procedures of installation, they can be installed rather easily. However, it was not easy for me to compile/link the fortran sample program provided by CDC/NOAA to read the data.

IDL

  • [GOOD] A function to read NetCDF data is provided by the language
  • [BAD] it is necessary to convert 2-byte packed data to float numbers manually

Actually I don't know how to use IDL very well, but I found IDL has a function to directly read NetCDF formatted files. I tried to read the data, and it worked. It did not automatically convert 2-byte packed data to float numbers. Since I am not so familiar with IDL, I sought other ways, although IDL would be very useful.

GrADS

  • [GOOD] The command 'sdfopen' opens reanalysis data very easily
  • [GOOD] 2-byte packed data are automatically converted to float numbers
  • [GOOD] GrADS commands work as usual
  • [BAD] It is limited and slow to make various computations using the data

GrADS worked pretty well. Just by typing

ga-> sdfopen [filename.nc]

GrADS opens a reanalysis data file and automatically converts the 2-byte packed data to float numbers. Once the file is open, GrADS commands work as usual. I could define variables to store some portions of the data, and I could make a contour plots. It is possible to 'fwrite' by GrADS (to write data into a file in the GrADS format), which enables to convert NetCDF formatted files to GrADS formatted files. Thus, fortran program could be used to read the GrADS formatted files, which would be much easier for programmers who are not familiar with NetCDF formats. However, this procedure requires additional disk space to store the GrADS formatted files before making various computations using the data.

Matlab

  • [GOOD] easy to open files and read the data
  • [GOOD] easy to manipulate the data after reading
  • [BAD] a little slow especially to output to files
  • [BAD] An additional toolbox for reading NetCDF is required

How to read with Matlab

  • Download NetCDF toolbox for Matlab
  • Install it
  • Test to read reanalysis data
  • Convert 2-byte packed data to meaningful numbers

This is an example to read 6-hourly snapshots of height field data in 2004.

nc=netcdf('hgt.2004.nc','nowrite'); var=nc{'hgt'}(it,:,:,:)*nc{'hgt'}.scale_factor(:)+nc{'hgt'}.add_offset(:);

'var' is a 3-d array which contains height field data in the form of (level,lat,lon) at an instant time step 'it'. Note that the order is opposite from a usual array. 'it' indicates time, e.g. it=1 means January 1 00Z, it=5 means January 2 00Z, and so on if this is 6-hourly snapshot data.

Download and Installation

http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html is the website that contains how to download and install the toolbox. I got no problem to download and install in a RedHat-like Linux PC with Matlab6 R13.

  • download mexcdf60
  • test if 'mexcdf60' works
  • run netcdf_install.m

These were all I did to install the toolbox.

Explanation

nc{'hgt'}

indicates an object of the variable named 'hgt'.

(:)

makes the object concrete, that is,

nc{'hgt'}(:,:,:,:)

is not an object but an array that contains 4-d height fields for whole 2004. Numbers of the columns depend on the dimension of the variable. Now, dimensions of (:,:,:,:) are like (time,level,longitude,latitude), whose order is opposite from usual arrays.

nc{'lat'}

also exists. 'hgt' is one of the variables contained in the .nc file. 'ncdump' shows what kind of variables are contained in the file. Usually, 'level', 'lat', 'lon', and 'time' exist in addition to an atmospheric variable.

Attributes of the variables are indicated by '.'. For example,

nc{'hgt'}.scale_factor is an object of one of the attributes of 'hgt' variable named 'scale_factor'.

(:) makes the object concrete, that is,

nc{'hgt'}.scale_factor(:) is not an object but a value of the attribute.