Memorandum on reading NCEP/NCAR Reanalysis data of NetCDF format

by Takemasa Miyoshi on 10/13/2004
Department of Meteorology, University of Maryland, College Park

The purpose of this document

Background

NCEP/NCAR Reanalysis data is freely available on the web at http://www.cdc.noaa.gov/cdc/reanalysis/. The data is in the NetCDF format which can be read by various ways such as C, Fortran, IDL, Matlab, and GrADS. There are tips for reading the data files provided in their website. A sample program in fortran is provided there, but I had a difficulty to run it using NetCDF and UDUNITS libraries. Thus, I tried GrADS, IDL, and Matlab, and finally, I am satisfied with using Matlab to read and manipulate the data.

Introduction of several methods to read the data

Several methods include

Fortran

To use fortran to read the Reanalysis data, NetCDF and UDUNITS libraries are necessary. They can be downloaded from UNIDATA's website. By usual procedures of installation, they can be installed rather easily. However, it was not easy for me to compile/link the fortran sample program provided by CDC/NOAA to read the data.

IDL

Actually I don't know how to use IDL very well, but I found IDL has a function to directly read NetCDF formatted files. I tried to read the data, and it worked. It did not automatically convert 2-byte packed data to float numbers. Since I am not so familiar with IDL, I sought other ways, although IDL would be very useful.

GrADS

GrADS worked pretty well. Just by typing
ga-> sdfopen [filename.nc]
GrADS opens a reanalysis data file and automatically converts the 2-byte packed data to float numbers. Once the file is open, GrADS commands work as usual. I could define variables to store some portions of the data, and I could make a contour plots. It is possible to 'fwrite' by GrADS (to write data into a file in the GrADS format), which enables to convert NetCDF formatted files to GrADS formatted files. Thus, fortran program could be used to read the GrADS formatted files, which would be much easier for programmers who are not familiar with NetCDF formats. However, this procedure requires additional disk space to store the GrADS formatted files before making various computations using the data.

Matlab

This is what I did, which is described below in detail.

How to use Matlab to read reanalysis data

This is an example to read 6-hourly snapshots of height field data in 2004.
nc=netcdf('hgt.2004.nc','nowrite');
var=nc{'hgt'}(it,:,:,:)*nc{'hgt'}.scale_factor(:)+nc{'hgt'}.add_offset(:);
'var' is a 3-d array which contains height field data in the form of (level,lat,lon) at an instant time step 'it'. Note that the order is opposite from a usual array. 'it' indicates time, e.g. it=1 means January 1 00Z, it=5 means January 2 00Z, and so on if this is 6-hourly snapshot data.

Download and Installation

http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html is the website that contains how to download and install the toolbox. I got no problem to download and install in a RedHat-like Linux PC with Matlab6 R13. These were all I did to install the toolbox.

Explanation

nc{'hgt'}
indicates an object of the variable named 'hgt'.
(:)
makes the object concrete, that is,
nc{'hgt'}(:,:,:,:)
is not an object but an array that contains 4-d height fields for whole 2004. Numbers of the columns depend on the dimension of the variable. Now, dimensions of (:,:,:,:) are like (time,level,longitude,latitude), whose order is opposite from usual arrays.
nc{'lat'}
also exists. 'hgt' is one of the variables contained in the .nc file. 'ncdump' shows what kind of variables are contained in the file. Usually, 'level', 'lat', 'lon', and 'time' exist in addition to an atmospheric variable.

Attributes of the variables are indicated by '.'. For example,
nc{'hgt'}.scale_factor
is an object of one of the attributes of 'hgt' variable named 'scale_factor'.
(:)
makes the object concrete, that is,
nc{'hgt'}.scale_factor(:)
is not an object but a value of the attribute.

Acknowledgement

When I was struggling with the NetCDF data, Shu-Chih Yang suggested me to use Matlab and guided me how to use it. Many thanks to Shu-Chih.
(c)2004 Takemasa Miyoshi