IDL

Interactive Display Language (IDL)


IDL is perhaps the most versatile package available for data display. A part of this versatility is very powerful processing of data in the form of arrays, which makes IDL similar to MATLAB in its capabilities. The drawback of this versatility is the need to allow many commands in the procedures for creating images and plots. For this reason it is best to address the array handling and programming aspects of IDL before turning to the actual data display for which IDL was created.


Table of Contents


The IDL Environment

The Meteorology Department has IDL installed on many workstations. These include the halo and the stratus computers (not nimbus). The halo computers can be accessed remotely via SSH. If you are using halo computers remotely, you must set up your X windows environment properly.

Once you are in the IDL environment, you may define variables and arrays, perform mathematical operations on them, and display the data produced. Any values you define will be saved in memory until you exit. To enter and exit the environment, from the UNIX command line type the following:

% idl ; OR
% idlde ; for the GUI version

> exit


Defining Variables and Arrays

Once you are in the IDL environment, defining variables and arrays is simple.

An integer
> x = 2
> print, x
> help, x
A real number (float)
> x = 2.0
> print, x
> help, x
A string
> x = '2'
> print, x
> help, x

> A = [1, 3, 4]
> print, A
> help, A

> B = [ [1,2,3], [4,5,6] ]
> print, B

You must use square brackets to define arrays. This distinquishes them from the ordinary brackets used for mathematical operations or for functions.

Any defined array variable may be combined symbolically as the dimensions allow, but beware of syntax:

> C = [A, A]
> print, C
> C = [ [A],[A] ]
> print, C
> D = [A,B] (whoops!)
> D = [ [A],[B] ]
> print, D

Basic arithmetic operations may be used in variable and array definitions as well. Most arithmetic operations will be performed on each element of the array independantly. If array sizes don't match, IDL will match up leading elements.

> testarr = 3 + A
> print, testarr

> testarr = 3*A
> print, testarr

> print, A
> print, B
> print, A + B
> print, A*B

IDL has many "built in" functions (as compared to "user-defined" functions discussed below). For example, built in functions can be used to calculate statistics.

> max_A = max(A)
> print, max_A
> mean_A = mean(A)
> print, mean_A


Type Conversions

IDL will automatically determine which data type a variable is from context.

> print, A
> print, A/3
> print, A/3.0
> print, float(A)/3


Array Operations

Indexing - all IDL arrays are zero indexed, which means the first element is accessed by using an index of zero. For multiple indices the sequence is [x,y] or [column, row]. Note how this differs from most mathematical indexing.

> c = [ [1,2,3],[4,5,6],[7,8,9] ]
> print, c[0,0]
> print, c[0,1]
> print, c[1,1:2]
> print, c[0,*]

Note that if a single index is supplied, IDL will count array items row-by-row like reading a book.

> print, c[4]

Selecting array items by criteria - the "where" function may be used to B return a vector of indices satisfying any given criteria.

> gt3 = where(c gt 3)
> print, gt3
> print, c(gt3)

Generating special Arrays - the most common arrays needed are either filled with zeros, or with values in ascending order. Functions are available for making both integer and floating versions of these.

> int_zeros = intarr(3,4)
> print, int_zeros
> float_zeros = fltarr(3,4)
> print, float_zeros
> count_int = indgen(3,4)
> print, count_int
> count_float = findgen(3,4)
> print, count_float

Array Multiplication - note that all vectors in IDL are defined as rows. This means IDL must assume row/column orientation by context. This leads to more than one possible definition for array multiplication. It is suggested that before using array multiplication blindly, the user test to see if the results will be as expected. The differences are easier to see with vectors rather than multi-index arrays:

> a = [1,2,3]
> b = [4,5]
> print, a#b
> print, b#a
> print, a##b

Note that the above matrix multiplication should not be confused with the element by element multiplication of two arrays using the scalar multiplication symbol '*':

> print,a*b


Built-in Functions and Arrays

IDL has the usual set of built-in mathematical functions such as cosine, exponentiation, etc. It is important to realize that these functions operate on arraysof numbers as easily as single numbers. Using this feature is much faster than using a loop to operate on each number individually, as well as being much easier to program. For example, assume you have a set of angles (in radians) that you want to find the cosine of:

angles = [0, pi/6, pi/3, pi/2]

If we were to program this with a loop (as you would need to do in C or Fortran) it would look like this:

> pi = 3.1416
> angles = [0,pi/6,pi/3,pi/2]
> x = fltarr[4]
> for a = 0,3 do x[a] = cos(angles[a])
> print, x
But you could do the same thing this way:
> pi = 3.1416
> angles = [0,pi/6,pi/3,pi/2]
> x = cos(angles)
> print, x
As a general rule, when working with arrays you should avoid loops as much as possible. The execution time can be decreased by an order of magnitude or more. It's also easier to read and write such programs.


Files: reading and writing

To write an array to a file, the file must be opened and assigned a file ID.

> a = indgen(3,4)
> print, a
> openw, 1, "testfile"
> printf, 1, a
> close, 1

To retrieve the array, we must know the size of the array and create a variable to receive it.

> openr, 1, "testfile"
> b = intarr(3,4)
> print, b
> readf, 1, b
> print, b
> close, 1

You may have several data types on each line of a file. If you have the variables predifined IDL can read this:

> a = 1 & b = 2.7 & c = 'hello'
> openw,1,'testfile'
> printf,1, a, b, c
> close,1

... later on ....

> a=0 & b=0.0 & c=''
> openr,1,'testfile'
> readf,1, a, b c
> close,1
> print,a,b,c

This would work fine, but beware if you put the string at the front of the line: unless told otherwise, IDL will alway read a string clear to the end of the line, so everything you put after a string will be read as a long string variable: numbers, spaces and all! To avoid this you need to read (and often write) using format statements, which resemble those used in fortran.

> a = 1
> b = 2.7
> c = 'hello'
> formatstring = '(a5,2x,f3.1,2x,I1)'
> openw,1,'test.txt'
> printf,1,c,b,a, format=formatstring
> close,1

> openr,1,'test.txt'
> readf,1,c,b,a, format = formatstring
> close,1
> print, a, b, c

After seeing how this works, try doing it without the format string. It doesn't work! If you are simply reading and writing numbers, format statements are not necessary except to make your files look pretty.


About Functions and Procedures

Except for arithmetic expressions and variable assignments, every command you issue in IDL will be either a function or a procedure. There are many predefined functions and procedures in IDL; the final section in this seminar will show you how to write your own.

A function accepts a list of arguments and returns a single variable value, which may be an array. The syntax is

> value = function_name(argument1, argument2,...)

In the previous section arrays of zeros were produced using the intarr function, in which the created array was the returned value and the dimensions were the arguments.

Procedures accept a list of arguments and may perform any set of operations available to IDL. The syntax is

procedure_name, argument1, argument2,.....

The print statement is a procedure, as will be the commands used to plot data in the next section.

Note on argument lists: normally the argument list must be in the same order as defined in the procedure or function. But if you know the variable names for each argument, you can assign the arguments without worrying about the order. For example, pretend someone has written a function called pointslope that calculates the slope between two points. The code for this function would look like this:

function pointslope, x1,y1,x2,y2
slope = float(y2-y1)/(x2-x1)
return, slope
end

Once this is compiled you could call this with actual numbers:

> slopevalue = pointslope(5,3,4,6)

They could have written the function like this:

function pointslope, X1=x1input,Y1=y1input,X2=x2input,Y2=y2input
slope = float(Y2-Y1)/(X2-X1)
return, slope
end

You may call this with the numbers in the order you prefer if you use the '=' sign:

> slopevalue = pointslope(X1=5,X2=4,Y1=3,Y2=6)

In the first case the order must match that of the function definition, but for the second case you may use whatever order you want provided you know the variable names used in the function.

Some arguments are binary, with values of either 0 or 1. In this case the two statements below are equivalent and either may be used in the argument list:

argument1 = 1     or     /argument1


Plotting Data

The various plotting procedures have some arguments that must be included, plus a large number of arguments with default values that can be changed by assigning values within the argument list. We will place the required arguments on the first line, with a few of the useful optional arguments on the following lines. The dollar symbol tells IDL that the next line is a continuation of the same command. To avoid overlapping windows from erasing each other, you'll want to issue the following command before running the plot commands:

> window, retain=2

The plot procedure requires vectors for the x and y values.

plot, x, y,  $
        xrange = [xmin,xmax], yrange = [ymin,ymax], $
        title = 'title', xtitle = 'xtitle', ytitle = 'ytitle'

example:
> x = indgen(6)
> print, x
> y = x^2
> plot,x,y
> plot, x, y, xrange=[2,6]

If you want to make a scatter plot, use the psym keyword with the plot command:

plot, x, y, psym=n, $
       xrange = [xmin,xmax], yrange = [ymin,ymax], $
       title = 'title', xtitle = 'xtitle', ytitle = 'ytitle'

example:
> x = [1,3,2,5,3,7]
> y = [0,6,4,3,5,1]
> plot, x, y, psym=1, title='Scatter Plot'

The contour procedure typically requires 3 two-dimensional arrays: one for the actual values of the data, one for the x coordinates, and one for the y coordinates. If only one array is provided, the x and y coordinates are assumed to be the x and y index values of the array.

contour, value_arr, x_arr, y_arr, $
     xrange = [xmin,xmax], yrange = [ymin,ymax], $
     levels = clev_vector, /fill,  /irregular

The fill keyword converts the plot from labelled contours to color coded filled contours. If you have vectors instead of 2D arrays for your input, you will need to use the irregular keyword.

example: > z = indgen(5,5)
> print, z
> contour, z
> contour, z, /fill

The surface procedure requires the same array inputs as the contour procedure, and can also work with a single array.

surface, value_arr, x_arr, y_arr, $
     xrange = [xmin,xmax], yrange = [ymin,ymax], $
     xtitle = 'xtitle', ytitle = 'ytitle', ztitle = 'ztitle'
example:
> surface, z

Often you simply want to look at the pixels exactly as they are arranged in the array. The TV function will do this, but it uses the default maximum brightness of one byte, or 255. Values higher than this will wrap around. To automatically scale your values to between 0 and 255, use the TVscl function:

> array = indgen(400,400,/long) ; a little smaller than the default window
> TVscl, array

This is often used to look at satellite data.

Saving Graphics to JPEG or TIFF Format

After you have created a graphical image but before it has been erased, it can be saved to a common format. Without going into too many explanitory details, the commands would be as follows:

> image = tvrd(true=1)
> write_jpeg, 'filename.jpg', image, quality=100, true=1

> image = tvrd(true=1)
> write_tiff, 'filename.tif', reverse(image, 3)

Note that the filename you are saving to must be a string variable (in quotes). The quality keyword for the JPG procedure indicates the amount of compression, and hence the final image quality.

Saving Graphics to PS Format

This is really opening up a can of worms... but it is the recommended choice for creating pretty publishable plots and figures. Instead of plotting to the "display" as was just done, plots are saved into a postscript "device." This "device" must be set BEFORE sending the plotting commands. After the plotting is completed, the device must be closed. Much too complicated to deal with in an intro seminar.


Writing Procedures

All IDL commands may be written in sequence in a procedure file that can be executed by typing the procedure name at the IDL prompt. The example below may be typed into a text file and saved with the name example.pro : IDL procedures MUST have the pro extension. Note the use of semicolons for comments.

pro example
; this procedure will plot the square root of the integers
; between two inputted values.  If any of the integers are
; negative, it will convert them to positive before proceeding.

print, "enter xlow, xhigh"
read,  xlow, xhigh

diff = xhigh - xlow
intvector = indgen(diff) + xlow
print, "the vector of integers is", intvector

; test for negatives
if (xlow lt 0) then print, 'negative values will be changed'

absintvector = abs(intvector)
root = sqrt(absintvector)

plot, intvector, root

end
You must save this file (if using a word processor make sure to save it as a text file!) using the '.pro' extension, then compile it. The IDL GUI interface has a compile option, but from the command line you would issue the following commands to compile and run the above procedure:
> .compile path/example.pro
> example

Actually, since the name of the file and the procedure are the same, you may not even have to issue the '.compile' command, since IDL will go looking for the file when you enter the 'example' command. If IDL doesn't find the file, try moving it to the RSI/IDL folder (if you are using the GUI) or including the complete file path in the filename (if using the command line).


System or Shell Commands

Occasionally you need to do something while in IDL that IDL can't do by itself, such as create directories or run programs written in another code. One way to do this is to precede an operating system command with a '$':

> $ mkdir newdirectory

This works great in interactive mode, but if you are using it to run a program from an IDL procedure, and the output is needed for the next line, this command will not wait for the result. So if you need to issue an operating system command from a procedure, use the 'spawn' command:

> spawn, 'wgrib -o outfile infile'

Note the use of quotes.


IDL Help

IDL has online help!! At the IDL command prompt, type "?" and it will bring up an indexed PDF file. If you have questions about specific commands, you can type "? command" at the prompt.


IDL Etc

IDL is an incredibly powerful software package, but it is a monster that must be tamed slowly. IDL can do perform just about any type of calculation, or create any type of plot, image or figure. However, the caveat is that some of the simplest things in concept (like annotation on a figure, or setting the color scale correctly), can test even the most advanced IDL programmer.

Finally, customized software can be created using IDL, including widget based software (sliders, buttons, etc). For example, users in this department have created software for analyses of satellite image data. These useful applications may include 1) selection of part of an image by mouse, 2) Changing the color scale through sliders, and 3) performing statistics on the selected box.