Previous: Intrinsic functions, Up: ncap2 netCDF Arithmetic Processor


4.1.5 Intrinsic mathematical functions

ncap2 supports the standard mathematical functions supplied with most operating systems. Standard calculator notation is used for addition +, subtraction -, multiplication *, division /, exponentiation ^, and modulus %. The available elementary mathematical functions are:

abs(x)
Absolute value Absolute value of x. Example: abs(-1) = 1
acos(x)
Arc-cosine Arc-cosine of x where x is specified in radians. Example: acos(1.0) = 0.0
acosh(x)
Hyperbolic arc-cosine Hyperbolic arc-cosine of x where x is specified in radians. Example: acosh(1.0) = 0.0
asin(x)
Arc-sine Arc-sine of x where x is specified in radians. Example: asin(1.0) = 1.57079632679489661922
asinh(x)
Hyperbolic arc-sine Hyperbolic arc-sine of x where x is specified in radians. Example: asinh(1.0) = 0.88137358702
atan(x)
Arc-tangent Arc-tangent of x where x is specified in radians between -pi/2 and pi/2. Example: atan(1.0) = 0.78539816339744830961
atanh(x)
Hyperbolic arc-tangent Hyperbolic arc-tangent of x where x is specified in radians between -pi/2 and pi/2. Example: atanh(3.14159265358979323844) = 1.0
ceil(x)
Ceil Ceiling of x. Smallest integral value not less than argument. Example: ceil(0.1) = 1.0
cos(x)
Cosine Cosine of x where x is specified in radians. Example: cos(0.0) = 1.0
cosh(x)
Hyperbolic cosine Hyperbolic cosine of x where x is specified in radians. Example: cosh(0.0) = 1.0
erf(x)
Error function Error function of x where x is specified between -1 and 1. Example: erf(1.0) = 0.842701
erfc(x)
Complementary error function Complementary error function of x where x is specified between -1 and 1. Example: erfc(1.0) = 0.15729920705
exp(x)
Exponential Exponential of x, e^x. Example: exp(1.0) = 2.71828182845904523536
floor(x)
Floor Floor of x. Largest integral value not greater than argument. Example: floor(1.9) = 1
gamma(x)
Gamma function Gamma function of x, Gamma(x). The well-known and loved continuous factorial function. Example: gamma(0.5) = sqrt(pi)
ln(x)
Natural Logarithm Natural logarithm of x, ln(x). Example: ln(2.71828182845904523536) = 1.0
log(x)
Natural Logarithm Exact synonym for ln(x).
log10(x)
Base 10 Logarithm Base 10 logarithm of x, log10(x). Example: log(10.0) = 1.0
nearbyint(x)
Round inexactly Nearest integer to x is returned in floating point format. No exceptions are raised for inexact conversions. Example: nearbyint(0.1) = 0.0
pow(x,y)
Power Value of x is raised to the power of y. Exceptions are raised for domain errors. Due to type-limitations in the C language pow function, integer arguments are promoted (see Type Conversion) to type NC_FLOAT before evaluation. Example: pow(2,3) = 8
rint(x)
Round exactly Nearest integer to x is returned in floating point format. Exceptions are raised for inexact conversions. Example: rint(0.1) = 0
round(x)
Round Nearest integer to x is returned in floating point format. Round halfway cases away from zero, regardless of current IEEE rounding direction. Example: round(0.5) = 1.0
sin(x)
Sine Sine of x where x is specified in radians. Example: sin(1.57079632679489661922) = 1.0
sinh(x)
Hyperbolic sine Hyperbolic sine of x where x is specified in radians. Example: sinh(1.0) = 1.1752
sqrt(x)
Square Root Square Root of x, sqrt(x). Example: sqrt(4.0) = 2.0
tan(x)
Tangent Tangent of x where x is specified in radians. Example: tan(0.78539816339744830961) = 1.0
tanh(x)
Hyperbolic tangent Hyperbolic tangent of x where x is specified in radians. Example: tanh(1.0) = 0.761594155956
trunc(x)
Truncate Nearest integer to x is returned in floating point format. Round halfway cases toward zero, regardless of current IEEE rounding direction. Example: trunc(0.5) = 0.0
The complete list of mathematical functions supported is platform-specific. Functions mandated by ANSI C are guaranteed to be present and are indicated with an asterisk 1. and are indicated with an asterisk. Use the ‘-f’ (or ‘fnc_tbl’ or ‘prn_fnc_tbl’) switch to print a complete list of functions supported on your platform. This prints a list of functions and whether they are supported for netCDF variables of intrinsic type NC_FLOAT and NC_DOUBLE. 2

EXAMPLES

See the ncap.in and ncap2.in scripts released with NCO for more complete demonstrations of ncap and ncap2 functionality, respectively (these scripts are available on-line at http://nco.sf.net/ncap.in and http://nco.sf.net/ncap2.in).

Define new attribute new for existing variable one as twice the existing attribute double_att of variable att_var:

     ncap2 -O -s 'one@new=2*att_var@double_att' in.nc out.nc

Average variables of mixed types (result is of type double):

     ncap2 -O -s 'average=(var_float+var_double+var_int)/3' in.nc out.nc

Multiple commands may be given to ncap2 in three ways. First, the commands may be placed in a script which is executed, e.g., tst.nco. Second, the commands may be individually specified with multiple ‘-s’ arguments to the same ncap2 invocation. Third, the commands may be chained together into a single ‘-s’ argument to ncap2. Assuming the file tst.nco contains the commands a=3;b=4;c=sqrt(a^2+b^2);, then the following ncap2 invocations produce identical results:

     ncap2 -O -v -S tst.nco in.nc out.nc
     ncap2 -O -v -s 'a=3' -s 'b=4' -s 'c=sqrt(a^2+b^2)' in.nc out.nc
     ncap2 -O -v -s 'a=3;b=4;c=sqrt(a^2+b^2)' in.nc out.nc

The second and third examples show that ncap2 does not require that a trailing semi-colon ‘;’ be placed at the end of a ‘-s’ argument, although a trailing semi-colon ‘;’ is always allowed. However, semi-colons are required to separate individual assignment statements chained together as a single ‘-s’ argument.

ncap2 may be used to “grow” dimensions, i.e., to increase dimension sizes without altering existing data. Say in.nc has ORO(lat,lon) and the user wishes a new file with new_ORO(new_lat,new_lon) that contains zeros in the undefined portions of the new grid.

     defdim("new_lat",$lat.size+1); // Define new dimension sizes
     defdim("new_lon",$lon.size+1);
     new_ORO[$new_lat,$new_lon]=0.0f; // Initialize to zero
     new_ORO(0:$lat.size-1,0:$lon.size-1)=ORO; // Fill valid data

The commands to define new coordinate variables new_lat and new_lon in the output file follow a similar pattern. One would might store these commands in a script grow.nco and then execute the script with

     ncap2 -O -v -S grow.nco in.nc out.nc

Imagine you wish to create a binary flag based on the value of an array. The flag should have value 1.0 where the array exceeds 1.0, and value 0.0 elsewhere. This example creates the binary flag ORO_flg in out.nc from the continuous array named ORO in in.nc.

     ncap2 -O -s 'ORO_flg=(ORO > 1.0)' in.nc out.nc

Suppose your task is to change all values of ORO which equal 2.0 to the new value 3.0:

     ncap2 -s 'ORO_msk=(ORO==2.0);ORO=ORO_msk*3.0+!ORO_msk*ORO' in.nc out.nc

This creates and uses ORO_msk to mask the subsequent arithmetic operation. Values of ORO are only changed where ORO_msk is true, i.e., where ORO equals 2.0. In the future, ncap2 will support the Fortran90 where construct to further simplify this syntax.

This example uses ncap2 to compute the covariance of two variables. Let the variables u and v be the horizontal wind components. The covariance of u and v is defined as the time mean product of the deviations of u and v from their respective time means. Symbolically, the covariance

[u'v'] = [uv]-[u][v] where [x] denotes the time-average of x and x'

denotes the deviation from the time-mean. The covariance tells us how much of the correlation of two signals arises from the signal fluctuations versus the mean signals. Sometimes this is called the eddy covariance. We will store the covariance in the variable uprmvprm.

     ncwa -O -a time -v u,v in.nc foo.nc # Compute time mean of u,v
     ncrename -O -v u,uavg -v v,vavg foo.nc # Rename to avoid conflict
     ncks -A -v uavg,vavg foo.nc in.nc # Place time means with originals
     ncap2 -O -s 'uprmvprm=u*v-uavg*vavg' in.nc in.nc # Covariance
     ncra -O -v uprmvprm in.nc foo.nc # Time-mean covariance

The mathmatically inclined will note that the same covariance would be obtained by replacing the step involving ncap2 with

     ncap2 -O -s 'uprmvprm=(u-uavg)*(v-vavg)' foo.nc foo.nc # Covariance

As of NCO version 3.1.8 (December, 2006), ncap2 can compute averages, and thus covariances, by itself:

     ncap2 -O -s 'uavg=u.avg($time);vavg=v.avg($time);uprmvprm=u*v-uavg*vavg' \
              -s 'uprmvrpmavg=uprmvprm.avg($time)' in.nc foo.nc

We have not seen a simpler method to script and execute powerful arithmetic than ncap2.

ncap2 utilizes many meta-characters (e.g., ‘$’, ‘?’, ‘;’, ‘()’, ‘[]’) that can confuse the command-line shell if not quoted properly. The issues are the same as those which arise in utilizing extended regular expressions to subset variables (see Subsetting Variables). The example above will fail with no quotes and with double quotes. This is because shell globbing tries to interpolate the value of $time from the shell environment unless it is quoted:

     ncap2 -O -s 'uavg=u.avg($time)'  in.nc foo.nc # Correct (recommended)
     ncap2 -O -s  uavg=u.avg('$time') in.nc foo.nc # Correct (and dangerous)
     ncap2 -O -s  uavg=u.avg($time)   in.nc foo.nc # Fails ($time = '')
     ncap2 -O -s "uavg=u.avg($time)"  in.nc foo.nc # Fails ($time = '')

Without the single quotes, the shell replaces $time with an empty string. The command ncap2 receives from the shell is uavg=u.avg(). This causes ncap2 to average over all dimensions rather than just the time dimension, and unintended consequence.

We recommend using single quotes to protect ncap2 command-line scripts from the shell, even when such protection is not strictly necessary. Expert users may violate this rule to exploit the ability to use shell variables in ncap2 command-line scripts (see CCSM Example). In such cases it may be necessary to use the shell backslash character ‘\’ to protect the ncap2 meta-character.

Whether a degenerate record dimension is desirable or undesirable depends on the application. Often a degenerate time dimension is useful, e.g., for concatentating, but it may cause problems with arithmetic. Such is the case in the above example, where the first step employs ncwa rather than ncra for the time-averaging. Of course the numerical results are the same with both operators. The difference is that, unless ‘-b’ is specified, ncwa writes no time dimension to the output file, while ncra defaults to keeping time as a degenerate (size 1) dimension. Appending u and v to the output file would cause ncks to try to expand the degenerate time axis of uavg and vavg to the size of the non-degenerate time dimension in the input file. Thus the append (ncks -A) command would be undefined (and should fail) in this case. Equally important is the ‘-C’ argument (see Subsetting Coordinate Variables) to ncwa to prevent any scalar time variable from being written to the output file. Knowing when to use ncwa -a time rather than the default ncra for time-averaging takes, well, time.


Footnotes

[1] ANSI C compilers are guaranteed to support double precision versions of these functions. These functions normally operate on netCDF variables of type NC_DOUBLE without having to perform intrinsic conversions. For example, ANSI compilers provide sin for the sine of C-type double variables. The ANSI standard does not require, but many compilers provide, an extended set of mathematical functions that apply to single (float) and quadruple (long double) precision variables. Using these functions (e.g., sinf for float, sinl for long double), when available, is (presumably) more efficient than casting variables to type double, performing the operation, and then re-casting. NCO uses the faster intrinsic functions when they are available, and uses the casting method when they are not.

[2] Linux supports more of these intrinsic functions than other OSs.