Converts ancillary and dump files from 32-bit -> 64-bit, or from 64-bit -> 32-bit. Also unpacks any fields which are packed with the WGDOS or Cray 32-bit methods (new functionality, 28 Jan 2004; bugfix 23 March 2004; further bugfix 21 June 2006).
conv48 infile outfile
32 -> 64 bit conversion, with unpacking
conv84 infile outfile
64 -> 32 bit conversion, with unpacking
conv88 infile outfile
unpacking only, input and output both at 64-bit
conv44 infile outfile
unpacking only, input and output both at 32-bit
This is less likely to be useful, so
conv44 executable is
not built by default. Type:
Assumes native byte-ordering. Use
bigend before and/or
afterwards as required.
Should be portable (tested on linux, and earlier version briefly tested on Compaq alpha).
There is a previous version available which does not support unpacking but is possibly better tested. Download convancil-20021122.tar.gz (NB includes bugfix 22 Nov 02) and build/use similarly. With this version, for each record with packed fields, it will issue a message that it is writing zeros instead of unpacked values.
NB the same will happen (zeros written, message issued), even with the later version, for fields which are packed with the GRIB packing method. If this is a problem, please ask, as code for GRIB unpacking is not included but is available and could be incorporated on request.
The 64 -> 32 bit conversion/unpacking can also be achieved using
ieee -32" on a Cray. (Though
not do 32 -> 64 bit conversion.)
Thanks to Jeff Cole for the unpacking code.
Subsets UM ancillary / dump / fields files by writing out a new file with only the desired PP records, but still in the UM file format. The purpose in almost all cases is to retain a smaller file. Supports files in either little- or big-endian byte ordering, and at either 32-bit or 64-bit (via different executables).
The following record selection criteria are currently supported:
Keep instantaneous fields. This is the main use envisaged when writing this utility. The intended purpose is to retain the fields in a UM dump file which are necessary for restarting the model, while losing the fields related to climate meaning. The climate mean system adds a number of time-averaged fields at the end of each start dump; these in many cases considerably add to the size of the start dumps, unnecessarily if archiving start dumps for the purpose of starting new model runs.
Prompt for each record. This is the most generic option, although maybe cumbersome for regular use. With each PP record, some information is displayed (mainly stash codes and level), and then the user says whether to include it or not. To reduce the amount of responding to prompts, it is possible to respond with a number so that the next N records are included or excluded.
Include all records. This will include all valid records, but will still remove space reserved for unused PP headers in the file. In this configuration, the utility (probably) duplicates Jeff Cole's "umpack" utility.
Include fields from specifed STASH section. This is probably more useful for use with field files of diagnostics than with start dumps. (To remove unnecessary fields from a start dump, it is probably better to use the "instantaneous fields" than to use this option specifying section 0, because the climate means may include prognostic fields.)
A command line utility. Usage message from 64-bit version follows. The 32-bit version is used identically via the other executable. (Note that the precision and byte ordering options must match the input file, and the output file will have the same; this is not the utility for converting these things. Expect to see many errors if you get this wrong.)
subset_um_64 -- subsets records in a UM file (64-bit version) Usage: subset_um_64 [options] input_file output_file Options are: record selection options: -I keep instantaneous fields [default] -P prompt for each record -A include all records (just removes unused headers, like 'umpack') -S SECT include fields from stash section SECT (e.g. 0 for prognostic) (do not specify multiple criteria; only the last will have effect) byte-ordering options: -b big-endian -l little-endian -n native [default] -s swapped (non-native) other options: -c clobber existing output file -v be more verbose -h display this help message
Sorry, these require the distinctly non-free package IDL.
rdancil.pro: read an
ancillary file into IDL arrays.
wrancil.pro: write an
ancillary file from IDL arrays.
wrancil,filename, $ fixhdr,intc,realc,levdepc,rowdepc,coldepc,fieldc,extrac, $ temphist,cfi1,cfi2,cfi3,lookup,data, $ [/swap],[bits=bits] rdancil -- same syntax(where
bitsmay be 64 (the default) or 32, and "
/swap" is for byte-swapped files).
New: As an alternative there are versions which returns "data" as an array of
pointers (pointing to an array of values for each record) rather than one long
array of values. See
generate structure, containing named fields, from "lookup" array of PP
the reverse operation (requires
display a few key fields from lookup table.
Example: "rewind" all data times in 32-bit restart dump by 100 years
rdancil,'xabtea.daj4310', $ fixhdr,intc,realc,levdepc,rowdepc,coldepc,fieldc,extrac, $ temphist,cfi1,cfi2,cfi3,lookup,data, $ bits=32 ;; change date in fixed-length header fixhdr=fixhdr-100 fixhdr=fixhdr-100 ;; change PP headers s=pparray_to_struct(lookup) ok=where(s.lbyr ne -32768) s.lbyr[ok]=s.lbyr[ok]-100 s.lbyrd[ok]=s.lbyrd[ok]-100 lookup=struct_to_pparray(s) wrancil,'xabtea.da94310', $ fixhdr,intc,realc,levdepc,rowdepc,coldepc,fieldc,extrac, $ temphist,cfi1,cfi2,cfi3,lookup,data, $ bits=32
A program to do this, built on
wrancil.pro above, is documented on a separate page.
write an ancillary file based on input ancillary, but with either 2D fields
spread to 3D, or with 3D fields zonally averaged to 2D (depending on options
Usage: see comment lines at top of script.
(NB requires all above scripts.)
umtodate reports the date corresponding to a UM filename,
umdate (new) reports the dates in a UM ancillary file header,
sortum sorts a list of UM filenames,
with correct chronological order for monthly or seasonal mean files,
datetoum turns a date into a UM date code.
(and add execute permissions).
umdatetake a list of filenames either on standard input or on the command line.
umdatehas some additional flags; type
umdatefor a usage message.
datetoumprompts interactively for the date.
umtodatealso handles files which have a ".nc" extension, etc.
umdatereports all the dates in the fixed header of an ancillary file. It is not dependent on the filename, so it can handle long and short time conventions also. But the file must exist, and must be ancillary file format (not, say, netcdf).
% ls xabbqa.dah8cg0 xabbqa.dah9210 xabbqa.pmh9apr.nc xabbqa.pmh9mar.nc xabbqa.dah9110 xabbqa.dah92g0 xabbqa.pmh9feb.nc xabbqa.pmh9may.nc xabbqa.dah91g0 xabbqa.pmh8dec.nc xabbqa.pmh9jan.nc % sortum * xabbqa.dah8cg0 xabbqa.dah9110 xabbqa.dah91g0 xabbqa.dah9210 xabbqa.dah92g0 xabbqa.pmh8dec.nc xabbqa.pmh9jan.nc xabbqa.pmh9feb.nc xabbqa.pmh9mar.nc xabbqa.pmh9apr.nc xabbqa.pmh9may.nc % ls | sortum | umtodate xabbqa.dah8cg0: 00:00, 16 Dec 1978 xabbqa.dah9110: 00:00, 1 Jan 1979 xabbqa.dah91g0: 00:00, 16 Jan 1979 xabbqa.dah9210: 00:00, 1 Feb 1979 xabbqa.dah92g0: 00:00, 16 Feb 1979 xabbqa.pmh8dec.nc: dec 1978 xabbqa.pmh9jan.nc: jan 1979 xabbqa.pmh9feb.nc: feb 1979 xabbqa.pmh9mar.nc: mar 1979 xabbqa.pmh9apr.nc: apr 1979 xabbqa.pmh9may.nc: may 1979
% ls xaaqdo@pdw14c1 xaaqdo@pdw15c1 % umdate * xaaqdo@pdw14c1: First 1991-09-01 00:00:00, Last 3213-12-01 01:00:00, Interval 2004-01-25 01:01:00 xaaqdo@pdw15c1: First 1991-09-01 00:00:00, Last 3214-12-01 01:00:00, Interval 2004-01-25 07:15:47
Convert UM dumps to NetCDF
There are various versions of this wrapper to convsh "out there". This one also supports creating zonal means, and extracting a single named field. (Also uses long field names based on the stash code, as per xconv, rather than the PP code which is the convsh default.)
um2nc [-z] [(-f|-freg) 'pattern'] infile1 [[infile2 ...] outfile]
-zto take zonal mean.
-fregto write only field with matching pattern.
-ffor exact match;
-fregfor case-insensitive regexp match.
outfilenot specified, outfile is
infile.zm.ncif zonal mean).
um2nc xabcda.pa????? myfile.nc
um2nc -f 'LW HEATING RATES' xabcda.pa????? lwhr.nc
um2nc -z -freg '^u.*wind' xabcda.pa????? ubar.nc
(In case you still want it, here's the previous version.)
Returns full pathname of most recent UM leave file(s) (matching given stems if specified at start of filename), and optionally extracts some information (timesteps, run time).
lastleave [options] [-~] [stem [stem...]]
-b num: produce code benchmark, assuming
numtimesteps per day
-tsfor this to work)
-c: search also for comp.leave files
-C: search ONLY for comp.leave files
-d dir:look for leave files in given directory
$MY_OUTPUT, or defaults to
-num): show num leave files (for each stem). (Show all if num=0.)
-p: show number of processors
-s: show size
-t: show time and date
-T: show running time (in qsmain)
-ts: count the "ATMOS TIMESTEP" lines
-~: abbreviate home directory path to "
lastleave-- last .leave (or .archive) file in any job
lastleave xaad-- in any job from experiment xaad
lastleave -C xaada-- last comp.leave file in job xaada
lastleave -n 3-- last 3 leave files
more `lastleave` -- display most recent leave file
lastleave -ts -T -b48-- show code benchmarks for 1/2 hour timestep
Edits the SUBMIT script of a processed UMUI job, changing
This section is almost entirely redundant due to the subsequent release of later versions of xconv, but is kept here at the end of the page just in case it is of any remaining use (unlikely).
NOTE: the following files are for xconv 1.05. A better version, 1.90, is available from Jeff Cole, but is currently (Jan '04) pre-release so is not on the web. Please contact jeff AT met.rdg.ac.uk
These static executables for linux: xconv, convsh are build with this patch, which causes the "unlimited" flag to be added to the time dimension when writing NetCDF files. This permits the use of time-domain processing (e.g. concatenation), using the NCO utilities (see description and source).
Usage: as per unmodified versions.
(NB despite being static executables, requires tcl/tk version 4.3 installed.)
Last edited: 1 February 2007