Viewing
To use the computer facilities effectively the whole job chain is not carried out in just one script, but divided into several different scripts to perform the main sub-tasks of the cclm chain.
subchain - main script
Purpose
This main script calls the job scripts of the directory scripts. subchain sends the following sub-tasks to the compute server.The script reads settings from the file job_settings.

Contents
The subchain script contains several sections (called actions) which are executed dependent on the subchain [action] call. If you just call subchain without any argument you will get a list of valid actions. The actions start and clean are called interactively. The other actions are called internally by the subchain script, only in case of a restart in case of an error they may be called interactively.
The subchain script first checks whether the file date.log exists. During the simulation this file holds the current date of the CCLM runs. When calling subchain the first time this file does not exist and will be created as well as the directory structure of the job.
Available actions:
  • start
    This is the very first call to start the simulation. The file date.log and the directory structure of the job will be created.
  • prep [YYYYMM]
    prepares some environment variables and submits the prep.job.sh for the month taken from the file date.log or optionally by the argument YYYYMM .
  • int2lm [YYYYMM]
    prepares some environment variables and submits the INT2LM job int2lm.job.sh for the month taken from the file date.log or optionally by the argument YYYYMM .
  • cclm [noprep]
    prepares some environment variables and submits the CCLM job cclm.job.sh for the month taken from the file date.log. It also calls subchain prep ${NEXT DATE} which starts a pre-processing job for the next month, if this is not suppressed by calling this action with the argument noprep.
  • arch [YYYYMM]
    prepares some environment variables and submits the archiving job arch.job.sh for the month taken from the file date.log or optionally by the argument YYYYMM .
  • post [YYYYMM]
    prepares some environment variables and submits the post-processing job post.job.sh for the month taken from the file date.log or optionally by the argument YYYYMM .
  • clean
    deletes all files and directories created for this experiment. You can use this actions if you want to start the simulation from scratch.
  • create
    this actions creates just the directory structure for your experiment. It is only needed in rare cases when the directory structure is corrupted.

prep.job.sh - pre-pre-processing script
Purpose
All necessary pre-pre-processing is performed by this script, e.g. copying boundary data from the archive and transforming them into int2lm compatible netCDF format. The script can be changed to perform copying from tape or via ftp or scp.

Contents
The script includes the job_settings file.
If ITYPE INT2LM == 1
Prepares the coarse grid initial or boundary data (e.g. from reanalysis, GCM or a coarser CCLM) for use in the INT2LM job for a specific month. This script has to be adjusted by the user depending on how and where the coarse grid data are stored. The caf or cas data files must be copied into the directory ${SCRATCHDIR}/${EXPID}/output/prep/YYYY_MM. At the end of this job an INT2LM job is initiated by subchain int2lm ${CURRENT DATE}.

 If the coarse grid data are internally compressed netCDF files, perform a decompression in this template by calling nccopy -k 2 infile outfile. This is because compressed input data slow down the reading by INT2LM significantly!



If ITYPE INT2LM /= 1
Assumes that the laf and lbf files of an INT2LM run already exist. These data are copied to ${SCRATCHDIR}/${EXPID}/output/int2lm/YYYY_MM. At the end of this job an CCLM job is initiated by subchain cclm.
int2lm.job.sh - pre-processing script
Purpose
This script prepares input data for the subsequent COSMO-CLM simulation: the external parameters for the surface and the global boundary meteorological fields at the domains boundaries are interpolated/re-projected to the model grid. Those parameters which are not explicitly set here are set to their default values during the simulation. For more information on the name list please visit the Namelist-Tool web page.

Contents
The script includes the job_settings file. Itcontains the namelist for INT2LM. After creating the input namelist files the INT2LM executable is called with the system specific MPI call. At the end of this job an CCLM job is initiated by subchain cclm. The user may change the namelist settings and the MPI call.
cclm.job.sh - cclm run script
Purpose
This script contains name list parameters for the model grid, as well as model options for dynamics, physics, I/O, and diagnostics, etc. Those parameters which are not explicitly set here are set to their default values during the simulation. For more information on the name list variables look please visit the Namelist-Tool web page. The standard GRIBOUT namelist settings are such that the groups out01 and out02 contain the quantities needed for a further downscaling with CCLM.

Contents
The script includes the job_settings file. It contains the namelist for CCLM. After creating the input namelist files CCLM executable is called with the system specific MPI call. At the end of this job an archive job is initiated by subchain arch if the total current month has been run by the CCLM. If the CCLM is run in shorter time length than a month and the end of the month has not been reached yet, the next CCLM chunk is initiated by subchain cclm. The user may change the namelist settings and the MPI call.
arch.job.sh - archive script
Purpose
Archiving of the results is performed in this script, e.g. compressing and combining output directories into tar files. In the standard script archiving is performed on hard disk. However, the script can be changed to perform archiving on tape or via ftp or scp. In the standard output the directories out01 and out01 are combined into a separate tar file for further nesting of CCLM with higher resolution.

Contents
The script includes the job_settings file. The results from the CCLM simulation for the current month are tarred and copied to the archive. At the end of this job a post-processing job is initiated by subchain post. An optional check with SAMOA can be performed,
post.job.sh - post-processing script
Purpose
Any post-processing needed is performed here. Archiving of postprocessing results is also done here. Post-processing may take a lot of time, depending on its complexity. This may slow down the whole chain. Therefore it is run in parallel to the rest of the chain.

Contents
The script includes the job_settings and functions.sh files. A first post-processing is performed with this template. The following post-processing is performed by default. However, the user can adjust the script to his needs. The following steps are performed:
  • the grid boxes that will be cut off from the results are taken from the environment variable NBOUNDCUT (to be set in the file job_settings)
  • a file with cut off boundary is calculated from the lffd${YDATE_START}c.nc file and stored as ${EXPID}_c.nc in the work/post directory.
  • time series functions are defined. For p- and z-levels extra functions are defined.
  • the time series functions are then called for user selected quantities. The resulting time series are noted under ${WORKDIR}/${EXPID}/post/YYYY_MM.
  • depending on the setting of the environment variable ITYPE_COMPRESS_POST (to be set in the file job_settings) a compression is applied.

 The file job_settings needs to be adapted by inserting the correct paths and several general model parameters.

The file job_settings includes all user specific settings for the subchain job, e.g. the experiment id, paths for input and output, start and and date, namelist settings which occur both in int2lm and cclm, and more. Please check the file job_settings for further information.
Start the chain simulation by
./subchain start

After the successful simulation output archived data are stored under cclm-sp/chain/arch/$EXPID and post-processed time series are stored under cclm-sp/chain/work/$EXPID/post.

 Before running a simulation on machines other than the DKRZ HLRE ”Mistral” and with other batch scripts change the sbatch commands in the top-level script subchain, and the batch settings and exe call in ll the scripts in the scripts directory.


Every experiment you create start by
./subchain start

If the job chain crashes due to any reason you can start the chain after you fixed the problem with any of the scripts by
./subchain scriptname [YYYYMM]
(scriptnames are: prep, int2lm, cclm, arch, post). If you do not provide the date YYYYMM, the date is taken from the file date.log. The cclm cript always takes the date from the date.log file, however, you can call it with the option noprep, in case the input data are already calculated by prep and int2lm.
There are two additional commands:
subchain create just creates the directory tree without starting the chain. This can be of help if one or more directories where deleted by mistake.
subchain clean deletes all added files but keeps changes made in the templates. This can be of help when you performed several changes in the template files and the simulation crashes. You may correct the templates files, perform a subchain clean followed by a subchain start.
 Viewing