Environmental Sciences
De e-Ciencia
Tabla de contenidos |
Analysis of Water Quality in Reservoirs
Using the DYRESM/CAEDYM model as a simple approach (one-dimensional) for the analysis of water quality in catchments.
DYRESM/CAEDYM is a product developed by the Centre for Water Research from the University of Western Australia. Software must be downloaded from their web site, and free user registration is required.[1]
After downloading the software the installation process is straightforward as the model comes presented in a tar-ball including all the necessary binary files for execution (createDYref, createDYsim, extractDYinfo, dycd, ncgen and ncdump). A post-simulation analysis software (Modeller) is also available, and its use is highly recommended for graphical analysis and representation. Several data packs are also available for testing purposes.
Input Data
The model requires seven files as input data:
- An Stage file containing morphometry information about the location (eg: test.stg)
- A Meteorological data file with the meteorological conditions during the simulation span (eg: test.met)
- An Inflow file with variable measures relative to the water inflows to the reservoir (eg: test.inf)
- A Withdrawals file containing measures of the output water flux in the reservoir (eg: test.wdr)
- A file containing an initial profile of the variables under analysis in the water reservoir (eg: test.pro)
- A Simulation parameter file (eg: test.par)
- A Simulation configuration file (eg: test.cfg)
For further information on file structure or data format please refer to CWR website. [2]
Simulation
Simulation execution process is linear and comprehends four steps:
- #createDYref test.stg test.met test.inf test.wdr DYref.nc
- #createDYsim test.pro test.par DYsim.nc
- #extractDYinfo DYref.nc DYsim.nc test.cfg
- #dycd
Step 1 creates a DYRESM reference file (DYref.nc)
Step 2 creates a DYRESM unconfigured simulation file (DYsim.nc)
It is mandatory that the simulation file receive this name, as dycd, the proper simulator, expect this filename.
Step 3 creates a DYRESM configured simulation file (Updates DYsim.nc)
Step 4 executes the simulation and updates DYsim.nc so it contains the output data.
It is convenient to redirect dycd output to a file as it writes a large amount of useful information to the standard output.
#dycd > dylog.log
Grid submitting files
The only important detail when submitting DYRESM jobs to the Grid is that the set of binaries required have a size exceeding 20MBytes so it is
required to store them in one Storage Element and recall them using the LCG tools prior to each simulation.
Taking this into account, and supposing that the binaries are stored in /grid/ienvmod/icoteril/DYRESM/binaries.tar:
dyresm.jdl
# Mandatory attributes
Executable = "dyresm.sh";
StdOutput = "dyresm.out";
StdError = "dyresm.err";
# Environment variables
Environment = {"LFC_HOST=lfc01.lip.pt"};
# I/O files to be staged from/to the User Interface
InputSandbox = {"dyresm.sh","inputdata.tar"};
OutputSandbox = {"dyresm.out","dy.log","dyresm.err","DYsim.nc"};
dyresm.sh
#!/bin/sh # Debug info echo \| Execution start: `date` \| Host: `hostname` \| User: `whoami` \| Path: `pwd` \| lcg-cp -v --vo ienvmod lfn:/grid/ienvmod/icoteril/DYRESM/binaries.tar file:///tmp/myfile.tar cp /tmp/myfile.tar binaries.tar rm /tmp/myfile.tar tar xvf DY.tar chmod u+x createDYref chmod u+x createDYsim chmod u+x extractDYinfo chmod u+x dycd tar xvf inputdata.tar ./createDYref test.stg test.met test.inf test.wdr DYref.nc ./createDYsim test.pro test.par DYsim.nc ./extractDYinfo DYref.nc DYsim.nc test.cfg ./dycd # Debug info echo \| Execution end: `date` \|
Output Processing
After retrieving the simulation output from the worker node we can use Modeller to visualize the results, following this simple steps:
- Interpolate Grid - Sim File menu -
- Visualise the selected results -Visualise menu -
Contact: Ignacio Coterillo Coz, email: coterili@ifca.unican.es
Distributed Hydrological Modeling of Watersheds
Distributed Hydrologic Balance using the Soil Moisture Distribution and Routing Model (SMDR [3]) on top of open-source Geographic Information System (GIS) GRASS. [4]
GRASS stands for Geographic Resources Analysis Support System and it is a Free Software/Open Source GIS allowing data management, image processing, graphics production, spatial modelling and visualization of data. It is released under GNU General Public License (GPL).[5]
SMDR available version is developed in the form of perl and shell scripts using GRASS commands for operations.
Being a distributed model (this means, a grid of points distributed over the area of study), the number of points involved in the different computations increases squarely so the possibility of running some, if not all, of this computations in a parallel way is very interesting looking forward to time reduction during the simulation process. We aim to do this by paralizing the main calculus functions of GRASS using Message Passing Interface libraries.
Installing GRASS
The installation process is described for a normal user (not root) with no special privileges
As we intend to test different modifications in GRASS code, which means several re-compile processes and test executions, we will use a local-referred installation creating a proper environment for the aplication.
- We need to download the GRASS source code, (we use Version 6.2.1) [6] and check the availability of required libraries in our system [7].
- For the Requirements our system doesn't meet we need to install them locally, creating a folder like, i.e:
/home/icoteril/usr
where we are going to replicate the /usr structure for GRASS.
- When the requirements are met, install GRASS in this same folder:
/home/icoteril/usr/ - Create a Tar ball containing the full local
usrfolder and sub-folders and store it in a Grid Storage Element (SE) using the LCG tools
lcg-cr --vo ienvmod -d i2gse01.ifca.es file:/home/icoteril/SMDR_Scripts.tar -l lfn:/grid/ienvmod/icoteril/usr.tar
- Also you will ned a valid location to run GRASS. Once you have created it locally just proceed like in the previous step and create a tar-ball from it, uploading it to a SE.
lcg-cr --vo ienvmod -d i2gse01.ifca.es file:/home/icoteril/SMDR_Scripts.tar -l lfn:/grid/ienvmod/icoteril/location.tar
Grid submitting files
In addition to the usr folder structure we need to send to the worker node the database we intend to use as base for our computations. This means sending and additional tar file containing the database folder structure, whose size is variable but commonly will be big enough (several Mbytes) as to need a Grid Storage Element too.
grass_job.jdl
# Mandatory atributes
Executable = "grass.sh";
StdOutput = "grass.out";
StdError = "grass.err";
# I/O files to be staged from/to the User Interface
InputSandbox = {"grass.sh"};
OutputSandbox = {"grass.out","grass.err"}
grass_job.sh
This code executes the GRASS g.version command after defining the needed environment variables.
#!/bin/bash echo -e " User: `whoami` \n Host Info: `uname -a` \n Path: `pwd` \n Date: `date`" lcg-cp --vo ienvmod lfn:/grid/ienvmod/icoteril/usr.tar file:/tmp/usr.tar lcg-cp --vo ienvmod lfn:/grid/ienvmod/icoteril/maps.tar file:/tmp/maps.tar HOMEDIR=`pwd` mv /tmp/usr.tar $HOMEDIR mv /tmp/maps.tar $HOMEDIR tar -xvvf usr.tar tar -xvvf maps.tar chmod u+x $HOMEDIR/usr/local/grass-6.2.1/bin/g.version ### Grass Environment Variables GISBASE=$HOMEDIR/usr/local/grass-6.2.1 GISDBASE=$HOMEDIR GISRC=$HOMEDIR/.grassrc6 ### Parameters LOCATION_NAME=cuenca1 MAPSET=PERMANENT LOCATION=$GISDBASE/$LOCATION_NAME/$MAPSET # Generating GISRC file cat << EOF > $GISRC GISDBASE: $GISDBASE LOCATION_NAME: $LOCATION_NAME MAPSET: $MAPSET PAINTER: ppm MAPLP: stuff.ppm EOF # Additional definitions ETC=$GISBASE/etc PATH=$GISBASE/bin:$GISBASE/scripts:$GISBASE/garden/bin:$PATH export GISBASE GISDBASE ETC PATH GISRC export LD_LIBRARY_PATH=$HOMEDIR/usr/local/grass-6.2.1/lib:$LD_LIBRARY_PATH g.version exit 0
Contact: Ignacio Coterillo Coz, email: coterili@ifca.unican.es
