Introduction

In the previous section you submitted the job "hostname.jdl" to gilda. If you now look at the source you will see the following:

Line Number	Code
1	Type = "Job";
2	JobType = "Normal";
3	Executable = "/bin/hostname";
4	StdOutput = "hostname.out";
5	StdError = "hostname.err";
6	OutputSandbox = {"hostname.err","hostname.out"};
7	Arguments = "-f";
8	RetryCount = 7;

As you can see the jdl file is basically a list of parameters and the associated values where the value can actually be a single value, a list of values or a complex expression. For the purposes of this tutorial all of these parameters (except Arguments) must be present in each jdl file and have appropriate values set. There are cases where not all of these parameters are needed, but this tutorial will not be looking at these cases.
To explain what is going on here we will look at each line in turn:

1. The value of Type must always be Job. This line exists in the jdl for historical reasons only.
2. The JobType specifies the style of the job. For all the exercises in this tutorial a job type of Normal will be used. Possible other values include Interactive and Checkpointable for jobs which require a more sophisticated level of control during their run.
3. The Executable parameter specifies the executable/command that will be run on the CE. There are two possibilities here. Firstly you can specify an executable that lies already on the remote CE. In this case the absolute path, possibly including environmental variables should be specified. The other possibility is to provide a local filename as the executable, which will be looked at in the second exercise.
4. The standard output (STDOUT) of the executable specified will be saved to the filename given as the value to the StdOutput parameter. Note this file will be local to the remote CE.
5. As for STDOUT but this time for standard error (STDERR).
6. The OutputSandbox parameter specifies a list of files that were generated by the executable and that you want to retrieve (the easiest way of viewing simple files).
7. Any arguments that are needed by the executable are specified in the Arguments parameter. Note that this must a single string value. Multiple arguments are passed as part of the string in the same way as they would on the command line (normally just space separated).
8. This final parameter (RetryCount) specifies how many times the job should be resubmitted in the case of a grid component failing (not the same as an error in the executable).

Exercise 1

Modify "hostname.jdl" so that instead of calling /bin/hostname it calls the script "start.sh" instead. This file (as with all files used in this tutorial) already exists in your account and looks like the following:

#!/bin/sh sleep 5 hostname -f

You will need to change the executable to /bin/sh and the argument to start.sh. You will also need to use the new parameter InputSandbox. The InputSandbox is a list of all the files that need to be sent (in EGEE terminology 'staged') to the CE. For this example you need to send start.sh . A typical usage of bash scripts in the grid is to set environmental variables the actual executable needs and to run a combination of different programs consecutively on the same CE. Once you have made these edits run this job and check your output to see if your job was successful. As stated previously a successful job will have an empty "hostaname.err" whilst "hostname.out" will contain the name of the CE the job ran on.

Whilst you are waiting for your job to return its output, open a new browser window (or just click the following link) at . Now click on the Monitoring button. You should now be seeing a window similar to the following image:

You are now looking at GridIce, a system for monitoring the state of the grid, in particular the usage of different parts of the grid. This version is just monitoring gilda. Explore gilda by clicking on the names of the different sites and then the names of the grid elements at each site.

Exercise 2

As well as bash scripts you can write your own programs in C. Modify your jdl file to run the executable "myhostname". The source for myhostname can be found in "myhostname.c" and is:

#include <stdio.h> #include <malloc.h> #define BUF_SIZE 1000 int main( int argc, char *argv[]) {     char *hostname;     hostname = (char *) malloc(BUF_SIZE);     gethostname(hostname,255);     printf("host is %s\n",hostname);     free(hostname);     return 1; }

Note that this time you do not need the Arguments parameter. After making the changes run this job and check for success (this should be done after every exercise).

Exercise 3

In exercise 2 we saw how a C program could be run as a grid job by sending the program as part of the input sandbox. In many cases however this is not a practical solution, for example the executable itself is very large and requires many files in many folders, or alternately you do not have access to the same compiler and environment as is used on the CE (as is the case in this tutorial). The solution then is to compile your program on the CE itself. The easiest way to do this is by using a bash script to run the compiler and then run the executable. Modify your jdl file so that the script "buildandrun.sh" is run with the parameter "myhostname". You will need to ensure that both "buildandrun.sh" and "myhostname.c" are staged to the CE. The script "buildandrun.sh" is simply:

#!/bin/sh gcc $1.c -o $1 ./$1

Exercise 4

In all of the previous examples you have been working with programs that had no dependencies on software other than the software installed on all WN's. It is normally not a realistic option to send all of this dependant software with the jdl, and in many cases the software installed already on some of the WN's. Similarly there has been no question raised as to what hardware the WN has (e.g. number of CPU's or amount of memory). The jdl file is able to handle these job requirements by the use of the GLUE Schema. All CE's advertise what software their WN's have installed and what their hardware/operating system is to a database on the Information Service. When deciding where to send a job the Resource Broker matches the requirements you have specified against all of the CE's in your VO to see which (if any) can run your job. Consider the following example (the file "demtools.jdl" in your account):

Type = "Job"; JobType = "Normal"; Executable = "/bin/sh"; StdOutput = "demtools.out"; StdError = "demtools.err"; InputSandbox = {"start_demtools.sh","mount_sainte_helens_WA.dem","grand_canyon_AZ.dem"}; OutputSandbox = {"demtools.out", "demtools.err", "mount_sainte_helens_WA.ppm", "mount_sainte_helens_WA.wrl","grand_canyon_AZ.ppm","grand_canyon_AZ.wrl"}; RetryCount = 7; Arguments = "start_demtools.sh"; Requirements = Member("DEMTOOLS-1.0",other.GlueHostApplicationSoftwareRunTimeEnvironment) || Member("DEMTOOLS",other.GlueHostApplicationSoftwareRunTimeEnvironment);

The final line in this file states that the job requires the CE to either support DEMTOOLS or DEMTOOLS-1.0. If you wish to run this example then when you have the output you should open in your browser either "mount_sainte_helens_WA.wrl" or "grand_canyon_AZ.wrl"

The Glue Schema supports many such job requirements. For this exercise add to your jdl the requirement

other.GlueCEUniqueID == "grid010.ct.infn.it:2119/jobmanager-lcgpbs-short";

When you submit the job and reach the point where you usually let the RB choose where to run the job, have a look at the other options in the drop down menu, you should not be surprised at what the other options are.

Previous Section

Next Section