Changes between Version 9 and Version 10 of net.sf.basedb.opengrid/install


Ignore:
Timestamp:
Aug 21, 2020, 8:27:21 AM (4 years ago)
Author:
Nicklas Nordborg
Comment:

Updated documentation with Slurm information

Legend:

Unmodified
Added
Removed
Modified

  • net.sf.basedb.opengrid/install

    v9 v10  
    1 = Installing the Open Grid Scheduler package =
     1= Installing the Job Scheduler package =
    22
    33== Installation and updating ==
    44
    5  1. Download the latest `opengrid-x.y.tar.gz` file from the [wiki:net.sf.basedb.opengrid Open Grid Scheduler main page].
     5 1. Download the latest `opengrid-x.y.tar.gz` file from the [wiki:net.sf.basedb.opengrid Job scheduler main page].
    66 2. Unpack the downloaded file to a directory of your choice.
    77 3. Copy the `opengrid.jar` file to your BASE plug-ins directory. Look in your
    … …  
    1515 6. Log in to BASE as an administrator and go to the '''Administrate->Plug-ins & Extensions->Overview''' page.
    1616 7. Run the installation wizard and select `opengrid.jar` for installation.
    17  8. Go to '''Administrate->Services''' and check that the ''Open Grid Scheduler service'' is running.
     17 8. Go to '''Administrate->Services''' and check that the ''Job scheduler service'' is running.
    1818 
    1919== Configuration ==
    2020
    21 Configuration settings are stored in the `opengrid-config.xml` file which should be located in the BASE `WEB-INF/classes` directory. The file is an XML file with a top-level `<opengrid>` tag and then one or more `<cluster>` tags. Each `<cluster>` tag defines a unique user + Open Grid Cluster combination. The following attributes are defined for the `<cluster>` tag:
     21Configuration settings are stored in the `opengrid-config.xml` file which should be located in the BASE `WEB-INF/classes` directory. The file is an XML file with a top-level `<opengrid>` tag and then one or more `<cluster>` tags. Each `<cluster>` tag defines a unique user + cluster combination. The following attributes are defined for the `<cluster>` tag:
    2222
    2323|| '''Attribute''' || '''Required''' || '''Description''' ||
     24|| type || no || The type of cluster. Valid values are `opengrid` and `slurm`. If not specified `opengrid` is assumed. (Since 1.4) ||
    2425|| name || yes || A readable name that is intended to be used in interfaces with users. ||
    25 || address || yes || Network address or IP number to the master host of the Open Grid Cluster. ||
     26|| address || yes || Network address or IP number to the master host of the cluster. ||
    2627|| port || no || Port number that accepts SSH connections (default value is 22) ||
    2728|| fingerprint || yes || SSH fingerprint. Either the MD5 hash formatted as a 16 two-digit hexadecimal numbers separated with ':', or (since 1.1) the SHA-256 hash in Base64-enocding. ||
    28 || user || yes || Username to use when connecting to the Open Grid Cluster. ||
    29 || password || no || Password to use when connecting to the Open Grid Cluster. Optional since version 1.2, which added support for private key files. ||
     29|| user || yes || Username to use when connecting to the cluster. ||
     30|| password || no || Password to use when connecting to the cluster. Optional since version 1.2, which added support for private key files. ||
    3031
    3132Example:
    … …  
    3334#!xml
    3435<cluster
     36  type="opengrid"
    3537  name="Open Grid"
    3638  address="grid.example.com"
    … …  
    4244}}}
    4345
    44 Access to the Open Grid Cluster is via SSH and since version 1.2 we support both username+password and private key authentication. In the former case a username and password must be specified in the `<cluster>` tag. The latter case is enabled by including a sub-tag `<key-file>`. The value should be the full path to a file containing a SSH private key. The following attributes are defined for the `<key-file>` tag:
     46Access to the cluster is via SSH and since version 1.2 we support both username+password and private key authentication. In the former case a username and password must be specified in the `<cluster>` tag. The latter case is enabled by including a sub-tag `<key-file>`. The value should be the full path to a file containing a SSH private key. The following attributes are defined for the `<key-file>` tag:
    4547
    4648|| '''Attribute''' || '''Required''' || '''Description''' ||
    … …  
    5557
    5658
    57 You may add as many `<cluster>` tags as you like if you have more than one Open Grid Cluster or if you want to configure access for multiple users to the same cluster. The only restriction is that the combination of `user`, `address` and `port` must be unique. Internally, an ID for each definition is created by combining the three values. Note that the port number is always included even if it is not present in the configuration file. The example about will get an ID like `griduser@grid.example.com:22`. The ID is important since this is what other extensions have to use in order to find the correct Open Grid Cluster and to be able to connect to it and submit jobs.
     59You may add as many `<cluster>` tags as you like if you have more than one cluster or if you want to configure access for multiple users to the same cluster. The only restriction is that the combination of `user`, `address` and `port` must be unique. Internally, an ID for each definition is created by combining the three values. Note that the port number is always included even if it is not present in the configuration file. The example about will get an ID like `griduser@grid.example.com:22`. The ID is important since this is what other extensions have to use in order to find the correct cluster and to be able to connect to it and submit jobs.
    5860
    5961Inside each `<cluster>` tag there are also several sub-tags that need to be configured:
    6062
    6163|| '''Sub-tag''' || '''Required''' || '''Default value''' || '''Description''' ||
    62 || `<job-folder>` || yes || || The path to a folder on the Open Grid Cluster that BASE can use to send job scripts and data files to/from the cluster. This folder must be accessible from all nodes in the cluster. A unique subfolder is created for each job that is submitted to the cluster. Job scripts may access this subfolder using the `${WD}` variable. Files are NOT automatically deleted after the job has finished. ||
    63 || `<tmp-folder>` || no || ${TMPDIR} || The path to a directory for storing temporary working data. It is recommended that the path is to a local disk on each node. The default value is to use the folder assigned by the Open Grid Cluster. Job scripts may access this subfolder using the `${TMPDIR}` variable. This folder and all files within it is typically deleted once the job has finished. ||
     64|| `<job-folder>` || yes || || The path to a folder on the cluster that BASE can use to send job scripts and data files to/from the cluster. This folder must be accessible from all nodes in the cluster. A unique subfolder is created for each job that is submitted to the cluster. Job scripts may access this subfolder using the `${WD}` variable. Files are NOT automatically deleted after the job has finished. ||
     65|| `<tmp-folder>` || no || `${TMPDIR}` || The path to a directory for storing temporary working data. It is recommended that the path is to a local disk on each node. The default value is to use the folder assigned by the cluster. Job scripts may access this subfolder using the `${TMPDIR}` variable. This folder and all files within it is typically deleted once the job has finished. ||
    6466|| `<tmp-folder-debug>` || no ||  || Alternative temporary folder that is used when submitting jobs with the debug flag. This can for example be set to a location that is not deleted automatically. If no value is specified the regular temporary folder is used. ||
    65 || `<date-command>` || no || date +'%Y-%m-%d %T' || A command to run on the Open Grid Cluster to get the current date and time. This information is used for correcting the running time of jobs if the clocks are different on the BASE server and Open Grid Cluster. The command must return the date and time in YYYY-MM-DD hh:mm:ss format (for example: 2017-01-12 10:40:15) ||
    66 || `<host-info-command>` || no || uname -srmo || A command to run on the Open Grid Cluster to get information about the operating system. It is used only for informational purposes. ||
    67 || `<opengrid-info-command>` || no || qstat -help | head -n1 || A command to run on the Open Grid Cluster to get information about the Open Grid software. This is currently only used for informational purposes, but in the future this information may be used for feature-detection. ||
    68 || `<job-agent-id>` || no || || Links the Open Grid Cluster to a job agent that is defined in BASE via the external ID. When this value exists the job agent is used as a proxy for access permissions. BASE users need to have USE permission for the job agent in order to use this Open Grid Cluster. Note that the job agent is not used for anything else. Do not set a server and/or port. The job agent software should not be installed on the Open Grid Cluster. Open Grid Clusters that are not linked to a job agent proxy can be used by all users. ||
    69 || `<nodes>` || no || || A list with one or more `<node name="..." />` elements identifying individual nodes in the Open Grid Cluster. Individual nodes are not used by this extension, but may be required by other extensions for doing Open Grid related tasks that can't be scheduled as jobs (for example, parsing out data from result files that should be stored in BASE). The list of nodes that can be used for this is configured here simply as a service for other extensions. Typically, one or two nodes can be set aside for this and it is recommended that actions are quick and not too resource consuming. Extensions that require access to nodes should document this requirement. ||
     67|| `<date-command>` || no || `date +'%Y-%m-%d %T'` || A command to run on the cluster to get the current date and time. This information is used for correcting the running time of jobs if the clocks are different on the BASE server and the cluster. The command must return the date and time in `YYYY-MM-DD hh:mm:ss` format (for example: `2017-01-12 10:40:15`) ||
     68|| `<host-info-command>` || no || `uname -srmo` || A command to run on the cluster to get information about the operating system. It is used only for informational purposes. ||
     69|| `<opengrid-info-command>` || no || `qstat -help | head -n1` (!OpenGrid)[[BR]] `sinfo -V` (Slurm) || A command to run on the cluster to get information about the cluster software. This is currently only used for informational purposes, but in the future this information may be used for feature-detection. ||
     70|| `<job-agent-id>` || no || || Links the cluster to a job agent that is defined in BASE via the external ID. When this value exists the job agent is used as a proxy for access permissions. BASE users need to have USE permission for the job agent in order to use the cluster. Note that the job agent is not used for anything else. Do not set a server and/or port. The job agent software should not be installed on the cluster. Clusters that are not linked to a job agent proxy can be used by all users. ||
     71|| `<nodes>` || no || || A list with one or more `<node name="..." />` elements identifying individual nodes in the cluster. Individual nodes are not used by this extension, but may be required by other extensions for doing tasks that can't be scheduled as jobs (for example, parsing out data from result files that should be stored in BASE). The list of nodes that can be used for this is configured here simply as a service for other extensions. Typically, one or two nodes can be set aside for this and it is recommended that actions are quick and not too resource consuming. Extensions that require access to nodes should document this requirement. ||