Package net.sf.basedb.opengrid.config

Class JobConfig

java.lang.Object
net.sf.basedb.opengrid.config.AbstractLockable<JobConfig>
net.sf.basedb.opengrid.config.JobConfig
public class JobConfig extends AbstractLockable<JobConfig>
Configuration settings related to a job definition that is about to be run on an Open Grid or Slurm cluster. The information becomes readonly when a JobDefinition instance has been created. A single configuration instance can be used with multiple jobs.

Note that options for the Open Grid Engine and Slurm can be very different and it is best if the client uses either setQsubOption(String, String) or setSbatchOption(String, String) depending on the type of the cluster.

Some options can be automatically converted between the two systems. This will only happen if options for one system has been set, but no options for the other. It is also possible to force conversion by directly calling convertQsubToSbatchOptions() or convertSbatchToQsubOptions().

Since:
1.0
Author:
nicklas

  • Field Details

    • DEFAULT

      public static final JobConfig DEFAULT
      A (locked) configuration instance with default settings and no batch configuration.

    • priority

      private Integer priority

    • nice

      private Integer nice

    • qsubOptions

      private Map<String,String> qsubOptions

    • sbatchOptions

      private Map<String,String> sbatchOptions

    • autoConvertOptions

      private boolean autoConvertOptions

    • batchConfig

      private BatchConfig batchConfig

  • Constructor Details

    • JobConfig

      public JobConfig()
      Create a new job configuration instance with default settings.

    • JobConfig

      public JobConfig(boolean autoConvertOptions)
      Create a new job configuration instance.
      Parameters:
      autoConvertOptions - If TRUE, options for qsub or sbatch are automatically converted
      Since:
      1.4

    • JobConfig

      private JobConfig(boolean autoConvertOptions, BatchConfig batchConfig)

  • Method Details

    • openGridPriorityToBasePriority

      public static int openGridPriorityToBasePriority(Integer priority)
      Convert an Open Grid job priority value to a BASE job priority value. 1024 -- 1 → 1 -- 4 0 or missing → 5 -1 -- -1023 → 6 -- 10

    • openGridPriorityToSlurmNice

      public static Integer openGridPriorityToSlurmNice(Integer priority)
      Convert an Open Grid job priority value to a 'nice' value used in Slurm. The main difference is that the scale is reversed. Eg. a positive priority value mean a negative nice value. Note that in Slurm the range for 'nice' is between +/- 2147483645, but in order to keep things simple we only reverse the sign. Null values are returned as null.
      Since:
      1.4

    • slurmNiceToOpenGridPriority

      public static Integer slurmNiceToOpenGridPriority(Integer nice)
      Convert a Slurm nice value to Open Grid priority. To keep things simple we only reverse the sign. If the resulting priority is outside the allowed range (-1023..+1024), the value is automatically changed to the nearest limit.
      Since:
      1.4
      See Also:

    • basePriorityToOpenGridPriority

      public static int basePriorityToOpenGridPriority(Integer priority)
      Convert a BASE job priority value to an Open Grid job priority value. 1-4 → 1024 -- 1 5 or missing → 0 6-10 → -1 -- -1023

    • setBatchConfig

      public void setBatchConfig(BatchConfig batchConfig)
      Set the batch configuration to use with this job configuration.
      Since:
      1.5

    • getBatchConfig

      public BatchConfig getBatchConfig()

    • setFailImmediately

      @Deprecated public void setFailImmediately(boolean failImmediately)
      Deprecated.
      In 1.5, "set -e' is no longer included in the generated scripts automatically
      Set this option if the job script should exit as soon as an error is encountered. This is on by default.

    • getFailImmediately

      @Deprecated public boolean getFailImmediately()
      Deprecated.

    • setCreatePrivateFiles

      @Deprecated public void setCreatePrivateFiles(boolean privateFiles)
      Deprecated.
      In 1.5, "umask" is no longer included in the generated scripts
      Set this option to automatically include a "umask" command that makes all files created by the job script private (eg, umask u=rwx,g=,o=) This is on by default.

    • getCreatePrivateFiles

      @Deprecated public boolean getCreatePrivateFiles()
      Deprecated.

    • setPriority

      public void setPriority(Integer priority)
      Set the priority of the job. Valid range is from -1023 (low priority) to +1024 (high priority). Priorities above 0 require special privileges. The default priority is unspecified (which typically means 0 on the Open Grid Cluster but this may depend on the user that is logged in). Also updates the 'nice' value.

    • getPriority

      public Integer getPriority()

    • getSlurmNice

      public Integer getSlurmNice()
      Get the 'nice' value (used in Slurm) that corresponds to the priority value used in Open Grid.
      Since:
      1.4
      See Also:

    • setSlurmNice

      public void setSlurmNice(Integer nice)
      Set the priority of the job using a Slurm 'nice' value. Valid range is from -2147483645 (high priority) to +2147483645 (low priority). Values below typically requires special privileges. The defaul 'nice' value is unspecified (which typically means 0 but this may depend on other options). Also updates the 'priority' value.
      Since:
      1.4

    • getBASEPriority

      public int getBASEPriority()
      Shortcut for calling openGridPriorityToBasePriority(Integer).

    • setQsubOption

      public void setQsubOption(String option, String value)
      Set an option for the 'qsub' command. See http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html for a lot more information. The options will go into the script file using the #$ prefix: #$ -option value Use an empty string as value for flag-like options: #$ -option It is not recommended to override options that are automatically set when submitting the job. This is currently: -terse, -S, -N, -wd, -o, -e and -p.
      Parameters:
      option - The name of the option (with or without the hyphen)
      value - The value of the option (null will remove it)

    • getQsubOption

      public String getQsubOption(String option)
      Get the value of a qsub option.
      Parameters:
      option - The name of the option (with or without the hyphen)

    • getQsubOptions

      public Map<String,String> getQsubOptions()
      Get all options that has been specified for qsub so far.
      Returns:
      A read-only map with the qsub options

    • appendQsubOptionsToScript

      public void appendQsubOptionsToScript(StringBuilder script, Set<String> ignore)
      Add all qsub options in this configuration to the script. Each option will create a line: #$ -<option> <value>

    • appendQsubOptionsToScript

      public void appendQsubOptionsToScript(ScriptBuilder script, Set<String> ignore)
      Add all qsub options in this configuration to the script. Each option will create a line: #$ -<option> <value>
      Since:
      1.5

    • setSbatchOption

      public void setSbatchOption(String option, String value)
      Set an option for the 'sbatch' command (Slurm). See https://slurm.schedmd.com/sbatch.html for a lot more information. The options will go into the script file using the #SBATCH prefix: #SBATCH --option=value Use an empty string as value for flag-like options: #SBATCH --option It is not recommended to override options that are automatically set when submitting the job. This is currently: (--parsable, --job-name, -J, --chdir, -D, --output, -o, --error, -e).
      Parameters:
      option - The name of the option (with or without the hyphen)
      value - The value of the option (null will remove it)
      Since:
      1.4

    • getSbatchOption

      public String getSbatchOption(String option)
      Get the value of a sbatch option.
      Parameters:
      option - The name of the option (with or without the hyphen)
      Since:
      1.4

    • getSbatchOptions

      public Map<String,String> getSbatchOptions()
      Get all options that has been specified for sbatch so far.
      Returns:
      A read-only map with the sbatch options
      Since:
      1.4

    • appendSbatchOptionsToScript

      public void appendSbatchOptionsToScript(StringBuilder script, Set<String> ignore)
      Add all sbatch options in this configuration to the script. Each option will create a line: #SBATCH --<option>=<value> or: #SBATCH -<option>=<value>

    • appendSbatchOptionsToScript

      public void appendSbatchOptionsToScript(ScriptBuilder script, Set<String> ignore)
      Add all sbatch options in this configuration to the script. Each option will create a line: #SBATCH --<option>=<value> or: #SBATCH -<option>=<value>

    • checkValid

      protected void checkValid(boolean forLock)
      The priority must be between -1023 and +1024 and 'nice' must be between -2147483645 and +2147483645. Other generic qsub or sbatch options are not checked.
      Overrides:
      checkValid in class AbstractLockable<JobConfig>
      Parameters:
      forLock - If TRUE, the instance will be locked after validation

    • autoConvertOptions

      private void autoConvertOptions()

    • convertOptionsTo

      public void convertOptionsTo(ClusterType type)
      Convert options to the variant used for the given cluster type. Note that the list of supported options that can be converted are very limited.
      Since:
      1.8
      See Also:

    • convertQsubToSbatchOptions

      public void convertQsubToSbatchOptions()
      Convert options set for 'qsub' (Open Grid) to options for 'sbatch' (Slurm). Supported options are: -pe smp X-Y: --nodes=1, --ntasks=1, --cpus-per-task=max(X, Y) -q NAME: --partition=NAME (since 1.8)
      Since:
      1.4

    • convertSbatchToQsubOptions

      public void convertSbatchToQsubOptions()
      Convert options set for 'sbatch' (Slurm) to options for 'qsub' (Open Grid). Supported options are: --cpus-per-task=N: -pe smp N -p, --partition=NAME: -q NAME (since 1.8)
      Since:
      1.4

    • getMaxNumberInString

      private String getMaxNumberInString(String s)
      Get the highest number that can be found in the given string. Use to extract the number of requested slots in, for example, -pe smp 8-16.