Add support for setting more options for submitting jobs to the cluster

[Nicklas Nordborg]

When submitting jobs to the cluster we need to specify some options regarding the resources that are required, etc. We currently have support for setting number of cores, priority and partition. The cluster is about to be updated and we may have to also specify memory requirements and maybe other things.

We should at least implement support for adding options to reggie-config.xml that are included in the job submission.

It should be possible to manually change defaults via the submission wizards. This doesn't have to be a general functionality that is implemented now, but it should be possible to add it if needed later.

[Nicklas Nordborg]

In 7346:

References #1522: Add support for setting more options for submitting jobs to the cluster

Introduced <submit> and <submit-debug> tags in the reggie-config.xml file. The tags can be used to set options for the queue system. The options must be specified one per line with '=' as a separtor between key and value. Empty lines and lines starting with # are ignored (white-space is removed from the beginning and end of each line first).

The <parallel-environment> should no longer be used. The number of CPUs should be specified with --cpus-per-task=N in the <submit> tag instead. Note that while there was a default value for <parallel-environment> settings, there is no default for <submit> settings.

The configuration has been updated for ASCAT jobs so far.

[Nicklas Nordborg]

In 7348:

References #1522: Add support for setting more options for submitting jobs to the cluster

Replaced all <parallel-environment> tags in reggie-config.xml with <submit> tags.

[Nicklas Nordborg]

In 7349:

References #1522: Add support for setting more options for submitting jobs to the cluster

Most job creator implementations now use the <submit> tags instead of <parallel-environment> except the demux and FASTQ importer since they are a bit different.

[Nicklas Nordborg]

In 7350:

References #1522: Add support for setting more options for submitting jobs to the cluster

The FASTQ importer now uses <submit-import> instead of hard-coded settings.

[Nicklas Nordborg]

In 7351:

References #1522: Add support for setting more options for submitting jobs to the cluster

Fixed also for the demux jobs. The possibility to set a different number of slots in the manual start demux wizard has been removed.

[Nicklas Nordborg]

In 7353:

References #1522: Add support for setting more options for submitting jobs to the cluster

We need version 1.11 of the job scheduler for supporting options that are flags.

[Nicklas Nordborg]

In 7358:

References #1522: Add support for setting more options for submitting jobs to the cluster

Implemented functionality for manually overriding configured job submission options. It has been implemented on the server side for ASCAT, but there is no support yet in the wizard.

[Nicklas Nordborg]

In 7359:

References #1522: Add support for setting more options for submitting jobs to the cluster

Implemented functionality for manually overriding configured job submission options on the server side for all other job creator implementations.

[Nicklas Nordborg]

In 7360:

References #1522: Add support for setting more options for submitting jobs to the cluster

Implemented functionality for manually overriding job options in the ASCAT wizard.

[Nicklas Nordborg]

In 7361:

References #1522: Add support for setting more options for submitting jobs to the cluster

Implemented functionality for manually overriding job options in some more wizards.

[Nicklas Nordborg]

In 7362:

References #1522: Add support for setting more options for submitting jobs to the cluster

Fixed an issue with settings when a ParameterSet was specified for the selected software item. This can affect which options that are used from the configuration file.

[Nicklas Nordborg]

In 7363:

References #1522: Add support for setting more options for submitting jobs to the cluster

Fixed for the DNA Bwa-mem alignment wizard.

[Nicklas Nordborg]

In 7364:

References #1522: Add support for setting more options for submitting jobs to the cluster

Manually overriding the options should work now for all other wizards.