Opened 2 years ago
Last modified 2 years ago
#1374 closed task
Implement support for running jobs in a Singularity container — at Initial Version
Reported by: | Nicklas Nordborg | Owned by: | Nicklas Nordborg |
---|---|---|---|
Priority: | major | Milestone: | Reggie v4.38 |
Component: | net.sf.basedb.reggie | Keywords: | |
Cc: |
Description
In a Singularity container we can install all programs that are needed for a specific analysis step without having to worry about exactly which versions that are installed on the host system. In the simplest form a script can be started with:
singularity exec /path/to/container.sif job.sh
but we will at least also need to specify --bind
options so that the container can access project-archive
, run-archive
, and other directories where we store data.
We also need to split the job.sh
script, since we need the options for qsub
. For example:
start_job.sh is submitted to the queue system and contains the options is responsible for starting the job.sh
script:
# --- start of qsub options --- #$ -S /bin/sh #$ -terse .... # --- end of qsub options --- singularity exec /path/to/container.sif job.sh
Also, since the container typically know and have control over the paths were the programs are installed we should not have to specify them in the configuration file.
Note that we still want to support the non-container way of running jobs.