Opened 2 years ago

Last modified 2 years ago

#1374 closed task

Implement support for running jobs in a Singularity container — at Initial Version

Reported by: Nicklas Nordborg Owned by: Nicklas Nordborg
Priority: major Milestone: Reggie v4.38
Component: net.sf.basedb.reggie Keywords:
Cc:

Description

In a Singularity container we can install all programs that are needed for a specific analysis step without having to worry about exactly which versions that are installed on the host system. In the simplest form a script can be started with:

singularity exec /path/to/container.sif job.sh

but we will at least also need to specify --bind options so that the container can access project-archive, run-archive, and other directories where we store data.

We also need to split the job.sh script, since we need the options for qsub. For example:

start_job.sh is submitted to the queue system and contains the options is responsible for starting the job.sh script:

# --- start of qsub options ---
#$ -S /bin/sh
#$ -terse
....
# --- end of qsub options ---
singularity exec /path/to/container.sif job.sh

Also, since the container typically know and have control over the paths were the programs are installed we should not have to specify them in the configuration file.

Note that we still want to support the non-container way of running jobs.

Change History (0)

Note: See TracTickets for help on using tickets.