Implement a wizard for creating panel-of-normals

[Nicklas Nordborg]

This is part of the WGS variant calling pipeline (see #1536).

We should create a set of wizards that can build a panel-of-normals VCF. The first wizard should take normal WGS alignments and run Mutect2 variant calling on them. The resulting VCF files should be stored in a (semi-temporary) location that is different from the regular project-archive.

When we have all normal VCF results a second wizard can be used to combine them into the final panel-of-normals. When this has been completed the VCF files created by the first step can be deleted.

[Nicklas Nordborg]

In 7387:

References #1537: Implement a wizard for creating panel-of-normals

Started to create a wizard for starting variant calling of normal samples that should be part of the panel-of-normals.

[Nicklas Nordborg]

In 7388:

References #1537: Implement a wizard for creating panel-of-normals

Started to implement the job script. The Base Recalibration step should work.

[Nicklas Nordborg]

In 7389:

References #1537: Implement a wizard for creating panel-of-normals

Implemented the variant calling step with Mutect2. The resulting VCF files are stored per chromosome in a special directory that is separate from the regular project archive. We may have to merge the VCF files before building the panel-or-normals or it might be possible to build one panel per chromosome and merge after that.

It should also be possible to run multiple Mutect2 processes in parallel to speed up things. A single process uses about 4 full CPUs, but it will not improve after that.

[Nicklas Nordborg]

In 7390:

References #1537: Implement a wizard for creating panel-of-normals

Implemented parallel execution of Mutect2 jobs. One job per chromosome is started in the background, but we also limit the number of concurrent jobs so that the we don't use more than the max number of assigned threads.

Also noticed that a *.stats text file is created by Mutect2. It contains the number of callable bases, and we extract that value and save it to our own stats.out file.

BAM files are now copied to the local node before starting the analysis.

[Nicklas Nordborg]

In 7391:

References #1537: Implement a wizard for creating panel-of-normals

Added chrY to the list of chromosomes.

Some minor changes to progress reporting.

Merge the VCF files into a single VCF file.

[Nicklas Nordborg]

In 7392:

References #1537: Implement a wizard for creating panel-of-normals

Started to implement the confirmation wizard. It should list and display some information about the variant call, but the registration is not working yet.

[Nicklas Nordborg]

In 7393:

References #1537: Implement a wizard for creating panel-of-normals

The merge step was missing a program parameter.

[Nicklas Nordborg]

In 7394:

References #1537: Implement a wizard for creating panel-of-normals

It should now be possible to confirm the panel-of-normal variant calling.

[Nicklas Nordborg]

In 7395:

References #1537: Implement a wizard for creating panel-of-normals

A wizard for building the final panel-of-normal VCF has been implemented. The script will leave the final VCF file in the working directory of the job. It have to be moved manually to a more suitable location in the directory with other references.

[Nicklas Nordborg]

In 7396:

References #1537: Implement a wizard for creating panel-of-normals

Added possibility to select a software for the panel-of-normal variant calling.

[Nicklas Nordborg]

In 7397:

References #1537: Implement a wizard for creating panel-of-normals

Changed the CallableBases to a long annotation type. Existing annotations need to be updated manually during the installation.

1. Rename the existing CallableBases annotation to CallableBasesOld before installation
2. Install the new Reggie version
3. Export the ID, CallableBasesOld and CallableBases for all existing items that have a value already.
4. Import the annotations again, but switch the columns.
5. After the import it should be possible to remove the CallableBasesOld annotation.