Opened 15 months ago

Closed 15 months ago

Last modified 13 months ago

#1161 closed task (fixed)

Update picard to 2.20

Reported by: Nicklas Nordborg Owned by: Nicklas Nordborg
Priority: major Milestone: Reggie v4.23
Component: net.sf.basedb.reggie Keywords:
Cc:

Description (last modified by Nicklas Nordborg)

http://broadinstitute.github.io/picard/

I think we should manage without any custom modifications. See #1160.

We need to test the "Check data files" functionality and the demux scripts.

Change History (11)

comment:1 Changed 15 months ago by Nicklas Nordborg

Description: modified (diff)

comment:2 Changed 15 months ago by Nicklas Nordborg

In 5527:

References #1161: Update picard to 2.20

Changed the default configuration file to use picard 2.20.0.

Tested with "Check data files" and it works for successful runs if we just read the response from 'stderr' instead of from 'stdout'. If we force a failure by using an incorrect read string it is reported with an error icon, but the error message contains a lot of other output before the actual error (this is what we anticipated from #1160).

comment:3 Changed 15 months ago by Nicklas Nordborg

In 5528:

References #1161: Update picard to 2.20

Updated to the new command line syntax. For more information see https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)

This change also requires that -Dpicard.useLegacyParser=false is added to the command line when starting picard and a new wrapper (picard2) for this has been added to the pipeline scripts.

comment:4 Changed 15 months ago by Nicklas Nordborg

In 5529:

References #1161: Update picard to 2.20

Adding the picard2 wrapper that expects the command line to be in the new syntax. See https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)

comment:5 Changed 15 months ago by Nicklas Nordborg

In 5531:

References #1161: Update picard to 2.20

Implemented a method for parsing output from picard and extracting lines that could be related to an error (non-zero exit code). hopefully it will be able to present the actual error message to user near the top of the output while skippen other unrelated informational messages.

comment:6 Changed 15 months ago by Nicklas Nordborg

In 5534:

References #1161 and #1146. The MIPs demux is now running with picard 2.20. Changes that were needed:

  • Use new paramater syntax (including values in reggie-config.xml)
  • No TILE_LIMIT for ExtractIlluminaBarcodes? when debugging
  • Parsing the demultiplex_metrics.txt file required a differnt regular expression since there are extra columns in places
  • Wrapping picard output with 'stdwrap.sh' to make sure that messages go to stdout or stderr depending on exit code

comment:7 Changed 15 months ago by Nicklas Nordborg

In 5535:

References #1161: Update picard to 2.20

Removed all ADAPTERS_TO_CHECK parameters from the config since there is an warning in the output about this and the picard documentation says that the parameter has been deprectated and is not used: http://broadinstitute.github.io/picard/command-line-overview.html#IlluminaBasecallsToFastq

comment:8 Changed 15 months ago by Nicklas Nordborg

In 5536:

References #1161: Update picard to 2.20

Updated the RNA-seq demux to use picard 2. Same changes as in [5534].

comment:9 Changed 15 months ago by Nicklas Nordborg

In 5537:

References #1161: Update picard to 2.20

Updated MarkDuplicates step after Hisat/Tophat? alignment to use picard 2.

comment:10 Changed 15 months ago by Nicklas Nordborg

Resolution: fixed
Status: newclosed

comment:11 Changed 13 months ago by Nicklas Nordborg

In 5584:

References #1161: Update picard to 2.20

If we set the IGNORE_UNEXPECTED_BARCODES parameter when running IlluminaBaseCallsToFastq we no longer have to specify entries for 'N' or other unexpected barcods in the multiplex file. This can make the demux quicker.

Note that the file needed by ExtractIlluminaBarcodes still need all possible barcode entries if we want to be able to detect mistakes.

Note: See TracTickets for help on using tickets.