Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#1161 closed task (fixed)

Update picard to 2.20

Reported by: Nicklas Nordborg Owned by: Nicklas Nordborg
Priority: major Milestone: Reggie v4.23
Component: net.sf.basedb.reggie Keywords:
Cc:

Description (last modified by Nicklas Nordborg)

http://broadinstitute.github.io/picard/

I think we should manage without any custom modifications. See #1160.

We need to test the "Check data files" functionality and the demux scripts.

Change History (11)

comment:1 by Nicklas Nordborg, 5 years ago

Description: modified (diff)

comment:2 by Nicklas Nordborg, 5 years ago

In 5527:

References #1161: Update picard to 2.20

Changed the default configuration file to use picard 2.20.0.

Tested with "Check data files" and it works for successful runs if we just read the response from 'stderr' instead of from 'stdout'. If we force a failure by using an incorrect read string it is reported with an error icon, but the error message contains a lot of other output before the actual error (this is what we anticipated from #1160).

comment:3 by Nicklas Nordborg, 5 years ago

In 5528:

References #1161: Update picard to 2.20

Updated to the new command line syntax. For more information see https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)

This change also requires that -Dpicard.useLegacyParser=false is added to the command line when starting picard and a new wrapper (picard2) for this has been added to the pipeline scripts.

comment:4 by Nicklas Nordborg, 5 years ago

In 5529:

References #1161: Update picard to 2.20

Adding the picard2 wrapper that expects the command line to be in the new syntax. See https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)

comment:5 by Nicklas Nordborg, 5 years ago

In 5531:

References #1161: Update picard to 2.20

Implemented a method for parsing output from picard and extracting lines that could be related to an error (non-zero exit code). hopefully it will be able to present the actual error message to user near the top of the output while skippen other unrelated informational messages.

comment:6 by Nicklas Nordborg, 5 years ago

In 5534:

References #1161 and #1146. The MIPs demux is now running with picard 2.20. Changes that were needed:

  • Use new paramater syntax (including values in reggie-config.xml)
  • No TILE_LIMIT for ExtractIlluminaBarcodes when debugging
  • Parsing the demultiplex_metrics.txt file required a differnt regular expression since there are extra columns in places
  • Wrapping picard output with 'stdwrap.sh' to make sure that messages go to stdout or stderr depending on exit code

comment:7 by Nicklas Nordborg, 5 years ago

In 5535:

References #1161: Update picard to 2.20

Removed all ADAPTERS_TO_CHECK parameters from the config since there is an warning in the output about this and the picard documentation says that the parameter has been deprectated and is not used: http://broadinstitute.github.io/picard/command-line-overview.html#IlluminaBasecallsToFastq

comment:8 by Nicklas Nordborg, 5 years ago

In 5536:

References #1161: Update picard to 2.20

Updated the RNA-seq demux to use picard 2. Same changes as in [5534].

comment:9 by Nicklas Nordborg, 5 years ago

In 5537:

References #1161: Update picard to 2.20

Updated MarkDuplicates step after Hisat/Tophat alignment to use picard 2.

comment:10 by Nicklas Nordborg, 5 years ago

Resolution: fixed
Status: newclosed

comment:11 by Nicklas Nordborg, 5 years ago

In 5584:

References #1161: Update picard to 2.20

If we set the IGNORE_UNEXPECTED_BARCODES parameter when running IlluminaBaseCallsToFastq we no longer have to specify entries for 'N' or other unexpected barcods in the multiplex file. This can make the demux quicker.

Note that the file needed by ExtractIlluminaBarcodes still need all possible barcode entries if we want to be able to detect mistakes.

Note: See TracTickets for help on using tickets.