Demux fails with OutOfMemoryError

[Nicklas Nordborg]

Most likely it is related to #1613 [7854] since this updated to picard version used for demuxing. In the log file there is a warning about using a deprecated parameter. The settings for the Small RNA demux is using the new parameter with a different value.

Here is part of the error log:

java.lang.RuntimeException: WARNING  2024-11-05 07:53:55  IlluminaBasecallsToFastq  Setting deprecated parameter `MAX_READS_IN_RAM_PER_TILE` use ` MAX_RECORDS_IN_RAM` instead
07:53:55.571 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/conda/share/picard-slim-3.2.0-0/picard.jar!/com/intel/gkl/native/libgkl_compression.so
[64.252s][warning][gc,alloc] pool-2-thread-1: Retried waiting for GCLocker too often allocating 256 words
[182.499s][warning][gc,alloc] pool-2-thread-4: Retried waiting for GCLocker too often allocating 256 words
[496.116s][warning][gc,alloc] pool-2-thread-4: Retried waiting for GCLocker too often allocating 954483 words
[1243.565s][warning][gc,alloc] pool-2-thread-3: Retried waiting for GCLocker too often allocating 1005333 words
ERROR  2024-11-05 08:20:24  ThreadPoolExecutorWithExceptions  A thread failed:
java.lang.OutOfMemoryError: Java heap space
  at picard.illumina.parser.BarcodeParser$BarcodeDataIterator.next(BarcodeParser.java:73)
  at picard.illumina.parser.BarcodeParser$BarcodeDataIterator.next(BarcodeParser.java:58)
  at picard.illumina.parser.PerTileParser.next(PerTileParser.java:111)
  at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:123)
  at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
  at picard.illumina.SortedBasecallsConverter$TileProcessor.run(SortedBasecallsConverter.java:192)
  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  at java.base/java.lang.Thread.run(Thread.java:840)
...

Hopefully it will help to change -MAX_READS_IN_RAM_PER_TILE 5000000 to -MAX_RECORDS_IN_RAM 1000000.

[Nicklas Nordborg]

Milestone renamed

[Nicklas Nordborg]

In 7923:

Fixes #1622: Demux fails with OutOfMemoryError