Opened 3 years ago
Last modified 3 years ago
#1325 closed defect
Calculating the average fragment size needs fixing due to Hisat bug — at Initial Version
| Reported by: | Nicklas Nordborg | Owned by: | Nicklas Nordborg |
|---|---|---|---|
| Priority: | major | Milestone: | Reggie v4.32 |
| Component: | net.sf.basedb.reggie | Keywords: | |
| Cc: |
Description
It seems that Hisat 2.1 has a bug that in some cases output a negative TLEN for both read pairs. This affects the singlecolumnaverager.awk pipeline script that calculates the average fragment size. The script only consider reads with TLEN>0 and TLEN<500 and will miss both reads in pairs that are affected by the Hisat bug.
It seems like the bug is only present for fragments that are shorter than the read length. It can be seen a sharp edge in the distribution of TLEN values (>0). See the attached image.
The solution is to let the singlecolumnaverager.awk consider both positive and negative TLEN values and then use the absolute value in the calculations. The final average and standard deviation should be the same, but the number of fragments need to be divided by 2.
