Even more functionality in the Flagged alignment wizard

[Nicklas Nordborg]

See #1050.

• [Done] Implement possibility to flag and add/modify comments when scheduling re-extractions. Since it takes about a week or more to get the result it is good to be able keep some information about why the re-extraction was made in the first place.

• [Done] Introduce a DoNotUse flag that should be set on the top-most item that is considered incorrect. The top-most item is typically a Specimen, Lysate, RNA, PreNormalizedRNA or Library. The flag should then be copied downstreams to all child items. A DoNotUseComment that goes along with the flag is probably also a good idea.

• [Done] Alignments that are flagged as DoNotUse should be removed from the Flagged alignments list and disabled for future genotype checks. (Update) Disabling is the default, but there should also be an option to keep them available for genotype checks. Remaining items in the list should be checked again and items that no longer get any genotype warning should also be removed.

• A lot of other wizards need to be updated to check the DoNotUse flag. For example:
  • When manually selecting RNA for library preparation a warning should be displayed.
  • Demuxing should skip flagged libraries. The rules for generating warnings, etc. need to be updated.
  • and more...

[Nicklas Nordborg]

Copied to ticket description.

[Nicklas Nordborg]

(In [4970]) References #1054: Even more functionality in the Flagged alignment wizard

Extended the "Schedule re-processing" functionality with the ability to add or modify comments (QC_GENOTYPE_COMMENT) and flag more alignments. The functionality is available when a filter has been selected.

[Nicklas Nordborg]

(In [4972]) References #1054: Even more functionality in the Flagged alignment wizard

Added two new annotation types: DoNotUse and DoNotUseComment.

They should be set on any item (from Specimen all the way down to RawBioassay) to indicate that it should not be used in the future for some reason (as indicated by the comment). The DoNotUse is a string enumeration with possible values: Specimen, Lysate, RNA, Library. The value should reflect the top-most item where there is a problem. For example, if an RNA is contaminated, then DoNotUse=RNA for the RNA and all items downstreams.

[Nicklas Nordborg]

(In [4973]) References #1054: Even more functionality in the Flagged alignment wizard

Adding 'debug'-level messages from servlets that are displayed in the regular result list after the registration. Messages should be prefixed with [Debug].

[Nicklas Nordborg]

(In [4974]) References #1054: Even more functionality in the Flagged alignment wizard

Preparing the template HTML with support for the "resolve" mode by adding some more CSS rules and a "Register" button.

[Nicklas Nordborg]

(In [4975]) References #1054: Even more functionality in the Flagged alignment wizard

Implemented functionality for marking items as "Ok to use" or "Do not use". The selection can be done on Specimen, Lysate, RNA or Library level and will affect all downstream child items.

The functionality is only available if a filter is active. If the filter is a Qiacube or Libplate filter it is a requirement that an option is selected for all flagged items. If the "Do not use" option is used a comment is required.

[Nicklas Nordborg]

(In [4976]) References #1054: Even more functionality in the Flagged alignment wizard

Added an indicator for re-processing that is enabled if there is no other alignment for the same specimen. Alignments from the same RNA (if a Qiacube filter is active) or library (if HET filter or libplate filter is active) are not counted as a separate alignment.

When the High HET filter is active there is also an indicator for DoNotUse that is active if there is another alignement without an "error" condition.

[Nicklas Nordborg]

(In [4977]) References #1054: Even more functionality in the Flagged alignment wizard

Messages are registration now reports number of specimen, lysates, rna and libraries instead of total number. Also fixed so that messages are correct for "one" and "multiple" items.

[Nicklas Nordborg]

(In [4978]) References #1054: Even more functionality in the Flagged alignment wizard

Sorting items in order from Specimen->Lysate->RNA->Library to make results predictable in the case that two or more items from the same parent-child chain are selected.

[Nicklas Nordborg]

(In [4979]) References #1054: Even more functionality in the Flagged alignment wizard

When a RNA is selected for DoNotUse we should also find the corresponding DNA and FlowThrough. This is done by looking for the other items that have the same Lysate parent and same Qiacube date and run number.

[Nicklas Nordborg]

(In [4980]) References #1054: Even more functionality in the Flagged alignment wizard

Display existing DoNotUse information for alignments. If a re-process option is selected from an item marked as DoNotUse a warning icon is displayed. In the "Resolve" mode alignments marked as DoNotUse can't be modified.

[Nicklas Nordborg]

(In [4981]) References #1054: Even more functionality in the Flagged alignment wizard

Added an option to keep an alignment that was marked as DoNotUse available for future genotype checks (eg. QC_GENOTYPE_STATUS=Checked instead of QC_GENOTYPE_STATUS=Disabled).

For example, if we have a QiaCube shift on positions 8-12, a verified RNA on position 6 but no information about the RNA at position 7 (and no remaining specimen) we can mark it as DoNotUse but keep it enabled for genotype checks in case we get a second sample from the same patient in the future.

Note that there is no possibility yet to change the DoNotUse to OkToUse if the second sample confirms that the original sample was correct.

[Nicklas Nordborg]

(In [4982]) References #1054: Even more functionality in the Flagged alignment wizard

Added PreNormalizedRNA as an option for the DoNotUse annotation. The options is of course only available if the alignment has a pre-normalied RNA item in the parent chain.

A minor complication is present when filtering on a QiaCube. It is possible that alignments from both a pre-normalized and a regular RNA exists and that there is a mixup that only affects the pre-normalized RNA. But the "Resolve" options are grouped by QiaCube positions so there is only one set of options for the two alignments. If the PreNormalizedRNA option is selected, only the pre-normalized RNA is marked as DoNotUse and we rely on the auto-unflag option to fix the regular RNA as if the Ok checkbox had been selected.

[Nicklas Nordborg]

(In [4983]) References #1054: Even more functionality in the Flagged alignment wizard

The "Create mRNA plate wizard" has been updated to display a warning when selecting a RNA marked with DoNotUse.

[Nicklas Nordborg]

(In [4984]) References #1054: Even more functionality in the Flagged alignment wizard

The "Create manual pool wizard" has been updated to display a warning when selecting a library marked with DoNotUse.

[Nicklas Nordborg]

(In [4985]) References #1054: Even more functionality in the Flagged alignment wizard

The "Start demux and merge" wizard now checks for libraries marked with DoNotUse. It will display information about them and provide an option to either exclude or include them in the demux. The default is to exclude such libraries.

The actual demux process will still demux all data, but it will collect the data to a virtual library "IGNORED". The main reason for doing this is to avoid that the checks that are made efter the demux to generate warnings for high levels of unmapped reads.

[Nicklas Nordborg]

(In [4989]) References #1054: Even more functionality in the Flagged alignment wizard

The "Start Hisat alignment" wizard now checks for items marked with DoNotUse. It will display a warning that must be confirmed before using such items.

[Nicklas Nordborg]

(In [4990]) References #1054: Even more functionality in the Flagged alignment wizard

The StringTie, Tophat and Cufflinks wizards has been updated to display a warning when selecting a library marked with DoNotUse.

Also changes in the Hisat wizard to avoid duplicates in the selection list.

[Nicklas Nordborg]

(In [4991]) References #1054: Even more functionality in the Flagged alignment wizard

The Gene and Pilot report wizards has been updated to display a warning when selecting a library marked with DoNotUse.

[Nicklas Nordborg]

(In [4992]) References #1054: Even more functionality in the Flagged alignment wizard

The Release exporter plug-in has been updated to require a verification from the user if the selected list contains one or more raw bioassays marked with DoNotUse.

[Nicklas Nordborg]

(In [4993]) References #1054: Even more functionality in the Flagged alignment wizard

The YellowLabel tracking and reporting wizard has been updated to display a warning for items marked with DoNotUse and requires verification if publishing any of those items.

[Nicklas Nordborg]

(In [4994]) References #1054: Even more functionality in the Flagged alignment wizard

The Define a new sample outtake wizard has been updated to display a warning and require verification if the selected source list contains a RNA or DNA item marked with DoNotUse.

[Nicklas Nordborg]

(In [4995]) References #1054: Even more functionality in the Flagged alignment wizard

The Flagged RNA wizard has been updated to display a warning for items marked with DoNotUse.

[Nicklas Nordborg]

(In [4997]) References #1054: Even more functionality in the Flagged alignment wizard

Added an icon that indicates if an alignment has been scheduled for re-processing to make that information available in all 3 modes and not just in the "Add comments and schedule re-processing" mode.

[Nicklas Nordborg]

(In [4998]) References #1054: Even more functionality in the Flagged alignment wizard

Fixes an issue with displaying current re-processing information.

[Nicklas Nordborg]

(In [5001]) References #1054: Even more functionality in the Flagged alignment wizard

Added functionality for keeping track of changes that are made in the forms. Changed elements are highlighted.

A warning message is displayed if there are changes that would be lost due to changing filter settings or navigating to a different page.

[Nicklas Nordborg]

(In [5003]) References #1054: Even more functionality in the Flagged alignment wizard

If we mark a 'Specimen' with DoNotUse we also need to find other sample child items.

[Nicklas Nordborg]

(In [5005]) References #1054: Even more functionality in the Flagged alignment wizard

Fixed an issue with an unexpected error ('e2' must contain at least one element) when no alignments matches the filter (eg. no High HET).

[Nicklas Nordborg]

(In [5009]) References #1054: Even more functionality in the Flagged alignment wizard

Updated Case summry with information about the DoNotUse annotation if present.

[Nicklas Nordborg]

(In [5010]) References #1054: Even more functionality in the Flagged alignment wizard

Removing left-over code that was never used in the final version.

[Nicklas Nordborg]

(In [5014]) References #1054: Even more functionality in the Flagged alignment wizard

The "Genotype quality control" wizard now checks if existing alignments have been marked as DoNotUse. A GOOD MATCH between two alignments is displayed as a warning (DONOTUSE MATCH) with a flag recommendation.

[Nicklas Nordborg]

(In [5017]) References #1054: Even more functionality in the Flagged alignment wizard

It is now possible to register partial libplates or QiaCubes as resolved. The reason is that there may be more than one mistake that is present on a single batch.

For example, we found a QiaCube run with a single "High HET" item and a swap betweeen 2 other items. The "High HET" is probably a contaminated library and the RNA was already scheduled for re-processing. It should not be solved until the new data is available, but we wanted to solve the swap immediately (marking the two RNA items as DoNotUse).

The possibility to bypass the strict registration rules becomes enabled after trying (and failing) to register an incomplete form. Note that it is only the DoNotUse/OkToUse options that can be bypassed. Comments are still required for all items marked with DoNotUse.

[Nicklas Nordborg]

(In [5018]) References #1054: Even more functionality in the Flagged alignment wizard

Added "Library" option to the "DoNotUse" list also when a QiaCube filter is active and there is only a single alignment for the specific position.

[Nicklas Nordborg]

(In [5024]) References #1054: Even more functionality in the Flagged alignment wizard

It is now possible to repeat a DoNotUse registration. The example use case is as before (see [5017]) that a High HET library is found flagged as DoNotUse. At a later date a swap is discovered that also involves the contaminated library. Before this change it was not possible to re-register a DoNotUse for the RNA since it was disabled due to the library already being flaggd.

This change makes it possible to re-register a DoNotUse flag for all items. Existing flags are not overwritten but it is possible to update the comment by making a second registration. For example, registering a DoNotUse on the RNA will not update anything on items marked with DoNotUse=Library. Repeating the registration one more time with DoNotUse=Library will update the comment on those items.

[Nicklas Nordborg]

(In [5025]) References #1054: Even more functionality in the Flagged alignment wizard

Fixes a javascript error in demux start when there is no items marked with DoNotUse.

[Nicklas Nordborg]

(In [5063]) References #1054: Even more functionality in the Flagged alignment wizard

Removed debug output.