Create import wizard for transfer of external samples

[Nicklas Nordborg]

See also #1478.

We will get information about transfered samples from an Excel file. The only example we have so far have the following columns:

• PADRemiss
• SCAN-B nummer
• TYP
• Ankomstdatum
• Clarity ID
• Utlämning datum till CMD
• Förvaring

The most important column is the Clarity ID column since we will use it to find an existing Specimen item. This will be the most common case and it should result in placing the found Specimen (and the downstreams Lyste, RNA, DNA and FlowThrough) items in the next free location in the storage boxes. I think that we should also verify the SCAN-B nummer so that it matches what we have.

If the Clarity ID doesn't match it will be more complicated. We have not yet decided how this should be handled, but items will somehow be created and placed on the next free locations in the storage boxes.

The column 'Förvaring' is the name of the transport boxes (there is 1 box for each type). For each transport box we will create 5 item lists and add the Specimen, Lysate, RNA, DNA and FlowThrough items to them. The item lists will be used to generate lab protocols for how to move the tubes to the new storage boxes. The intention is to work with one transport box at a time.

[Nicklas Nordborg]

In 7139:

References #1480: Create import wizard for transfer of external samples

Started to implement an import wizard. The "Aliquot importer" for the MIPs pipeline has been used as a model for the functionality. The wizard will currently parse the file (CSV or Excel) and after minimal validation display the information.

There is not yet any code for mapping to existing items or creating new items, etc.

[Nicklas Nordborg]

In 7140:

References #1480: Create import wizard for transfer of external samples

The wizard now assign new storage box locations to each entry in the file. There must be enough empty wells in existing storage boxes or an error is generated.

[Nicklas Nordborg]

In 7141:

References #1480: Create import wizard for transfer of external samples

Changes in the error handling. There can now be multiple errors for a single entry.

Started to implement the first two mapping variants.

1. A match is made to an existing Specimen via the Clarity ID.
2. A match is made to an existing NoSpecimen via the SCAN-B ID and the NoSpecimen must have EXTERNAL_SPECIMEN_EXISTS=Yes

[Nicklas Nordborg]

In 7142:

References #1480: Create import wizard for transfer of external samples

Started to implement creation of work item lists and updating the storage locations of existing matched items and some preparations for handling the case were new items need to be created.

[Nicklas Nordborg]

In 7143:

References #1480: Create import wizard for transfer of external samples

Implemented the rest of the mapping methods to find an existing item to link the import to.

Added check for consent.

Re-arranged the table a bit.

[Nicklas Nordborg]

In 7144:

References #1480: Create import wizard for transfer of external samples

Implemented support for creating all missing items.

Added messages for debugging.

Added a new activity type that can have two counters.

[Nicklas Nordborg]

In 7145:

References #1480: Create import wizard for transfer of external samples

Some fixes to ensure annotations are set as we want them.

ExternalSpecimenExists=Yes on all items (will be reset when they have been registered as moved).

Laterality is copied from Case for new Specimen items.

It should be BiopsyType, NOT SpecimenType to specify core biopsies.

[Nicklas Nordborg]

In 7146:

References #1480: Create import wizard for transfer of external samples

Added ExternalRef and OtherPathNote to the Case summary for Specimen and NoSpecimen items.

[Nicklas Nordborg]

In 7147:

References #1480: Create import wizard for transfer of external samples

Added links to the Case summary for most items in the table.

Added possibility to filter that table depending on mapping type, error/warning status, etc.

[Nicklas Nordborg]

In 7148:

References #1480: Create import wizard for transfer of external samples

Changed order and severity of some checks.

[Nicklas Nordborg]

In 7149:

References #1480: Create import wizard for transfer of external samples

Imporvements to the filtering options.

[Nicklas Nordborg]

In 7153:

References #1480: Create import wizard for transfer of external samples

Added links to the Case summary also for the Clarity ID and PAD values. Normally, they should lead to the same item, but there are sometimes mismatches.

[Nicklas Nordborg]

In 7154:

References #1480: Create import wizard for transfer of external samples

If the PAD has a suffix we remove it before searching.

[Nicklas Nordborg]

In 7155:

References #1480: Create import wizard for transfer of external samples

Ignore PAD differences due to upper/lower-case letters.

[Nicklas Nordborg]

In 7156:

References #1480: Create import wizard for transfer of external samples

Allow entries with a duplicate Clarity ID to map since that will make filtering error/mapping types more clear.

[Nicklas Nordborg]

In 7157:

References #1480: Create import wizard for transfer of external samples

If we can't map to anything and the SCAN-B ID is missing it is an error.

If a duplicated mapping is found reset the 'canImport' and 'action' of the first mapped item.

[Nicklas Nordborg]

In 7158:

References #1480: Create import wizard for transfer of external samples

Added support for selecting an alternate action when a NoSpecimen item has been matched by SCAN-B ID and the PAD doesn't match. In this case it may be that a new Specimen item should be created instead of converting the existing NoSpecimen.

When there is a PAD mismatch, try to find an item with the PAD from the file. This information may help to diagnose the mismatch.

[Nicklas Nordborg]

In 7159:

References #1480: Create import wizard for transfer of external samples

Include 'Other path. note' information for items with error and/or warnings.

The "Comment" field is filled with the text from the "Description" for mapped Specimen or NoSpecimen.

[Nicklas Nordborg]

In 7160:

References #1480: Create import wizard for transfer of external samples

Added a link for downloading the select file.

[Nicklas Nordborg]

In 7161:

References #1480: Create import wizard for transfer of external samples

Added support for storing comments that have been entered in the wizard.

The alternate action possibility for NoSpecimen item has been implemented so that a new Specimen is created instead.

[Nicklas Nordborg]

In 7162:

References #1480: Create import wizard for transfer of external samples

Added a progress report to the validation and import.

Modified some of the messages after validation and import.

[Nicklas Nordborg]

In 7163:

References #1480: Create import wizard for transfer of external samples

Set "AutoProcessing=Disable" on new items so that other wizards can avoid picking them up after registration. Some of the existing wizards may need to updated to check for this.

[Nicklas Nordborg]

In 7164:

References #1480: Create import wizard for transfer of external samples

Added "Flagged External Specimen" item list that is used to store Specimen that we have received but that we don't have any processing or sequencing information for.

[Nicklas Nordborg]

In 7165:

References #1480: Create import wizard for transfer of external samples

Added a counter for the number of items marked with ExternalSpecimenExists.

[Nicklas Nordborg]

In 7166:

References #1480: Create import wizard for transfer of external samples

If there are warnings append them to the ImportWarnings annotation.

[Nicklas Nordborg]

In 7167:

References #1480: Create import wizard for transfer of external samples

The counter for external specimen should not include those that already have an assigned storage location. The ExternalSpecimenExists annotation is not reset until the actual transfer has been registered.

[Nicklas Nordborg]

In 7168:

References #1480: Create import wizard for transfer of external samples

Added TransportMessages annotation. This should be used to store information on the work item lists about samples that should not transfered to new storage locations. For example, if it matches a case with Consent=No/NotAsked.

[Nicklas Nordborg]

In 7170:

References #1480: Create import wizard for transfer of external samples

Lysates with AutoProcessing annotation should not be selected for extraction.

[Nicklas Nordborg]

In 7174:

References #1480: Create import wizard for transfer of external samples

Item lists are annotated with ExteranlRef=transport box name

[Nicklas Nordborg]

In 7176:

References #1480: Create import wizard for transfer of external samples

Improvements to the table that is split into columns coming from the import file and columns with data from lorry. Added an icon that shows a tooltip with the raw line text.

[Nicklas Nordborg]

In 7177:

References #1480: Create import wizard for transfer of external samples

Cases where SCAN-B id is missing and there is no match against the Clarity ID is handled as if the sample is not a SCAN-B sample. Import is allowed with a message to leave the tube in the box.

[Nicklas Nordborg]

In 7185:

References #1480: Create import wizard for transfer of external samples

Save a link between the item lists representing transport boxes and the file that was for the import.

[Nicklas Nordborg]

In 7190:

References #1480: Create import wizard for transfer of external samples

Moved the wizard to the Sample processing section.

[Nicklas Nordborg]

In 7191:

References #1480: Create import wizard for transfer of external samples

Added support for saving comments per set of transport boxes. The comment is stored as the description on the item lists that are created to represent the transport boxes.

[Nicklas Nordborg]

In 7193:

References #1480: Create import wizard for transfer of external samples

Allow duplicates in the import file if the SCAN-B ID also match. In this case, only a warning is generated and the duplicate is ignored. If the SCAN-B ID doesn't match an error is generated for both entries.

[Nicklas Nordborg]

In 7194:

References #1480: Create import wizard for transfer of external samples

Fixes some display issues when selecting a different file after the first one.