Opened 5 years ago

Closed 5 years ago

#1075 closed task (fixed)

Change the release exporter to work with list of patients

Reported by: Nicklas Nordborg Owned by: Nicklas Nordborg
Priority: major Milestone: Reggie v4.21
Component: net.sf.basedb.reggie Keywords:
Cc:

Description

The current release exporter plug-in works with a list of raw bioassays and is then working it's way up to patient. The new implementation should start with a list of patients and work it's way down. It should also include other item types, such as Blood, DNA, FlowThrough, etc. Items need to be filtered based on conditions that has not yet been fully determined. For example, items marked with the DoNotUse annotation should not be included.

The old export of tab-separated files will not work with the new approach and should be removed.

Change History (30)

comment:1 by Nicklas Nordborg, 5 years ago

(In [5089]) References #1075: Change the release exporter to work with list of patients

The release exporter plug-in now accepts an item list with patient/biosource items. The rest of the code has been modified so that the exporter code can be compiled and run without crashing. It is currently only exporting the patient data to a .json file.

comment:2 by Nicklas Nordborg, 5 years ago

(In [5090]) References #1075: Change the release exporter to work with list of patients

Removed code for generating tab-separated files with cohort data.

comment:3 by Nicklas Nordborg, 5 years ago

(In [5091]) References #1075: Change the release exporter to work with list of patients

Changed CohortWriter API so that implementations can return multiple items.

Updated PatientWriter implementation to only accept actual patient items. Added implementations for No, NotAsked, RetroNo and Retract items.

comment:4 by Nicklas Nordborg, 5 years ago

(In [5096]) References #1075: Change the release exporter to work with list of patients

Changed CaseWriter implementation so that it loads all cases related to a patient. The current query filter is Consent=Yes and DoNotUse=null. Also updated the IncaWriter so that it now also works.

comment:5 by Nicklas Nordborg, 5 years ago

(In [5097]) References #1075: Change the release exporter to work with list of patients

Updated the SpecimenWriter implementation. It will currently load specimen data that have DoNotUse=null. Histology and scoring information is not loaded.

comment:6 by Nicklas Nordborg, 5 years ago

(In [5098]) References #1075: Change the release exporter to work with list of patients

Refactored the query generation/execution part of the exporter. The new QueryManager class is responsible for creating and handling all queries that are used for loading release data at all leveles.

comment:7 by Nicklas Nordborg, 5 years ago

(In [5099]) References #1075: Change the release exporter to work with list of patients

Implemented the LysateWriter implementation. The only filter so far is DoNotUse=null.

comment:8 by Nicklas Nordborg, 5 years ago

(In [5100]) References #1075: Change the release exporter to work with list of patients

Updated the RnaWriter implementation. It will currently load RNA items that have DoNotUse=null and at least one child RNAQC item with a RIN or RQS value.

comment:9 by Nicklas Nordborg, 5 years ago

(In [5101]) References #1075: Change the release exporter to work with list of patients

Updated the LibraryWriter implementation. It will currently load Library items that have DoNotUse=null and a name not ending with '.dil'.

comment:10 by Nicklas Nordborg, 5 years ago

(In [5102]) References #1075: Change the release exporter to work with list of patients

Updated the MergedWriter implementation. It will currently load MergedSequences items that have DoNotUse=null and AnalysisResult=Successful. It will currently only output the simple annotation values. Information from the demux, sequencing and pools are not yet included.

comment:11 by Nicklas Nordborg, 5 years ago

(In [5103]) References #1075: Change the release exporter to work with list of patients

Updated the RawBioAssayWriter implementation with two new subclasses StringTieWriter and CufflinksWriter. Common annotations from the alignment and masked parent are exporter, but not files yet.

comment:12 by Nicklas Nordborg, 5 years ago

(In [5104]) References #1075: Change the release exporter to work with list of patients

Updated the MergedWriter implementation to include information from the demux, sequencing runs and pooled libraries.

comment:13 by Nicklas Nordborg, 5 years ago

(In [5105]) References #1075: Change the release exporter to work with list of patients

Cleaning up unused code.

comment:14 by Nicklas Nordborg, 5 years ago

(In [5106]) References #1075: Change the release exporter to work with list of patients

Added support for exporting data files. Completed the implementation for merged and aligned bioassays and cufflinks and stringtie raw data.

comment:15 by Nicklas Nordborg, 5 years ago

(In [5107]) References #1075: Change the release exporter to work with list of patients

Implemented support for exporting Histology and the GoodStain sample. Since we are no longer moving from the bottom up there is no lysate that can give us a partition date, so the implementation will currently select the hostology item with the latest partition date.

The release export should now be complete in the sense that all types of items can be fully exported with the same information as before. It still remains to implement more detailed filtering rules.

comment:16 by Nicklas Nordborg, 5 years ago

Status: newassigned

comment:17 by Nicklas Nordborg, 5 years ago

(In [5124]) References #1075: Change the release exporter to work with list of patients

The LibColumn annotations should be exported as an Integer. It was accidentally changed to a String in [5090] by using a formatter.

comment:18 by Nicklas Nordborg, 5 years ago

(In [5130]) References #1075: Change the release exporter to work with list of patients

Removed old and unused code.

comment:19 by Nicklas Nordborg, 5 years ago

(In [5131]) References #1075: Change the release exporter to work with list of patients

Added writer implementations for DNA and FlowThrough. They both have a single filter: DoNotUse=null. On the FlowThrough QiacubeBatchNo is the only annotations. DNA also has the same NanoDrop annotations as RNA but doens't have RNAQC.

Version 0, edited 5 years ago by Nicklas Nordborg (next)

comment:20 by Nicklas Nordborg, 5 years ago

(In [5132]) References #1075: Change the release exporter to work with list of patients

Changed the DNA and FlowThrough writers to work with specific dao objects instead of generic extracts. This should be safer. (Belongs to ticket #1081)

Last edited 5 years ago by Nicklas Nordborg (previous) (diff)

comment:21 by Nicklas Nordborg, 5 years ago

(In [5133]) References #1075: Change the release exporter to work with list of patients

Changed the DNA and FlowThrough writers to work with specific dao objects instead of generic extracts. This should be safer. (Belongs to ticket #1081)

Last edited 5 years ago by Nicklas Nordborg (previous) (diff)

comment:22 by Nicklas Nordborg, 5 years ago

(In [5139]) References #1075: Change the release exporter to work with list of patients

Cleanup.

comment:23 by Nicklas Nordborg, 5 years ago

(In [5164]) References #1075: Change the release exporter to work with list of patients

Added filter creationDate <> null for Lysate, Histology and Library to prevent the inclusion of items that has only been pre-registered and not yet performed in the lab. Other items are either not registered until they have been cretate (RNA, DNA, etc) or have other annotations that can be used instead (for example AnalsysisResult=Successful for derived bioassay items.

comment:24 by Nicklas Nordborg, 5 years ago

(In [5197]) References #1075: Change the release exporter to work with list of patients

Added filter for library items to only include libraries on a libplate with PlateProcessingResult=Successful.

comment:25 by Nicklas Nordborg, 5 years ago

(In [5198]) References #1075: Change the release exporter to work with list of patients

Added filter for library items that exclude libraries that have at least one alignment on the "Flagged alignments" list.

comment:26 by Nicklas Nordborg, 5 years ago

(In [5199]) References #1075: Change the release exporter to work with list of patients

Removed the AnalysisResult=Successful filter when loading aligned sequences since this value is not set by the legacy pipeline (Tophat/Cufflinks).

comment:27 by Nicklas Nordborg, 5 years ago

(In [5200]) References #1075: Change the release exporter to work with list of patients

Simplified progress reporting.

comment:28 by Nicklas Nordborg, 5 years ago

(In [5231]) References #1075: Change the release exporter to work with list of patients

Fixes a NullPointerException if there is no Histology item.

comment:29 by Nicklas Nordborg, 5 years ago

(In [5234]) References #1075: Change the release exporter to work with list of patients

Replaces a NullPointerException with a more descriptive ItemNotFoundException if a parent derived bioassay with the expected name can't be found. This should never happen unless something has been inocrrectly changed (eg. a manual rename och relink).

comment:30 by Nicklas Nordborg, 5 years ago

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.