Export information intended for INCA

[olle]

It should be possible to export information intended for INCA, a swedish IT-platform for management of databases for cancer patients regarding care and research.

The export should be based on the one for monthly operation list, with the following modifications:

1. Patients with blood samples should be included, even if no "Specimen" or "NoSpecimen" exist.
2. Up to two rows should be included for each patient, one for each laterality represented by a known sample (Specimen/NoSpecimen). If no Specimen or NoSpecimen exists for a patient, a Blood sample should be used.
3. A "Sample" column should show the sample sub-type (Specimen/NoSpecimen/BloodOnly). The sample sub-type shown for a patient should be the first existing with priority "Specimen" > "NoSpecimen" > "Blood".
4. Core biopsy items should be included.
5. Instead of operation date, the sampling date for the included sample should be exported.
6. A "PatientId" column should be included.
7. The default "Month/Year" time period should be "All".
8. Column headers "PersonalNr" and "OpDate" should be exchanged for "PersonalNo" and "Date", respectively.

[olle]

Ticket accepted.

[olle]

Ticket description updated with specification of the export format.

[olle]

Traceability note:

• The monthly operation list export was introduced in Ticket #385 (Export information needed by the tumor registry).
• The monthly operation list was last updated in Ticket #486 (Monthly operation list export should not have optional columns), where the optional columns "Specimen" and "PatientId" were excluded.

[olle]

Background info:

• INCA is an abbreviation of the the Swedish expression "Informationsnätverk för cancervården", loosely translated as "Information Network for Cancer Care".
• The INCA export file is intended to be used when requesting information from the INCA database, which contains diagnoses for cancer cases.

[olle]

Ticket description updated regarding the date used in the export format.

[olle]

Ticket description updated.

[olle]

Design discussion.

Since the export has many features in common with the monthly operation list export, the design will be based on the design of the latter. The monthly operation list export is managed by JSP script export_monthly_oplist.jsp in reggie/resources/personal/ and java servlet ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/. Choice of the optional columns is handled by boolean variables exportSubtype and exportPatientId, whose values (0 or 1) are transferred from the JSP script to the servlet when calling the latter with variable cmd set to "ExportMontlyOpList".

A new JSP script export_inca.jsp in reggie/resources/personal/ is created based on export_monthly_oplist.jsp:

1. JSP script index.jsp in reggie/resources/ is updated with a new entry "INCA export" in the "Export/import information to/from external registers" section. The new entry is linked to new JSP script export_inca.jsp in reggie/resources/personal/.
2. New JSP script export_inca.jsp will also call servlet ExportServlet.java, but with request parameter cmd set to "ExportINCA" instead of "ExportMonthlyOpList".
3. Boolean request parameters exportSubtype and exportPatientId are set to 1 instead of 0, since the "Sample" and "PatientId" columns should be included in the INCA export file.
4. A new time selection option "all" is introduced, coupled to setting request parameter time to 1, corresponding to 1970.01.01 00:00:00 001. Option "all" is set as default option.

Java servlet ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ is updated to manage cmd "ExportINCA" in addition to "ExportMonthlyOpList":

1. If request parameter time is set to 1, all samples will be processed, irrespective of sampling time.
2. Blood samples will be processed for the INCA export, as will specimens with biopsy types SpecimenCoreBiopsy and SpecimenNeedle.
3. Due to the differences in sample selection between the monthly operation list export and the INCA export, a special code section is introduced for the latter. For the INCA export, the selection is performed in two steps:
   a. In the first step a sample may be entered in one or both of two HashMaps of type HashMap<String,Sample>, one for each laterality, patientNumberSampleLeftHashMap and patientNumberSampleRightHashMap.
   b. A new private method HashMap<String, Sample> updatePatientNumberSampleHashMap(DbControl dc, SnapshotManager manager, HashMap<String, Sample> patientNumberSampleHashMap, String patientNumber Sample s, String laterality) manages the priorities between different samples, and decides whether a sample should be entered in the HashMap, optionally replacing a previous entry.
   c. Samples with known laterality will only be processed for update of the corresponding HashMap, while samples with unknown laterality will be processed for update of both HashMaps.
   d. In step 2, a sample that is represented in one of the HashMaps will produce a row in the export file. An exception is made for blood samples, for which are required that the sample should be represented in both HashMaps for a row to be printed, since other samples have higher priorities than blood samples, and the latter have unknown laterality.
4. Minor changes in the export table headers are made, i.e. exchanging "PersonalNr" and "OpDate" for "PersonalNo" and "Date", respectively.
5. The export filename will have prefix "incexport-" instead of "opdates-". When time option "all" is selected, the default export filename will include the default start year 2010, a hyphen "-", and the current year and month, e.g. "incaexport-2010-2013-04.tsv" for an export created during April 2013.

[olle]

(In [1957]) Refs #487. First version of the INCA export.

[olle]

Functionality description update:

The INCA export file is intended to be used when requesting information from the INCA database, which contains diagnoses for cancer cases. In order to ensure that the most relevant diagnose is returned, the sample selection should be updated as follows:

1. All samples of subtype "Specimen" and "NoSpecimen" should be included for each laterality, up to one for each biopsy type and date.
2. As previously, a blood sample should only be included, when no other sample subtype exist for a patient. Only the earliest blood sample should be included.
3. A new column indicating the sample biopsy type should be included in the export file, placed directly after the sample subtype column. For samples with biopsy type null, biopsy type should be reported as "default" for samples of subtype "Specimen" and "NoSpecimen", but "not_available" for blood samples.

[olle]

Design update:

1. Since the export format for the monthly operation list and the INCA export now differ considerably, it was decided to introduce a new inner class PersonalIncaExportData for the INCA export, while PersonalOpDate will be used for the monthly operation list export. Class PersonalIncaExportData will contain the same private final attributes as PersonalOpDate plus a new String biopsyType. The public methods boolean equals(Object obj) and int compareTo(PersonalIncaExportData o) will be extended to check the subtype and biopsyType values, in that order, if all other checked values are equal.
2. Private method HashMap<String, Sample> updatePatientNumberSampleHashMap(...) will be simplified by storing a new sample if no sample has been stored for a patient, and replace a stored blood sample with a new sample of other subtype, or a new blood sample with earlier sampling date than the stored one.
3. When assembling samples for the INCA export in the second step of the procedure, only blood samples will be checked against the hash maps.
4. Sample biopsy type will be included in the export, in a column directly after the the sample subtype column.

[olle]

(In [1958]) Refs #487. Second version of the INCA export. Class/file ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ has been updated to include all samples of type Specimen and NoSpecimen for each laterality, up to one for each biopsy type and date, and include the sample biopsy type in the exported data:

1. Since the export format for the monthly operation list and the INCA export now differ considerably, a new inner class PersonalIncaExportData is introduced for the INCA export, while PersonalOpDate will be used for the monthly operation list export. Class PersonalIncaExportData contains the same private final attributes as PersonalOpDate plus a new String biopsyType. The public methods boolean equals(Object obj) and int compareTo(PersonalIncaExportData o) are extended to check the subtype and biopsyType values, in that order, if all other checked values are equal.
2. Private method HashMap<String, Sample> updatePatientNumberSampleHashMap(...) is simplified by storing a new sample if no sample has been stored for a patient, and replace a stored blood sample with a new sample of other subtype, or a new blood sample with earlier sampling date than the stored one.
3. When assembling samples for the INCA export in the second step of the procedure, only blood samples will be checked against the hash maps.
4. Sample biopsy type will be included in the export, in a column directly after the the sample subtype column.

[olle]

Ticket closed as an INCA export corresponding to current specification has been developed.