Missing sample data report should support sample list export

[olle]

Missing sample data report should support sample list export. An "Export" button should be added beside the "Restart" button at the end of the report view. Clicking the "Export" button should create a sample list, that can be downloaded or opened by a viewer program.

The export list should have the following properties:

1. The list should consist of tab-separated columns.
2. When sample type "Specimen" or "No specimen" is selected, a common export list should be created, while a separate list should be created when sample type "Blood" is selected.
3. The export list should include all samples with missing data of the sample types in question, irrespective of any blood sample filter (only relevant for blood samples).
4. The export list for sample types "Specimen" and "No specimen" should include the following columns:
   a. Sample name
   b. Subtype
   c. Personal number
   d. All first names
   e. Family name
   f. PAD
   g. Laterality
   h. Sampling date
   i. RNALater date
   j. Biopsy type
   
5. The export list for sample type "Blood" should include the following columns:
   a. Sample name
   b. Subtype
   c. Personal number
   d. All first names
   e. Family name
   f. Blood sampling date
   g. Blood freezer date
   h. Blood sample type
   

[olle]

Ticket accepted.

[olle]

Traceability note:

• The missing sample data report was introduced in Ticket #439 (Missing sample data report).
• The blood sample filter was introduced in Ticket #505 (Sample source reports should be updated to support follow-up blood samples), where it was added to the sample count, overview, and missing items reports.
• The monthly operation list export was introduced in Ticket #385 (Export information needed by the tumor registry).
• INCA export was introduced in Ticket #487 (Export information intended for INCA).

[olle]

Ticket description updated regarding values to report and their order. Also clarification of text.

[olle]

Design discussion.

1. In the last step before a report is generated, the "Generate" button is coupled to function goCreate() in JSP script samplereportgenerator.jsp in reggie/resources/reports/, which in turn sends an Ajax request to java servlet SampleReportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/. The new "Export" button will be coupled to the same function, but java servlet ExportServlet.java should be called instead of SampleReportServlet.java, in order to separate code for export from that for generating data for screen tables. This can be accomplished by adding a boolean flag exportData as argument to function goCreate(), and let button "Generate" call goCreate(false), while button "Export" calls goCreate(true). Function goCreate(exportData) will be updated to call java servlet ExportServlet.java instead of SampleReportServlet.java, when argument ExportData has value true.
2. Java servlet ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ should be updated:
   a. Protected method void doGet(HttpServletRequest req, HttpServletResponse resp) should be updated for command "ExportMissingSampleDataSampleList" to call new private method Set<MissingSampleDataExportData> createMissingSampleDataExport(DbControl dc, String sampleType, String bloodSampleFilter) to create a missing sample data sample list for export.
   b. New private method Set<MissingSampleDataExportData> createMissingSampleDataExport(DbControl dc, String sampleType, String bloodSampleFilter) calls new private method List<Sample> createSampleListForMissingSampleDataReport(DbControl dc, String sampleType, String bloodSampleFilter) to obtain a list of samples of the desired sample type(s). This list will be processed to obtain a list, that only contains samples with missing data. The missing data sample list is used to create a list of MissingSampleDataExportData objects, that will returned.
   c. New private method List<Sample> createSampleListForMissingSampleDataReport(DbControl dc, String sampleType, String bloodSampleFilter) performs a database query and creates a sample list for the missing sample data report. The list will both include samples with and without missing data, and has to be processed afterwards to obtain a list, that only contains samples with missing data.
   d. New inner class MissingSampleDataExportData added. Its public method String toString() will a return a tab-separated line with data for the missing sample data export list, with contents depending on the subtype attribute.

[olle]

(In [2206]) Refs #565. Missing sample data report updated with sample list export:

1. JSP script samplereportgenerator.jsp in reggie/resources/reports/ updated with a new "Export" button beside the "Restart" button at the end of the report view. The new "Export" button is coupled to the same function goCreate() as the "Generate" button in the last step before a report is generated, but java servlet ExportServlet.java should be called instead of SampleReportServlet.java. This is accomplished by adding a boolean flag exportData as argument to function goCreate(), and let button "Generate" call goCreate(false), while button "Export" calls goCreate(true). Function goCreate(exportData) is updated to call java servlet ExportServlet.java instead of SampleReportServlet.java, when argument ExportData has value true.
2. Java servlet ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ updated:
   a. Protected method void doGet(HttpServletRequest req, HttpServletResponse resp) updated for command "ExportMissingSampleDataSampleList" to call new private method Set<MissingSampleDataExportData> createMissingSampleDataExport(DbControl dc, String sampleType, String bloodSampleFilter) to create a missing sample data sample list for export.
   b. New private method Set<MissingSampleDataExportData> createMissingSampleDataExport(DbControl dc, String sampleType, String bloodSampleFilter) calls new private method List<Sample> createSampleListForMissingSampleDataReport(DbControl dc, String sampleType, String bloodSampleFilter) to obtain a list of samples of the desired sample type(s). This list is processed to obtain a list, that only contains samples with missing data. The missing data sample list is used to create a list of MissingSampleDataExportData objects, that will returned.
   c. New private method List<Sample> createSampleListForMissingSampleDataReport(DbControl dc, String sampleType, String bloodSampleFilter) performs a database query and creates a sample list for the missing sample data report. The list will both include samples with and without missing data, and has to be processed afterwards to obtain a list, that only contains samples with missing data.
   d. New inner class MissingSampleDataExportData added. Its public method String toString() will a return a tab-separated line with data for the missing sample data export list, with contents depending on the subtype attribute.

[olle]

Ticket description updated by fixing typo.

[olle]

Design update:

• It was decided to move the new functionality to a new entry "Export missing data sample list" under Reggie section "Export/import information to/from external registers". The added "Export" button beside the "Restart" button at the end of the report view should therefore be removed.

[olle]

(In ​[2210]) Refs #565. Missing data sample list export updated to move the functionality from the missing sample data report to a new entry "Export missing data sample list" under Reggie section "Export/import information to/from external registers":

1. JSP script index.jsp in reggie/resources/ updated with new entry "Export missing data sample list" under Reggie section "Export/import information to/from external registers", with link to new JSP script export_missing_data_sample_list.jsp in reggie/resources/personal/.
2. JSP script samplereportgenerator.jsp in reggie/resources/reports/ updated by removal of the added "Export" button beside the "Restart" button at the end of the report view in the missing sample data report, thus removing the changes added in change set ​[2206].
3. New JSP script export_missing_data_sample_list.jsp in reggie/resources/personal/ added. It is based on JSP script export_inca.jsp, but has a menu "Sample type(s)" with options "Specimen/No specimen" and "Blood". It calls java servlet ExportServlet.java with command ExportMissingSampleDataSampleList and the selected menu option.
4. Java servlet ExportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ updated to accept the new menu option "Specimen/No specimen".

[olle]

Ticket closed since the requested functionality has been added.

[olle]

(In [2220]) Refs #565. Missing data sample list export updated in preview for sample types "Specimen/No specimen" to report number of list items of each subtype in the header text:

1. JSP script export_missing_data_sample_list.jsp in reggie/resources/personal/ updated in function goExport(preview) to report number of list items of each subtype in the header text for sample types "Specimen/No specimen".

[olle]

(In [2223]) Refs #574. Refs #565. Refs #569. Consent and missing sample data report wizards updated to support missing data sample list export for users with PatientCurator role:

Java servlet SampleReportServlet in reggie/src/net/sf/basedb/reggie/servlet/ update:

1. Private method JSONObject createConsentCountReport(DbControl dc, JSONObject json, Date startDate, Date endDate) updated to check if the user does not have permission to read patient names, and transfer the result in the returned JSON object in variable with key "permissionDeniedForPatientName". The same flag is already returned by private method JSONObject createMissingSampleDataReport(DbControl dc, JSONObject json, Date startDate, Date endDate, String sampleType, String bloodSampleFilter).

JSP script samplereportgenerator.jsp in reggie/resources/reports/ update:

1. Function goCreate() updated with argument exportOpt:
   a. If the value of argument exportOpt is "preview" or "export", the consent count and missing sample data report wizards directs the Ajax request to servlet ExportServlet.java to obtain data for corresponding the missing data sample list.
   b. If the value of argument exportOpt is "preview", a preview is shown created by the one of the new functions createMissingConsentDataSampleListPreview(allLines, previewTitle, previewList) or createMissingDataSampleListPreview(allLines, sampleTypes, previewTitle, previewList), that corresponds to the used report wizard.
   c. If the value of argument exportOpt is "export", the output from the Export servlet request is redirected to be downloaded or opened with a program of the user's choice.
2. Function goCreate(exportOpt) updated for the consent and missing sample data reports to check if the user has permission to read patient names, and if so, add two new buttons to the right of the "Restart" button at the end, "Export preview" and "Export", coupled to calling function goCreate(exportOpt) with argument exportOpt having values "preview" and "export", respectively.
3. New function createMissingConsentDataSampleListPreview(allLines, previewTitle, previewList) added. It creates a sample list preview like the one created by JSP script export_missing_consent_data_sample_list.jsp in reggie/resources/personal/, as described in Ticket #569 (Sample list export for missing consent data).
4. New function createMissingDataSampleListPreview(allLines, sampleTypes, previewTitle, previewList) added. It creates a sample list preview like the one created by JSP script export_missing_data_sample_list.jsp in reggie/resources/personal/, as described in Ticket #565 (Missing sample data report should support sample list export).
5. HTML part updated with new style section for the sample list preview, button definitions for the "Export preview" and "Export" buttons, and new div tags for the optional export preview.