Overview report should show number of patients having a sample type item

[olle]

Overview report should show number of patients having a sample type item, where a patient with several samples of the same type should only be counted once for that sample type.

[olle]

Ticket accepted.

[olle]

Traceability note:

• The report generator was introduced in Ticket #339 (Report generator). It included a sample count report.
• The overview report was introduced in Ticket #438 (Overview report).
• The overview report was updated in Ticket #505 (Sample source reports should be updated to support follow-up blood samples).

[olle]

Design discussion:

The overview report currently contains two tables, one showing number of items of different kinds, and one showing number of patients with different combinations of sample items.

• A new table should be inserted between the two, showing number of patients with at least one item of Blood sample, Follow-up blood sample, Specimen, and NoSpecimen, respectively. Columns entries are non-exclusive, i.e. a patient may be represented in several columns.

Note: All data in the new table, except that for follow-up blood samples, is already available by summing a number of column entries in the table showing number of patients with different combinations of sample items, as is shown below:

#Pwb = Number of patients with blood sample(s)
#Pws = Number of patients with specimen(s)
#Pwns = Number of patients with "no specimens(s)"
#P(b) = Number of patients with blood samples only
#P(s) = Number of patients with specimens only
#P(ns) = Number of patients with "no specimens" only
#P(b+s) = Number of patients with blood samples and specimens
#P(b+ns) = Number of patients with blood samples and "no specimens"
#P(s+ns) = Number of patients with specimens and "no specimens"
#P(b+s+ns) = Number of patients with blood samples, specimens, and "no specimens"

#Pwb = #P(b) + #P(b+s) + #P(b+ns) + #P(b+s+ns)
#Pws = #P(s) + #P(b+s) + #P(s+ns) + #P(b+s+ns)
#Pwns = #P(ns) + #P(b+ns) + #P(s+ns) + #P(b+s+ns)

The new table is therefore mostly a convenience feature.

[olle]

Design implementation:

The overview report is part of the sample source report, that is managed by JSP script samplereportgenerator.jsp in reggie/resources/reports/, and Java servlet SampleReportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/. Most required information is already available in HashSets in the latter, keeping track of ID's of patients having different sample types (the exception is follow-up blood samples, for which a new HashSet should be added). The information is made available to the JSP script by adding more items to the returned JSON object.

Java servlet SampleReportServlet.java updates in private method JSONObject createOverviewReport(DbControl dc, JSONObject json, Date startDate, Date endDate, String bloodSampleFilter):

1. Four new JSON keys are defined for the required sample type information, and the JSON statistics objects are initialized for these keys.
2. A new HashSet is defined for keeping track of ID's of patients having follow-up blood samples, and is filled with data.
3. The HashSets keeping track of ID's of patients having different sample types are used to add data to the JSON statistic objects for the new JSON keys.

JSP script samplereportgenerator.jsp updates:

1. Function goCreate() should be updated in the section for optional extra tables to add a table for the overview report, created by new function createOverviewPatientWithSampleTable(report).
2. New function createOverviewPatientWithSampleTable(report) added. It creates the new table using data from JSON object report, calling new function addDataRowsToOverviewPatientWithSampleTable(report, reportTable) to set the main table entries.
3. New function addDataRowsToOverviewPatientWithSampleTable(report, reportTable) added. It fills the main "patient with sample" table with data from the JSON object report.

[olle]

(In [2178]) Refs #550. Overview report updated to show number of patients having a sample type item:

Java servlet SampleReportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ updated in private method JSONObject createOverviewReport(DbControl dc, JSONObject json, Date startDate, Date endDate, String bloodSampleFilter):

1. Four new JSON keys are defined for the required sample type information, and the JSON statistics objects are initialized for these keys.
2. A new HashSet is defined for keeping track of ID's of patients having follow-up blood samples, and is filled with data.
3. The HashSets keeping track of ID's of patients having different sample types are used to add data to the JSON statistic objects for the new JSON keys.

JSP script samplereportgenerator.jsp in reggie/resources/reports/ updates:

1. Function goCreate() updated in the section for optional extra tables to add a table for the overview report, created by new function createOverviewPatientWithSampleTable(report).
2. New function createOverviewPatientWithSampleTable(report) added. It creates the new table using data from JSON object report, calling new function addDataRowsToOverviewPatientWithSampleTable(report, reportTable) to set the main table entries.
3. New function addDataRowsToOverviewPatientWithSampleTable(report, reportTable) added. It fills the main "patient with sample" table with data from the JSON object report.

[olle]

Ticket closed as the requested functionality has been added.

[olle]

Ticket reopened as more functionality was desired for the report in question.

[olle]

Design update:

• The overview report should allow a time period filter, where a start and end date may be entered. The time period filter should work as in the other reports:
  
  a. If the start date is left blank, it is set to the earliest site start date.
  b. If the end date is left blank, it is set to the current date.
  c. Items with dates before the start date or after the end date should be excluded, all other included, i.e. items with dates equal to the start or end date should be included.

• Handling items with missing dates is non-trivial, since the overview report contains a table with columns representing different types of items. Tables for a single type of item normally contain a "Total" column that includes items with dates outside the selected time period, as well as items with unknown dates. The following choices were made for the overview report:
  
  a. The time period filter should only be applied to sample items, while patients and consents are derived from these.
  b. If an item lacks creation date, the entry (registration) date is used instead.
  c. The number of items of different types with unknown date should be reported in a comment line at the end of the report.

[olle]

Design note:

Some consequences of the design choices for the time period filter in the overview report:

• The number of "Yes" consents for a time period may differ in the overview report and the Consent report, as the overview uses the corresponding sample date, while the latter uses the consent date.
• Some items in the "Patient" column in the overview report for non-overlapping time periods may still be identical, as a patient who has undergone surgery for cancer in both breasts, but at different dates, will be represented by two sample dates.

[olle]

Design implementation:

Overview report update to allow time period filter:

JSP script samplereportgenerator.jsp in reggie/resources/reports/ updates:

1. Function gotoStep2() updated to display the time period input fields.
2. Function goCreate() updated to report the number of items of different types with unknown date in a comment line at the end of the report.

Java servlet SampleReportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ updated in private method JSONObject createOverviewReport(DbControl dc, JSONObject json, Date startDate, Date endDate, String bloodSampleFilter):

1. Four new JSON keys are defined to keep track of the number of items of different types with unknown date, and the JSON statistics objects are initialized for these keys.
2. The number of sample items of different types with unknown date are counted, and the results are added to the returned JSON object via the new JSON keys.
3. The selected time period filter is applied to the sample items, where the entry (registration) date is used for items with unknown creation date.

[olle]

(In [2183]) Refs #550. Overview report updated to allow time period filter:

JSP script samplereportgenerator.jsp in reggie/resources/reports/ updates:

1. Function gotoStep2() updated to display the time period input fields.
2. Function goCreate() updated to report the number of items of different types with unknown date in a comment line at the end of the report.

Java servlet SampleReportServlet.java in reggie/src/net/sf/basedb/reggie/servlet/ updated in private method JSONObject createOverviewReport(DbControl dc, JSONObject json, Date startDate, Date endDate, String bloodSampleFilter):

1. Four new JSON keys are defined to keep track of the number of items of different types with unknown date, and the JSON statistics objects are initialized for these keys.
2. The number of sample items of different types with unknown date are counted, and the results are added to the returned JSON object via the new JSON keys.
3. The selected time period filter is applied to the sample items, where the entry (registration) date is used for items with unknown creation date.

[olle]

Ticket closed as the requested functionality has been added.