Level 3 exporter

[Nicklas Nordborg]

An export plug-in should be developed that exports data from Relax to a level 3 publication server.

We will re-use ideas from the "Release exporter" (#938) that is exporting data from the production server to Relax.

• The export create JSON files with data in the same format as before. In principle it should be possible to use the relax importer also on the level 3 server, though me might need more and/or different functionality in the future.
• The files are saved on some local file server

There are some differences:

• The exporter must work with different releases, which means that we can't hard-code annotations, etc. that should be exported since not all annotations are present in all releases. The starting point is that all non-INCA annotations should be exported and only white-listed INCA annotations. White-listing can be done by creating annotation type categories. It should be possible to create multiple categories and then choose which one to use when creating the export.
• Only white-listed files should be exported. Details need to be worked out.
• Scripts for copying data files need a different solution since we can't assume that the level 3 publication server can access the internal file server. The export/import step should work even if we have to transfer files, for example, with an USB drive.

[Nicklas Nordborg]

(In [4571]) References #994: Level 3 exporter

Started to implement the level 3 exporter. It uses bits and pieces of code from Reggie and from the Cohort exporter plug-in (#987). The goal is that the exporter should generate JSON files that are compatible with the files generated by the "Release exporter" in Reggie (#938). This would make it possible to use Relex for importing also on the level 3 server. The current implementation is not able to do this yet.

• It should export the item chain and all annotations defined on each level (this depends on the categories each annotation type belongs to).
• The source list must be a list with raw bioassays.
• There is no special handling for INCA annotations yet (the are all exported).
• No file data is exported
• No scripts for copying files
• Incorrect value for the DataFilesFolder annotation (we must export the default value and ignore the project-specific value since the imported will auto-prepend the release-specific path).
• And possible more things...

[Nicklas Nordborg]

(In [4572]) References #994: Level 3 exporter

The DataFilesFolder annotation is now exported with the default value instead of the project-specific value.

The plug-in now asks for a category with INCA annotation types. If selected, only the INCA annotations that are part of the category are included in the export. If not category is selected, all INCA annotations are exported.

[Nicklas Nordborg]

(In [4575]) References #994: Level 3 exporter

FPKM file information is now exported for raw bioassays with cufflinks raw data.

[Nicklas Nordborg]

(In [4576]) References #994: Level 3 exporter

Creating and exporting some eadditional files needed to make the import work.

• index.json: Information about the raw bioassay list
• batch-index-lookup.json: Not used, but we need an empty file
• files.json: Filenames of all item JSON files
• exportcomplete: Without it, the importer will wait forever.

The JSON files that are exported should now be compatible with the Reggie exporter. This means that the relax plug-in can be used on the level 3 server for importing data to it. Still missing scripts for copying files.

[Nicklas Nordborg]

(In [4577]) References #994: Level 3 exporter

Bash scripts for creating the release folder structure and rsync-ing cufflinks files are now created.

[Nicklas Nordborg]

(In [4578]) References #994: Level 3 exporter

Adding Level3Archive file server intended for exporting files that are going to the level-3 server. Using the ReleaseArchive server for this has been explicitely disabled since it may overwrite the releases coming from Reggie.

[Nicklas Nordborg]

(In [4579]) References #994: Level 3 exporter

The exporter no longer include annotation categories that doesn't contain any INCA annotations in the selection list for the "INCA annotations" category.

[Nicklas Nordborg]

(In [4608]) References #994: Level 3 exporter

Added a configuration file for relax. So far, the only setting is <server-mode> which can be set to RELAX or LEVEL3.

This information is used to determine default configuration settings for some plug-ins. For example, the Level3ExporterPlugin will not be able to export to the ReleaseArchive server in RELAX mode since that may cause data originating from lorry to be overwritten.

[Nicklas Nordborg]

(In [4613]) References #994: Level 3 exporter

More missing files that should have been added in [4608].

[Nicklas Nordborg]

(In [4629]) References #994: Level 3 exporter

Removed debug code (the plug-in will no longer report an error if the export is successful).