Data Release Page not Updating

[nithujohn]

Hi @scottcain, it looks like the data release page is not updating for a while (since May 6th). Please have a look. Also the front page is partially updating. Thanks!

[scottcain]

I just wrote a comment on #222 This is a critical bug that is being addressed now but unfortunately is not an easy fix.

[b-f-chan]

Assign to @UmmulkiramR to investigate - There is currently a limit to the download size due to one of the libraries we are using in the Singularity code, I think @joneubank first suggestion was to investigate that library and see if there is something else out there we can use (please connect with him for details)

[ghost]

Debugging guidelines by Jaser - https://wiki.oicr.on.ca/display/OV/Fix+latest+release+data+not+being+created

[joneubank]

Our dataset has grown to now be too large to build a single archive with the tools in our code base. We have theorized that the issue is based on the Java language limit on Array length (MAX_INTEGER which is 2^32 elements, approx 2Gb size limit) that is being overcome by our archive library when it converts the data to a byte array. This means that we will need to either find a new library that is not limited in this way, or we will need to stop building an archive of the full data set.

[ghost]

We have a temporary fix for this Provide isolated fasta downloading functionality #279 and Change Tar library header properties for unstuck bundling issues #39

We will move this to prod in day or two. Currently placing this ticket in backlog as we wait for a long term solution to this.

[UmmulkiramR]

@sifavahora
Just to be clear - as we have discussed, only the isolated fasta downloading functionality #279 would be moved to prod.

Code changes to resolve virusseq/singularity#39 would be moved to testing once you give a go-ahead.

[ghost]

We decided to use solution virusseq/singularity#39 as that was the best.
This bug is resolved. However we are planning to redesign the page, Will capture requirements in subsequent tickets.
Closing this ticket.