You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, RawFit.mdata, ReducedFit.mdata are properties that read and return the data from the files every time they are used. There could be two alternatives with different advantages and drawbacks:
Make them cached properties so that they read the file only one, saving quite a bit of disk read access. This would make some functions and views slightly faster.
Make RawFit.mdata a cached property that reads data only one. Make ReducedFit.mdata a cached property that computes the value on the fly, applying the master calibrations frames on the fly to the rawfit. This would make it the pipeline processes only slightly slower, however, for the iop4admin views using the data, such as the stats on the detailed views, it might add some seconds to the page load. Some tests are needed to check if these differences are significant (applying the calibration frames, once they are built, is quite fast).
The main advantage of this point is that disk space would be reduced by a factor of x5. Right now, since the reduced FITS are saved using float64 to keep precision (#30 ), they take x4 times the size of the raw images (uint16 usually). This method would completely eliminate these files from the disk. the .build_file would only build and save the WCS header to disk (a few kilobytes at most), and the .header would read this.
The text was updated successfully, but these errors were encountered:
Alternatively, we could make ReducedFit.mdata return the files from the disk if they exist, otherwise just build it, save it, and return it. Then we could delete old reduced fits file and only keep the last months, that are the ones more likely to be consulted in the admin, freeing quite a lot of space too. This however will mean more writes, which is what wores out the disk more (also not very important, but still).
Right now, RawFit.mdata, ReducedFit.mdata are properties that read and return the data from the files every time they are used. There could be two alternatives with different advantages and drawbacks:
The main advantage of this point is that disk space would be reduced by a factor of x5. Right now, since the reduced FITS are saved using float64 to keep precision (#30 ), they take x4 times the size of the raw images (uint16 usually). This method would completely eliminate these files from the disk. the .build_file would only build and save the WCS header to disk (a few kilobytes at most), and the .header would read this.
The text was updated successfully, but these errors were encountered: