explore use of parquet version of Argo (from D. Nicholson & E. Milanese)

[gaelforget]

Hi @enrico-mi and @dnicholson

this is an extended version of the example code you provided, turned into a Pluto notebook (see below) and where I added a few things :

• plotting methods and visuals
• get_subset_region and get_subset_float functions
• functions to extract positions, incl via Distributed + SharedArrays

Pluto notebook

These recative notebooks are most portable as they include the environment (Project.toml + Manifest.toml are embedded).

To run the notebook you want to do this :

• link the folder that contains Argo*.parquet to data_pq in this notebook's folder
• julia -e 'using Pluto; Pluto.run(notebook="ArgoParquetDev.jl")' in terminal window

Or you can execute the notebooks via the Julia REPL :

using Pluto; Pluto.activate_notebook_environment("ArgoParquetDev.jl")
using Pkg; Pkg.instantiate()
include("ArgoParquetDev.jl")

notes

Would be great if you could try it out @enrico-mi and @dnicholson

My hope is that moving forward you will also be willing to help maintain the notebook (when e.g. file formats may evolve from the current prototype) and improve it (incl. with markdown narrative, metadata etc).

For now this is not tested as part of the CI, or advertised in the docs, but the idea is that once there is a public data release (on Dataverse or zenodo?) then we can run a version of this notebook automatically via GitHub action and advertise it in the docs examples.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.76%. Comparing base (4952e75) to head (984dc14).

Additional details and impacted files

@@           Coverage Diff           @@
##           master      #81   +/-   ##
=======================================
  Coverage   76.76%   76.76%           
=======================================
  Files          13       13           
  Lines        1399     1399           
=======================================
  Hits         1074     1074           
  Misses        325      325           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.