diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index d985d8f..f09efc6 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-29T20:42:15","documenter_version":"1.4.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-29T21:14:55","documenter_version":"1.4.0"}} \ No newline at end of file diff --git a/dev/index.html b/dev/index.html index 2203200..e1c8e5e 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,9 +1,9 @@ Home · ZarrDatasets.jl

ZarrDatasets

ZarrDatasets.ZarrDataset — Type
ds = ZarrDataset(url::AbstractString,mode = "r";
-                 _omitcode = 404,
+                 _omitcode = [404,403],
                  maskingvalue = missing)
 ZarrDataset(f::Function,url::AbstractString,mode = "r";
-                 maskingvalue = missing)

Open the zarr dataset at the url or path url. Only the read-mode is currently supported. ds supports the API of the JuliaGeo/CommonDataModel.jl. The experimental _omitcode allows to work-around servers that return HTTP error different than 404 for missing chunks.

Example:

using ZarrDatasets
+                 maskingvalue = missing)

Open the zarr dataset at the url or path url. The mode can only be "r" (read-only) or "c" (create). ds supports the API of the JuliaGeo/CommonDataModel.jl. The experimental _omitcode allows to work-around servers that return a HTTP error different than 404 for missing chunks.

Example:

using ZarrDatasets
 url = "https://s3.waw3-1.cloudferro.com/mdl-arco-time-035/arco/MEDSEA_MULTIYEAR_PHY_006_004/med-cmcc-ssh-rean-d_202012/timeChunked.zarr"
 ds = ZarrDataset(url);
 # see the metadata
@@ -21,4 +21,4 @@
 
 zos1 = ZarrDataset(url) do ds
   ds["zos"][:,:,end,1]
-end # implicit call to close(ds)
source

Differences between Zarr and NetCDF files

  • All metadata (in particular attributes) is stored in JSON files for the Zarr format with the following implications:
    • JSON does not distinguish between integers and real numbers. They are all considered as generic numbers. Whole numbers are loaded as Int64 and decimal numbers Float64. It is not possible to store the number 1.0 as a real number.
    • The order of keys in a JSON document is undefined. It is therefore not possible to have a consistent ordering of the attributes or variables.
    • The JSON standard does not allow NaN, +Inf, -Inf (https://github.com/capnproto/capnproto/issues/261).
+end # implicit call to close(ds)source

Differences between Zarr and NetCDF files

diff --git a/dev/search_index.js b/dev/search_index.js index df535a9..51d9a4d 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"#ZarrDatasets","page":"Home","title":"ZarrDatasets","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Modules = [ZarrDatasets]","category":"page"},{"location":"#ZarrDatasets.ZarrDataset","page":"Home","title":"ZarrDatasets.ZarrDataset","text":"ds = ZarrDataset(url::AbstractString,mode = \"r\";\n _omitcode = 404,\n maskingvalue = missing)\nZarrDataset(f::Function,url::AbstractString,mode = \"r\";\n maskingvalue = missing)\n\nOpen the zarr dataset at the url or path url. Only the read-mode is currently supported. ds supports the API of the JuliaGeo/CommonDataModel.jl. The experimental _omitcode allows to work-around servers that return HTTP error different than 404 for missing chunks.\n\nExample:\n\nusing ZarrDatasets\nurl = \"https://s3.waw3-1.cloudferro.com/mdl-arco-time-035/arco/MEDSEA_MULTIYEAR_PHY_006_004/med-cmcc-ssh-rean-d_202012/timeChunked.zarr\"\nds = ZarrDataset(url);\n# see the metadata\ndisplay(ds)\n# load the variable time\ntime = ds[\"time\"][:]\n# load the the attribute long_name for the variable zos\nzos_long_name = ds[\"zos\"].attrib[\"long_name\"]\n# load the global attribute\ncomment = ds.attrib[\"comment\"]\n# query the dimension of the variable zos\nsize(ds[\"zos\"])\nclose(ds)\n\nExample with a do-block:\n\nusing ZarrDatasets\nurl = \"https://s3.waw3-1.cloudferro.com/mdl-arco-time-035/arco/MEDSEA_MULTIYEAR_PHY_006_004/med-cmcc-ssh-rean-d_202012/timeChunked.zarr\"\n\nzos1 = ZarrDataset(url) do ds\n ds[\"zos\"][:,:,end,1]\nend # implicit call to close(ds)\n\n\n\n\n\n","category":"type"},{"location":"#Differences-between-Zarr-and-NetCDF-files","page":"Home","title":"Differences between Zarr and NetCDF files","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All metadata (in particular attributes) is stored in JSON files for the Zarr format with the following implications:\nJSON does not distinguish between integers and real numbers. They are all considered as generic numbers. Whole numbers are loaded as Int64 and decimal numbers Float64. It is not possible to store the number 1.0 as a real number.\nThe order of keys in a JSON document is undefined. It is therefore not possible to have a consistent ordering of the attributes or variables.\nThe JSON standard does not allow NaN, +Inf, -Inf (https://github.com/capnproto/capnproto/issues/261).","category":"page"}] +[{"location":"#ZarrDatasets","page":"Home","title":"ZarrDatasets","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Modules = [ZarrDatasets]","category":"page"},{"location":"#ZarrDatasets.ZarrDataset","page":"Home","title":"ZarrDatasets.ZarrDataset","text":"ds = ZarrDataset(url::AbstractString,mode = \"r\";\n _omitcode = [404,403],\n maskingvalue = missing)\nZarrDataset(f::Function,url::AbstractString,mode = \"r\";\n maskingvalue = missing)\n\nOpen the zarr dataset at the url or path url. The mode can only be \"r\" (read-only) or \"c\" (create). ds supports the API of the JuliaGeo/CommonDataModel.jl. The experimental _omitcode allows to work-around servers that return a HTTP error different than 404 for missing chunks.\n\nExample:\n\nusing ZarrDatasets\nurl = \"https://s3.waw3-1.cloudferro.com/mdl-arco-time-035/arco/MEDSEA_MULTIYEAR_PHY_006_004/med-cmcc-ssh-rean-d_202012/timeChunked.zarr\"\nds = ZarrDataset(url);\n# see the metadata\ndisplay(ds)\n# load the variable time\ntime = ds[\"time\"][:]\n# load the the attribute long_name for the variable zos\nzos_long_name = ds[\"zos\"].attrib[\"long_name\"]\n# load the global attribute\ncomment = ds.attrib[\"comment\"]\n# query the dimension of the variable zos\nsize(ds[\"zos\"])\nclose(ds)\n\nExample with a do-block:\n\nusing ZarrDatasets\nurl = \"https://s3.waw3-1.cloudferro.com/mdl-arco-time-035/arco/MEDSEA_MULTIYEAR_PHY_006_004/med-cmcc-ssh-rean-d_202012/timeChunked.zarr\"\n\nzos1 = ZarrDataset(url) do ds\n ds[\"zos\"][:,:,end,1]\nend # implicit call to close(ds)\n\n\n\n\n\n","category":"type"},{"location":"#Differences-between-Zarr-and-NetCDF-files","page":"Home","title":"Differences between Zarr and NetCDF files","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All metadata (in particular attributes) is stored in JSON files for the Zarr format with the following implications:\nJSON does not distinguish between integers and real numbers. They are all considered as generic numbers. Whole numbers are loaded as Int64 and real numbers Float64. It is not possible to store the number 1.0 as a real number.\nThe order of keys in a JSON document is undefined. It is therefore not possible to have a consistent ordering of the attributes or variables.\nThe JSON standard does not allow the values NaN, +Inf, -Inf which is problematic for attributes (zarr-python #412,\nzarr-specs #81). However, there is a special case for the fill-value to handle NaN, +Inf and -Inf.","category":"page"}] }