NexradLevel2 Bug related to conflicting sizes for dimension

[kmuehlbauer]

tracked also in #258

Bug related to conflicting sizes for dimension

• Previously unreported bug.
• This error does not occur in the pyart reader !
• In the MCVE below, we first decompress the file.
• Error: ValueError: conflicting sizes for dimension 'azimuth': length 360 on 'azimuth' and length 358 on {'azimuth': 'DBZH', 'range': 'DBZH'}

import os
import fsspec
import xradar as xd
import subprocess

local_tmp_dir = "/tmp"
fs_args = {}
_ = fs_args.setdefault("anon", True)
fs = fsspec.filesystem("s3", **fs_args)

s3_filepath = "noaa-nexrad-level2/2010/01/01/KABR/KABR20100101_000618_V03.gz"
local_filepath = os.path.join(local_tmp_dir, os.path.basename(s3_filepath))
fs.download(s3_filepath, local_tmp_dir)
subprocess.run(["gzip", "-d", local_filepath], check=True)

dt = xd.io.open_nexradlevel2_datatree(local_filepath.replace(".gz", ""))

[kmuehlbauer]

The reader breaks when trying to access the last sweep:

import xarray as xr
ds = xr.open_dataset(local_filepath.replace(".gz", ""), engine="nexradlevel2", group="sweep_6") 
ds

All sweeps before that will read perfectly fine. Investigating the root cause.

[mgrover1]

Ohh thanks for raising the issue here!

[kmuehlbauer]

So, I've found the root cause of the ValueError:

xradar/xradar/io/backends/nexrad_level2.py

Lines 1240 to 1246 in 830d86b

	# get rays and bins
	nrays = (
	datastore.ds["record_end"]
	- datastore.ds["record_number"]
	+ 1
	- len(datastore.ds["intermediate_records"])
	)

In the above code we retrieve the number of rays by calculating

nrays = n_end - n_start + 1 - n_intermed

where n_intermed is the numbers of intermediate records within the data records. In the above case in the final record, those intermediate records are after n_end and should not be subtracted. So the solution is to count the number of intermediate records which fall inside n_start and n_end.