grib2io v2.0.0b1

[EricEngle-NOAA]

grib2io v2.0.0 Beta 1 Release Notes

Introduction

The entire grib2io codebase has been rewritten. grib2io v1 was an optimized version of its predecessor, ncepgrib2 (formerly a part of the pygrib package). grib2io v2 has taken a different approach to handling the vast amount of GRIB2 metadata.

The Grib2Message object uses dynamic inheritance where it is comprised of a base object (for GRIB2 Sections 0 and 1). The rest of the GRIB2 message makes use of templates, so grib2io defines each grid, product, and data representation template as a class. When a GRIB2 message is read from file or created from scratch, Grib2Message.__new__() is used to create the custom Grib2Message object based on the template numbers.

In the Grib2Message object, GRIB2 section metadata values are stored in class variables of the section name (e.g. section0, section1, etc). The Grib2Message base class and template classes contain metadata that are class variable that are dataclass field objects. Their default values are descriptor classes that live in grib2io/templates. Each GRIB2 metadata attribute has a descriptor class defined for it. This allows for real-time mapping between a GRIB2 numeric metadata value stored in the section array and its "decoded" state.

Why refactor in this manner? The Grib2Message object now becomes a lightweight object container. The GRIB2 metadata only exists in 1 location -- its respective section array and you access the decoded variable through its descriptor class, not another variable in the object.

Highlights

• Rewrite of Grib2Message object leveraging dynamic inheritance; template classes; and descriptor classes for metadata (see Introduction above for more information). (Use descriptor classes for section templates and metadata. #39)

• Added support for spatial interpolation using NCEPLIBS-sp and NCEPLIBS-ip libraries. (Interpolation #52)

• Improved perforamance when indexing GRIB2 messages. Instead of storing many attributes of each message, the actual Grib2Message object is stored along with message offset information.

• IMPORTANT: For grib2io.open, GRIB2 message number counting now begins at zero (0).

• Improved grib2io.open.select() functionality. Since each Grib2Message object is indexed, the select method can easily access ANY Grib2Message metadata attribute.

• Improved performance when unpacking data values. The GRIB2 data section is only read from file when Grib2Message.data is referenced. (Only read data section when calling data() method #40)

• Added support for Reduced and Rotated Lat/Lon grids. (Add support to compute lat/lon values for rotated grids #42 and Support for Reduced Regular and Gaussian Lat/Lon Grids #59)

• Added support for Product Definition Template 4.48. (Add support for product definition template 4.48 #49)

• Date and time metadata objects are now of type datetime.datetime or datetime.timedelta. (Use datetime objects for GRIB2 date/time metadata #1)

• Improved xarray backend support.

Full Changelog: v1.1.0...v2.0.0b1

---

This discussion was created from the release grib2io v2.0.0b1.

[kevmcgrath]

Hi Eric,

Thank you for this update. This fixes my problem of packing NDFD data from GRIBs generated by degrib, which had been resulting in an "invalid pointer" error. However, the .pack() command is still hanging when I attempt to pack data from NDFD GRIBs generated by wgrib2. Example:

---

• Generate single-message GRIB using wgrib2:

$ wget https://tgftp.nws.noaa.gov/SL.us008001/ST.opnl/DF.gr2/DC.ndfd/AR.conus/VP.001-003/ds.qpf.bin
$ /apps/ops/para/libs/intel/19.1.3.304/wgrib2/2.0.8/bin/wgrib2 ds.qpf.bin -match "(^1:)" -grib_out qpf_VP.001-003_wgrib2.grb

---

• Read single-message GRIB generated above and attempt to pack:

import grib2io
sourceGrib = 'qpf_VP.001-003_wgrib2.grb'
grbs_qpf06 = grib2io.open(sourceGrib)
qpf06Message = grbs_qpf06[0]
qpf06Message.pack()

HANGS

Let me know if you'd like for me to create an 'issue' for this.

Thanks,
Kevin

[EricEngle-NOAA]

Before getting into grib2io, I wanted to make you aware that the wgrib2 option you are using -grib_out seemingly unpacks and repacks data. If you look at the original ds.qpf.bin, GRIB2 messages are using complex packing, but the file coming out of wgrib2 using -grib_out is simple packing. If you use `-grib', then bytes are simply put from old to new files without modifications.

Now grib2io should and must accommodate the this message and I am looking into that.

[EricEngle-NOAA]

This will be fixed in beta 2.

[kevmcgrath]

Thank you, Eric!