Handle BOMs when loading HRA criteria tables

[phargogh]

This PR corrects (and tests for) loading the HRA criteria table using the UTF-8-SIG encoding.

Fixes #1460

Checklist

• Updated HISTORY.rst and link to any relevant issue (if these changes are user-facing)
  - [ ] Updated the user's guide (if needed)
  - [ ] Tested the Workbench UI (if relevant)

[phargogh]

With the update to use utils.read_csv_to_dataframe, the only value-add here is the test to make sure we can handle the BOM when parsing.

[dcdenu4]

BOMs away!

[dcdenu4]

Oops, I was a little premature with my approval. Looks like the Windows tests are failing on this.

[dcdenu4]

Thanks @phargogh , still some funky Windows things unfortunately!

[dcdenu4]

utf-8-sig includes the BOM when writing the file. Cool!

[dcdenu4]

This bit is still failing on Windows and I think it's because of the following from Python's codec docs (https://docs.python.org/3/library/codecs.html#encodings-and-unicode)

On encoding the utf-8-sig codec will write 0xef, 0xbb, 0xbf as the first three bytes to the file. On decoding utf-8-sig will skip those three bytes if they appear as the first three bytes in the file. In UTF-8, the use of the BOM is discouraged and should generally be avoided.

So maybe it's being stripped out when opening and decoding? Although this is contradictory to what I'm seeing in my terminal:

>>> with open("test-bom.csv", 'w', encoding='utf-8-sig') as my_table:
...     my_table.write("HABITAT NAME,eelgrass,,,hardbottom,,,CRITERIA TYPE")
>>>
>>> with open("test-bom.csv") as table:
...     print(table.read())
...
HABITAT NAME,eelgrass,,,hardbottom,,,CRITERIA TYPE

Which makes sense because:

To increase the reliability with which a UTF-8 encoding can be detected, Microsoft invented a variant of UTF-8 (that Python calls "utf-8-sig") for its Notepad program: Before any of the Unicode characters is written to the file, a UTF-8 encoded BOM (which looks like this as a byte sequence: 0xef, 0xbb, 0xbf) is written. As it’s rather improbable that any charmap encoded file starts with these byte values (which would e.g. map to

LATIN SMALL LETTER I WITH DIAERESIS
RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
INVERTED QUESTION MARK

So, I'm not sure why the test assertion error looks like the BOM is being decoded and stripped, where as when I open and read it I get the funky character mapping...

[phargogh]

Thanks @dcdenu4 ! It looks like the missing part was to open it up in binary mode ... that's going to bypass any interpretation or OS-specific assumptions of whether to strip those leading bytes.

[dcdenu4]

Thanks James. There are some Workbench related dependencies failing, but they're unrelated.