Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle BOMs when loading HRA criteria tables #1461

Merged
merged 6 commits into from
Dec 11, 2023
Merged

Handle BOMs when loading HRA criteria tables #1461

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
753bcb3
Fixing and testing UTF-8-SIG for the criteria table.
phargogh Dec 5, 2023
dcf897b
Noting change in HISTORY.
phargogh Dec 5, 2023
3c07a04
Merge branch 'main' of https://github.com/natcap/invest into bugfix/1…
phargogh Dec 5, 2023
b19b946
Using utf-8-sig encoding directly.
phargogh Dec 7, 2023
4dbe7f5
Making sure the new file is closed after we're done with it. RE:#1460
phargogh Dec 7, 2023
0bc66be
Doing the test by opening the file in binary mode. RE:#1460
phargogh Dec 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions tests/test_hra.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,12 +331,11 @@ def test_criteria_table_parsing_with_bom(self):
from natcap.invest import hra

criteria_table_path = os.path.join(self.workspace_dir, 'criteria.csv')
with open(criteria_table_path, 'w') as criteria_table:
bom_char = "\uFEFF" # byte-order marker in 16-bit hex value
with open(criteria_table_path, 'w', encoding='utf-8-sig') as criteria_table:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utf-8-sig includes the BOM when writing the file. Cool!

criteria_table.write(
textwrap.dedent(
f"""\
{bom_char}HABITAT NAME,eelgrass,,,hardbottom,,,CRITERIA TYPE
"""\
HABITAT NAME,eelgrass,,,hardbottom,,,CRITERIA TYPE
HABITAT RESILIENCE ATTRIBUTES,RATING,DQ,WEIGHT,RATING,DQ,WEIGHT,E/C
recruitment rate,2,2,2,2,2,2,C
connectivity rate,2,2,2,2,2,2,C
Expand All @@ -351,6 +350,11 @@ def test_criteria_table_parsing_with_bom(self):
management effectiveness,2,2,1,2,2,1,E
"""
))

# Sanity check: make sure the file has the expected BOM
bom_char = "\uFEFF" # byte-order marker in 16-bit hex value
assert open(criteria_table_path).read().startswith(bom_char)

target_composite_csv_path = os.path.join(self.workspace_dir,
'composite.csv')
hra._parse_criteria_table(criteria_table_path,
Expand Down
Loading