Reading a CSV file exported from Excel leaves UTF-8 BOM characters in file

[martinstreicher]

I created a small Excel XLS file. I saved it and exported XLSX and CSV to use in a test suite for a Tabular-based POR utility class. I chose the UTF-8 CSV option from Excel to create the CSV.

When I read the file, the UTF-BOM remains in the first column name. You can see them from the byte-wise deconstruction of the column name.

(byebug) data.first.keys.first.to_s.bytes
[239, 187, 191, 99, 111, 108, 117, 109, 110, 95, 49]
(byebug) :column_1.to_s.bytes
[99, 111, 108, 117, 109, 110, 95, 49]

While I did save XLS as UTF-8 CSV instead of plain CSV, I'd still like the option to remove the BOM. I won't know whether other incoming CSV files will have the BOM or not.

[martinstreicher]

Thoughts? I'm happy to send a PR. Specify format as utf8-csv to flag that the BOM needs to be removed?