Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean empty lines in csv before reading #140

Open
nailend opened this issue Nov 10, 2023 · 1 comment
Open

Clean empty lines in csv before reading #140

nailend opened this issue Nov 10, 2023 · 1 comment

Comments

@nailend
Copy link
Collaborator

nailend commented Nov 10, 2023

A common reading error occurs when having multiple empty lines at the end of a csv file.
tableschema.exceptions.CastError: Row length 0 doesn't match fields count 3

tableschema is expecting 3 columns but as an empty line doesn't have any column separators, there are none.
This could be easily caught by cleaning the csv files from any empty lines while reading

@nailend
Copy link
Collaborator Author

nailend commented Nov 10, 2023

its actually more complicated then I thought, as the csv would have to be modified itself, not sure if this is a good idea...

empty lines at the end could be detected, using raw_read()

raw_r = r.raw_read()
str(raw_r[-2:]) == "b'\\n\\n'"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant