Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation logic for domain specific nad fields #45

Merged
merged 1 commit into from
Mar 21, 2024

Conversation

sb-2011
Copy link
Collaborator

@sb-2011 sb-2011 commented Mar 20, 2024

Created the DataValidator class to contain validation logic and added domain specific checks. Also did some minor refactoring.

@sb-2011 sb-2011 requested a review from akuny March 20, 2024 20:00
Copy link
Collaborator

@akuny akuny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one comment concerning a potentially dead method of the DataReader class.

@@ -40,24 +33,25 @@ def validate_column_map(self):
)

def rename_columns(self, gdf: GeoDataFrame) -> GeoDataFrame:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this rename_columns method used anywhere? If not, we probably want to remove it for the time being; if it is in use somewhere, testing it would be a good idea.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is used in the read_file_in_batches() method before yielding a batch. It is needed to apply the crosswalk/column mapping.

@sb-2011 sb-2011 merged commit 1a91e06 into main Mar 21, 2024
1 check passed
@sb-2011 sb-2011 deleted the add-validation-for-domains branch March 21, 2024 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants