Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Polars and other dataframes #482

Merged
merged 4 commits into from
Dec 15, 2023

Conversation

johalnes
Copy link
Contributor

Overview

As discussed in #474, added Python dataframe interchange protocol.

This open up the possibility for using the chain ladder package with multiple different dataframe packages, like Spark, Polars, Vaex etc.

Behind the scene it uses the "new" pandas function pd.api.interchange.from_dataframe(data) that magically converts a dataframe to the correct types and other dataframe adjustments to make it pandas compatible.

Testing

Tested with the example from issue #474 and added test to ensure same result with polars and pandas

@johalnes
Copy link
Contributor Author

I'm using Ruff for linting and formatting, which complains about some best practices not being followed. Mostly unused imports, isort complains about sorting of imports, but also that type(value) is str should be changed with is instance(value, str) according to some PEP.

Should I create a new issue and pull request with ruff as formatter and use auto fix to remove those smaller "issues"?

Copy link

codecov bot commented Dec 15, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ac3fa37) 81.70% compared to head (63cea7f) 81.73%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #482      +/-   ##
==========================================
+ Coverage   81.70%   81.73%   +0.02%     
==========================================
  Files          80       80              
  Lines        4712     4719       +7     
  Branches      806      807       +1     
==========================================
+ Hits         3850     3857       +7     
  Misses        659      659              
  Partials      203      203              
Flag Coverage Δ
unittests 81.73% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbogaardt jbogaardt merged commit e5ceaf6 into casact:master Dec 15, 2023
3 checks passed
@jbogaardt
Copy link
Collaborator

Thanks for these updates @johalnes !

@jbogaardt
Copy link
Collaborator

I'm using Ruff for linting and formatting, which complains about some best practices not being followed. Mostly unused imports, isort complains about sorting of imports, but also that type(value) is str should be changed with is instance(value, str) according to some PEP.

Should I create a new issue and pull request with ruff as formatter and use auto fix to remove those smaller "issues"?

Sure!

@johalnes johalnes deleted the feature/474_interchange_api branch December 16, 2023 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants