-
Notifications
You must be signed in to change notification settings - Fork 908
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DO NOT MERGE] cudf-polars chunked parquet reader #16789
[DO NOT MERGE] cudf-polars chunked parquet reader #16789
Commits on Jul 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7742b8b - Browse repository at this point
Copy the full SHA 7742b8bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ef0b49f - Browse repository at this point
Copy the full SHA ef0b49fView commit details -
Configuration menu - View commit details
-
Copy full SHA for e9fd96d - Browse repository at this point
Copy the full SHA e9fd96dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9d69621 - Browse repository at this point
Copy the full SHA 9d69621View commit details -
Configuration menu - View commit details
-
Copy full SHA for 918a40e - Browse repository at this point
Copy the full SHA 918a40eView commit details -
Configuration menu - View commit details
-
Copy full SHA for f8f2d0d - Browse repository at this point
Copy the full SHA f8f2d0dView commit details -
Configuration menu - View commit details
-
Copy full SHA for bcedb6b - Browse repository at this point
Copy the full SHA bcedb6bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6f2d406 - Browse repository at this point
Copy the full SHA 6f2d406View commit details -
Configuration menu - View commit details
-
Copy full SHA for f3bbd3f - Browse repository at this point
Copy the full SHA f3bbd3fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1d4c30c - Browse repository at this point
Copy the full SHA 1d4c30cView commit details
Commits on Jul 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for abcf22b - Browse repository at this point
Copy the full SHA abcf22bView commit details
Commits on Aug 2, 2024
-
Merge pull request rapidsai#16347 from wence-/wence/fea/polars-engine…
…-config Use new polars engine config object in cudf-polars callback
Configuration menu - View commit details
-
Copy full SHA for 62a5dbd - Browse repository at this point
Copy the full SHA 62a5dbdView commit details
Commits on Aug 5, 2024
-
Adapt to IR changes in polars 1.4 (rapidsai#16494)
## Description <!-- Provide a standalone description of changes in this PR. --> <!-- Reference any issues closed by this PR with "closes rapidsai#1234". --> <!-- Note: The pull request title will be included in the CHANGELOG. --> Adapts to IR changes in polars 1.4 and handles nrows/skiprows a little more correctly. ## Checklist - [ ] I am familiar with the [Contributing Guidelines](https://github.com/rapidsai/cudf/blob/HEAD/CONTRIBUTING.md). - [ ] New or existing tests cover these changes. - [ ] The documentation is up to date with these changes. --------- Co-authored-by: Lawrence Mitchell <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 7d0c7ad - Browse repository at this point
Copy the full SHA 7d0c7adView commit details
Commits on Aug 6, 2024
-
Implement polars string Replace and ReplaceMany (rapidsai#16039)
Add support for ``pl.col.str.replace`` and ``pl.col.str.replace_many`` Authors: - Thomas Li (https://github.com/lithomas1) Approvers: None URL: rapidsai#16039
Configuration menu - View commit details
-
Copy full SHA for 5de29b3 - Browse repository at this point
Copy the full SHA 5de29b3View commit details
Commits on Aug 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7f6b00f - Browse repository at this point
Copy the full SHA 7f6b00fView commit details
Commits on Aug 20, 2024
-
Backport: Remove cuDF dependency from pylibcudf column from_device te…
…sts (rapidsai#16441) (rapidsai#16605) Authors: - Thomas Li (https://github.com/lithomas1) - https://github.com/brandon-b-miller Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#16605
Configuration menu - View commit details
-
Copy full SHA for 822e7d0 - Browse repository at this point
Copy the full SHA 822e7d0View commit details -
Implement scan-based whole-frame aggregations for cudf-polars (rapids…
…ai#16509) contributes to rapidsai#16478 This implements "cum_min", "cum_max", "cum_prod", "cum_sum" "cum_count" is not implemented for now, since there's no exact libcudf match (I imagine the non-grouped case is also not used that much but haven't checked). I suppose we could implement it by creating a column of 1s and copying the null mask over, and doing a cum_sum on that. Let me know if you want to try that. Authors: - Thomas Li (https://github.com/lithomas1) Approvers: - https://github.com/brandon-b-miller URL: rapidsai#16509
Configuration menu - View commit details
-
Copy full SHA for 152111b - Browse repository at this point
Copy the full SHA 152111bView commit details
Commits on Aug 21, 2024
-
Merge pull request rapidsai#16599 from wence/fix/remove-placeholder-c…
…olumn Use a key column rather than a placeholder for count agg
Configuration menu - View commit details
-
Copy full SHA for 13a1493 - Browse repository at this point
Copy the full SHA 13a1493View commit details
Commits on Aug 22, 2024
-
Implement order preserving groupby in cudf-polars (rapidsai#16555)
## Description - Closes rapidsai#16477
Configuration menu - View commit details
-
Copy full SHA for 7cf3289 - Browse repository at this point
Copy the full SHA 7cf3289View commit details
Commits on Aug 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f6c938f - Browse repository at this point
Copy the full SHA f6c938fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4ded370 - Browse repository at this point
Copy the full SHA 4ded370View commit details
Commits on Aug 27, 2024
-
Correctly export empty column names in DataFrame.to_polars (rapidsai#…
…16596) polars.from_arrow renames empty column names (see pola-rs/polars#11632). This causes problems when round-tripping specially crafted dataframes. Avoid the problem by constructing the table with fake names and then renaming.
Configuration menu - View commit details
-
Copy full SHA for edabb67 - Browse repository at this point
Copy the full SHA edabb67View commit details -
Configuration menu - View commit details
-
Copy full SHA for a4c35e9 - Browse repository at this point
Copy the full SHA a4c35e9View commit details -
Add more
cudf-polars
unaryops (rapidsai#16579)Add support for additional unaryops through `cudf-polars`. Closes rapidsai#16566 --------- Co-authored-by: Lawrence Mitchell <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 0a95b2c - Browse repository at this point
Copy the full SHA 0a95b2cView commit details -
Merge pull request rapidsai#16667 from wence-/wence/merge-2408
Forward-merge 24.08
Configuration menu - View commit details
-
Copy full SHA for cc892fc - Browse repository at this point
Copy the full SHA cc892fcView commit details -
Add
pylibcudf
/cudf-polars
stringstrip
(rapidsai#16504)Add support for string `strip` in `pylibcudf` and `cudf-polars`. --------- Co-authored-by: Lawrence Mitchell <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 41a3a95 - Browse repository at this point
Copy the full SHA 41a3a95View commit details
Commits on Aug 28, 2024
-
cudf-polars
/pylibcudf
string -> date parsing (rapidsai#16306)This PR adds datetime/timestamp parsing from string columns in pylibcudf and cudf-polars. Closes rapidsai#16174
Configuration menu - View commit details
-
Copy full SHA for 0bf68d4 - Browse repository at this point
Copy the full SHA 0bf68d4View commit details
Commits on Aug 29, 2024
-
Support quantile in cudf_polars (rapidsai#16093)
Support `pl.Expr.quantile` in cudf-polars. --------- Co-authored-by: Vyas Ramasubramani <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 40d33cb - Browse repository at this point
Copy the full SHA 40d33cbView commit details
Commits on Aug 30, 2024
-
Implement handlers for first/last in groupby (rapidsai#16688)
Since the full-frame `Agg` handler for first and last doesn't construct a request (because we can do it without a `from_scalar` call), we didn't handle these in a groupby context. Fortunately it is easy to add.
Configuration menu - View commit details
-
Copy full SHA for 95da2c5 - Browse repository at this point
Copy the full SHA 95da2c5View commit details -
Ensure IR validation always checks for empty columns
We were previously not calling the superclass __post_init__ in custom validations of IR nodes. This meant that we would sometimes fail to raise when the schema contained an EMPTY column. Since we can't really compute with these types, we just fall back.
Configuration menu - View commit details
-
Copy full SHA for 434afab - Browse repository at this point
Copy the full SHA 434afabView commit details -
Need to check for nulls in nested dtypes
To do this we need to inspect the polars datatypes, since by the time we've converted to the pylibcudf one, the nested element types have been lost. We don't do this eagerly during dtype conversion because we still want to allow scalar literals with null dtype that will then be cast to a non-null dtype.
Configuration menu - View commit details
-
Copy full SHA for 385ae98 - Browse repository at this point
Copy the full SHA 385ae98View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1cf1146 - Browse repository at this point
Copy the full SHA 1cf1146View commit details -
Move creation of regex program to initialisation
This way if we don't support any features of the pattern, we correctly fall back to CPU.
Configuration menu - View commit details
-
Copy full SHA for de445a3 - Browse repository at this point
Copy the full SHA de445a3View commit details -
Merge pull request rapidsai#16703 from wence-/wence/fea/polars-reject…
…-invalid-regex Move creation of regex program to initialisation
Configuration menu - View commit details
-
Copy full SHA for f39713e - Browse repository at this point
Copy the full SHA f39713eView commit details -
Configuration menu - View commit details
-
Copy full SHA for ad364c6 - Browse repository at this point
Copy the full SHA ad364c6View commit details
Commits on Sep 2, 2024
-
Merge pull request rapidsai#16702 from wence-/wence/fea/polars-no-emp…
…ty-columns Disallow producing dataframes with Empty columns
Configuration menu - View commit details
-
Copy full SHA for d158b22 - Browse repository at this point
Copy the full SHA d158b22View commit details
Commits on Sep 3, 2024
-
Partially reject dynamic groupby (rapidsai#16720)
We are not yet exposing the actual information that the groupby is dynamic, but this catches a bunch of cases. Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#16720
Configuration menu - View commit details
-
Copy full SHA for b550645 - Browse repository at this point
Copy the full SHA b550645View commit details
Commits on Sep 4, 2024
-
Implement Kleene logic handling for Any/All and bitwise Or/And (rapid…
…sai#16476) We previously didn't support this case correctly, but it's not too bad. This would be much easier if we could do it in libcudf, hence: rapidsai#16475
Configuration menu - View commit details
-
Copy full SHA for eb2a23e - Browse repository at this point
Copy the full SHA eb2a23eView commit details -
Some fixes for unary functions (rapidsai#16719)
Correctly handle `pow` and `log` by translating to binary expressions when we observe the node. Upgrade our minimum supported polars version (so that we see all these function names from the rust IR). Also tighten check for which groupby-aggs are supported when the expression contains a unary function.
Configuration menu - View commit details
-
Copy full SHA for ebc3bbe - Browse repository at this point
Copy the full SHA ebc3bbeView commit details -
Implement unpivot in cudf-polars (rapidsai#16689)
Add support for unpivoting a DataFrame. We raise for cases where the concatenation of the value columns produces a cast that is not supported by standard fixed-width unary casting.
Configuration menu - View commit details
-
Copy full SHA for 5d262df - Browse repository at this point
Copy the full SHA 5d262dfView commit details -
Small scan-handler fixes (rapidsai#16721)
Reject two more edge cases that we do not support. We could easily support the case where the parquet read just needs to read the metadata, but it is low priority, so have not done so here.
Configuration menu - View commit details
-
Copy full SHA for c76e90b - Browse repository at this point
Copy the full SHA c76e90bView commit details
Commits on Sep 5, 2024
-
Implement cudf-polars datetime extraction methods (rapidsai#16500)
--------- Co-authored-by: brandon-b-miller <[email protected]> Co-authored-by: Bradley Dice <[email protected]> Co-authored-by: Lawrence Mitchell <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ccb8061 - Browse repository at this point
Copy the full SHA ccb8061View commit details
Commits on Sep 6, 2024
-
Polars 1.7 will change a minor thing in the IR, adapt to that (rapids…
…ai#16755) This field renaming was due to a recent refactor in (as-yet-unreleased) polars 1.7.
Configuration menu - View commit details
-
Copy full SHA for feb2e63 - Browse repository at this point
Copy the full SHA feb2e63View commit details -
Run polars test suite (defaulting to GPU) in CI (rapidsai#16710)
## Description We implement a small pytest plugin that defaults the polars engine to GPU (by monkeypatching `LazyFrame.collect`, yet another reason to have a global default somehow). As well as this, we collate all the known failures and classify them. ## Checklist - [x] I am familiar with the [Contributing Guidelines](https://github.com/rapidsai/cudf/blob/HEAD/CONTRIBUTING.md). - [x] New or existing tests cover these changes. - [x] The documentation is up to date with these changes.
Configuration menu - View commit details
-
Copy full SHA for 6d2e455 - Browse repository at this point
Copy the full SHA 6d2e455View commit details
Commits on Sep 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 24f9516 - Browse repository at this point
Copy the full SHA 24f9516View commit details
Commits on Sep 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4bbbdc2 - Browse repository at this point
Copy the full SHA 4bbbdc2View commit details