-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add dataset-level stats #2297
Commits on Nov 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d9bc2a5 - Browse repository at this point
Copy the full SHA d9bc2a5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ca0100 - Browse repository at this point
Copy the full SHA 6ca0100View commit details -
chore: apply clippy suggestions
warning: usage of `FromIterator::from_iter` --> src/cmd/stats.rs:789:27 | 789 | work_br = csv::ByteRecord::from_iter(vec![&*header].into_iter().chain(stat)); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use `.collect()` instead of `::from_iter()`: `vec![&*header].into_iter().chain(stat).collect::<csv::ByteRecord<_>>()` | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#from_iter_instead_of_collect = note: `-W clippy::from-iter-instead-of-collect` implied by `-W clippy::pedantic` = help: to override `-W clippy::pedantic` add `#[allow(clippy::from_iter_instead_of_collect)]` warning: implicitly cloning a `ByteRecord` by calling `to_owned` on its dereferenced type --> src/cmd/stats.rs:803:31 | 803 | stats_br_vec.push(dataset_stats_br.to_owned()); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider using: `dataset_stats_br.clone()` | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#implicit_clone = note: `-W clippy::implicit-clone` implied by `-W clippy::pedantic` = help: to override `-W clippy::pedantic` add `#[allow(clippy::implicit_clone)]` warning: implicitly cloning a `ByteRecord` by calling `to_owned` on its dereferenced type --> src/cmd/stats.rs:812:31 | 812 | stats_br_vec.push(dataset_stats_br.to_owned()); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider using: `dataset_stats_br.clone()` | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#implicit_clone warning: implicitly cloning a `ByteRecord` by calling `to_owned` on its dereferenced type --> src/cmd/stats.rs:825:31 | 825 | stats_br_vec.push(dataset_stats_br.to_owned()); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider using: `dataset_stats_br.clone()` | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#implicit_clone
Configuration menu - View commit details
-
Copy full SHA for 8fae48f - Browse repository at this point
Copy the full SHA 8fae48fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 68f3830 - Browse repository at this point
Copy the full SHA 68f3830View commit details -
refactor: improve dataset-level stats calculation, use a helper closu…
…re write_dataset_stat
Configuration menu - View commit details
-
Copy full SHA for 3bb256d - Browse repository at this point
Copy the full SHA 3bb256dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 85c330b - Browse repository at this point
Copy the full SHA 85c330bView commit details
Commits on Nov 19, 2024
-
refactor
: get_stats_records helper to ignore dataset-level stats an……d use simd_json instead of serde_json
Configuration menu - View commit details
-
Copy full SHA for 12bdfff - Browse repository at this point
Copy the full SHA 12bdfffView commit details -
refactor: get_stats_records - reduce cloning; align stats.jsonl loadi…
…ng approach also refactor csv_to_jsonl to pass output_jsonl by reference instead of by value
Configuration menu - View commit details
-
Copy full SHA for 642fd74 - Browse repository at this point
Copy the full SHA 642fd74View commit details -
refactor:
stats
- csv_to_jsonl - pass stats_pathbuf by reference in……stead of by value
Configuration menu - View commit details
-
Copy full SHA for 0b072c8 - Browse repository at this point
Copy the full SHA 0b072c8View commit details -
refactor: use
qsv__
instead of_qsv_
as prefix for qsv dataset le……vel objects so as not to trigger select where objects starting with _ is a sentinel for last column
Configuration menu - View commit details
-
Copy full SHA for b05c576 - Browse repository at this point
Copy the full SHA b05c576View commit details -
tests: adjust
index
andjson
tests to account for new dataset-lev……el stats also add assert_succes to select frequency and tojsonl tests to help in debugging
Configuration menu - View commit details
-
Copy full SHA for 672796e - Browse repository at this point
Copy the full SHA 672796eView commit details