v0.9.1

What's Changed

• Fix Jekyll Docs Build by @soldni in #55
• Adding Citation text back to README by @soldni in #56
• Bump rustix from 0.37.20 to 0.37.25 by @dependabot in #59
• Documentation on BaseParallelProcessor by @soldni in #62
• Add download instruction by @Muennighoff in #63
• Fix spawn method for multiprocessing by @soldni in #64
• Fix hardcoded URL by @soldni in #65
• Fix Accidental Override of Boolean Value by @soldni in #66

New Contributors

• @Muennighoff made their first contribution in #63

Full Changelog: v0.9.0...v0.9.1

v0.9.0

What's Changed

• Skipping AWS checks when aws access key is not available by @soldni in #28
• env variable is not passed to tests by @soldni in #29
• Fix make by @chris-ha458 in #24
• Fix make more by @chris-ha458 in #31
• ff by @soldni in #36
• Adding C4 example, dryrun mode, profiling taggers by @soldni in #37
• Only run Python style checks on source and tests by @soldni in #38
• fix rust parts by @chris-ha458 in #23
• Add rust unit tests by @chris-ha458 in #35
• Bump webpki from 0.22.0 to 0.22.2 by @dependabot in #52
• Adding Tokenizer, Writing Documentation, Misc Bugs & CLI improvements by @soldni in #54

New Contributors

• @chris-ha458 made their first contribution in #24
• @dependabot made their first contribution in #52

Full Changelog: v0.8.0...v0.9.0

v0.8.0

What's Changed

• Analyzer to save and plot taggers distribution by @soldni in #21
• Scripts to compute statistics by @soldni in #22

Full Changelog: v0.7.0...v0.8.0

v0.7.0

What's Changed

• CLI improvements, remove need of experiment name by @soldni in #20

Full Changelog: v0.6.5...v0.7.0

v0.6.5

What's Changed

• added validation of configs, tagger bugfixes by @soldni in #18
• upping version by @soldni in #19

Full Changelog: v0.6.4...v0.6.5

v0.6.4

What's Changed

• adding tests in CI by @soldni in #17
• Added tests for local/remote bindings for deduper/mixer by @soldni in #15

Full Changelog: v0.6.3...v0.6.4

v0.6.3

What's Changed

• Mixer can use s3 or local paths by @rodneykinney in #14

New Contributors

• @rodneykinney made their first contribution in #14

Full Changelog: v0.6.2...v0.6.3

v0.6.2

What's Changed

• Add Dirk as an author by @dirkgr in #2
• README, tests by @kyleclo in #1
• ff main by @soldni in #4
• Kylel/test taggers by @kyleclo in #5
• ff soldni/cli branch by @soldni in #7
• add tests for data types by @kyleclo in #8
• ff by @soldni in #9
• CLI for dolma by @soldni in #6
• Readme and instructions by @soldni in #11
• Update README.md by @ianand in #12
• Fixing Build Issues by @soldni in #13

New Contributors

• @dirkgr made their first contribution in #2
• @kyleclo made their first contribution in #1
• @soldni made their first contribution in #4
• @ianand made their first contribution in #12

Full Changelog: https://github.com/allenai/dolma/commits/v0.6.2