Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable activation of subset selection feature per dataset with related args from recipe.yaml #528

Open
eshwarprasadS opened this issue Jan 30, 2025 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@eshwarprasadS
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Currently, datamixing.py consumes only two fields from recipe.yaml pertaining to dataset mixing. These two fields are: path and sampling_size. It would be beneficial if we can make changes to datamixing.py to enable a more nuanced yaml that will contain which subsetting method to use, and its pertaining parameters as sub fields.

Describe the solution you'd like
We could easily enable subset selection from here as an option, if datamixing.py could parse a yaml that is designed like so (for example):

datasets:
  - path: instructlab_community.jsonl
    sampling_size: 1.0
    sampling_method: random
    sampling_params:
      seed: 42  # Optional: for reproducible random sampling

  - path: another_dataset.jsonl
    sampling_size: 0.8
    sampling_method: diversity
    sampling_params:
      num_folds: 5     # Optional: number of folds for diversity sampling
      epsilon: 0.1     # Optional: epsilon parameter for diversity calculation

Additional context
This would enable a minimal entry point to calling the Subset Selection feature. For the purposes of this issue, it could simply call to a dummy method, once the yaml is correctly parsed, and it is recognized as a call to Subset Selection.

@bbrowning
Copy link
Contributor

This seems like a reasonable way to expose subset selection, so a +1 from me on the general direction of this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants