Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixing the clm-prompt-tuning that was causing unequal lengths in the label token id #487

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fixing the clm-prompt-tuning that was causing unequal lengths in the label token id #487

wants to merge 1 commit into from

Commits on Apr 29, 2024

  1. Update clm-prompt-tuning.ipynb 

    The code snippet needs a correction in the line:
labels["input_ids"][i] = [-100] * (max_length - len(sample_input_ids)) + label_input_ids
Change it to:
labels["input_ids"][i] = [-100] * (max_length - len(label_input_ids)) + label_input_ids
This adjustment ensures that the label token ids are padded or truncated based on their own length, aligning with Hugging Face's recommended practice and avoiding issues with unequal lengths in input and label token ids. The same changes need to be corrected in documentation as well is been mentioned  in the https://huggingface.co/docs/peft/main/en/task_guides/prompt_based_methods and https://huggingface.co/docs/peft/main/en/task_guides/clm-prompt-tuning
    @bpkapkar
    bpkapkar authored Apr 29, 2024
    Configuration menu
    Copy the full SHA
    102bc82 View commit details
    Browse the repository at this point in the history