Asking to pad but the tokenizer does not have a padding token

[christiandarkin]

I keep getting the error: Asking to pad but the tokenizer does not have a padding token- but there doesn't seem to be an option for it in the gui. Any idea of how to fix this? (I'm trying to train with "TheBloke/Wizard-Vicuna-7B-Uncensored-HF"

Thanks.

[zetavg]

Add these lines

if tokenizer.pad_token is None:
    print(f"Tokenizer has no pad_token set, setting it to 1.")
    tokenizer.pad_token_id = 1

here: https://github.com/zetavg/LLaMA-LoRA-Tuner/blob/98dba8d/llama_lora/lib/finetune.py#L246

[xzuyn]

Shouldn't it be tokenizer.pad_token_id = 0 since the generation_config.json is this?

{
  "_from_model_config": true,
  "bos_token_id": 1,
  "eos_token_id": 2,
  "pad_token_id": 0,
  "transformers_version": "4.29.2"
}

Also unrelated. Do you plan on continuing this project? So far this is the only trainer I've found to work when using ROCm versions of PyTorch and bitsandbytes.